KR102641492B1 - Polyp detection method, device and computer program using endoscopy image - Google Patents

Abstract

A polyp detection method using an endoscopic image according to an embodiment of the present invention, the method comprising preparing at least a first deep learning model and a second deep learning model, and receiving an endoscopic image taken from the human body. and detecting the presence or absence of a polyp and location information of the polyp using one of the first deep learning model and the second deep learning model with respect to the received endoscopic image, and based on the presence or absence of the polyp over time. Comprising a step of calculating a polyp probability score for a polyp detected, determining whether to display a polyp on the endoscope image based on the calculated score, and outputting the polyp display on the endoscope image, and the detection step. The deep learning model used is determined based on the polyp likelihood score.

Description

Polyp detection method, device and program using endoscopic images {POLYP DETECTION METHOD, DEVICE AND COMPUTER PROGRAM USING ENDOSCOPY IMAGE}

The present invention relates to a method, device, and program for detecting polyps using an endoscope. More specifically, it relates to a polyp detection method, device, and program from an endoscopic image that can detect polyps more accurately while ensuring real-time polyp detection.

In general, the core purpose of endoscopy is to fundamentally block cancer by discovering and removing polyps (small raised tissue on the inner wall that must be removed when discovered), which are precursor lesions of cancer.

Endoscopy (particularly colonoscopy) is a test performed one-on-one between a doctor and a patient, and has the disadvantage of being labor-intensive and time-consuming. In addition, the test is repetitive, and polyp detection may fail due to various factors such as the tester's skill level, tester's condition, fatigue, and mistakes during the test process. The failure rate of polyp detection through endoscopy is known to be as high as 30%.

Recently, technology for automatically detecting polyps from endoscopic images using deep learning is known.

The endoscope has a bendable structure, and the examiner bends the endoscope and moves the imaging device installed at the tip of the endoscope closer to the suspected polyp area to finally confirm the presence of a polyp. Conventional polyp detection technology using deep learning learning automatically detects suspected polyps in the endoscope image using deep learning learning and displays them on the endoscope image. Through this, the examiner's polyp detection failure rate can be reduced.

However, for the purpose of endoscope detection technology, the deep learning polyp detection model must detect suspected polyps in the endoscope image in real time and display them on the endoscope image. However, if the processing speed of the deep learning polyp detection model is slow, such real-time processing is not possible. There are problems that can be difficult or require expensive equipment.

KR 10-2021-0150695 A (2021. 12. 13.)

The present invention is intended to solve the above-mentioned problems, and seeks to provide a polyp detection method, device, and program from an endoscopic image that can detect polyps more accurately while ensuring real-time polyp detection.

The problem to be solved by the present invention is not limited to the content mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

According to an embodiment of the present invention, a polyp detection method using an endoscopic image includes preparing at least a first deep learning model and a second deep learning model, receiving an endoscopic image taken from the human body, and receiving the endoscopic image. Detecting the presence or absence of a polyp and location information of the polyp using one of the first deep learning model and the second deep learning model for an endoscopic image, and the polyp detected based on the presence or absence of the polyp over time Comprising a step of calculating a polyp probability score, determining whether to display a polyp on the endoscopic image based on the calculated score, and outputting a polyp display on the endoscopic image, and deep used in the detection step. The learning model may be determined based on the polyp likelihood score.

The second deep learning model may have a lower polyp detection processing speed than the first deep learning model.

In addition, if the calculated score is more than the reference value, a polyp is displayed, and if the polyp probability score is more than the reference value set smaller than the reference value, the second deep learning model can be applied.

According to one embodiment of the present invention, when the polyp likelihood score is less than the reference value continues for more than a certain period of time, the second deep learning model may be applied.

In addition, according to one embodiment of the present invention, the step of calculating a polyp probability score for a polyp detected based on the presence or absence of a polyp over time includes comparing the polyp detected in the frame with the polyp detected in the previous frame. In the step of calculating the polyp probability score by adding a predetermined score to the polyp probability score in the previous frame for the polyp detected in the previous frame and the current frame, and for the polyp newly detected in the frame, a predetermined score is added to the polyp probability score. It may be calculated as a probability score, and for polyps detected in the previous frame but not detected in the current frame, the step may include calculating the polyp probability score by subtracting a predetermined score from the polyp probability score in the previous frame.

In addition, the gradation of the polyp indication is determined according to the polyp probability score, and if the polyp probability score is higher than a reference value set lower than the polyp probability score corresponding to the highest polyp indication, the second deep learning model may be applied.

In addition, according to another embodiment of the present invention, a polyp detection device using an endoscopic image includes an image acquisition unit that receives an endoscopic image taken from the human body, and a first deep learning model and a first deep learning model for the received endoscopic image. 2 A polyp detection unit that detects the presence or absence of a polyp and the location information of the polyp using one of the deep learning models, and a polyp probability calculation unit that calculates a polyp probability score for the detected polyp based on the presence or absence of the polyp over time. Based on the calculated score, it includes a polyp display determination unit that determines whether or not to display a polyp on the endoscopic image, and a polyp display unit that outputs a polyp display on the endoscopic image, and the deep learning model used in the polyp detection unit determines the possibility of a polyp. It can be decided based on the score.

According to another embodiment of the present invention, there is a polyp detection program using an endoscopic image, the polyp detection program is a computer program coupled to a processor and stored in a computer-readable recording medium, and the computer program detects the polyp described above. The method may be configured to be executed by the processor by instructions stored in the recording medium.

According to a polyp detection method, device, and program using an endoscopic image according to an embodiment of the present invention, a first deep learning model and a second deep learning model are used according to a polyp probability score calculated based on the presence or absence of polyps over time. Since one of the following is used, polyps can be detected and displayed more accurately while ensuring real-time polyp detection.

According to another embodiment of the present invention, by varying the polyp display according to the polyp probability score calculated based on the presence or absence of polyps over time, the problem of the polyp display momentarily blinking is solved, The examiner can intuitively recognize the possibility of a latent polyp. In addition, even if the number of polyp signs increases, the examiner can easily determine which area should be diagnosed through endoscopy, so there is less need to reduce the number of polyp signs by raising the standard for polyp judgment, which can lower the risk of polyp detection failure.

In addition, according to another embodiment of the present invention, a more accurate deep learning model is used for the highest polyp display or a potential polyp display that persists for a certain period of time, thereby ensuring real-time polyp detection and detecting polyps more accurately. It can be displayed.

Figure 1 is a block diagram of a polyp detection device according to an embodiment of the present invention.
Figure 2 shows an example of an endoscopic image acquired by the image acquisition unit 20.
Figure 3 is an example explaining the polyp probability score calculated for each polyp.
Figure 4 is a diagram illustrating the polyp probability score according to the time trend of the polyp.
Figure 5 shows an example of detecting the presence or absence of a polyp and polyp location information in the first frame 1.
Figure 6 shows an example of detecting the presence or absence of a polyp and polyp location information in frame 2, which is the next frame after the first frame 1.
Figure 7 shows an example of an endoscopic image with the highest polyp indicated.
Figure 8 shows an example of an endoscopic image displaying an area where a plurality of detected polyps are gathered in close proximity.
Figure 9 shows an example of an endoscopic image with additional markings for polyps that are judged to be latent polyps for more than a certain period of time.
Figure 10 is a flowchart of a polyp detection method according to an embodiment of the present invention.
Figure 11 is a diagram illustrating in more detail the step of calculating a polyp probability score for a detected polyp based on the presence or absence of a polyp over time.

Since the present invention can make various changes and have various embodiments, specific embodiments will be described in detail with reference to the drawings. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention. While describing each drawing, similar reference numerals are used for similar components. Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component without departing from the scope of the present invention, and similarly, the second component may also be named a first component. The term “and/or” includes any of a plurality of related description items or a combination of a plurality of related description items. When a component is referred to as being “connected” or “connected” to another component, it should be understood that it may be directly connected to or connected to that other component, but that other components may exist in between. something to do. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between. The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof. Additionally, it is to be understood that the terms "first" and "second" are used herein for distinguishing purposes only and do not in any way indicate or anticipate order or priority. Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, be interpreted as having an ideal or excessively formal meaning. It doesn't work.

Throughout the specification and claims, when it is said that a part includes a certain component, this does not mean that other components are excluded, but that other components may be further included, unless specifically stated to the contrary. Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a block diagram of a polyp detection device according to an embodiment of the present invention.

Referring to Figure 1, the polyp detection device according to an embodiment of the present invention includes an image acquisition unit 10, a polyp detection unit 20, a polyp probability calculation unit 30, a polyp display determination unit 40, and a polyp display unit ( 50) may be included.

The image acquisition unit 10 according to an embodiment of the present invention acquires images captured by endoscopic equipment. For example, endoscope equipment can capture images at 60 frames per second, but is not necessarily limited to this. The image acquisition unit 10 according to an embodiment of the present invention may extract the endoscopic image at 20 or 40 frames per second, but is not necessarily limited thereto.

The polyp detection unit 20 according to an embodiment of the present invention can automatically detect the presence or absence of a polyp and polyp location information using a deep learning model for the extracted endoscopic image. Here, the deep learning-based polyp detection model is a model obtained by learning multiple endoscopic images, and can extract polyp location information from the endoscopic images.

For example, a deep learning polyp detection model uses rectangular polyp coordinates and a polyp score (in this specification, 'polyp score' is provided by processing the frame in question by a deep learning polyp detection model, scoring the probability that a specific area is a polyp). It is possible to detect the presence of a polyp by calculating the polyp score and comparing it with the threshold. For example, if the threshold value of the polyp score for determining the suspected polyp area is, for example, 700, if the polyp score for the suspected polyp area in one frame is 700 or more (e.g., 720), the suspected polyp area is a polyp. It is detected as

If it is determined that it is necessary to detect polyps more accurately based on the presence or absence of polyps over time, the polyp detection unit 20 according to an embodiment of the present invention applies a more accurate deep learning polyp detection model. Polyps can be detected.

For example, according to one embodiment of the present invention, the polyp detection unit 20 may determine a deep learning polyp detection model based on the polyp probability score calculated by the polyp probability calculation unit 30, which will be described later.

The polyp probability calculation unit 30 calculates a polyp probability score for the detected polyp based on the presence or absence of a polyp over time. That is, the polyp detected in the current frame can be compared with the polyp detected in the previous frame, and based on this comparison, a predetermined score can be applied to calculate a polyp probability score over time. For example, this polyp probability score is calculated in such a way that a predetermined score is added if a polyp is detected in the frame, and a predetermined score is subtracted if a polyp is not detected in the frame, so that the polyp probability score is calculated cumulatively. .

It is possible to determine whether the detected polyps are the same by comparing the position coordinates of the polyp detected in the frame with the position coordinates of the polyp detected in the previous frame. The polyp probability score calculation unit 30 calculates a polyp probability score by adding a predetermined score to the polyp probability score in the previous frame for the polyp detected in the previous frame and the current frame, and adds a polyp probability score to the polyp newly detected in the frame. For polyps that were detected in the previous frame but not detected in the current frame, a predetermined score is calculated as the polyp probability score, and the polyp probability score can be calculated by subtracting a predetermined score from the polyp probability score in the previous frame.

Figure 2 shows an example of an endoscopic image acquired by the image acquisition unit 20. In the first frame 1, the first polyp (1) and the second polyp (2) are detected as polyps by the first deep learning model, and in the next consecutive frame, frame 2, the first polyp (1) and the third polyp (3) ) is detected as a polyp by the first deep learning model, the polyp probability score calculated for each polyp may be as follows.

Figure 3 is an example illustrating the polyp probability score calculated for each polyp.

As can be seen in FIG. 3, since the first polyp 1 was detected in frame 1 and frame 2, a predetermined score (e.g., 1) is added to determine the polyp probability of the first polyp 1 in frame 2. The score is 2. Since the second polyp 2 was detected in frame 1 but not in frame 2, a predetermined score (e.g., 1) is subtracted, and the polyp probability score of the second polyp 2 in frame 2 is 0. Since the third polyp (3) was not detected in frame 1 but was detected in frame 2, the polyp probability score is 1.

Figure 4 is a diagram illustrating the polyp probability score according to the time trend of the polyp.

As shown in Figure 4, as the number of polyps detected increases, the polyp probability score increases, and as the number of polyps not detected increases, the polyp probability score decreases.

According to one embodiment of the present invention, the polyp probability score is based on the above cumulative method, but may be additionally adjusted. For example, if polyps are detected continuously (e.g., polyps detected more than 5 times), 3 can be additionally added to the polyp probability score, and if polyps are not detected continuously (e.g., polyps not detected more than 5 times), 3 can be additionally added to the polyp probability score. , additionally, 3 can be subtracted from the polyp probability score.

In addition, although the score is explained as being accumulated by adding or subtracting the same score (e.g., 1) depending on whether or not a polyp is detected, the score may not be the same score. For example, a score may be calculated in proportion to the difference between the polyp score and the reference value, and this score may be added or subtracted depending on whether or not a polyp is detected. That is, if the polyp score is 780 and the reference value is 700, the difference is 80, and if the polyp score is 900, the difference is 200. In this case, the points added to the polyp likelihood score are not the same, and may be set in proportion to the difference and added to the polyp likelihood score.

In the polyp detection unit according to an embodiment of the present invention, the first deep learning model and the second deep learning model may be prepared at different processing speeds per frame. Typically, the lower the processing speed of a deep learning model (i.e., the longer it takes to process), the more accurate polyp detection is possible. Therefore, when a deep learning model with a low processing speed is used, the processing speed per frame becomes slow, and there is a risk that real-time processing of the endoscopic image will be delayed. Considering this, in one embodiment of the present invention, while detecting the presence or absence of a polyp and polyp location coordinates by applying the first deep learning model, when a more accurate judgment is required, the processing speed per frame is higher than that of the first deep learning model. A second deep learning model with a low value can be applied. Through this, concerns about real-time processing delays for endoscopic images can be minimized, while polyps can be displayed by judging certain polyps based on stricter standards.

For example, when the reference value for whether to display a polyp is 10, and the polyp probability score is 8 or more, the presence of a polyp can be detected by applying the second deep learning model. In this case, when the polyp probability score exceeds 9 and reaches 10 through the second deep learning model, the polyp display determination unit 40 determines the polyp display, and accordingly, the polyp display unit 50 displays the polyp on the endoscope image. is output.

Additionally, in cases where a polyp is repeatedly detected and not detected as a polyp depending on the frame, an accurate judgment regarding the polyp may be required from the examiner's point of view. That is, when the polyp likelihood score stays within a predetermined score range for a certain period of time, the second deep learning model can be applied.

The above-mentioned case was explained on the assumption that the polyp indication was single. According to an embodiment of the present invention, as will be described later, the polyp indication may be determined differently based on the presence or absence of a polyp over time and displayed on the endoscopic image. In other words, if a polyp is detected, the detected polyp is displayed, but the polyp display is determined differently based on the presence or absence of the polyp over time. Below, when the polyp indication is determined differently based on the presence or absence of a polyp over time, an application example of the first deep learning model and the second deep learning model over time will be described.

In conventional polyp detection technology using deep learning, there are often cases where a specific area is judged to be a suspected polyp area for various reasons such as bubbles or foreign substances that appear and disappear momentarily, or the shooting angle, but is no longer judged to be a suspected polyp area. For example, if the threshold value of the polyp score for determining the suspected polyp area is, for example, 700, in one frame, the polyp score for the suspected polyp area is 700 or more (e.g., 720), and the polyp score for the suspected polyp area is 700 or more (e.g., 720), and the In the next frame, if the polyp score for the suspected polyp area is less than 700 (eg, 650), the suspected polyp area is displayed as a polyp in the endoscopic image, and then the square disappears. In other words, the polyp indicator blinks (false positive).

The present inventor focused on the fact that this problem cannot be fundamentally solved by raising the standard for determining polyps. In other words, raising the standard for polyp judgment may have the effect of reducing the false positive rate, but the problem of the polyp area indicator blinking still exists, and this cannot be completely solved. In addition, false positives mean that the examiner may need to bring the endoscopic imaging device closer to the suspected polyp area to ultimately determine whether there is a tumor, and these are noises and are not subject to removal. In other words, if the false positive rate is reduced by raising the standard for polyp detection, there is a problem that the risk of polyp detection failure increases.

According to one embodiment of the present invention, the polyp display determination unit 30 may vary the polyp display 60 according to the polyp probability score. For example, the gradation of the polyp mark 60 on the image may be varied. For example, the higher the score, the color tone can be changed step by step or continuously so that the examiner can intuitively recognize it.

As shown in Figures 5 and 6, the polyp probability score of the first polyp 1 increases over time, and may be displayed in a darker color tone in frame 2 than in frame 1. Conversely, the second polyp 2 may be displayed in a lighter shade in frame 2 than in frame 1, as the polyp probability score decreases over time. The third polyp 3 is newly detected in frame 2 and can be displayed in a color tone corresponding to the polyp probability score of 1. Here, it should be understood that the change in color tone is exaggerated for explanation purposes. That is, considering that the frame per second is 60, the color tone of the first polyp 1 is not set to change noticeably as shown in Figures 5 and 6 as it moves from frame 1 to frame 2. This may not be the case, and in reality, the color tone may be set to change more gradually.

Based on the disclosure of Figures 5 and 6, when comparing the existing endoscopic detection method and the present invention, in the existing endoscopic detection method, the second polyp 2 is not detected in frame 2, so the polyp is displayed in frame 2. disappears, the second polyp (2) blinks in the endoscopic image. On the other hand, in the present invention, the second polyp 2 still has a polyp mark, and the examiner can intuitively recognize the possibility of the second polyp 2 being a polyp through the color tone of the polyp mark.

Additionally, according to one embodiment of the present invention, if the polyp probability score is less than a predetermined score, the polyp may not be displayed. For example, if the polyp probability score is -4 or less, the polyp may not be displayed in the frame.

According to one embodiment of the present invention, for example, when the polyp display is varied according to the polyp probability score, there may be a highest polyp display 62 in which the polyp display no longer changes even if the polyp probability score increases. For example, if the polyp probability score is 10 or more, the polyp indication may no longer change. In this case, the polyp indication for polyps with a polyp probability score of 10 or more is the highest polyp indication (62), which is recognized by the examiner as a polyp that is highly suspected to be a polyp.

When displaying the highest polyp 62, a polyp display method that is different from the display method of the remaining polyp displays can be applied. For example, as shown in Figure 7, if the remaining polyp display method is a color change method, the highest polyp display 62 with a polyp probability score of 10 or more is displayed in a different color or has a different polyp display shape (e.g., It can be displayed by changing the square display to a circular display. Through this, the examiner can intuitively recognize potential polyps and final suspected polyps.

In this way, the final suspected polyp and the potential polyp are distinguished, but the possibility of a potential polyp is intuitively recognized for the potential polyp, so even if the number of polyps is somewhat increased, the examiner can easily figure out which area should be diagnosed through endoscopy. You can. Accordingly, there is less need to reduce polyp indications by raising the standard for polyp judgment, and the risk of polyp detection failure can be lowered.

Since this highest polyp mark 62 is recognized by the examiner as a polyp that is highly suspected of being a polyp, it may be desirable to determine whether or not to mark the highest polyp through more accurate judgment. Therefore, if the best polyp indication is provided when the polyp likelihood score is 10, the first deep learning model is applied until the polyp likelihood score reaches 9, and then the first deep learning model is used for polyps with a polyp likelihood score of 9 or more. A second deep learning model with a reference value higher than the reference value can be applied.

Alternatively, if a latent polyp is repeatedly judged to be a polyp and not judged to be a polyp depending on the frame, an accurate judgment regarding the polyp may be required from the examiner's point of view. That is, if the time smaller than the polyp probability score corresponding to the highest polyp indication 62 continues for more than a certain period of time, the second deep learning model with a low processing speed per frame can be applied.

According to one embodiment of the present invention, when a plurality of detected polyps are gathered together on an endoscope image, a method of displaying an area where a plurality of detected polyps are gathered may be applied, rather than a method of displaying individual detected polyps. For example, according to one embodiment of the present invention, not only the final suspected polyp but also the potential polyp is displayed differently according to the passage of time, so there may be many polyp indications. From the examiner's point of view, in order to reduce the risk of polyp detection failure, it is necessary to check areas with many potential polyps by approaching the area with an endoscopic imaging device. Therefore, so that the examiner can intuitively recognize areas with many potential polyps and guide the movement of the endoscope, when multiple detected polyps with polyp probability scores below a certain score are gathered in close proximity, individual detected polyps are not displayed, but rather are displayed. , it is possible to display an area where multiple detected polyps are gathered in close proximity. Figure 8 is an example of a diagram showing a polyp display 66 indicating an area where a plurality of detected polyps are gathered in close proximity.

According to one embodiment of the present invention, there may be cases where a latent polyp is repeatedly judged to be a polyp and not judged to be a polyp depending on the frame. From the examiner's point of view, in order to reduce the risk of polyp detection failure, these potential polyps need to be confirmed by accessing the endoscope camera. In order for the examiner to intuitively recognize these potential polyps and guide the movement of the endoscope, if a polyp is judged to be a potential polyp for more than a certain period of time (i.e., if the polyp probability score stays within a certain range for a certain period of time), a mark indicating this is additionally added. can do. Figure 9 is an example of an endoscopic image with additional markings for polyps that are judged to be latent polyps for more than a certain period of time.

Figure 10 is a flowchart of a polyp detection method according to an embodiment of the present invention.

Referring to Figure 10, the polyp detection method according to an embodiment of the present invention includes preparing at least a first deep learning model and a second deep learning model (S100) and receiving an endoscopic image taken from the human body. (S200) and detecting the presence or absence of a polyp and location information of the polyp using at least one of the first deep learning model and the second deep learning model with respect to the received endoscopic image (S300), and time A step of calculating a polyp probability score for the detected polyp based on the presence or absence of a polyp according to the trend (S400), and a step of determining whether to display the polyp on the endoscope image based on the calculated score (S500); A step of outputting a polyp indication on an endoscope image (S600) may be included. Here, the deep learning model used in step S300 is determined based on the polyp likelihood score.

Figure 11 is a diagram illustrating in more detail the step of calculating a polyp probability score for a detected polyp based on the presence or absence of a polyp over time. As shown in Figure 11, the step of calculating a polyp probability score for a polyp detected based on the presence or absence of a polyp over time (S400) involves comparing the polyp detected in the frame with the polyp detected in the previous frame. It may be comprised of a step (S420) and a step (S440) of calculating a polyp probability score over time by applying a predetermined score based on the comparison result.

The device described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general-purpose or special-purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may perform an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device may include multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible. Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium, or It may be permanently or temporarily embodied in a device or a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media. The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. Computer-readable media may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and floptical disks. Included are magneto-optical media such as magneto-optical media, and hardware devices specifically configured to store and perform program instructions such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa. The above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications and variations will be possible to those skilled in the art without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

1: Primary polyp
2: Secondary polyp
3: Tertiary polyp
10: Image acquisition unit
20: Polyp detection unit
30: Polyp probability calculation unit
40: Polyp indication decision part
50: Polyp display unit
60: Polyp sign
62: Top polyp display

Claims (15)

A polyp detection method using a polyp detection device including an image acquisition unit, a polyp detection unit, a polyp probability calculation unit, a polyp display determination unit, and a polyp display unit,
Preparing, by the polyp detection unit, a first deep learning model configured to detect the presence or absence of a polyp and location information of the polyp, and a second deep learning model configured to detect the presence or absence of a polyp and location information of the polyp;
receiving, by the image acquisition unit, an endoscopic image taken from the human body;
The polyp detection unit detects the presence or absence of a polyp and location information of the polyp using the first deep learning model and the second deep learning model with respect to the received endoscopic image;
The polyp probability calculation unit calculates a polyp probability score for the detected polyp based on the presence or absence of the polyp over time;
The polyp display determination unit determines whether to display the polyp on the endoscopic image based on the calculated score;
The polyp display unit includes a step of outputting a polyp display on an endoscope image,
The second deep learning model has a lower polyp detection processing speed than the first deep learning model,
In the detection step, the presence or absence of a polyp and location information of the polyp are detected using the first deep learning model, and then, according to the polyp probability score, the second deep learning model is used instead of the first deep learning model. Thus, the presence or absence of a polyp and the location information of the polyp are detected,
Polyp detection method using a polyp detection device.
delete According to paragraph 1,
If the calculated score is higher than the reference value, the polyp display unit displays the polyp,
Polyp detection method using a polyp detection device.
According to paragraph 3,
If the polyp probability score is greater than the reference value set to be smaller than the reference value, a second deep learning model is applied in the polyp detection unit,
Polyp detection method using a polyp detection device.
According to paragraph 3,
If the polyp probability score is smaller than the reference value for more than a certain period of time, a second deep learning model is applied in the polyp detection unit,
Polyp detection method using a polyp detection device.
According to paragraph 1,
The step of calculating a polyp probability score for a detected polyp based on the presence or absence of a polyp over time is,
Comparing the polyp detected in the current frame with the polyp detected in the previous frame;
For polyps detected in the previous frame and the current frame, a polyp probability score is calculated by adding a predetermined score to the polyp probability score in the previous frame, and for polyps newly detected in the current frame, a predetermined score is calculated as the polyp probability score. And, for polyps detected in the previous frame but not detected in the current frame, calculating a polyp probability score by subtracting a predetermined score from the polyp probability score in the previous frame,
Polyp detection method using a polyp detection device.
According to clause 6,
The gradation of the polyp indication is determined according to the polyp probability score, and if the polyp probability score is higher than a reference value set lower than the polyp probability score corresponding to the highest polyp indication, a second deep learning model is applied in the polyp detection unit,
Polyp detection method using a polyp detection device.
A polyp detection device using endoscopic images,
An image acquisition unit that receives an endoscopic image taken from the human body,
Prepare a first deep learning model configured to detect the presence or absence of a polyp and location information of the polyp, and a second deep learning model configured to detect the presence or absence of a polyp and location information of the polyp, with respect to the received endoscopic image. A polyp detection unit that detects the presence or absence of a polyp and location information of the polyp using the first deep learning model and the second deep learning model,
A polyp probability calculation unit that calculates a polyp probability score for the detected polyp based on the presence or absence of polyp over time,
a polyp display determination unit that determines whether to display a polyp on the endoscopic image based on the calculated score;
It includes a polyp display unit that outputs the polyp display on the endoscope image,
The second deep learning model has a lower polyp detection processing speed than the first deep learning model,
In the polyp detection unit, the presence or absence of a polyp and location information of the polyp are detected using the first deep learning model, and then, according to the polyp probability score, the second deep learning model is used instead of the first deep learning model. Thus, the presence or absence of a polyp and the location information of the polyp are detected,
Polyp detection device using endoscopic images.
delete According to clause 8,
If the calculated score is higher than the standard value, the polyp display unit displays the polyp,
Polyp detection device using endoscopic images.
According to clause 10,
If the polyp probability score is greater than the reference value set to be smaller than the reference value, a second deep learning model is applied in the polyp detection unit,
Polyp detection device using endoscopic images.
According to clause 10,
If the polyp probability score is smaller than the reference value for more than a certain period of time, a second deep learning model is applied in the polyp detection unit,
Polyp detection device using endoscopic images.
According to clause 8,
In the polyp probability calculation section,
Compare the polyp detected in the current frame with the polyp detected in the previous frame,
For polyps detected in the previous frame and the current frame, a polyp probability score is calculated by adding a predetermined score to the polyp probability score in the previous frame, and for polyps newly detected in the current frame, a predetermined score is calculated as the polyp probability score. And, for polyps detected in the previous frame but not detected in the current frame, a polyp probability score is calculated by subtracting a predetermined score from the polyp probability score in the previous frame,
Polyp detection device using endoscopic images.
According to clause 13,
The gradation of the polyp indication is determined according to the polyp probability score, and if the polyp probability score is higher than a reference value set lower than the polyp probability score corresponding to the highest polyp indication, a second deep learning model is applied in the polyp detection unit,
Polyp detection device using endoscopic images.
It is a polyp detection program using endoscopic images.
The polyp detection program is a computer program coupled to a processor and stored in a computer-readable recording medium,
The computer program is configured to execute the polyp detection method according to one of claims 1, 3, and 4 to 7 on the processor by instructions stored in the recording medium.