KR102641489B1 - Polyp detection method, device and computer program using endoscopy image - Google Patents

Abstract

According to an embodiment of the present invention, a polyp detection method using an endoscopic image includes the steps of receiving an endoscopic image taken from the human body, and determining the presence or absence of a polyp using a deep learning model for the received endoscopic image. A step of detecting location information, a step of calculating a polyp probability score for the detected polyp based on the presence or absence of a polyp over time, and based on the calculated score, determining the polyp display on the endoscope image for each frame. It includes the step of displaying the determined polyp mark on the endoscope image.

Description

Polyp detection method, device and program using endoscopic images {POLYP DETECTION METHOD, DEVICE AND COMPUTER PROGRAM USING ENDOSCOPY IMAGE}

The present invention relates to a method, device, and program for detecting polyps using an endoscope. More specifically, it relates to a polyp detection method, device, and program from an endoscopic image that allows the examiner to more intuitively identify potential polyps and guide the movement of the endoscope.

In general, the core purpose of endoscopy is to fundamentally block cancer by discovering and removing polyps (small raised tissue on the inner wall that must be removed when discovered), which are precursor lesions of cancer.

Endoscopy (particularly colonoscopy) is a test performed one-on-one between a doctor and a patient, and has the disadvantage of being labor-intensive and time-consuming. In addition, the test is repetitive, and polyp detection may fail due to various factors such as the tester's skill level, tester's condition, fatigue, and mistakes during the test process. The failure rate of polyp detection through endoscopy is known to be as high as 30%.

Recently, technology for automatically detecting polyps from endoscopic images using deep learning is known.

The endoscope has a bendable structure, and the examiner bends the endoscope and moves the imaging device installed at the tip of the endoscope closer to the suspected polyp area to finally confirm the presence of a polyp. Conventional polyp detection technology using deep learning learning automatically detects suspected polyps in the endoscope image using deep learning learning and displays them on the endoscope image. Through this, the examiner's polyp detection failure rate can be reduced.

However, for various reasons such as bubbles or foreign substances that appear and disappear momentarily, or the angle of imaging, a specific area is detected as a suspected polyp area, but is not detected as a suspected polyp area, and the indication of the suspected polyp area on the endoscopic image often blinks. (False Positive), as a result, it is difficult for the examiner to intuitively identify potential polyps, and in some cases, too many polyp suspicion area indicators may blink, causing discomfort to the examiner.

KR 10-2021-0150695 A (2021. 12. 13.)

The present invention is intended to solve the above-mentioned problems, and aims to provide a method, device, and program for detecting polyps from an endoscopic image, which allows the examiner to intuitively recognize the possibility of a potential polyp without the problem of the polyp area display blinking. do.

The problem to be solved by the present invention is not limited to the content mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

According to an embodiment of the present invention, a polyp detection method using an endoscopic image includes the steps of receiving an endoscopic image taken from the human body, and determining the presence or absence of a polyp using a deep learning model for the received endoscopic image. A step of detecting location information, a step of calculating a polyp probability score for the detected polyp based on the presence or absence of a polyp over time, and based on the calculated score, determining the polyp display on the endoscope image for each frame. It may include the step of displaying the determined polyp mark on the endoscope image.

The step of calculating a polyp probability score for a polyp detected based on the presence or absence of a polyp over time includes comparing the polyp detected in the frame with the polyp detected in the previous frame, and based on the comparison, It may include calculating a polyp probability score over time by applying the score.

In addition, the step of applying the predetermined score to calculate the polyp probability score over time includes, for polyps detected in the previous frame and the current frame, adding a predetermined score to the polyp probability score in the previous frame to obtain the polyp probability score. For polyps newly detected in the frame, a predetermined score is calculated as the polyp probability score, and for polyps detected in the previous frame but not detected in the current frame, a predetermined score is subtracted from the polyp probability score in the previous frame. It may include calculating a polyp probability score.

According to one embodiment of the present invention, the predetermined score may be determined based on the difference between the polyp score provided by the deep learning model and the polyp detection reference value.

If the polyp probability score is below the reference value, the polyp may not be labeled as a polyp.

A gradation of the polyp mark may be determined according to the polyp probability score, and the highest polyp mark may be different from the remaining polyp marks in color or shape.

In addition, according to another embodiment of the present invention, when a plurality of polyps with a polyp probability score smaller than the polyp probability score corresponding to the highest polyp indication are gathered, the area where the polyps are gathered may be displayed. Additionally, if the polyp probability score is lower than the polyp probability score corresponding to the highest polyp indication persists for more than a certain period of time, an indication indicating this may be applied to the polyp indication.

According to another embodiment of the present invention, a polyp detection device using an endoscopic image includes an image acquisition unit that receives an endoscopic image taken from the human body, and a deep learning model for the received endoscopic image to detect the presence or absence of a polyp. and a polyp detection unit that detects location information of the polyp, a polyp probability calculation unit that calculates a polyp probability score for the detected polyp based on the presence or absence of the polyp over time, and based on the calculated score, on the endoscope image. It may include a polyp display determination unit that determines the polyp display for each frame, and a polyp display unit that displays the determined polyp display on the endoscope image.

The polyp probability calculation unit may compare the polyp detected in the current frame with the polyp detected in the previous frame, and apply a predetermined score based on the comparison to calculate a polyp probability score over time.

In addition, the polyp probability calculation unit calculates the polyp probability score by adding a predetermined score to the polyp probability score in the previous frame for the polyp detected in the previous frame and the current frame, and calculates the polyp probability score for the polyp newly detected in the frame. The score is calculated as a polyp probability score, and for polyps detected in the previous frame but not detected in the current frame, the polyp probability score can be calculated by subtracting a predetermined score from the polyp probability score in the previous frame.

In addition, the predetermined score may be determined based on the difference between the polyp score provided by the deep learning model and the polyp detection reference value, and if the polyp probability score is less than the reference value, the polyp may not be displayed for the polyp.

A gradation of the polyp mark may be determined according to the polyp probability score, and the highest polyp mark may be different from the remaining polyp marks in color or shape.

According to another embodiment of the present invention, when a plurality of polyps with a polyp probability score smaller than the polyp probability score corresponding to the highest polyp indication are gathered, the area where the polyps are gathered may be displayed. Additionally, if the polyp probability score is lower than the polyp probability score corresponding to the highest polyp indication persists for more than a certain period of time, an indication indicating this may be applied to the polyp indication.

According to another embodiment of the present invention, there is a polyp detection program using an endoscopic image, the polyp detection program is a computer program coupled to a processor and stored in a computer-readable recording medium, and the computer program detects the polyp described above. The method may be configured to be executed by the processor by instructions stored in the recording medium.

According to the polyp detection method, device, and program using an endoscopic image according to an embodiment of the present invention, the polyp display is instantaneous as the polyp display is varied according to the polyp probability score calculated based on the presence or absence of polyps over time. This has the effect of solving the flickering problem.

In addition, since the polyp probability score calculated based on the presence or absence of polyps over time is reflected in the polyp display, the examiner can intuitively recognize the polyp probability of the potential polyp.

In addition, according to one embodiment of the present invention, the final suspected polyp and the potential polyp are distinguished, but the possibility of polyps of the potential polyp is intuitively recognized, so even if the number of polyp indications increases somewhat, the examiner selects which part. It is easy to determine whether a diagnosis should be made through endoscopy. Accordingly, there is less need to reduce polyp indications by raising the standard for polyp judgment, and the risk of polyp detection failure can be lowered.

Figure 1 is a block diagram of a polyp detection device according to an embodiment of the present invention.
Figure 2 shows an example of detecting the presence or absence of a polyp and polyp location information in the first frame 1.
Figure 3 shows an example of detecting polyp presence and polyp location information in frame 2, which is the next frame after the first frame 1.
Figure 4 is an example illustrating the polyp likelihood score calculated for each polyp in the first frame 1 of Figure 2 and the polyp likelihood score calculated for each polyp in frame 2 of Figure 3.
Figure 5 is a diagram illustrating the polyp probability score according to the time trend of the polyp.
Figure 6 shows an example of an endoscopic image with the highest polyp indicated.
Figure 7 shows an example of an endoscopic image displaying an area where a plurality of detected polyps are gathered in close proximity.
Figure 8 shows an example of an endoscopic image with additional markings for polyps that are judged to be latent polyps for more than a certain period of time.
Figure 9 is a flowchart of a polyp detection method according to an embodiment of the present invention.
Figure 10 is a diagram illustrating in more detail the step of calculating a polyp probability score for a detected polyp based on the presence or absence of a polyp over time.

Since the present invention can make various changes and have various embodiments, specific embodiments will be described in detail with reference to the drawings. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention. While describing each drawing, similar reference numerals are used for similar components. Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component without departing from the scope of the present invention, and similarly, the second component may also be named a first component. The term “and/or” includes any of a plurality of related description items or a combination of a plurality of related description items. When a component is referred to as being “connected” or “connected” to another component, it should be understood that it may be directly connected to or connected to that other component, but that other components may exist in between. something to do. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between. The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof. Additionally, it is to be understood that the terms "first" and "second" are used herein for distinguishing purposes only and do not in any way indicate or anticipate order or priority. Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless clearly defined in the present application, be interpreted as having an ideal or excessively formal meaning. It doesn't work.

Throughout the specification and claims, when it is said that a part includes a certain component, this does not mean that other components are excluded, but that other components may be further included, unless specifically stated to the contrary. Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a block diagram of a polyp detection device according to an embodiment of the present invention.

Referring to Figure 1, the polyp detection device according to an embodiment of the present invention includes an image acquisition unit 10, a polyp detection unit 20, a polyp probability calculation unit 30, a polyp display determination unit 40, and a polyp display unit ( 50) may be included.

The image acquisition unit 10 according to an embodiment of the present invention acquires images captured by endoscopic equipment. For example, endoscope equipment can capture images at 60 frames per second, but is not necessarily limited to this. The image acquisition unit 10 according to an embodiment of the present invention may extract the endoscopic image at 20 or 40 frames per second, but is not necessarily limited thereto.

The polyp detection unit 20 according to an embodiment of the present invention can automatically detect the presence or absence of a polyp and polyp location information using a deep learning model for the extracted endoscopic image. Here, the deep learning-based polyp detection model is a model obtained by learning multiple endoscopic images, and can extract polyp location information from the endoscopic images.

For example, a deep learning polyp detection model uses rectangular polyp coordinates and a polyp score (in this specification, 'polyp score' is provided by processing the frame in question by a deep learning polyp detection model, scoring the probability that a specific area is a polyp). It is possible to detect the presence of a polyp by calculating the polyp score and comparing it with the threshold.

As described above, in the conventional polyp detection technology using deep learning, a specific area is judged to be a suspected polyp area for various reasons such as bubbles or foreign substances that appear and disappear momentarily, or the shooting angle, but is not judged to be a suspected polyp area. Occurs often. For example, if the threshold value of the polyp score for determining the suspected polyp area is, for example, 700, in one frame, the polyp score for the suspected polyp area is 700 or more (e.g., 720), and the polyp score for the suspected polyp area is 700 or more (e.g., 720), and the In the next frame, if the polyp score for the suspected polyp area is less than 700 (eg, 650), the suspected polyp area is displayed as a polyp in the endoscopic image, and then the square disappears. In other words, the polyp indicator blinks (false positive).

The present inventor focused on the fact that this problem cannot be fundamentally solved by raising the standard for determining polyps. In other words, raising the standard for polyp judgment may have the effect of reducing the false positive rate, but the problem of the polyp area indicator blinking still exists, and this cannot be completely solved. In addition, false positives mean that the examiner may need to bring the endoscopic imaging device closer to the suspected polyp area to ultimately determine whether there is a tumor, and these are noises and are not subject to removal. In other words, if the false positive rate is reduced by raising the standard for polyp detection, there is a problem that the risk of polyp detection failure increases.

According to one embodiment of the present invention, the polyp probability calculation unit 30 calculates a polyp probability score for the detected polyp based on the presence or absence of a polyp over time. That is, the polyp detected in the current frame can be compared with the polyp detected in the previous frame, and based on this comparison, a predetermined score can be applied to calculate a polyp probability score over time. For example, this polyp probability score may be calculated in such a way that if a polyp is detected in the frame, a predetermined score is added, and if a polyp is not detected in the frame, a predetermined score is subtracted, and the predetermined score is calculated cumulatively.

It is possible to determine whether the detected polyps are the same by comparing the position coordinates of the polyp detected in the frame with the position coordinates of the polyp detected in the previous frame. The polyp probability score calculation unit 30 calculates a polyp probability score by adding a predetermined score to the polyp probability score in the previous frame for the polyp detected in the previous frame and the current frame, and adds a polyp probability score to the polyp newly detected in the frame. For polyps that were detected in the previous frame but not detected in the current frame, a predetermined score is calculated as the polyp probability score, and the polyp probability score can be calculated by subtracting a predetermined score from the polyp probability score in the previous frame.

Figure 2 shows an example of an endoscopic image in which a polyp is detected and displayed in the first frame 1. Figure 3 shows an example of an endoscopic image in which a polyp is detected and displayed in frame 2, which is the next frame after the first frame 1.

Figure 4 is an example illustrating the polyp likelihood score calculated for each polyp in the first frame 1 of Figure 2 and the polyp likelihood score calculated for each polyp in frame 2 of Figure 3.

As can be seen in Figure 4, since the first polyp 1 was detected in frame 1 and frame 2, a predetermined score (e.g., 1) is added to determine the polyp probability of the first polyp 1 in frame 2. The score is 2. Since the second polyp 2 was detected in frame 1 but not in frame 2, a predetermined score (e.g., 1) is subtracted, and the polyp probability score of the second polyp 2 in frame 2 is 0. Since the third polyp (3) was not detected in frame 1 but was detected in frame 2, the polyp probability score is 1.

Figure 5 is a diagram illustrating the polyp probability score according to the time trend of the polyp.

As shown in Figure 5, as the number of polyps detected increases, the polyp probability score increases, and as the number of polyps not detected increases, the polyp probability score decreases.

According to one embodiment of the present invention, the polyp display determination unit 30 may vary the polyp display 60 according to the polyp probability score. For example, the gradation of the polyp mark 60 on the image may be varied. For example, the higher the score, the color tone can be changed step by step or continuously so that the examiner can intuitively recognize it.

As shown in Figures 2 and 3, the polyp probability score of the first polyp 1 increases over time, and may be displayed in a darker color tone in frame 2 than in frame 1. Conversely, the second polyp 2 may be displayed in a lighter shade in frame 2 than in frame 1, as the polyp probability score decreases over time. The third polyp 3 is newly detected in frame 2 and can be displayed in a color tone corresponding to the polyp probability score of 1. Here, it should be understood that the change in color tone is exaggerated for explanation purposes. That is, considering that the frame per second is 60, the color tone of the first polyp 1 is not set to change noticeably as shown in Figures 2 and 3 as it moves from frame 1 to frame 2. This may not be the case, and in reality, the color tone may be set to change more gradually.

Based on the disclosure of Figures 2 and 3, when comparing the existing endoscopic detection method and the present invention, in the existing endoscopic detection method, the second polyp 2 is not detected in frame 2, so the polyp is displayed in frame 2. disappears, the second polyp (2) blinks in the endoscopic image. On the other hand, in the present invention, the second polyp (2) still has a polyp mark (60), and the examiner can intuitively recognize the possibility of a polyp of the second polyp (2) through the color tone of the polyp mark (60). .

Additionally, according to one embodiment of the present invention, if the polyp probability score is less than a predetermined score, the polyp may not be displayed. For example, if the polyp probability score is -4 or less, the polyp may not be displayed in the frame.

According to one embodiment of the present invention, the polyp probability score is based on the above cumulative method, but may be additionally adjusted. For example, if polyps are detected continuously (e.g., polyps detected more than 5 times), 3 can be additionally added to the polyp probability score, and if polyps are not detected continuously (e.g., polyps not detected more than 5 times), 3 can be additionally added to the polyp probability score. , additionally, 3 can be subtracted from the polyp probability score.

In addition, although the score is explained as being accumulated by adding or subtracting the same score (e.g., 1) depending on whether or not a polyp is detected, the score may not be the same score. For example, a score may be calculated in proportion to the difference between the polyp score and the reference value, and this score may be added or subtracted depending on whether or not a polyp is detected. That is, if the polyp score is 780 and the reference value is 700, the difference is 80, and if the polyp score is 900, the difference is 200. In this case, the points added to the polyp likelihood score are not the same, and may be set in proportion to the difference and added to the polyp likelihood score.

According to one embodiment of the present invention, for example, when the polyp display is varied according to the polyp probability score, there may be a highest polyp display 62 in which the polyp display no longer changes even if the polyp probability score increases. For example, if the polyp probability score is 10 or more, the polyp indication may no longer change. In this case, the polyp indication for polyps with a polyp probability score of 10 or more is the highest polyp indication (62), which is recognized by the examiner as a polyp that is highly suspected to be a polyp.

When displaying the highest polyp 62, a polyp display method that is different from the display method of the remaining polyp displays can be applied. For example, as shown in Figure 6, if the remaining polyp display method is a color change method, the highest polyp display 62 with a polyp probability score of 10 or more is displayed in a different color or has a different polyp display shape (e.g., It can be displayed by changing the square display to a circular display. Through this, the examiner can intuitively recognize potential polyps and final suspected polyps.

In this way, the final suspected polyp and the potential polyp are distinguished, but the possibility of a potential polyp is intuitively recognized for the potential polyp, so even if the number of polyps is somewhat increased, the examiner can easily figure out which area should be diagnosed through endoscopy. You can. Accordingly, there is less need to reduce polyp indications by raising the standard for polyp judgment, and the risk of polyp detection failure can be lowered.

According to one embodiment of the present invention, when a plurality of detected polyps are gathered together on an endoscope image, a method of displaying an area where a plurality of detected polyps are gathered may be applied, rather than a method of displaying individual detected polyps. For example, according to one embodiment of the present invention, not only the final suspected polyp but also the potential polyp is displayed differently according to the passage of time, so there may be many polyp indications. From the examiner's point of view, in order to reduce the risk of polyp detection failure, it is necessary to check areas with many potential polyps by approaching the area with an endoscopic imaging device. Therefore, so that the examiner can intuitively recognize areas with many potential polyps and guide the movement of the endoscope, when multiple detected polyps with polyp probability scores below a certain score are gathered in close proximity, individual detected polyps are not displayed, but rather are displayed. , it is possible to display an area where multiple detected polyps are gathered in close proximity. Figure 7 is an example of a diagram showing a polyp display 66 indicating an area where a plurality of detected polyps are gathered in close proximity.

According to one embodiment of the present invention, there may be cases where a latent polyp is repeatedly judged to be a polyp and not judged to be a polyp depending on the frame. From the examiner's point of view, in order to reduce the risk of polyp detection failure, these potential polyps need to be confirmed by accessing the endoscope camera. In order for the examiner to intuitively recognize these potential polyps and guide the movement of the endoscope, if a polyp is judged to be a potential polyp for more than a certain period of time (i.e., if the polyp probability score stays within a certain range for a certain period of time), a mark indicating this is additionally added. can do. Figure 8 is an example of an endoscopic image with additional markings 68 for polyps that are judged to be latent polyps for more than a certain period of time.

According to one embodiment of the present invention, when more accurate judgment is required, the presence or absence of polyps and polyp location coordinates can be detected by applying a more accurate deep learning model.

As described above, the deep learning model according to an embodiment of the present invention is a model that learns the location of polyps from a plurality of endoscopic images, calculates polyp location coordinates and polyp scores, and determines whether this polyp score exceeds a predetermined reference value. Depending on whether it is a polyp or not, it is determined whether it is a polyp or not. According to one embodiment of the present invention, while detecting the presence or absence of a polyp and the polyp location coordinates by applying the first deep learning model, when a more accurate judgment is required, a second deep learning model with a reference value higher than the reference value of the first deep learning model is used. A learning model can be applied. Through this, the final suspected polyp can be judged based on more stringent standards and displayed.

For example, as described above, when the polyp display is different depending on the polyp probability score, there may be a highest polyp display 62 in which the polyp display no longer changes even if the polyp probability score increases. Since this highest polyp mark 62 is recognized by the examiner as a polyp that is highly suspected of being a polyp, it may be desirable to determine whether or not to mark the highest polyp through more accurate judgment. Therefore, if the best polyp indication (62) is provided when the polyp likelihood score is 10, the first deep learning model is applied until the polyp likelihood score reaches 8, and then the first deep learning model is used for polyps with a polyp likelihood score of 8 or more. A second deep learning model with a reference value higher than the reference value of the learning model can be applied.

According to another embodiment of the present invention, the first deep learning model and the second deep learning model may be prepared at different processing speeds per frame. Typically, the lower the processing speed of a deep learning model (i.e., the longer it takes to process), the more accurate polyp detection is possible. Therefore, when a deep learning model with a low processing speed is used, the processing speed per frame becomes slow, and there is a risk that real-time processing of the endoscopic image will be delayed. Considering this, in one embodiment of the present invention, while detecting the presence or absence of a polyp and polyp location coordinates by applying the first deep learning model, when a more accurate judgment is required, the processing speed per frame is higher than that of the first deep learning model. A second deep learning model with a low value can be applied. Through this, concerns about real-time processing delays for endoscopic images can be minimized, while polyps can be displayed by judging certain polyps based on stricter standards.

For example, the first deep learning model can be applied until the polyp likelihood score reaches 8, and then the second deep learning model with a low processing speed per frame can be applied to polyps with a polyp likelihood score of 8 or more. Alternatively, if a latent polyp is repeatedly judged to be a polyp and not judged to be a polyp depending on the frame, an accurate judgment regarding the polyp may be required from the examiner's point of view. That is, if the time smaller than the polyp probability score corresponding to the highest polyp indication 62 continues for more than a certain period of time, the second deep learning model with a low processing speed per frame can be applied.

Figure 9 is a flowchart of a polyp detection method according to an embodiment of the present invention.

Referring to FIG. 9, the polyp detection method according to an embodiment of the present invention includes receiving an endoscopic image taken from the human body (S100), and determining the presence or absence of a polyp using a deep learning model for the received endoscopic image. And a step of detecting the location information of the polyp (S200), a step of calculating a polyp probability score for the detected polyp based on the presence or absence of the polyp over time (S300), and based on the calculated score, an endoscope Determining the polyp mark on the image for each frame (S400); It may include outputting the determined polyp mark on the endoscope image (S500).

Figure 10 is a diagram illustrating in more detail the step of calculating a polyp probability score for a detected polyp based on the presence or absence of a polyp over time. As shown in Figure 10, the step of calculating a polyp probability score for a polyp detected based on the presence or absence of a polyp over time (S300) involves comparing the polyp detected in the frame with the polyp detected in the previous frame. It may be comprised of a step (S320) and a step (S340) of calculating a polyp probability score over time by applying a predetermined score based on the comparison result.

The device described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general-purpose or special-purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may perform an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device may include multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible. Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium, or It may be permanently or temporarily embodied in a device or a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media. The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. Computer-readable media may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa. The above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications and variations will be possible to those skilled in the art without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

1: Primary polyp
2: Secondary polyp
3: Tertiary polyp
10: Image acquisition unit
20: Polyp detection unit
30: Polyp probability calculation unit
40: Polyp indication decision part
50: Polyp display unit
60: Polyp sign
62: Highest polyp display

Claims (19)

A polyp detection method using a polyp detection device including an image acquisition unit, a polyp detection unit, a polyp probability calculation unit, a polyp display determination unit, and a polyp display unit,
receiving, by the image acquisition unit, an endoscopic image captured in the human body;
The polyp detection unit detects the presence or absence of a polyp and location information of the polyp using a deep learning model with respect to the received endoscopic image;
The polyp probability calculation unit calculates a polyp probability score for the detected polyp based on the presence or absence of the polyp over time;
The polyp display determination unit determines a polyp display on the endoscope image for each frame based on the calculated score;
The polyp display unit includes a step of displaying the determined polyp display on the endoscope image,
The step of calculating a polyp probability score for a detected polyp based on the presence or absence of a polyp over time is,
Comparing the polyp detected in the current frame with the polyp detected in the previous frame;
Based on the comparison, applying a predetermined score to calculate a polyp probability score over time,
The step of calculating the polyp probability score over time by applying the predetermined score is,
For polyps detected in the previous frame and the current frame, a polyp probability score is calculated by adding a predetermined score to the polyp probability score in the previous frame, and for polyps newly detected in the current frame, a predetermined score is calculated as the polyp probability score. And, for polyps detected in the previous frame but not detected in the current frame, a polyp probability score is calculated by subtracting a predetermined score from the polyp probability score in the previous frame, and the polyp probability score is a cumulative value of the predetermined scores. doing,
Polyp detection method using a polyp detection device.
delete delete According to paragraph 1,
The polyp probability calculation unit determines the predetermined score based on the difference between the polyp score provided by the deep learning model and the polyp detection reference value,
Polyp detection method using a polyp detection device.
According to paragraph 1,
If the polyp probability score is below the reference value, the polyp display unit does not display the polyp for the polyp.
Polyp detection method using a polyp detection device.
According to paragraph 1,
The polyp display determination unit determines a gradation of the polyp display according to the polyp probability score,
Polyp detection method using a polyp detection device.
According to clause 6,
The top polyp mark is different in color or shape from the rest of the polyp marks.
Polyp detection method using a polyp detection device.
In clause 7,
When a plurality of polyps with a polyp probability score smaller than the polyp probability score corresponding to the highest polyp display are gathered, the polyp display unit displays the area where the polyps are gathered,
Polyp detection method using a polyp detection device.
In clause 7,
If the polyp probability score is less than the polyp probability score corresponding to the highest polyp indication persists for more than a certain period of time, the polyp display unit displays the polyp indication on the endoscope image as an indicator indicating this,
Polyp detection method using a polyp detection device.
A polyp detection device using endoscopic images,
An image acquisition unit that receives an endoscopic image taken from the human body,
A polyp detection unit that detects the presence or absence of a polyp and location information of the polyp using a deep learning model for the received endoscopic image;
A polyp probability calculation unit that calculates a polyp probability score for the detected polyp based on the presence or absence of polyp over time,
A polyp display determination unit that determines the polyp display on the endoscope image for each frame based on the calculated score;
It includes a polyp display unit that displays the determined polyp display on the endoscope image,
In the polyp probability calculation section,
Compare the polyp detected in the current frame with the polyp detected in the previous frame, and based on the comparison, apply a predetermined score to calculate a polyp probability score over time,
In calculating the polyp probability score, the polyp probability calculation unit calculates the polyp probability score by adding a predetermined score to the polyp probability score in the previous frame for the polyp detected in the previous frame and the current frame, and calculates the polyp probability score anew in the frame. For detected polyps, a predetermined score is calculated as the polyp probability score, and for polyps detected in the previous frame but not detected in the current frame, a predetermined score is subtracted from the polyp probability score in the previous frame to calculate the polyp probability score. The polyp probability score is the cumulative value of a predetermined score,
Polyp detection device using endoscopic images.
delete delete According to clause 10,
The predetermined score is determined based on the difference between the polyp score provided by the deep learning model and the polyp detection reference value,
Polyp detection device using endoscopic images.
According to clause 10,
If the polyp probability score is below the standard value, the polyp is not indicated.
Polyp detection device using endoscopic images.
According to clause 10,
The gradation of the polyp display is determined according to the polyp probability score,
Polyp detection device using endoscopic images.
According to clause 15,
The top polyp mark is different in color or shape from the rest of the polyp marks.
Polyp detection device using endoscopic images.
According to clause 16,
If multiple polyps with a polyp probability score smaller than the polyp probability score corresponding to the highest polyp display are gathered, the area where the polyps are gathered is displayed.
Polyp detection device using endoscopic images.
According to clause 16,
If the polyp probability score is lower than the polyp probability score corresponding to the highest polyp indication for more than a certain period of time, a mark indicating this is applied to the polyp indication,
Polyp detection device using endoscopic images.
It is a polyp detection program using endoscopic images.
The polyp detection program is a computer program coupled to a processor and stored in a computer-readable recording medium,
The computer program is configured to execute the polyp detection method according to one of claims 1 and 4 to 8 on the processor by instructions stored in the recording medium.