KR102684605B1 - System and method for analyzing upper gastrointestinal endoscopy image, and a recording medium having computer readable program for executing the method - Google Patents

Abstract

The upper gastrointestinal endoscopy image analysis system includes an anatomical location map setting unit, an image acquisition unit, an anatomical location label determination unit, an anatomical location map output unit, and an anatomical observation path output unit. The anatomical position map setting unit sets a label list of anatomical positions of the upper gastrointestinal tract to be photographed, and the image acquisition unit acquires images from a camera attached to the end of the upper gastrointestinal endoscope insertion tube and transmits them to the anatomical position label determination unit. , the anatomical position label determination unit 130 determines the label of the anatomical position from the transmitted image, and the anatomical position map output unit outputs the anatomical position map using the determined label of the anatomical position. Additionally, the anatomical observation path output unit outputs the path along which anatomical observation was made during the endoscopic procedure and accompanying information on the anatomical location map.

Description

Upper gastrointestinal endoscopy image analysis system, method, and recording medium recording a computer-readable program for executing the method {SYSTEM AND METHOD FOR ANALYZING UPPER GASTROINTESTINAL ENDOSCOPY IMAGE, AND A RECORDING MEDIUM HAVING COMPUTER READABLE PROGRAM FOR EXECUTING THE METHOD}

The present invention relates to an upper gastrointestinal endoscope image analysis technology for diagnosing and treating lesions by inserting an endoscope into the human body and directly observing the esophagus, stomach, and duodenum bulb through images. More specifically, it relates to an upper gastrointestinal endoscope image analysis technology that effectively performs endoscopic imaging of the upper gastrointestinal tract. This relates to a system that provides upper gastrointestinal endoscopy image analysis information so that it can be photographed and analyzed with completeness.

Upper Gastrointestinal Endoscopy is a method of diagnosing and treating lesions by inserting an endoscope into the human body and directly observing images of the esophagus, stomach, and duodenal bulb. During endoscopy, detailed observation and photography are very important to avoid blind spots regarding the anatomical location. In particular, the photographs taken are used for exchange of opinions among medical staff, follow-up observation of lesions, education, explanations to examinees, and future use. It is very important for reasons such as quality control of endoscopes and response to potential medical lawsuits.

For this reason, the European Society of Gastrointestinal Endoscopy recommends taking at least 8 to 10 images from major areas of the esophagus, stomach, and duodenum, even when there are no lesions, to manage the endoscopic quality.

In addition, endoscopic procedures require a lot of skill, as the Korean Society of Gastrointestinal Endoscopy requires that the procedure be performed hundreds of times (approximately 1,000 cases of gastroscopy and 150 cases of colonoscopy) in the presence of a specialist after obtaining a specialist qualification.

As mentioned above, academic guidelines are provided for correct insertion, accurate observation, and appropriate photography to avoid blind spots in key anatomical locations during endoscopic surgery, and endoscopic photography and procedures suitable for quality control are provided. emphasizes the importance of

However, in the prior art, complete anatomical observation with an endoscope without blind spots is entirely dependent on the experience and skill of the performing medical staff. For this reason, endoscopic medical staff who do not have much experience are required to perform the procedure thoroughly without blind spots. It requires a lot of time and effort.

In other words, when performing upper gastrointestinal endoscopy, conventional upper gastrointestinal endoscopic imaging recommendations require images to be taken at multiple specific anatomical locations even when there is no lesion, and the upper part of the esophagus, stomach, duodenum, etc. according to each experience of the medical staff. Images were taken at anatomical locations in the gastrointestinal tract.

Therefore, endoscopic medical staff who do not have much experience must not only keep in mind the anatomical location and observation path that must be taken for accurate procedures, but also take images according to the observation path during the insertion and retrieval stages. Because you have to continuously think and judge about the plan during the procedure, more time and money are required.

Sometimes, even experienced medical staff miss capturing images at a specific anatomical location during the insertion and retrieval stage, and missing images at a specific anatomical location results in a decrease in the quality control of the endoscope.

KR 101049507 B1

In the present invention, the problems raised in the prior art are solved by solving the problems raised in the prior art, by outputting the path along which anatomical observation was made during the endoscopic procedure and the accompanying time information on the anatomical location map, so that the anatomical location or lesion missed at the insertion stage during the endoscopic procedure The purpose is to visualize and provide observation route information for suspicious areas, etc., and to enable medical staff to refer to this and perform a complete endoscopic examination at the retrieval stage.

In addition, even in the stage of analyzing images taken after upper gastrointestinal endoscopy, the path through which anatomical observation was made during the endoscopic procedure and accompanying time information are output to the anatomical location map, so that images are taken at the insertion and retrieval stages during the endoscopic procedure. The purpose is to enable each medical staff to perform endoscopic examinations with completeness and efficiency in future endoscopic procedures by enabling analysis of the anatomical distribution information of each image and the behavior according to the time required.

To achieve the above object, the upper gastrointestinal endoscopy image analysis system according to the present invention includes an anatomical location map setting unit, an image acquisition unit, an anatomical location label determination unit, an anatomical location map output unit, and an anatomical observation path output unit. do.

According to this configuration, the anatomical position map and observation path are output on the screen where the endoscopic image is output or at a random location of the endoscope system, thereby visualizing the anatomical position map and observation path of the images acquired in the insertion and retrieval stages. By providing this, the medical staff can refer to this and perform a complete endoscopic examination at the retrieval stage.

In addition, in the step of analyzing the captured image after the upper gastrointestinal endoscopy, the label of the anatomical position determined for the stored image is used to output it to the anatomical position map, and the anatomy during the endoscopic procedure is displayed on the anatomical position map. By outputting the path of observation and accompanying time information, it is possible to analyze the anatomical distribution information of each image taken during the insertion and retrieval stages during the endoscopic procedure and the behavior according to the time required, which can be used later in the endoscopic procedure. Each medical staff member will be able to perform endoscopic examinations with completeness and efficiency.

At this time, in the anatomical position map setting unit, the main anatomical positions to be observed during the endoscopic procedure can be set by combining multiple types of anatomical diagrams and anatomical position labels. According to this configuration, an anatomical location map that is familiar and intuitive to medical staff in the field of upper gastrointestinal endoscopy can be set, thereby increasing user convenience.

In addition, in addition to location maps for arbitrary anatomical locations, it is possible to set up an anatomical location map of the upper gastrointestinal tract for more simplified locations by compartmentalizing major anatomical imaging locations, making it easier to set up a location map. do.

The anatomical position label determination unit uses an artificial intelligence algorithm to determine the position label of the image transmitted from the image acquisition unit. The artificial intelligence algorithm used to determine the location label of an image is a method of extracting and using hand-crafted features, or a method using deep learning, by applying a number of artificial intelligence algorithms. It can be implemented.

The anatomical location map output unit displays and outputs an anatomical location map using the location label determined and delivered by the anatomical location label determination unit. It determines whether or not an image is acquired for a location preset in the anatomical location map and outputs it on the location map. Additionally, images acquired at a location not preset in the anatomical location map are also output to the location map.

With this configuration, it is possible to prevent missing images from an already established anatomical location, and it is also possible to recognize whether a significant area is suspected to be a lesion or requires additional observation, thereby managing the quality of the endoscopic procedure. can be improved.

According to the present invention, the anatomical position label determined for the image acquired during the upper gastrointestinal endoscopy is output to the anatomical position map, and the anatomical position map includes the path where anatomical observation was made during the endoscopic procedure. By outputting auxiliary time information, the observation path information for anatomical locations or suspected lesion areas missed during the insertion stage during endoscopic procedures is visualized and provided, and the medical staff can refer to this to perform a complete endoscopic examination at the retrieval stage. It becomes possible.

In addition, in the step of analyzing the captured image after the upper gastrointestinal endoscopy, the label of the anatomical position determined for the stored image is used to output it to the anatomical position map, and the anatomy during the endoscopic procedure is displayed on the anatomical position map. By outputting the path of observation and accompanying time information, it is possible to analyze the anatomical distribution information of each image taken during the insertion and retrieval stages during the endoscopic procedure and the behavior according to the time required, which can be used later in the endoscopic procedure. Each medical staff member will be able to perform endoscopic examinations with completeness and efficiency.

At this time, not only does it determine whether or not an image is acquired for a location preset in the anatomical location map and output it to the location map, but it also outputs an image acquired in a location not preset in the anatomical location map to the location map, thereby detecting the lesion. It is possible to easily observe significant areas that are suspicious or require additional observation.

In addition, by using simple schematic diagrams and expanded diagrams familiar to endoscopic medical staff, it is possible to set anatomical positions familiar to each medical staff's experience, enabling intuitive procedures and enabling easier and faster endoscopic observation. do.

1 is a schematic block diagram of an endoscopic image analysis system during an endoscopic procedure, according to an embodiment of the present invention.
Figure 2 is a schematic flowchart of an analysis method performed by the endoscopic image analysis system of Figure 1;
Figure 3 is a schematic block diagram of an endoscopic image analysis system when analyzing images taken after an endoscopic procedure, according to another embodiment of the present invention.
FIG. 4 is a schematic flowchart of an analysis method performed by an endoscopic image analysis system when analyzing an image taken after an endoscopic procedure according to the embodiment of FIG. 3.
Figure 5 is a table showing examples of anatomical location labels and descriptions of the upper gastrointestinal tract.
Figures 6 and 7 show examples of simple and expanded schematic diagrams of the upper gastrointestinal tract.
Figures 8 and 9 are, respectively, an example view of an anatomical location map of the upper gastrointestinal tract using a simple schematic diagram, and an example view of an anatomical location map of the upper gastrointestinal tract using an expanded schematic diagram.
10 and 11 are diagrams showing an example output of an anatomical position map of the upper gastrointestinal tract using a simple schematic diagram and an example diagram of an anatomical position map output of the upper gastrointestinal tract using an expanded schematic diagram, respectively.
12 to 14 are diagrams illustrating output examples of the steps of setting, inserting, and retrieving an anatomical location map and observation path of the upper gastrointestinal tract using a simple schematic diagram.
Figures 15 to 17 are diagrams showing example outputs of the setting, insertion, and retrieval stages of an anatomical location map and observation path of the upper gastrointestinal tract using an expanded schematic diagram.

Hereinafter, preferred embodiments of the present invention will be described with reference to the attached drawings.

FIG. 1 is a schematic block diagram of an endoscopic image analysis system during an endoscopic procedure, according to an embodiment of the present invention, and FIG. 2 is a schematic flowchart of an endoscopic image analysis system according to an embodiment of the present invention. In FIG. 1, the endoscopic image analysis system 100 includes an anatomical position map setting unit 110, an image acquisition unit 120, an anatomical position label determination unit 130, an anatomical position map output unit 140, and an anatomical position map output unit 140. It includes an enemy observation path output unit 150.

The anatomical position map setting unit 110 sets a label list of anatomical positions of the upper gastrointestinal tract to be photographed, and the image acquisition unit 120 acquires images from a camera attached to the end of the upper gastrointestinal endoscope insertion tube to capture the anatomy. The anatomical position label determination unit 130 determines the label of the anatomical position from the transmitted image, and the anatomical position map output unit 140 determines the determined anatomical position. Print it on the anatomical location map using the labels. Additionally, the anatomical observation path output unit 150 outputs the path along which anatomical observation was made during the endoscopic procedure and accompanying information on the anatomical location map.

Figure 3 is a schematic block diagram of an endoscopic image analysis system when analyzing images taken after an endoscopic procedure according to an embodiment of the present invention, and Figure 4 is a schematic block diagram of an endoscopic image analysis system taken after an endoscopic procedure according to an embodiment of the present invention. This is a schematic flowchart of the endoscopic image analysis system when analyzing one image.

In FIG. 3, the endoscopic image analysis system 300 includes an anatomical location map setting information acquisition unit 310, a storage image acquisition unit 320, an anatomical location label determination unit 330, and an anatomical location map output unit 340. , and includes an anatomical observation path output unit 350.

First, the anatomical location map setting information and the stored image are read and transmitted to the anatomical location label determination unit 330, and the anatomical location label determination unit 330 determines the label of the anatomical location from the delivered image. The anatomical location map output unit 340 uses the label of the determined anatomical location to output it to the anatomical location map. The anatomical observation path output unit 350 outputs the path along which anatomical observation was made during the endoscopic procedure and accompanying information on the anatomical location map.

Upper gastrointestinal endoscopy is a method of diagnosing and treating lesions by inserting an endoscope with a small camera into the human body and directly observing images of the esophagus, stomach, and duodenum.

In order to verify that upper gastrointestinal endoscopy has been appropriately performed to control the quality of endoscopic procedures, it is necessary to take images at multiple specific anatomical locations even when there is no lesion during upper gastrointestinal endoscopy, to exchange opinions among medical staff, It is very important for reasons such as follow-up of lesions, education, explanation to examinees, quality management of future endoscopy, and response to potential medical lawsuits.

As above, when performing upper gastrointestinal endoscopy, the medical staff performing the procedure plans and performs the procedure by considering the main anatomical locations to be observed during the endoscopic procedure, according to each experience and the society's recommendations for quality control. The main anatomical positions to be observed, which are basically planned by the medical staff, are designated and set in advance in the anatomical position map setting unit 110.

To set the main anatomical positions to be observed in the anatomical position map setting unit 110, the anatomical label list of the upper gastrointestinal tract can be used, or the anatomical schematic diagram and label can be used together. Figure 5 is a table showing examples of anatomical location labels and descriptions of the upper gastrointestinal tract, and Figures 6 and 7 are diagrams showing examples of simple schematic diagrams and expanded schematic diagrams of the upper gastrointestinal tract.

Using the anatomical location labels and schematic diagrams shown in FIGS. 5 to 7, an anatomical location map of the upper gastrointestinal tract can be established for any specific anatomical locations to be observed, as shown in FIGS. 8 and 9. Figures 8 and 9 are, respectively, an example view of an anatomical location map of the upper gastrointestinal tract using a simple schematic diagram, and an example view of an anatomical location map of the upper gastrointestinal tract using an expanded schematic diagram.

The anatomical location label determination unit 130 determines the location label of the image transmitted from the image acquisition unit 120 using an artificial intelligence algorithm and transmits it to the anatomical location map output unit. The method of determining the location label of the image is to extract and use hand-crafted features of the image, or to apply a number of artificial intelligence algorithms, such as deep learning. It can be implemented.

In the anatomical position map output unit 140, the position label determined by the anatomical position label determination unit 130 for the acquired image and transmitted is the label set in the anatomical position map setting unit 110 previously set. If they match, they are displayed and output as shown in Figures 10 and 11.

Figures 10 and 11 are diagrams showing an example output of an anatomical position map of the upper gastrointestinal tract using a simple schematic diagram and an example diagram of an anatomical position map output of the upper gastrointestinal tract using an expanded schematic diagram, respectively. In Figures 10 and 11, a green circle is a location where an anatomical location map is set and an image is acquired, a red circle is a location where an anatomical location map is set and an image is not acquired, and a yellow circle is a location where an anatomical location map is not set. It indicates the location where the image was acquired.

As described above, in addition to outputting the anatomical position map of the upper gastrointestinal tract, examples of the observation path output during the anatomical position map setting and endoscope insertion and retrieval stages during endoscopic procedures are shown in Figures 12 to 17.

Figures 12 to 14 are illustrations of an anatomical location map of the upper gastrointestinal tract using a simple schematic diagram and output examples of the observation path setting, insertion, and retrieval steps, and Figures 15 to 17 are an anatomical location map of the upper gastrointestinal tract using an expanded schematic diagram. and an example output diagram of the observation path setting, insertion, and retrieval steps.

In Figures 12 to 17, the green circle is a location where the anatomical location map is set and an image is acquired, the red circle is a location where the anatomical location map is set and the image is not acquired, and the yellow circle is a location where the anatomical location map is not set. Each location where the image was acquired is indicated, and the arrow indicates the anatomical observation path.

Although the present invention has been described in terms of some preferred embodiments, the scope of the present invention should not be limited thereby, but should extend to modifications and modifications of the above embodiments as supported by the claims.

100, 300: Upper gastrointestinal endoscopy image analysis system
110: Anatomical location map setting unit
120: Image acquisition unit
130, 330: Anatomical location label determination unit
140, 340: Anatomical location map output unit
150, 350: Anatomical observation path output unit
310:: Anatomical location map setting information acquisition unit
320: Storage image acquisition unit

Claims (12)

an anatomical location map setting unit that sets labels corresponding to at least some anatomical locations of the upper gastrointestinal tract;
An image acquisition unit that acquires an endoscopic image of the upper gastrointestinal tract;
an anatomical position label determination unit that determines whether the label corresponding to the acquired image is set;
an anatomical location map output unit that outputs the determined label when the label set corresponding to the acquired image is determined; and
An upper gastrointestinal endoscopy image analysis system comprising an anatomical observation path output unit that outputs path information including image acquisition order information corresponding to the output labels.
In claim 1,
an anatomical location map setting information acquisition unit that acquires preset label information corresponding to the anatomical locations; and
Further comprising a storage image acquisition unit that acquires a pre-stored storage image of the endoscope of the upper gastrointestinal tract,
The anatomical position label determination unit determines whether the label corresponding to the acquired stored image is set,
The anatomical location map output unit outputs a label corresponding to the stored image,
The anatomical observation path output unit further outputs path information including shooting order information of stored images corresponding to the output labels.
In claim 1,
The upper gastrointestinal endoscopy image analysis system wherein the anatomical observation path output unit further outputs output time information of the output labels.
In claim 1,
An upper gastrointestinal endoscopy image analysis system, characterized in that the output of the label is performed on a preset anatomical location map.
In claim 1,
An upper gastrointestinal endoscopy image analysis system, wherein the label is set using a preset anatomical label list.
In claim 1,
An upper gastrointestinal endoscopy image analysis system, wherein the label is set using a preset anatomical diagram.
In claim 1,
An upper gastrointestinal endoscopy image analysis system, wherein the determination of whether the label is set is performed by an artificial intelligence algorithm.
In claim 1,
The anatomical location map output unit further outputs labels that are not determined to correspond to the acquired image among the set labels, distinguishing them from the determined labels.
In claim 8,
The anatomical location map output unit outputs a label that is distinct from the output labels in response to the acquired image when an image is acquired for anatomical locations in the upper gastrointestinal tract for which no corresponding label has been set. An upper gastrointestinal endoscopy image analysis system characterized by further output.
In claim 9,
The anatomical observation path output unit performs an image acquisition procedure corresponding to the labels output when images are acquired for anatomical positions for which no corresponding labels are set among the anatomical positions of the upper gastrointestinal tract. An upper gastrointestinal endoscopy image analysis system characterized in that it further outputs path information including information.
An upper gastrointestinal endoscopic image analysis method performed by an upper gastrointestinal endoscopic image analysis system, comprising:
An anatomical location map setting step of setting labels corresponding to at least some anatomical locations of the upper gastrointestinal tract;
An image acquisition step of acquiring an endoscopic image of the upper gastrointestinal tract;
An anatomical location label determination step of determining whether the label corresponding to the acquired image is set;
an anatomical location map output step of outputting the determined label when the label set corresponding to the acquired image is determined; and
An upper gastrointestinal endoscopic image analysis method comprising an anatomical observation path output step of outputting path information including image acquisition order information corresponding to the output labels.
A recording medium recording a computer-readable program for executing the method of claim 11.

Images