JP4749732B2 - Medical image processing device - Google Patents

Description

The present invention relates to determination of properties of interest from the large amount of image data to an organism to efficiently perform medical image processing equipment.

Conventionally, in endoscopy using an endoscope, in-vivo image data captured by an endoscope device or an endoscope observation device is immediately displayed on a display device such as a CRT, It is generally stored externally as moving image data, and a doctor performs a diagnosis by viewing a moving image or a frame image in the moving image as a still image during or after an examination.
In recent years, a swallowable capsule endoscope has been used.
For example, as disclosed in Japanese Patent Application Laid-Open No. 2004-645, image data captured in the body by a capsule endoscope is sequentially accumulated as moving image data by wireless communication, and a doctor moves a moving image or moving image. The frame image inside is viewed as a still image and diagnosed.

Also, as disclosed in Japanese Patent Application Laid-Open No. 2004-188026, there is an apparatus that applies image analysis processing to a still image and displays the analysis result on an endoscopic image or another display area. .
By using this image analysis result, a doctor can make a diagnosis based on an objective determination criterion such as an image analysis value such as IHb or blood vessel analysis, which is not subjective.
JP 2004-645 A JP 2004-188026 A

However, when viewing a moving image after an endoscopy by a doctor or viewing a moving image using a capsule endoscope, the number of frame images included in the moving image is enormous. It is a very labor-intensive work to find out from among the images, extract the frame images of each part one by one, apply image analysis processing, and make a diagnosis based on each image analysis result. Become.
For such a problem, the image analysis apparatus for the still image is applied, the same image analysis processing is applied to all the frame images included in the moving image, and the result is stored. It is possible to realize the system.
However, since the same image analysis processing is applied to all the frame images at once, the processing time increases, and it is necessary to wait for a long time until the processing result is obtained. Also, when applying image analysis processing by changing parameters, it takes a long time to obtain an appropriate processing result. In addition, there is a drawback that the amount of data that needs to be stored before obtaining an appropriate analysis processing result also increases.

In screening for esophageal examination using an endoscopic device, the presence or absence of Barrett mucosa or Barrett's esophagus is examined. The Barrett mucosa is a gastric esophageal junction as a connection between the esophagus and the stomach where the squamous epithelium forming the esophagus is replaced by the gastric mucosa due to the influence of reflux esophagitis or the like, and is also called a columnar epithelium. A diagnosis of Barrett's esophagus is made when this Barrett mucosa occurs 3 cm or more from the normal mucosal boundary and circumferentially with respect to the cross section of the esophageal lumen.
Barrett's esophagus is increasing, especially in Westerners, and is a major problem because adenocarcinoma occurs with a high probability, so early detection of Barrett's mucosa is very important.
Therefore, there is a demand for a medical image processing apparatus that can objectively determine biometric features such as Barrett's esophagus and Barret's mucosa and provide the determination result to the operator.

(Object of invention)
The present invention has been made in view of the above, to provide a medical image processing equipment that can be efficiently determined from a large amount of image data such as moving image data of the attention to characteristics such as Barrett's esophagus Objective.

The medical image processing apparatus of the present invention includes image extracting means for extracting frame image data from in-vivo moving image data captured by an in-vivo imaging device or a plurality of still image data captured continuously,
Image analysis means for analyzing the frame image data extracted by the image extraction means, and outputting an image analysis result;
With
The image analysis means includes
First biological feature detection means for detecting a first biological feature;
Based on the detection result of the first biometric feature detection means, a second biometric feature is obtained with respect to a frame image captured forward or backward in time from the image used for detection by the first biometric feature detection means. Second biological feature detection means for detecting features;
Based on the detection result of the second biological feature detection means, a property determination means for determining the properties of the living body and outputting the determination result;
It is characterized by providing.
With the above configuration, when determining the properties of the living body, the second biological feature can be detected after the first biological feature is detected. Therefore, attention is paid to the Barrett's esophagus from a large amount of image data. This makes it possible to determine the property to be performed efficiently. That is, when the first biometric feature is not detected, the detection of the second biometric feature is omitted so that efficient image analysis processing can be performed.

  According to the present invention, it is possible to efficiently determine the property of interest such as Barrett's esophagus even in the case of a large amount of image data.

  Embodiments of the present invention will be described below with reference to the drawings.

  1 to 16 relate to a first embodiment of the present invention, FIG. 1 shows an overall configuration of an endoscope system including the first embodiment of the present invention, and FIG. 2 shows an internal configuration by oral insertion of an endoscope. FIG. 3 shows an example of an endoscopic image obtained by imaging the vicinity of the boundary between the esophagus and the stomach, and FIG. 4 shows a functional view of the image processing apparatus in this embodiment. FIG. 5 shows that moving image data stored in the image storage unit is stored as a collection of still image data. FIG. 6 shows analysis results stored in the analysis information storage unit, information stored in the processing program storage unit, and the like. FIG. 7 shows a monitor display example in which the analysis result is displayed together with the endoscopic image. The flowchart of the process sequence which performs the property determination of Barrett's esophagus by a present Example is shown, FIG. 9 shows the process sequence which performs the detection process of a gastroesophageal junction part with information, such as an image used or produced | generated.

10 shows a fence-like blood vessel end point boundary and the like, FIG. 11 shows a flowchart of the fence-like blood vessel extraction process in FIG. 9, FIG. 12 shows an image example for explaining the operation when performing the process of FIG. 10 shows a flowchart of the bullet mucous membrane determination processing in FIG. 10, FIG. 14 shows a flowchart of a modified example of FIG. 9, FIG. 15 shows an example of an image used for explaining the operation of FIG. The flowchart of a determination process is shown.
An endoscope system 1 shown in FIG. 1 includes a medical image processing apparatus (comprising an endoscope observation apparatus 2 and a personal computer that performs image processing on an image obtained by the endoscope observation apparatus 2). Hereinafter, the image processing apparatus is simply abbreviated as 3) and a monitor 4 that displays an image processed by the image processing apparatus 3.

The endoscope observation device 2 is inserted into a body cavity and forms an endoscope 6 that forms an in-vivo imaging device that images the inside of the body, a light source device 7 that supplies illumination light to the endoscope 6, and the endoscope 6. A camera control unit (abbreviated as CCU) 8 that performs signal processing on the image pickup means, and a monitor 9 that displays an endoscopic image picked up by the image pickup device when a video signal output from the CCU 8 is input. Have.
The endoscope 6 includes an insertion portion 11 that is inserted into a body cavity and an operation portion 12 that is provided at the rear end of the insertion portion 11. A light guide 13 that transmits illumination light is inserted into the insertion portion 11.

The rear end of the light guide 13 is connected to the light source device 7. Then, the illumination light supplied from the light source device 7 is transmitted by the light guide 13 and emitted from the distal end surface attached to the illumination window provided at the distal end portion 14 of the insertion portion 11 (transmitted illumination light), and the affected part Illuminate the subject.
An imaging device 17 is provided by an objective lens 15 attached to an observation window adjacent to the illumination window and, for example, a charge coupled device (abbreviated as CCD) 16 serving as a solid-state imaging device disposed at an imaging position of the objective lens 15. It is. The optical image formed on the imaging surface of the CCD 16 is photoelectrically converted by the CCD 16.
The CCD 16 is connected to the CCU 8 via a signal line, and the CCD 16 outputs a photoelectrically converted image signal when a CCD drive signal from the CCU 8 is applied. This image signal is signal-processed by a video processing circuit in the CCU 8 and converted into a video signal. This video signal is output to the monitor 9, and an endoscopic image is displayed on the display surface of the monitor 9. This video signal is also input to the image processing device 3.

In this embodiment, the endoscope 6 is inserted from the mouth portion into the distal end portion 14 of the insertion portion 11 and inserted from the esophagus to the vicinity of the boundary of the stomach, and the normal portion of the esophagus that becomes the detection mucosa near the boundary. Used when performing endoscopy to check for the presence of Barrett mucosa as a mucous membrane (specifically squamous epithelium) that has become denatured and presents the properties of the gastric mucosa. .
In this case, a video signal corresponding to an endoscopic image obtained by imaging the surface of a living body mucosa in the body is also input to the image processing device 3, and the Barret mucosa is processed by the image processing method described below for the video signal. Detection (determination) processing is performed to determine whether or not the disease has reached the disease of Barrett's esophagus.

The image processing device 3 includes an image input unit 21 to which a video signal corresponding to an endoscopic image input from the endoscope observation device 2 is input, and image processing for image data input from the image input unit 21. CPU 22 as a central processing unit for performing the processing, and a processing program storage unit 23 for storing a processing program (control program) that causes the CPU 22 to execute image processing.
In addition, the image processing apparatus 3 includes an image storage unit 24 that stores image data input from the image input unit 21, an analysis information storage unit 25 that stores analysis information processed by the CPU 22, and a process performed by the CPU 22. A hard disk 27 serving as a storage device for storing the image data and analysis information, etc., stored via the storage device interface 26, a display processing unit 28 for performing display processing for displaying the image data and the like processed by the CPU 22, and a user Includes an input operation unit 29 including a keyboard for inputting data such as image processing parameters and performing an instruction operation.

The video signal generated by the display processing unit 28 is output to the display monitor 4, and a processed image subjected to image processing is displayed on the display surface of the display monitor 4. The image input unit 21, CPU 22, processing program storage unit 23, image storage unit 24, analysis information storage unit 25, storage device interface 26, display processing unit 28, and input operation unit 29 are connected to each other via a data bus 30. Has been.
In this embodiment, the region to be examined or diagnosed is the periphery of the junction between the esophagus and the stomach, and by performing image analysis on the image obtained by the endoscope 6, there is a possibility of the Barrett esophagus in the periphery. It is determined whether or not there is a site.

For this reason, the insertion part 11 of the endoscope 6 is inserted into the patient's mouth from the distal end side to perform imaging. FIG. 2 is a diagram showing a body cavity site where the endoscope tip is located when the endoscope 6 is orally inserted into the body cavity of a patient. The distal end portion 14 of the endoscope 6 enters the esophagus 33 from the esophageal entrance 32 by being inserted into the mouth 31, and proceeds to the stomach 36 side through the epithelial boundary 34 and the gastroesophageal junction 35 along the way. Further, it passes through the cardia 37 and reaches the inside of the stomach 36.
By performing an operation of inserting the endoscope 6, moving image data captured in the above order is acquired. The moving image data acquired in this way is stored in the image storage unit 24, and image analysis is performed on the frame image of the still image constituting the moving image data.
FIG. 3 is a schematic view of an endoscopic image captured in the vicinity of the boundary between the esophagus 33 and the stomach 36. In this endoscopic image, the cardia 37 is an entrance into the stomach and opens and closes.
The fence-like blood vessels 38 that run substantially radially outside the cardia 37 are blood vessels that exist only on the esophagus 33 side and extend vertically along the lumen of the esophagus 33.

In addition, the epithelial boundary (illustrated by the alternate long and short dash line) 39 from the mucosal tissue on the esophagus 33 side and the mucosal tissue side on the stomach 36 has a strong reddish mucosal tone from the cardia (the distributed epithelium is called columnar epithelium), The opposite direction is a whitish mucosal color tone (the distributed epithelium is called squamous epithelium), so that the epithelial boundary can be determined by endoscopic observation.
A line connecting the end points of the fence-like blood vessel 38 (shown by a broken line) is a boundary line that is difficult to identify easily by endoscopic observation (there is no actual line), and this is a gastroesophageal junction. 35, which is the tissue boundary between the stomach 36 and the esophagus 33.
In normal cases, the epithelial boundary 39 is near the gastroesophageal junction 35, but the squamous epithelium forming the esophagus 33 is replaced by the mucosa of the stomach 36 (columnar epithelium or Barrett mucosa) due to the influence of reflux esophagitis or the like. Then, this epithelial boundary 39 rises to the esophagus 33 side.

A disease called Barrett's esophagus is diagnosed when this Barrett's mucosa occurs 3 cm or more from the normal mucosal boundary and is omnidirectional with respect to the esophageal lumen cross section.
FIG. 4 shows a functional configuration of the main part of the image processing apparatus 3.
The moving image data captured by the endoscope 6 and input to the image processing device 3 is stored as moving image data Vm1, Vm2,... In an image storage unit 24 as an image storage (image recording) means. Is done.
In this case, the moving image data Vm1, Vm2,... Has a data structure in which still images are accumulated over time. Therefore, each moving image data Vm1, Vm2,... Is, for example, as shown in FIG. 5, still image data Vs0, Vs1,..., VsM with frame numbers 0, 1,..., MAX_COUNT (where M = MAX_COUNT). And stored in the image storage unit 24.

At the same time, the frame time stored in the image storage unit 24 is also stored. The still image data may be stored as image data compressed by JPG or the like.
When image processing is started, still image data in the range of the specified frame number in the moving image data Vm1, for example, read from the image storage unit 24, is an image extraction block configured by software by the CPU 22 and the processing program. 41 is extracted.
The extracted still image data is sequentially sent to the image analysis block 42 and the display processing block 43.
The image analysis block 42 includes an epithelial boundary detection block 44 that detects an epithelial boundary, a gastroesophageal junction detection block 45 that detects a gastroesophageal junction, and a Barrett esophagus determination block 46 that determines whether or not it is a Barrett esophagus. ing.

The epithelial boundary detection block 44 detects an epithelial boundary line existing in the image as a point sequence by detecting, for example, a difference in mucosal color tone in the image as an edge.
The gastroesophageal junction detection block 45 detects, for example, a line connecting the end points of the fence-like blood vessel as a point sequence (details of the detection method will be described later).
The Barrett's esophageal determination block 46 calculates features such as the shape of the epithelial boundary, the striped residual of the squamous epithelium, the distance from the epithelial boundary to the gastroesophageal junction, the standard deviation of the distance, the maximum value / minimum value of the distance, It is determined whether or not the imaged target site is Barrett's esophagus.
Information on the determination result determined by the Barrett's esophageal determination block 46 is stored in the analysis information storage unit 25 and is also sent to the display processing block 43 and displayed on the monitor 4 via the image extraction block 41. The information of the determination result by the analysis by the image analysis block 42 is displayed.

6A shows an example of an analysis result stored in the analysis information storage unit 25, and FIG. 6B is used or set when the analysis processing is performed by the processing program storage unit 23. An example of information is shown.
FIG. 7 shows a display example in which the determination result information is displayed on the monitor 4 in the analyzed still image.
In this embodiment, as will be described with reference to FIG. 8, when determining whether or not there is still image data including Barrett's esophagus from moving image data, a determination target region for performing Barrett's esophageal property determination The reference region (gastroesophageal junction in this embodiment) having the first feature (amount) imaged before or after (may be substantially simultaneous) with respect to the image obtained by imaging Whether or not there is a determination process is performed.

Then, when it is determined that the reference region is imaged by the determination process, the second feature (quantity) (in this embodiment, the epithelial boundary in the frame behind or ahead of the frame) The image processing handcuffs that detect the Barrett's esophagus based on the detection result of the second feature are used to efficiently determine the property of the Barrett's esophagus that is the target of property determination. It is a feature.
By performing such image analysis processing, the processing for detecting the second feature can be omitted for an image that does not have the first feature, and the target property determination target can be omitted. The property determination result can be obtained efficiently in a short time so that a large amount of image data can be handled.
Next, the operation of the image processing apparatus 3 of the present embodiment will be described with reference to the flowchart of FIG.

When the user designates a file name of moving image data from the input operation unit 29 to the CPU 22 that performs processing in accordance with the processing program, the CPU 22 reads the maximum number of frames of the designated moving image data from the image storage unit 24. Then, as shown in FIG. 6B, the value is substituted into a parameter MAX_COUNT that represents the maximum number of frames, and processing according to the processing program is started.
In the first step S1, the CPU 22 initializes the frame number variable COUNT, that is, sets COUNT = 0.
In the next step S2, the CPU 22 compares the frame number variable COUNT with MAX_COUNT. If COUNT> MAX_COUNT, this process is terminated.

If the determination in step S2 is reversed, that is, if COUNT ≦ MAX_COUNT, the process proceeds to step S3. In this step S3, the image extraction block 41 extracts an image with frame number = COUNT.
In the next step S4, the gastroesophageal junction detection block 45 performs gastroesophageal junction detection processing in this embodiment as processing for detecting the first feature from the image of the frame number.
As a result of the detection processing, it is determined whether or not the gastroesophageal junction 35 exists as shown in step S5 depending on whether or not a point sequence of lines indicating the gastroesophageal junction 35 is obtained.
If it is determined in step S5 that the gastroesophageal junction 35 does not exist, the process proceeds to the next step S6, the value of the frame number variable COUNT is incremented by one, and the process returns to step S2, from step S2 to step S2. The process of S6 is repeated.

On the other hand, if it is determined in step S5 that the gastroesophageal junction 35 is present, the second feature of step S7 is detected, and whether or not it is a Barrett's esophagus that is a property determination target based on the detection result. The process proceeds to the determination processing side.
In step S7, in order to start the Barrett's esophagus determination process, a variable N is set, specifically, a variable N = 0 is set.
In the next step S8, a predetermined constant MAX_N, more specifically, the maximum frame number to be subjected to the determination process as to whether or not it is Barrett's esophagus is compared. If the comparison result is N> MAX_N, this process ends. In the present embodiment, whether the frame number is after the maximum frame number set in advance as described above is not determined as to whether it is a Barrett esophagus.

On the other hand, if the comparison result in step S8 is opposite, that is, if N ≦ MAX_N, the process proceeds to step S9, and in this step S9, the image extraction block 41 extracts an image of frame number = COUNT + N. That is, an image in which N frames have elapsed in time is extracted from the image in which the gastroesophageal joint 35 is detected (at this time, since N is in an initially set state of 0, the gastroesophageal joint 35 is initially detected. From this image, Barrett's esophagus determination processing is performed, and as will be understood from the subsequent processing, an image captured later in time is sequentially subjected to determination processing whether or not it is Barrett's esophagus).
In step S10, the gastroesophageal junction detection block 45 performs a process for detecting the gastroesophageal junction 35 from the image of the frame number.

In the next step S11, the epithelial boundary detection block 44 performs a detection process of the epithelial boundary 34 as a second feature detection process from the image of the frame number. In addition, the process which detects the epithelial boundary 34 detects the epithelial boundary 34 by the process from step S1 to S4 of FIG. 4 in Japanese Patent Application No. 2004-360319, for example. As described above, the squamous epithelium on the esophagus side and the columnar epithelium on the stomach side have different color tones, so after performing edge processing and thinning processing on the endoscopic image data, By connecting, the epithelial boundary 34 can be calculated (detected).
In the next step S12, the Barrett's esophageal determination block 46 uses the line point sequence indicating the gastroesophageal junction 35 detected in step S10 and the line point sequence indicating the epithelial boundary 34 detected in step S11. It is determined whether or not the site for property determination in the captured image is the Barrett esophagus.

Specifically, it can be determined whether or not it is a Barrett's esophagus by a process described in the Barret mucosa determination process of FIG. 13 described later.
The bullet esophageal determination block 46 passes the determination result on whether or not it is a bullet esophagus and the frame number to the display processing block 43 in step S13. The display processing block 43 extracts the image data indicated by the designated frame number from a buffer (not shown) therein, and superimposes the determination result on the image data. Then, the image data is sent to the monitor 4 and the image is displayed on the display surface together with the determination result.
For example, when it is determined that it is Barrett's esophagus, as shown in FIG. 6B, for example, “suspected Barrett's esophagus” is displayed in this determination target image.
After the process of step S13, the variable N is increased by 1 in the next step S14, and then the process returns to step S8. Then, the processing from step S8 to step S14 is repeated. In this way, when the variable N exceeds the maximum value MAX_N, this process ends.

According to the present embodiment performing such a configuration and processing, when performing image analysis of whether or not it is Barrett's esophagus for still image data to be analyzed that constitutes moving image data of a captured endoscopic image Then, a process for detecting an image having a feature of the gastroesophageal joint 35 serving as an end point of the fence-like blood vessel existing in the periphery of the site where the Barrett's esophagus is determined is performed in the order of imaging. Then, the bullets are detected based on the detection process of the features of the epithelial boundary 34 and the positional relationship between the detection result and the gastroesophageal joint 35, which are necessary for determining the characteristics of the bullet esophagus for the images after the image having the characteristics. Since it is judged whether it is an esophagus, it can be judged efficiently whether it is a Barrett's esophagus.
In addition, as described below, Barrett's esophagus and Barret's mucosa (Barrett's epithelium) as pre-symptoms leading to the disease of the Barrett's esophagus can also be determined, so that determination suitable for early treatment and the like can be performed.

Also, in this embodiment, a maximum frame number for determining whether or not it is a Barrett's esophagus is set in advance, and it is determined whether or not it is a Barrett's esophagus for an image having a frame number after the maximum frame number. Therefore, it is possible to prevent time from being spent on an image that does not need to be subjected to property determination as to whether or not it is Barrett's esophagus.
That is, as shown in FIG. 2, when the inside of the esophagus 33 is sequentially imaged from the mouth 31 side and the inside of the stomach 36, that is, the inside of the cardia 37, the Barrett esophagus is applied to the image inside the stomach. The image does not need to be subjected to property determination. In such a case, by setting the frame number of the image as MAX_N, it is possible not to perform the property determination as to whether or not it is Barrett's esophagus.

Next, the detection process of the gastroesophageal junction 35 will be described with reference to FIGS. In the following, a description will be given of an image analysis process in which the detection of the gastroesophageal joint 35 is followed by the detection of the epithelial boundary 34 and the Barrett mucosa determination process. The purpose of this image analysis processing is to provide an apparatus and method for appropriately determining whether or not it is a Barrett mucosa. By performing such an image analysis process, it is possible to appropriately determine whether or not it is a Barrett mucosa. it can.
The processing procedure and generated data in this case are shown in FIG. The left side in FIG. 9 shows the processing contents, and the inside of the right side of the frame shows information such as images used or generated.

When the image analysis process starts, edge extraction processing is performed on the processing target image in the first step S21. In this edge extraction process, for example, an edge image is generated by applying a band pass filter to a G color component image in an RGB image. An edge extraction method using a bandpass filter is a known technique. Further, an edge image may be generated from the processing target image using a luminance component. When not only the edge of the blood vessel but also the edge of another shape (contour) is extracted, if the edge of the shape extracted by applying the bandpass filter to the R component of the processing target image is excluded, only the edge of the blood vessel Can be extracted.
In addition, the process part applicable to the gastroesophageal detection process of step S4 of FIG. 8 uses step S21 to step S26 in FIG.

In the next step S22 in FIG. 9, the edge image is binarized to generate a binarized image. In the binarization processing in this embodiment, each pixel of the binarized image is determined to be 0 or 1 by comparing the pixel value of each pixel of the edge image with a specified threshold value.
In the next step S23, a known thinning method is applied to the binarized image to generate a thinned image by thinning processing.
In the next step S24, a fence-like blood vessel extraction process for extracting a fence-like blood vessel peculiar to the esophagus 33 is performed on the thinned image, and the extracted fence-like blood vessel information is stored. A flowchart of this process is shown in FIG. 11 (this description will be described later).

In the next step S25, the end point coordinates of the fence-like blood vessel stored in the fence-like blood vessel extraction process are acquired, the boundary line generation process for connecting the end point coordinate point sequence with the line segment is performed, and the boundary line information is generated (acquired). . FIG. 10A shows boundary line information generated by this processing, more specifically, a fence-like blood vessel end point boundary.
In the next step S26, an image including the boundary line information (fence-like blood vessel end point boundary) acquired by the boundary line image generation process, and the previously acquired dark part and epithelial boundary 34 is generated. This image is shown in FIG.
In the next step S27, Barrett's esophagus determination processing is performed to determine whether or not it is Barrett's esophagus or Barrett's mucosa from the positional relationship between the previously acquired squamous epithelium and columnar epithelial boundary 34. Details of the processing will be described later with reference to FIG.

In this way, the Barrett's esophagus or Barrett mucosa is determined, the determination result is displayed, and the process is terminated.
Next, with reference to FIG. 11, the fence-like blood vessel extraction process in step S24 of FIG. 9 will be described.
When the fence-like blood vessel extraction process starts, an unprocessed line segment is acquired from the thinned image in the first step S31. An image example in that case is shown in FIG.
In the next step S32, the number of pixels of the line segment is calculated as a line segment length L. Then, in the next step S33, the calculated line segment length L is compared with a predetermined threshold value thre1 to determine its magnitude. In this determination process, if L> ther1, the process proceeds to the next step S34, and if L ≦ ther1, it is determined that the line segment is not a fence-like blood vessel, and the process proceeds to step S41. In this embodiment, for example, ther1 = 50.

  In step S34, the number of branches / intersections C and the number of refraction points B are calculated, and in step S35, the magnitude of the predetermined threshold value ε is determined. If C ≦ Cth and B <ε, the process proceeds to the next step S36, and if C> Cth or B ≧ ε, the line segment is not a fence-like blood vessel to be extracted, but a dendritic blood vessel, and the step The process moves to S41. In this embodiment, Cth = 0 and ε = 3.

Of the two end points of the line segment in step S36, an end point closer to the previously acquired image dark part is acquired, and in step S37, a vector v connecting the end point and the dark part center is calculated.
In the next step S38, an angle θ formed by a straight line connecting the vector v and the line segment end point and start point is calculated. Then, in the next step S39, the magnitude of the calculated angle θ and the predetermined threshold value thre2 is determined.

In the determination process in step S39, if θ <thre2 (for example, θ1 in FIG. 12B), the process proceeds to the next step S40, and conversely if θ ≧ thre2 (for example, θ2 in FIG. 12B), It is determined that the line segment is not a fence-like blood vessel, and the process proceeds to step S41. In this embodiment, thre2 = 45 °.
In step S40, the line segment extracted in step S31 is determined as a fence-like blood vessel that satisfies the determination condition in step S39, and information about the line segment as the fence-like blood vessel information (the above-described line segment length L, line) is determined. Minute branch / intersection number C, refraction point number B, line segment coordinate point sequence, end point coordinate, angle θ) are stored. In this way, a fence-like blood vessel can be extracted as shown in FIG.

In step S41, the presence / absence of an unprocessed line segment is determined. If there is an unprocessed line segment, a loop process to step S31 is performed, and if there is no unprocessed line segment, this process ends. In steps S36 to S39, only a blood vessel extending in the dark part direction may be extracted using a matched filter.
Next, referring to FIG. 13, the bullet mucous membrane determination process in step S28 of FIG. 9 will be described. In step S51, the boundary line image generated by the above-described boundary line image generation process is acquired. This is shown in FIG.
In the next step S52, the entire image is divided by a predetermined number, that is, N rays. For example, FIG. 10C shows the case of division with N = 8.
In the next step S53, a variable i indicating the i-th radiation [i] is set to an initial value 1.

In the next step S54, a point P1 where the i-th radiation [i] and the epithelial boundary 34 intersect and a point P2 where the i-th radiation [i] and the formed boundary intersect are calculated. An image obtained by calculating the points P1 and P2 is shown in FIG.
In the next step S55, a distance Q [i] between the points P1 and P2 is calculated.
In the next step S56, it is determined whether or not all the radiation has been processed. That is, it is determined whether or not i is the number of radiations N. If N is not reached, i is incremented by 1 in step S57, and then the process returns to step S54 and the same processing is performed for all radiations. If so, the process proceeds to step S58.
When the distance Q [i] between the points P1 and P2 is calculated for all the radiations, that is, N in this way, the variance σ is calculated using the N distances Q [i] in step S58.

In the next step S59, the magnitude of the variance σ and a predetermined threshold value thre3 is determined. If σ> thre3, the process proceeds to step S60. Conversely, if σ ≦ thre3, it is determined that the image is not a finding of Barrett's mucosa, and the process is terminated. In this embodiment, thre3 = 5.
In step S60, if the image acquired in step S51 satisfies the determination condition of step S59, it is determined that the image is a Barrett mucosal finding image, and the determination result is displayed, notified, saved, etc. The process ends.
By determining whether or not it is a Barrett mucosa (Barrett epithelium) according to the processing shown in FIGS. 9 to 13, it is possible to make a highly accurate determination.

In other words, in such a process, the gastroesophageal junction 35 and the epithelial boundary 34 are detected, and whether or not it is a Barrett mucosa is determined from the detection result. It can be carried out.
By changing a part of the processing of FIG. 13 as follows, the radius (or diameter) of the esophagus 33 in the image is calculated (estimated), and a known statistical value is adopted as the value of the radius. Alternatively, a determination that is quantitatively close to the Barrett mucosa including the Barrett esophagus may be performed. In FIG. 13, for example, a process of calculating a distance (referred to as R [i] for the sake of clarity) between the dark center O-point P1 (or the dark center O-point P2) between steps S55 and S56. Do.

Then, the distance R [i] is calculated together with the distance Q [i] between the points P1 and P2 for all the radiation [i] through the determination process in step S56. Thereafter, instead of calculating the variance σ of the distance Q [i] in step S58 of FIG. 13, the average value Rav of the distance R [i] is calculated, and the average value Rave is calculated in the vicinity of the epithelial boundary 34 in the esophagus 33. The evaluation value (estimated value) of the radius is used.
The statistical radius value Rs (cm) of the esophagus 33 in the case of a normal adult or a person similar in shape to the patient is stored in advance in a memory or the like, and the point P1 using the radius value Rs as the average value Rav. -Evaluate the average value of the distance Q [i] between P2.
And it determines whether the average value of the distance Q [i] is 3.0 cm or more, and when it is 3.0 cm or more, it determines with Barrett's esophagus.

Further, when the average value of the distance Q [i] is about 1.5 cm, for example, it is determined that the Barrett mucosa has progressed considerably. Further, it may be determined that an initial symptom of Barrett mucosa appears as in the case where the average value of the distance Q [i] is about 0.5 cm, for example.
Thus, according to this modification, it is possible to determine whether or not the Barrett's esophagus is in a quantitatively close state, and it is possible to quantitatively determine the degree of progression of the symptom even in the case of Barrett's mucosa. By displaying the results, it is possible to facilitate early treatment.
Further, instead of the bullet mucous membrane determination processing shown in FIG. 9, it may be performed as shown in the flowchart of the modification shown in FIG. 14.

The changes from the flowchart of the above-described process in this modification are shown in FIG. 14 instead of the blood vessel end point extraction process in step S25, the boundary line generation process in step S26, and the boundary line image generation process in step S27 in FIG. In this way, the epithelial boundary fence-like blood vessel image generation process of step S61 is performed. The bullet mucous membrane determination process of step S62 is performed using the epithelial boundary fence-like blood vessel image generated by this process.
In the epithelial boundary fence-like blood vessel image generation processing in step S61, the fence-like blood vessel image acquired in the above-described fence-like blood vessel extraction process and the epithelial boundary fence-like blood vessel image including the dark part and the epithelial boundary line acquired in advance are displayed. It is generated as shown in FIG.
In the next step 62, Barrett mucosa determination processing is performed using the epithelial boundary fence-like blood vessel image generated in the previous step S61. A flowchart of this bullet mucous membrane determination processing is shown in FIG.

As shown in FIG. 16, in the first step S63, an image including the previously acquired epithelial boundary 34 and the fence-like blood vessel is acquired.
In the next step S64, the number of fence-like blood vessels J that intersect with the epithelial boundary line is initialized, that is, J = 0 is set.
In the next step S65, a blood vessel to be processed is acquired from Q fence-like blood vessels. In the next step S66, it is determined whether or not the blood vessel to be processed intersects with the epithelial boundary 34. When intersecting, it progresses to the following step S67, and 1 is added to the number J of fence-like blood vessels, and when not intersecting, it returns to the process of step S65, acquires the next process target blood vessel, and repeats the same process.
After performing step S67, it is determined whether or not all the fence-like blood vessels have been processed in the next step S68. If there is an unprocessed fence-like blood vessel, the process returns to step S65, and the same process is repeated. If it has already been carried out with respect to the fence-like blood vessel, the process proceeds to the next step S69.

FIG. 15B shows an image example in which the number of fence-like blood vessels J intersecting with the epithelial boundary 34 is calculated in this way. In the case of FIG. 15B, the number Q of fence-like blood vessels is 7, and 6 (= J) of them intersect with the epithelial boundary 34.
In step S69, the magnitude of J / Q and a predetermined threshold value thre4 are determined. If J / Q> thre4, the process proceeds to the next step S70. Conversely, if J / Q ≦ thre4, it is determined that the image is not a finding of Barrett's mucosa, and this process ends. In this embodiment, thre4 = 0.5.
In step S70, when the image acquired in step S63 satisfies the determination conditions of step S66 and step S69, it is determined that the image is a Barrett mucosal finding image, and is displayed on the monitor 4, for example, and this process ends. .

In this modification, it can be determined whether or not it is a Barrett mucosa depending on how many end points of the fence-like blood vessel exist inside the epithelial boundary 34.
As described above, according to the present embodiment, when analyzing whether or not the Barrett's esophagus is a large amount of still image data constituting the moving image data of the endoscopic image, the determination of the Barrett's esophagus is performed. A process for detecting an image having characteristics of the gastroesophageal junction 35 serving as the gastric end point of the fence-like blood vessel as the first characteristic part existing in the peripheral part is performed, and an image subsequent to the image detected by the process is displayed. On the other hand, since the detection process of the epithelial boundary 34 as the second characteristic part is performed to determine whether or not it is a Barrett esophagus, it is possible to efficiently determine whether or not it is a Barrett esophagus or the like. . Therefore, it is possible to reduce the labor of extraction work by manual work.

In addition, as described above, the gastroesophageal joint 35 set as the first feature part for which the feature is first detected is also used for property determination of Barrett's esophagus, so that the detection of the feature is effective. Available.
In addition, since the Barrett mucosa (Barrett epithelium) as a pre-symptom leading to Barrett's esophageal disease can also be determined, determination suitable for early treatment or the like can be performed.
It is also possible not to perform the process on an image that does not need to be determined whether or not it is a Barrett esophagus.
Although the present embodiment has been described using moving image data, the present invention may be applied to a plurality of continuous still image data in one examination (can be applied to other embodiments as well).

Next, a second embodiment of the present invention will be described with reference to FIGS. In the first embodiment described above, in order to determine whether or not it is Barrett's esophagus, the gastric esophageal joint 35 is first detected to detect an image including the gastroesophageal joint 35. .
Since the detection processing of the gastroesophageal joint 35 is heavy in the detection processing, the processing speed is slow, and there is room for improvement in processing speed and detection speed. Cardiac detection processing can be considered as detection processing for a living body part that can be processed at a higher speed than the detection of the gastroesophageal joint 35 and is expected to have high detection accuracy.
In this embodiment, attention is paid to this point to improve the processing speed and the like.
As in the first embodiment, the hardware configuration of the image processing apparatus according to the present embodiment can be the same as that in the first embodiment. Moreover, the functional structure comprised by CPU22 by the processing program of a present Example is shown in FIG. The configuration shown in FIG. 17 is a configuration in which a cardia detection block 47 is further provided in the image analysis block 42 in the configuration of FIG.

A flowchart of the process according to this embodiment is shown in FIG. The Barrett's esophagus is determined according to this flowchart, and the determination result is displayed.
The cardia detection block 47 detects, for example, the cardia 37 present in the image data from the detection of the dark part and the shape of the detected dark part and the sharpness of the brightness change in the vicinity of the edge of the dark part. Details thereof will be described later with reference to FIG.
The processing procedure shown in FIG. 18 performs the detection process of the cardia 37 in step S4 ′ instead of the detection processing of the gastroesophageal joint 35 in step S4 in the processing procedure of the flowchart in FIG. Instead of the determination process for the presence of the gastroesophageal joint 35 in step S5, the determination process for the presence of the cardia 37 in step S5 ′ is performed.

In the processing procedure of FIG. 8, instead of extracting the image of frame number = COUNT + N in step S9, the image of frame image = COUNT-N is extracted as shown in step S9 ′ of FIG.
That is, in Example 1, since the gastroesophageal junction 35 is a peripheral part of the part for determining the Barrett's esophagus, an image temporally examined from a state in which the part is detected in the image is examined. As a result, an image closer to the peripheral portion of the part can be obtained, which corresponds to an imaging condition when imaging is performed while the distal end portion 14 of the endoscope 6 is inserted.
On the other hand, as can be seen from FIG. 2, the cardia 37 is a part that passes through the gastroesophageal junction 35 and the epithelial boundary 34 and becomes an entrance into the stomach. A frame image that is traced back by N frames is extracted. Then, it is sequentially determined whether or not it is Barrett's esophagus in a direction that goes back to the image captured earlier from this image.

According to the present embodiment, after detecting the cardia 37 that is lighter in the detection process than the detection of the gastroesophageal joint 35, the Barrett's esophagus determination process is performed based on the image in which the cardia 37 is detected. Therefore, the Barrett's esophagus can be determined in a shorter time.
Therefore, it can transfer to the Barrett's esophagus determination processing side more efficiently, and the effect similar to Example 1 can be acquired in a short time.

Next, the detection process of the cardia 37 will be described with reference to FIGS. FIG. 19 shows together with data using the processing flow for generating the closed cardia when closed or generated data. When the cardia 37 detection process starts, an edge detection process is performed on the processing target image to generate an edge image as shown in step S71.
In this embodiment, the edge extraction process generates an edge image by applying a bandpass filter to the R color component image.
An edge extraction method using a bandpass filter is a known technique. Further, an edge image may be generated from the processing target image using a luminance component.

In the next step S72, the edge image is binarized to generate a binarized image. In the binarization processing in this embodiment, each pixel of the binarized image is determined to be 0 or 1 by comparing the pixel value of each pixel of the edge image with a specified threshold value.
In the next step S73, a known thinning method is applied to the binarized image to generate a thinned image by thinning processing. An example of the generated thinned image is shown in FIG.
In the next step S74, a branch / intersection calculation process for calculating branches / intersections in all thin lines in the thinned image is performed. FIG. 20B shows an example of the branch / intersection calculated for the thinned image of FIG. FIG. 20B shows a case where the number Hc of branches / intersections is 5.

The coordinates of the branch / intersection calculated by the branch / intersection calculation process in step S74 are stored as branch / intersection information.
In the next step S75, concentration degree calculation processing is performed to calculate the concentration level of the branch / intersection from the coordinate value of the branch / intersection, thereby calculating the concentration information.

This concentration degree calculation process will be described with reference to the flowchart of FIG. In the first step S77, the coordinate values of the Nc branch / intersection described above are acquired. In the next step S78, both variances σx and σy are calculated as the variance σx of the x coordinate values and the variance σy of the y coordinate values of the Nc branches / intersections. Then, in the next step S79, the calculated both variances σx and σy are stored as the concentration degree information, and this process is terminated.
Instead of obtaining the variance in the concentration degree calculation process, a standard deviation, a variation coefficient, or an average value of distances from the centers of gravity of Nc branches / intersections may be obtained and used as the degree of concentration information.
Returning to FIG. 19, using the concentration level information calculated by the concentration level calculation process in step S <b> 75, a closing cardia determination process is performed in the next step S <b> 76.

In this closing cardia determination processing, as shown in step S76a of FIG. 22, the number of branches / intersections Nc and the degree of concentration information (σx, σy) are determined by comparing the magnitude relationship between predetermined threshold values thre_N and thre_x, thre_y, respectively. I do.
When it is determined in step S76a that the conditions of Nc> thre_N, σx <thre_x, and σy <thre_y are satisfied, the closed cardia 37 is determined as shown in step S76b. On the other hand, when it is determined that the condition of step S76a is not satisfied, that is, when Nc ≦ thre_N, or σx ≧ thre_x, or σy ≧ thre_y, it is determined that it is not the closed cardia 37 as shown in step S76c.
After performing the cardia determination process in this manner, the cardia detection process shown in FIG. 19 ends. In this way, the cardia 37 can be detected.

As described above, according to the present embodiment, the process for detecting the cardia 37 first is performed, and when the cardia 37 is detected, the process for determining the Barrett esophagus to be determined with respect to the image going back in time. Therefore, the Barrett's esophagus can be determined efficiently even for a large amount of image data.
Further, when the Barrett's esophagus is determined as described in Example 1, the Barrett mucosa can also be determined, and this determination is effective for early treatment.
FIG. 23 shows a flowchart of cardia detection in the modification.
In this modification, in the processing flowchart shown in FIG. 19, the edge of step S <b> 81 for the purpose of detecting an open cardia instead of the branch / intersection calculation processing in step S <b> 74 and the concentration calculation processing in the next step S <b> 75. The component generation angle calculation process is performed, and the opening cardia determination process of detecting (determining) the opening cardia in step S82 is performed based on the generation angle information calculated by the edge component generation angle calculation process.

The processing shown in FIG. 23 is the same from step S71 to step S73 in FIG.
As shown in step S71, edge extraction processing is performed on the processing target image to generate an edge image, and binarization processing in step S72 is performed on the edge image to generate a binarized image. The thinned image shown in FIG. 24A is generated by the thinning process in S73.
Next, FIG. 24 (FIG. 24) in which an image is superimposed on a thinned image obtained by the above thinning process by performing a dark part binarization process using a dark part extraction threshold to extract an image dark part in advance for an image including a dark part. The edge component generation angle calculation process of step S81 is performed on the image shown in B), and generation angle information of a large edge angle is calculated.

And it is determined whether it is an opening cardia by the opening cardia determination process of step S82.
FIG. 25 is a flowchart showing details of the edge component generation angle calculation processing in step S81 of FIG.
In the first step S83, one feature point of the dark part in the image is selected. In this embodiment, for example, the barycentric point of the dark part is calculated.
In the next step S84, the image is divided into a plurality of, for example, M areas in the circumferential direction by radial lines around the calculated feature point such as the center of gravity or the center point.
In the next step S85, one line segment i is extracted (acquired) from the thin lines in the thinned image shown in FIG.

In the next step S86, an angle θ [i] at which the extracted line segment i exists is calculated. That is, the number Ni of regions where the line segment i exists is counted, and the angle θ [i] where the line segment i exists is calculated from the count by θ [i] = Ni × (360 / M) °.
An example calculated in this way is shown in FIG. FIG. 24C shows a case where the number of areas Ni from the dividing line area where the line segment i is 0 ° to the dividing line area where the line segment i is 270 ° is six.
In the next step S87, the presence / absence of an unprocessed line segment is determined. If there is an unprocessed line segment, the process returns to step S85 to acquire the unprocessed line segment and perform the same process.
On the other hand, if there is no unprocessed line segment, the edge component generation angle calculation process ends. Returning to FIG. 23, it is determined whether or not the opening is a cardia opening by the opening cardia determination process of the next step S82 using the angle information of the angle θ [i] generated by the edge component generation angle calculation process of step S81.

In the opening cardia determination process, for example, as shown in step S82a of FIG. 26, the magnitude θ [i] is determined based on the predetermined threshold value thre5. That is, it is determined whether or not the condition θ [i]> thre5 is satisfied. If the condition θ [i]> thre5 is satisfied, it is determined that the edge of the opening is as shown in step S82b. Conversely, if the condition is not satisfied, it is determined that the edge is not the edge of the opening. To do. In this way, the process for detecting the cardia ends.
In this way, the cardia opening can be detected.
And by performing the process which detects a Barrett's esophagus from the image which detected the cardia, a Barrett's esophagus etc. can be determined efficiently.

Next, Embodiment 3 of the present invention will be described with reference to FIGS. The hardware configuration of the image processing apparatus of the present embodiment is the same as that of the first embodiment, and FIG. 1 can be employed. Further, FIG. 27 shows a functional configuration of a main part constituted by the CPU 22 that executes the processing program in the present embodiment. The configuration shown in FIG. 27 is a configuration in which a processing continuation determination block 48 is further provided in the image analysis block 42 in the configuration of FIG.
The processing continuation determination block 48 determines whether or not there is a point sequence of the epithelial boundary detected by the epithelial boundary detection block 44, and controls the operation of the image analysis block 42 according to the determination result.
A flowchart of a processing procedure according to the present embodiment is shown in FIG. The Barrett's esophagus is determined according to this flowchart, and the determination result is displayed.

The processing method according to the flowchart shown in FIG. 28 detects the epithelial boundary 34 that can be detected by a simpler process instead of the process of detecting the gastroesophageal junction 35 in the processing procedure of FIG. 8 in the first embodiment. Process.
First, an image in which the epithelial boundary 34 is detected is retrieved, and if an image in which the epithelial boundary 34 is detected is retrieved, the process proceeds to the processing side for determining Barrett's esophagus.
The processing procedure will be described below with reference to the flowchart of FIG. Since the first steps S1 to S3 are the same as those in the flowchart of FIG. 8, the description thereof is omitted.

An image whose frame number is COUNT in step S3 is extracted, and the detection process of the epithelial boundary 34 in the next step S91 is performed.
Then, for the detected epithelial boundary 34, the processing continuation determination block 48 in the next step S92 determines whether or not a sequence of lines indicating the epithelial boundary 34 is obtained in step S91, thereby determining the epithelial boundary 34. The presence / absence of the presence or absence is determined.
If it is determined in step S92 that the epithelial boundary 34 does not exist, the process proceeds to step S6, the value of the frame number variable COUNT is incremented by one, and the process returns to step S2, and the processes from step S2 to step S92 are performed. That is, the analysis operation for detecting the epithelial boundary 34 is continued.

  On the other hand, if it is determined in step S92 that the epithelial boundary 34 is present, the analysis operation for detecting the epithelial boundary 34 is shifted to step S93, and the analysis operation for determining Barrett's esophagus is performed.

In step S93, the gastroesophageal junction detection block 45 performs a process for detecting the gastroesophageal junction 35 from the image of the frame number.
After performing the detection process of the gastroesophageal joint 35, in the next step S94, the Barrett's esophageal determination block 46 includes a point sequence of lines indicating the gastroesophageal joint 35 detected in step S93 and the epithelium detected in step S91. It is determined whether or not the target site in the captured image is the Barrett esophagus using the point sequence of the line indicating the boundary 34.
If it is determined that the esophagus is a Barrett esophagus, the Barrett esophagus determination block 46 passes the determination result and the frame number to the display processing block 43 in step S95. This display processing block 43 extracts the image of the designated frame number from the buffer, and superimposes the determination result on the image data. For example, the display is as shown in FIG.

In the next step S96, COUNT is incremented by one. In the next step S97, an image of the next frame number (= COUNT) is newly acquired. And the detection process of the epithelial boundary 34 is performed with respect to the image.
In the next step S98, it is determined whether or not the epithelial boundary 34 is present by the previous detection process.

In step S98, the processing continuation determination block 48 determines whether or not the epithelial boundary 34 is present by determining whether or not a point sequence of lines indicating the epithelial boundary 34 is obtained in the previous step S97.
If it is determined in step S98 that the epithelial boundary 34 does not exist, the processing loop from step S93 to step S98, that is, the analysis processing operation on the Barrett's esophageal determination processing side is exited, and this processing ends.

On the other hand, if it is determined that the epithelial boundary 34 exists, the process returns to step S93, and the process for detecting the gastroesophageal junction 35 is performed to continue the Barrett's esophageal determination process. In this way, when the presence of the epithelial boundary 34 is detected, the processing in the above processing loop is repeated, and when the presence of the epithelial boundary 34 is no longer detected, this processing is terminated.
According to this embodiment that operates in this manner, the epithelial boundary 34 is first detected, and the image from which the epithelial boundary 34 is detected by this detection processing moves to a processing side that determines the presence or absence of Barrett's esophagus. If the presence of the epithelial boundary 34 is no longer detected again on the processing side, this processing is terminated, so that the Barrett's esophagus can be determined efficiently.

That is, according to the present embodiment, when the processing continuation determining means is provided to detect only the peripheral portion of the epithelial boundary 34 necessary for determining the property of Barrett's esophagus, the epithelial boundary 34 is not detected in the image. Since the control for ending the Barrett's esophagus determination process is performed, the image required for the Barrett's esophagus determination process is extracted and the Barrett's esophagus determination process is performed without requiring time and effort. Can do.
That is, the Barrett's esophagus determination process can be performed in a shorter time and with less effort than in the first embodiment.

In this embodiment, after detecting the epithelial boundary 34 from the frame image, the temporally rearward frame image in which the frame images are continuous is used as a target image for the next epithelial boundary detection. Accordingly, the frame image may be acquired retroactively in time to be a detection target image.
In addition, the acquisition interval of consecutive frame images is N (where N is a natural number of 1, 2, 3,...), And is incremented to COUNT ← COUNT + N in step S97 to designate the next acquisition frame image. May be.
Next, a modification of this embodiment will be described. When it is determined whether each image is a Barrett's esophagus, the image of the Barrett's esophagus is originally captured due to the effects of noise, halation, light quantity change over time, shading, etc. included in the image data. There is a possibility that it is erroneously determined that it is not Barrett's esophagus.

Therefore, this modification is improved by the processing procedure shown in FIG. The processing procedure according to the flowchart shown in FIG. 29 is the same processing from step S1 to step S94 in the processing procedure of FIG. 28 (however, since the variable Nb of the frame number is used in FIG. 29, this variable Nb is changed in step S1. Initialized to 0).
In FIG. 28, the determination result is displayed in the next step S95 from the determination result in step S94. However, in this modified example, after the determination process in step S94, the detection process of the epithelial boundary 34 is performed instead of the next frame number. In addition, the Barrett's esophagus determination process is performed by performing a detection process of the gastroesophageal joint 35 or the like.
In this way, when the image has no epithelial boundary 34, it is comprehensively determined whether or not it is Barrett's esophagus from all the determination results that have been determined whether or not it is Barrett's esophagus, and the determination result Is displayed.

Hereinafter, the processing procedure will be described with reference to FIG. Steps S1 to S94 are the same as those in FIG. 28 (however, as described above, in step S1, the variable Nb of another frame number is also initialized to 0), and thus the description thereof is omitted.
In step S94, it is determined whether or not it is Barrett's esophagus based on the position where the epithelial boundary 34 exists in step S92 and the detection process of the gastroesophageal junction 35 in step S93. Then, the determination result of the Barrett esophagus is temporarily accumulated, and the variable of the frame number Nb is incremented by one in the next step S101. In the next step S102, an image having a frame number COUNT + Nb in which Nb is increased by one is extracted.

In the next step S103, the epithelial boundary 34 is detected. After this detection process, in the next step S104, it is determined whether or not the epithelial boundary 34 exists. If it exists, the process returns to step S93, the detection process of the gastroesophageal junction 35 is performed, and the bullet is similarly processed. Processing such as determination of whether or not it is an esophagus is performed.
On the other hand, if it is determined that the epithelial boundary 34 does not exist, the process proceeds to step S105, and all the determination results of Barrett's esophagus acquired by the processing from step S93 to step S104 are acquired.
Then, in the next step S106, it is comprehensively determined whether or not it is a Barrett esophagus from the determination results of all these Barrett esophagus. Then, in the next step S107, the comprehensive determination result is displayed superimposed on the image, and this process ends.

The Barrett esophagus determination block 46 determines whether or not the Barrett's esophagus is comprehensive in Step S106 as follows.
For example, the ratio Na / Nb between the number Na of images determined to be Barrett's esophagus and the number Nb of images determined to be Barrett's esophagus is calculated. If the ratio Na / Nb is greater than 0.8, the Barrett esophageal determination block 46 determines that the imaging target is Barrett's esophagus, and in Step S107, all Nb images used for Barrett's esophageal determination. Is superimposed with the information “suspected of Barrett's esophagus”.
According to this modified example in which the processing operation is performed in this way, the same effect as that of the third embodiment is obtained, and further, the information on the result of determining whether or not it is Barrett's esophagus in each of the plurality of images is more comprehensively used. Since it determines, it can be determined in a more reliable state whether it is Barrett's esophagus.

In the above description, the endoscope 6 having an elongated insertion portion has been described in the case of performing image processing on the case of an endoscope image captured by being inserted into the body. The present invention can be similarly applied to the case of an endoscopic image captured by a capsule endoscope to be imaged. In the case of a capsule endoscope, it is usually an in-vivo imaging device that continuously captures still images at regular intervals. Further, in this case, imaging is performed while moving to the esophagus 33, stomach, small intestine, and large intestine without returning after swallowing from the mouth. Even in such a case, the present invention is applicable.
In addition, the present invention includes a case of an embodiment configured by partially combining the above-described embodiments.

[Appendix]
1. In Claim 1, the said property determination means superimposes and outputs the said determination result on the image which performed the said determination result.
2. The property determination unit according to claim 1, wherein the property determination unit outputs the determination result based on the detection result of the second biological feature detection unit, and when a plurality of the determination results are obtained, based on the plurality of determination results. Output comprehensive judgment results.
3. In Claim 1, the said property determination means is a Barrett's esophagus determination means which performs the property determination of Barrett's esophagus as a disease in an esophagus.
4). In Claim 1, the said property determination means is a bullet determination means which determines the property determination of the Barrett esophagus as a disease in the esophagus and the presence of the Barrett mucosa before the disease of the Barrett esophagus.

5. In Claim 1, the said property determination means is a Barrett mucosa determination means for determining the presence of Barrett mucosa in which the mucosa on the esophagus side of the gastroesophageal junction in the esophagus has been degenerated into columnar epithelium as the gastric mucosa.

6). In Claim 1, the first biological feature detection means includes a gastroesophageal junction as a boundary between the stomach and the esophagus in the esophagus, or an epithelial boundary serving as a boundary between the squamous epithelium on the esophagus and the columnar epithelium on the stomach, Alternatively, either the cardia that becomes the entrance of the stomach from the esophagus is detected.
7). In Supplementary Note 3 or 4, the first or second biological feature detection means detects a site or mucosal boundary used for property determination of the Barrett esophagus.
8). In Appendix 7, the site or mucosal boundary is a gastroesophageal junction or epithelial boundary.

9. 2. The gastroesophageal junction as a boundary between the stomach and the esophagus in the esophagus, or an epithelial boundary serving as a boundary between the squamous epithelium on the esophagus and the columnar epithelium on the stomach side. To detect.

10. In Supplementary Note 3, the Barrett's esophageal determination means determines the property of Barrett's esophagus based on the detection result of the gastroesophageal junction and the epithelial boundary.
11. In Supplementary Note 4, the bullet determining means determines the property of the bullet esophagus or the bullet mucosa based on the detection result of the gastroesophageal junction and the epithelial boundary.
12 Additional remark 10 or 11 has a fence-like blood vessel detecting means for detecting a fence-like blood vessel in order to detect the gastroesophageal junction.
13. In Supplementary Note 6, the first biological feature detecting means detects a cardia closing as the cardia or a cardia opening.
14 The moving image data or still image data is generated by an endoscope having an insertion portion or a capsule endoscope swallowed from a mouth.

15. An image extraction processing step for extracting frame image data from in-vivo moving image data captured by the in-vivo imaging device or a plurality of still image data continuously captured;
Image analysis processing step for performing image analysis of the frame image data extracted in the image extraction processing step and outputting an image analysis result;
With
The image analysis processing step includes
A first biometric feature detection processing step for performing a process of detecting a first biometric feature;
Based on the detection result of the first biometric feature detection processing step, the second is applied to the frame image captured forward or backward in time from the image used for detection in the first biometric feature detection processing step. A second biometric feature detection processing step for performing a process for detecting the biometric feature, and a process for performing a process for determining the properties of the living body based on the detection result of the second biometric feature detection processing step and outputting the determination result A determination processing step;
A medical image processing method comprising:

16. In Additional Statement 15, in the second biological feature detection processing step, a predetermined number of frame images are acquired based on the detection result of the first biological feature detection processing step, and the second biological feature detection step is performed on the frame image. Performs feature detection processing.
17. In Supplementary Note 15, the second biological feature detection processing step is based on the detection result of the first biological feature detection processing step.
Sequentially acquiring frame images captured forward or backward in time from the frame image processed by the first biological feature detection processing step;
The acquisition of the frame image is interrupted based on the detection result when the first biological feature detection processing step is applied to the acquired frame image.

18. In Supplementary Note 15, the property determination processing step superimposes and outputs the determination result on the image on which the determination result has been performed.
19. In Supplementary Note 15, the property determination processing step outputs the determination result based on the detection result of the second biological feature detection processing step and, when a plurality of the determination results are obtained, based on the plurality of determination results. Output comprehensive judgment results.
20. In Supplementary Note 15, the property determination processing step is a Barrett esophagus determination processing step for determining the property of Barrett's esophagus as a disease in the esophagus.
21. In Supplementary Note 15, the property determination processing step is a Barret determination processing step that determines the property of Barrett's esophagus as a disease in the esophagus and the presence of Barrett mucous membrane before the disease of Barrett's esophagus.

22. In Supplementary Note 15, the property determination processing step is a Barret mucosa determination processing step for determining the presence of Barret mucosa in which the mucosa on the esophagus side of the gastroesophageal junction in the esophagus has degenerated into a columnar epithelium as the gastric mucosa.
23. In Supplementary Note 15, the first biological feature detection processing step includes a gastroesophageal junction as a boundary between the stomach and the esophagus in the esophagus, or an epithelial boundary serving as a boundary between the squamous epithelium on the esophagus and the columnar epithelium on the stomach, Alternatively, either the cardia that becomes the entrance of the stomach from the esophagus is detected.

24. A line structure extraction means for extracting a line structure from a medical image obtained by imaging the mucosal surface in the esophagus;
Terminal calculation means for calculating each terminal on the stomach side from the shape feature amount of the line structure;
Fence-like blood vessel determining means for determining whether or not the line structure extracted from the shape feature amount is a fence-like blood vessel;
A fence-like blood vessel detection device comprising:
25. A first step of inputting a medical image obtained by imaging a biological mucosal surface;
A second step of extracting a line structure from the medical image;
A third step of calculating a shape feature amount of the line structure;
A fourth step of determining whether the mucosal structure is a fence-like blood vessel from the shape feature amount;
A method for detecting a fence-like blood vessel, comprising:
26. Appendix 24 is characterized in that the shape feature value is a branch / intersection or a generation angle.

(Background of appendices 24-26)
Conventionally, there has been no effective apparatus or method for determining the Barrett esophagus or Barrett mucosa.
For this reason, in order to provide an effective apparatus or method for determining the Barrett's esophagus or Barrett's mucosa, the configurations of Supplementary notes 24 to 26 were adopted.
(Effects of Supplementary Notes 24-26)
Since the end of the gastric esophagus at the gastroesophageal junction at the boundary between the stomach and the esophagus is detected, it is possible to appropriately determine the Barrett esophagus or Barrett mucosa from the detection result of the epithelial boundary. it can.

27. An epithelial boundary and gastroesophageal junction detection means for detecting an epithelial boundary and a gastroesophageal junction from a medical image obtained by imaging the mucosal surface in the esophagus;
Barret determination means for determining the presence of Barrett mucosa or Barrett's esophagus from the detected epithelial boundary and gastroesophageal junction,
A bullet detection device comprising:
28. An epithelial boundary & gastroesophageal junction detection step for performing a process of detecting the epithelial boundary and the gastroesophageal junction from a medical image obtained by imaging the mucosal surface in the esophagus;
A bullet determination processing step for determining the presence of a bullet mucosa or a bullet esophagus from the detected epithelial boundary and gastroesophageal junction;
A bullet detection method comprising:

(Background to Appendix 27 and 28)
Conventionally, there has been no effective apparatus or method for determining the Barrett esophagus or Barrett mucosa.
For this reason, in order to provide an effective apparatus or method for determining the Barrett's esophagus or Barrett's mucosa, the configurations of Supplementary Notes 27 and 28 were adopted.
(Effects of Supplementary Notes 27 and 28)
It is possible to detect the gastroesophageal junction at the boundary between the stomach and the esophagus and the epithelial boundary, and appropriately determine the Barrett esophagus or Barrett mucosa from these detection results.

  In order to determine the properties in the case of an esophageal bullet etc. from a large amount of endoscopic image data obtained by imaging the inside of the esophagus etc., a process for detecting the first feature such as the gastroesophageal junction and epithelial boundary around the region Detecting by detecting the second feature such as an esophageal bullet to be judged for the subsequent images from the image judged to have the feature by sequentially changing the frame number until the presence is judged. Transition to processing. By performing such processing, it is possible to perform more efficient processing than when performing the second feature detection processing and determination processing from the beginning.

FIG. 1 is a block diagram showing an overall configuration of an endoscope system including a first embodiment of the present invention. FIG. 2 is a diagram schematically showing each part of the upper digestive tract that is endoscopically examined by oral insertion of an endoscope. FIG. 3 is a diagram showing an example of an endoscopic image obtained by imaging the vicinity of the boundary between the esophagus and the stomach. FIG. 4 is a diagram illustrating a functional configuration of a main part of the image processing apparatus according to the present exemplary embodiment. FIG. 5 is a diagram showing that moving image data stored in an image storage unit is stored as a collection of still image data. FIG. 6 is a diagram illustrating analysis results stored in the analysis information storage unit, information stored in the processing program storage unit, and the like. FIG. 7 is a diagram illustrating a monitor display example in which an analysis result is displayed together with an endoscopic image. FIG. 8 is a flowchart of a processing procedure for determining the properties of Barrett's esophagus according to this embodiment. FIG. 9 is a flowchart showing a processing procedure for performing detection processing of a gastroesophageal junction together with information such as images used or generated. FIG. 10 is a diagram showing a fence-like blood vessel end point boundary and the like. 11 is a flowchart showing details of the fence-like blood vessel extraction process in FIG. FIG. 12 is a diagram illustrating an example of an image for explaining an operation when the processing of FIG. 11 is performed. FIG. 13 is a flowchart showing details of the bullet mucous membrane determination process in FIG. FIG. 14 is a flowchart of a modification of FIG. FIG. 15 is a diagram showing an example of an image used for explaining the operation of FIG. 16 is a flowchart showing details of the bullet mucous membrane determination process in FIG. FIG. 17 is a diagram illustrating a functional configuration of a main part of the image processing apparatus according to the second embodiment of the present invention. FIG. 18 is a flowchart of a processing procedure for determining the properties of Barrett's esophagus according to this embodiment. FIG. 19 is a flowchart showing a processing procedure for performing cardia detection processing together with information such as images used or generated. FIG. 20 is an explanatory diagram of the operation of FIG. FIG. 21 is a flowchart showing details of the concentration degree calculation processing of FIG. 22 is a flowchart showing the closing cardia determination process of FIG. FIG. 23 is a flowchart showing a processing procedure for performing cardia detection processing in a modified example together with information such as an image to be used or generated. FIG. 24 is a diagram for explaining the operation of FIGS. The flowchart figure which shows the detail of the edge component production | generation angle calculation process in FIG. FIG. 26 is a flowchart showing the opening cardia determination processing of FIG. FIG. 27 is a diagram illustrating a functional configuration of a main part of the image processing apparatus according to the second embodiment of the present invention. FIG. 28 is a flowchart of a processing procedure for determining the properties of Barrett's esophagus according to this embodiment. FIG. 29 is a flowchart of a processing procedure for determining the properties of Barrett's esophagus according to a modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Endoscope system 2 ... Endoscope observation apparatus 3 ... Image processing apparatus 4 ... Monitor 6 ... Endoscope 7 ... Light source device 8 ... CCU
11 ... Insertion part 14 ... Tip part 16 ... CCD
17 ... Imaging device 21 ... Image input unit 22 ... CPU
22a ... Geometric image conversion means 22b ... Development view output means 22c ... Epithelial boundary detection means 22d ... Epithelial boundary analysis means 23 ... Processing program storage section 24 ... Image storage section 25 ... Analysis information storage section 27 ... Hard disk 28 ... Display processing Unit 29 ... Input operation unit 33 ... esophagus 34 ... epithelial boundary 35 ... gastroesophageal junction 36 ... stomach 37 ... cardia 38 ... fenced blood vessel 39 ... epithelial boundary 41 ... image extraction block 42 ... image analysis block 43 ... display processing block 44 ... Epithelial boundary detection block 45 ... Gastroesophageal junction detection block 46 ... Barrett's esophageal decision block 47 ... Cardia detection block Agent Patent attorney Susumu Ito

Claims (3)

Image extracting means for extracting frame image data from in-vivo moving image data captured by the in-vivo imaging device or a plurality of still image data continuously captured;
Image analysis means for analyzing the frame image data extracted by the image extraction means, and outputting an image analysis result;
With
The image analysis means includes
First biological feature detection means for detecting a first biological feature;
Based on the detection result of the first biometric feature detection means, a second biometric feature is obtained with respect to a frame image captured forward or backward in time from the image used for detection by the first biometric feature detection means. Second biological feature detection means for detecting features;
Based on the detection result of the second biological feature detection means, a property determination means for determining the properties of the living body and outputting the determination result;
A medical image processing apparatus comprising:   The second biometric feature detection unit acquires a predetermined number of frame images based on the detection result of the first biometric feature detection unit, and performs a second biometric feature detection process on the frame image. The medical image processing apparatus according to claim 1. The second biological feature detection means is based on the detection result of the first biological feature detection means,
Sequentially obtaining frame images captured forward or backward in time with respect to the frame image processed by the first biological feature detection means;
2. The medical image processing apparatus according to claim 1, wherein the acquisition of the frame image is interrupted based on the detection result when the processing of the first biological feature detection means is applied to the acquired frame image. .

Images