JP2006166939A - Image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing method capable of detecting the presence of a specified biological mucous membrane without being affected by an inappropriate part such as a dark part. <P>SOLUTION: The tone feature amount of each pixel is calculated by using an Hb index for an eliminated unrequired area image 32 which is an eliminated unrequired area such as the dark part from a source image 31. The tone feature amount is compared with a threshold, a type labeling processing for indicating whether or not the possibility of a specified biological mucous membrane is high is performed, and then whether or not the source image 31 includes Barrett's epithelium as the specified biological mucous membrane is judged. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing method for determining the properties of a biological mucosa in the vicinity of the stomach boundary from the esophagus.

In recent years, endoscope apparatuses that perform endoscopy and the like using an endoscope have been widely adopted in the medical field and the industrial field. In the medical field, for example, by inserting an insertion portion of an endoscope into a body cavity and observing the examination target site, it is used to diagnose whether the examination target site is a normal mucosa or a lesioned mucosa.
In such a case, if it is possible to determine whether the mucosa is a normal mucosa or a lesioned mucosa by performing image processing from an endoscopic image, the surgeon should focus on the determination result site. Thus, an efficient diagnosis can be performed.

For example, Japanese Patent Laid-Open No. 10-014864 as a first conventional example discloses an image processing apparatus that calculates a feature amount from a region of interest set by a region-of-interest setting unit and discriminates and classifies a lesion.
Further, in Japanese Patent Laid-Open No. 2000-155840 as a second conventional example, preprocessing such as inverse γ correction and shading correction is performed so that the feature amount can be calculated while further reducing the difference in imaging conditions. An image processing apparatus is disclosed.
JP 10-014864 A JP 2000-155840 A

According to the second conventional example, the difference in imaging conditions in the first conventional example can be reduced.
However, even in the second conventional example, if there is a dark part, an increase in the amount of illumination light can partially make an image captured in a brighter state, but the dark part can be completely eliminated by increasing the amount of illumination light. Is almost impossible.
More specifically, from the esophagus to the stomach boundary, the mucosal epithelium (squamous epithelium) as a living mucosa in the esophagus was devastated and replaced with the same properties as the gastric mucosa (degenerated) Screening Barrett's epithelium (Barrett's mucosa), which becomes a diseased mucosa, is useful information for diagnosis.

In this case, the Barrett epithelium, which has been degenerated in the same manner as the gastric mucosa due to the squamous epithelium devastation, has the property of easily becoming cancerous tissue, and is very useful if it can be detected early.
In this way, in endoscopy from the esophagus to the vicinity of the boundary of the stomach, in the part from the luminal esophagus to the deep stomach, for example, a dark part often exists in the image. There is a possibility that more appropriate determination or detection cannot be performed due to such a dark portion or the like. In other words, preprocessing such as inverse γ correction and shading correction can be improved by, for example, brightening part of the dark part, but it can be said that it is impossible to eliminate the dark part. For this reason, the accuracy of the image processing result cannot be improved due to the dark area or the like.

(Object of invention)
The present invention has been made in view of the above-described points, and provides an image processing method capable of more appropriately detecting the presence of a specific biological mucous membrane to be detected without being affected by an inappropriate part such as a dark part. The purpose is to do.
In addition, the present invention more appropriately detects the presence of Barrett epithelium (Barrett mucosa) as a specific biological mucous membrane to be detected without being affected by an inappropriate part such as a dark part from the esophagus to the stomach boundary. An object of the present invention is to provide an image processing method capable of

The image processing method of the present invention includes a first step of inputting an image signal corresponding to an image of two or more colors obtained by imaging a biological mucous membrane,
A second step of setting a processing target region to exclude pixels inappropriate for the detection target in the image;
A third step of calculating a tone characteristic amount for each of the pixels in at least the processing target region;
A fourth step of detecting the presence of a specific biological mucous membrane based on the calculated color tone feature amount;
It is characterized by comprising.
With the above step configuration, by excluding pixels inappropriate for the detection target such as a dark part, the presence of a specific biological mucous membrane to be detected can be detected with higher accuracy.

  According to the present invention, it is possible to more accurately detect the presence of a specific biological mucous membrane as a detection target by excluding pixels inappropriate for the detection target such as a dark part.

  Embodiments of the present invention will be described below with reference to the drawings.

1 to 10 relate to a first embodiment of the present invention, FIG. 1 shows a configuration of an endoscope system having a function of an image processing method of the first embodiment of the present invention, and FIG. 2 shows an image of the first embodiment. 3 shows processing functions for realizing the processing method, images to be used, etc., FIG. 3 shows processing contents of unnecessary area deletion processing for deleting unnecessary areas, and FIG. 4 shows image examples such as original images and unnecessary area deleted images. FIG. 5 shows the processing contents of region labeling.
FIG. 6 shows the processing contents of the mucous membrane feature quantity calculation, FIG. 7 shows the configuration of the endoscope system provided with the first modification, and FIG. 8 is for realizing the image processing method in the second modification. FIG. 9 shows an example of an image divided into rectangular small areas, and FIG. 10 shows the processing contents of mucous membrane feature amount calculation in the second modified example.

An endoscope system 1 shown in FIG. 1 includes an endoscope observation apparatus 2 and an image processing apparatus 3 including a personal computer that performs image processing on an image obtained by the endoscope observation apparatus 2. And a display monitor 4 for displaying an image processed by the image processing device 3.
The endoscope observation apparatus 2 includes an endoscope 6 that is inserted into a body cavity, a light source device 7 that supplies illumination light to the endoscope 6, and a camera control that performs signal processing on an imaging unit of the endoscope 6. A unit (abbreviated as CCU) 8 and a monitor 9 for displaying an endoscopic image picked up by an image pickup device when a video signal output from the CCU 8 is input.
The endoscope 6 includes an insertion portion 11 that is inserted into a body cavity and an operation portion 12 that is provided at the rear end of the insertion portion 11. A light guide 13 that transmits illumination light is inserted into the insertion portion 11.

The rear end of the light guide 13 is connected to the light source device 7. Then, the illumination light supplied from the light source device 7 is transmitted by the light guide 13 and emitted from the distal end surface attached to the illumination window provided at the distal end portion 14 of the insertion portion 11 (transmitted illumination light), and the affected part Illuminate the subject.
An imaging device 17 is provided by an objective lens 15 attached to an observation window adjacent to the illumination window and, for example, a charge coupled device (abbreviated as CCD) 16 serving as a solid-state imaging device disposed at an imaging position of the objective lens 15. It is. The optical image formed on the imaging surface of the CCD 16 is photoelectrically converted by the CCD 16.
The CCD 16 is connected to the CCU 8 via a signal line, and the CCD 16 outputs a photoelectrically converted image signal when a CCD drive signal from the CCU 8 is applied. This image signal is signal-processed by a video processing circuit in the CCU 8 and converted into a video signal. This video signal is output to the monitor 9, and an endoscopic image is displayed on the display surface of the monitor 9. This video signal (image signal) is also input to the image processing device 3.

In this embodiment, the endoscope 6 is inserted into the distal end portion 14 of the insertion portion 11 from the mouth, inserted from the esophagus to the vicinity of the stomach boundary, and a specific biological mucosa serving as a detection target mucosa in the vicinity of the boundary. It is used when performing endoscopy to determine whether or not the Barrett epithelium is present.
In this case, the video signal corresponding to the endoscopic image is also input to the image processing apparatus 3, and detection of whether or not the Barrett epithelium exists is performed on the video signal by an image processing method as described below. (Decision) Processing is performed.

The image processing device 3 includes an image input unit 21 to which a video signal corresponding to an endoscopic image input from the endoscope observation device 2 is input, and image processing for image data input from the image input unit 21. CPU 22 as a central processing unit for performing the processing, and a processing program storage unit 23 for storing a processing program (control program) that causes the CPU 22 to execute image processing.
In addition, the image processing apparatus 3 includes an image storage unit 24 that stores image data input from the image input unit 21, an analysis information storage unit 25 that stores analysis information processed by the CPU 22, and a process performed by the CPU 22. A hard disk 27 serving as a storage device for storing the image data and analysis information, etc., stored via the storage device interface 26, a display processing unit 28 for performing display processing for displaying the image data and the like processed by the CPU 22, and a user Includes an input operation unit 29 including a keyboard for inputting data such as image processing parameters and performing an instruction operation.

The video signal generated by the display processing unit 28 is output to the display monitor 4, and a processed image subjected to image processing is displayed on the display surface of the display monitor 4. The image input unit 21, CPU 22, processing program storage unit 23, image storage unit 24, analysis information storage unit 25, storage device interface 26, display processing unit 28, and input operation unit 29 are connected to each other via a data bus 30. Has been.
FIG. 2 is a block diagram showing an image processing function for performing image processing for detecting the presence of a bullet epithelium as a detection target mucosal tissue by the CPU 22 in accordance with a processing program stored in the processing program storage unit 23.
As shown in FIG. 1, the CPU 22 deletes an unnecessary area from the original image 31 and generates an unnecessary area deleted image 32 for the original image 31 to be subjected to image processing, and the unnecessary area. It has a function with a color tone processing unit 35 that performs color tone processing on the region deleted image 32 and generates a color tone processed image 34.

In addition, the CPU 22 performs a region labeling process on the color tone processed image 34 to generate a region labeling image 36, and calculates a mucosal feature amount for the region labeling image 36 to obtain mucosal information. It has a function with the mucous membrane feature amount calculation unit 39 that outputs 38.
The video signal output from the CCU 22 is input to the image input unit 21 as an endoscopic image, and becomes digital original image data by inverse γ correction conversion which is a known technique for A / D conversion and linear correction, In accordance with the processing program, the CPU 22 performs the following processing.
First, the CPU 22 performs unnecessary area deletion processing for deleting unnecessary areas from the original image data. This unnecessary area deletion processing replaces the pixel values of pixels unnecessary for the determination of the mucosal tissue type with (0, 0, 0) for the pixel values (R, G, B) of the color original image data, An unnecessary area deleted image 32 from which unnecessary areas have been deleted is generated. The unnecessary area is a dark part or a high luminance halation part in the endoscopic image.

FIG. 4A shows a representative example of the original image 31.
This unnecessary area deletion processing is shown in FIG. When the unnecessary area deletion process starts, as shown in step S1, the CPU 22 determines whether or not the pixel values (R, G, B) of the pixels constituting the color original image data are dark pixels. I do.
That is, the CPU 22 uses the threshold values (or reference values) Rcut, Gcut, and Bcut for determining whether or not the pixel is a dark pixel, and determines whether or not the pixel is R <Rcut, G <Gcut, and B <Bcut. Judgment. Here, the threshold values Rcut, Gcut, and Bcut are predetermined values that have a relatively low luminance value for determining whether or not the pixel is a dark portion, and are set in the processing program.
In step S1, if there is a pixel corresponding to a dark pixel, the CPU 22 replaces the pixel value (R, G, B) of the pixel with the pixel value (0, 0, 0) as shown in step S2. Move on to the next step S3.

In step S1, if there is no pixel corresponding to the dark part, the process proceeds to step S3.
In step S3, the CPU 22 determines whether or not the pixel value (R, G, B) is a halation pixel for each pixel constituting the color original image data.
That is, the CPU 22 determines whether or not the pixel is R> Rhal, G> Ghal, and B> Bhal using threshold values Rhal, Ghal, and Bhal for determining whether or not the pixel is a halation pixel. Here, the threshold values Rhal, Ghal, and Bhal are predetermined values that have relatively high luminance values for determining whether or not the pixel is a halation pixel, and are set in the processing program.
A user such as an operator can also change the dark threshold values Rcut, Gcut, Bcut and the halation threshold values Rhal, Ghal, Bhal from the input operation unit 29.

If there is a pixel corresponding to halation in step S3, the CPU 22 replaces the pixel value (R, G, B) of the pixel with the pixel value (0, 0, 0) as shown in step S4. The area deletion process ends. Also, in step S3, if there is no pixel corresponding to halation, the unnecessary area deletion process is terminated.
When the unnecessary area deletion process of FIG. 3 is completed, an unnecessary area deleted image 32 from which the unnecessary area has been deleted is generated. FIG. 4B shows a specific example of the unnecessary area deleted image 32 generated after the unnecessary area deletion processing is completed for the original image 31 of FIG.
Next, the color tone processing unit 35 calculates a color tone feature amount based on the pixel value for each pixel of the unnecessary region deleted image 32.
However, the color tone processing unit 35 does not perform the calculation process of the color tone feature amount for the pixel having the pixel value (0, 0, 0) which is determined as an unnecessary area in the processing of FIG.

As the color tone feature amount, a hemoglobin (abbreviated as Hb) index having a high correlation with blood flow, which is used in a known technique, that is, Hbindex (abbreviated as IHb) is used. This IHb is a luminance signal Vg for each pixel on the endoscopic image (G signal reflecting light near 560 nm that is most absorbed by hemoglobin) and a luminance signal Vr (light near 650 nm that is less modified by hemoglobin). (R signal to be reflected) is extracted, and a value obtained by logarithmically converting the two ratios, that is, a calculation formula: log ₂ (R / G) is used. This process is shown by step S5 in FIG.
The calculated color feature value by IHb is stored as a pixel value at the same position as the pixel of the original image. For example, it is stored in the image storage unit 24 or the analysis information storage unit 25. In this way, the color tone processed image 34 is generated. FIG. 4C shows an example of the color tone processed image 34 after the color tone feature amount calculation processing is performed on the unnecessary area deleted image 32 of FIG. 4B. Note that FIG. 4C is displayed as a pseudo-colored image, for example.

In this case, when IHb in the color tone processed image 34 is higher than the predetermined threshold value θ, the image to be imaged is highly likely to include Barrett epithelium (Barrett mucosa).
Next, with respect to the color tone processed image 34, the region labeling processing unit 37 generates a region labeling image 36 based on a processing flow described later.
The region labeling image is a labeling image that stores the type of epithelium at the pixel position to be imaged. In this embodiment, 1 is set when the imaging target at the image position is a squamous epithelium of a normal mucosa, 2 is set when it is a Barrett epithelium as a denatured mucosa having changed properties, and 0 is set when it is an unnecessary region. It is set to store.
Next, the process flow of area labeling will be described with reference to FIG.

When the process flow of area labeling starts, the CPU 22 acquires the pixel value C of the unprocessed pixel from the color tone processed image 34 in the first S11.
In the next S12, the CPU 22 determines whether or not the pixel value C is zero. If it is determined that the pixel value C is not 0, the process proceeds to the next step S13. Conversely, if it is determined that the pixel value C is 0, the process proceeds to S16.
In step S13, the CPU 22 determines the magnitude of the predetermined threshold value θ and the pixel value C described above according to the processing program. If the determination result is C> θ, the process proceeds to the next step S14. On the other hand, if it is determined in step S13 that C ≦ θ, the process proceeds to step S15. Note that a threshold value θ for performing the determination classification is set from the sample data of IHb sampled in advance from the normal mucosa and the Barrett epithelium as the denatured mucosa.
In step S14, the CPU 22 sets the corresponding pixel value of the area labeling image to 2 and proceeds to step S17. In step S15, the CPU 22 sets the corresponding pixel of the area labeling image to 1, and proceeds to step S17.

In step S16, the CPU 22 sets the response pixel value of the area labeling image to 0, and proceeds to step S17.
In step S17, the CPU 22 determines whether or not there is an unprocessed pixel. If there is an unprocessed pixel, the process returns to step S11. If there is no unlabeled pixel, the region labeling process ends. To do.
In this way, by the region labeling process, the region labeling image 36 in which each pixel of the color tone processed image 34 is labeled with the pixel value 0, 1, or 2 is generated. FIG. 4D shows an example of the region labeling image 36.
Next, the mucous membrane feature quantity calculation unit 39 calculates various epithelial mucosal feature quantities for the region labeling image 36.
In this embodiment, the number of pixels of each epithelial mucosa type, the boundary pixel point sequence of the region labeling image based on a known boundary line tracking algorithm, the pixel number ratio of the epithelial mucosa type (number of pixels of Barrett epithelium / squamous epithelium) The number of pixels) is obtained and stored as epithelial mucosa information.

The mucous membrane feature quantity calculation unit 39 compares the pixel number ratio of the above-mentioned epithelial mucosa type with a predetermined threshold value τ, determines the presence or absence of the Barrett epithelium, and stores the determination result as epithelial mucosa information. . In this case, it is determined that the Barrett epithelium is present in the image when the pixel number ratio of the epithelial mucosa is ≧ τ.
On the other hand, when the pixel number ratio of the epithelial mucosa type is smaller than τ, it is determined that there is no Barrett epithelium in the image.
FIG. 6 shows the determination process of the bullet epithelium by the mucous membrane feature quantity calculation unit 39.
As shown in step S21, the CPU 22 counts and calculates the number of pixels N1 of the pixel value 1 labeled with the squamous epithelium and the number of pixels N2 of the pixel value 2 labeled with the bullet epithelium. In the next step S22, the CPU 22 determines whether or not the pixel number ratio N2 / N1 of the epithelial mucosa type is N2 / N1 ≧ τ using the threshold value τ for determining the presence or absence of the bullet epithelium.
If the condition of N2 / N1 ≧ τ is satisfied according to the determination result of step S22, as shown in step S23, the CPU 22 determines that the bullet epithelium is present in the original image, and the result is stored in the hard disk 27. Save to etc.

On the other hand, if N2 / N1 <τ based on the determination result in step S22, as shown in step S24, the CPU 22 determines that there is no Barrett epithelium in the original image, and stores the result in the hard disk 27 or the like. save.
After step S23 and step S24, the process proceeds to step S25, where the CPU 22 sends the determination result to the display monitor 4 via the display processing unit 28, and displays the determination result on the display surface of the display monitor 4. Then, this process ends.
In the above description, the pixel number ratio N2 / N1 of the epithelial mucosa type is used, but N2 / (N1 + N2) may be used instead of the pixel number ratio N2 / N1. In this case, the determination condition of N2 / N1 ≧ τ is N2 / (N1 + N2) ≧ (τ + 1). That is, even if the total number of pixels (N1 + N2) in the processing target area is used, the same determination can be made.

As described above, according to the present embodiment, since it is determined by image analysis whether the Barrett epithelium as a specific biological mucous membrane to be detected exists in the image, early detection of the Barrett epithelium without subjective judgment is possible. It becomes easy to do. Also, by displaying the determination result, it is possible for the surgeon to notify that there is a high possibility that Barrett epithelium is present in the image, so the surgeon can more efficiently diagnose whether it is a degenerative part or not. it can.
Note that, for the purpose of noise removal, the region labeling image may be subjected to expansion and contraction processing, which is a known technique, immediately after processing by the region labeling processing unit 37. In the embodiment of the present invention, IHb is used as the color tone feature quantity, but other feature quantities such as R / G, R / (R + G + B) may be used. Further, for example, the hue and saturation in the IHS color space can be used.
FIG. 7 shows a configuration of a main part in the endoscope system 1B of the first modification. In the endoscope system 1B, in the endoscope system 1 shown in FIG. 1, the insertion portion 11 is inserted into the body cavity, for example, at the proximal end side of the insertion portion 11 in the endoscope 6 constituting the endoscope observation apparatus 2. An insertion amount detection device 41 for detecting the insertion amount to be inserted is provided. The insertion amount detection device 41 sends information on the detected insertion amount to, for example, the CPU 22 of the image processing device 3.
The insertion amount detection device 41 includes a plurality of rollers 42 that rotatably contact the outer peripheral surface of the insertion portion 11 and a rotation amount of a rotary encoder or the like that is attached to one rotation shaft of these rollers 42 and detects the rotation amount. And a detection sensor 43.

Then, the signal detected by the rotation amount detection sensor 43 is input to, for example, the CPU 22 in the image processing apparatus 3, and the CPU 22 is an image in which the captured image includes the portion to be detected from the information on the insertion amount. It is determined whether or not. That is, the CPU 22 has a function of a detection target image determination unit 22a that determines whether or not a captured image is an image including a detection target part from the information on the insertion amount.
In this case, when the insertion portion 11 of the endoscope 6 is inserted from the mouth of the patient, the position of the distal end portion 14 when it is inserted deep in the esophagus is detected from the value of the rotation amount detection sensor 43 at the time of insertion. can do.
Also, a condition for determining whether an image having a resolution higher than the resolution that can be determined by the above-described image processing method or an image including a part to be detected is determined by the imaging characteristics of the imaging device 17 including the CCD 16. And put it in

In other words, there are cases where an image including the part to be detected can be obtained from a distance, but if the distant portion is regarded as insufficient resolution for image analysis or the amount of illumination light cannot be sufficient, detection is possible. The image is not determined to include the target part.
An image picked up by the CCD of the endoscope 6 is subjected to signal processing by the CCU 8 and then always inputted to the image input unit 21 of the image processing device 3. The images are sequentially stored in the hard disk 27 or the like.
In this case, the CPU 22 saves information on the amount of insertion as supplementary information in images that are sequentially stored in the hard disk 27 or the like. Alternatively, the CPU 22 may store determination information (identification information) for determining whether or not the captured image is an image including a portion to be detected from the information on the insertion amount as supplementary information.

Then, the CCU 22 performs unnecessary area deletion processing and the like on the image including the determination target part from the determination information, as in the case of the first embodiment. That is, in this modification, the original image subjected to image analysis in the first embodiment is a selected image that is automatically determined as a detection target based on the determination information.
Therefore, according to the present modification, the image to be analyzed is limited to an image that objectively meets a preset condition, so that a more objective determination result can be obtained.
In addition, since an image to be detected can be appropriately detected or selected and set using the insertion amount detection device 41 or the like, it is possible to reduce image processing on an unnecessary image obtained by imaging a portion that is not the detection target. .
Moreover, even if an image is capturing an image of a region to be detected, if the image is far from the imaging device 17, the accuracy of image analysis is insufficient, but according to the present modification, the imaging device 17 In consideration of this characteristic, when the distance is more than a predetermined distance, the accuracy by image analysis can be ensured by setting the determination condition that the region to be detected is not imaged.

Next, a second modification will be described. This modification is different from the first embodiment in part of the processing program. The processing functions of the CPU 22B that processes in accordance with the processing program according to this modification are as shown in FIG.
The processing function shown in FIG. 8 is that a rectangular small area division processing unit 52 that generates a rectangular small area division processing group 51 is provided for the area labeling image 36 in the processing function of FIG. The rectangular small area division processing unit 52 divides the area labeling image 36 to generate a rectangular small area dividing processing group 51.
FIG. 9 shows an image of the rectangular small region 53 generated by being schematically divided.
As shown in FIG. 8, the rectangular small region division processing unit 52 performs mucous membrane feature amount calculation on the rectangular small region dividing processing group 51 when generating the rectangular small region dividing processing group 51 for the region labeling image 36. The configuration is applicable.

In the present modification, the mucous membrane feature quantity calculating unit 39B calculates the mucosal information 38 for each rectangular small region 53, and compares the mucosa type pixel number ratio of the epithelium with a preset threshold value τ. Thus, the attribute of the rectangular small area 53 is determined.
For example, when the pixel number ratio of the epithelial mucosa type is equal to or greater than τ, the attribute of the rectangular small region is set to the bullet epithelium. On the other hand, when the pixel number ratio of the epithelial mucosa type is smaller than τ, the attribute of the rectangular small region is set as the squamous epithelium.
In addition, the mucous membrane feature quantity calculation unit 39B counts the number of attributes determined in the rectangular small region 53, and the attribute ratio (the rectangular small region count number of Barrett's epithelium / the rectangular small region count number of the squamous epithelium) and a predetermined number. The threshold value ε is compared.
For example, when the attribute ratio ≧ ε, it is determined that the Barrett epithelium is present in the image. On the other hand, when the attribute ratio <ε, it is determined that the bullet epithelium does not exist in the image.

FIG. 10 shows a processing flow of mucous membrane feature amount calculation by the mucous membrane feature amount calculation unit 39B.
When the mucous membrane feature amount calculation process starts, as shown in step S31, the CPU 22 calculates the pixel number n1 of the pixel value 1 and the pixel number n2 of the pixel value 2 in each rectangular small region by labeling.
In the next step S32, the CPU 22 determines whether or not the pixel number ratio n2 / n1 is equal to or greater than a predetermined threshold value τ. That is, the CPU 22 determines whether n2 / n1 ≧ τ. If this condition is satisfied, as shown in step S33, the CPU 22 sets the attribute of the rectangular small region that satisfies the condition as Barrett epithelium.
On the other hand, when the condition of n2 / n1 ≧ τ is not satisfied, the CPU 22 sets the attribute of the rectangular small region as squamous epithelium as shown in step S34.

After the processing in steps S33 and S34, in step S35, the CPU 22 determines whether or not there is an unprocessed rectangular small region. If there is an unprocessed rectangular small region, the process returns to step S31, and steps S31 to S35. Repeat the process.
Then, after performing the processing of steps S31 to S35 for all the rectangular small regions, as shown in step S36, the number m1 of the rectangular small regions with the attribute of the bullet epithelium and the number of the rectangular small regions with the attribute of the squamous epithelium. m2 is calculated.
In the next step S37, the CPU 22 determines whether or not the pixel number ratio m2 / m1 is equal to or greater than a predetermined threshold value ε. That is, the CPU 22 determines whether m2 / m1 ≧ ε. If this condition is satisfied, as shown in step S38, the CPU 22 determines that there is a bullet epithelium in the image, stores the determination result in the hard disk 27, and displays it on the display monitor 4.

On the other hand, if the condition of m2 / m1 ≧ ε is not satisfied, the CPU 22 determines that the bullet epithelium exists in the image as shown in step S39, stores the determination result in the hard disk 27, and displays the display monitor. 4 is displayed.
According to this modification, the image is divided into small areas having a certain size, and each small area is a region having a possibility of Barrett epithelium as a detection target mucous membrane for each pixel in the small region. Since it is determined whether or not it is a Barrett epithelium from those determination results, it can be more accurately determined whether or not it is a detection target mucosa.
That is, in the first embodiment, each pixel in the entire image is labeled as to whether or not it is a mucous membrane to be detected, and whether or not a bullet epithelium is included is determined based on the ratio of the number of types of pixels based on the labeling result. Therefore, the determination result of a certain part is reflected in the determination result in the same way, whether it is a nearby part or a distant part.
On the other hand, in this modified example, since the determination is performed by dividing into small regions, the determination in each part can be performed more appropriately, so the reliability of the final determination result Can be improved.

Next, Embodiment 2 will be described with reference to FIGS. FIG. 11 shows functional processing contents performed by the CPU 22 based on the processing program in the second embodiment. In this embodiment, the hardware configuration is the same as that of the image processing apparatus 3 shown in FIG. 1, and the processing program stored in the processing program storage unit 23 is partially different.
In this embodiment, the original image is used as a moving image, for example, and the image of each frame of the moving image is sequentially processed.
For this reason, what was demonstrated in Example 1 is shown with the same code | symbol, and the description is abbreviate | omitted. As shown in FIG. 11, in this embodiment, in the processing program of FIG. 2, the mucous membrane feature amount calculation processing is performed instead of the mucosal feature amount calculation unit 39 that calculates the mucous membrane feature amount for the region labeling image 36. The feature amount processing unit 60 calculates various types of mucosal feature amounts for the region labeling image 36 by the mucosal feature amount processing unit 60 and sequentially stores them in the mucous membrane information buffer 61.

The mucous membrane information buffer 61 is a ring buffer as shown in FIG. 12, and the position of the buffer for the next saving is circulated, so when a predetermined number of mucosal information is generated, the previous information is overwritten. To save. In the example of FIG. 12, for example, the mucosa information 62a, 62b, 62c, 62d and the mucosa information are sequentially stored in time series. Then, when mucosal information is input next, the newest mucosal information is overwritten on the oldest mucosal information 62a. In FIG. 12, although four cases of mucous membrane information 62a to 62d are shown, a ring buffer having a larger storage capacity may be used.
The mucous membrane feature quantity processing unit 60 determines the presence or absence of a bullet epithelium in the image from the pixel number ratio of each epithelial type in the region labeling image 36, and stores the determination result in the mucosal information buffer 61 as the mucosal information 62. It is configured.

In the mucosa information 62 of the mucosa information buffer 61, the mucosa type is determined by the mucosa type determination unit 63.
A processing flow in the mucous membrane type determining unit 63 is shown in FIG.
In the first step S41, the CPU 22 acquires all the mucosa information 62 stored in the mucosa information buffer 61.
In the next step S42, the CPU 22 counts the number of mucous membrane type determination results that are Barrett's epithelium from each mucous membrane information 62, and sets the count to C.
In the next step S43, the CPU 22 compares the predetermined threshold value θ set by the processing program with the count number C described above. That is, it is determined whether or not C> θ.

When C> θ, as shown in step S44, the CPU 22 determines that the bullet epithelium is present in the original image.
On the other hand, when C ≦ θ, as shown in step S45, the CPU 22 determines that there is no bullet epithelium in the original image.
In step S46 after steps S44 and 45, the CPU 22 displays the determination result on the display monitor 4 and ends this processing.
In addition to the determination result, the count number C may be displayed on the display monitor 4.
According to the present embodiment, since the mucous membrane type is determined from the mucosal type result for a plurality of images, it is possible to prevent erroneous determination of the mucous membrane type due to an image with poor observation conditions.

Further, according to the present embodiment, whether or not the Barrett epithelium exists is determined by using the moving image captured by the imaging means provided at the distal end portion 14 while inserting the insertion portion 11 of the endoscope 2 as an original image. Since the determination can be made sequentially, the surgeon can make a diagnosis from the determination result to the vicinity of the detection target site efficiently (or quickly).
Further, according to the present embodiment, when the insertion portion 11 is inserted from the distal end side and inserted into the deep side of the esophagus, the Barrett epithelium is present in the case where the Barrett epithelium is present near the boundary of the stomach. Since the probability of displaying a determination result in which there is an error increases, the diagnosis of Barrett's epithelium can be performed efficiently from the display.

In addition, if the count number C is also displayed on the display monitor 4 as described above, when the bullet epithelium exists, the value of the count number C generally increases as it approaches the site where the bullet epithelium exists. Therefore, diagnosis of Barrett's epithelium can be efficiently performed from the display.
Instead of sequentially inputting each frame of the moving image sequentially, the original image may be input at a constant frame interval.
Further, the mucous membrane feature quantity processing unit 60 is configured not to store the mucosa information 62 when the number of pixels of the pixel value = 0 included in the region labeling image 36 is equal to or larger than a predetermined threshold, It may be configured not to store information inappropriate for the determination.

Next, Embodiment 3 will be described with reference to FIGS. The purpose of this embodiment is to provide an image processing method capable of accurately detecting (determining) the presence or absence of a mucous membrane to be detected. For this purpose, a small region that is not suitable for the detection target, such as a dark part in an image, is detected. And from the detection result, refer to the position information of the dark part in the image, and further exclude from the detection object what is determined as a small area to be excluded adjacent to the dark part. A process of generating an image of the region is performed.
FIG. 14 shows functional processing contents performed by the CPU 22 based on the processing program in the third embodiment. In this embodiment, the hardware configuration is the same as that of the image processing apparatus 3 shown in FIG. 1, and the processing program stored in the processing program storage unit 23 is partially different.
The mucous membrane type determination process in the present embodiment is configured to continuously execute the processing program A shown in FIG. 14A and the processing program B shown in FIG. The processing program B uses the processing target area information 74 that is the result of the processing program A.

A processing block of the processing program A that is a feature of the present embodiment will be described.
The original image 31 is input to a rectangular small region division processing unit 71, and the rectangular small region division processing unit 71 divides the endoscopic image into rectangular small region image groups 72 as shown in FIG. Note that the small area is not limited to a rectangle.
The rectangular small region image group 72 is input to the processing target region determination unit 73, and the processing target region determination unit 73 performs processing shown in the flow shown in FIG. 16 to generate processing target region information 74.
In step S51, a rectangular small region 75 is acquired from the rectangular small region image group 72, and the number of dark area pixels in the rectangular small region 75 is counted in order to detect a processing target excluded small region.
In this embodiment, the dark pixel is a pixel whose pixel values are R <Rcut and G <Gcut and B <Bcut. Let the count number be Cn (where n is the serial number of the small rectangular area). Here, Rcut and the like are those described in the first embodiment. Note that a threshold different from that in the first embodiment may be set.

In step S52, it is determined whether or not there is a rectangular small area 75 to be processed (the process of step S1 is not executed). If there is a rectangular small area 75 to be processed, the process returns to step S51 to perform count processing. If there is no rectangular small area 75 to be processed, the process proceeds to step S53.
In step S53, based on the number Cn of dark area pixels in the small rectangular area 75, a dark area determination process to be excluded from the processing target is performed. The determination in this embodiment is to determine the rectangular small region 75 having the maximum Cn calculated for each rectangular small region 75 as the dark region.
Next, in step S54, the dark area is excluded from the process target area, and the process target area is determined based on the position of the dark area in the original image 31.

The determination method will be described with reference to FIG.
A double hatched portion (cross hatched portion) 76a in FIG. 17 indicates the dark portion area determined in step S53, and lists all the patterns that can be realized when divided into 3 × 3, that is, nine patterns. . A hatched portion 76b indicates a small area to be excluded from the processing target.

In step S54, it is determined which pattern in FIG. 17 corresponds to the dark area according to the position of the dark area in the original image 31, and the dark area and the processing exclusion small area are excluded according to the corresponding pattern. It is determined that only a small area is to be processed.
For example, when the upper right corner is detected as a dark part as shown in FIG. 18, two on both sides adjacent to the double hatched part 76a are excluded as shown in the uppermost pattern on the rightmost side in FIG. .
Then, processing target area information 74 is generated for processing a small area 76c that is not hatched at all other than the double hatched portion 76a and the hatched portion 76b in FIG. That is, the processing target area information 74 is generated (determined) from the detected information of the double hatched portion 76a.

For this reason, in the storage means such as the processing program storage unit 23 and the analysis information storage unit 25, processing is performed according to which block position in the original image 31 the block of the dark area (small area) detected in advance is located. Excluded block information for determining a block to be excluded from the target is registered (stored). Specifically, a plurality of patterns are stored as excluded block information as shown in FIG. Then, by referring to the excluded block information, the processing target area can be determined according to the position of the dark area.
The unnecessary area deletion unit 33B in the processing program B uses the above-described processing target area information 74 to delete unnecessary areas by replacing pixels that are not processing targets with (R, G, B) = (0, 0, 0). An image 32 is generated.

The subsequent processing is the same as, for example, the mucous membrane information generation processing content of FIG.
Note that the processing target region determination unit 73 counts the number of pixels with R = 255 in the processing target region for each small region 75, and captures an image when the number of pixels with R = 255 is equal to or greater than a predetermined number of pixels. It may be configured that the small region 75 is excluded from the processing target region on the assumption that it is too close to the target.
Note that the size (number of pixels) of the small area 75 may be selected and set. Further, depending on the size, a plurality of dark portions may be detected, and their surrounding portions may be excluded from the processing target region based on the arrangement, distribution, and the like. In this way, the processing target area can be set more appropriately.

According to the present embodiment, in addition to the small area including the dark part, it is determined whether or not the small area adjacent to the dark part is an inappropriate small area for processing, and the inappropriate area is processed. Therefore, the remaining part becomes more suitable for the detection target, and therefore the detection accuracy (determination accuracy) of a specific biological mucous membrane can be further improved.
Supplementally, when the dark area and the area close thereto are distributed in, for example, concentric circles or other shapes according to the site in the body cavity, if the dark area is detected by simply setting the small area 75, There is a possibility that the dark part and the area close thereto cannot be reliably excluded from the detection target.
Even in such a case, a region that is not suitable for the detection target can be efficiently and appropriately excluded from the position, arrangement, or distribution of the dark region detected as described above. Although it is possible to exclude pixels in the dark portion for each pixel, this takes time. On the other hand, in the present embodiment, since blocks are used, inappropriate blocks can be efficiently excluded.

In the above description, the case of an image obtained by the endoscope 2 having an elongated insertion portion has been described. However, as described below, an image obtained by using a capsule endoscope is used. May be.
As shown in FIG. 19, a capsule endoscope system 81 having a modified example includes a capsule endoscope apparatus (hereinafter abbreviated as a capsule endoscope) 82 that images a body cavity when a patient swallows, and a patient And an external device 83 that receives and records image data from the capsule endoscope 82, and an image processing device 84 to which an image is input from the external device 83.
The capsule endoscope 82 is, for example, an LED 85 as an illuminating unit in a capsule container, an objective lens 86 that connects an image of the illuminated subject, and a CCD 87 that constitutes an imaging unit that performs imaging. A control circuit 88 that performs signal processing on an image signal captured by the CCD 87, a wireless circuit 89 that performs processing to wirelessly transmit the captured image, and a battery 90 that supplies power to each circuit and the like. .

The extracorporeal device 83 receives radio waves from the antenna 89 a of the radio circuit 89 of the capsule endoscope 82 via the plurality of antennas 91 a, 91 b, 91 c, and sends the signal to the control circuit 93. This control circuit 93 converts it into a video signal and outputs it to the image processing device 84.
Then, the image processing device 84 performs one image processing in the above-described embodiment.
The control circuit 93 has a position detection function 93a that estimates the position of the capsule endoscope 82 using a plurality of antennas 91a to 91c. The position detection function 93a may be used to select and set the detection target image as described in the first modification of the first embodiment.
Note that embodiments configured by partially combining the above-described embodiments or changing the order of processing steps also belong to the present invention.

[Appendix]
1. In the fourth step of performing the process of detecting the presence of a specific biological mucosa in claim 1 or 2,
A process for detecting the presence of a specific biological mucous membrane is performed based on the number of pixels A in which the value of the color tone characteristic amount is in a specific range and the total number B of pixels in the processing target area.
2. In Additional Description 1, a process of determining a specific biological mucous membrane when the ratio A / B between the number of pixels A and the total number of pixels B is larger than a predetermined value is performed.
3. In Additional Description 1, when the ratio B / A between the number of pixels A and the total number of pixels B is smaller than a predetermined value, a process of determining a normal biological mucous membrane is performed.
4). In the fourth step of performing the process of detecting the presence of a specific biological mucosa in claim 3,
A process for detecting the presence of a specific biological mucous membrane is performed based on the number of images A in which the value of the color tone characteristic amount is within a specific range and the total number B of pixels in the block.

5. In Claims 1-3, when the image is a plurality of images, when the number of images determined to be highly likely to include a specific biological mucous membrane is greater than a predetermined number, the captured image is a specific biological mucous membrane. It is characterized in that it is determined that the image is an image.
6). In Supplementary Note 5, the plurality of images is a continuous frame image obtained by imaging the living body mucous membrane or a continuous image obtained by imaging the living body mucous membrane at a predetermined frame interval, and is configured by a predetermined number of images.
7). Appendix 5 is characterized in that an image in which an area occupied by an image in the processing target area is smaller than a predetermined value is excluded from the plurality of images.
8). In Claims 1 to 4 and Supplementary Notes 1 to 7, the second step detects a detection target exclusion region in which a pixel whose color signal value is within a specific range is extracted and excluded from the detection target, and A process of setting an image of the processing target area according to a position of the detection target exclusion area in the image is performed.

9. In Supplementary Note 8, the image is divided into a plurality of blocks, and a block pattern is selected by collating with a plurality of previously registered block patterns based on the block position where the detection target exclusion region exists,
A process for forming an image of the processing target area is performed based on the selected block pattern.
10. In any one of claims 1 to 4 and appendices 1 to 9, the color tone feature amount is based on a ratio of a plurality of color signals.
11. In Claims 1 to 4 and Supplementary Notes 1 to 10, the image is an image captured by an endoscope having an elongated insertion portion.
12 In Claims 1 to 4 and Supplementary Notes 1 to 10, the image is a captured image by a capsule endoscope.

13. In Claims 1 to 4 and Supplementary Notes 1 to 12, the third step performs a process of calculating a hemoglobin index as the color tone feature amount.
14 In Supplementary Note 13, the fourth step includes a representative value of a hemoglobin index sampled from a biological mucosa having a normal hemoglobin index value and a representative value of a hemoglobin index sampled from a specific denatured mucosa having a changed property. The threshold value set for distinguishing both mucous membranes is used, and the presence of the specific denatured mucosa is detected (determined) by comparing with the threshold value.
15. The supplementary note 14 is characterized in that the normal biological mucous membrane to be sampled is a mucosa of squamous epithelium in the esophagus, and the specific abnormal biological mucosa to be sampled is a biological mucous membrane of Barrett epithelium in which the squamous epithelium is denatured. .

16. In any one of claims 1 to 4 and appendices 1 to 15, the second step is characterized in that a dark portion is detected and excluded as a pixel inappropriate for the detection target.
17. In any one of claims 1 to 4 and appendices 1 to 16, the second step includes a process of detecting and excluding a high-luminance halation part as a pixel inappropriate for the detection target.
18. The image determination for performing determination processing as to whether or not the first image is an image of a biological mucous membrane at a position near a boundary between the esophagus and the stomach. It has a step.
19. An image input unit that inputs an image signal corresponding to an image of two or more colors obtained by imaging a biological mucous membrane; a processing target region setting unit that sets a processing target region that excludes pixels inappropriate for detection in the image;
A feature amount calculation unit that calculates a color tone feature amount at least for each pixel in the processing target region;
A biological mucous membrane detection unit that detects the presence of a specific biological mucous membrane based on the calculated color tone feature amount;
An image processing apparatus comprising:

20. In an image processing method for performing image processing on an image and calculating a predetermined feature amount,
A first step of performing a process of dividing the image into a plurality of blocks;
A second step of performing a process of detecting an inappropriate block including pixels inappropriate for calculating the feature amount;
A third step of performing a process of setting a block to be excluded from the processing target area according to the position of the inappropriate block in the image;
An image processing method comprising:
21. Appendix 20 is characterized in that the third step performs a process of setting a block to be excluded from the processing target area with reference to information registered in advance.
(Background to Appendix 20 and 21)
Same as the text background.
(Operational effects of appendices 20 and 21)
In addition to blocks that contain pixels that are inappropriate for feature value calculation directly, blocks that contain pixels that are inappropriate for feature value calculation can also be excluded from the processing target area, improving the accuracy of feature value detection. Can be improved.

  The image obtained by imaging the mucous membrane near the boundary between the esophagus and the stomach in the body cavity is degenerated from the normal mucous membrane of the esophagus by calculating the color characteristic amount such as the hemoglobin index by excluding inappropriate parts such as dark areas. By determining the presence or absence of Barrett's epithelium as a mucous membrane, it is possible to support the endoscopy by the operator.

1 is a block diagram showing a configuration of an endoscope system having a function of an image processing method according to a first embodiment of the present invention. FIG. 3 is a diagram illustrating a processing function for realizing the image processing method according to the first embodiment, an image to be used, and the like. The flowchart figure which shows the processing content of the unnecessary area | region deletion process which deletes an unnecessary area | region. The figure which shows image examples, such as an original image and an unnecessary area | region deletion image. The flowchart figure which shows the processing content of area | region labeling. The flowchart figure which shows the processing content of mucous membrane feature-value calculation. The figure which shows a part of structure of the endoscope system provided with the 1st modification. The figure which shows the processing function for implement | achieving the image processing method in a 2nd modification, the image to be used, etc. FIG. The figure which shows the example of an image divided | segmented into the rectangular small area | region. The flowchart figure which shows the processing content of the mucous membrane feature-value calculation in a 2nd modification. FIG. 10 is a diagram illustrating a processing function for realizing the image processing method according to the second embodiment of the present invention, an image to be used, and the like. The figure which shows the ring buffer which stores mucous membrane information. The flowchart figure which shows the process of mucous membrane determination. FIG. 10 is a diagram illustrating a processing function and an image to be used for realizing the image processing method according to the third embodiment of the present invention. The figure which shows the image divided | segmented into the rectangular small area image group. The flowchart figure which shows the processing content of process target area | region determination. Explanatory drawing in the case of determining a process target area | region based on the detected dark part area | region. The figure which shows the example of an image of the detected dark part. The block diagram which shows the structure of the capsule type endoscope system provided with the capsule type endoscope in a modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Endoscope system 2 ... Endoscope observation apparatus 3 ... Image processing apparatus 4 ... Display monitor 6 ... Endoscope 7 ... Light source device 8 ... CCU
11 ... Insertion part 15 ... Objective lens 16 ... CCD
17 ... Imaging device 21 ... Image input unit 22 ... CPU
23 ... Processing program storage unit 24 ... Image storage unit 25 ... Analysis information storage unit 27 ... Hard disk 28 ... Display processing unit 29 ... Input operation unit 31 ... Original image 32 ... Unnecessary area deletion image 33 ... Unnecessary area deletion unit 34 ... Tone processing Image 35 ... Color tone processing unit 36 ... Area labeling image 37 ... Area labeling processing unit 38 ... Mucosal information 39 ... Mucosal feature calculation unit Agent Patent attorney Susumu Ito

Claims (4)

A first step of performing a process of inputting an image signal corresponding to an image of two or more colors obtained by imaging a biological mucous membrane;
A second step of performing a process of setting a processing target area by excluding pixels inappropriate for the detection target in the image;
A third step of performing a process of calculating a color tone characteristic amount for each pixel at least in the processing target area;
A fourth step of performing a process of detecting the presence of a specific biological mucous membrane based on the calculated color tone feature amount;
An image processing method comprising: The third step of performing the process of calculating the color tone feature amount is as follows:
The image processing method according to claim 1, wherein the image is divided into a plurality of blocks, and the tone characteristic amount calculation process is performed for each block.   In the fourth step, the presence of a specific biological mucous membrane is detected for each block, and when the number of blocks determined that the specific biological mucosa exists is larger than a predetermined number, 3. The image processing method according to claim 2, wherein a determination process is performed if the image exists in the image.   The images are a plurality of time-series images, and when the number of cases where it is determined in the fourth step that a specific biological mucosa exists in the plurality of images is greater than a predetermined number, The image processing method according to claim 1, wherein a process for determining that the specific mucous membrane is a specific biological mucous membrane is performed.