JP2001148862A - Image processing unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing unit that can obtain a color slurring corrected image where a color slurring part is not conspicuous by avoiding incomplete correction even when an object image obtained from an image pickup means is distorted. SOLUTION: Color slurring is discriminated for each pixel from each of R, G, B color images obtained with the image pickup means, color slurring information of each pixel of an image with the least color slurring is read (S21), and when the color slurring quantity is deviated from a permissible range, the pixel data of the image with the least color slurring and the pixel data of the image with the 2nd least color slurring are read and both data are synthesized at a prescribed ratio (S26). The above processing is repeated by the number of entire pixels of the image (S28), and a color slurring corrected image is generated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing apparatus suitable for obtaining an image in which color misregistration is inconspicuous.

[0002]

2. Description of the Related Art At present, a gastrointestinal tract such as the esophagus, stomach, small intestine, and large intestine and a trachea such as a lung are observed by inserting a scope into a body cavity, and a treatment tool inserted into a treatment tool channel as necessary. 2. Description of the Related Art Medical endoscopes for performing various treatments using a medical device are generally used.

In particular, an electronic endoscope using an image pickup means such as a charge-coupled device (CCD) can display a moving image on a color monitor in real time, thereby reducing the fatigue of an operator who operates the electronic endoscope. It is widely used because it can be.

As a method of obtaining a color image with an electronic endoscope, there are a simultaneous method in which a color mosaic filter is inserted in front of an image pickup means, and a method in which a monochrome CCD is used as an image pickup device, and an R light is applied to the front of a lamp of a light source device for illuminating a subject. , G,
A plane-sequential method of sequentially inserting each filter of B to obtain a color image has been put to practical use.

[0005] A surface-sequential type electronic endoscope (hereinafter referred to as "surface-sequential type endoscope") has advantages such as good color reproducibility and the ability to reduce the size of imaging means.
Since the R, G, and B images are captured in chronological order, there is a disadvantage that a color shift occurs when the relative positional relationship between the subject and the imaging unit changes.

In particular, when an endoscope image is displayed as a still image, a noticeable color shift hinders accurate diagnosis.
For example, in Japanese Patent Application Laid-Open No. 3-146024 or Japanese Patent Publication No. 2836158, a shift vector for each area between R, G, and B images is obtained, and the position of the image is shifted for each area according to the shift vector, and interpolation is performed. A method of correcting a color shift by processing has been proposed.

[0007]

However, according to the techniques described in these prior arts, the correction of color misregistration becomes incomplete if the subject image is distorted in the area as the area becomes larger. There is a problem.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described points, and has no area, and creates a color misregistration corrected image which is not incompletely corrected even when a subject image is distorted. An object of the present invention is to provide an image processing apparatus capable of making a color shift portion inconspicuous.

[0009]

A first image processing apparatus according to the present invention comprises: a frame sequential imaging unit for capturing an image of a subject for each of a plurality of color components; A color misregistration amount detecting means for repeatedly detecting a color misregistration amount between images when a combination of an image of a specific color as a reference and an image of a color other than the reference is performed a predetermined number of times within the predetermined time; Discriminating means for discriminating a plurality of color misregistration amounts smaller than other color misregistration amounts among the color misregistration amounts repeatedly detected by the color misregistration amount detecting means; A synthesizing means for synthesizing a combination of images at a predetermined ratio when a color shift amount occurs.

In such a configuration, a subject is imaged for each of a plurality of color components by a frame sequential imaging means, and an image of a specific color as a reference and an image of a color other than the reference among images obtained for each color component are obtained. The amount of color shift between images when
A predetermined number of times of repetition detection within a predetermined time, a plurality of small color misregistration amounts are compared with other color misregistration amounts among the repeatedly detected color misregistration amounts, and a plurality of obtained relatively small color misregistration amounts are obtained. Are combined at a predetermined ratio to obtain a color misregistration corrected image.

The second image processing apparatus includes a frame sequential imaging unit that captures an image of a subject for each of a plurality of color components, and a specific color as a reference among images for each color component obtained by the frame sequential imaging unit. A color misregistration amount detecting means for repeatedly detecting a color misregistration amount between images obtained by combining the image with the image of a color other than the reference within a predetermined time, and a color misregistration amount repeatedly detected by the color misregistration amount detecting means. Judging means for judging at least the first color misregistration amount and the second smallest color misregistration amount among the color misregistration amounts, and at least the first color misregistration amount based on the judgment result by the judging means. And a combining means for combining the images combined when the second color shift amount occurs and the images combined when the second color shift amount occurs at a predetermined ratio.

In such a configuration, a subject is imaged for each of a plurality of color components by a frame sequential imaging means, and an image of a specific color as a reference and an image of a color other than the reference among the obtained images of the respective color components are obtained. The amount of color misregistration between the images when combined with the image is repeatedly detected for a predetermined number of times within a predetermined time, and at least the first color misregistration amount and the second smallest second color misregistration amount among the repeatedly detected color misregistration amounts. The images combined at least when the first color shift amount occurs and the images combined when the second color shift amount occurs based on the determination result. Are synthesized at a predetermined ratio to obtain a color misregistration corrected image.

[0013]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows a configuration of a frame sequential endoscope apparatus in which a color misregistration correction apparatus employed in the present embodiment is incorporated.

As shown in FIG. 1, a frame sequential endoscope apparatus according to the present embodiment includes a frame sequential endoscope 1 having a built-in image pickup means for picking up an object for each of a plurality of color components, and a video processor 2. , A color shift correction device 3, an image processing device 4, and a display device 5.

The frame sequential endoscope 1 has an elongated insertion portion,
The insertion section is inserted into a living body to perform imaging. An illumination optical system and an image pickup means are provided at the distal end of the insertion portion, and red light, green light, and blue light sent at predetermined timing from a light source device (not shown) are sequentially emitted from the illumination optical system.

The reflected light of red light, green light and blue light applied to the inside of the living body is photoelectrically converted for each color component by an image pickup means such as a built-in CCD, and is output to the video processor 2. After being processed into video signals of R image, G image, and B image by a video signal processing circuit built in the processor 2, the video signal is output to the color misregistration correction device 3. The configuration and operation of the color misregistration correction device 3 will be described later.

The endoscope image output from the color shift correcting device 3 and corrected for color shift is subjected to various image processing such as color tone emphasis and frequency emphasis by an image processing device 4 and then output to a display device 5. Is displayed.

In the color misregistration correction apparatus 3 according to the present embodiment, the color misregistration occurring at the time of freeze is shifted horizontally and vertically with respect to the G image by the R and B images. Are selected and combined to create a color misregistration corrected image.

FIG. 2 is a block diagram of the color misregistration correction device 3. The color misregistration correction device 3 includes an image memory 6 that stores an input endoscope image, a CPU 7 that can execute a color misregistration determination program and a correction image creation program, and a color that stores color misregistration information for each pixel of a plurality of images. A misregistration determination result storage means 8;
And so on.

The input endoscope image is stored in the color misregistration correction image memory 9 when observing a moving image, and is output to the image processing device 4. However, when the image freezes, the image is stored in the image memory 6. A color shift correction process is performed. That is, since the input endoscope image is an image captured by the frame sequential type endoscope 1, a color shift occurs.

The image stored in the image memory 6 is sent to the CPU
7, the R image and the B image are read at a read start address different from that of the G image (in a state shifted in the horizontal and vertical directions), and color misregistration is determined.

For example, if the read start address of the G image is α, the read start addresses of the R and B images are read as α + β. Then, the R image and the B image are
The image is shifted to the left by β pixels from the G image.

More specifically, the color misregistration correction processing is performed according to the flowcharts shown in FIGS. That is,
In the color misregistration determination routine shown in FIGS. 3 and 4, color misregistration determination processing is performed. Subsequently, in the color misregistration correction image creation routine shown in FIG. 5, a color misregistration corrected image is created based on the color misregistration determination result. .

In the color misregistration determination routine shown in FIGS. 3 and 4, first, the R image and the B image stored in the read start address A of the image memory 6 are read in step S1, and the read is performed in step S2. Read start address B shifted by β pixels from start address A
Read out the G image stored in.

Then, the process proceeds to step S3, in which the determination of color shift is performed for each pixel, and the result is temporarily stored in the color shift determination result storage means 8 until the number of pixels of the entire image is determined. .

When the color misregistration determination for the number of pixels of the entire image is completed, the process proceeds to step S4, and the color misregistration determination for each image is performed using the color misregistration determination result for each pixel.

Based on the color misregistration for each image determined in step S4, color misregistration correction processing is performed in step S5 and subsequent steps.

First, in step S5, it is checked whether or not the color shift determined in step S4 is the smallest among the images determined so far. Proceeding to S6, the storage of the image which is stored in the color shift determination result storage means 8 and which has been determined to have the second lowest color shift so far is deleted, and in step S7,
The information of the image determined to have the least color shift is 2
The information is stored as the information of the image with the least color shift.

In step S8, the information of the image to be determined this time (the read start address of the R image and the B image and the color shift determination result for each pixel) is used as information of the image with the least color shift. Are stored in the color misregistration determination result storage means 8.

Then, the process proceeds to step S9, where the number of determinations n
Is checked whether a predetermined number of times no has been reached. If not (n <no), step S10
Then, a value α shifted downward or rightward by a predetermined amount is added to the read start address A, the next read start address A is set (A = A + α), and the process returns to step S1.

When the number of determinations n reaches a predetermined number of times no (n = no), the routine exits from the routine and a color misregistration corrected image is created.

On the other hand, if it is determined in step S5 that the color misregistration determined in step S4 is not the least color misregistration, the process branches to step S11, and the second least color misregistration has been determined so far. It is checked whether the image is an image or not, and when it is determined that the color misregistration is the second smallest, the process proceeds to step S12, and the color misregistration result storage unit 8 stores the second color misregistration. In step S13, the information of the image to be determined this time (the read start addresses of the R image and the B image and the color shift determination result for each pixel) is deleted. 2
It is stored in the color misregistration determination result storage unit 8 as the information of the image not in the second order, and the process returns to step S9.

If it is determined in step S11 that the color misregistration of the image to be determined this time is not the first or second least, the process proceeds to step S14, and the color misregistration determination result storage means is executed. Then, the color misregistration information of the currently determined image temporarily stored in step S8 is deleted, and step S9 is executed.
Return to

As described above, by reading the image information from the image memory 6 a predetermined number of times and performing the color shift determination, the color shift determination result storing means 8 stores the image with the least color shift and the second color shift. Is stored.

Thereafter, an image is read from the image memory 6 by using the color shift information by the corrected image creation program executed by the CPU 7, and the image with the least color shift and the image with the second lowest color shift are synthesized. Thus, a color misregistration corrected image is created and recorded in the color misregistration corrected image memory 9.

The process of creating the color misregistration corrected image is specifically performed by a color misregistration corrected image creation routine shown in FIG.

In this routine, first, at step S21
Then, the color shift information for each pixel of the image with the least color shift is read out from the color shift determination result storage means 8, and in step S22, the color shift amount falls within a predetermined allowable range based on the color shift information. Check if it is.

If the amount of color misregistration is within the allowable range, the process proceeds to step S23, in which the pixel data of the image with the least color misregistration is read from the read start address stored in the color misregistration determination result storage means 8. Image memory 6 based on
Read from

If the color misregistration amount does not fall within the predetermined allowable range, the flow branches to step S24, and the color misregistration result storage means 8 stores the color misregistration information of the image with the second smallest color misregistration. Originally read out the color shift information of the same pixel,
The process proceeds to step S25 to check whether or not the color shift amount falls within a predetermined range based on the color shift information.

If the amount of color misregistration is not within the predetermined range, the process returns to step S23.

If it is determined in step S25 that the amount of color misregistration is within the predetermined range, the process proceeds to step S26, where the pixel data of the image with the least color misregistration and the pixel data of the image with the second least color misregistration are displayed. The data is read from the image memory 6 based on the read start address stored in the color misregistration determination result storage means 8, and the two pixel data are combined at a ratio of, for example, first: second = 7: 3. I do.

Then, step S23 or step S
When the process proceeds from step 26 to step S27, the pixel data of the image with the least color shift read out in step S23 or the pixel data synthesized in step S26 is stored in the color shift correction image memory 9.

Then, the process proceeds to a step S28, where it is determined whether or not a corrected image has been created for all the pixels of the image.
If there is an uncreated pixel, the process returns to step S21.

This routine is repeated for the number of pixels of the entire image, and when corrected images of all the pixels have been created, the process exits the routine via step S28, and the color shift correction of one image is completed.

The color misregistration corrected image created by the color misregistration corrected image creation routine is output to the image processing device 4 and
After predetermined image processing, the image is displayed on the display device 5.

Incidentally, in the frame sequential endoscope 1, a color shift occurs due to a difference in the image capturing timing between the R image and the G image.

FIG. 6 shows an R image obtained by irradiating illumination light in the R wavelength region, FIG. 7 shows a G image obtained by irradiating illumination light in the G wavelength region, and FIG. 3 shows a composite image of an R image, a G image, and a B image.

Here, assuming that the subject image 10 has moved from left to right, it is assumed that the G image is picked up after the R image is picked up.
It is assumed that the subject image 10 is distorted as shown in FIG. For the sake of simplicity, the description of the B image is omitted because it is assumed that the same image as the G image has been obtained.

As shown in FIG. 8, an image obtained by combining the R image, the G image, and the B image is a color endoscope image in which a color shift occurs in a hatched portion. A color shift correction image obtained when this image is input to the color shift correction device 3 described in the present embodiment will be described.

When an image in which a color shift has occurred as shown in FIG. 8 is input to the color shift correcting device 3, the above-described FIGS.
In the color misregistration determination routine shown in FIG. 9, the read start addresses of the R image and B image when forming an image with the least color misregistration as shown in FIG. 9, information on the color misregistration amount for each pixel, and FIG. The read start address of the R image and the B image and the information of the color shift amount for each pixel when constructing the second image with the least color shift as shown are stored in the color shift determination result storage means 8.

Next, the processing for correcting the color shift is performed according to the above-described color shift correction image creation routine shown in FIG.

FIG. 9 shows an image with the least color shift and its two-dimensional graph (hereinafter, “profile”).
Referring to the information on the amount of color shift in the profile 12 on the line 11 traversing the subject image 10 as shown in the drawing, the amount of color shift in the section 13 exceeds the allowable range. Therefore, section 13
In the pixel of the above, the same pixel of the second image with less color shift is referred to.

FIG. 10 shows an image having the second least color shift and its profile. Section 13 in profile 15 on line 14 crossing subject image 10 shown in FIG.
Looking at the information on the color misregistration amount of the pixel corresponding to, it can be seen that the color misregistration amount falls within a predetermined allowable range.

Therefore, the two image data are combined to create a color misregistration corrected image. Here, the image data with the least color shift is combined at 70%, and the image data with the second least color shift at 30%. Note that the combination ratio is merely an example, and the image ratio may be replaced with image data having the second smallest color shift, or the combination ratio may be changed according to the color shift amount.

FIG. 11 shows the obtained color misregistration corrected image and the profile 17 on the line 16 crossing the subject image 10. As can be seen from the figure, since the subject image 10 is distorted, it is possible to reduce the color shift in the section 13 in which the correction was not sufficient only by shifting the R, G, and B images.

In this embodiment, the color misregistration correction device 3
After that, the image processing apparatus 4 is connected, so that problems such as the color misregistration part being emphasized and obstructing observation are reduced.

By performing a filtering process such as smoothing on the color misregistration corrected image so as to make the boundary of the synthesized portion inconspicuous, it is possible to obtain an image which can be more easily observed.

[Appendix] As described above, according to the present invention, the following configuration can be obtained. (1) An image storage means for storing an image picked up by the electronic endoscope apparatus, in a color shift correction apparatus for a plane sequential endoscope connected to or incorporated in the plane sequential endoscope apparatus, The image of each color of R, G, B is read from the image storage means,
A color misregistration judging unit for judging a color misregistration amount, a color misregistration corrected image information storage unit for storing a plurality of pieces of information of an image determined to have a small color misregistration amount by the color misregistration determining unit, When the color misregistration determination performed by the means is completed, a color misregistration corrected image is created by combining a plurality of images when the color misregistration amount is small based on the information stored in the color misregistration corrected image information storage means. A color shift correction apparatus for a field sequential endoscope, comprising:

(2) In (1), there is provided an image processing means which is provided at a subsequent stage of the color misregistration corrected image synthesizing means and performs image processing on the color misregistration corrected image.

In these arrangements, a color shift determination is performed for an image obtained by shifting each of the R, G, and B images in the horizontal and vertical directions, and the color shift for each pixel of the image determined to have a small color shift is performed. Is stored in a memory. After repeating this operation a plurality of times, a color misregistration corrected image is created by combining the image with the least color misregistration and the image with the second least color misregistration based on the color misregistration information stored in the memory.
Therefore, even when the subject image obtained by the imaging unit is distorted, it is possible to obtain an image with reduced color shift.

[0061]

As described above, according to the present invention,
Since an image with a small amount of color misregistration is synthesized to obtain a color misregistration corrected image, there is no need to provide an area, and even if the subject image obtained by the imaging means is distorted, the correction does not become incomplete. Thus, it is possible to obtain a color misregistration corrected image in which the color misregistration portion is not noticeable.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a field sequential endoscope device incorporating a color misregistration correction device.

FIG. 2 is a block diagram of a color misregistration correction device.

FIG. 3 is a flowchart showing a color misregistration determination routine (part 1).

FIG. 4 is a flowchart illustrating a color misregistration determination routine (part 2);

FIG. 5 is a flowchart illustrating a color shift correction image creation routine.

FIG. 6 is an explanatory diagram of an R image obtained by irradiating illumination light in an R wavelength region.

FIG. 7 is an explanatory diagram of a G image obtained by irradiating illumination light in a G wavelength region.

FIG. 8 is an explanatory diagram of a composite image of an R image, a G image, and a B image.

FIG. 9 is an explanatory diagram showing the relationship between the read start addresses of the R and B images and the amount of color misregistration when forming an image with the least color misregistration.

FIG. 10 is an explanatory diagram showing the relationship between the read start address of the R image and the B image and the amount of color misregistration when constructing the second image with the least color misregistration.

FIG. 11 is an explanatory diagram showing a relationship between a color misregistration correction image and a profile on a line crossing a subject.

[Explanation of symbols]

 3 Color shift correction device 10 Subject image

Continued on the front page (72) Inventor Katsuichi Imaizumi 2-43-2 Hatagaya, Shibuya-ku, Tokyo Inside Olympus Optical Industrial Co., Ltd. (72) Inventor Tatsuhiko Suzuki 2-43-2 Hatagaya, Shibuya-ku, Tokyo Olympus Within Optical Industry Co., Ltd. (72) Inventor Fumiyuki Okawa 2-43-2 Hatagaya, Shibuya-ku, Tokyo Inside Olympus Optical Industry Co., Ltd.

Claims (2)

[Claims] 1. A plane-sequential imaging unit that images a subject for each of a plurality of color components, and a reference color image and a reference color image among images for each color component obtained by the plane-sequential imaging unit. A color misregistration amount detecting means for repeatedly detecting a color misregistration amount between images when a combination of the color misregistration image and the color misregistration amount is detected by the color misregistration amount detecting means within a predetermined time; Discriminating means for discriminating a plurality of small color misregistration amounts as compared with other color misregistration amounts; and combining a plurality of relatively small image misregistration amounts obtained by the discriminating means. An image processing apparatus, comprising: combining means for combining at a ratio. 2. A plane-sequential imaging unit for imaging an object for each of a plurality of color components, and a reference color image and a reference color image among images for each color component obtained by the plane-sequential imaging unit. A color misregistration amount detecting means for repeatedly detecting a color misregistration amount between images when a combination of the color misregistration image and the color misregistration amount is detected by the color misregistration amount detecting means within a predetermined time; Determining means for determining at least the smallest first color shift amount and the second smallest color shift amount; and combining at least when the first color shift amount is generated based on the determination result by the determining means. An image processing apparatus, comprising: combining means for combining, at a predetermined ratio, the combined images and the combined images when the second color shift amount occurs.