JP2003224859A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide image processing technology for effectively suppressing production of a false color while reducing the occurrence of loss of colors. <P>SOLUTION: The image processing apparatus 10 of this invention applies interpolation processing to image signals of each pixel received from a CCD of a Bayer system by using image signals of pixels in the vicinity of the target pixel to output an image signal with three primary color components from each pixel. The image signal in a color space having the three primary color components is converted into image signals in a color space having a luminance signal Y and color difference signals Cr, Cb. Further, an HPF 72 or the like detects a degree of inclusion of a high frequency component of the luminance signal Y in the vicinity of each pixel, and an absolute value adder circuit 81 and an LPF 72 extract a low saturation area on the basis of the color difference signals Cr, Cb. Then a chroma killer circuit 80 applies suppression control of the color difference signals Cr, Cb depending on the degree of inclusion of the high frequency component as to pixels in the low saturation area and provides an output of the result. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a configuration of an image processing apparatus used in a digital camera and the like and an image processing method.

[0002]

2. Description of the Related Art In digital cameras and digital video cameras, CCD (Ch
arge Coupled Device) is used. The CCD is configured by arranging a large number of photodiodes that control photoelectric conversion in a matrix.

An optical image of a subject incident on a CCD through an optical system including an image pickup lens is photoelectrically converted by a photodiode and then accumulated. The charges accumulated in the photodiodes arranged in a matrix are sequentially taken out from the CCD, transferred to the image processing circuit, and detected as an image signal.

As described above, the CCD plays a role of accumulating electric charges according to the lightness and darkness of an optical image and outputting it as an electric signal, but in order to obtain a color image, three primary color components (R, G, B) are used. It is necessary to obtain an electric signal corresponding to.

A Bayer color filter array 100 used in a single-plate color image pickup apparatus will be described with reference to FIG.

Bayer color filter array 100
Arranging G filters contributing to the luminance signal in a checkered pattern, and arranging R and B filters in the remaining part in a checkered pattern. The filters of each color arranged in this way are
It corresponds to each pixel (photodiode), and the charge corresponding to the primary color component of any one of R, G, and B is accumulated in each pixel.

According to the Bayer method, each pixel outputs an electric signal for any of the primary color components, so that it is necessary to supplement the missing color information for each pixel.

For example, a pixel covered by an R filter has only R information, and therefore an interpolation process is performed to estimate G and B based on the values of peripheral pixels. In this way, a color image is captured using one CCD.

As described above, in the Bayer method in which the interpolation processing is performed by using the information of the neighboring pixels, it is preferable that the same subject be imaged in the area in which the interpolation processing is performed. For example, when the pixels are mutually interpolated in the area 110 indicated by the thick line in FIG. 9, it is preferable that the same subject be imaged in the area 110.

However, depending on the contrast pattern of the subject, there may be high-frequency components in a region narrower than the region 110, and in such a case, so-called false color will occur. False colors occur at high frequencies,
It occurs in the contours, lines, and fine patterns of the subject, and cannot be completely removed by optical LPF (low-pass filter), interpolation processing, or the like.

For example, as shown in FIG.
It is assumed that the object of the boundary 121 or the dot 122 is imaged within 0. In this case, since the lower right pixel 114 in the area 110 has only the information of B,
Information on R and G is interpolated from the pixels 111, 112, 113 and the like. For this reason, the interpolation process is performed using information about the subject that does not originally exist in the pixel 114, and a false color occurs.

Therefore, a process of suppressing saturation of a portion where a false color is generated and making the generated false color inconspicuous is performed.

FIG. 11 shows a conventional block diagram of a processing device for suppressing false color. The electrical signals of the RGB primary color components output from the CCD are input to the color difference matrix circuit 93. In the color difference matrix circuit 93, the color space having RGB primary color components has a luminance component (Y) and a color difference component (C
r, Cb).

Luminance component Y of the converted electric signal
Is input to the contour enhancement filter 95. In the edge enhancement filter 95, the luminance component Y is HPF (high-pass
After passing through the filter), the amplifier, and the base clip, high-frequency components above a predetermined value are detected.

The detected high frequency component is added to the luminance component Y that has passed through the LPF. As a result, the luminance component Y
Will be subjected to contour enhancement processing.

On the other hand, for the detected high frequency part,
As described above, there is a high probability that false color will occur. Therefore,
Information of the region (pixel) detected as a high frequency component by the contour enhancement filter 95 is input to the color difference suppression circuit 97, and color difference suppression control is performed on the color difference components Cr and Cb of the pixel. In this way, false colors caused by Bayer image formation are made inconspicuous.

[0017]

However, in the false color suppression control described above, pixels to be suppressed are determined from the output of the contour emphasis filter 95 that emphasizes the edges of the object, so that all high frequency components of the object are suppressed. It will be. In other words, the color difference is suppressed in all parts where the high frequency component (edge component) is strong.

In this case, although it is possible to prevent the occurrence of false color at the edge portion, there is a problem that a phenomenon called "color loss" occurs. "Color loss" is a phenomenon (color loss) in which a color is lost in an edge portion where the color difference is suppressed by the above-described conventional technique.

Therefore, in view of the above problems, an object of the present invention is to provide an image processing technique capable of effectively suppressing the occurrence of false color while reducing the occurrence of color loss.

[0020]

In order to achieve the above object, the invention of claim 1 is an image processing apparatus for performing a predetermined image processing on an image signal input from an image pickup element, wherein the image pickup element is 3 Of the primary color components, three types of photoelectric conversion elements that respectively output image signals of the primary color components are arranged in accordance with a predetermined arrangement rule for each pixel,
Interpolation means for performing an interpolation process on the image signal of each pixel input from the image pickup device by using the image signals of neighboring pixels, and obtaining an image signal of three primary color components for each pixel;
A color space conversion means for converting an image signal in a color space having a primary color component into an image signal in a color space having a luminance component and a color component, and the luminance component near each pixel based on the signal value of the luminance component. High-frequency component detection means for detecting the inclusion degree of high-frequency components of the signal value, low-saturation area extraction means for extracting a low-saturation area based on the signal value of the color component, and pixels in the low-saturation area With regard to (4), a suppressing unit that suppresses the signal value of the color component based on the detection result of the inclusion degree of the high frequency component is provided.

According to a second aspect of the present invention, in the image processing apparatus according to the first aspect of the present invention, the suppressing unit controls the degree of suppression of the signal value of the color component of the pixel according to the degree of inclusion of the high frequency component. It is characterized by changing.

According to a third aspect of the present invention, in the image processing apparatus according to the first or second aspect of the present invention, the low saturation area extracting means extracts the low saturation area using a low pass filter. And

According to a fourth aspect of the present invention, three types of photoelectric conversion elements for outputting image signals of any one of the three primary color components are input from an image pickup element arranged for each pixel according to a predetermined arrangement rule. An image processing method for subjecting an image signal to predetermined image processing, wherein interpolation processing is performed using an image signal of a pixel in the vicinity of an image signal of each pixel input from the image pickup device, and three primary color components are obtained for each pixel. Of the image signal of the color space having the three primary color components, the step of converting the image signal of the color space having the three primary color components into the image signal of the color space having the luminance component and the color component, and based on the signal value of the luminance component, Detecting a degree of inclusion of a high frequency component of the signal value of the luminance component in the vicinity of each pixel, extracting a low saturation area based on the signal value of the color component, and within the low saturation area For pixels, characterized in that it comprises the steps of: suppressing a signal value of the color component based on the amount of about the detection results of the high-frequency component.

According to a fifth aspect of the present invention, an image pickup device in which three types of photoelectric conversion elements for outputting image signals of any one of the three primary color components to the computer are arranged for each pixel in accordance with a predetermined arrangement rule. The image signal of the three primary color components for each pixel, which is generated by performing the interpolation process on the image signal of each pixel input from, using the image signal of the neighboring pixel, has the luminance component and the color component. Acquiring in a state of being converted into an image signal of a color space; detecting the degree of inclusion of a high frequency component of the signal value of the luminance component in the vicinity of each pixel based on the signal value of the luminance component; Extracting a low-saturation area based on the signal value of the component, and for the pixels in the low-saturation area, the color formation based on the detection result of the content of the high-frequency component. Characterized in that the step of suppressing the signal value of a program for execution.

[0025]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. First, a schematic configuration of a digital camera 31 equipped with the image processing apparatus 10 (see FIG. 1) according to the present embodiment will be described.

{1. Schematic configuration of digital camera} FIG.
FIG. 3 is a front view of the digital camera 31. The digital camera 31 is composed of a box-shaped camera body 32 and a rectangular parallelepiped image pickup unit 33. On the front side of the imaging unit 33,
A zoom lens 34 which is an imaging lens is provided, and an optical finder 35 is provided.

The grip portion 3 is provided at one end of the camera body 32.
6 and the built-in flash 37 in the upper center of the front side
Is provided, and a shutter button 38 is provided on the upper surface side.

Although not shown, a liquid crystal display (LCD) for performing monitor display of picked-up images, reproduction display of recorded images, and the like is provided on the back side of the camera body 32. In addition, a power switch, various operation buttons, and the like are provided on the back side of the camera body 32.

{2. Configuration of Image Processing Device} FIG. 1 is a block configuration diagram of an image processing device 10 according to the present embodiment, which is mounted inside a digital camera 31.

<2-1. Bayer CCD and Interpolation Processing> CCD 1 is a single-plate CCD equipped with a Bayer color filter array. A large number of photodiodes that control photoelectric conversion are two-dimensionally arranged in a matrix in the CCD 1 to correspond to each pixel, and each pixel is covered with one of the color filters of the primary color components (R, G, B). It is being appreciated.

In the present embodiment, as in the color filter array 100 shown in FIG. 5, G that contributes to the luminance signal in a checkered pattern is formed.
No. 1 filter is arranged, and R and B filters are further arranged in a checkered pattern on the remaining part. In this way, the charge corresponding to the primary color component of R, G, or B is accumulated in each pixel.

As the Bayer type color filter array, there are several types such as a type in which Gs are arranged in the vertical direction in addition to the type shown in FIG. 9, but the image processing apparatus of the present embodiment. The type of color filter array applicable to the above is not particularly limited. However, in the interpolation processing described later, it is necessary to perform processing according to the type of color filter array.

The charges accumulated in the CCD 1 are sequentially taken out line by line and output as a one-dimensional electric signal. Further, an electric signal of each pixel is converted into a 12-bit digital electric signal in an A / D conversion circuit (not shown), and then input to a WB (white balance) circuit 2 to perform RGB level conversion. To adjust the white balance.

After the white balance is adjusted, the electric signal of each pixel is input to the interpolation circuit 3 and the interpolation process is performed for each pixel.

That is, each pixel has one of R, G, and B.
Since it only has information about one primary color component, it performs interpolation processing to infer information about the other primary color components based on the values of surrounding pixels. By this interpolation processing, 12-bit information for each of R, G, and B is given to each pixel. That is, the image signals (R, G, B) of the three primary color components for each pixel are generated.

After the interpolation processing is performed, the electric signals (R, G, B) of each pixel are output in the linear matrix circuit 4.
After being subjected to a predetermined correction process, it is input to the γ correction circuit 5. In the γ correction circuit 5, the electrical signals (R, G, B) are converted by the γ correction table (RGB gamma LUT) 51 into
Correction is performed according to the reproduction characteristics of the display.
Further, in the γ correction circuit 5, the 12-bit electric signal is compressed to 8 bits.

<2-2. Color space conversion processing> The electrical signals (R, G, B) compressed to 8 bits are then input to the color difference matrix circuit 6. The color difference matrix circuit 6 is
The conversion matrix includes a Y matrix 61, a Cr matrix 62, and a Cb matrix 63.
The color space having the primary color components (R, G, B) is converted into the color space having the luminance component (Y) and the color difference components (Cr, Cb).

The following expression 1 expresses the Y matrix 61, the Cr matrix 62, and the Cb matrix 63 by a color difference matrix of 3 rows and 3 columns. Therefore, it can be expressed that the RGB color space is converted into a color space having a luminance component and a color difference component by the color difference matrix.

[0039]

[Equation 1]

<2-3. Contour enhancement filter processing> The luminance component Y output from the color difference matrix circuit 6 (in the following description, an electric signal of the luminance component is appropriately expressed as a luminance signal Y) is next subjected to a contour (edge) enhancement filter. It is input to the circuit 7. In the contour emphasis filter circuit 7, the luminance signal Y is branched into two. Among the branched luminance signals Y, one passes through an HPF (high-pass filter) 72 and one passes through an LPF (low-pass filter) 71.

From the luminance signal Y that has passed through the HPF 72,
A high-frequency component is detected, and the detected high-frequency component is input to and amplified by the amplifier 73, and then only the high-frequency component having a predetermined value or more is detected and output by the base clip 74.

Then, in the adder 75, the LPF 71
By adding a high frequency component of a predetermined value or more output from the base clip 74 to the brightness signal Y that has passed through, the brightness signal Y is output after the contour is emphasized.

The luminance signal Y is stored in a buffer (not shown), and the HPF process of the target pixel is performed based on the signal Y stored in this buffer. For example, in the present embodiment, the HPF 72 is represented in a matrix form of 5 rows and 5 columns, and an edge is formed using the luminance signal Y of the target pixel and the luminance signal Y of 24 pixels near the target pixel. The extraction process will be performed. Therefore,
It is necessary that at least these 25 pixel luminance signals Y are accumulated in the buffer. In order to meet such a need, for example, the pixel values of a plurality of pixels included in horizontal lines to which the target pixel (target pixel) belongs are centered on the upper and lower two lines (five in total). It suffices to provide a buffer having a capacity for performing

The same applies to the processing using the LPF 71. The color difference signals Cr and Cb are stored in a buffer having a predetermined capacity according to the size of each filter, and then the predetermined LPF processing is performed. To be done. Further, the same applies to the processing using the LPF 82 described later.

<2-4. High Frequency Component Detection Processing> Next, the high frequency component detection processing will be described.

The output value from the HPF 72 of the contour emphasis filter circuit 7 represents the detection result of how much the high frequency component is included, that is, the inclusion degree of the high frequency component (in other words, the detection level of the edge component). Further, since this output value is obtained for each pixel, information regarding the degree of inclusion of the high frequency component at each pixel position is obtained. And the edge chroma killer setting LUT
By appropriately correcting this output value with reference to a (Lookup Table) 86, the degree of suppression of the electric signals Cr and Cb of both color difference components (hereinafter also referred to as color difference signals) is obtained. This makes it possible to obtain the degree of suppression of the color difference signals Cr and Cb at each pixel position according to the degree of inclusion of the high frequency component at each pixel position.

Specifically, the output value from the HPF 72 is
After being corrected with reference to the edge chroma killer setting LUT 86, it is input to the masking circuit 83 as a value representing the degree of suppression. Thereafter, the suppression degree is further corrected in the masking circuit 83 in consideration of the saturation information described later, and then the color difference signals Cr and Cb are suppressed by the multipliers 84 and 85 according to the corrected suppression degree. Done.

FIG. 3 shows the setting LUT of the edge chroma killer.
It is the figure which graphically showed the setting content of 86 as the suppression degree curve LC. In the figure, the horizontal axis represents the content of high-frequency components, and the vertical axis represents the degree of suppression. That is, the suppression control is performed such that the higher the content of the high frequency component is, the higher the suppression is applied.

For example, when the content of the high frequency component in the pixel n is An, the color difference component in the pixel n is suppressed by about 20%. When the high frequency component content of the pixel m is Am, the color difference component of the pixel m is suppressed by about 60%. As described above, the pixel m having a relatively high content of the high frequency component is suppressed more than the pixel n having a relatively low content of the high frequency component. According to this, the degree of suppression of the signal value of the color component of the pixel (that is, the degree of suppression control) is changed according to the degree of inclusion of the high frequency component, so that the output image does not have unnaturalness. Suppression control becomes possible.

However, in order to remove the noise component, it is preferable to set the suppression degree to a low value so that the suppression degree does not become excessively large when the content of the high frequency component is small. For example, as shown in FIG. 3, when the content of high frequency components is smaller than An, the suppression degree curve L
It is preferable that C exists below the virtual direct proportional line LL.

<2-5. Low Saturation Area Extraction Process> Next, the low saturation area extraction process will be described.

In this low saturation area extraction processing, based on the color difference signals (electric signals of color difference components) Cr and Cb,
Extract low saturation areas. Here, the “low-saturation area” means a portion where the saturation is lower than a predetermined level in a region near each pixel position.

Specifically, first, the absolute value adding circuit 81 adds the absolute value of the color difference signal Cr (= RY) and the absolute value of the color difference signal Cb (= BY). Here, R is an image signal of a red component, B is an image signal of a blue component, Y
Represents a luminance signal.

Then, a high-frequency component is removed by performing processing using the LPF 82 having a predetermined size (for example, a size of 5 pixels × 5 pixels) with each pixel as the center position. By using the LPF 82, it is possible to remove a high frequency component associated with a hue transition or the like from the output value of the absolute value adding circuit 81, so that a highly saturated portion can be extracted more accurately.

The output value from the LPF 82 is input to the masking circuit 83. In this masking circuit 83, the output value from the LPF 82 is NOT processed for each bit.
Mask processing is performed using a value that is inverted by performing calculation (hereinafter, also referred to as an inverted value). Where LP
The inversion value with respect to the output value from F82 has a magnitude relationship opposite to that of the original output value, and thus has a larger value as the saturation is lower. That is, this inversion value is an index value representing the degree of low saturation. Therefore, it is possible to extract the pixel having the inversion value larger than the predetermined value as the pixel in the low saturation area. Also, as described above, L
By using the PF 82, it is possible to remove the high frequency component associated with the hue transition or the like from the output value of the absolute value adding circuit 81, so that the low saturation area can be extracted more accurately.

<2-6. Suppression processing of color difference signal> Next,
The color difference signal suppression processing will be described.

The chroma killer circuit 80 calculates the degree of suppression according to the content of the high frequency component of the luminance signal Y and the saturation calculated based on the color difference signals Cr and Cb, and the color difference according to the calculated degree of suppression. Suppression processing is performed on the signals Cr and Cb.

Here, the case where the suppression processing for the color difference signals Cr and Cb is performed on the pixels in the low saturation area in accordance with the degree of inclusion of the high frequency component will be exemplified. As a result, a relatively large suppression process can be performed on pixels having a high degree of high frequency component content and low saturation.

Specifically, the output value from the LPF 82 and the output value from the HPF 72 corrected by using the edge chroma killer setting LUT 86 are the masking circuit 8
Input to 3. In this masking circuit 83, the LPF
A mask process is performed using a value (inversion value) obtained by inverting the output value from 82 by performing a NOT operation on each bit. Specifically, this inversion value is set to a predetermined threshold value LV.
Then, the binarized value is used as a mask, and the masked operation is performed on the output value from the HPF 72 that has been subjected to the conversion process using the edge chroma killer setting LUT.

Here, as described above, the inversion value with respect to the output value from the LPF 82 is a value that becomes larger as the saturation becomes lower, that is, an index value indicating the degree of low saturation. The output value from the HPF 72 corrected by using the edge chroma killer setting LUT 86 is a value that increases as the content of the high frequency component increases, that is, an index value indicating the content of the high frequency component. Then, in consideration of these values, the suppression degree is obtained as a larger value for a pixel having low saturation as the inclusion degree of the high frequency component is larger.

Then, the obtained suppression degree is multiplied by the multiplier 84,
The color difference signals Cr and Cb, which have been subjected to the suppression processing, are obtained by inputting them to 85 and multiplying them by the color difference signals Cr and Cb.

With the above processing, the image processing apparatus according to the present embodiment outputs the luminance signal Y subjected to the edge enhancement processing and the color difference signals Cr and Cb subjected to the suppression control. . The output luminance signal Y and color difference signals Cr and Cb are recorded in the buffer 9 and then, for example, J
Processing according to a compression standard such as PEG is performed, and the image file is stored in a memory card or the like.

As described above, in the present embodiment, since the suppression control is performed only for the pixels extracted as the pixels in the area with low saturation, the suppression control is performed for all edge components. The problem caused by performing the suppression control can be solved.

Conventionally, all the edge components are suppressed, so that the effect of suppressing the generated false color can be obtained, but on the other hand, the color is suppressed even for the edge portion having a large saturation, and it is not possible to suppress. There was a problem that natural color loss occurs. According to the present embodiment, since the color is suppressed only in the low-saturation edge portion, it is possible to obtain a natural image in which the color loss is reduced by eliminating the color loss in the high-saturation portion. become.

{3. Color Difference Suppression Control Processing} FIG. 4 is a flowchart of the color difference suppression control processing described above. Further, FIGS. 5 to 8 are diagrams showing an example of each processed signal in one predetermined horizontal line in the image. In the graphs of FIGS. 5 to 8, the horizontal axis represents the horizontal position x in the image,
The vertical axis represents the magnitude of each signal. The color difference suppression control processing will be described below with reference to these drawings.

First, in step S1 (FIG. 4), C
The RGB image signals output from the CD 1 are input. This image signal is one of the RGB primary color components in each pixel.
It is an image signal of any one of the primary color components.

In the next step S2, interpolation processing is performed on the input pixel using neighboring pixels.
By this step, image signals of three primary color components are obtained in each pixel (step S2).

In step S3, the image signals of the RGB three primary color components are the luminance signal Y and the color difference signals Cr and Cb.
Is converted to.

In FIG. 5, each signal Y, Cr, Cb is shown. More specifically, FIG. 5A shows the luminance signal Y, FIG. 5B shows the color difference signal Cr (= RY), and FIG. 5C shows the color difference signal Cb (= B). -Y) is represented.

In the leftmost section DX1, a relatively large number of pixels in which both the red component and the blue component have strong colors, that is, purple, are distributed.

In the rightmost section DX3,
A portion where the color difference signals Cr and Cb are almost zero and the luminance signal is high, that is, white pixels are relatively distributed. Further, in the color difference signals Cr and Cb in the section DX3, there are pulse-like portions. This pulsed part is
This is a portion representing the false color generated by the above-described interpolation processing (step S2). That is, in the section DX3, the pulse-shaped pseudo-colored portion exists in the white-colored region as a whole.

Further, also in the section DX2, a pulse-like false color portion similarly occurs. However, section DX2
Differs from the section DX3 in that it has a larger saturation than the section DX3.

According to the suppression processing described below, the color loss is prevented by not changing the false color portion in the section DX2, and the pulse-like false color portion in the low saturation area of the section DX3 is reduced. Will be possible.

In step S4, the inclusion degree of the high frequency component in the vicinity of the target pixel is detected based on the luminance signal Y obtained in step S3.

FIG. 6 is a diagram showing the luminance signal Y and the signal YP detected as a value indicating the content of the high frequency component. More specifically, FIG. 6B shows a signal YP obtained by further modifying the luminance signal Y processed by the HPF 72 with reference to the edge chroma killer setting LUT 86.
5A shows the same luminance signal Y as that shown in FIG.

As shown in FIG. 6B, this HPF 72
Functions as a differential filter, and the edge part is extracted. Further, this edge portion exists at the same position as the position of the false color generating portion. This is because the above "false color" is caused by the existence of the edge portion.

Then, in step S5, the saturation in the vicinity of the target pixel is detected based on the color difference signals Cr and Cb obtained in step S3. The operation of extracting the low saturation area will be described with reference to FIG. 7.

First, FIG. 7C shows the signal SG1 as the addition result of the absolute value adding circuit 81. This signal SG1 is specifically a signal obtained by adding the absolute value of the color difference signal Cr and the absolute value of the color difference signal Cb. Note that in FIGS. 7A and 7B, the same signal C as in FIGS. 5B and 5C, respectively is used.
r and Cb are represented.

Further, FIG. 7D shows a signal SG2 obtained by further processing the signal SG1 by the LPF 82.
It is a figure showing. This gives
It is possible to obtain the signal SG2 in which the “smoothing process” has been performed and the high frequency component has been removed.

Further, FIG. 7 (e) shows that this signal SG2 is x
It is a figure showing the signal SG3 turned up about the axis and raised overall by a predetermined value (maximum value). This signal SG3
Is a signal corresponding to the inverted value of the signal SG2 and having a larger value as the saturation is lower, that is, a signal indicating the degree of low saturation.

Further, this signal SG3 is a predetermined threshold value LV.
The signal SG4 modified according to whether or not the above is generated. FIG. 7F is a diagram showing this signal SG4. The value of the signal SG4 becomes “1” when the original signal SG3 is equal to or larger than the threshold LV, and becomes the value “0” when the original signal SG3 is smaller than the threshold LV. The area where the signal SG4 has the value "1" (section DX4 in the figure) is an area with relatively low saturation, that is, a "low saturation area".
Here, for simplification of description, it is assumed that the section DX4, which is the low saturation area, is detected as a section that is substantially the same as the section DX3.

Then, in step S6, the color difference signals Cr and Cb are suppressed for each target pixel in accordance with the degree of inclusion of the high frequency component near the target pixel and the degree of saturation near the target pixel. This suppression operation will be described with reference to FIG.

Specifically, as shown in FIG. 8C, the signal S is obtained by masking the signal YP with the signal SG4.
Generate G5. This signal SG5 is zero outside the low-saturation area, and becomes a larger value in the low-saturation area as the content of high-frequency components at each position increases. In FIG. 8A, the signal SG
4 is represented, and the signal YP is represented in FIG.

Then, the corrected color difference signal Cr is generated by multiplying the generated signal SG5 by the color difference signal Cr. Similarly, the corrected color difference signal Cb is generated by multiplying the signal SG5 and the color difference signal Cb. In addition,
FIG. 8D shows the corrected color difference signal Cr, and FIG. 8E shows the corrected color difference signal Cb. The color difference signals Cr and Cb before correction are shown in FIGS. 7A and 7B, respectively.

Then, in step S7, for each pixel of interest, the luminance signal Y and the corrected color difference signals Cr, Cb
And are output.

By performing the suppression processing using the above-mentioned signal SG5, it is possible to perform the suppression processing only for the pixels in the low saturation area, and the suppression degree for each pixel is determined for that pixel. The larger the degree of inclusion of the high frequency component, the larger the value. As a result, the occurrence of false colors can be effectively suppressed.

In the example of FIGS. 5 to 8, the false color in the white area (low chroma area) in the section DX4 is removed. Since the false color has a characteristic that it is particularly noticeable in the low saturation area, it is possible to obtain a more natural image by removing the false color from the low saturation area.

On the other hand, in the high saturation area in the section DX2,
False color remains due to lack of suppression (or suppression of suppression). In the high-saturation area, false colors are less noticeable, while color loss is more noticeable. Therefore, by utilizing such characteristics, by not removing the false color (or suppressing the degree of removal) in the high saturation area, it is possible to prevent "color loss" from occurring in the high saturation area and It is possible to obtain a clear image. That is, it is possible to effectively suppress the occurrence of false color.

The image processing apparatus according to this embodiment described above is a CCD equipped with various color filter arrays such as a Bayer system such as a digital camera and a digital video.
It can be applied to any device that inputs an image signal from.

{4. Modification} <Use of Saturation> In the above-described embodiment, the low saturation area is discriminated based on the binarized signal SG4, but more precise suppression processing is performed using the signal SG3 before being binarized. May be performed. That is, the suppression processing may be performed according to the degree of saturation. More specifically, the suppression processing may be performed by using a larger suppression degree as the saturation is lower.

More specifically, the signal SG3 representing the degree of low saturation is used.
Is multiplied by the signal YP representing the degree of inclusion of high frequency components to generate a suppression signal, and the generated signal is further multiplied by the color difference signals Cr and Cb. This makes it possible to generate the color difference signals Cr and Cb that have been subjected to the suppression processing. Therefore, the suppression processing can be performed using a higher suppression degree as the content of the high frequency component is higher, and the suppression processing can be performed using a higher suppression degree as the saturation is lower.

In this case, since more precise suppression processing can be performed according to the degree of saturation, it is possible to more accurately suppress the occurrence of false color. In addition,
In this case, determining the degree of suppression using the signal SG3 indicating the degree of low saturation determines whether or not each pixel is a pixel within the low saturation area (in other words, the low saturation area (Extracting and subjecting the pixels in the low saturation area to the suppression processing of the color difference signal).

<Color Signals Other than Color Difference Signals> In the above embodiment, the signals representing the color components (color signals) are:
The case where the color difference signal is suppressed is illustrated, but the present invention is not limited to this. For example, the suppression processing may be performed on a color signal other than the color difference signal (for example, a signal representing only saturation) using the same idea as above.

<Other Type CCD> Furthermore, in the above embodiment, the case where the Bayer type CCD in which the respective color component pixels are arranged in a checkered pattern is used is illustrated, but the present invention is not limited to this.

For example, for an image picked up by a CCD having a stripe arrangement in which lines in which R components and B components are alternately arranged and lines in which G components are arranged in a row are alternately arranged, The invention may be applied.

Further, instead of the primary color CCD having each color of R (red), G (green) and B (blue), a complementary color CCD having each color of C (cyan), M (magenta) and Y (yellow) is used. You may use it.

As described above, the information (image signal) of the neighboring pixels
In the case of adopting the method of performing the interpolation process by utilizing the, it is possible to effectively suppress the occurrence of false color.

<Realization Using Computer> In the above embodiment, the hardware circuit in the digital camera 31 realizes the predetermined image processing, but the present invention is not limited to this. It is also possible to realize the above image processing by executing a predetermined software program (also simply referred to as “program”).

Such a program is recorded on a computer-readable recording medium (flexible disk,
CD-ROM, DVD (Digital Versatile Disk), etc.)
It may be provided in a state recorded in. The user can cause the computer to function as the image processing apparatus having the above-described image processing function by causing the computer to read such a recording medium. Also,
Such a program may be provided not only as a recording medium but also via a network. In this case, the program loaded by the computer via the network may be executed on the computer.

Further, in this case, the digital camera 31 generates the image signals of the three primary color components for each pixel by the interpolation process using the image signals of the pixels in the vicinity of each pixel, and the generated three primary color components are generated. The image signal (image data) may be read by a predetermined computer. After that, the computer performs a color space conversion operation on this image signal (image data) to convert the image signal into an image signal in a color space having a luminance component and a color component, and to perform 3 for each pixel. The image signal of the primary color component can be acquired. Alternatively, the digital camera 31 side may convert in advance the image signals of the three primary color components for each pixel into an image signal of a color space having a luminance component and a color component, and read the converted image signal into a computer. You may In this way, the computer acquires the image signals of the three primary color components for each pixel in the state of being converted into the image signal of the color space having the luminance component and the color component, and then performs the operation of step S4 and thereafter. You can do this.

<Line Segment Extraction> Further, in the above embodiment, the HPF 72 is used to detect the content of the high frequency component, but the present invention is not limited to this, and the line segment component is detected as the high frequency component. Good. Specifically, a line segment component can be extracted from the luminance signal Y by using the Sobel line segment extraction filter for the luminance signal Y branched in the contour emphasis filter circuit 7. Sobel line segment extraction filter A
1 to A4 are shown in Formula 2.

[0102]

[Equation 2]

In Equation 2, the filter A1 is a spatial filter that detects the boundary of the oblique line that rises to the right, the filter A2 is a spatial filter that detects the boundary of the vertical line, and the filter A3 is that of the oblique line that descends to the right. The filter A4 is a spatial filter that detects boundaries, and the filter A4 is a spatial filter that detects boundaries of horizontal lines. Each line segment extraction filter A1 to A4 is expressed in a matrix form of 3 rows and 3 columns, and line segment extraction processing is performed using the luminance signal Y of the target pixel and the luminance signal Y of pixels in the vicinity of the target pixel. Be seen.

Specifically, taking the case where the line segment extraction filter A1 is applied to the pixel of interest as an example, the values of the luminance signal Y of the pixel of interest and the eight neighboring pixels are used as the respective components of the filter A1 as coefficients. Multiply to obtain the sum of these calculated values. Then, when the absolute value of the total value becomes larger than the predetermined value, the pixel of interest is judged by the filter A1 as a line segment component. And
The calculated value (total value) represents the detection level of the line segment in the pixel of interest. That is, each line segment filter A1
The calculated value (total value) using any of the spatial filters A4 to A4 means that the larger the value, the greater the degree of detection as a line segment.

The line segment component can be detected as a high frequency component by using such a line segment detection filter. According to this, it is possible to perform the suppression control only on the edge region that can be recognized as a line segment, and not perform the suppression control on the high frequency component region such as a dot. Therefore, it is possible to effectively prevent the occurrence of the false color while more accurately preventing the “color loss” of the dot portion.

[0106]

As described above, the claims 1 to 5 are as follows.
According to the invention described in (1), since the signal value of the color component is suppressed based on the detection result of the inclusion degree of the high frequency component for the pixel in the low saturation area, the occurrence of the false color while reducing the color loss is caused. Can be effectively suppressed.

In particular, according to the second aspect of the present invention, the degree of suppression of the signal value of the color component of the pixel is changed according to the content of the high frequency component, so that the false color is generated more accurately. It can be prevented.

According to the third aspect of the invention, the low-saturation area can be more accurately extracted by using the low-pass filter.

[Brief description of drawings]

FIG. 1 is a block configuration diagram of an image processing apparatus 10 according to an embodiment of the present invention.

FIG. 2 is a front view of a digital camera 31.

FIG. 3 is a diagram showing a graph of setting contents of an edge chroma killer setting LUT 86.

FIG. 4 is a flowchart showing color difference suppression control processing.

FIG. 5 is a diagram showing an example of each processed signal Y, Cr, Cb before color difference suppression control processing in a predetermined one horizontal line in an image.

FIG. 6 is a diagram showing a luminance signal Y and a signal YP detected as a value indicating the degree of inclusion of high-frequency components.

FIG. 7 is a diagram illustrating an extraction operation of a low saturation area.

FIG. 8 is a diagram illustrating a suppression operation.

FIG. 9 is a diagram showing an example of a color filter array in a Bayer CCD.

FIG. 10 is a diagram showing a state in which an optical image of a subject having a high frequency component is formed on a Bayer CCD.

FIG. 11 is a block diagram of a conventional image processing apparatus.

[Explanation of symbols]

10 Image processing device 100 color filter array 31 digital camera 80 Chromakiller circuit 81 Absolute value addition circuit 93 Color difference matrix circuit 95 Edge enhancement filter 97 Color difference suppression circuit Cr, Cb color difference signal Y luminance signal

Continued front page    F-term (reference) 5B057 BA02 CA01 CA08 CA12 CA16                       CB01 CB08 CB12 CB16 CC01                       CE17 CE18 CH08                 5C065 AA01 AA03 BB01 BB12 BB48                       CC02 CC03 DD01 EE06 GG02                       GG13 GG23                 5C066 AA01 CA09 CA17 EA13 EC05                       GA01 GA02 GB03 JA01 KC02                       KC03 KC07 KE03 KM02                 5C077 LL19 MM03 MP08 PP01 PP31                       PP32 PP34 PP37 PP48 PP49                       PQ08 PQ12 RR19 TT09                 5C079 HB01 HB04 HB11 LA14 LA28                       LB01 NA03

Claims (5)

[Claims] 1. An image processing apparatus for performing predetermined image processing on an image signal input from an image pickup device, wherein the image pickup device outputs an image signal of any one of the three primary color components. A photoelectric conversion element of a kind is arranged according to a predetermined arrangement rule for each pixel, and an interpolation process is performed using image signals of neighboring pixels with respect to image signals of each pixel input from the image pickup element, Interpolation means for obtaining an image signal of three primary color components for each pixel; color space conversion means for converting an image signal of a color space having three primary color components into an image signal of a color space having a luminance component and a color component; A high-frequency component detection unit that detects the degree of inclusion of a high-frequency component in the signal value of the luminance component near each pixel based on the signal value of the luminance component, and a low-saturation area is extracted based on the signal value of the color component. Low-saturation area extraction means, and for pixels in the low-saturation area, suppression means for suppressing the signal value of the color component based on the detection result of the inclusion degree of the high-frequency component, A characteristic image processing device. 2. The image processing apparatus according to claim 1, wherein the suppressing unit is configured to contain the high frequency component in accordance with a degree of inclusion thereof.
An image processing apparatus, wherein the degree of suppression of a signal value of a color component of the pixel is changed. 3. The image processing apparatus according to claim 1, wherein the low saturation area extraction unit extracts a low saturation area by using a low pass filter. 4. A predetermined type of image signal input from an image pickup device in which three types of photoelectric conversion elements that output image signals of any one of the three primary color components are arranged according to a predetermined arrangement rule for each pixel. An image processing method for performing image processing, comprising: interpolating an image signal of each pixel input from the image sensor using image signals of neighboring pixels to obtain an image signal of three primary color components for each pixel. And a step of converting an image signal of a color space having three primary color components into an image signal of a color space having a luminance component and a color component, and the step of converting the image signal in the vicinity of each pixel based on the signal value of the luminance component. Detecting a degree of inclusion of a high frequency component of a signal value of a luminance component, extracting a low saturation area based on the signal value of the color component, and regarding pixels in the low saturation area An image processing method characterized by comprising the steps of: suppressing a signal value of the color component based on the amount of about the detection results of the high-frequency component. 5. A pixel input to a computer from an image pickup device in which three types of photoelectric conversion elements for outputting image signals of any one of the three primary color components are arranged in accordance with a predetermined arrangement rule for each pixel. The image signal of the three primary color components for each pixel, which is generated by performing an interpolation process on the image signal of the pixel using the image signals of neighboring pixels, is an image signal of a color space having a luminance component and a color component. And a step of detecting the degree of inclusion of a high frequency component of the signal value of the luminance component in the vicinity of each pixel, based on the signal value of the luminance component, in the signal value of the color component. And extracting a low-saturation area based on the detection result of the content of the high-frequency component for pixels in the low-saturation area. Program for the steps of the execution.