JPH11177995A - Image processing device and method and camera - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device and method which surely determines a vertical or horizontal correlation that is more intensive than the other and which can reproduce an image with no breaking. SOLUTION: This device processes output signals of an ACCD area sensor which has a color filter of a primary color RGB Bayer array on its light receiving surface. meanwhile, an fs/2 detection part 16 detects the presence of a frequency component that is multiplied by 1/2 of th especial sampling frequency (fs) of an input image. At the same time, it's determined whether the detected frequency component is achromatic or not. When an fs/2 frequency component is detected, the R/G/B signals whose fs/2 frequency components are eliminated by a chromatic color processing circuit 102 are primarily outputted in a chromatic mode. Then the R/G/B signals which have undergone a horizontal interpolation 103 or vertical interpolation 104 are primarily outputted via an achromatic color processing circuit 101 in an achromatic mode.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, an image processing method, and a camera, and more particularly, to an image processing apparatus for processing an output signal of a solid-state imaging device having a color filter on a light receiving surface, and a processing method thereof. The present invention relates to a camera using these.

[0002]

2. Description of the Related Art Conventionally, to remove a half of the spatial sampling frequency fs for the purpose of removing aliasing noise before image information enters a solid-state imaging device,
Band limitation was performed using an optical LPF (Low Pass Filter). As described above, by performing the band limitation using the optical LPF, the output signal of the solid-state imaging device does not include a frequency component near 1/2 of the spatial sampling frequency fs. The sense of image will be impaired. On the other hand, in order not to impair the sense of resolution of the captured image, the optical L
Instead of using PF, the spatial sampling frequency fs
The response at a point higher than half the frequency of?

[0003]

However, fs /
If the response at a point higher than the frequency 2 is set to 0, not only a false signal is generated but also the color
When signal processing by correlation detection is performed in a solid-state imaging device having a color filter of a primary color Bayer array of (red), G (green), and B (blue), fs / fs If there is a spatial frequency of 2, it becomes impossible to determine whether the vertical correlation is strong or the horizontal correlation is strong, which causes image quality deterioration.

That is, in FIG. 16, a signal of a horizontal fs / 2 image (b) or a vertical fs / 2 signal is applied to a pixel array of a solid-state image sensor corresponding to a primary color RGB Bayer array (a).
When the signal of the image (c) of s / 2 is input, the output signal (d) in the case where attention is paid only to the pixel of G becomes the horizontal fs
The same holds true for / 2 (b) and vertical fs / 2 (c). Therefore, only by looking at the G output signal, the vertical correlation is strong (horizontal fs / 2) or the horizontal correlation is strong (vertical fs /
2) cannot be determined.

In FIG. 16A, Gr / Gb
Indicates G pixels in the R row / G pixels in the B row, respectively.
Also, in FIG. 3D, R / G pixels are displayed in black, and attention should be paid to gray (scattered) pixels and white pixels.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to reliably determine whether a vertical correlation or a horizontal correlation is strong, and reproduce an image without failure. It is an object of the present invention to provide an image processing apparatus and an image processing method which enable the above.

[0007]

An image processing apparatus according to the present invention is an image processing apparatus for processing an output signal of a solid-state imaging device having a color filter having a predetermined color arrangement on a light receiving surface, and comprising: 1 / Sampling frequency
A frequency detection circuit that detects the presence of the doubled frequency component, a color information detection circuit that detects information about the color, and an input based on the detection result of the color information detection circuit when the frequency detection circuit detects the frequency component. A correction processing circuit for performing a correction process on the image.

Further, an image processing method according to the present invention is an image processing method for processing an output signal of a solid-state imaging device having a color filter having a predetermined color array on a light receiving surface. In addition to detecting the presence of a frequency component that has been multiplied by a factor of two, information about a color is detected. When the presence of the frequency component is detected, a correction process is performed on the input image based on the information about the color.

In the image processing apparatus and the processing method having the above-described configuration, for example, when signal processing by correlation detection is performed, one signal for the spatial sampling frequency of the input image is used.
If there is a frequency component of / 2, it cannot be determined whether the vertical correlation is strong or the horizontal correlation is strong. Therefore, detection of a frequency component of 1/2 of the spatial sampling frequency and detection of information on a color are performed. , An achromatic color (= white) is detected. However, the detection of the achromatic color is meaningful only when a frequency component of １ ／ of the spatial sampling frequency is detected. This is because if it is an achromatic color, it can be determined whether the horizontal correlation or the vertical correlation is strong even in a region of 1/2 of the spatial sampling frequency.

Then, 1/2 of the spatial sampling frequency
If it is determined that the color component is an achromatic color, the horizontal or vertical interpolation is performed using the numerical value calculated by the achromatic color detection. Thus, in the case of an achromatic color, the characteristics can be improved to a resolution corresponding to １ ／ of the spatial sampling frequency. On the other hand, if it is determined that the color is not an achromatic color, that is, it is a chromatic color,
R, G, and B signals that do not include the frequency component 2 are output. As a result, it is possible to prevent erroneous detection of correlation detection near 1/2 of the spatial sampling frequency.

[0011]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 is a block diagram showing a basic configuration of an image processing apparatus according to the present invention. Here, the color solid-state imaging device to be processed by the image processing apparatus has, for example, R (red) G shown in FIG.
(Green) B (blue) primary color Bayer array color filter 11
On the light-receiving surface.

The color array is not limited to the primary color Bayer array, and the color filters are not limited to the RGB primary color array. Even in the case of other primary color arrays, complementary colors are used. Color array (for example, Ye / Cy
/ Mg / G) is also applicable. Also,
As the solid-state imaging device 12, a so-called all-pixel readout type CCD (Charge) that independently reads out signal charges of all pixels is used.
Coupled Device) Although a solid-state image sensor (hereinafter, referred to as a CCD area sensor) is used, the present invention is also applicable to a CCD solid-state image sensor that is not an all-pixel readout method.

R output from the CCD area sensor 12
The GB point sequential data is supplied to the detection unit 14 and the interpolation unit 15 after signal processing such as black level clamping and white balance is performed in the signal processing unit 13. The detection unit 14 detects an optimal interpolation method from the input RGB point sequential data, and sends the interpolation information to the interpolation unit 15. The interpolation unit 15 calculates R based on the interpolation information input from the detection unit 14.
Interpolation processing is performed on the GB point sequential data, and the data is output.

As shown in FIG. 3, the detecting section 14 detects an fs / 2 detecting section 16 for detecting a half frequency component of the spatial sampling frequency fs and a pixel to be interpolated (hereinafter referred to as an interpolated pixel). A total of 4 directions, ie, four directions at an angle that is an integral multiple of 90 ° with respect to the top, bottom, left and right, ie, two opposite directions in the vertical (V) direction and two opposite directions in the horizontal (H) direction
VH correlation detector 17 for detecting the degree of correlation in the direction, and the degree of correlation in the four directions forming an oblique direction of the upper right, upper left, lower left, and lower right with respect to the interpolated pixel, that is, forming an angle of 45 ° with the above four directions And an oblique correlation detection unit 18 for detecting the correlation.

In this embodiment, in addition to the four directions (up, down, left, and right),
Although a configuration for detecting the degree of correlation in a total of eight directions of four diagonal directions is taken as an example, a configuration for detecting the degree of correlation only in four directions of up, down, left, and right may be used. However, in the following description, the case of eight directions will be described as an example.

On the other hand, as shown in FIG.
An R interpolation unit 19 that performs an interpolation process on R pixel information based on interpolation information given from the detection unit 14, a G interpolation unit 20 that performs an interpolation process on G pixel information, and a B pixel information A B interpolation unit 21 that performs interpolation processing on fs / fs / based on detection information of the detection unit 14 and performs a correction processing that suppresses generation of noise on a hatch at fs / 2.
And a two-correction processing unit 22.

The fs / 2 detector 16, the VH correlation detector 17, and the oblique correlation detector 18 in the detector 14 will be described below.
Each configuration example will be described.

FIG. 5 is a block diagram showing an example of a specific configuration of the fs / 2 detector 16 in the detector 14.
FIG. 6 shows the spatial frequency-response characteristics of the pass filter in the fs / 2 detector 16.

The fs / 2 detector 16 has a filter characteristic in which the maximum value of the response of the passing frequency is a spatial frequency which is 1/2 of the spatial sampling frequency fs of the input image.
That is, the fs / 2 pass filter 23 having the characteristic A of FIG.
A filter characteristic in which the maximum value of the response of the passing frequency is a quarter frequency of the spatial sampling frequency fs of the input image, that is, an fs / 4 pass filter 24 having the characteristic B of FIG. 6, and these pass filters 23 and 24 Which detects a frequency component of fs / 2 based on each pass frequency component of f
An s / 2 detection circuit 25 is provided.

The fs / 2 detector 16 further has a luminance correction circuit 26 for correcting the level of the luminance Y0 by multiplying the level by a predetermined correction coefficient. Then, the luminance level corrected by the luminance correction circuit 26 is supplied to the fs / 2 detection circuit 25. The fs / 2 detection circuit 25 outputs fs /
Each of the pass frequency components of the 2,2 and fs / 4 pass filters 23 and 24 is compared, and the presence or absence of the fs / 2 frequency component is detected from the comparison result.

More specifically, if the difference between the pass frequency components of the fs / 2 and fs / 4 pass filters 23 and 24 is obtained, and the difference obtained by subtracting the brightness level corrected by the brightness correction circuit 26 from the difference is positive. For example, it is assumed that a frequency component of fs / 2 exists. Here, the reason why the luminance level is subtracted from the difference between the pass frequency components of the fs / 2 and fs / 4 pass filters 23 and 24 is to suppress the threshold from fluctuating due to the luminance. Thus, the fs / 2 detection system 27 is configured. FIG. 7 is a conceptual diagram of the detection of the luminance Y0.

The fs / 2 detector 16 further includes information on color, specifically, an achromatic color detection system 2 for detecting whether or not the subject is an achromatic color (= white) in a 5 × 5 pixel area, for example.
8 are provided. This achromatic color detection system 28 has an R / G
A horizontal color difference level calculating circuit 29 for calculating the horizontal color difference level of the / B signal, a vertical color difference level calculating circuit 30 for calculating the vertical color difference level, and based on the respective outputs of these color difference level calculating circuits 29, 30. fs / 2 correlation detection, and the horizontal / vertical interpolation coefficient WhHGain / Wh
An achromatic color is detected based on the outputs of the fs / 2 correlation detection circuit 31 that outputs VGain and the color difference level calculation circuits 29 and 30, and an interpolation coefficient Wh corresponding to the achromatic color level is detected.
It has an achromatic color detection circuit 32 that outputs Gain.

FIG. 8 shows fs / 2 in the fs / 2 detection circuit 25.
It is a conceptual diagram of s / 2 detection (A) and achromatic color detection (B). First, in the fs / 2 detection system 27 of FIG.
The horizontal fs / 2 component and the vertical fs / 2 component that have passed through the fs / 2 pass filter 23 are converted into absolute values (ABS; absolut).
e) After being converted into absolute values by the circuits 33 and 34, they are added by the adder 35. Similarly, the horizontal fs / 4 component and the vertical fs / 4 component that have passed through the fs / 4 pass filter 24 are converted into absolute values by absolute value conversion circuits 36 and 37, and then added to by an adder 38.
Is added. In the luminance correction circuit 26, a process of multiplying the level of the luminance Y0 by a predetermined correction coefficient α is performed by the multiplier 39.

The added output of the adder 35 becomes one input A of a subtractor 41 after being multiplied by a predetermined correction coefficient β in a multiplier 40. The addition output of the adder 38 is
2, a value obtained by multiplying the level of the luminance Y0 by a predetermined correction coefficient α is added, and an adder 43 adds a predetermined offset value γ
, Is the other input B of the subtractor 41. Here, the correction coefficient β and the offset value γ are parameters for determining the threshold.

The subtractor 41 subtracts one input A from the other input B. That is, in the subtractor 41, fs /
The two components (A) and the fs / 4 component (B) were compared (A-
B), the difference (A−B) is positive, that is, fs / 2 component> f
If the component is the s / 4 component, a determination result indicating that the frequency component of fs / 2 exists is issued. Then, the difference (A−B) is calculated through the lookup table (LUT) 44 using the interpolation coefficient G.
Output as 2Gain. That is, the detected f
As the frequency component of s / 2 increases, the look-up table 44 outputs a larger value of the interpolation coefficient G2Gain.

Next, in the achromatic color detection system 28 of FIG. 8B, the R / G / B signals pass through the bandpass filters 45 and 46 in the vertical and horizontal directions, and
After being converted into absolute values by 7, 48, the band-pass filter 4
Low-pass filters 49,5 in the direction orthogonal to
Pass through zero. Thereby, the color difference level Wh in the vertical direction is obtained.
V and the horizontal color difference level WhH are calculated. The vertical and horizontal color difference levels WhV and Wh
H is supplied to the subtractor 51 as its two inputs A and B, and also to the minimum value calculation circuit 52.

The subtractor 51 subtracts one input A from the other input B to obtain a vertical color difference level WhV.
(A) is compared with the horizontal color difference level WhH (B). From this comparison result, it can be determined whether the correlation in the vertical direction is strong or the correlation in the horizontal direction is strong. That is,
When WhH> WhV, it is determined that the vertical correlation is strong, that is, vertical stripes. When WhV> WhH, it is determined that the vertical correlation is strong, that is, vertical stripes. Then, the subtraction output (A-B) of the subtractor 51 is output through a look-up table 53 as an interpolation coefficient WhVGain / WhHGain according to the degree of vertical / horizontal correlation. Note that the vertical interpolation coefficient WhVGain and the horizontal interpolation coefficient WhHGain have a complementary relationship, and WhVGain + WhHGain is always constant.

The minimum value calculation circuit 52 outputs the smaller one of the vertical color difference level WhV and the horizontal color difference level WhH to the subtractor 54 as one input A.
Supply as As the other input B of the subtractor 54,
A reference value obtained by multiplying the luminance Y0 by a predetermined correction coefficient δ by a multiplier 55 and further subtracting a predetermined offset value ε by a subtractor 56 is supplied. The subtractor 54 calculates the color difference level Wh
The numerical value A, which is the smaller of V and WhH, is compared with the reference value B (AB), and the comparison result is negative, that is, the numerical value A is equal to the reference value B.
If it is smaller than this, it is determined to be achromatic (white). The subtraction output (A−B) of the subtractor 54 is passed through a look-up table 57 to obtain an interpolation coefficient WhG corresponding to the achromatic color level.
ain is output.

FIG. 9 is a block diagram showing an example of a specific configuration of the VH correlation detecting section 17 and the oblique correlation detecting section 18. As shown in FIG.

The VH correlation detector 17 calculates a correlation value indicating the degree of correlation based on the pixel information of the pixel on the right side of the interpolation pixel. A left correlation value calculation circuit 62 that calculates a correlation value based on the pixel information, an upper correlation value calculation circuit 63 that calculates a correlation value based on pixel information of a pixel above the interpolation pixel, and pixel information of a pixel below the interpolation pixel. Lower correlation value calculation circuit 64 for calculating a correlation value based on
A correlation value → interpolation gain conversion circuit 65 which converts each correlation value calculated in 1 to 64 into an interpolation gain and outputs the result.

In the VH correlation detecting section 17 having the above configuration,
The correlation value → interpolation gain conversion circuit 65 outputs horizontal / vertical interpolation gains RGain, LGane, TGain, BGain.
in and gain RGainD, L for horizontal / vertical RB interpolation
GainD, TGainD, and BGainD are output as interpolation coefficients.

The oblique correlation detector 18 calculates an upper correlation value calculation circuit 66 for calculating a correlation value based on the pixel information of the upper right pixel of the interpolation pixel, and calculates a correlation value based on the pixel information of the upper left pixel of the interpolation pixel. Upper left correlation value calculation circuit 6 for calculating a correlation value
7, a lower left correlation value calculation circuit 68 for calculating a correlation value based on pixel information of a lower left pixel of the interpolation pixel, and a right calculation device for calculating a correlation value based on pixel information of a lower right pixel of the interpolation pixel. Lower correlation value calculation circuit 69 and these correlation value calculation circuits 6
Each of the correlation values calculated in 6 to 69 is converted into an interpolation gain, and oblique interpolation gains D1Gain to D4Ga are used as interpolation coefficients.
It is composed of a correlation value → interpolation gain conversion circuit 70 that outputs in.

Further, since the four horizontal and vertical directions and the four diagonal directions are not orthogonal to each other, the VH-diagonal comparison circuit 60 compares the horizontal and vertical correlation values with the diagonal correlation values to obtain a diagonal interpolation correction gain. VHGAIN, DGai
n is calculated as an interpolation coefficient.

Next, an example of each configuration of the R / G / B interpolation unit and the fs correction processing unit 22 in the interpolation unit 15 will be described.

FIG. 10 is a block diagram showing an example of a specific configuration of the G interpolation unit 20. In FIG. 10, G data after color separation is supplied to four horizontal and vertical directions, that is, right, left, upper, and lower correlation processing circuits 71, 72, 73, and 74, respectively. , 72,
At 73 and 74, the interpolation data Gr, Gl, G in four directions
t and Gb are generated. These interpolation data Gr, Gl,
Gt and Gb are generated by passing the LPF in the direction of interpolation.

The interpolation data Gr, Gl, Gt, Gb are supplied to multipliers 75, 76, 77, 78, respectively, to obtain the interpolation coefficients determined by the VH correlation detecting section 17 shown in FIG. LGane, TGai
n and BGain, respectively. Then, the addition is performed by the adders 79 to 81, whereby the G interpolation processing is performed. The G image data after the interpolation processing is expressed by equation (1). G = Gr × RGain + Gl × LGain + Gt × TGain + Gb × BGain (1)

Similarly, in the four oblique directions, the upper right, upper left, lower left, and lower right correlation processing circuits 82, 83,
At 84 and 85, interpolation data in each direction of D1 / D2 / D3 / D4 is generated, and multipliers 86, 87, 88 and 8 are generated.
9, the oblique interpolation gains D1Gain, D2Gain, D3 determined by the oblique correlation detector 18 shown in FIG.
After each of the gains is multiplied by Gain and D4Gain, the addition is performed by adders 90 to 92 to perform an interpolation process.

The G image data Gvh obtained by the horizontal / vertical four-direction correlation detection / interpolation and the G image data Gd obtained by the diagonal four-direction correlation detection / interpolation are mixed according to the situation, as shown in equation (2) Is changed and added. The adjustment of the mixture ratio is performed by oblique interpolation correction gains VHGain and DGain calculated by the VH-oblique comparison circuit 60 shown in FIG. G = Gvh × VHGain + Gd × DGain (2)

FIG. 11 is a block diagram showing an example of a specific configuration of the R and B interpolation units 19 and 21. The configurations of the R and B interpolation units 19 and 21 are basically the same as the configuration of the G interpolation unit 20 shown in FIG. Therefore, FIG.
1, the same parts as those in FIG. 10 are denoted by the same reference numerals. However, the method of generating the interpolation data in each of the right, left, upper and lower correlation processing circuits 71 ', 72', 73 'and 74' is more complicated than in the case of G. There are two reasons for this: to perform correlation detection for RB interpolation and to add a G / 2 component to RB.

Therefore, the correlation detection for RG interpolation is separately performed for horizontal (right, left) interpolation data and vertical (up, down) interpolation data. Specifically, R pixel and B pixel
When interpolating the R signal / B signal from the pixel signal, the horizontal and vertical RG interpolation gains RGainD and LGane described above are used.
D, TGainD, and BGainD are calculated and used as interpolation coefficients dedicated to R / B. Since this is not the gist of the present invention, its detailed description is omitted here.

FIG. 12 is a block diagram showing an example of a specific configuration of the fs / 2 correction processing section 22.

In FIG. 12, the RGB signals are supplied to an achromatic color processing circuit 101 and a chromatic color processing circuit 102. The achromatic processing circuit 101 includes a horizontal interpolation circuit 103 that performs horizontal interpolation on an input image and a vertical interpolation circuit 10 that performs vertical interpolation on an input image.
4 and an interpolation coefficient WhHGa given from the fs / 2 detector 16 for each output of the interpolation circuits 103 and 104.
in, WhVGain Multipliers 105 and 106
And an adder 107 for adding the multiplied outputs of the multipliers 105 and 106.

The signal passed through the achromatic color processing circuit 101 is multiplied by the interpolation coefficient WhGain given from the fs / 2 detector 16 by the multiplier 108, and then becomes one input of the adder 109. As the other input of the adder 109, a signal passed through the chromatic color processing circuit 102 is added to the multiplier 11
0 and the interpolation coefficient WhG given from the fs / 2 detector 16
ain is multiplied by an interpolation coefficient WhGain ′. The interpolation coefficient WhGain 'is calculated, for example, by subtracting the interpolation coefficient WhGain from a constant. As shown in the characteristic diagram of FIG. 13, the chromatic color processing circuit 102 uses an fs / 2 trap circuit having a filter characteristic in which the response of a frequency component of 空間 of the spatial sampling frequency fs of the input image is 0. It is configured.

The addition output of the adder 109 is output to the multiplier 111
And the interpolation coefficient G2Ga given from the fs / 2 detector 16
After being multiplied by in, it becomes one input of the adder 112. Further, each interpolation output RIP / GIP / BIP supplied from each of the R / G / B interpolation units 19, 20, and 21 is calculated by the multiplier 113 from the interpolation coefficient G2Gain given from the fs / 2 detection unit 16. After being multiplied by the interpolation coefficient G2Gain ', it becomes the other input of the adder 112. The interpolation coefficient G2Gain 'is calculated, for example, by subtracting the interpolation coefficient G2Gain from a constant. FIG.
5 is a conceptual diagram of a correction process in the fs / 2 correction processing unit 22. As is evident from this conceptual diagram, the correction process is R
/ G / B for each signal.

In the image processing apparatus having the above structure, fs /
First, the fs / 2 detection system 27 shown in FIG.
Thus, the presence or absence of the fs / 2 frequency component is detected. That is, the G signal after color separation is passed through the fs / 2 pass filter 23 and the fs / 4 pass filter 24,
In the 2 detection circuit 25, a value obtained by multiplying the level of the luminance Y0 by a predetermined correction coefficient α is subtracted from the difference, and if the subtraction result is positive, it is assumed that a frequency component of fs / 2 exists.

The achromatic color detection system 28 detects whether the subject is achromatic in a region of, for example, 5 × 5 pixels. That is, as is clear from FIG. 8B, after passing through the band-pass filters 45 and 46 in the vertical and horizontal directions, the absolute values are obtained by the absolute value conversion circuits 47 and 48, The smaller numerical value (WhV / WhH) of the values passed through the low-pass filters 49 and 50 in the direction orthogonal to the filters 45 and 46 is the luminance Y0.
Is smaller than a value obtained by multiplying by a predetermined correction coefficient δ, it is determined to be achromatic (white).

In the fs / 2 correction processing section 22, the interpolation coefficients G2Gain and G2Gain 'given from the fs / 2 detection system 27 of the fs / 2 detection section 16 and the interpolation coefficient WhHGai given from the achromatic color detection system 28.
n, WhVGain and interpolation coefficients WhGain, Wh
Based on Gain ', the optimum correction processing is executed when the presence of the frequency component of fs / 2 is not detected, or when it is detected and the color component is achromatic or chromatic. Each of these correction processes will be described below.

First, when the presence of the frequency component of fs / 2 is not detected, there is no problem of erroneous detection of the correlation detection near fs / 2, so that ordinary RGB interpolation processing is performed. That is, since the detection component of fs / 2 is small,
The interpolation coefficient G2Gain given from the fs / 2 detection system 27 of the fs detection unit 16 is small and the interpolation coefficient G2Gai
Since n ′ is large, the mixing ratio in the adder 112 is large on each interpolation output RIP / GIP / BIP side supplied from each of the R / G / B interpolation units 19, 20, and 21.
The interpolation output RIP / GIP / BIP is mainly output.

Next, when the presence of the frequency component of fs / 2 is detected, a correction process is performed based on the determination result of the achromatic color detection system 28. First, when it is determined that the color is not an achromatic color, that is, it is a chromatic color, the interpolation coefficient WhGain provided from the achromatic color detection system 28 is small and the interpolation coefficient Wh
Since hGain 'is large, the ratio of mixing in the adder 109 is large on the chromatic color processing circuit 102 side, and the ratio of R / G / B from which the fs / 2 frequency component has been removed in the chromatic color processing circuit 102 is reduced. Each signal is mainly output. At this time, since the mixing ratio in the adder 112 is at the time of detecting the frequency component of fs / 2, the ratio of the chromatic color processing circuit 102 is large.

On the other hand, when the presence of the frequency component of fs / 2 is detected, if it is determined that the color is achromatic, the horizontal correlation is strong or the vertical correlation is low even in the region of fs / 2. Since it is possible to judge whether the image data is strong or not, in the achromatic processing circuit 101 shown in FIG. 12, the numerical value calculated by the achromatic color detection, that is, the interpolation coefficient Wh given from the achromatic color detection system 28
Horizontal interpolation circuit 10 using HGain and WhVGain
The horizontal interpolation or the vertical interpolation is performed by changing the mixing ratio of the signal having passed through No. 3 and the signal having passed through the vertical interpolation circuit 104. In this case, since an achromatic color is premised, RB is substituted by G, as is clear from the conceptual diagram of FIG.

As described above, in the case where the presence of the frequency component of fs / 2 is detected, if it is determined that the color component is a chromatic color, interpolation processing is performed by each of the R / G / B interpolation units 19, 20, and 21. Without using the interpolation output RIP / GIP / BIP,
By using the R / G / B signal from which the frequency component of fs / 2 has been removed, when performing signal processing by correlation detection, even if there is a spatial frequency of fs / 2, fs / 2 is used.
Since it is possible to prevent erroneous detection of nearby correlation detection, it is possible to reproduce an image without failure.

Further, when the fs / 2 frequency component is detected and the color is determined to be achromatic, the R / G / B signal from which the fs / 2 frequency component has been removed is not used, By performing the horizontal interpolation or the vertical interpolation using the numerical values calculated by the color detection, it is possible to increase the resolution when an achromatic color is imaged to fs / 2. As an example, VGA (Video Graphics Array)
In the case of (640 × 480) pixels, for an achromatic subject, the resolution can be improved to 480 lines corresponding to fs / 2.

FIG. 15 is a schematic configuration diagram showing an example of the camera according to the present invention. In FIG. 15, incident light from a subject is imaged on a light receiving surface (imaging surface) of the CCD area sensor 122 by an optical system including a lens 121 and the like.
On the light receiving surface of the CCD area sensor 122, a color filter 123 having a color array of, for example, a primary color Bayer array is provided. The CCD area sensor 122 is driven and controlled by a CCD drive circuit 124 such as exposure, reading out and transferring signal charges, and the like.

The output signal of the CCD area sensor 122 is supplied to an image processing device 125, where various signal processing is performed. The image processing apparatus 125 detects correlations in eight horizontal, vertical, and diagonal directions, performs adaptive interpolation processing based on the detection results, and detects an achromatic color when detecting a frequency component of fs / 2. The image processing apparatus according to the above embodiment having a configuration in which the interpolation processing is changed in the case of a chromatic color is used.

As described above, for example, in a camera using the CCD area sensor 122 having the color filters 123 of the primary color Bayer array as an image pickup device, a spatial frequency component of fs / 2 exists when performing signal processing by correlation detection. However, it is possible to prevent erroneous detection of the correlation detection near fs / 2, to enable reproduction of an image without failure, and to eliminate the need to set the response near fs / 2 to 0 using an optical LPF. Since the spatial frequency that becomes 0 can be increased, the resolution / resolution of the reproduced image can be improved.

[0056]

As described above, according to the present invention,
When processing the output signal of a solid-state imaging device having a color filter on a light receiving surface, when the presence of a frequency component of fs / 2 is detected and the presence of the frequency component is detected,
When it is determined to be chromatic, the R / G / B signals from which the fs / 2 frequency component has been removed are used, so that erroneous detection of correlation detection near fs / 2 can be prevented. Image can be reproduced, and when it is determined to be achromatic, by performing horizontal interpolation or vertical interpolation using the numerical value calculated by achromatic color detection,
It is possible to increase the resolution when capturing an achromatic color to fs / 2.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a basic configuration of an image processing apparatus according to the present invention.

FIG. 2 is a primary color Bayer array diagram of a color filter.

FIG. 3 is a block diagram illustrating an example of a configuration of a detection unit.

FIG. 4 is a block diagram illustrating an example of a configuration of an interpolation unit.

FIG. 5 is a block diagram illustrating an example of a specific configuration of an fs / 2 detection unit.

FIG. 6 is a characteristic diagram of a spatial frequency-response of a pass filter in an fs / 2 detection unit.

FIG. 7 is a conceptual diagram of detection of luminance Y0.

FIG. 8 is a conceptual diagram of fs / 2 detection;
(B) shows an achromatic color detection system, and (B) shows an achromatic color detection system.

FIG. 9 is a block diagram illustrating an example of a specific configuration of a VH correlation detection unit and an oblique correlation detection unit.

FIG. 10 is a block diagram illustrating an example of a specific configuration of a G interpolation unit.

FIG. 11 is a block diagram illustrating an example of a specific configuration of an R and B interpolation unit.

FIG. 12 is a block diagram illustrating an example of a specific configuration of an fs / 2 correction processing unit.

FIG. 13 is a characteristic diagram of the fs / 2 trap circuit.

FIG. 14 is a conceptual diagram of an fs / 2 correction process.

FIG. 15 is a schematic configuration diagram illustrating an example of a camera according to the present invention.

FIG. 16 is a diagram for explaining a problem.

[Explanation of symbols]

11, 123: color filter, 12, 122: CCD
Area sensor, 14 ... detector, 15 ... interpolator, 16 ... f
s / 2 detection section, 17 VH correlation detection section, 18 diagonal correlation detection section, 19 R interpolation section, 20 G interpolation section, 21 B interpolation section, 22 fs / 2 correction processing section, 23 fs / 2 pass filter, 24 ... fs / 4 pass filter, 25 ... fs /
2 detection circuit, 26 ... luminance correction circuit, 27 ... fs / 2 detection system, 28 ... achromatic color detection system, 29 ... horizontal color difference level calculation circuit, 30 ... vertical color difference level calculation circuit, 31 ... fs / 2 correlation detection circuit, 32 ... Achromatic color detection circuit

Claims (17)

[Claims] 1. An image processing apparatus for processing an output signal of a solid-state imaging device having a color filter having a predetermined color array on a light receiving surface, comprising a frequency component that is １ ／ times a spatial sampling frequency of an input image. A frequency detection circuit for detecting the presence of, a color information detection circuit for detecting information about a color, and when the frequency detection circuit detects the frequency component,
An image processing apparatus comprising: a correction processing circuit that performs a correction process on an input image based on a detection result of the color information detection circuit. 2. The frequency detection circuit according to claim 1, wherein a maximum value of a response of a passing frequency is a first filter whose spatial frequency is a half of a spatial sampling frequency of the input image, and a maximum value of a response of a passing frequency is By comparing a second filter having a spatial frequency of 1/4 of the spatial sampling frequency of the input image with each pass frequency component of the first and second filters, a frequency component of 1/2 of the spatial sampling frequency is obtained. The image processing apparatus according to claim 1, further comprising: a detection unit configured to detect an image. 3. The image according to claim 2, wherein said frequency detection circuit has means for subtracting an average signal level of a pixel near a pixel to be interpolated from a passing frequency component of said first filter. Processing equipment. 4. The color information detection circuit detects a change amount in a horizontal direction and a vertical direction of an input image signal centering on a pixel to be interpolated, and the change amount detection circuit detects the change amount. 2. The image processing apparatus according to claim 1, further comprising: an achromatic color detection circuit configured to detect an achromatic color level based on the amount of change in each of the horizontal direction and the vertical direction. 5. The achromatic color detection circuit detects an achromatic color level by comparing a minimum value of each of the horizontal and vertical change amounts detected by the change amount detection circuit with a predetermined reference value. 5. The method according to claim 4, wherein
The image processing apparatus according to any one of the preceding claims. 6. The correction processing circuit according to claim 1, wherein said correction processing circuit performs a horizontal interpolation or a vertical interpolation on the input image, and removes a half frequency component of a spatial sampling frequency from the input image. A first mixing step of mixing a signal having passed through the processing circuit for chromatic colors, the signal having passed through the processing circuit for achromatic colors, and the signal having passed through the processing circuit for chromatic colors at a first mixing ratio according to a detection output of the color information detection circuit; The image processing apparatus according to claim 1, further comprising a circuit. 7. A color arrangement of the color filter is R (red).
7. The image processing apparatus according to claim 6, wherein a G (green) B (blue) primary color Bayer array is used, and the achromatic processing circuit performs horizontal interpolation or vertical interpolation on the G input image. . 8. The color information detection circuit includes an achromatic color detection circuit that detects an achromatic color level based on each horizontal and vertical change amount of an input image signal around a pixel to be interpolated, The first mixing circuit changes the first mixing ratio so that the ratio of the signal passing through the achromatic color processing circuit increases when the achromatic color level detected by the achromatic color detection circuit is large. 7. The image processing apparatus according to claim 6, wherein: 9. The mixing device according to claim 1, wherein the color information detection circuit compares the amounts of change in the horizontal direction and the vertical direction detected by the change amount detection circuit, and sets a second mixture ratio according to the comparison result. A ratio setting circuit, wherein the achromatic color processing circuit includes a first signal processing system that performs horizontal interpolation on the input image, a second signal processing system that performs vertical interpolation on the input image, 7. The image according to claim 6, further comprising a second mixing circuit that mixes the signal having passed through the first signal processing system and the signal having passed through the second signal processing system at the second mixing ratio. Processing equipment. 10. The mixing ratio setting circuit according to claim 1, wherein when a change amount in the vertical direction is larger than a change amount in the horizontal direction, a ratio of the signal having passed through the first signal processing system is increased. 10. The image processing method according to claim 9, wherein when the amount of change in the horizontal direction is larger than that of the second processing, the second mixture ratio is changed so that the ratio of the signal that has passed through the second signal processing system is increased. apparatus. 11. An image processing method for processing an output signal of a solid-state imaging device having a color filter having a predetermined color array on a light receiving surface, comprising a frequency component that is １ ／ times a spatial sampling frequency of an input image. An image processing method comprising: detecting the presence of a color component; detecting information on a color; and detecting the presence of the frequency component, performing a correction process on an input image based on the information on the color. 12. In the detection of the information relating to the color, the amounts of change in the horizontal direction and the vertical direction of the input image signal are detected around the pixel to be interpolated, and the achromatic color level is detected based on the amounts of change. The method of claim 11, wherein
The image processing method described in the above. 13. The image according to claim 12, wherein the detection of the achromatic color level is performed by comparing a minimum value of the amount of change in each of the horizontal direction and the vertical direction with a predetermined reference value. Processing method. 14. In the correction processing, a signal obtained by performing horizontal interpolation or vertical interpolation on an input image and a signal obtained by removing a frequency component of 空間 of a spatial sampling frequency from the input image are obtained by: 13. The image processing method according to claim 12, wherein mixing is performed at a mixing ratio according to the achromatic color level. 15. A solid-state imaging device having a color filter having a predetermined color arrangement on a light-receiving surface, an optical system for forming an incident light from a subject on a light-receiving surface of the solid-state imaging device, and a solid-state imaging device. A camera comprising: an image processing device that processes an output signal, the image processing device comprising: a frequency detection circuit that detects the presence of a frequency component that is half the spatial sampling frequency of an input image; A color information detection circuit that detects information about the frequency component, and when the frequency detection circuit detects the frequency component,
A correction processing circuit for performing correction processing on the input image based on the detection result of the color information detection circuit. 16. The color information detection circuit detects a change amount of a horizontal direction and a vertical direction of an input image signal centering on a pixel to be interpolated, and the color information detection circuit detects the change amount of the input image signal. 16. The camera according to claim 15, further comprising: an achromatic color detection circuit that detects an achromatic color level based on the amount of change in each of the horizontal direction and the vertical direction. 17. The achromatic color processing circuit, wherein the correction processing circuit performs horizontal interpolation or vertical interpolation on an input image;
1/2 of the spatial sampling frequency for the input image
And a signal passing through the achromatic color processing circuit and a signal passing through the chromatic color processing circuit are mixed at a mixing ratio according to the detection output of the color information detection circuit. The camera according to claim 15, further comprising a mixing circuit.