JP2012239242A - Luminance signal generation device and luminance signal generation method and image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate a luminance signal from a picture signal read out of an image pickup device using color filters of a primary color Bayer array, in which luminance signal occurrence of a folding distortion in the diagonal direction component of a red or blue color subject is suppressed.SOLUTION: A first high frequency signal is generated from the picture signal of pixels in all colors of an image pickup device by limiting the band of the spatial frequency thereof. Also, a second high frequency signal is generated from the picture signal of pixels in one of red and blue colors by limiting the band of the spatial frequency thereof. Then, a high frequency signal is generated for a pixel of interest by adding the first and the second high frequency signals together while applying a weight in such a way that the closer to one of red and blue colors the pixel of interest is, the higher will be a ratio of the second high frequency signal. The high frequency signal for the pixel of interest is added to a luminance signal generated for the pixel of interest, whereby an edge enhanced luminance signal for the pixel of interest is generated.

Description

  The present invention relates to a luminance signal generation device, a luminance signal generation method, and an imaging device. In particular, the present invention relates to an apparatus and method for generating a luminance signal from a signal obtained by an image sensor using a primary color Bayer array color filter, and an imaging apparatus using them.

  In order to generate a color image using an image sensor that can detect the amount of light, such as a CCD image sensor or a CMOS image sensor, a configuration in which light transmitted through a color filter is incident on the image sensor is generally used.

  There are various types of color filters depending on the type of color used and the arrangement of colors assigned to each pixel. The color types are primary colors (red, green, blue) or complementary colors (cyan, magenta, yellow). For the color arrangement, the Bayer arrangement is widely used.

  FIG. 24 is a diagram showing one unit of the primary color Bayer array. Actually, the same arrangement is repeated according to the number of pixels of the image sensor. R is red, G1 and G2 are green, and B is blue.

  FIG. 25 shows a luminance generation that realizes an OutOfGreen method in which a luminance signal (OG signal) is generated only from a green (G) signal, among conventional methods for generating a luminance signal using the color filters of the primary color Bayer array shown in FIG. It is a figure which shows the structural example of a circuit.

  First, the zero insertion circuit 2201 is applied to the RAW signal 2200 obtained by digitizing the output (2204) of the image sensor, and values other than the G pixel are set to 0 (2205). Next, a low-pass filter (V-LPF) circuit 2202 for limiting the vertical band and a low-pass filter (H-LPF) circuit 2203 for limiting the horizontal band are applied to obtain a luminance signal. Hereinafter, a luminance signal obtained by the OutOfGreen method is referred to as an OG signal.

  Further, as another example of the conventional method for generating the luminance signal using the color filters of the primary color Bayer array shown in FIG. 24, there is an SWY method for generating the luminance signal (SWY signal) using all the RGB pixels.

FIG. 26 is a diagram illustrating a configuration example of a luminance generation circuit that realizes the SWY method.
As is clear from comparison with FIG. 25, the SWY method is a method of obtaining a luminance signal without using the zero insertion circuit 2201 in the OutOfGreen method. Hereinafter, a luminance signal obtained by the SWY method is referred to as a SWY signal.

FIG. 27 is a diagram showing resolvable spatial frequency characteristics of the OG signal and the SWY signal.
The x-axis indicates the frequency space in the horizontal (H) direction of the subject, the y-axis indicates the frequency space in the vertical (V) direction, and the spatial frequency increases as the distance from the origin increases. Since the OG signal generates a luminance signal from only the G signal, the resolution limit in the horizontal and vertical directions is equal to the Nyquist frequency (on the axis, π / 2) of the image sensor. However, since there are lines in which no pixels exist in the oblique direction, the limit resolution frequency in the oblique direction is lower than that in the horizontal and vertical directions, and as a result, the spatial frequency region 2400 on the rhombus becomes a resolvable spatial frequency.

  On the other hand, since the SWY signal is generated using all pixels, when the subject is achromatic, the square frequency 2401 as shown in the figure has a spatial frequency that can be resolved. However, for a red subject, for example, luminance signals are not output from pixels other than the R pixel, and therefore, resolution is performed only in a spatial frequency range 2402 that is half of the horizontal and vertical directions compared to an achromatic subject.

  Patent Document 1 proposes a method of replacing the oblique region 2403 of the OG signal in FIG. 27 with the SWY signal. However, since the resolution limit frequency of the SWY signal is lowered for the chromatic object, the OG signal is replaced with the SWY signal only when the oblique region 2403 is an achromatic object. Thereafter, an edge enhancement component is detected using the generated luminance signal and added to the luminance signal to generate a final luminance signal.

  On the other hand, Patent Document 2 proposes a method of generating a luminance signal by using a plurality of interpolation filters prepared in advance according to the angle of the subject and then performing edge enhancement to generate a final luminance signal.

  Patent Document 3 proposes a method of weighted addition of the OG signal and the SWY signal according to the hue and saturation of the subject, as in Patent Document 1. Specifically, the SWY signal is used for the low saturation subject, the SWY signal is used for the high saturation subject and Mg (magenta) and G (green) subjects, and the OG signal is used for the other chromatic subjects. Thereafter, an edge enhancement component is calculated from the MIX signal and added to a separately generated luminance signal to generate a final luminance signal.

  Further, in the method described in Patent Document 4, the first high-frequency signal is generated using the OG signal, and the angle adaptive type in which not only the horizontal and vertical directions but also the diagonal band is limited from the signals of all the color pixels. A second high-frequency signal is generated using a luminance signal generated by the SWY method. Then, the first high-frequency signal and the second high-frequency signal are weighted and added according to the spatial frequency of the signal to generate a third high-frequency signal, and the OG signal and the third high-frequency signal are added to obtain the final luminance. Generate a signal.

JP 2003-348609 A JP 2003-196649 A Japanese Patent No. 3699873 JP 2008-72377 A

  However, when the technique described in Patent Document 1 is applied to a signal obtained by using a color filter with a primary color Bayer array, if the SWY signal generated by the luminance signal generation circuit shown in FIG. There was an adverse effect of generating a false resolution signal (folding distortion) in the vicinity. This is presumably because the band in the H and V directions is limited in the circuit of FIG. 26, and the band limitation in the 45 ° and 135 ° directions is not sufficient.

  Further, since the OG signal is used for the chromatic color subject, the oblique resolution of the chromatic color subject is not improved. Furthermore, when an edge emphasis signal is generated with a luminance signal after a part of the OG signal (oblique area 2403) is replaced with the SWY signal, the switching portion between the OG signal and the SWY signal is emphasized, and an unnatural texture is likely to occur. .

  In the method of Patent Document 2, it is necessary to prepare and hold a plurality of interpolation filters according to the accuracy of the subject in advance. When the color configuration is, for example, NTSC-RGB space, it is necessary to prepare an interpolation filter that maintains the luminance signal configuration ratio R: G: B = 3: 6: 1. As a result, the filter coefficient is limited, and there is a problem that optimum filter processing cannot be performed for the angle of the subject. Similarly to Patent Document 1, since edge enhancement is performed after luminance signals calculated by a plurality of interpolation methods are mixed, subtle switching of luminance signals with different calculation methods is emphasized.

  Further, in the method of Patent Document 3, since the SWY signal is used instead of the OG signal by the Mg and G subjects, the resolution of subjects other than Mg and G among the chromatic subjects is not improved. Furthermore, as in the methods of Patent Document 1 and Patent Document 2, edge enhancement is performed after weighted addition of luminance signals calculated by a plurality of methods, so that the switching portion of the method is emphasized.

In Patent Document 4, in order to suppress image quality deterioration in a red subject, a red region is detected, and the final luminance signal is generated using the first high-frequency signal generated from the OG signal. For this reason, there are problems such as a phenomenon that a red subject is blurred, particularly a blonde that is difficult to distinguish from red, and a problem that the aliasing distortion of the red diagonal line cannot be removed.
The same problem occurs for a blue subject depending on the spectral distribution characteristics of the color filter.

  The present invention has been made in view of such problems of the prior art. An object of the present invention is to provide a luminance signal generation apparatus and a luminance signal generation method capable of suppressing the occurrence of aliasing distortion in a diagonal component for a red or blue subject.

  The above-described object is a luminance signal generation device that generates an edge-enhanced luminance signal for each pixel from an image signal read from an image sensor having a primary color Bayer array color filter, and generates the luminance signal from the image signal. A luminance signal generating means for generating, a first high-frequency signal generating means for generating a first high-frequency signal using image signals of pixels of all colors of the image sensor, and one of red and blue among the image sensors. Second high-frequency signal generation means for generating a second high-frequency signal using the image signal of the pixel, and an index indicating a measure of whether the pixel of interest is a color close to either red or blue from the image signal Based on the calculation means and the index, the first high-frequency signal and the second high-frequency signal are set so that the ratio of the second high-frequency signal increases as the pixel of interest is closer to either red or blue. Addition First weighted addition means for generating a high-frequency signal for the pixel of interest by addition, and edge enhancement for the pixel of interest by adding the high-frequency signal for the pixel of interest to the luminance signal generated by the luminance signal generation means for the pixel of interest It is achieved by a luminance signal generating apparatus characterized by having addition means for generating a luminance signal.

  Another object of the present invention is to provide a luminance signal generation method for generating an edge-enhanced luminance signal for each pixel from an image signal read from an image sensor having a primary color Bayer array color filter. A luminance signal generating step of generating a luminance signal from the image signal, and a first high frequency signal in which the first high frequency signal generating means generates the first high frequency signal using the image signals of the pixels of all colors of the image sensor. And a second high-frequency signal generating step in which the second high-frequency signal generating means generates a second high-frequency signal using an image signal of one of red or blue pixels in the image sensor. A calculating unit that calculates an index indicating a measure of whether the pixel of interest is a color close to either red or blue from the image signal; and a first weighted addition unit that determines whether the pixel of interest is based on the index Red or The first high-frequency signal for the pixel of interest is generated by weighted addition of the first high-frequency signal and the second high-frequency signal so that the closer the color is to one of blue, the higher the ratio of the second high-frequency signal is. And an adding step in which the addition means adds a high-frequency signal for the pixel of interest to the luminance signal generated by the luminance signal generation step for the pixel of interest to generate an edge-enhanced luminance signal for the pixel of interest. It is also achieved by a luminance signal generation method characterized by comprising:

  With such a configuration, according to the present invention, it is possible to generate a luminance signal that suppresses the occurrence of aliasing distortion in a diagonal component even for a red or blue subject.

It is a block diagram which shows the structural example of the luminance signal generation apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows the example of the frequency characteristic of the low pass filter used for the luminance signal generation apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows the example of the coefficient of the two-dimensional spatial filter used for the luminance signal generation apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structural example of the angle signal generation circuit in the 1st Embodiment of this invention. It is a block diagram which shows the structural example of the 1st HVD weighted addition circuit in the 1st Embodiment of this invention. FIG. 6 is a diagram illustrating an input / output characteristic example of a first coefficient calculation circuit in FIG. 5. FIG. 2 is a block diagram illustrating a configuration example of a color processing circuit according to the first embodiment of the present invention. It is a block diagram which shows the structural example of the 1st and 2nd GR weighted addition circuit in the 1st Embodiment of this invention. It is a figure which shows the example of the input-output characteristic of the 1st red degree calculation circuit in FIG. It is a figure which shows the input-output characteristic example of the 2nd redness degree calculation circuit in FIG. FIG. 9 is a diagram illustrating an input / output characteristic example of a second coefficient calculation circuit in FIG. 8. It is a block diagram which shows the structural example of the luminance signal generation apparatus which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the adaptive interpolation circuit in the luminance signal generation apparatus which concerns on the 2nd Embodiment of this invention. It is a figure explaining operation | movement of the DiffH signal generation circuit in FIG. 13, and a DiffV signal generation circuit. It is a figure which shows the input-output characteristic example of the G coefficient calculation circuit in FIG. It is a block diagram which shows the structural example of the diagonally weighted addition circuit in the luminance signal generation apparatus which concerns on the 2nd Embodiment of this invention. It is a figure which shows the frequency space in which an adaptive OG signal and a SWY signal can be resolved. FIG. 17 is a diagram illustrating an input / output characteristic example of a second SWY coefficient calculation circuit in FIG. 16. FIG. 17 is a block diagram illustrating a configuration example of a third SWY coefficient calculation circuit in FIG. 16. It is a figure which shows the input-output characteristic example of CalSwyUse3 in FIG. FIG. 17 is a block diagram illustrating a configuration example of a fourth SWY coefficient calculation circuit in FIG. 16. It is a figure which shows the input-output characteristic example of CalSwyUse4 in FIG. FIG. 17 is a block diagram illustrating a configuration example of a fifth SWY coefficient calculation circuit in FIG. 16. It is a figure which shows 1 unit of a primary color Bayer arrangement | sequence. It is a block diagram which shows the structural example of the luminance signal generation circuit by OG system. It is a block diagram which shows the structural example of the luminance signal generation circuit by a SWY system. It is a figure which shows the resolvable spatial frequency characteristic of OG signal and SWY signal.

(First embodiment)
Hereinafter, the present invention will be described in detail based on preferred and exemplary embodiments with reference to the drawings.
FIG. 1 is a block diagram showing a configuration example of a luminance signal generation apparatus according to the first embodiment of the present invention. The luminance signal generation device according to the present embodiment can be suitably realized by a signal processing circuit that performs signal processing such as so-called development processing in an imaging device that uses an imaging device having a color filter with a primary color Bayer array.

  The luminance signal generation device generates a luminance signal for each pixel using the image signal for each pixel read from the image sensor. Therefore, although not particularly described, the following processing is executed for each target pixel while sequentially updating the target pixel.

(Calculation of the first luminance signal)
An image signal read out from an image sensor (not shown) having a primary color Bayer array color filter and subjected to digitization and white balance processing is input as a RAW signal 100.

  A vertical low-pass filter (V-LPF) 101 and a horizontal low-pass filter (H-LPF) 102 serving as a first low-pass filter limit the horizontal and vertical bands of the RAW signal 100 to generate a luminance signal Yhv_1.

FIG. 2 is a diagram illustrating an example of frequency characteristics of the low-pass filter used in the luminance signal generation apparatus of the present embodiment. In FIG. 2, the x-axis indicates the spatial frequency and the y-axis indicates the pass gain. The V-LPF 101 and the V-LPF 102 have a frequency characteristic 200 in which the gain is 0 at the Nyquist frequency N of the image sensor. On the other hand, as will be described later, the V-LPF 116 and the H-LPF 117 have a frequency characteristic 201 in which the gain is 0 at a frequency (N / 2) that is half the Nyquist frequency N of the image sensor.
Normally, the V-LPF 101 and the H-LPF 102 having the frequency characteristic 200 are applied to a monochrome (achromatic) subject.

  The luminance signal Yhv_1 output from the H-LPF 102 is supplied to the first HVD weighted addition circuit 114. The luminance signal Yhv_1 is also supplied to the two-dimensional low-pass filters D45-LPF103 and D135-LPF106, the vertical bandpass filter (V-LPF) 109, and the horizontal bandpass filter (H-BPF) 111.

  FIG. 3 is a diagram illustrating an example of coefficients of a two-dimensional spatial filter used in the luminance signal generation apparatus according to the present embodiment. 3, (a) shows an example of D45-LPF 103, (b) shows an example of D135-LPF 106, (c) shows an example of D135-BPF 104 described later, and (d) shows an example of a coefficient of D45-BPF 107 described later.

The D45-LPF 103 generates a luminance signal Yd45_1 in which the band of the luminance signal Yhv_1 in the 45-degree direction is limited, and supplies the luminance signal Yd45_1 to the first HVD weighted addition circuit 114.
The D135-LPF 106 generates a luminance signal Yd135_1 in which the band of the luminance signal Yhv_1 in the 135 degree direction is limited, and supplies the luminance signal Yd135_1 to the first HVD weighted addition circuit 114.

  The first HVD weighted adder circuit 114 weights and adds the luminance signal Yhv_1 band-limited in the horizontal and vertical directions, and the luminance signal Yd45_1 and luminance signal Yd135_1 band-limited in the 45-degree direction or 135-degree direction, respectively. 1 luminance signal Yh_1 is generated.

Next, a weighted addition method in the first HVD weighted addition circuit 114 will be described.
The first HVD weighted addition circuit 114 weights and adds the three luminance signals with a weighted addition coefficient corresponding to the angle of the subject.

  First, an OG signal is generated from the RAW signal 100 by the OG signal generation circuit 140. The OG signal generation circuit 140 generates an OG signal by the method described with reference to FIG. The angle signal generation circuit 150 generates an angle signal angleSigD indicating the angle of the subject from the OG signal, and supplies the angle signal angleSigD to the first HVD weighted addition circuit 114.

FIG. 4 is a block diagram illustrating a configuration example of the angle signal generation circuit. The OG signal 300 is input to D135-BPF301 and D45-BPF303, which are two-dimensional bandpass filters. D135-BPF301 and D45-BPF303 have coefficients shown in (c) and (d) of FIG. 3, respectively, and emphasize the 45-degree line and the 135-degree line.
Absolute value circuits (ABS) 302 and 304 generate 45 degree line detection signals and 135 degree line detection signals from the outputs of D135-BPF 301 and D45-BPF 303, respectively.

  The subtraction circuit 305 subtracts the 135 degree line detection signal from the ABS 304 from the 45 degree line detection signal from the ABS 302 and sets the subtraction result as an angle signal (angleSigD) 306. The angle signal angleSigD means a 45 degree line if positive and a 135 degree line if negative. The region where the amplitude of the output signal is small means horizontal and vertical lines.

FIG. 5 is a block diagram showing a configuration example of the first HVD weighted addition circuit 114 as the first luminance signal generation means.
The first coefficient calculation circuit 1141 calculates and outputs a weighted addition coefficient D_UseRatio based on the value of the angle signal angleSigD. The polarity discriminating circuit 1142 discriminates the polarity of the angle signal angleSigD and controls the selector 1143 based on the polarity (positive / negative), thereby supplying one of the luminance signal Yd45_1 and the luminance signal Yd135_1 to the first arithmetic circuit 1144. Specifically, the polarity determination circuit 1142 supplies the selector 1143 to supply the luminance signal Yd45_1 to the first arithmetic circuit 1144 when the polarity of the angle signal angleSigD is positive and the luminance signal Yd135_1 when it is negative. Control.

  The first arithmetic circuit 1144 uses the weighted addition coefficient D_UseRatio from the first coefficient calculation circuit 1141 to calculate the first luminance signal Yh_1 by weighted addition of the luminance signal Yhv_1 and the luminance signal Yd45_1 or the luminance signal Yd135_1. To do.

FIG. 6 is a diagram illustrating an input / output characteristic example of the first coefficient calculation circuit 1141.
Since the absolute value (amplitude) of the angle signal angleSigD is from 0 to a preset threshold value P1 (0 ≦ | angleSigD | ≦ P1) is a horizontal or vertical signal, the first coefficient calculation circuit 1141 sets the coefficient D_UseRatio to 0. And On the other hand, the coefficient D_UseRatio is set to 128 in a section (P2 <| angleSigD |) equal to or greater than the threshold value P2 representing a complete oblique signal. On the other hand, linear interpolation output is performed in the interval between the threshold values P1 and P2 (P1 <| angleSigD | <P2).

  In this way, the first coefficient calculation circuit 1141 obtains the weighted addition coefficient D_UseRatio based on the absolute value | angleSigD | of the angle signal and supplies it to the first arithmetic circuit 1144.

  The first arithmetic circuit 1144 generates the first luminance signal Yh_1 using the following equations (1-1) and (1-2).

  Here, Yhv_1 indicates a luminance signal whose band is limited in the horizontal and vertical directions, and Yd45_1 and Yd135_1 indicate a 45 degree luminance signal and a 135 degree luminance signal whose band is further limited in the 45 degree direction and the 135 degree direction, respectively.

(Calculation of the first high-frequency signal)
The luminance signal Yhv_1 whose vertical and horizontal bands are limited by the V-LPF 101 and the H-LPF 102 is displayed in the vertical and horizontal directions by the vertical bandpass filter (V-BPF) 109 and the horizontal bandpass filter (H-BPF) 111. Edge components are detected. Each edge component is multiplied by a gain set in advance by a vertical gain circuit (V-Gain) 110 and a horizontal gain circuit (H-Gain) 112 and then added by an adder circuit 113 to obtain a first horizontal vertical edge signal ( High-frequency signal) AChv_1.

  On the other hand, the luminance signal whose band in the 45-degree direction is further limited by the D45-LPF 103 is D135-BPF 104 for detecting an AC component (edge component or high-frequency component) in the 135-degree direction (coefficients are shown in FIG. 3C). ), The edge component of the 45 degree line is extracted. Thereafter, the gain is adjusted by the gain circuit 105, and a 45-degree line edge signal ACd45_1 is calculated. Similarly, the edge signal in the 135 degree direction is extracted from the luminance signal whose band in the 135 degree direction is further limited by the D135-LPF 106 by the D45-BPF 107 (coefficients are shown in FIG. 3D). Thereafter, the gain is adjusted by the gain circuit 108, and a 135-degree line edge signal Acd135_1 is calculated.

  The second HVD weighted addition circuit 115 functions as first high-frequency signal generation means. In the second HVD weighted addition circuit 115, the weighted addition coefficient D_UseRatio and the three edge signals used when generating the first luminance signal Yh_1 are weighted and added according to the following equations (1-3) and (1-4). A first high-frequency signal AC_1 is generated.

  The second HVD weighted addition circuit 115 has the same configuration as the first HVD weighted addition circuit 114 shown in FIG. 5 except that AChv_1, ACd45_1, and ACd135_1 are input instead of Yhv_1, Yd45_1, and Yd135_1. Good. Alternatively, the weighted addition coefficient D_UseRatio and the polarity information of the angle signal angleSigD can be acquired from the first HVD weighted addition circuit 114. In this case, the first coefficient calculation circuit 1141 and the polarity determination circuit 1142 are unnecessary.

(Calculation of second luminance signal (for red subject))
The second luminance signal for the red subject is generated by the same processing as the first luminance signal, but the characteristics of the band limiting filter in the processing are different.

  The RAW signal 100 is limited in the horizontal and vertical bands by the V-LPF 116 and the H-LPF 117. The V-LPF 116 and the H-LPF 117 as the second low-pass filter have the frequency characteristics 201 of FIG. That is, the band of the RAW signal 100 is limited to about half of the frequency characteristic 200 of the V-LPF 101 and the H-LPF 102 (about half of the Nyquist frequency of the image sensor).

  This is because, in the case of a primary color filter, particularly in a red subject, the output signal value from the green pixel and the blue pixel is almost 0. Therefore, the Nyquist frequency by spatial sampling by the image sensor is a normal achromatic subject in both the vertical and horizontal directions. Because it becomes half of

  The luminance signal Yhv_2 output from the H-LPF 117 is processed in the same manner as the luminance signal Yhv_1, and the second luminance signal Yh_2 and the second high-frequency signal AC_2 are generated by the third and fourth weighted addition circuits 129 and 130. Is done.

  Specifically, the luminance signal Yhv_2 is supplied to the third HVD weighted addition circuit 129 as the signals Yd45_2 and Yd135_2 after the bands in the 45 degree direction and the 135 degree direction are limited by the D45-LPF 118 and the D135-LPF 121. Further, the luminance signal Yhv_2 is supplied to the third HVD weighted addition circuit 129 as it is.

  The third HVD weighted addition circuit 129 serving as the second luminance signal generation unit has the same configuration as the first HVD weighted addition circuit 114, and calculates the weighted addition coefficient D_UseRatio from the angle signal angleSigD. Then, the third weighted addition circuit 129 generates the second luminance signal Yh_2 using the following equations (1-5) and (1-6).

  Here, Yhv_2 indicates a luminance signal whose band is limited in the horizontal and vertical directions, and Yd45_2 and Yd135_2 indicate a 45 degree luminance signal and a 135 degree luminance signal whose band is further limited in the 45 degree direction and the 135 degree direction, respectively. As described above, these signals are different from Yhv_1, Yd45_1, and Yd135_1 in the band limitation in the vertical direction and the horizontal direction.

(Calculation of second high-frequency signal (for red subject))
The luminance signal Yhv_2 whose band in the vertical and horizontal directions is limited by the V-LPF 116 and the H-LPF 117 is displayed in the vertical and horizontal directions by the vertical bandpass filter (V-BPF) 124 and the horizontal bandpass filter (H-BPF) 126. Edge components are detected. Each edge component is multiplied by a gain set in advance by a vertical gain circuit (V-Gain) 125 and a horizontal gain circuit (H-Gain) 127, and then added by an adder circuit 128 to obtain a second horizontal / vertical edge signal ( High frequency signal) AChv_2.

  On the other hand, the luminance signal whose band in the 45-degree direction is further limited by the D45-LPF 118 is D135-BPF 119 for detecting the AC component (edge component or high-frequency component) in the 135-degree direction (the coefficients are shown in FIG. 3C). ), The edge component of the 45 degree line is extracted. Thereafter, the gain is adjusted by the gain circuit 120, and a 45-degree line edge signal ACd45_2 is calculated. Similarly, the edge signal in the 135 degree direction is extracted from the luminance signal whose band in the 135 degree direction is further limited by the D135-LPF 121 by the D45-BPF 122 (coefficients are shown in FIG. 3D). Thereafter, the gain is adjusted by the gain circuit 123, and a 135-degree line edge signal Acd135_2 is calculated.

  The fourth HVD weighted addition circuit 130 functions as second high frequency signal generation means. In the fourth HVD weighted addition circuit 130, the weighted addition coefficient D_UseRatio and the three edge signals used when generating the second luminance signal Yh_2 are weighted and added according to the following equations (1-7) and (1-8), A second edge signal AC_2 is generated.

  The fourth HVD weighted addition circuit 115 has the same configuration as the first HVD weighted addition circuit 114 shown in FIG. 5 except that AChv_2, ACd45_2, and ACd135_2 are input instead of Yhv_1, Yd45_1, and Yd135_1. Good. Alternatively, the weighted addition coefficient D_UseRatio and the polarity information of the angle signal angleSigD can be acquired from the third HVD weighted addition circuit 129. In this case, the first coefficient calculation circuit 1141 and the polarity determination circuit 1142 are unnecessary.

(Weighted addition of first and second luminance signal and high frequency signal)
The first luminance signal Yh_1 and the second luminance signal Yh_2 generated by the first and third HVD weighted addition circuits 114 and 129 are weighted by the first GR weighted addition circuit 131 as the first weighted addition means. Addition results in a third luminance signal Yh_3.

  The first GR weighted addition circuit 131 of the present embodiment uses 100% of the first luminance signal Yh_1 for the normal subject other than the red subject. Further, the first GR weighted addition circuit 131 uses 100% of the second luminance signal Yh_2 signal in which the red subject is limited to the half of the first luminance signal Yh_1 in the vertical and horizontal bands.

  By performing such weighted addition, when information of all pixels is used at the time of generating a luminance signal of an imaging device having a primary color filter (for example, when the SWY method is used), aliasing distortion is also generated for a red subject. A sufficiently suppressed luminance signal can be generated.

FIG. 7 is a block diagram illustrating a configuration example of the color processing circuit 160 in FIG.
The color processing circuit 160 generates a phase angle signal Chromaθ and a saturation signal ChromaSig from the RAW signal 100 as signals for determining the redness of the subject, and sends them to the first and second GR weighted addition circuits 131 and 132. Supply.

  In FIG. 7, the RAW signal 100 is subjected to interpolation calculation (low-pass filter processing) for each pixel in the color interpolation circuit 601, and then the color difference signal RY (Ry) signal and BY (By ) Converted to a signal.

  The hue angle detection circuit 603 generates the hue angle signal Chromaθ by applying the following expression (1-9) to the color difference signal generated for each pixel.

  The saturation calculation circuit 604 generates the saturation signal ChromaSig by applying the following equation (1-10) to the color difference signal generated by the color difference signal generation circuit 602.

  The hue angle signal Chromaθ and the saturation signal ChromaSig generated by the color processing circuit 160 are output to the first and second GR weighted addition circuits 131 and 132.

FIG. 8 is a block diagram illustrating a configuration example of the first and second GR weighted addition circuits 131 and 132.
The first redness calculation circuit 1311 has input / output characteristics as shown in FIG. 9, and converts the hue angle signal Chromaθ into a first redness RedSig_1 as one of indices indicating the redness of the subject. .

  In the example shown in FIG. 9, the range of the hue angle signal Chromaθ from 0 to a predetermined threshold P1 (Chromaθ ≦ P1) and the threshold P4 or more (P4 ≦ Chromaθ) represents colors other than red. Set RedSig_1 to 0. On the other hand, the range from the threshold value P2 to P3 (P2 ≦ Chromaθ ≦ P3) represents red, and the first redness RedSig_1 is set to 128. In the other ranges (P1 <Chromaθ <P2, P3 <Chromaθ <P4), the first redness RedSig_1 having a value obtained by linear interpolation of 128 and 0 is output. P1 to P4 can be determined in advance as redness thresholds.

The second redness calculation circuit 1312 has input / output characteristics as shown in FIG. 10, and the saturation signal ChromaSig is converted into the second redness RedSig_2 as one of indices indicating the measure of the redness of the subject. Convert.
In the example shown in FIG. 10, the value of the saturation signal ChromaSig from 0 to a predetermined threshold C1 (ChromaSig ≦ C1) is low saturation and is affected by noise, so RedSig_2 is set to 0. The range of the threshold C2 or more (C2 ≦ ChromaSig) represents red, and the second redness RedSig_2 is set to 128. In the other range (C1 <ChromaSig <C2), the second redness RedSig_2 having a value obtained by linear interpolation of 128 and 0 is output. C1 to C2 can be determined in advance as redness thresholds.

  The third redness calculation circuit 1313 multiplies the first redness RedSig_1 and the second redness RedSig_2 according to the following equation (1-11) to calculate the third redness RedSig_3.

  The second coefficient calculation circuit 1314 has input / output characteristics as shown in FIG. 11, and outputs a GR weighted addition coefficient Yh2UseRatio from the third redness RedSig_3.

  In the example shown in FIG. 11, the value of the third redness RedSig_3 from 0 to a predetermined threshold Th1 (RedSig_3 ≦ Th1) is not red, so the GR weighted addition coefficient Yh2UseRatio is set to 0. The range of threshold Th2 or more (Th2 ≦ RedSig — 3) represents red, and the GR weighted addition coefficient Yh2UseRatio is set to 128. In the other range (Th1 <RedSig_3 <Th2), a GR weighted addition coefficient Yh2UseRatio having a value obtained by linear interpolation of 128 and 0 is output. Th1 to Th2 can be determined in advance as redness thresholds.

  With such input / output characteristics, 100% of the second luminance signal Yh_2 is used for a red subject, and 100% of the first luminance signal Yh_1 is used for a non-red subject, and the first and second luminances are used for an intermediate subject. Signals Yh_1 and Yh_2 are used in proportion to redness.

  Finally, the second arithmetic circuit 1315 uses the weighted addition coefficient Yh2UseRatio and, from the first luminance signal Yh_1 and the second luminance signal Yh_2, according to the following equations (1-12) and (1-13), A third luminance signal Yh_3 is calculated. Similarly, in the second GR weighted addition circuit 132 as the second weighted addition means, the first high frequency signal AC_1 and the second high frequency signal AC_2 are weighted and added, and the third high frequency signal AC_3 is obtained. Generate.

  In the second GR weighted addition circuit 132, the weighted addition coefficient Yh2UseRatio may be acquired from the first GR weighted addition circuit 131 and used. In this case, the second GR weighted addition circuit 132 may have only the second arithmetic circuit 1315 in FIG.

Alternatively, the second GR weighted addition circuit 132 may calculate the weighted addition coefficient Yh2UseRatio, and the first GR weighted addition circuit 131 may acquire the weighted addition coefficient Yh2UseRatio from the second GR weighted addition circuit 132. In this case, the first GR weighted addition circuit 131 need only have the second arithmetic circuit 1315 in FIG.
The adder circuit 133 adds the third luminance signal Yh_3 and the third high-frequency signal AC_3, and outputs the addition result as a final luminance signal.

  In the present embodiment, the description has been made assuming that a red subject is used as a subject color whose horizontal and vertical resolutions are reduced. However, depending on the spectral distribution characteristics of the color filter, the output of the R pixel and the G pixel may be almost zero for a blue subject. In this case, it is possible to detect the blueness instead of the redness, apply the same processing to the blue subject, and introduce the second luminance signal Yh_2 to generate the luminance signal. Further, the second luminance signal Yh_2 may be used for both the red and blue subjects.

  As described above, according to the present embodiment, in the luminance signal generation device that generates a luminance signal from an image signal obtained from an image sensor using a color filter with a primary color Bayer array, the vertical and horizontal bands are assigned to the first and second bands. The first luminance signal is generated by limiting to the range of 1. Further, the second luminance signal is generated by limiting the vertical and horizontal bands to the second range narrower than the first range. Then, as the red or blue color of the subject is stronger, the ratio of the second luminance signal is increased and the first and second luminance signals are weighted and added to generate a final luminance signal. Therefore, it is possible to generate a luminance signal that sufficiently suppresses the occurrence of aliasing distortion for a red subject and a blue subject.

  In the present embodiment, the first high-frequency signal and the second high-frequency signal having different band limitation ranges are similarly generated for the high-frequency signal, and the second red color or blue color of the subject becomes stronger. The ratio of the high frequency signal is increased and the first and second high frequency signals are weighted and added. Since a luminance signal is generated using such a high-frequency signal, a high-resolution luminance signal with reduced blur can be generated for a red subject and a blue subject.

(Second Embodiment)
FIG. 12 is a block diagram illustrating a configuration example of a luminance signal generation apparatus according to the second embodiment of the present invention. In FIG. 12, the same components as those in the first embodiment are denoted by the same reference numerals, and a duplicate description is omitted. In the present embodiment, it is assumed that all LPFs have the frequency characteristics indicated by 200 in FIG.

(Calculation of the first high-frequency signal)
It is generated by the same method as the first high-frequency signal AC_1 in the first embodiment.

  The RAW signal 100 has V-LPF 1017 and H-LPF 1018 that are equivalent to the V-LPF 101 and H-LPF 102 in the first embodiment, and the bands in the vertical and horizontal directions are limited, respectively. The V-BPF 109 and H-BPF 111 detect edge components in the vertical and horizontal directions. Each edge component is multiplied by a gain set in advance by the gain circuits 110 and 112 and added by the adding circuit 113 to calculate a first horizontal / vertical edge signal AChv_1.

  On the other hand, from the D45-LPF 1019 equivalent to the D45-LPF 103 in the first embodiment, the luminance signal Yhv_1 whose band in the 45-degree direction is limited is extracted by the D135-BPF 104 as the edge component of the 45-degree line. Then, the gain is adjusted by the gain circuit 105, and the first 45-degree line edge signal ACd45_1 is calculated.

Similarly, in the luminance signal Yhv_1 whose band in the 135 degree direction is limited by the D135-LPF 1022 equivalent to the D135-LPF 106 in the first embodiment, an edge component in the 135 degree direction is extracted by the D45-BPF 107. Then, the gain is adjusted by the gain circuit 108, and the first 135-degree line edge signal Acd135_1 is calculated.
The first HVD weighted addition circuit 115 weights and adds the three edge signals using the weighted addition coefficient D_UseRatio calculated in the first embodiment to generate a first high-frequency signal AC_1.

(Calculation of second high-frequency signal (for red subject))
Although it is generated by a method almost the same as the second high-frequency signal in the first embodiment, in this embodiment, all pixel signals of the RAW signal 100 are not used to obtain a high-frequency component, and 0 is inserted in addition to the R pixel. The difference from the first embodiment is that a processed signal is used. In this way, by setting the pixels other than the R pixel to 0, it is possible to prevent the aliasing distortion from the color pixel other than the R pixel to the luminance signal with a minute amplitude when photographing the red subject. The second embodiment is also different from the first embodiment in that the vertical and horizontal bands of the RAW signal are limited to the same level as the first high-frequency signal.

  In the RAW signal 100, pixel signal values other than the R pixel are replaced with 0 by the 0 insertion circuit 1001. Thereafter, the V-LPF 1004 and the H-LPF 1005 limit the vertical and horizontal bands to the Nyquist frequency or lower. Further, edge components in the horizontal and vertical directions are detected by the V-BPF 124 and the H-BPF 126, respectively multiplied by gains by the gain circuits 125 and 127, and then added by the adder circuit 128 to calculate an edge signal AChv_2.

On the other hand, for the image signal whose band in the 45-degree direction is limited by the D45-LPF 1006, the edge component of the 45-degree line is extracted by the D135-BPF 119, the gain is adjusted by the gain circuit 120, and the edge signal ACd45_2 is calculated. Similarly, for the image signal whose band in the 135 degree direction is limited by the D135-LPF 1009, the edge component of the 135 degree line is extracted by the D45-BPF 122, and the gain is adjusted by the gain circuit 123 to calculate the edge signal ACd135_2. .
The second HVD weighted addition circuit 130 weights and adds the three edge signals using the weighted addition coefficient D_UseRatio to generate a second edge signal AC_2.

(Calculation of third high-frequency signal)
In the first embodiment, since the high-frequency signal is calculated using the SWY signal, the resolution near the horizontal and vertical Nyquist frequencies deteriorates. Therefore, the third high-frequency signal is generated using the adaptively interpolated OG signal to prevent resolution degradation near the Nyquist frequency.

The 0 insertion circuit 1032 to the addition circuit 1038 function as third high frequency signal generation means for generating a high frequency signal from the image signal of the green pixel.
In the RAW signal 100, pixel signal values other than G pixels are replaced with 0 in the 0 insertion circuit 1032 and the adaptive interpolation circuit 1033 generates an adaptive OG signal by adaptive interpolation of the RAW signal 100 inserted in 0.

FIG. 13 is a block diagram illustrating a configuration example of the adaptive interpolation circuit 1033 in the present embodiment.
The adaptive interpolation circuit 1033 receives both the RAW signal 100 in which pixel signal values other than G pixels are replaced with 0 by the 0 insertion circuit 1032 and the RAW signal 100 before replacement.

  The zero-replaced RAW signal 100 is limited in the vertical band by the V-LPH 1102 to generate a Gv signal. The Gv signal is input to the G weighted addition circuit 1105 and input to the H-LPF 1103. The H-LPF 1103 limits the horizontal band of the Gv signal, and generates a Ghv signal in which the vertical and horizontal bands are limited. The Ghv signal is input to the G weighted addition circuit 1105. The H-LPF 1104 limits the horizontal band of the zero-substituted RAW signal 100, generates a Gh signal, and inputs the Gh signal to the G weighted addition circuit 1105.

  In this way, signals other than G pixels are set to 0 (1) Gv signal in which the vertical band is limited, (2) Gh signal in which the horizontal band is limited, and (3) Vertical and horizontal directions A Ghv signal with a limited bandwidth is generated and is input to the G-weighted addition circuit 1105.

  On the other hand, the DiffH signal generation circuit (DiffH) 1106 and the DiffV signal generation circuit (DiffV) 1107 generate a DiffH signal and a DiffV signal from the RAW signal 100 in which 0 is not inserted as follows.

FIG. 14 is a diagram for explaining the operation of the DiffH signal generation circuit 1106 and the DiffV signal generation circuit 1107. When the target pixel is P22, the DiffH signal generation circuit 1106 and the DiffV signal generation circuit 1107 generate a DiffH signal and a DiffV signal according to the following equations (2-1) and (2-2).
DiffH = | P21-P23 | + | 2 × P22-P20-P24 | (2-1)
DiffV = | P12-P32 | + | 2 × P22-P02-P42 | (2-2)
Note that P12 to P42 in the equation are signal values of corresponding pixels in FIG.

When the subject is a vertical stripe, the DiffH signal is large, and when the subject is a horizontal stripe, the value of the DiffV signal is large. The subtractor 1108 subtracts the DiffV signal from the DiffH signal, and outputs a DiffHV signal indicating the degree to which the target pixel corresponds to the tilted line.
That is,
DiffHV = DiffH-DiffV
It is.

  A coefficient calculation circuit (Tsig) 1109 calculates a weighted addition coefficient based on the DiffHV signal and supplies it to the G weighted addition circuit 1105.

FIG. 15 is a diagram illustrating an example of input / output characteristics of the G coefficient calculation circuit 1109.
The horizontal axis indicates the DiffHV signal, and the vertical axis indicates the value of the weighted addition coefficient Tsig. When DiffHV ≦ Th0, since DiffV is larger than DiffH, it is determined that the region is a horizontal stripe region. In this case, a Tsig value (Tsig = −128) in which the G weighted addition circuit 1105 uses the Gh signal 100% is output.

The section of Th1 <DiffHV <Th2 is determined as an oblique area because DiffV and DiffH are close to each other. In this section, a Tsig value (Tsig = 0) at which the G weighted addition circuit 1105 uses 100% of the Ghv signal is output.
Further, when DiffHV ≧ Th3, since DiffH is larger than DiffV, it is determined to be a vertical stripe region. In this case, a Tsig value (Tsig = 128) using 100% of the Gv signal is output.

  In the section of Th0 <DiffHV ≦ Th1, Tsig determined so as to change linearly from −128 to 0 is output. Similarly, Tsig determined so as to change linearly from 0 to 128 is output in the section of Th2 ≦ DiffHV <Th3.

  Based on the Tsig value determined in this way, the G weighted addition circuit 1105 performs weighted addition using the following equations (2-3) and (2-4) for the Gh signal, Gv signal, and Ghv signal. And an adaptive OG signal is generated.

  In the adaptive OG signal generated by the adaptive interpolation circuit 1033, the edge component in the vertical direction is detected by the V-BPF 1034, and the gain is adjusted by the gain circuit 1035. On the other hand, in the adaptive OG signal, the horizontal edge component is detected by the H-BPF 1036 and the gain is adjusted by the gain circuit 1037. The vertical and horizontal edge components after gain adjustment are added by the adder circuit 1038 and sent to the obliquely weighted adder circuit 1039 as the third high-frequency signal AC_3.

(Weighted addition of first and third high-frequency signals)
The oblique weighted addition circuit 1039 weights and adds the first high-frequency signal AC_1 generated from all the color pixels (SWY signal) and the third high-frequency signal AC_3 generated from the adaptive OG signal according to the following equation (2-5). Then, the fourth high-frequency signal AC_4 is generated.

FIG. 16 is a block diagram illustrating a configuration example of the oblique weighted addition circuit 1039 of the present embodiment.
The oblique weighted addition circuit 1039 as the second weighted addition means obtains the first to fourth SWY coefficients SWYUseRatio_1 to 4 by the first to fourth SWY coefficient calculation circuits 1060 to 1063. Then, in the fifth SWY coefficient calculation circuit 1064, the final weighted addition coefficient SWYUseRatio is obtained from the first to fourth SWY coefficients SWYUseRatio_1 to 4. The third arithmetic circuit 1065 performs weighted addition according to the above equation (2-5) to obtain the fourth high-frequency signal AC_4.

(1) Calculation of SWYUseRatio_1 (first SWY coefficient) FIG. 17 is a diagram illustrating a frequency space in which the adaptive OG signal and the SWY signal can be resolved.
Here, the region where the third high-frequency signal AC_3 is affected by the aliasing distortion is an oblique region 1400. Therefore, if the oblique region 1400 is replaced with the first high-frequency signal AC_1, the fourth high-frequency signal AC_4 having no aliasing distortion can be generated. Therefore, the weighted addition coefficient can be calculated by using the angle signal AngleSigD used in the first embodiment.

  The angle signal AngleSigD is a 45-degree line if positive, a 135-degree line if negative, and closer to the 45-degree and 135-degree lines if the absolute value of the signal value is large, and closer to the 0-degree and 90-degree lines if it is small. It has the characteristics to become.

  As a result, the first SWY coefficient SWYUseRatio_1 can be obtained from the angle signal AngleSigD by the first SWY coefficient calculation circuit 1060. Specifically, the same input / output characteristics (in FIG. 6, the output value D_UseRatio is SWYUseRatio_1 in FIG. 6) used in the first embodiment are set to the first SWY coefficient calculation circuit 1060. You just have to.

(2) Calculation of SWYUseRatio_2 (second SWY coefficient) The horizontal and vertical region 1401 in FIG. 17 uses the third high-frequency signal AC_3 generated from the adaptive OG signal. This is because the third high-frequency signal AC_3 is based on the adaptive OG signal, so that the resolution is better than the second high-frequency signal AC_2 generated from the SWY signal whose band is always limited in both the horizontal and vertical directions. by.

  The horizontal / vertical region 1401 can be determined using the DiffH signal and the DiffV signal described with reference to FIGS. 13 and 14. As described above, when the subject is a vertical stripe, the value of DiffH increases and DiffV becomes zero. When the subject is a horizontal stripe, the value of DiffV increases and becomes DiffH0.

  Therefore, the second SWY coefficient SWYUseRatio_2 can be calculated from the DiffHV signal by using the second SWY coefficient calculation circuit 1061 having the input / output characteristics shown in FIG. In FIG. 18, the value of the input signal | DiffHV | is from 0 to P1 and is a diagonal line because the value is small, and the second SWY coefficient SWYUseRatio_2 is set to 128. Further, P2 or more are completely horizontal stripes or vertical stripes, and the second SWY coefficient SWYUseRatio_2 is set to 0. In the interval from P1 to P2, the value of the second SWY coefficient SWYUseRatio_2 is a value obtained by linearly interpolating 128 and 0.

(3) Calculation of SWYUseRatio_3 (third SWY coefficient) In the region near the horizontal and vertical Nyquist frequencies (Nyquist region) 1402 in FIG. 17, the third high-frequency signal AC_3 is used for the same reason as (2). preferable. The Nyquist area 1402 can be determined by detecting a phase shift between the G1 pixel signal and the G2 pixel signal.

FIG. 19 is a block diagram illustrating a configuration example of the third SWY coefficient calculation circuit 1062.
The 0 insertion circuits 1601 and 1064 respectively generate signals having pixel signal values other than the G1 and G2 pixels of the RAW signal 100 as 0. Values are interpolated to pixels set to 0 by low-pass filters (V-LPF 1602, H-LPF 1603, V-LPF 1605, and H-LPF 1066) for horizontal and vertical interpolation.

  Next, the subtraction circuit 1607 obtains the difference between the interpolated G1 signal (output of the H-LPF 1063) and the interpolated G2 signal (output of the H-LPF 1066), and calculates the absolute value of the difference as an absolute value circuit (abs). Obtained in 1608 and output as a region discrimination signal sigDiffG.

  A third SWY coefficient SWYUseRatio — 3 is calculated from the area discrimination signal sigDiffG using CalSwyUse3 1609 having input / output characteristics as shown in FIG. CalSwyUse3 1609 may be an arithmetic circuit or a table.

  Since the value of the area discrimination signal sigDiffG is small from 0 to P1, it is a diagonal line, and the third SWY coefficient SWYUseRatio_3 is set to 128. Also, since P2 or more are completely horizontal stripes or vertical stripes, the third SWY coefficient SWYUseRatio_3 is set to zero. In the interval from P1 to P2, the value of the third SWY coefficient SWYUseRatio_3 is a value obtained by linearly interpolating 128 and 0.

(4) Calculation of SWYUseRatio — 4 (fourth SWY coefficient) It is preferable that the low frequency region 1403 in FIG. 17 uses the first high frequency signal AC_1. The low frequency region 1403 can be detected by applying a filter that detects the low frequency region to the RAW signal.

FIG. 21 is a block diagram illustrating a configuration example of the fourth SWY coefficient calculation circuit 1063.
The RAW signal 100 is limited in the vertical and horizontal bands by the V-LPF 1801 and the H-LPF 1802. Thereafter, a band-pass filter (H-BPF) 1803 in the horizontal direction and a band-pass filter (V-BPF) 1804 in the vertical direction are applied separately. The subtracter 1805 subtracts the output of the V-BPF 1804 from the output of the H-BPF 1803. An absolute value circuit (abs) 1806 calculates the absolute value of the subtraction result and outputs it as a low-frequency detection signal LowF.

  Then, a fourth SWY coefficient SWYUseRatio_4 is calculated using CalSwyUse4 1807 having input / output characteristics as shown in FIG. CalSwyUse4 1807 may be an arithmetic circuit or a table.

The value of the low-frequency detection signal LowF from 0 to P1 is small because it is a high-frequency region, and the fourth SWY coefficient SWYUseRatio_4 is set to 128. Also, since P2 and above are in the low frequency region, the fourth SWY coefficient SWYUseRatio_4 is set to zero. In the interval from P1 to P2, the value of the fourth SWY coefficient SWYUseRatio_4 is a value obtained by linearly interpolating 128 and 0.
The fifth SWY coefficient calculation circuit 1064 calculates a final weighted addition coefficient SWYUseRatio from the first SWY coefficient SWYUseRatio_1 to the fourth SWY coefficient SWYUseRatio_4.

FIG. 23 is a block diagram illustrating a configuration example of the fifth SWY coefficient calculation circuit 1064.
The multiplier 2004 multiplies the first SWY coefficient SWYUseRatio_1 and the second SWY coefficient SWYUseRatio_2, and the shifter 2005 performs a 7-bit shift operation in the right direction. The shifter 2005 multiplies the output of the multiplier 2004 by 1/128.

  The output of the shifter 2005 is multiplied by the third SWY coefficient SWYUseRatio_3 by the multiplier 2006, and the multiplication result is shifted 7 bits to the right by the shifter 2007. Finally, the output of the shifter 2007 is multiplied by the fourth SWY coefficient SWYUseRatio_4 by the multiplier 2008, and the multiplication result is output as the final weighted addition coefficient SWYUseRatio.

That is,
SWYUseRatio = SWYUseRatio_1 * SWYUseRatio_2 / 128 * SWYUseRatio_3 / 128 * SWYUseRatio_4 * / 128
It is.

  The third arithmetic circuit 1065 performs weighted addition according to the above equation (2-5) to obtain the fourth high-frequency signal AC_4 from the first and third high-frequency signals AC_1 and AC_3.

(Weighted addition of second and fourth high-frequency signals)
The third GR weighted addition circuit 1051 in FIG. 12 weights and adds the fourth high-frequency signal AC_4 and the second high-frequency signal AC_2, and generates a fifth high-frequency signal AC_5 according to the following equation (2-6). The third GR weighted addition circuit 1051 is the second GR weighted addition in the first embodiment shown in FIG. 8 except that AC_1 is AC_4, AC_3 is AC_5, and the name of the weighted addition coefficient is AC2UseRatio. A configuration similar to that of the circuit 132 may be used.

(Generation of final luminance signal)
Returning to FIG. 12, the adaptive OG signal generated by the adaptive interpolation circuit 1033 is also supplied to the gain circuit 1046. The gain circuit 1046 multiplies the adaptive OG signal by a gain of 0.6 and outputs the result.

  On the other hand, the RAW signal 100 is supplied to the gain circuit 1047 after the pixel signal values other than the R pixel are replaced with 0 by the 0 insertion circuit 1040 and interpolated by the V-LPF 1041 and the H-LPF 1042. The gain circuit 1047 multiplies the R signal by a gain of 0.3 times and outputs the result.

  Further, the RAW signal 100 is supplied to the gain circuit 1048 after the pixel signal values other than the B pixel are replaced with 0 by the 0 insertion circuit 1043, interpolated by the V-LPF 1044 and the H-LPF 1045. A gain circuit 1048 multiplies the B signal by a gain of 0.1 times and outputs the result.

The output signals of the gain circuits 1047 and 1048 are added by the adder circuit 1049, and further added by the adder circuit 1050 with the output signal of the gain circuit 1046 to become the luminance signal Yh.
Thereafter, the adder circuit 1052 adds the fifth high-frequency signal AC_5 and the luminance signal Yh to generate a final luminance signal YhFinal with edge enhancement.

  As described above, in the present embodiment, the weighted addition is performed so that the oblique region of the third high-frequency signal generated from only the G pixel signal is replaced with the first high-frequency signal generated from the RGB color signal. A fourth high-frequency signal is generated. Therefore, the diagonal resolution of the conventional OG signal is replaced with the SWY signal, and the resolution is greatly improved.

  Further, separately from the fourth high frequency signal, the second high frequency signal is generated using only the R pixel signal, and the weight of the second high frequency signal is increased as the redness indicating the scale of the subject is red. Then, the final high frequency signal is generated by weighted addition of both. For this reason, even in a red subject, the resolution of an oblique region is improved and a luminance signal with enhanced edge can be generated.

  Note that the method of weighted addition of the second high-frequency signal and the fourth high-frequency signal is not limited to the above method. By applying a higher gain to the second high-frequency signal as the degree of redness indicating the scale of the subject being red is higher, and adding the second high-frequency signal to which the gain has been added to the fourth high-frequency signal, A configuration may be adopted in which the second high frequency signal and the fourth high frequency signal are weighted and added. In some cases, such weighted addition can generate a better luminance signal from the output of an image sensor provided with a color filter with spectral characteristics such that a certain amount of output can be obtained from the G pixel even for a red subject.

(Other embodiments)
The above-described embodiment can also be realized in software by a computer of a system or apparatus (or CPU, MPU, etc.).
Therefore, the computer program itself supplied to the computer in order to implement the above-described embodiment by the computer also realizes the present invention. That is, the computer program itself for realizing the functions of the above-described embodiments is also one aspect of the present invention.

  The computer program for realizing the above-described embodiment may be in any form as long as it can be read by a computer. For example, it can be composed of object code, a program executed by an interpreter, script data supplied to the OS, but is not limited thereto.

  A computer program for realizing the above-described embodiment is supplied to a computer via a storage medium or wired / wireless communication. Examples of the storage medium for supplying the program include a magnetic storage medium such as a flexible disk, a hard disk, and a magnetic tape, an optical / magneto-optical storage medium such as an MO, CD, and DVD, and a nonvolatile semiconductor memory.

  As a computer program supply method using wired / wireless communication, there is a method of using a server on a computer network. In this case, a data file (program file) that can be a computer program forming the present invention is stored in the server. The program file may be an executable format or a source code.

Then, the program file is supplied by downloading to a client computer that has accessed the server. In this case, the program file can be divided into a plurality of segment files, and the segment files can be distributed and arranged on different servers.
That is, a server apparatus that provides a client computer with a program file for realizing the above-described embodiment is also one aspect of the present invention.

  In addition, a storage medium in which the computer program for realizing the above-described embodiment is encrypted and distributed is distributed, and key information for decrypting is supplied to a user who satisfies a predetermined condition, and the user's computer Installation may be allowed. The key information can be supplied by being downloaded from a homepage via the Internet, for example.

Further, the computer program for realizing the above-described embodiment may use an OS function already running on the computer.
Further, a part of the computer program for realizing the above-described embodiment may be configured by firmware such as an expansion board attached to the computer, or may be executed by a CPU provided in the expansion board. Good.

Claims (5)

A luminance signal generation device that generates an edge-enhanced luminance signal for each pixel from an image signal read from an image sensor having a primary color Bayer array color filter,
Luminance signal generating means for generating a luminance signal from the image signal;
First high-frequency signal generation means for generating a first high-frequency signal using image signals of pixels of all colors of the image sensor;
A second high-frequency signal generating unit configured to generate a second high-frequency signal using an image signal of one of red or blue pixels of the image sensor;
A calculating means for calculating an index indicating a measure of whether the pixel of interest is a color close to either one of the red or blue from the image signal;
Based on the index, the first high-frequency signal and the second high-frequency signal are set so that the ratio of the second high-frequency signal becomes higher as the pixel of interest is closer to either red or blue. A weighted addition means for generating a high-frequency signal for the pixel of interest,
The luminance signal generating means includes addition means for adding a high-frequency signal for the target pixel to the luminance signal generated for the target pixel to generate an edge-enhanced luminance signal for the target pixel. Luminance signal generating device. And third high-frequency signal generating means for generating a third high-frequency signal from the image signal of the green pixel of the image sensor;
Inclination detecting means for detecting a degree corresponding to the inclined line of the target pixel;
The higher the degree that the pixel of interest corresponds to the inclined line is, the higher the ratio of the third high-frequency signal is, and the first high-frequency signal and the third high-frequency signal are weighted and added. A second weighted addition means for generating a high-frequency signal;
The first weighted addition means generates a high-frequency signal for the pixel of interest by performing weighted addition of the fourth high-frequency signal and the second high-frequency signal instead of the first high-frequency signal. The luminance signal generation apparatus according to claim 1. An image sensor having a primary color Bayer array color filter;
An imaging apparatus comprising: the luminance signal generation apparatus according to claim 1. A luminance signal generation method for generating an edge-enhanced luminance signal for each pixel from an image signal read from an image sensor having a primary color Bayer array color filter,
A luminance signal generating means for generating a luminance signal from the image signal;
A first high-frequency signal generating unit that generates a first high-frequency signal using image signals of pixels of all colors of the image sensor;
A second high-frequency signal generating step, wherein the second high-frequency signal generating means generates a second high-frequency signal using an image signal of one of the red and blue pixels of the image sensor;
A calculating step in which a calculating means calculates an index indicating a measure of whether the pixel of interest is a color close to either one of the red or blue from the image signal;
Based on the index, the first weighted addition means is configured to increase the ratio of the second high-frequency signal so that the pixel of interest has a color closer to either red or blue. A weighted addition of a signal and the second high-frequency signal to generate a high-frequency signal for the pixel of interest;
The adding means includes an adding step of adding a high-frequency signal for the target pixel to the luminance signal generated by the luminance signal generation step for the target pixel to generate an edge-enhanced luminance signal for the target pixel. A luminance signal generation method characterized by the above.   The program for making a computer perform each process of the luminance signal production | generation method of Claim 4.

Images