JP4859220B2 - Luminance signal creation device, brightness signal creation method, and imaging device - Google Patents

Description

  The present invention relates to a luminance signal generation apparatus, a luminance signal generation method, and an imaging apparatus. In particular, the present invention relates to an apparatus and method for creating a luminance signal from a signal obtained by an image sensor using a Bayer color filter, and an imaging apparatus using them.

  In order to create a color image using an image sensor that can detect the amount of light, such as a CCD image sensor or a CMOS image sensor, a configuration in which light transmitted through a color filter is incident on the image sensor is generally used.

  There are various types of color filters depending on the type of color used and the arrangement of colors assigned to each pixel. The color types are primary colors (red, green, blue) or complementary colors (cyan, magenta, yellow). For the color arrangement, the Bayer arrangement is widely used.

  FIG. 18 is a diagram showing one unit of the primary color Bayer array. Actually, the same arrangement is repeated according to the number of pixels of the image sensor. R is red, G1 and G2 are green, and B is blue.

  FIG. 19 is a diagram for explaining an OutOfGreen method for creating a luminance signal from only a green (G) signal, among conventional methods for creating a luminance signal using the color filters of the primary color Bayer array shown in FIG.

  First, in the RAW signal 1900 obtained by digitizing the output of the image sensor, values other than the G pixel are set to 0 (1901). Next, a low-pass filter (V-LPF) process (1902) for limiting the vertical band and a low-pass filter (H-LPF) process (1903) for limiting the horizontal band are performed to obtain a luminance signal. Hereinafter, a luminance signal obtained by the OutOfGreen method is referred to as an OG signal.

As another example of a conventional method for creating a luminance signal using a color filter with a primary color Bayer array shown in FIG. 18, there is a SWY method in which a luminance signal is created using all RGB pixels.
FIG. 20 is a diagram for explaining the SWY method.
The SWY method is a method for obtaining a luminance signal without performing the process (1901) of setting values other than G pixels to 0 in the OutOfGreen method. Hereinafter, a luminance signal obtained by the SWY method is referred to as a SWY signal.

FIG. 21 is a diagram showing resolvable spatial frequency characteristics of the OG signal and the SWY signal.
The x-axis indicates the frequency space in the horizontal (H) direction of the subject, the y-axis indicates the frequency space in the vertical (V) direction, and the spatial frequency increases as the distance from the origin increases. Since the OG signal creates a luminance signal only from the G signal, the resolution limit in the horizontal and vertical directions is equal to the Nyquist frequency (on the axis, π / 2). However, since there are lines in which no pixels exist in the oblique direction, the limit resolution frequency in the oblique direction is lower than that in the horizontal and vertical directions. As a result, the inside of the rhombus region 2100 in the figure becomes a resolvable spatial frequency.

  On the other hand, since the SWY signal is generated using all pixels, when the subject is achromatic, the square frequency 2101 as shown in the figure has a spatial frequency that can be resolved. However, in a red subject, for example, luminance signals are not output from pixels other than the R pixel, so that the resolution is only in a quarter range 2102 compared to an achromatic subject.

  Patent Document 1 proposes a method of replacing the oblique region 2103 of the OG signal in FIG. 21 with the SWY signal. However, since the resolution limit frequency of the SWY signal is lowered for the chromatic subject, the OG signal is replaced with the SWY signal only when the oblique region is an achromatic subject. Then, an edge emphasis component is detected using the created luminance signal and added to the luminance signal to create a final luminance signal.

  On the other hand, Patent Document 2 proposes a method of creating a luminance signal by using a plurality of interpolation filters prepared in advance according to the angle of the subject, and then performing edge enhancement to create a final luminance signal.

  Patent Document 3 proposes a method of weighted addition of the OG signal and the SWY signal according to the hue and saturation of the subject, as in Patent Document 1. Specifically, the SWY signal is used for the low saturation subject, the SWY signal is used for the high saturation subject and the Mg (magenta) and G (green) subjects, and the OG signal is used for the other chromatic subjects. Thereafter, an edge enhancement component is calculated from the MIX signal and added to a separately created luminance signal to create a final luminance signal.

JP 2003-348609 A JP 2003-196649 A Japanese Patent No. 3699873

  However, when the method described in Patent Document 1 is applied to a signal obtained using a color filter with a primary color Bayer array, using the SWY signal created by the method shown in FIG. There was an adverse effect of generating a resolution signal (folding distortion). This is considered to be due to the band limitation in the 45 ° and 135 ° directions being insufficient because the band limitation is performed only in the H direction and the V direction in the method of FIG.

  Further, since the OG signal is used for the chromatic subject, the resolution in the oblique direction for the chromatic subject is not improved. Furthermore, when an edge emphasis signal is created with a luminance signal after part of the OG signal is replaced with the SWY signal, the switching portion between the OG signal and the SWY signal is emphasized, and an unnatural texture is likely to occur.

  In the method of Patent Document 2, it is necessary to prepare and hold a plurality of interpolation filters according to the accuracy of the subject in advance. When the color configuration is, for example, NTSC-RGB space, it is necessary to prepare an interpolation filter that maintains the luminance signal configuration ratio R: G: B = 3: 6: 1. As a result, the filter coefficient is limited, and there is a problem that optimum filter processing cannot be performed for the angle of the subject. Similarly to Patent Document 1, since edge enhancement is performed after luminance signals calculated by a plurality of interpolation methods are mixed, subtle switching of luminance signals with different calculation methods is emphasized.

  Further, in the technique of Patent Document 3, since the subject of Mg and G uses the SWY signal instead of the OG signal, the resolution of subjects other than Mg and G among the chromatic subjects is not improved. Furthermore, as in the methods of Patent Document 1 and Patent Document 2, edge enhancement is performed after weighted addition of luminance signals calculated by a plurality of methods, so that the switching portion of the method is emphasized.

  The present invention has been made in view of such problems of the prior art. An object of the present invention is to provide a method and apparatus for creating a luminance signal capable of suppressing the occurrence of aliasing distortion in an oblique direction component.

The above-described object is a luminance signal generation device that generates a luminance signal from a signal of an object obtained by an image sensor having a color filter with a primary color Bayer array, and is a first image signal generated from only a G pixel signal of the image sensor . A first high-frequency signal generating means for generating a first high-frequency signal using one luminance signal, and a second high-frequency signal using a second luminance signal generated from signals of all the color pixels of the image sensor Second high-frequency signal creating means, weighted addition means for creating a third high-frequency signal by weighted addition of the first high-frequency signal and the second high-frequency signal according to the spatial frequency of the signal of the subject , adding the luminance signal and the third high-frequency signal, and an adding means for creating a third luminance signal and chromatic, weighted addition means, the first luminance signal, a band-pass filter 45 degree direction in the spatial frequency domain No. that processed The higher the absolute value of the difference between the first band-pass signal and the second band-pass signal obtained by subjecting the first luminance signal to band-pass filtering in the direction of 135 degrees in the spatial frequency domain, the larger the third high-frequency signal This is achieved by a luminance signal generating device that performs weighted addition for increasing the ratio of the second high-frequency signal .

Further, the above object is, from the signals of the subject obtained by the image pickup element having a primary color Bayer pattern type of color filter array, a luminance signal creating apparatus for creating a luminance signal of each pixel of the image sensor, G of the image sensor First high-frequency signal generating means for generating a first high-frequency signal using a first luminance signal generated from only pixels, and the horizontal, vertical, and 45 degree direction bands of all color pixel signals of the image sensor are limited. Second high-frequency signal generating means for generating a second high-frequency signal by using the second luminance signal, and a third that limits the horizontal, vertical, and 135-degree band of the signals of all the color pixels of the image sensor . Third high-frequency signal generating means for generating a third high-frequency signal using the luminance signal, selection means for selecting one of the second and third high-frequency signals, the first high-frequency signal, the second and second Of the three high-frequency signals, the selection means The one and the weighted sum-option, a weighted addition means for generating a fourth high-frequency signal, adding means first adds the luminance signal and the fourth high-frequency signal, and outputs a fourth luminance signal, have a, selection means, second greater than the absolute value of the signal absolute value of the first signal with a limited band of 45 degree direction of the luminance signal is band-limits the 135-degree direction of the first luminance signal This is also achieved by a luminance signal generating apparatus characterized by selecting a third high frequency signal if it is small .

The above-described object is a luminance signal generation method for generating a luminance signal from a signal of an object obtained by an image sensor having a primary color Bayer array color filter, which is generated only from a signal of a G pixel of the image sensor. A first high-frequency signal generating step of generating a first high-frequency signal using the first luminance signal, and a second high-frequency signal using the second luminance signal generated from signals of all the color pixels of the image sensor A second high-frequency signal generating step for generating the first high-frequency signal, a weighted addition step for generating a third high-frequency signal by weighted addition of the first high-frequency signal and the second high-frequency signal according to the spatial frequency of the signal of the subject , plus one of the luminance signal and the third high-frequency signal, and an adding step of generating a third luminance signal and chromatic, weighted addition process, the first luminance signal, 45 degree direction of the band in the spatial frequency domain Perform path filtering The larger the absolute value of the difference between the first bandpass signal and the second luminance band signal obtained by subjecting the first luminance signal to bandpass filtering in the direction of 135 degrees in the spatial frequency domain, the third This can also be achieved by a luminance signal generation method characterized by performing weighted addition for increasing the ratio of the second high-frequency signal to the high-frequency signal .

Another object of the present invention is to provide a luminance signal generation method for generating a luminance signal from a signal of a subject obtained by an image sensor having a primary color Bayer array color filter, which is generated only from G pixels of the image sensor . A first high-frequency signal generating step for generating a first high-frequency signal using one luminance signal, and a second luminance signal in which the horizontal, vertical, and 45-degree band bands of all color pixel signals of the image sensor are limited And a second high-frequency signal generating step for generating a second high-frequency signal using the first and third luminance signals in which the horizontal, vertical, and 135-degree bandwidths of the signals of all color pixels of the image sensor are limited. A third high-frequency signal creating step for creating three high-frequency signals, a selection step for selecting one of the second and third high-frequency signals, the first high-frequency signal, and the second and third high-frequency signals. Add one selected by the selection process Adding to the weighted addition step of generating a fourth high-frequency signal, a first adds the luminance signal and the fourth high-frequency signal, possess an adding step of outputting a fourth luminance signal, selection process If the absolute value of the first luminance signal with the band limited in the 45 degree direction is larger than the absolute value of the first luminance signal with the band limited in the 135 degree direction, the second high frequency signal is smaller. It is also achieved by a luminance signal generation method characterized by selecting a third high-frequency signal .

  With such a configuration, according to the present invention, it is possible to suppress the aliasing distortion of the luminance signal in the oblique direction.

Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the drawings.
FIG. 1 is a block diagram showing a configuration example of a luminance signal generating apparatus according to the first embodiment of the present invention. The luminance signal generation device of the present embodiment can be suitably realized in a signal processing circuit that performs signal processing such as so-called development processing in an imaging device that uses an imaging device including a color filter with a primary color Bayer array.

  An image signal (hereinafter referred to as a RAW signal 100) read out from an image sensor (not shown) and converted into digital data and subjected to white balance processing is input to the OG signal creation unit 101 and the SWY signal creation unit 103. The OG signal creation unit 101 creates a first luminance signal from the RAW signal 100. The first luminance signal is processed by the first high-frequency (edge) component detection processing unit 102 to create a first high-frequency signal. The first luminance signal is also supplied to the second high frequency component detection processing unit 104.

  Further, the SWY signal creation unit 103 creates a second luminance signal from the RAW signal. The second high frequency component detection processing unit 104 creates a second high frequency signal from the second luminance signal.

  The weighted addition processing unit 105 adds the first high-frequency signal and the second high-frequency signal to create a third high-frequency signal. Further, the adder 106 adds the first luminance signal (OG signal) and the third high-frequency signal to create a final luminance signal (third luminance signal).

Hereinafter, each signal processing unit will be described in detail.
(Process for creating first high-frequency signal (AC_OG signal))
FIG. 2 is a diagram illustrating a configuration example of the OG signal generation unit 101 and the first high-frequency component detection processing unit 102 in FIG.
The RAW signal 100 is input to the Zero insertion circuit 201, the DiffH signal generation circuit 206, and the DiffV signal generation circuit 207, respectively, inside the OG signal generation unit 101.

The zero insertion circuit 201 creates a zero insertion G signal by replacing signals other than G in the raw signal 100 with 0. The vertical low-pass filter (V-LPF) 202 and the first horizontal low-pass filter (H-LPF) 204 limit the vertical and horizontal bands of the zero insertion G signal and output them as Gv and Gh signals, respectively. To do.
The second horizontal low-pass filter (H-LPF) 203 limits the horizontal band of the Gv signal and outputs the Ghv signal.

In this way, (1) the Gv signal in which the vertical band is limited, (2) the Gh signal in which the horizontal band is limited, and (3) the vertical and horizontal bands of the Zero insertion G signal are limited. Ghv signals are generated and input to the weighted addition circuit 205.
On the other hand, a DiffH signal generation circuit (DiffH) 206 and a DiffV signal generation circuit (DiffV) 207 generate DiffH and DiffV signals from the RAW signal 100.

FIG. 3 is a diagram for explaining Diff signal generation processing. In this embodiment, DiffH and DiffV signals in the pixel of interest P22 are calculated according to the following equations (1) and (2).
DiffH = | P21−P23 | + | 2 × P22−P20−P24 | (Formula 1)
DiffV = | P12−P32 | + | 2 × P22−P02−P42 | (Formula 2)
In Expressions 1 and 2, P12 to P42 are signal values of corresponding pixels in FIG.

  When the subject is a vertical stripe, the DiffH signal is large, and when it is a horizontal stripe, the value of the DiffV signal is large. The subtracter 208 subtracts the DiffV signal from the DiffH signal and outputs the DiffHV signal. A weighted addition coefficient calculation circuit (Tsig) 209 calculates a weighted addition coefficient based on the DiffHV signal and supplies it to the weighted addition circuit 205.

FIG. 4 is a diagram illustrating an input / output characteristic example of the weighted addition coefficient calculation circuit 209.
The horizontal axis indicates the DiffHV signal, and the vertical axis indicates the value of the weighted addition coefficient Tsig. When DiffHV ≦ Th0, since DiffV is larger than DiffH, it is determined that the region is a horizontal stripe region. In this case, a Tsig value (Tsig = −128) in which the weighted addition circuit 205 uses the Gh signal 100% is output.

  The section of Th1 <DiffHV <Th2 is determined as an oblique area because DiffV and DiffH are close to each other. In this section, the Tsig value (Tsig = 0) at which the weighted addition circuit 205 uses the Ghv signal 100% is output.

  Further, when DiffHV ≧ Th3, since DiffH is larger than DiffV, it is determined to be a vertical stripe region. In this case, a Tsig value (Tsig = 128) using 100% of the Gv signal is output.

  In the section of Th0 ≦ DiffHV ≦ Th1, Tsig determined so as to change linearly from −128 to 0 is output. Similarly, Tsig determined so as to change linearly from 0 to 128 is output in the section of Th2 ≦ DiffHV ≦ Th3.

  The weighted addition circuit 205 weights and adds the Gh signal, the Gv signal, and the Ghv signal according to the following Expression 3 and Expression 4 based on the Tsig value determined in this way, and the first luminance signal (DC_OG signal) 210. Create

Ｔｓｉｇ＜０のとき、

When Tsig> = 0,

When Tsig <0,

  The first luminance signal that is the output of the weighted addition circuit 205 is supplied to the adder 106 (FIG. 1). The first luminance signal is also supplied to the first high frequency component detection processing unit 102.

  In the first high-frequency component detection processing unit 102, the first luminance signal is input to the horizontal low-pass filter (H-LPF) 211 and the vertical low-pass filter (V-LPF) 213.

  The first luminance signal is subjected to gain adjustment by a horizontal gain circuit (H_Gain) 212 after the horizontal band is limited by the H-LPF 211. The output of H_Gain 212 is a horizontal edge signal.

Similarly, the first luminance signal is subjected to the gain adjustment by the vertical gain circuit (V_Gain) 214 after the vertical band is limited by the V-LPF 213. The output of V_Gain 214 is a vertical edge signal.
Hereinafter, an edge signal may be described as an AC component.

  The adder 215 adds the horizontal and horizontal AC components and outputs the result as a first high-frequency signal (AC_OG signal). The first high frequency signal is supplied to the weighted addition processing unit 105 (FIG. 1).

(Process for creating second high-frequency signal (AC_SWY signal))
FIG. 5 is a diagram illustrating a configuration example of the SWY signal creation unit 103 and the second high-frequency component detection processing unit 104 in FIG.
(1) SWY signal generation as a base The RAW signal 100 is first generated in the vertical and horizontal bands by the vertical low-pass filter (V-LPF) 501 and horizontal low-pass filter (H-LPF) 502 of the SWY signal generation unit 103. And a general-purpose SWY luminance signal is created.

(2) Creation of AC_SWY signal The primary color Bayer array signal has good color separation of the three primary colors of RGB due to the spectral sensitivity characteristics of the filter. Therefore, if the band limiting filter is not applied to the diagonal direction in addition to the horizontal and vertical band limiting filters 501 and 502 used for generating the general-purpose SWY luminance signal, the luminance aliasing distortion cannot be sufficiently suppressed. Therefore, in this embodiment, in addition to the low-pass filter that limits the horizontal and vertical bands, an oblique LPF that limits the 45-degree and 135-degree bands is provided. Then, one of the two types of band-limited signals is selected based on the angle information of the subject.

  The general-purpose SWY signal output from the H-LPF 502 is limited in the 45-degree band by the first two-dimensional filter (D45-LPF) 503. Thereafter, the two-dimensional bandpass filter (D135-BPF) 504 for detecting the edge of the 45-degree line and the gain circuit 505 generate an edge enhancement signal (AC_45) for the oblique 45-degree line.

  On the other hand, the band of the general-purpose SWY signal in the direction of 135 degrees is limited by the second two-dimensional filter (D135-LPF) 506. Thereafter, a two-dimensional bandpass filter (D45-BPF) 507 for detecting an edge of the 135 ° line and a gain circuit 508 generate an edge enhancement signal (AC_135) for the diagonal 135 ° line.

  One of the AC_45 signal and the AC_135 signal is selected by the selector 509 in accordance with the accuracy determination signal (angleSigD) output from the angle determination signal generation circuit 510. Then, it is supplied to the weighted addition processing unit 105 as a second high-frequency signal (AC_AWY) that emphasizes the edge in the oblique direction.

FIG. 6 is a block diagram illustrating a configuration example of the angle determination signal generation circuit 510.
A DC_OG signal created by the OG signal creation unit 101 from the RAW signal 100 is input to the angle determination signal creation circuit 510. A 45 degree line detection signal is created from the DC_OG signal by a band pass filter 601 (D135-BPF) that emphasizes the 45 degree line and an absolute value circuit (ABS) 602. Also, a 135 degree line detection signal is created from the DC_OG signal by a band pass filter (D45-BPF) 603 that emphasizes the 135 degree line and an absolute value circuit (ABS) 604.

  The subtracter 605 subtracts the 135 degree line detection signal output from the absolute value circuit 604 from the 45 degree line detection signal output from the absolute value circuit 602. The subtraction result is supplied to the selector 509 as an angle determination signal (angleSigD). The selector 509 selects and outputs the AC_45 signal if the angle determination signal (angleSigD) is positive, and the AC_135 signal if it is negative.

(Third luminance signal creation process)
The weighted addition processing unit 105 performs weighted addition on the first high-frequency signal (AC_OG) and the second high-frequency signal (AC_SWY) created as described above according to the following Equation 5, and obtains a third high-frequency signal (ACsig). ).

ここで、SWYUseRatioは加重加算係数である。

Here, SWYUseRatio is a weighted addition coefficient.

  FIG. 22 is a diagram illustrating a configuration example of the weighted addition processing unit 105. In the present embodiment, the weighted addition coefficient SWYUseRatio is calculated based on the first to fifth weighted addition coefficients SWYUseRatio_1 to SWYUseRatio_5.

  The SWYUseRatio_1 calculation circuit 1054 to SWYUseRatio_5 calculation circuit 1058 calculate first to fifth weighted addition coefficients SWYUseRatio_1 to SWYUseRatio_5. The SWYUseRatio calculation circuit 1059 calculates the final weighted addition coefficient SWYUseRatio from the first to fifth weighted addition coefficients SWYUseRatio_1 to SWYUseRatio_5. Then, the SWYUseRatio calculation circuit 1059 gives SWYUseRatio / 128 and (1−SWYUseRatio / 128) to the multipliers 1052 and 1051 as coefficients. Multipliers 1052 and 1051 multiply the AC_OG signal and the AC_SWY signal by a coefficient, respectively, and output the result. The outputs of the multipliers 1052 and 1051 are added by the adder 1053, and the addition result is output to the adder 106 for addition with the DC_OG signal.

Hereinafter, a method for calculating the weighted addition coefficient SWYUseRatio will be described.
(1) Calculation of SWYUseRatio_1 FIG. 17 is a diagram for explaining how the luminance signal generation apparatus according to the present embodiment uses the OG signal and the SWY signal in the same frequency space as in FIG. Here, a region where the first high-frequency signal (AC_OG) is affected by the aliasing distortion is an oblique region 1700. Therefore, if the region 1700 is replaced with the second high-frequency signal (AC_SWY), a high-frequency signal without aliasing distortion can be created. Therefore, the weighted addition coefficient can be calculated by using the angle determination signal used when the AC_SWY signal is created.

  The angle discrimination signal angleSigD created by the angle discrimination signal creation circuit 510 at the time of AC_SWY calculation indicates a 45-degree line area if positive, and a 135-degree line area if negative. It has a feature close to the 0 degree and 90 degree lines.

  A configuration example of the SWYUseRatio_1 calculation circuit 1054 is shown in FIG. The SWYUseRatio_1 calculation circuit 1054 obtains the absolute value of the angle determination signal angleSigD acquired from the first high-frequency component detection processing unit 102 by the absolute value circuit (abs) 701. Then, SWYUseRatio_1 is calculated using CalSwyUse1 702 having input / output characteristics as shown in FIG. CalSwyUse1 702 may be an arithmetic circuit or a table.

  In FIG. 7, the section where the absolute value of the angle determination signal angleSig is from 0 to Th3 is close to the 0 degree and 90 degree lines because the value is small, and SWYUseRatio_1 = 0 is output. On the other hand, the section where the absolute value of the angle discrimination signal angleSig is equal to or greater than Th4 is a diagonal line because the value is large, and SWYUseRatio_1 = 128 is output to replace the second high-frequency signal. Here, it is assumed that 128 = 100%. In the interval from the absolute value Th3 to Th4, a value obtained by linear interpolation in the range of 0 to 128 is output.

(2) Calculation of SWYUseRatio_2 The horizontal and vertical region 1701 in FIG. 17 uses the first high-frequency signal (AC_OG). This is because the OG signal is an adaptive OG signal that limits the band in the horizontal or vertical direction, so that the resolution is improved over the SWY signal that limits the band in both the horizontal and vertical directions.

Hereinafter, as one of the methods for discriminating the horizontal and vertical regions, a method of using the pixel difference value and calculating SWYUseRatio_2 based on this size will be described.
The absolute value of the DiffHV signal calculated by the first high-frequency component detection processing unit 102 can be used as the magnitude of the pixel difference value. As described above, when the subject is a vertical stripe, the value of the DiffH signal is large and the DiffV signal is zero. When the subject is a horizontal stripe, the value of the DiffV signal is large and the DiffH signal is 0. Therefore, it can be seen that the larger the absolute value of the DiffHV signal, which is the difference between DiffH and DiffV, is closer to the horizontal or vertical line.

  A configuration example of the SWYUseRatio_2 calculation circuit 1055 is shown in FIG. The SWYUseRatio_2 calculation circuit 1055 obtains the absolute value of the DiffHV signal acquired from the first high-frequency component detection processing unit 102 by the absolute value circuit (abs) 804. Then, SWYUseRatio_2 is calculated using CalSwyUse2 805 having input / output characteristics as shown in FIG. CalSwyUse2 805 may be an arithmetic circuit or a table.

  When the absolute value of the input signal DiffHV is from 0 to Th5, the value is small, so it is a diagonal line, and SWYUserRatio_2 = 256 is output. Th6 and above are completely horizontal and vertical lines, and SWYUserRatio_2 = 0 is output. Also, linearly interpolated values are output in the section from Th5 to Th6.

(3) Calculation of SWYUseRatio_3 The first high-frequency signal (AC_OG) is preferably used for the region (false color region) 1702 in the vicinity of horizontal and vertical Nyquist in FIG. 17 for the same reason as (2). The Nyquist region can be determined by detecting a phase shift between the G1 and G2 signals.

A configuration example of the SWYUseRatio_3 calculation circuit 1056 is shown in FIG.
In the RAW signal 100, 0 is inserted into signals other than G1 in the Zero insertion circuit 1001, and 0 is inserted into signals other than G2 in the Zero insertion circuit 1002. The interpolation value is inserted into the pixels set to 0 by the low-pass filter circuit (1002, 1003, 1005, 1006) for horizontal and vertical interpolation.

  Next, the outputs of the HLPFs 1003 and 1006 are subtracted by a subtractor 1007 and an absolute value is calculated by an absolute value circuit 1008. This absolute difference value is sigDiffG for determining the region.

  Then, SWYUseRatio_3 is calculated using CalSwyUse3 1009 having input / output characteristics as shown in FIG. CalSwyUse3 1009 may be an arithmetic circuit or a table.

  Since the input signal sigDiffG has a small value from 0 to Th7, it is a diagonal line, and SWYUserRatio_3 = 256 is output. On the other hand, Th8 and above are completely horizontal and vertical lines, and SWYUserRatio_3 = 0 is output. Further, linearly interpolated values are output in the section from Th7 to Th8.

(4) Calculation of SWYUseRatio — 4 It is preferable to use the first high frequency signal (AC_OG) for the low frequency region 1703 in FIG. The low frequency region can be detected by applying a filter that detects the low frequency region to the RAW signal.

A configuration example of the SWYUseRatio_4 calculation circuit 1057 is shown in FIG.
The RAW signal 100 is limited in the vertical and horizontal bands by the low-pass filters 1201 and 1202. After that, a horizontal bandpass filter (H-BPF) 1203 and a vertical bandpass filter (V-BPF) 1204 are applied separately. Then, the subtracter 1205 subtracts the output of the V-BPF 1204 from the output of the H-BPF 1203, and the absolute value circuit 1206 takes the absolute value and outputs it as a low-frequency detection signal (LowF signal).

  Then, SWYUseRatio — 4 is calculated using CalSwyUse4 1207 having input / output characteristics as shown in FIG. CalSwyUse4 1207 may be an arithmetic circuit or a table.

  Since the value from 0 to Th9 of the input signal LowF is small, it is a high frequency region, and SWYUserRatio_4 = 256 is output. On the other hand, Th10 or higher is a low frequency region, and SWYUserRatio_4 = 0 is output. Further, linearly interpolated values are output in the section from Th9 to Th10.

(5) Calculation of SWYUseRatio_5 The red subject is likely to output 0 other than the R pixel. For this reason, when a luminance signal is created by the SWY method, aliasing distortion occurs at a frequency equal to or higher than half the Nyquist frequency. Therefore, the second high-frequency signal (AC_SWY) is a signal with much aliasing distortion for a red subject. Therefore, the color of the subject is determined, and when the color is red, the first high-frequency signal (AC_OG) is used even in the oblique region 1700.

A configuration example of the SWYUseRatio_5 calculation circuit 1058 is shown in FIG.
The color interpolation circuit 1401 performs pixel color interpolation processing by a predetermined method from the RAW signal 100. A color difference signal creation circuit 1402 creates a color difference signal (R-Y signal and BY signal) from the RGB values of the color interpolation result. The hue angle detection circuit 1403 obtains the hue angle signal θ from the color difference signal using the following formula 6.

そして、図１５に示すような入出力特性を有するCalSwyUse5 １４０４を用い、SWYUseRatio_5を算出する。CalSwyUse5 １４０４は、演算回路であっても、テーブルであっても良い。

Then, SWYUseRatio_5 is calculated using CalSwyUse5 1404 having input / output characteristics as shown in FIG. CalSwyUse5 1404 may be an arithmetic circuit or a table.

  When the input signal θ is from 0 to Th11, Th14 and above are other than red, and SWYUserRatio_5 = 256 is output. On the other hand, Th12 to Th13 are red, and SWYUserRatio_5 = 0 is output. Also, linearly interpolated values are output in the section from Th11 to Th12 and the section from Th13 to Th14.

(6) Calculation of SWYUseRatio The final SWYUseRatio is calculated by the SWYUseRatio calculation circuit 1059 from SWYUseRatio_1 to SWYUseRatio_5 calculated as described above.

A configuration example of the SWYUseRatio calculation circuit 1059 is shown in FIG.
After SWYUseRatio_1 (1600) and SWYUseRatio_2 (1601) are multiplied by multiplier 1605, shifter 1606 performs an 8-bit shift operation in the right direction. The output value of the multiplier 1605 is reduced to 1/256 by the shifter 1606.

  SWYUseRatio_3 (1602), SWYUseRatio_4 (1603), and SWYUseRatio_5 (1604) are also multiplied by multipliers 1607, 1609, and 1611, respectively, and then 1/256 by shifters 1608, 1610, and 1612.

  Thereby, SWYUseRatio having a value of 0 to 128 is obtained. Further, SWYUseRatio is 1/128 by divider 1613 and provided to multiplier 1052. Further, (1-SWYUseRatio) is obtained by the subtractor 1614 and is given to the multiplier 1051.

  Multipliers 1051 and 1052 multiply the AC_OG signal and the AC_SWY signal by coefficients, respectively, and output the result. By adding these outputs by the adder 1053, ACsig (third high-frequency signal) of Expression (5) is obtained.

  As described above, according to the present embodiment, the first high-frequency signal created from the OG signal and the second high-frequency signal created from the SWY signal are weighted and added to the DC_OG signal that is the base luminance signal. To do. Therefore, unlike the prior art, the switching point of the luminance signal having a different calculation method is not enhanced by the edge enhancement processing performed in the subsequent stage.

  Furthermore, in the present embodiment, the second high-frequency signal is generated using the SWY signal that is band-limited not only in the vertical and horizontal directions but also in the 45-degree and 135-degree directions. As a result, it is possible to suppress the luminance reversal in the oblique direction that occurs with the conventional SWY signal.

  Furthermore, in the present embodiment, an oblique line discrimination signal that determines the usage rate of the first high-frequency signal and the second high-frequency signal is detected using the OG signal. Therefore, for example, an OG signal is used in a red subject area even in an oblique region, and aliasing distortion that is likely to occur in a red subject when the SWY signal is used in the oblique region can be suppressed.

Furthermore, in this embodiment, a signal with high resolution can be obtained by using the first high-frequency signal (AC_OG) in the horizontal and vertical Nyquist frequency regions.
Furthermore, in this embodiment, it is possible to obtain a signal with a high resolution feeling by using the first high-frequency signal (AC_OG) in the low-frequency region.
Furthermore, in the present embodiment, since the first high-frequency signal (AC_OG) is used for a red subject that is not good at the SWY signal, aliasing distortion with respect to the red subject can be suppressed.

(Other embodiments)
The above-described embodiment can also be realized in software by a computer of a system or apparatus (or CPU, MPU, etc.).
Therefore, the computer program itself supplied to the computer in order to implement the above-described embodiment by the computer also realizes the present invention. That is, the computer program itself for realizing the functions of the above-described embodiments is also one aspect of the present invention.

  The computer program for realizing the above-described embodiment may be in any form as long as it can be read by a computer. For example, it can be composed of object code, a program executed by an interpreter, script data supplied to the OS, but is not limited thereto.

  A computer program for realizing the above-described embodiment is supplied to a computer via a storage medium or wired / wireless communication. Examples of the storage medium for supplying the program include a magnetic storage medium such as a flexible disk, a hard disk, and a magnetic tape, an optical / magneto-optical storage medium such as an MO, CD, and DVD, and a nonvolatile semiconductor memory.

  As a computer program supply method using wired / wireless communication, there is a method of using a server on a computer network. In this case, a data file (program file) that can be a computer program forming the present invention is stored in the server. The program file may be an executable format or a source code.

Then, the program file is supplied by downloading to a client computer that has accessed the server. In this case, the program file can be divided into a plurality of segment files, and the segment files can be distributed and arranged on different servers.
That is, a server apparatus that provides a client computer with a program file for realizing the above-described embodiment is also one aspect of the present invention.

  In addition, a storage medium in which the computer program for realizing the above-described embodiment is encrypted and distributed is distributed, and key information for decrypting is supplied to a user who satisfies a predetermined condition, and the user's computer Installation may be allowed. The key information can be supplied by being downloaded from a homepage via the Internet, for example.

Further, the computer program for realizing the above-described embodiment may use an OS function already running on the computer.
Further, a part of the computer program for realizing the above-described embodiment may be configured by firmware such as an expansion board attached to the computer, or may be executed by a CPU provided in the expansion board. Good.

It is a block diagram which shows the structural example of the luminance signal production apparatus which concerns on embodiment of this invention. FIG. 2 is a diagram illustrating a configuration example of an OG signal creation unit 101 and a first high frequency component detection processing unit 102 in FIG. 1. It is a figure explaining the luminance signal production | generation apparatus Diff signal production | generation process which concerns on embodiment of this invention. It is a figure which shows the input-output characteristic example of the weighted addition coefficient calculation circuit 209 in the luminance signal production apparatus which concerns on embodiment of this invention. FIG. 2 is a diagram illustrating a configuration example of a SWY signal creation unit 103 and a second high frequency component detection processing unit 104 in FIG. 1. It is a block diagram which shows the structural example of the angle discrimination | determination signal preparation circuit 510 in the luminance signal preparation apparatus concerning embodiment of this invention. It is a figure which shows the angle discrimination | determination signal-SWYUseRatio_1 input-output characteristic of SWYUseRatio_1 calculation circuit 1054 in the luminance signal production apparatus which concerns on embodiment of this invention. It is a figure which shows the structural example of SWYUseRatio_2 calculation circuit 1055 in the luminance signal production apparatus which concerns on embodiment of this invention. It is a figure which shows the DiffHV signal-SWYUseRatio_2 input-output characteristic of SWYUseRatio_2 calculation circuit 1055 in the luminance signal production apparatus which concerns on embodiment of this invention. It is a figure which shows the structural example of SWYUseRatio_3 calculation circuit 1056 in the luminance signal production apparatus which concerns on embodiment of this invention. It is a figure which shows the sigDiffG-SWYUseRatio_3 input-output characteristic of SWYUseRatio_3 calculation circuit 1056 in the luminance signal production apparatus concerning the embodiment of the present invention. It is a figure which shows the structural example of SWYUseRatio_4 calculation circuit 1057 in the luminance signal production apparatus which concerns on embodiment of this invention. It is a figure which shows the LowF signal-SWYUseRatio_4 input-output characteristic of SWYUseRatio_4 calculation circuit 1057 in the luminance signal production apparatus which concerns on embodiment of this invention. It is a figure which shows the structural example of SWYUseRatio_5 calculation circuit 1058 in the luminance signal production apparatus which concerns on embodiment of this invention. It is a figure which shows the hue angle signal-SWYUseRatio_5 input-output characteristic of SWYUseRatio_5 calculation circuit 1058 in the luminance signal production apparatus which concerns on embodiment of this invention. It is a figure which shows the structural example of SWYUseRatio calculation circuit 1059 in the luminance signal production apparatus which concerns on embodiment of this invention. It is a figure explaining how the luminance signal production apparatus which concerns on embodiment of this invention uses an OG signal and a SWY signal in the frequency space similar to FIG. It is a figure which shows 1 unit of a primary color Bayer arrangement | sequence. It is a figure explaining an OutOfGreen system. It is a figure explaining a SWY system. It is a figure which shows the spatial frequency characteristic which can resolve OG signal and SWY signal. It is a figure which shows the structural example of the weighted addition process part 105 in the luminance signal production apparatus which concerns on embodiment of this invention. It is a figure which shows the structural example of SWYUseRatio_1 calculation circuit 1054 in the luminance signal production apparatus which concerns on embodiment of this invention.

Claims (14)

A luminance signal generating device that generates a luminance signal from a signal of a subject obtained by an image sensor having a primary color Bayer array color filter,
First high-frequency signal creating means for creating a first high-frequency signal using a first luminance signal created only from the signal of the G pixel of the image sensor ;
Second high-frequency signal creating means for creating a second high-frequency signal using a second luminance signal created from signals of all the color pixels of the image sensor ;
Weighted addition means for creating a third high-frequency signal by weighted addition of the first high-frequency signal and the second high-frequency signal according to the spatial frequency of the signal of the subject ;
Adding the first luminance signal and the third high-frequency signal, it has a adding means for generating a third luminance signal,
The weighted addition means includes a first bandpass signal obtained by performing bandpass filter processing in a 45-degree direction in the spatial frequency domain on the first luminance signal, and 135 in the spatial frequency domain on the first luminance signal. The weighted addition is performed to increase the ratio of the second high-frequency signal in the third high-frequency signal as the absolute value of the difference from the second band-pass signal subjected to the band-pass filter processing in the direction of magnitude increases. Luminance signal generation device characterized. The weighted addition means includes absolute value of the difference value of the at least one set of pixel signals present in the upper and lower positions across the target pixel of the image sensor, the at least one set of pixel signals present on the left and right sides of the said target pixel the object is to determine the degree of vertical stripe or lateral stripe on the basis of the absolute value of the difference value, the closer to vertical stripes or horizontal stripes, a weighted addition to increase the ratio of the first high-frequency signal in the third high-frequency signal luminance signal generation apparatus according to claim 1 Symbol placement and performing. The weighted addition unit, the low-frequency component of the signal G1 pixels existing on both sides of the R pixel of the G pixel, the absolute value of the difference signal between the low frequency component of the signal of the G2 pixels existing on both sides of the B pixel the larger, the third luminance signal generation apparatus according to claim 1 or claim 2, characterized by performing weighted addition of increasing the proportion of the first RF signal in the high-frequency signal. The weighted addition means increases the ratio of the first high-frequency signal in the third high-frequency signal as the absolute value signal of the output signal of the filter that detects the low-frequency region of the signal of the subject obtained from the image sensor increases. luminance signal generation apparatus according to any one of claims 1 to 3, characterized by performing weighted addition of increasing. The weighted addition means detects the color of the subject from the signal of the subject obtained from the image sensor, and the ratio of the first high-frequency signal in the third high-frequency signal is increased as the subject color is closer to red. luminance signal generation apparatus according to any one of claims 1 to 4, characterized by performing weighted addition of increasing. A luminance signal generating device that generates a luminance signal of each pixel of the image sensor from a signal of an object obtained by an image sensor including a color filter of a primary color Bayer array,
First high-frequency signal creating means for creating a first high-frequency signal using a first luminance signal created only from the G pixel of the image sensor ;
Second high-frequency signal generating means for generating a second high-frequency signal using a second luminance signal in which the horizontal, vertical and 45 degree band of the signals of all the color pixels of the image sensor are limited;
Third high-frequency signal generating means for generating a third high-frequency signal using a third luminance signal in which the horizontal, vertical, and 135-degree band bands of the signals of all the color pixels of the image sensor are limited;
Selecting means for selecting one of the second and third high-frequency signals;
Weighted addition means for weighted addition of the first high-frequency signal and one of the second and third high-frequency signals selected by the selection means to create a fourth high-frequency signal;
It said first luminance signal and the fourth adds the high-frequency signal, possess adding means for outputting a fourth luminance signal and,
If the absolute value of the signal that limits the band in the 45 degree direction of the first luminance signal is larger than the absolute value of the signal that restricts the band in the 135 degree direction of the first luminance signal, the selection means The luminance signal generating apparatus is characterized in that the third high-frequency signal is selected if it is small . The weighted addition means includes absolute value of the difference value of the at least one set of pixel signals present in the upper and lower positions across the target pixel of the image sensor, the at least one set of pixel signals present on the left and right sides of the said target pixel Based on the difference between the absolute values of the difference values, the degree of the subject being vertical stripes or horizontal stripes is determined, and the weighted addition increases the ratio of the first high-frequency signal in the fourth high-frequency signal as the vertical stripe or horizontal stripe is closer The luminance signal generating apparatus according to claim 6, wherein: The weighted addition means has a large absolute value of a difference signal between the low frequency component of the G1 pixel signal existing on both sides of the R pixel in the G pixel and the low frequency component of the G2 pixel signal existing on both sides of the B pixel. The luminance signal generating apparatus according to claim 6 or 7 , wherein weighted addition is performed to increase a ratio of the first high-frequency signal to the fourth high-frequency signal. The weighted addition unit, as the absolute value signal of the output signal of the filter for detecting a low-frequency region of the subject signal obtained from the image sensor is large, the ratio of the first high-frequency signal in the fourth high-frequency signal to perform a higher weighted addition luminance signal generation apparatus according to any one of claims 6 to 8, characterized in. The weighted addition unit, the detecting the color of the object from the signal of the subject obtained from the imaging element, the ratio of the first high-frequency signal color of the object in the more the fourth high-frequency signal close to red luminance signal generation apparatus according to any one of claims 6 to 9, characterized in that performing the weighted addition of increasing. An image sensor having a primary color Bayer array color filter;
An imaging device comprising the luminance signal generation device according to any one of claims 1 to 10 . A luminance signal creating method for creating a luminance signal from a signal of a subject obtained by an image sensor having a color filter of a primary color Bayer array,
A first high-frequency signal creating step of creating a first high-frequency signal using a first luminance signal created only from the G pixel signal of the image sensor ;
A second high-frequency signal creating step of creating a second high-frequency signal using a second luminance signal created from signals of all the color pixels of the image sensor ;
A weighted addition step of creating a third high-frequency signal by weighted addition of the first high-frequency signal and the second high-frequency signal according to a spatial frequency of the signal of the subject ;
It said first adding the luminance signal and the third high-frequency signal, possess an adding step of generating a third luminance signal,
In the weighted addition step, the first luminance signal is obtained by performing a band-pass filter process in a 45 degree direction in the spatial frequency domain on the first luminance signal, and the first luminance signal is 135 in the spatial frequency domain. The weighted addition is performed to increase the ratio of the second high-frequency signal in the third high-frequency signal as the absolute value of the difference from the second band-pass signal subjected to the band-pass filter processing in the direction of magnitude increases. A characteristic luminance signal generation method. A luminance signal creating method for creating a luminance signal from a signal of a subject obtained by an image sensor having a color filter of a primary color Bayer array,
A first high-frequency signal creating step of creating a first high-frequency signal using a first luminance signal created only from the G pixel of the image sensor ;
A second high-frequency signal generating step of generating a second high-frequency signal using a second luminance signal in which the horizontal, vertical and 45 degree band of the signal of all color pixels of the image sensor is limited;
A third high-frequency signal generating step of generating a third high-frequency signal by using a third luminance signal in which the horizontal, vertical and 135 degree bands of the signals of all the color pixels of the image sensor are limited;
A selection step of selecting one of the second and third high-frequency signals;
A weighted addition step of weighted addition of the first high-frequency signal and one of the second and third high-frequency signals selected by the selection step to create a fourth high-frequency signal;
Adds the first luminance signal and the fourth high-frequency signal, it possesses an adding step of outputting a fourth luminance signal,
If the absolute value of the signal that limits the band in the 45 degree direction of the first luminance signal is larger than the absolute value of the signal that restricts the band in the 135 degree direction of the first luminance signal, the selecting step A luminance signal generating method comprising: selecting the third high frequency signal if the signal is small . A program for causing a computer to execute each step of the luminance signal generation method according to claim 12 or 13 .

Images