JP2009253439A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of preventing the decline of resolution even when there is a high luminance object, improving image quality and improving the connection of the low frequency and high frequency of luminance signals. <P>SOLUTION: The imaging apparatus provided with an imaging element 5 including photoelectric conversion elements 51R, G, and B for detecting RGB light respectively and a photoelectric conversion element 51W for detecting the luminance components of the light includes: an RGBW interpolation part 12 for generating R signals, G signals, B signals and signals (W signals) of the luminance components using imaging signals from the respective photoelectric conversion elements at pixel positions corresponding to the respective photoelectric conversion elements; a luminance signal generation part 13 for generating the luminance signals Y at the pixel positions from the RGB signals generated at the pixel positions; and a luminance signal generation means for combining composite signals Y(L) in which the low frequency components of the luminance signals Y generated at the pixel position is combined with the low frequency components of the W signals generated at the pixel position with the high frequency components of the W signals generated at the pixel position and generating the luminance signals Y' configuring pixel data corresponding to the pixel position. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image pickup apparatus having a solid-state image pickup element including at least three types of color detection photoelectric conversion elements for detecting light of different colors and a luminance detection photoelectric conversion element for detecting a luminance component of light. .

  Conventionally, an R photoelectric conversion element that detects R (red) light, a G photoelectric conversion element that detects G (green) light, a B photoelectric conversion element that detects B (blue) light, and a luminance component of light are detected. An imaging apparatus having a solid-state imaging element including a W photoelectric conversion element is proposed (see, for example, Patent Document 1).

  In such an imaging apparatus, as a luminance signal of each pixel data of color image data to be generated, an RGB luminance signal generated by a color signal obtained from the R photoelectric conversion element, the G photoelectric conversion element, and the B photoelectric conversion element is used. It can be considered that the W luminance signal generated from the signal obtained from the W photoelectric conversion element is used. When the W luminance signal is used, the luminance is different due to the difference in spectral characteristics between the W photoelectric conversion element, the R photoelectric conversion element, the G photoelectric conversion element, and the B photoelectric conversion element as compared with the case where the RGB luminance signal is used. Reproducibility decreases. When the RGB luminance signal is used, the resolution is lower than when the W luminance signal is used.

  Therefore, as in the imaging device of Patent Document 1, a luminance signal obtained by combining a low-frequency component of an RGB luminance signal and a high-frequency component of a W luminance signal is used as the luminance signal of each pixel data of color image data to be generated. In addition, both luminance reproducibility and resolution can be achieved.

  However, when the level of the W luminance signal is lower than the level of the RGB luminance signal, when the low frequency component of the RGB luminance signal and the high frequency component of the W luminance signal are synthesized, the level of the luminance signal obtained by the synthesis is higher than that of the W luminance signal. If there is a high-luminance subject, the signal level of the subject portion is saturated and the high-frequency component of the luminance signal is crushed, which may reduce the resolution. Further, when the frequency characteristics of the RGB luminance signal and the W luminance signal are different, the frequency connection is deteriorated by the synthesis, and the image quality is deteriorated.

  Patent Document 2 discloses a single-plate color camera that eliminates a difference in sensitivity of luminance information caused by a plurality of color filters arranged on a light receiving surface of a solid-state image sensor and can reproduce the structure of a subject more realistically. ing.

  Patent Document 3 discloses an imaging method in which processing for a luminance signal is performed independently of processing for a color signal, color reproduction and luminance reproduction can be improved, and side effects that the color signal has on the luminance signal can be reduced. .

JP 2007-243334 A JP-A-8-46980 JP 2001-54133 A

  The present invention has been made in view of the above circumstances, and it is possible to improve image quality by preventing degradation of resolution even when there is a high-luminance subject while achieving both luminance reproducibility and resolution. An object is to provide an imaging device.

  The imaging apparatus of the present invention includes a solid-state imaging device including at least three types of color detection photoelectric conversion elements that detect light of different colors and a luminance detection photoelectric conversion element that detects a luminance component of light. An imaging device that uses a signal obtained from the color detection photoelectric conversion element and the luminance detection photoelectric conversion element at a coordinate position of pixel data that constitutes color image data to be generated. ) Component signal, G (green) component signal, B (blue) component signal, and luminance component signal generating means; and the R component signal generated at the coordinate position; First luminance signal generation means for generating a first luminance signal at the coordinate position from the G component signal and the B component signal, and a low frequency of the first luminance signal generated at the coordinate position Component and generated at the coordinate position The luminance that composes the pixel data at the coordinate position by synthesizing the synthesized signal obtained by synthesizing the low frequency component of the luminance component signal and the high frequency component of the luminance component signal generated at the coordinate position Second luminance signal generation means for generating a second luminance signal as a signal.

  In the imaging apparatus according to the aspect of the invention, the first luminance signal generation unit may convert the R component signal, the G component signal, and the B component signal generated at the coordinate position with a predetermined luminance generation coefficient. The first luminance signal is generated by weighted addition, and the predetermined luminance generation coefficient is obtained by using only the signal obtained from the color detection photoelectric conversion element as the predetermined luminance generation coefficient. It is set to a value that minimizes the level difference between the low frequency component of the first luminance signal and the synthesized signal when the value is optimal for generation.

  In the imaging apparatus according to the aspect of the invention, the first luminance signal generation unit may convert the R component signal, the G component signal, and the B component signal generated at the coordinate position into the color detection photoelectric signal. When generating color image data only with a signal obtained from the conversion element, the first luminance signal is generated by weighted addition with an optimal luminance generation coefficient, and before the generation of the first luminance signal, Level adjustment means for adjusting the level of a signal obtained from the photoelectric conversion element for color detection, wherein the level adjustment means has a minimum level difference between the low frequency component of the first luminance signal and the synthesized signal. The level of the signal is adjusted using such a gain.

  The image pickup apparatus of the present invention includes a storage unit that stores the luminance generation coefficient that minimizes the level difference for each subject light source, and the first luminance signal generation unit uses the subject light source set at the time of shooting. The first luminance signal is generated using the corresponding luminance generation coefficient.

  The imaging apparatus of the present invention includes storage means for storing a gain that minimizes the level difference for each subject light source, and the level adjustment means uses the gain corresponding to the subject light source set at the time of shooting. The level of the signal is adjusted.

  In the imaging apparatus according to the aspect of the invention, the first luminance signal generation unit may convert the R component signal, the G component signal, and the B component signal generated at the coordinate position into the color detection photoelectric signal. When generating image data only with a signal obtained from the conversion element, the first luminance signal is generated by weighted addition with an optimal luminance generation coefficient, and the low-frequency component of the first luminance signal and the first luminance signal are generated. The level of the luminance component signal generated at the coordinate position is set to the level of the R component signal, the G component signal generated at the coordinate position, and Level adjustment means for adjusting based on the B component signal is provided.

  In the imaging apparatus according to the aspect of the invention, the level adjustment unit may obtain the R component signal, the G component signal, and the B component signal generated at the coordinate position by weighted addition with a predetermined coefficient. The level adjustment is performed by adding the adjustment data to the luminance component signal generated at the coordinate position.

  The image pickup apparatus of the present invention includes a storage unit that stores the predetermined coefficient for each subject light source, and the level adjustment unit uses the predetermined coefficient corresponding to the subject light source set at the time of shooting to calculate the luminance component. Adjust the signal level.

  According to the present invention, it is possible to provide an imaging apparatus capable of improving image quality while preventing a decrease in resolution even when there is a high-luminance subject while achieving both luminance reproducibility and resolution. Further, by using the low frequency component of the signal obtained from the fourth photoelectric conversion element as the low frequency component of the second luminance signal, the connection between the low frequency component and the high frequency component of the second luminance signal is improved. be able to.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is a schematic partial plan view of an image sensor mounted on the image pickup apparatus according to the first embodiment of the present invention.
A solid-state imaging device 5 shown in FIG. 1 includes a photoelectric conversion element 51R (character “R” in the figure) that detects R light arranged in a square lattice in a horizontal direction on a semiconductor substrate and a vertical direction perpendicular thereto. ), Photoelectric conversion element 51G for detecting G light (character “G” in the figure), and photoelectric conversion element 51B for detecting B light (character “B” in the figure) And a photoelectric conversion element 51W for detecting luminance components of light arranged in a square lattice pattern in the horizontal direction on the semiconductor substrate and in the vertical direction perpendicular thereto (see FIG. W photoelectric conversion element group consisting of “W” inside), and these positions are shifted in the horizontal and vertical directions by about ½ of the respective photoelectric conversion element arrangement pitch. Is arranged. The arrangement pitch of each photoelectric conversion element of the RGB photoelectric conversion element group and each photoelectric conversion element of the W photoelectric conversion element group is the same.

  The arrangement of the photoelectric conversion elements of the RGB photoelectric conversion element group includes a GR photoelectric conversion element array, which is a line composed of photoelectric conversion elements 51G and 51R arranged in the vertical direction, and photoelectric conversion elements 51B and photoelectric elements arranged in the vertical direction. It can be said that the BG photoelectric conversion element rows, which are rows made of the conversion elements 51G, are alternately arranged in the horizontal direction.

  The arrangement of each photoelectric conversion element of the W photoelectric conversion element group can be said to be a plurality of W photoelectric conversion element arrays that are arrays of photoelectric conversion elements 51W arranged in the vertical direction.

  On the right side of each photoelectric conversion element array, a vertical charge transfer unit (for transferring charges accumulated in the photoelectric conversion elements constituting each photoelectric conversion element array in the vertical direction corresponding to each photoelectric conversion element array ( (Not shown) is formed. A horizontal charge transfer unit (not shown) for transferring the charges transferred here in the horizontal direction is connected to the end of the vertical charge transfer unit, and the horizontal charge transfer is connected to the end of the horizontal charge transfer unit. An output amplifier (not shown) that outputs an imaging signal corresponding to the charge transferred through the unit is provided. In addition, the structure according to which the signal according to the electric charge accumulate | stored in each photoelectric conversion element is output outside by the MOS circuit which consists of a MOS transistor may be sufficient.

  Each photoelectric conversion element has the same structure, and the components of light to be detected differ depending on the filters formed above the respective light receiving surfaces.

  In the photoelectric conversion element 51R, an R color filter that transmits an R component of light is formed on the light receiving surface, thereby functioning as a photoelectric conversion element that detects R light. In the photoelectric conversion element 51G, a G color filter that transmits the G component of light is formed on the light receiving surface, thereby functioning as a photoelectric conversion element that detects G light. In the photoelectric conversion element 51B, a B color filter that transmits the B component of light is formed on the light receiving surface, and thus functions as a photoelectric conversion element that detects B light. In the photoelectric conversion element 51W, a luminance filter having spectral characteristics correlated with the luminance component of light is formed on the light receiving surface, thereby functioning as a photoelectric conversion element that detects the luminance component of light.

  The luminance filter corresponds to an ND filter, a transparent filter, a white filter, a gray filter, or the like, but the configuration in which light is directly incident on the light receiving surface without providing anything above the light receiving surface of the photoelectric conversion element 51W is also possible. It can be said that a filter is provided.

  Hereinafter, an R component imaging signal obtained from the photoelectric conversion element 51R is an R signal, a G component imaging signal obtained from the photoelectric conversion element 51G is a G signal, and a B component imaging signal obtained from the photoelectric conversion element 51B is a B signal. An imaging signal of a luminance component obtained from the photoelectric conversion element 51W is referred to as a W signal.

FIG. 2 is a block diagram showing a schematic configuration of the imaging apparatus according to the first embodiment of the present invention.
The image pickup apparatus shown in FIG. 2 includes the image pickup element 5 shown in FIG. 1, an analog front end (AFE) 11 that performs predetermined analog signal processing and digital conversion processing on analog signals output from the image pickup element 5, and an AFE 11. A digital signal processing unit 10 that generates color image data by performing interpolation calculation, gamma correction calculation, RGB / YC conversion processing, and the like on the digital imaging signal.

  The AFE 11 also performs level adjustment processing for adjusting the level of each signal by multiplying each of the R signal, G signal, and B signal by a uniform gain. The gain for adjusting the level is determined in advance according to the set photographing sensitivity (ISO), and the AFE 11 multiplies each signal by the gain corresponding to the set ISO sensitivity. Adjust the level. The imaging signal output from the AFE 11 is temporarily stored in a memory (not shown). In this memory, an imaging signal is stored in a state where an imaging signal obtained from the photoelectric conversion element is arranged at a pixel position corresponding to each photoelectric conversion element of the imaging element 5.

  The digital signal processing unit 10 interpolates signals from the surroundings to each pixel position on the memory to generate an R signal, G signal, B signal, and W signal, and a signal generated at each pixel position. The luminance signal generation unit 13 that generates the luminance signal Y from the color signal, the color difference signal generation unit 14 that generates the color difference signal from the signal generated at each pixel position, and the luminance signal Y and the W signal generated by the luminance signal generation unit 13 The low frequency component extraction unit 15, the low frequency / high frequency component separation unit 16, and the low frequency component synthesis unit constituting luminance signal generation means for generating a luminance signal Y ′ constituting one pixel data of color image data based on 17 and a low frequency / high frequency component synthesis unit 18.

  The RGBW interpolation unit 12 converts the pixel position from the R signal, G signal, B signal, and W signal stored in the memory to pixel positions corresponding to the photoelectric conversion elements 51R, 51G, 51B, and 51W on the memory. An R signal, a G signal, and a B signal using an imaging signal obtained from a photoelectric conversion element corresponding to 1 and an imaging signal obtained from a photoelectric conversion element of a type different from the photoelectric conversion element around the photoelectric conversion element , And a process of generating a W signal.

  For example, at a pixel position corresponding to the photoelectric conversion element 51R, an R signal obtained from the photoelectric conversion element 51R and an imaging signal obtained from the photoelectric conversion elements 51G, 51B, and 51W around the photoelectric conversion element 51R. Are used to generate an R signal, a G signal, a B signal, and a W signal. At a pixel position corresponding to the photoelectric conversion element 51G, the G signal obtained from the photoelectric conversion element 51G and the photoelectric conversion element 51G R signals, G signals, B signals, and W signals are generated using imaging signals obtained from the surrounding photoelectric conversion elements 51R, 51B, and 51W, and the photoelectric conversion is performed at the pixel position corresponding to the photoelectric conversion element 51B. An R signal, a G signal, a B signal, and a W signal are generated using the B signal obtained from the element 51B and the imaging signals obtained from the photoelectric conversion elements 51R, 51G, and 51W around the photoelectric conversion element 51B. , At the pixel position corresponding to the photoelectric conversion element 51W, the W signal obtained from the photoelectric conversion element 51W and the imaging signals obtained from the photoelectric conversion elements 51R, 51G, and 51B around the photoelectric conversion element 51W are used. R signal, G signal, B signal, and W signal are generated.

  By this processing, four imaging signals of R signal, G signal, B signal, and W signal are generated at pixel positions corresponding to the photoelectric conversion elements 51R, 51G, 51B, and 51W on the memory. Become.

The luminance signal generation unit 13 converts the R signal, the G signal, and the B signal at each pixel position generated by the RGBW interpolation unit 12 into predetermined coefficients (α, β, γ, as shown in the following equation (1). ) To generate a luminance signal Y at each pixel position. This arithmetic expression is a general expression used when obtaining a luminance signal constituting one pixel data of color image data.
Luminance signal Y = α × R signal + β × G signal + γ × B signal (1)

  The coefficients α, β, and γ are optimal values when color image data is generated using only imaging signals obtained from the photoelectric conversion elements 51R, 51G, and 51B. When color image data is generated using only the imaging signals obtained from the photoelectric conversion elements 51R, 51G, and 51B, the color image data is generated based on imaging signals obtained from a well-known single-plate imaging element having a Bayer array. Is the same as That is, the optimum value is a general coefficient (α = 0.3, β = 0) used when color image data is generated based on an imaging signal obtained from a known single-plate imaging device having a Bayer array. .59, γ = 0.11).

  The color difference signal generation unit 14 generates a color difference signal C at the pixel position from the R signal, the G signal, and the B signal generated at each pixel position by the RGBW interpolation unit 12. A known method can be adopted as a method of generating the color difference signal C.

  The low frequency component extraction unit 15 extracts a low frequency component of the luminance signal Y generated at an arbitrary pixel position by the luminance signal generation unit 13 and inputs the low frequency component to the low frequency component synthesis unit 17. The low frequency component extraction part 15 can be comprised by a low pass filter, for example.

  The low frequency / high frequency component separation unit 16 separates the W signal generated at the arbitrary pixel position into a high frequency component and a low frequency component, and inputs the low frequency component to the low frequency component synthesis unit 17 to reduce the high frequency component. This is input to the frequency / high frequency component synthesizer 18. As a method of separating the W signal into a high frequency component and a low frequency component, first, a low frequency component is extracted from the W signal by a low-pass filter, and then the extracted low frequency component signal is subtracted from the original W signal. An example is a method of obtaining a high frequency component of the W signal.

  The low frequency component combining unit 17 combines the low frequency component of the luminance signal Y generated at the arbitrary pixel position and the low frequency component of the W signal generated at the arbitrary pixel position with a predetermined weight. A signal Y (L) is generated and input to the low frequency / high frequency component synthesizer 18. For example, the low frequency component of the luminance signal Y and the low frequency component of the W signal are weighted at a ratio of 1: 1 and combined. In this case, the synthesis arithmetic expression is {(low frequency component of luminance signal Y) + (low frequency component of W signal)} / 2.

  The low frequency / high frequency component synthesizing unit 18 synthesizes the synthesized signal Y (L) corresponding to the arbitrary pixel position and the high frequency component of the W signal generated at the arbitrary pixel position to generate a luminance signal. Y ′ is generated.

  FIG. 3 is a diagram for explaining the effect of the signal synthesis processing by the low-frequency component synthesis unit 17 shown in FIG. FIG. 3 shows an example in which the low frequency component of the luminance signal Y and the low frequency component of the W signal are weighted by 1: 1 and combined.

As shown in FIG. 3, when the low frequency component of the luminance signal Y and the low frequency component of the W signal are combined with a weight of 1: 1, the combined signal becomes Y (L), and the low frequency component of the luminance signal Y is obtained. Compared with the case where the component is used as a low-frequency component of the luminance signal constituting the pixel data of the color image data, the amount of light until reaching the saturation level can be increased. As a result, even when there is a high-luminance subject, it is possible to improve the collapse of the high-frequency component of the luminance signal constituting the pixel data.
Further, compared with the case where the low frequency component of the luminance signal Y is used as the low frequency component of the luminance signal constituting the pixel data of the color image data, the connection between the low frequency component and the high frequency component of the luminance signal is improved. Can do.
Further, the luminance reproducibility can be improved as compared with the case where the W signal is used as it is as the luminance signal constituting the pixel data.

The operation of the imaging apparatus configured as described above will be described.
An image pickup signal that has been picked up and output from the image pickup device 5 is subjected to gain adjustment by the AFE 11 and digitally converted, and then stored in the internal memory of the image pickup apparatus. In the internal memory, an image pickup signal is stored in a state where an image pickup signal obtained from the photoelectric conversion element is arranged at a pixel position corresponding to each photoelectric conversion element of the image pickup element 5. Next, an R signal, a G signal, a B signal, and a W signal are generated at the pixel position corresponding to each photoelectric conversion element of the imaging element 5 from the imaging signal stored in the internal memory. Then, a color difference signal C and a luminance signal Y constituting pixel data corresponding to the pixel position are generated from a signal at the pixel position corresponding to each photoelectric conversion element of the image sensor 5. Next, based on the luminance signal Y and the W signal generated at the pixel position, a luminance signal Y ′ constituting pixel data corresponding to the pixel position is generated. By such processing, the color difference signal C and the luminance signal Y ′ are generated at the pixel position corresponding to each photoelectric conversion element of the imaging element 5.

  Finally, using the color difference signal C and the luminance signal Y ′ generated at the pixel position corresponding to each photoelectric conversion element of the image sensor 5, the image sensor 5 has an intermediate position between the photoelectric conversion elements of the RGB photoelectric conversion element group. The color difference signal C and the luminance signal Y ′ are interpolated at the pixel position corresponding to the corresponding pixel position and the intermediate position of each photoelectric conversion element of the W photoelectric conversion element group of the image pickup element 5, and the pixel data of the square array is obtained. The color image data is generated. The color image data is compressed and then recorded on a recording medium that can be output to the outside.

As described above, the imaging apparatus according to the present embodiment uses the low-frequency component of the luminance signal Y ′ generated at the pixel position corresponding to each photoelectric conversion element of the imaging element 5 as the low-frequency component of the luminance signal Y generated at the pixel position. A synthesized signal Y (L) obtained by synthesizing the frequency component and the low frequency component of the W signal generated at the pixel position, and the high frequency component of the luminance signal Y ′ as the high frequency of the W signal generated at the pixel position. As an ingredient. For this reason, the saturation of the low frequency component of the luminance signal Y ′ can be improved compared to the conventional case where the luminance signal Y ′ is a combined signal of the low frequency component of the luminance signal Y and the high frequency component of the W signal. It is possible to prevent degradation of the resolution by preventing the signal Y ′ from being crushed by high frequency components.
In addition, the connection between the low frequency component and the high frequency component of the luminance signal Y ′ can be improved as compared with the conventional case where the luminance signal Y ′ is a composite signal of the low frequency component of the luminance signal Y and the high frequency component of the W signal. .
Further, the luminance reproducibility can be improved as compared with the case where the W signal is used as it is as the luminance signal constituting the pixel data.

(Second embodiment)
In the first embodiment, as shown in FIG. 3, the luminance reproducibility of the combined signal Y (L) is better than the low frequency component of the W signal, but the color reproducibility is better than the low frequency component of the luminance signal Y. It gets worse. In the present embodiment, an imaging apparatus capable of further improving the color reproducibility and luminance reproducibility of the combined signal Y (L) generated by the imaging apparatus of the first embodiment will be described.

  In the imaging apparatus of the present embodiment, the coefficients (α, β, γ) used when the luminance signal generation unit 13 of the imaging apparatus shown in FIG. 2 generates the luminance signal Y are combined with the low-frequency component of the luminance signal Y. The luminance signal generation unit 13 of the imaging apparatus shown in FIG. 2 generates the luminance signal Y using this coefficient in advance by the least square method or the like so that the level difference with the signal Y (L) is minimized. As a result, the color reproducibility and luminance reproducibility are improved. The coefficients (α, β, γ) that minimize the level difference can be obtained from a signal obtained by photographing a predetermined image before product shipment, for example.

  For example, as shown in FIG. 3, when a luminance signal Y generated with coefficients (α, β, γ) that minimizes the level difference between Y (low frequency) and Y (L) is Y_corr, The signal Y (L) ′ output from the component synthesizer 17 is as illustrated.

  This signal Y (L) 'has the same characteristics as Y (low frequency) until halfway, and has a knee characteristic that changes to the same slope as W when Y_corr exceeds the amount of light that saturates. For this reason, saturation can be ensured while ensuring color reproducibility equivalent to Y (low frequency). The luminance reproduction in this case can be optimized for luminance lower than the knee characteristic, and the knee characteristic provides an advantage that the high frequency component of the luminance signal Y ′ is not crushed.

  In the imaging apparatus shown in FIG. 2, the coefficients (α, β, γ) are set to optimum values as described in the first embodiment, and gains to be multiplied by signals at the time of level adjustment in the AFE 11 are set. The luminance reproducibility and color reproducibility can also be improved by changing to a value that minimizes the level difference. The gain that minimizes the level difference can be obtained, for example, from a signal obtained by photographing a predetermined image before product shipment.

  Although not shown in FIG. 2, the imaging apparatus according to the present embodiment multiplies each of the R signal, the G signal, and the B signal by the white balance gain before the luminance signal generation unit 13 to obtain the white balance. A white balance correction unit is provided to correct the above. The white balance gain differs for each color signal and changes depending on the subject light source. For this reason, the white balance correction unit stores different white balance gains for each subject light source.

  Taking the case of changing the coefficients (α, β, γ) as an example in order to minimize the level difference, if the white balance gain is changed, the level difference naturally changes. Naturally, the coefficients (α, β, γ) that minimize the value also change. Therefore, coefficients (α, β, γ) that minimize the level difference are stored for each subject light source in the imaging device, and the luminance signal generation unit 13 sets the subject light source set at the time of photographing. It is preferable to generate the luminance signal Y by reading out the coefficient corresponding to. In this way, it is possible to cope with changes in the subject light source.

Further, in the case of changing the gain at the AFE 11 in order to minimize the level difference, if the white balance gain is changed, the level difference naturally changes, and the level difference is minimized. Naturally, such a gain also changes. For this reason, the gain that minimizes the level difference is stored for each subject light source in the imaging apparatus, and the AFE 11 reads the gain corresponding to the subject light source set at the time of shooting and performs gain adjustment. Preferably it is done. In this way, it is possible to cope with changes in the subject light source.
(Third embodiment)
In the imaging apparatus according to the second embodiment, the coefficient and gain are fixed, and the level difference is minimized by adjusting the level of the W signal.

  FIG. 4 is a block diagram illustrating a schematic configuration of an imaging apparatus according to the third embodiment of the present invention. In FIG. 4, the same components as those in FIG. The imaging apparatus shown in FIG. 4 has a configuration in which a W adjustment unit 19 is added to the imaging apparatus shown in FIG.

The W adjustment unit 19 adjusts the level of the W signal generated at each pixel position by the RGBW interpolation unit 12 by performing calculation according to the following equation (2) so that the level difference is minimized.
W signal after adjustment = W signal before adjustment + xR signal + yG signal + zB signal (2)

  In Expression (2), the R signal, the G signal, and the B signal are signals generated by the RGBW interpolation unit 12 at the same pixel position as the W signal to be adjusted. X, y, and z are weighting coefficients.

  The weighting coefficient may be obtained in advance from a signal obtained when a predetermined image is captured so that the level difference between the synthesized signal Y (L) and the low frequency component of the luminance signal Y is minimized.

  As described above, the luminance reproducibility and the color reproducibility can be improved by adjusting the level of the W signal in advance. Furthermore, the connection between the low frequency component and the high frequency component of the luminance signal Y ′ can be improved.

  Similarly, in the imaging apparatus of the present embodiment, a white balance correction unit is provided. That is, if the white balance gain is changed, the level difference naturally changes, and the coefficients (x, y, z) that minimize the level difference naturally change. For this reason, in the imaging apparatus, coefficients (x, y, z) that minimize the level difference are stored for each subject light source, and the W adjustment unit 19 sets the subject light source set at the time of photographing. It is preferable to adjust the level of the W signal by reading the corresponding coefficient. In this way, it is possible to cope with changes in the subject light source.

(Fourth embodiment)
The arrangement of the photoelectric conversion elements of the imaging element 5 described in the first embodiment to the third embodiment is not limited to that shown in FIG. 1, but includes the photoelectric conversion element 51R, the photoelectric conversion element 51G, and the photoelectric conversion element 51B. The photoelectric conversion element 51W may be configured in a one-dimensional or two-dimensional manner.

FIG. 5 is a diagram illustrating a modification of the arrangement of the photoelectric conversion elements of the image sensor 5.
As shown in FIG. 5, a large number of photoelectric conversion elements composed of photoelectric conversion elements 51R, 51G, 51B, 51W are arranged in a square lattice pattern on a semiconductor substrate, and an RGB photoelectric conversion element composed of photoelectric conversion elements 51R, 51G, 51B. The RGB photoelectric conversion element group and the W photoelectric conversion element group may be arranged in a checkered pattern so that the group and the W photoelectric conversion element group including the photoelectric conversion elements 51W form a checkered pattern.

(Fifth embodiment)
The filter provided above the RGB photoelectric conversion element group shown in FIGS. 1 and 5 may be a complementary color filter instead of a primary color filter.
Even when the complementary color filter is used, it is possible to generate an R component signal, a G component signal, a B component signal, and a luminance component signal at the pixel position corresponding to each photoelectric conversion element of the image sensor 5. Therefore, the present invention can be applied. Further, the number of colors of the filter is not limited to three, and may be four or more.

(Sixth embodiment)
In the imaging device of the first embodiment, there may be a photographing mode for generating color image data composed of the same number of pixel data as the number of photoelectric conversion elements constituting the RGB photoelectric conversion element group.

  In this case, the RGBW interpolation unit 12 uses the R signal, the G signal, the B signal, and the W signal output from the AFE 11 at the pixel position corresponding to each of the photoelectric conversion elements 51R, 51G, and 51B on the memory. Using an imaging signal obtained from a photoelectric conversion element corresponding to a position and an imaging signal obtained from a photoelectric conversion element of a type different from the photoelectric conversion element around the photoelectric conversion element, an R signal, a G signal, and a B signal A process of generating a signal and a W signal is performed.

  As the W signal to be generated at the pixel position corresponding to each of the photoelectric conversion elements 51R, 51G, and 51B, the W signal obtained from the photoelectric conversion element 51W adjacent to the pixel position may be used as it is.

  Then, the color difference signal C and the luminance signal Y are generated from the generated signal at the pixel position corresponding to each of the photoelectric conversion elements 51R, 51G, and 51B, and the luminance signal Y ′ is generated from the luminance signal Y and the W signal. Color image data may be completed.

That is, the pixel position where the luminance signal Y ′ and the color difference signal C are finally generated corresponds to the coordinate position of the pixel data constituting the color image data in the claims.
In the case of the first embodiment, the pixel positions corresponding to all the photoelectric conversion elements included in the solid-state imaging device 5 are the coordinate positions, and in the case of the sixth embodiment, an RGB photoelectric conversion element group is configured. The pixel position corresponding to the photoelectric conversion element is the coordinate position.

Schematic diagram of a partial plane of an image sensor mounted on the image pickup apparatus according to the first embodiment of the present invention. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus according to a first embodiment of the present invention. The figure for demonstrating the effect by the signal synthetic | combination process by the low frequency component synthetic | combination part shown in FIG. The block diagram which shows schematic structure of the imaging device which is 3rd embodiment of this invention. The figure which shows the modification of the arrangement | sequence of the photoelectric conversion element of the image pick-up element shown in FIG.

Explanation of symbols

5 Image sensor 12 RGBW interpolation unit 13 Luminance signal generation unit 51R, G, B Color detection photoelectric conversion element 51W Luminance detection photoelectric conversion element

Claims (8)

An imaging apparatus having a solid-state imaging device including at least three types of color detection photoelectric conversion elements for detecting light of different colors and a luminance detection photoelectric conversion element for detecting a luminance component of light,
R (red) component signal using the signals obtained from the color detection photoelectric conversion element and the luminance detection photoelectric conversion element at the coordinate position of the pixel data constituting the color image data to be generated; Signal generating means for generating a G (green) component signal, a B (blue) component signal, and a luminance component signal;
First luminance signal generation means for generating a first luminance signal at the coordinate position from the R component signal, the G component signal, and the B component signal generated at the coordinate position;
A synthesized signal obtained by synthesizing a low frequency component of the first luminance signal generated at the coordinate position and a low frequency component of the signal of the luminance component generated at the coordinate position, and the generated at the coordinate position An imaging apparatus comprising: a second luminance signal generating unit configured to combine a high frequency component of a luminance component signal and generate a second luminance signal that is a luminance signal that constitutes the pixel data at the coordinate position. The imaging apparatus according to claim 1,
The first luminance signal generation means weights and adds the R component signal, the G component signal, and the B component signal generated at the coordinate position with a predetermined luminance generation coefficient. The luminance signal of
The first luminance signal when the predetermined luminance generation coefficient is an optimum value when color image data is generated only from a signal obtained from the color detection photoelectric conversion element. An imaging device that is set to a value that minimizes the level difference between the low-frequency component and the synthesized signal. The imaging apparatus according to claim 1,
The first luminance signal generation means uses only the signal obtained from the color detection photoelectric conversion element for the R component signal, the G component signal, and the B component signal generated at the coordinate position. And generating the first luminance signal by weighted addition with an optimal luminance generation coefficient when generating color image data at
A level adjusting means for adjusting a level of a signal obtained from the photoelectric conversion element for color detection before the generation of the first luminance signal;
An image pickup apparatus in which the level adjusting unit adjusts the level of the signal using a gain that minimizes a level difference between a low frequency component of the first luminance signal and the synthesized signal. The imaging apparatus according to claim 2,
Storage means for storing the luminance generation coefficient that minimizes the level difference for each subject light source;
An imaging apparatus in which the first luminance signal generation unit generates the first luminance signal using the luminance generation coefficient corresponding to a subject light source set at the time of shooting. The imaging apparatus according to claim 3,
Storage means for storing a gain that minimizes the level difference for each subject light source;
An imaging apparatus in which the level adjusting means adjusts the level of the signal using the gain according to a subject light source set at the time of photographing. The imaging apparatus according to claim 1,
The first luminance signal generation means uses only the signal obtained from the color detection photoelectric conversion element for the R component signal, the G component signal, and the B component signal generated at the coordinate position. And generating the first luminance signal by weighted addition with an optimal luminance generation coefficient when generating image data at
The level of the luminance component signal generated at the coordinate position is set to the R position generated at the coordinate position so that the level difference between the low frequency component of the first luminance signal and the synthesized signal is minimized. An imaging apparatus comprising level adjusting means for adjusting based on a component signal, the G component signal, and the B component signal. The imaging apparatus according to claim 6,
Adjustment data obtained by weighting and adding the R component signal, the G component signal, and the B component signal generated at the coordinate position by a predetermined coefficient by the level adjusting means, An imaging apparatus that adjusts the level by adding to the luminance component signal generated at a position. The imaging apparatus according to claim 7,
Storage means for storing the predetermined coefficient for each subject light source;
An imaging apparatus in which the level adjusting means adjusts the level of the signal of the luminance component using the predetermined coefficient corresponding to a subject light source set at the time of shooting.

Images