JP2009253616A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of reducing color noise. <P>SOLUTION: A digital camera provided with a solid state imaging element 5 including a photoelectric conversion element 51R for detecting R light, a photoelectric conversion element 51G for detecting G light, a photoelectric conversion element 51B for detecting B light and a photoelectric conversion element 51W for detecting the luminance components of light includes: an RGBW interpolation part 17a for interpolating signals of a kind which can not be obtained from the photoelectric conversion elements corresponding to pixel positions from the periphery at the pixel positions corresponding to the respective photoelectric conversion elements included in the solid state imaging element 5 and generating R signals, G signals, B signals and signals of the luminance components at the pixel positions; a luminance/color difference signal generation part 17b for generating color difference signals C from the R signals, the G signals and the B signals; and a color difference signal correction means (a correction coefficient computing part 17c and a multiplier 17d) for correcting the color difference signals C on the basis of the R signals, the G signals, the B signals and the signals of the luminance components. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention provides a first photoelectric conversion element that detects light of a first color, a second photoelectric conversion element that detects light of a second color, and a third photoelectric element that detects light of a third color. The present invention relates to an imaging apparatus having a solid-state imaging device including a photoelectric conversion device and a fourth photoelectric conversion device that detects a luminance component of light.

  A first photoelectric conversion element that detects R light, a second photoelectric conversion element that detects G light, a third photoelectric conversion element that detects B light, and a fourth photoelectric element that detects a luminance component of light For example, Patent Documents 1 and 2 disclose an imaging apparatus having a solid-state imaging element including a conversion element.

  These image pickup devices use the signal from the fourth photoelectric conversion element to increase the brightness, but the color difference signal is not processed using the signal from the fourth photoelectric conversion element. The merit of providing the fourth photoelectric conversion element has not been fully utilized.

  On the other hand, in Patent Document 3, in an imaging apparatus including a solid-state imaging device including a photoelectric conversion element that can obtain a color signal and a photoelectric conversion element that can obtain a luminance signal, each pixel is determined based on the difference between the luminance signal and the color signal. A method for generating a RGB signal is disclosed. However, in this method, since the signal difference is obtained, an increase in noise occurs.

JP 2007-258686 A JP 2007-243334 A JP 2007-27667 A

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an imaging apparatus capable of reducing color noise.

  The imaging apparatus of the present invention detects a first photoelectric conversion element that detects light of a first color, a second photoelectric conversion element that detects light of a second color, and light of a third color. An image pickup apparatus having a solid-state image pickup device including a third photoelectric conversion device and a fourth photoelectric conversion device for detecting a luminance component of light, at a coordinate position of pixel data constituting color image data to be generated , A signal obtained from the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, a signal obtained from the third photoelectric conversion element, and a signal obtained from the fourth photoelectric conversion element Among these, at least a signal obtained from the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, and a signal obtained from the third photoelectric conversion element, a signal of an R (red) component And G (green) component signal and B (blue) RGB signal generation means for generating a component signal, and a color difference signal corresponding to the coordinate position from the R component signal, the G component signal, and the B component signal generated at the coordinate position. Using a signal obtained from the fourth photoelectric conversion element at the coordinate position, a correction luminance signal generation unit that generates a luminance component signal for correcting the color difference signal corresponding to the coordinate position. Means, a signal obtained from the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, a signal obtained from the third photoelectric conversion element, and a signal obtained from the fourth photoelectric conversion element Of the signals, at least the signal obtained from the first photoelectric conversion element, the signal obtained from the second photoelectric conversion element, and the signal obtained from the third photoelectric conversion element, are generated at the coordinate position. Based on the signal of the luminance component for the correction that is, and a color difference signal correcting means for correcting the color difference signal corresponding to the coordinate position.

  In the imaging apparatus according to the aspect of the invention, the color difference signal correction unit may include the R component signal, the G component signal, and the B component signal generated at the coordinate position by the RGB signal generation unit, and the coordinate position. The color difference signal corresponding to the coordinate position is corrected using the correction luminance component signal generated in step (b).

  In the image pickup apparatus of the present invention, the color difference signal correction unit includes a signal obtained from the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, a signal obtained from the third photoelectric conversion element, And a signal obtained from at least the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, and a signal obtained from the third photoelectric conversion element among the signals obtained from the fourth photoelectric conversion element The R component signal for correcting the color difference signal, the G component signal, and the B component signal are generated at the coordinate position by a method different from the RGB signal generating means. The color difference signal is corrected using the R component signal for correction, the G component signal, the B component signal, and the correction luminance component signal.

In the imaging apparatus of the present invention, the color difference signal correction unit includes a multiplication unit that multiplies the color difference signal by a correction coefficient to obtain a corrected color difference signal, and a correction coefficient calculation unit that calculates the correction coefficient. The correction coefficient calculation means uses the following equation, where K is the correction coefficient:
K = W / (α × R + β × G + γ × B + δ)
W: Signal of the correction luminance component generated at the coordinate position R: Signal of the R component generated at the coordinate position G: Signal of the G component generated at the coordinate position B: The coordinate position The B component signals α, β, γ, and δ generated in the above are the spectral characteristics of the fourth photoelectric conversion element, the first photoelectric conversion element, the second photoelectric conversion element, and the third photoelectric conversion element. When approximated by the product sum of each spectral characteristic of the photoelectric conversion element, the correction coefficient is determined by a coefficient determined so that an error between the approximated spectral characteristic and the spectral characteristic of the fourth photoelectric conversion element is minimized. Is calculated.

In the imaging apparatus of the present invention, the color difference signal correction unit includes a multiplication unit that multiplies the color difference signal by a correction coefficient to obtain a corrected color difference signal, and a correction coefficient calculation unit that calculates the correction coefficient. The correction coefficient calculation means uses the following equation, where K is the correction coefficient:
K = W / (α × R + β × G + γ × B + δ)
W: A signal of the correction luminance component generated at the coordinate position R: A signal of the correction R component generated at the coordinate position G: A signal of the G component for correction generated at the coordinate position Signal B: B component signal for correction generated at the coordinate position α, β, γ, δ: Spectral characteristics of the fourth photoelectric conversion element, the first photoelectric conversion element, the second photoelectric conversion element When approximated by the product sum of the respective spectral characteristics of the photoelectric conversion element and the third photoelectric conversion element, an error between the approximated spectral characteristic and the spectral characteristic of the fourth photoelectric conversion element is minimized. The correction coefficient is calculated according to the coefficient determined by

  In the imaging apparatus of the present invention, the correction coefficient calculation means clips the value of K to “1” when the value of K is larger than “1” as a result of the calculation.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which can reduce color noise can be provided.

  Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a schematic partial plan view of an image sensor mounted on an image pickup apparatus according to an embodiment of the present invention.
A solid-state imaging device 5 shown in FIG. 1 includes a photoelectric conversion element 51R (character “R” in the figure) that detects R light arranged in a square lattice in a horizontal direction on a semiconductor substrate and a vertical direction perpendicular thereto. ), Photoelectric conversion element 51G for detecting G light (character “G” in the figure), and photoelectric conversion element 51B for detecting B light (character “B” in the figure) And a photoelectric conversion element 51W for detecting luminance components of light arranged in a square lattice pattern in the horizontal direction on the semiconductor substrate and in the vertical direction perpendicular thereto (see FIG. W photoelectric conversion element group consisting of “W” inside), and these positions are shifted in the horizontal and vertical directions by about ½ of the respective photoelectric conversion element arrangement pitch. Is arranged. The arrangement pitch of each photoelectric conversion element of the RGB photoelectric conversion element group and each photoelectric conversion element of the W photoelectric conversion element group is the same.

  The arrangement of the photoelectric conversion elements of the RGB photoelectric conversion element group includes a GR photoelectric conversion element array that is a line made of photoelectric conversion elements 51G and 51R arranged in the vertical direction Y, and a photoelectric conversion element 51B that is aligned in the vertical direction Y. It can be said that the BG photoelectric conversion element rows, which are rows made up of and photoelectric conversion elements 51G, are alternately arranged in the horizontal direction.

  The arrangement of the photoelectric conversion elements of the W photoelectric conversion element group can be said to be a plurality of W photoelectric conversion element arrays that are arrays of photoelectric conversion elements 51W arranged in the vertical direction Y in the horizontal direction.

  On the right side of each photoelectric conversion element array, a vertical charge transfer unit (for transferring charges accumulated in the photoelectric conversion elements constituting each photoelectric conversion element array in the vertical direction corresponding to each photoelectric conversion element array ( (Not shown) is formed. A horizontal charge transfer unit (not shown) for transferring the charges transferred here in the horizontal direction is connected to the end of the vertical charge transfer unit, and the horizontal charge transfer is connected to the end of the horizontal charge transfer unit. An output amplifier (not shown) that outputs an imaging signal corresponding to the charge transferred through the unit is provided. In addition, the structure according to which the signal according to the electric charge accumulate | stored in each photoelectric conversion element is output outside by the MOS circuit which consists of a MOS transistor may be sufficient.

  Each photoelectric conversion element has the same structure, and the components of light to be detected differ depending on the filters formed above the respective light receiving surfaces.

  In the photoelectric conversion element 51R, an R color filter that transmits an R component of light is formed on the light receiving surface, thereby functioning as a photoelectric conversion element that detects R light. In the photoelectric conversion element 51G, a G color filter that transmits the G component of light is formed on the light receiving surface, thereby functioning as a photoelectric conversion element that detects G light. In the photoelectric conversion element 51B, a B color filter that transmits the B component of light is formed on the light receiving surface, and thus functions as a photoelectric conversion element that detects B light. In the photoelectric conversion element 51W, a luminance filter having spectral characteristics correlated with the luminance component of light is formed on the light receiving surface, thereby functioning as a photoelectric conversion element that detects the luminance component of light.

  The luminance filter corresponds to an ND filter, a transparent filter, a white filter, a gray filter, or the like, but the configuration in which light is directly incident on the light receiving surface without providing anything above the light receiving surface of the photoelectric conversion element 51W is also possible. It can be said that a filter is provided.

  Hereinafter, an R component imaging signal obtained from the photoelectric conversion element 51R is an R signal, a G component imaging signal obtained from the photoelectric conversion element 51G is a G signal, and a B component imaging signal obtained from the photoelectric conversion element 51B is a B signal. An imaging signal of a luminance component obtained from the photoelectric conversion element 51W is referred to as a W signal.

FIG. 2 is a diagram illustrating a schematic configuration of a digital camera that is an example of an imaging apparatus in which the solid-state imaging device illustrated in FIG. 1 is mounted.
The imaging system of the illustrated digital camera includes a photographic lens 1, a solid-state imaging device 5 shown in FIG. 1, a diaphragm 2 provided therebetween, an infrared cut filter 3, and an optical low-pass filter 4.

  A system control unit 11 that performs overall control of the electrical control system of the digital camera controls the flash light emitting unit 12 and the light receiving unit 13 and controls the lens driving unit 8 to adjust the position of the photographing lens 1 to the focus position and zoom. The exposure amount is adjusted by adjusting the aperture amount of the aperture 2 via the aperture drive unit 9.

  Further, the system control unit 11 drives the solid-state imaging device 5 via the imaging device driving unit 10 and outputs a subject image captured through the photographing lens 1 as an imaging signal. An instruction signal from the user is input to the system control unit 11 through the operation unit 14.

  The electric control system of the digital camera further includes an analog signal processing unit 6 that performs analog signal processing such as correlated double sampling processing and signal amplification processing connected to the output of the solid-state imaging device 5, and the analog signal processing unit 6. An A / D conversion circuit 7 that converts the output imaging signal into a digital signal is provided, and these are controlled by the system control unit 11. The imaging signal output from the A / D conversion circuit 7 is temporarily stored in the main memory 16. The main memory 16 stores the image pickup signal in a state where the image pickup signal obtained from the photoelectric conversion element is arranged at the pixel position corresponding to each photoelectric conversion element of the solid-state image pickup element 5.

  The electric control system of the digital camera includes a main memory 16, a memory control unit 15 connected to the main memory 16, and digital imaging signals from the A / D conversion circuit 7 for interpolation calculation, gamma correction calculation, and RGB / YC conversion. A digital signal processing unit (DSP) 17 that generates color image data by performing processing and the like, and compression / decompression that compresses the color image data generated by the digital signal processing unit 17 into a JPEG format or decompresses the compressed image data A processing unit 18, an external memory control unit 20 to which a detachable recording medium 21 is connected, and a display control unit 22 to which a liquid crystal display unit 23 mounted on the back of the camera or the like is connected include a control bus 24 and the data bus 25 are connected to each other and controlled by a command from the system control unit 11.

FIG. 3 is a diagram showing an internal configuration of the digital signal processing unit shown in FIG.
As shown in FIG. 3, the digital signal processing unit 17 includes an RGBW interpolation unit 17a that interpolates and generates R, G, B, and W signals at each pixel position on the main memory 16, and each pixel position. A luminance / color difference signal generation unit 17b that generates a luminance signal and a color difference signal at each pixel position from the signals in the signal, a correction coefficient calculation unit 17c that constitutes a color difference signal correction unit that corrects the color difference signal, and a multiplier 17d. ing.

  The RGBW interpolation unit 17a uses the R signal, the G signal, the B signal, and the W signal output from the A / D conversion circuit 7, and the pixel position corresponding to each of the photoelectric conversion elements 51R, 51G, 51B, and 51W on the memory. In addition, using a signal obtained from a photoelectric conversion element corresponding to the pixel position and a signal obtained from a photoelectric conversion element of a different type from the photoelectric conversion element around the photoelectric conversion element, an R signal, and G A process of generating a signal, a B signal, and a W signal is performed.

  For example, at the pixel position corresponding to the photoelectric conversion element 51R, the R signal obtained from the photoelectric conversion element 51R, the G signal obtained from the photoelectric conversion elements 51G and 51B around the photoelectric conversion element 51R, and B An R signal, a G signal, and a B signal are generated using the signal, and a W signal is generated using the W signal obtained from the photoelectric conversion element 51W around the photoelectric conversion element 51R. At the pixel position corresponding to the photoelectric conversion element 51G, a G signal obtained from the photoelectric conversion element 51G, and an R signal and a B signal obtained from the photoelectric conversion elements 51R and 51B around the photoelectric conversion element 51G. Are used to generate an R signal, a G signal, and a B signal, and a W signal is generated using a W signal obtained from the photoelectric conversion element 51W around the photoelectric conversion element 51G. At the pixel position corresponding to the photoelectric conversion element 51B, the B signal obtained from the photoelectric conversion element 51B and the R signal and G signal obtained from the photoelectric conversion elements 51R and 51G around the photoelectric conversion element 51B are provided. The R signal, the G signal, and the B signal are generated using the W signal, and the W signal is generated using the W signal obtained from the photoelectric conversion element 51W around the photoelectric conversion element 51B. At the pixel position corresponding to the photoelectric conversion element 51W, a W signal is generated using the W signal obtained from the photoelectric conversion element 51W, and the photoelectric conversion elements 51R, 51G, and 51B around the photoelectric conversion element 51W are generated. The R signal, G signal, and B signal are generated using the obtained R signal, G signal, and B signal.

  By this process, four signals of R signal, G signal, B signal, and W signal are generated at pixel positions corresponding to the photoelectric conversion elements 51R, 51G, 51B, 51W on the memory. .

  In the above description, the R, G, and B signals to be generated at each pixel position are generated using the color signals obtained from the photoelectric conversion element 51R, the photoelectric conversion element 51G, and the photoelectric conversion element 51B. However, this may be generated using the W signal obtained from the photoelectric conversion element 51W together with the color signal. Hereinafter, a method for generating the R, G, and B signals in this case will be described.

  FIG. 4 is a diagram showing the arrangement of the imaging signals output from the A / D conversion circuit 7. In FIG. 4, a circle with “R” inside indicates an R signal obtained from the photoelectric conversion element 51 </ b> R, a circle with “G” inside indicates a G signal obtained from the photoelectric conversion element 51 </ b> G, A circle with “B” inside shows the B signal obtained from the photoelectric conversion element 51B, and a circle with “W” inside shows the W signal obtained from the photoelectric conversion element 51W. The position where each signal exists corresponds to the position of the photoelectric conversion element from which the signal is output, and this position is defined as a pixel position. A number (01 to 32) indicating the position is given above each pixel position.

  As shown in FIG. 4, when the imaging signal output from the A / D conversion circuit 7 is viewed in the Z direction that intersects the horizontal direction (X direction) and the vertical direction (Y direction) at an angle of 45 degrees, A first column in which W signals and G signals are alternately arranged in the Z direction, and a second column in which R signals and B signals are alternately arranged in the Z direction across the W signal are orthogonal to the Z direction. The configuration is arranged alternately in the orthogonal direction. Further, when viewed in the Y direction, a third column in which G signals and R signals are alternately arranged in the Y direction and a fourth column in which G signals and B signals are alternately arranged in the Y direction are represented by W In this configuration, the signals are alternately arranged in the X direction across the fifth column in which the signals are arranged in the Y direction.

  The RGBW interpolation unit 17a determines in which direction each of a large number of imaging signals output from the A / D conversion circuit 7 is correlated by using, for example, a W signal adjacent to the imaging signal. For an imaging signal correlated in the Z direction or a direction orthogonal thereto, the color signal to be interpolated at the pixel position where the imaging signal exists is the same as the color signal to be interpolated around the pixel position. Luminance utilization estimation processing that estimates using a type of color signal and W signal, and a color signal to be interpolated at the pixel position uses the same type of color signal as the color signal to be interpolated around the pixel position The luminance signal non-use estimation process estimated in this way interpolates the color signal that does not exist at the pixel position where each of the R signal, G signal, and B signal exists.

  Further, the RGBW interpolation unit 17a should interpolate the color signal to be interpolated at the pixel position where the image signal exists for the image signal correlated in the X direction or the Y direction. G use estimation processing that estimates using the same kind of color signal and G signal as the color signal, and the same color signal as the color signal to be interpolated around the pixel position. The color signal that does not exist is interpolated at the pixel position where each of the R signal, the G signal, and the B signal exists by the G non-use estimation process that estimates using the color signal.

  For example, when determining in which direction G10 shown in FIG. 4 is correlated, the RGBW interpolation unit 17a performs the Y direction: (W05 + W13) / 2− (W06 + W14) / 2, the X direction: (W05 + W06) / 2- (W13 + W14) / 2, Z direction: (W06-W13), direction orthogonal to the Z direction: (W05-W14), and the direction in which the absolute value is the smallest among the calculation results, It is determined that the direction has a correlation with G10.

  In the following description, for convenience, the R signal existing at the pixel position ** is described as “R **”, the G signal existing at the pixel position ** is described as “G **”, and the pixel The B signal existing at the position ** is described as “B **”, and the W signal existing at the pixel position ** is described as “W **”.

  Hereinafter, a method for interpolating a color signal at a pixel position where an imaging signal having a correlation in the Z direction, for example, will be described in detail.

(A) When R signal is interpolated at a pixel position where B signal exists In this case, the RGBW interpolation unit 17a estimates an R signal to be interpolated by luminance non-use estimation processing. That is, the RGBW interpolation unit 17a uses the R signal to be interpolated at the pixel position where the B signal exists, and the R signal near the pixel position included in the second column including the B signal present at the pixel position. presume.
For example, when the R signal (R11) is interpolated at the pixel position 11, the synchronization processing unit 12 calculates the average of the R signals (R04, R18) in the vicinity of B11 included in the second column including B11. R11 is estimated from (1). R11 = (R04 + R18) / 2 (1)

(B) When interpolating the B signal at the pixel position where the R signal exists In this case, the RGBW interpolation unit 17a estimates the B signal to be interpolated by the luminance non-use estimation process. That is, the RGBW interpolation unit 17a uses the B signal to be interpolated at the pixel position where the R signal exists, and the B signal near the pixel position included in the second column including the R signal existing at the pixel position. presume.
For example, when the B signal (B18) is interpolated at the pixel position 18, the synchronization processing unit 12 calculates the average of the B signals (B11, B25) in the vicinity of R18 included in the second column including R18. B18 is estimated from (2). B18 = (B11 + B25) / 2 (2)

(C) When G Signal is Interpolated to the Pixel Position where the B Signal is Present and the Pixel Position where the R Signal is Present In this case, the RGBW interpolation unit 17a estimates the G signal to be interpolated by the luminance use estimation process. That is, the G signal to be interpolated to the pixel position where the B signal exists and the pixel position where the R signal exists respectively, and the W near the pixel position included in the second column including the color signal existing at the pixel position. The estimation is performed using the signal, the W signal in the vicinity of the pixel position in the first column adjacent to the second column, and the G signal in the vicinity of the W signal in the first column.

  For example, when the G signal (G11) is interpolated at the pixel position 11, it is assumed that the difference between the W signal and the G signal in the local region of the image is equal in an object correlated in the Z direction, that is, G11−. Based on the premise that the relationship of W11 = {(G06-W06) + (G15-W15)} / 2 is established, G11 = W11 + {(G06-W06) + (G15-W15)} / 2. G11 is estimated by the calculation of 3).

  Specifically, the RGBW interpolation unit 17a first uses the luminance signal (W07, W14) in the vicinity of the pixel position 11 included in the second column including B11 existing at the pixel position 11 at the pixel position 11. (W11) is interpolated, and the pixel position 06 which is the pixel position adjacent to the pixel position 11 in the direction orthogonal to the Z direction is near the pixel position 06 included in the first column including W06 existing at the pixel position 06. G06 is interpolated using the G signal (G03, G10), and the first column including W15 present at the pixel position 15 at the pixel position 15 that is the pixel position adjacent to the pixel position 11 in the direction orthogonal to the Z direction. G15 is interpolated using G signals (G12, G19) in the vicinity of the pixel position 15 included in.

  W11 is obtained, for example, by calculating the average of W07 and W14, G06 is obtained, for example, by calculating the average of G03 and G10, and G15 is obtained, for example, by calculating the average of G12 and G19. . After obtaining W11, G06, and G15, G11 is estimated by performing an operation of W11 + {(G06-W06) + (G15−W15)} / 2 according to the equation (3). In the same way, the G signal is also interpolated at the pixel position where the R signal exists.

  Note that Equation (3) estimates G11 based on the correlation between the pixel position 11 and the pixel positions 06 and 15, but the correlation between the pixel position 11 and the pixel position 06 or the pixel position 11 and the pixel. It is also possible to estimate G11 based on the correlation with the position 15. In this case, the above equation (3) may be used as modified as the following equation (4) or (5). G11 = W11 + (G06-W06) (4), G11 = W11 + (G15-W15) (5).

(D) When R signal and B signal are interpolated at the pixel position where the G signal exists In this case, the RGBW interpolation unit 17a estimates the R signal and the B signal to be interpolated by the luminance use estimation process. That is, the R signal to be interpolated at the pixel position where the G signal exists, the W signal in the vicinity of the pixel position included in the first column including the G signal existing at the pixel position, and the first column. The estimation is performed using the R signal in the vicinity of the pixel position in the second column and the W signal in the vicinity of the R signal in the second column. Also, the B signal to be interpolated at the pixel position where the G signal exists, the W signal near the pixel position included in the first column including the G signal present at the pixel position, and the first column adjacent to the W signal. Is estimated using the B signal in the vicinity of the pixel position in the second column and the W signal in the vicinity of the B signal in the second column.

For example, when the R signal (R19) is interpolated at the pixel position 19, in the subject having a correlation in the Z direction, the difference between the W signal and the R signal in the local region of the image is equal, that is, R19−. Based on the premise that the relationship of W19 = {(R18−W18) + (R20−W20)} / 2 is established, R19 = W19 + {(R18−W18) + (R20−W20)} / 2. R19 is estimated by the calculation of 6).
Similarly, based on the premise that the relationship B19−W19 = {(B11−W11) + (B27−W27)} / 2 is established, B19 = W19 + {(B11−W11) + (B27−W27)} / 2... B19 is estimated by the calculation of (7).

Specifically, the RGBW interpolation unit 17a uses the luminance signal (W19, W22) near the pixel position 19 included in the first column including G19 existing at the pixel position 19 as the luminance signal (W19). ) To the pixel position 18 that is the pixel position adjacent to the pixel position 19 in the X direction, and the W signal (W14, W21) in the vicinity of the pixel position 18 included in the second column including R18 existing at the pixel position 18. ) Is used to interpolate W18, and the W signal in the vicinity of the pixel position 20 included in the second column including R20 existing at the pixel position 20 is added to the pixel position 20 that is the pixel position adjacent to the pixel position 19 in the X direction. W20 is interpolated using (W16, W23).
In addition, the RGBW interpolation unit 17a sets the W signal (in the vicinity of the pixel position 11 included in the second column including B11 existing at the pixel position 11) to the pixel position 11 which is the pixel position adjacent to the pixel position 19 in the Y direction. W11 is interpolated using W07, W14), and the pixel position 27 that is adjacent to the pixel position 19 in the Y direction is adjacent to the pixel position 27 included in the second column including B27 that exists at the pixel position 27. W27 is interpolated using the W signal (W23, W30).

  For example, W19 is obtained by calculating the average of W15 and W22, W18 is obtained by calculating the average of W14 and W21, and W20 is obtained by calculating the average of W16 and W23, for example. , W11 is obtained, for example, by calculating the average of W07 and W14, and W27 is obtained, for example, by calculating the average of W23 and W30. After obtaining W18, W19, W20, W11, and W27, R19 is estimated by calculating W19 + {(R18−W18) + (R20−W20)} / 2 according to Equation (6), B19 is estimated by calculating W19 + {(B11-W11) + (B27-W27)} / 2 according to (7).

  Note that Equation (6) estimates R19 based on the correlation between the pixel position 19 and the pixel positions 18 and 20, but the correlation between the pixel position 19 and the pixel position 18 or the pixel position 19 and the pixel. It is also possible to estimate R19 based on the correlation with the position 20. In this case, the above formula (6) may be used as modified as the following formula (8) or (9). R19 = W19 + (R18−W18) (8), R19 = W19 + (R20−W20) (9).

  Expression (7) estimates B19 based on the correlation between the pixel position 19 and the pixel positions 11 and 27, but the correlation between the pixel position 19 and the pixel position 11 or the pixel position 19 and the pixel. It is also possible to estimate B19 based on the correlation with the position 27. In this case, the above equation (7) may be used as modified as the following equation (10) or (11). B19 = W19 + (B11−W11) (10), B19 = W19 + (B27−W27) (11).

  By performing the processes (a) to (d), it is possible to generate an R signal, a G signal, and a B signal at each pixel position corresponding to the photoelectric conversion elements 51R, 51G, and 51B. The R signal, the G signal, and the B signal may be generated by interpolation using the R signal, the G signal, and the B signal generated at other pixel positions at the pixel position corresponding to the photoelectric conversion element 51W.

  The luminance / color difference signal generation unit 17b generates a luminance signal Y and a color difference signal C corresponding to the pixel position from the R signal, G signal, and B signal at each pixel position generated by the RGBW interpolation unit 17a. As a method for generating the luminance signal Y and the color difference signal C, a known method can be adopted.

  The multiplier 17d multiplies the color difference signal at the arbitrary pixel position generated by the luminance / color difference signal generation unit 17b by the correction coefficient K calculated based on the four imaging signals at the arbitrary pixel position, thereby obtaining the color difference. Correct the signal.

  The correction coefficient calculator 17c calculates the correction coefficient K by the following equation (a).

  K = W / (α × R + β × G + γ × B + δ) (a)

  W in the equation (a) is a W signal generated at the arbitrary pixel position. R in the formula (a) is an R signal generated at the arbitrary pixel position. G in the formula (a) is a G signal generated at the arbitrary pixel position. B in the equation (a) is a B signal generated at the arbitrary pixel position. Α, β, γ, and δ in the formula (a) are approximated spectral spectra when the spectral characteristics of the photoelectric conversion element 51W are approximated by the product sum of the spectral characteristics of the photoelectric conversion elements 51R, 51G, and 51B. This coefficient is determined in advance so that an error between the characteristic and the spectral characteristic of the photoelectric conversion element 51W is minimized.

  For example, the same amount of light is applied to each of the photoelectric conversion elements 51R, 51G, 51B, and 51W, and the R signal, G signal, and B signal obtained from each of the photoelectric conversion elements 51R, 51G, and 51B by this light are expressed by the formula ( The values of α, β, γ, and δ may be determined so that the signal obtained by substituting into the denominator on the right side of a) is closest to the W signal obtained from the photoelectric conversion element 51W by this light.

  Thus, if the values of α, β, γ, and δ are optimized, the value of the correction coefficient K is almost “1”. Here, when the correction coefficient K is expressed separately as a signal component and a noise component, the following equation (b) is obtained.

  K = (Ws + Wn) / (α × Rs + β × Gs + γ × Bs + α × Rn + β × Gn + γ × Bn + δ) (b)

  Xs (X = W, R, G, B) in the formula (b) is a signal component included in the X signal. Xn (X = W, R, G, B) in the formula (b) is a noise component included in the X signal.

  Since the W signal has higher sensitivity than the R signal, the G signal, and the B signal, when the signal level of each signal is the same, noise components included in the R signal, the G signal, and the B signal are converted into the W signal. It becomes larger than the noise component contained. As the noise component contained in the R signal, G signal, and B signal increases, the value of the correction coefficient K decreases as can be seen from the equation (b). In addition, when the signal component included in the R signal, G signal, and B signal is reduced, the noise component included in the R signal, G signal, and B signal is relatively increased. In this case, the value of the correction coefficient K is “ Less than 1 ″.

  In general, when a noise component included in an R signal, a G signal, and a B signal as a generation source of a color difference signal increases, color noise tends to increase in proportion thereto. That is, the correction coefficient K can also be regarded as an index representing the degree of the magnitude of color noise included in the color difference signal.

  For example, if K = 1/2, even if the noise of the W signal is considered to be zero, the noise components of the R signal, the G signal, and the B signal are almost the same level as the signal components of the R signal, the G signal, and the B signal. It can be said that the color noise is large. Therefore, the color noise can be suppressed by generating the color difference signal C ′ by multiplying the color difference signal C obtained by the luminance / color difference signal generation unit 17b by the correction coefficient K in the multiplier 17d. .

The operation of the digital camera configured as described above will be described.
When shooting is performed, an image pickup signal is output from each photoelectric conversion element of the solid-state image pickup element 5, and after this image pickup signal is converted into a digital signal, it is temporarily stored in the main memory 16. After that, the image signal is interpolated using the image signal at the surrounding pixel position at the pixel position corresponding to each photoelectric conversion element of the solid-state image sensor 5 on the main memory 16 to correspond to each photoelectric conversion element. An R signal, a G signal, a B signal, and a W signal are generated at the pixel position.

  Next, a luminance signal Y and a color difference signal C are generated from the R signal, G signal, and B signal at each pixel position, and each of the R signal, G signal, B signal, and W signal at each pixel position is generated. A correction coefficient K corresponding to the pixel position is calculated. Then, the color difference signal C ′ is generated by multiplying the color difference signal C generated at each pixel position by the correction coefficient K corresponding to each pixel position.

  When the luminance signal Y and the color difference signal C ′ are generated at the pixel position corresponding to each photoelectric conversion element and the generation of the color image data is completed, the color image data is compressed and recorded on the recording medium 21.

  As described above, according to the digital camera of the present embodiment, the color difference signal generated at the pixel position is corrected according to the R signal, G signal, B signal, and W signal generated at each pixel position. Optimal correction can be performed for each location of the subject. For example, even for a subject with a mixture of bright and dark areas, the optimum color difference signal correction is automatically performed between the bright and dark areas, so image quality can be reduced without complicated processing. Can be prevented. Further, according to such a color difference signal correction method, since the hue does not change, the deterioration of the image quality can be prevented from this point.

  In the above description, the R, G, and B signals generated by the RGBW interpolation unit 17a as R, G, and B data to be substituted into the equation (a) by the correction coefficient calculation unit 17c, Although the example using B signal was demonstrated, it is not restricted to this. The correction coefficient calculation unit 17c generates R, G, and B signals at each pixel position by a method different from the method performed by the RGBW interpolation unit 17a, and substitutes them into the above equation (a). It may be used as B data.

  As a method different from the method performed by the RGBW interpolation unit 17a, for example, a method obtained by simplifying the method performed by the RGBW interpolation unit 17a can be employed. For example, when the RGBW interpolation unit 17a generates a signal using the R signal, the G signal, the B signal, and the W signal, the correction is performed using the data generated using only the R, G, and B signals. The coefficient K may be obtained.

  Hereinafter, other embodiments of the digital camera of this embodiment will be listed.

The correction coefficient calculator 17c preferably clips the value of the correction coefficient K to “1” when the calculated correction coefficient K is larger than “1”.
If the correction coefficient K is larger than “1”, it can be said that the color noise is low. For this reason, if the color difference signal C is multiplied by the correction coefficient K in this state, the color noise is emphasized. Therefore, in such a case, by setting the correction coefficient K to “1”, it is possible to prevent erroneous correction and to prevent image quality deterioration.

  The correction coefficient calculation unit 17c does not determine K based on the ratio of W and (α × R + β × G + γ × B + δ) as in equation (1), but instead of W and (α × R + β × G + γ × B + δ). The magnitudes may be compared, and the correction coefficient K may be determined according to the comparison result. For example, a value (A × W) obtained by multiplying the numerator on the right side of Equation (1) by a coefficient A (A × W) and a value obtained by multiplying the denominator on the right side of Equation (1) by a coefficient B (B × (α × R + β × G + γ × B + δ). )) May be compared while changing the coefficients A and B, and the correction coefficient K may be determined according to the magnitude relationship.

In this case, the correction coefficient K for each of the following conditions (a) to (i) is stored in the digital camera.
(A) If A = 1, B = 1 and (A × W) ≧ (B × (α × R + β × G + γ × B + δ)), the correction coefficient K = 1
(B) If A = 4, B = 3 and (A × W) = (B × (α × R + β × G + γ × B + δ)), the correction coefficient K = 0.75.
(C) If A = 2, B = 1 and (A × W) = (B × (α × R + β × G + γ × B + δ)), the correction coefficient K = 0.5.
(D) If A = 4, B = 1 and (A × W) = (B × (α × R + β × G + γ × B + δ)), the correction coefficient K = 0.25.
(E) If A = 10, B = 1, and (A × W) ≦ (B × (α × R + β × G + γ × B + δ)), the correction coefficient K = 0.1
(F) When A = 1 and B = 1, (A × W) <(B × (α × R + β × G + γ × B + δ)), and when A = 4 and B = 3, (A × W)> (B × (Α × R + β × G + γ × B + δ)), the correction coefficient K = 0.88
(G) A = 4, B = 3, (A × W) <(B × (α × R + β × G + γ × B + δ)), and A = 2, B = 1, (A × W)> (B × (Α × R + β × G + γ × B + δ)), the correction coefficient K = 0.63
(H) When A = 2 and B = 1, (A × W) <(B × (α × R + β × G + γ × B + δ)), and when A = 4 and B = 1, (A × W)> (B × (Α × R + β × G + γ × B + δ)), the correction coefficient K = 0.38.
(I) When A = 4 and B = 1, (A × W) <(B × (α × R + β × G + γ × B + δ)), and when A = 10 and B = 1, (A × W)> (B × (Α × R + β × G + γ × B + δ)), the correction coefficient K = 0.18

  The correction coefficient calculation unit 17c compares (A × W) and (B × (α × R + β × G + γ × B + δ)) using the imaging signal at each pixel position, and stores the correction coefficient K corresponding to the comparison result in the memory. Is read out and input to the multiplier 17d.

  In order to obtain the ratio of W to (α × R + β × G + γ × B + δ), a divider is required, but the divider requires an enormous amount of circuit. As described above, if the method of determining the correction coefficient K only by comparing the magnitudes of (A × W) and (B × (α × R + β × G + γ × B + δ)) is adopted, a divider is unnecessary and the size is reduced. And cost reduction.

  The arrangement of the photoelectric conversion elements of the solid-state imaging element 5 is not limited to that shown in FIG. 1, but the photoelectric conversion element 51R, the photoelectric conversion element 51G, the photoelectric conversion element 51B, and the photoelectric conversion element 51W are one-dimensional or Any configuration that is two-dimensionally arranged may be used.

FIG. 5 is a diagram illustrating a modification of the arrangement of the photoelectric conversion elements of the solid-state imaging device 5.
As shown in FIG. 5, a large number of photoelectric conversion elements composed of photoelectric conversion elements 51R, 51G, 51B, 51W are arranged in a square lattice pattern on a semiconductor substrate, and an RGB photoelectric conversion element composed of photoelectric conversion elements 51R, 51G, 51B. A group and a W photoelectric conversion element group made up of photoelectric conversion elements 51W may be arranged in a checkered pattern so that a checkered pattern is formed when different colors are applied to the respective photoelectric conversion elements. good.

The filter provided above the RGB photoelectric conversion element group shown in FIGS. 1 and 5 may be a complementary color filter instead of a primary color filter.
Even when the complementary color filter is used, it is possible to generate an R component signal, a G component signal, a B component signal, and a luminance component signal at the pixel position corresponding to each photoelectric conversion element of the solid-state imaging device. Therefore, the present invention can be applied.

  In the example shown in FIG. 1, color image data composed of the same number of pixel data as the number of all the photoelectric conversion elements included in the solid-state imaging element 5 is generated, but the photoelectric conversion elements constituting the RGB photoelectric conversion element group There may be a photographing mode for generating color image data composed of the same number of pixel data as the number of pixel data.

  In this case, the RGBW interpolation unit 17a uses the R signal, the G signal, the B signal, and the W signal output from the A / D conversion circuit 7 to correspond to each of the photoelectric conversion elements 51R, 51G, and 51B on the memory. Processing for generating an R signal, a G signal, a B signal, and a W signal is performed at the position.

  As the W signal to be generated at the pixel position, the W signal obtained from the photoelectric conversion element 51W adjacent to the pixel position may be used as it is. The R, G, and B signals to be generated at the pixel position may be generated using at least the R signal, the G signal, and the B signal among the R signal, the G signal, the B signal, and the W signal.

  Then, color difference signals and luminance signals are generated from the generated signals at pixel positions corresponding to the photoelectric conversion elements 51R, 51G, and 51B, and the color difference signals are corrected by the above-described method to complete color image data. Just do it.

That is, the pixel position where the luminance signal and the color difference signal are finally generated corresponds to the coordinate position of the pixel data constituting the color image data in the claims.
When generating color image data composed of the same number of pixel data as the number of all the photoelectric conversion elements included in the solid-state image sensor 5, the pixel positions corresponding to all the photoelectric conversion elements included in the solid-state image sensor 5 When generating color image data consisting of the same number of pixel data as the number of photoelectric conversion elements constituting the RGB photoelectric conversion element group at the coordinate position, it corresponds to the photoelectric conversion element constituting the RGB photoelectric conversion element group The pixel position to perform becomes the coordinate position.

Schematic diagram of a partial plane of an image sensor mounted on an image pickup apparatus according to an embodiment of the present invention The figure which shows schematic structure of the digital camera which is an example of the imaging device carrying the solid-state image sensor shown in FIG. The figure which shows the internal structure of the digital signal processing part shown in FIG. The figure which showed the arrangement | sequence of the imaging signal output from the A / D conversion circuit 7 The figure which shows the modification of the arrangement | sequence of the photoelectric conversion element of the solid-state image sensor 5 shown in FIG.

Explanation of symbols

5 Solid-state image sensor 17a RGBW interpolation unit 17b Luminance / color difference signal generation unit 17c Correction coefficient calculation unit 17d Multipliers 51R, 51G, 51B, 51W Photoelectric conversion elements

Claims (6)

A first photoelectric conversion element that detects light of a first color, a second photoelectric conversion element that detects light of a second color, and a third photoelectric conversion element that detects light of a third color; An imaging device having a solid-state imaging device including a fourth photoelectric conversion element for detecting a luminance component of light,
Obtained from the signal obtained from the first photoelectric conversion element, the signal obtained from the second photoelectric conversion element, and the third photoelectric conversion element at the coordinate position of the pixel data constituting the color image data to be generated Of the signal and the signal obtained from the fourth photoelectric conversion element, at least the signal obtained from the first photoelectric conversion element, the signal obtained from the second photoelectric conversion element, and the third photoelectric conversion element RGB signal generation means for generating an R (red) component signal, a G (green) component signal, and a B (blue) component signal using the signal obtained from
Color difference signal generating means for generating a color difference signal corresponding to the coordinate position from the R component signal, the G component signal, and the B component signal generated at the coordinate position;
A correction luminance signal generating means for generating a signal of a luminance component for correction of a color difference signal corresponding to the coordinate position using a signal obtained from the fourth photoelectric conversion element at the coordinate position;
Of the signal obtained from the first photoelectric conversion element, the signal obtained from the second photoelectric conversion element, the signal obtained from the third photoelectric conversion element, and the signal obtained from the fourth photoelectric conversion element , At least a signal obtained from the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, and a signal obtained from the third photoelectric conversion element, and the correction generated at the coordinate position And a color difference signal correcting unit that corrects a color difference signal corresponding to the coordinate position based on the luminance component signal. The imaging apparatus according to claim 1,
The color difference signal correcting unit is configured to generate the R component signal, the G component signal, and the B component signal generated at the coordinate position by the RGB signal generating unit, and the correction signal generated at the coordinate position. An image pickup apparatus that corrects a color difference signal corresponding to the coordinate position using a signal of a luminance component. The imaging apparatus according to claim 1,
The color difference signal correcting means is a signal obtained from the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, a signal obtained from the third photoelectric conversion element, and the fourth photoelectric conversion. Among the signals obtained from the element, at least the signal obtained from the first photoelectric conversion element, the signal obtained from the second photoelectric conversion element, and the signal obtained from the third photoelectric conversion element, The R component signal for correcting the color difference signal, the G component signal, and the B component signal are generated at the coordinate position by a method different from the RGB signal generating means, and the R component signal for correction is generated. An imaging device that corrects the color difference signal using a component signal, the G component signal, the B component signal, and the correction luminance component signal. The imaging apparatus according to claim 2,
The color difference signal correcting means includes a multiplying means for multiplying the color difference signal by a correction coefficient to obtain a corrected color difference signal, and a correction coefficient calculating means for calculating the correction coefficient.
The correction coefficient calculating means uses the correction coefficient as K and the following equation:
K = W / (α × R + β × G + γ × B + δ)
W: Signal of the correction luminance component generated at the coordinate position R: Signal of the R component generated at the coordinate position G: Signal of the G component generated at the coordinate position B: The coordinate position The B component signals α, β, γ, and δ generated in the above are the spectral characteristics of the fourth photoelectric conversion element, the first photoelectric conversion element, the second photoelectric conversion element, and the third photoelectric conversion element. When approximated by the product sum of each spectral characteristic of the photoelectric conversion element, the correction coefficient is determined by a coefficient determined so that an error between the approximated spectral characteristic and the spectral characteristic of the fourth photoelectric conversion element is minimized. An imaging device for calculating The imaging apparatus according to claim 3,
The color difference signal correcting means includes a multiplying means for multiplying the color difference signal by a correction coefficient to obtain a corrected color difference signal, and a correction coefficient calculating means for calculating the correction coefficient.
The correction coefficient calculating means uses the correction coefficient as K and the following equation:
K = W / (α × R + β × G + γ × B + δ)
W: A signal of the correction luminance component generated at the coordinate position R: A signal of the correction R component generated at the coordinate position G: A signal of the G component for correction generated at the coordinate position Signal B: B component signal for correction generated at the coordinate position α, β, γ, δ: Spectral characteristics of the fourth photoelectric conversion element, the first photoelectric conversion element, the second photoelectric conversion element When approximated by the product sum of the respective spectral characteristics of the photoelectric conversion element and the third photoelectric conversion element, an error between the approximated spectral characteristic and the spectral characteristic of the fourth photoelectric conversion element is minimized. An imaging device that calculates the correction coefficient according to a coefficient determined by The imaging apparatus according to claim 4 or 5, wherein
An imaging apparatus that clips the value of K to “1” when the value of K is greater than “1” as a result of the calculation by the correction coefficient calculation means.

Images