JP5033702B2 - Imaging device - Google Patents

Imaging device Download PDF

Info

Publication number
JP5033702B2
JP5033702B2 JP2008098353A JP2008098353A JP5033702B2 JP 5033702 B2 JP5033702 B2 JP 5033702B2 JP 2008098353 A JP2008098353 A JP 2008098353A JP 2008098353 A JP2008098353 A JP 2008098353A JP 5033702 B2 JP5033702 B2 JP 5033702B2
Authority
JP
Japan
Prior art keywords
signal
photoelectric conversion
conversion element
component
coordinate position
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP2008098353A
Other languages
Japanese (ja)
Other versions
JP2009253616A5 (en
JP2009253616A (en
Inventor
和田  哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to JP2008098353A priority Critical patent/JP5033702B2/en
Publication of JP2009253616A publication Critical patent/JP2009253616A/en
Publication of JP2009253616A5 publication Critical patent/JP2009253616A5/ja
Application granted granted Critical
Publication of JP5033702B2 publication Critical patent/JP5033702B2/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Color Television Image Signal Generators (AREA)

Description

本発明は、第一の色の光を検出する第一の光電変換素子と、第二の色の光を検出する第二の光電変換素子と、第三の色の光を検出する第三の光電変換素子と、光の輝度成分を検出する第四の光電変換素子とを含む固体撮像素子を有する撮像装置に関する。   The present invention provides a first photoelectric conversion element that detects light of a first color, a second photoelectric conversion element that detects light of a second color, and a third photoelectric element that detects light of a third color. The present invention relates to an imaging apparatus having a solid-state imaging device including a photoelectric conversion device and a fourth photoelectric conversion device that detects a luminance component of light.

R光を検出する第一の光電変換素子と、G光を検出する第二の光電変換素子と、B光を検出する第三の光電変換素子と、光の輝度成分を検出する第四の光電変換素子とを含む固体撮像素子を有する撮像装置が、例えば特許文献1、2に開示されている。   A first photoelectric conversion element that detects R light, a second photoelectric conversion element that detects G light, a third photoelectric conversion element that detects B light, and a fourth photoelectric element that detects a luminance component of light For example, Patent Documents 1 and 2 disclose an imaging apparatus having a solid-state imaging element including a conversion element.

これら撮像装置は、第四の光電変換素子からの信号を用いて輝度の高感度化を図っているが、色差信号については、第四の光電変換素子からの信号を利用した処理は行っておらず、第四の光電変換素子を設けたメリットが活かしきれていない。   These image pickup devices use the signal from the fourth photoelectric conversion element to increase the brightness, but the color difference signal is not processed using the signal from the fourth photoelectric conversion element. The merit of providing the fourth photoelectric conversion element has not been fully utilized.

一方、特許文献3には、色信号が得られる光電変換素子と、輝度信号が得られる光電変換素子とを含む固体撮像素子を備える撮像装置において、輝度信号と色信号との差分により、各画素のRGB信号を生成する方法が開示されている。しかし、この方法では、信号の差分を求めているためノイズ増加が発生してしまう。   On the other hand, in Patent Document 3, in an imaging apparatus including a solid-state imaging device including a photoelectric conversion element that can obtain a color signal and a photoelectric conversion element that can obtain a luminance signal, each pixel is determined based on the difference between the luminance signal and the color signal. A method for generating a RGB signal is disclosed. However, in this method, since the signal difference is obtained, an increase in noise occurs.

特開2007−258686号公報JP 2007-258686 A 特開2007−243334号公報JP 2007-243334 A 特開2007−27667号公報JP 2007-27667 A

本発明は、上記事情に鑑みてなされたものであり、色ノイズを低減することが可能な撮像装置を提供することを目的とする。   The present invention has been made in view of the above circumstances, and an object thereof is to provide an imaging apparatus capable of reducing color noise.

本発明の撮像装置は、第一の色の光を検出する第一の光電変換素子と、第二の色の光を検出する第二の光電変換素子と、第三の色の光を検出する第三の光電変換素子と、光の輝度成分を検出する第四の光電変換素子とを含む固体撮像素子を有する撮像装置であって、生成すべきカラー画像データを構成する画素データの座標位置に、前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、前記第三の光電変換素子から得られる信号、及び前記第四の光電変換素子から得られる信号のうち、少なくとも前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、及び前記第三の光電変換素子から得られる信号を用いて、R(赤色)成分の信号と、G(緑色)成分の信号と、B(青色)成分の信号とを生成するRGB信号生成手段と、前記座標位置に生成された前記R成分の信号、前記G成分の信号、及び前記B成分の信号から前記座標位置に対応する色差信号を生成する色差信号生成手段と、前記座標位置に、前記第四の光電変換素子から得られる信号を用いて、前記座標位置に対応する色差信号の補正用の輝度成分の信号を生成する補正用輝度信号生成手段と、前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、前記第三の光電変換素子から得られる信号、及び前記第四の光電変換素子から得られる信号のうち、少なくとも前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、及び前記第三の光電変換素子から得られる信号と、前記座標位置に生成された前記補正用の輝度成分の信号とに基づいて、前記座標位置に対応する色差信号を補正する色差信号補正手段とを備える。   The imaging apparatus of the present invention detects a first photoelectric conversion element that detects light of a first color, a second photoelectric conversion element that detects light of a second color, and light of a third color. An image pickup apparatus having a solid-state image pickup device including a third photoelectric conversion device and a fourth photoelectric conversion device for detecting a luminance component of light, at a coordinate position of pixel data constituting color image data to be generated , A signal obtained from the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, a signal obtained from the third photoelectric conversion element, and a signal obtained from the fourth photoelectric conversion element Among these, at least a signal obtained from the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, and a signal obtained from the third photoelectric conversion element, a signal of an R (red) component And G (green) component signal and B (blue) RGB signal generation means for generating a component signal, and a color difference signal corresponding to the coordinate position from the R component signal, the G component signal, and the B component signal generated at the coordinate position. Using a signal obtained from the fourth photoelectric conversion element at the coordinate position, a correction luminance signal generation unit that generates a luminance component signal for correcting the color difference signal corresponding to the coordinate position. Means, a signal obtained from the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, a signal obtained from the third photoelectric conversion element, and a signal obtained from the fourth photoelectric conversion element Of the signals, at least the signal obtained from the first photoelectric conversion element, the signal obtained from the second photoelectric conversion element, and the signal obtained from the third photoelectric conversion element, are generated at the coordinate position. Based on the signal of the luminance component for the correction that is, and a color difference signal correcting means for correcting the color difference signal corresponding to the coordinate position.

本発明の撮像装置は、前記色差信号補正手段が、前記RGB信号生成手段によって前記座標位置に生成された前記R成分の信号、前記G成分の信号、及び前記B成分の信号と、前記座標位置に生成された前記補正用の輝度成分の信号とを用いて、前記座標位置に対応する色差信号を補正する。   In the imaging apparatus according to the aspect of the invention, the color difference signal correction unit may include the R component signal, the G component signal, and the B component signal generated at the coordinate position by the RGB signal generation unit, and the coordinate position. The color difference signal corresponding to the coordinate position is corrected using the correction luminance component signal generated in step (b).

本発明の撮像装置は、前記色差信号補正手段が、前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、前記第三の光電変換素子から得られる信号、及び前記第四の光電変換素子から得られる信号のうち、少なくとも前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、及び前記第三の光電変換素子から得られる信号を用いて、前記RGB信号生成手段とは別の方法で、前記座標位置に前記色差信号の補正用の前記R成分の信号、前記G成分の信号、及び前記B成分の信号を生成し、前記補正用の前記R成分の信号、前記G成分の信号、及び前記B成分の信号と、前記補正用の輝度成分の信号とを用いて、前記色差信号を補正する。   In the image pickup apparatus of the present invention, the color difference signal correction unit includes a signal obtained from the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, a signal obtained from the third photoelectric conversion element, And a signal obtained from at least the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, and a signal obtained from the third photoelectric conversion element among the signals obtained from the fourth photoelectric conversion element The R component signal for correcting the color difference signal, the G component signal, and the B component signal are generated at the coordinate position by a method different from the RGB signal generating means. The color difference signal is corrected using the R component signal for correction, the G component signal, the B component signal, and the correction luminance component signal.

本発明の撮像装置は、前記色差信号補正手段が、前記色差信号に補正係数を乗じて補正後の色差信号を求める乗算手段と、前記補正係数を演算する補正係数演算手段とで構成され、前記補正係数演算手段が、前記補正係数をKとして、次の式、
K=W/(α×R+β×G+γ×B+δ)
W:前記座標位置に生成された前記補正用の輝度成分の信号
R:前記座標位置に生成された前記R成分の信号
G:前記座標位置に生成された前記G成分の信号
B:前記座標位置に生成された前記B成分の信号
α,β,γ,δ:前記第四の光電変換素子の分光特性を、前記第一の光電変換素子、前記第二の光電変換素子、及び前記第三の光電変換素子の各々の分光特性の積和で近似したときに、その近似した分光特性と前記第四の光電変換素子の分光特性との誤差が最小となるように決められた係数
により前記補正係数を演算する。
In the imaging apparatus of the present invention, the color difference signal correction unit includes a multiplication unit that multiplies the color difference signal by a correction coefficient to obtain a corrected color difference signal, and a correction coefficient calculation unit that calculates the correction coefficient. The correction coefficient calculation means uses the following equation, where K is the correction coefficient:
K = W / (α × R + β × G + γ × B + δ)
W: Signal of the correction luminance component generated at the coordinate position R: Signal of the R component generated at the coordinate position G: Signal of the G component generated at the coordinate position B: The coordinate position The B component signals α, β, γ, and δ generated in the above are the spectral characteristics of the fourth photoelectric conversion element, the first photoelectric conversion element, the second photoelectric conversion element, and the third photoelectric conversion element. When approximated by the product sum of each spectral characteristic of the photoelectric conversion element, the correction coefficient is determined by a coefficient determined so that an error between the approximated spectral characteristic and the spectral characteristic of the fourth photoelectric conversion element is minimized. Is calculated.

本発明の撮像装置は、前記色差信号補正手段が、前記色差信号に補正係数を乗じて補正後の色差信号を求める乗算手段と、前記補正係数を演算する補正係数演算手段とで構成され、前記補正係数演算手段が、前記補正係数をKとして、次の式、
K=W/(α×R+β×G+γ×B+δ)
W:前記座標位置に生成された前記補正用の輝度成分の信号
R:前記座標位置に生成された前記補正用のR成分の信号
G:前記座標位置に生成された前記補正用のG成分の信号
B:前記座標位置に生成された前記補正用のB成分の信号
α,β,γ,δ:前記第四の光電変換素子の分光特性を、前記第一の光電変換素子、前記第二の光電変換素子、及び前記第三の光電変換素子の各々の分光特性の積和で近似したときに、その近似した分光特性と前記第四の光電変換素子の分光特性との誤差が最小となるように決められた係数
により前記補正係数を演算する。
In the imaging apparatus of the present invention, the color difference signal correction unit includes a multiplication unit that multiplies the color difference signal by a correction coefficient to obtain a corrected color difference signal, and a correction coefficient calculation unit that calculates the correction coefficient. The correction coefficient calculation means uses the following equation, where K is the correction coefficient:
K = W / (α × R + β × G + γ × B + δ)
W: A signal of the correction luminance component generated at the coordinate position R: A signal of the correction R component generated at the coordinate position G: A signal of the G component for correction generated at the coordinate position Signal B: B component signal for correction generated at the coordinate position α, β, γ, δ: Spectral characteristics of the fourth photoelectric conversion element, the first photoelectric conversion element, the second photoelectric conversion element When approximated by the product sum of the spectral characteristics of the photoelectric conversion element and the third photoelectric conversion element, an error between the approximated spectral characteristic and the spectral characteristic of the fourth photoelectric conversion element is minimized. The correction coefficient is calculated according to the coefficient determined in (1).

本発明の撮像装置は、前記補正係数演算手段が、演算の結果、前記Kの値が“1”より大きかった場合、前記Kの値を“1”にクリップする。   In the imaging apparatus of the present invention, the correction coefficient calculation means clips the value of K to “1” when the value of K is larger than “1” as a result of the calculation.

本発明によれば、色ノイズを低減することが可能な撮像装置を提供することができる。   ADVANTAGE OF THE INVENTION According to this invention, the imaging device which can reduce color noise can be provided.

以下、本発明の実施形態について図面を参照して説明する。   Embodiments of the present invention will be described below with reference to the drawings.

図1は、本発明の実施形態である撮像装置に搭載される撮像素子の部分平面模式図である。
図1に示す固体撮像素子5は、半導体基板上の水平方向とこれに直交する垂直方向とに正方格子状に配列されたR光を検出する光電変換素子51R(図中に“R”の文字を付してある)、G光を検出する光電変換素子51G(図中に“G”の文字を付してある)、B光を検出する光電変換素子51B(図中に“B”の文字を付してある)からなるRGB光電変換素子群と、半導体基板上の水平方向とこれに直交する垂直方向とに正方格子状に配列された光の輝度成分を検出する光電変換素子51W(図中に“W”の文字を付してある)からなるW光電変換素子群とを備え、これらが、それぞれの光電変換素子配列ピッチの略1/2だけ、水平方向及び垂直方向にずれた位置に配置されている。尚、RGB光電変換素子群の各光電変換素子とW光電変換素子群の各光電変換素子の配列ピッチは同じである。
FIG. 1 is a schematic partial plan view of an image sensor mounted on an image pickup apparatus according to an embodiment of the present invention.
A solid-state imaging device 5 shown in FIG. 1 includes a photoelectric conversion element 51R (character “R” in the figure) that detects R light arranged in a square lattice in a horizontal direction on a semiconductor substrate and a vertical direction perpendicular thereto. ), Photoelectric conversion element 51G for detecting G light (character “G” in the figure), and photoelectric conversion element 51B for detecting B light (character “B” in the figure) And a photoelectric conversion element 51W for detecting luminance components of light arranged in a square lattice pattern in the horizontal direction on the semiconductor substrate and in the vertical direction perpendicular thereto (see FIG. W photoelectric conversion element group consisting of “W” inside), and these positions are shifted in the horizontal and vertical directions by about ½ of the respective photoelectric conversion element arrangement pitch. Is arranged. The arrangement pitch of each photoelectric conversion element of the RGB photoelectric conversion element group and each photoelectric conversion element of the W photoelectric conversion element group is the same.

RGB光電変換素子群の各光電変換素子の配列は、垂直方向Yに並ぶ光電変換素子51Gと光電変換素子51Rとからなる列であるGR光電変換素子列と、垂直方向Yに並ぶ光電変換素子51Bと光電変換素子51Gとからなる列であるBG光電変換素子列とを、水平方向に交互に配列したものと言うことができる。   The arrangement of the photoelectric conversion elements of the RGB photoelectric conversion element group includes a GR photoelectric conversion element array that is a line made of photoelectric conversion elements 51G and 51R arranged in the vertical direction Y, and a photoelectric conversion element 51B that is aligned in the vertical direction Y. It can be said that the BG photoelectric conversion element rows, which are rows made up of and photoelectric conversion elements 51G, are alternately arranged in the horizontal direction.

W光電変換素子群の各光電変換素子の配列は、垂直方向Yに並ぶ光電変換素子51Wからなる列であるW光電変換素子列を水平方向に複数配列したものと言うことができる。   The arrangement of the photoelectric conversion elements of the W photoelectric conversion element group can be said to be a plurality of W photoelectric conversion element arrays that are arrays of photoelectric conversion elements 51W arranged in the vertical direction Y in the horizontal direction.

各光電変換素子列の右側部には、各光電変換素子列に対応させて、各光電変換素子列を構成する光電変換素子に蓄積された電荷を垂直方向に転送するための垂直電荷転送部(不図示)が形成されている。垂直電荷転送部の終端には、ここを転送されてきた電荷を水平方向に転送するための水平電荷転送部(不図示)が接続され、この水平電荷転送部の終端には、該水平電荷転送部を転送されてきた電荷に応じた撮像信号を出力する出力アンプ(不図示)が設けられている。尚、各光電変換素子に蓄積された電荷に応じた信号を、MOSトランジスタからなるMOS回路によって外部に出力させる構成であっても良い。   On the right side of each photoelectric conversion element array, a vertical charge transfer unit (for transferring charges accumulated in the photoelectric conversion elements constituting each photoelectric conversion element array in the vertical direction corresponding to each photoelectric conversion element array ( (Not shown) is formed. A horizontal charge transfer unit (not shown) for transferring the charges transferred here in the horizontal direction is connected to the end of the vertical charge transfer unit, and the horizontal charge transfer is connected to the end of the horizontal charge transfer unit. An output amplifier (not shown) that outputs an imaging signal corresponding to the charge transferred through the unit is provided. In addition, the structure according to which the signal according to the electric charge accumulate | stored in each photoelectric conversion element is output outside by the MOS circuit which consists of a MOS transistor may be sufficient.

各光電変換素子は同一構造となっており、それぞれの受光面上方に形成されるフィルタによって、それぞれの検出する光の成分が異なるものとなっている。   Each photoelectric conversion element has the same structure, and the components of light to be detected differ depending on the filters formed above the respective light receiving surfaces.

光電変換素子51Rは、光のR成分を透過するRカラーフィルタが受光面上に形成されており、これにより、R光を検出する光電変換素子として機能する。光電変換素子51Gは、光のG成分を透過するGカラーフィルタが受光面上に形成されており、これにより、G光を検出する光電変換素子として機能する。光電変換素子51Bは、光のB成分を透過するBカラーフィルタが受光面上に形成されており、これにより、B光を検出する光電変換素子として機能する。光電変換素子51Wは、光の輝度成分と相関のある分光特性を持った輝度フィルタが受光面上に形成されており、これにより、光の輝度成分を検出する光電変換素子として機能する。   In the photoelectric conversion element 51R, an R color filter that transmits an R component of light is formed on the light receiving surface, thereby functioning as a photoelectric conversion element that detects R light. In the photoelectric conversion element 51G, a G color filter that transmits the G component of light is formed on the light receiving surface, thereby functioning as a photoelectric conversion element that detects G light. In the photoelectric conversion element 51B, a B color filter that transmits the B component of light is formed on the light receiving surface, and thus functions as a photoelectric conversion element that detects B light. In the photoelectric conversion element 51W, a luminance filter having a spectral characteristic correlated with the luminance component of light is formed on the light receiving surface, thereby functioning as a photoelectric conversion element that detects the luminance component of light.

輝度フィルタは、NDフィルタや、透明フィルタ、白色フィルタ、グレーのフィルタ等が該当するが、光電変換素子51Wの受光面の上方に何も設けずに光が直接受光面に入射する構成も、輝度フィルタを設けたということができる。   The luminance filter corresponds to an ND filter, a transparent filter, a white filter, a gray filter, or the like, but the configuration in which light is directly incident on the light receiving surface without providing anything above the light receiving surface of the photoelectric conversion element 51W is also possible. It can be said that a filter is provided.

以下では、光電変換素子51Rから得られるR成分の撮像信号をR信号、光電変換素子51Gから得られるG成分の撮像信号をG信号、光電変換素子51Bから得られるB成分の撮像信号をB信号、光電変換素子51Wから得られる輝度成分の撮像信号をW信号と言う。   Hereinafter, an R component imaging signal obtained from the photoelectric conversion element 51R is an R signal, a G component imaging signal obtained from the photoelectric conversion element 51G is a G signal, and a B component imaging signal obtained from the photoelectric conversion element 51B is a B signal. An imaging signal of a luminance component obtained from the photoelectric conversion element 51W is referred to as a W signal.

図2は、図1に示す固体撮像素子を搭載する撮像装置の一例であるデジタルカメラの概略構成を示す図である。
図示するデジタルカメラの撮像系は、撮影レンズ1と、図1に示す固体撮像素子5と、この両者の間に設けられた絞り2と、赤外線カットフィルタ3と、光学ローパスフィルタ4とを備える。
FIG. 2 is a diagram illustrating a schematic configuration of a digital camera that is an example of an imaging apparatus in which the solid-state imaging device illustrated in FIG. 1 is mounted.
The imaging system of the illustrated digital camera includes a photographic lens 1, a solid-state imaging device 5 shown in FIG. 1, a diaphragm 2 provided therebetween, an infrared cut filter 3, and an optical low-pass filter 4.

デジタルカメラの電気制御系全体を統括制御するシステム制御部11は、フラッシュ発光部12及び受光部13を制御し、レンズ駆動部8を制御して撮影レンズ1の位置をフォーカス位置に調整したりズーム調整を行ったりし、絞り駆動部9を介し絞り2の開口量を制御して露光量調整を行う。   A system control unit 11 that performs overall control of the electrical control system of the digital camera controls the flash light emitting unit 12 and the light receiving unit 13 and controls the lens driving unit 8 to adjust the position of the photographing lens 1 to the focus position and zoom. The exposure amount is adjusted by adjusting the aperture amount of the aperture 2 via the aperture drive unit 9.

又、システム制御部11は、撮像素子駆動部10を介して固体撮像素子5を駆動し、撮影レンズ1を通して撮像した被写体画像を撮像信号として出力させる。システム制御部11には、操作部14を通してユーザからの指示信号が入力される。   Further, the system control unit 11 drives the solid-state imaging device 5 via the imaging device driving unit 10 and outputs a subject image captured through the photographing lens 1 as an imaging signal. An instruction signal from the user is input to the system control unit 11 through the operation unit 14.

デジタルカメラの電気制御系は、更に、固体撮像素子5の出力に接続された相関二重サンプリング処理や信号増幅処理等のアナログ信号処理を行うアナログ信号処理部6と、このアナログ信号処理部6から出力された撮像信号をデジタル信号に変換するA/D変換回路7とを備え、これらはシステム制御部11によって制御される。A/D変換回路7から出力された撮像信号は、メインメモリ16に一時的に記憶される。メインメモリ16には、固体撮像素子5の各光電変換素子に対応する画素位置に、その光電変換素子から得られた撮像信号が配置された状態で撮像信号が記憶される。   The electric control system of the digital camera further includes an analog signal processing unit 6 that performs analog signal processing such as correlated double sampling processing and signal amplification processing connected to the output of the solid-state imaging device 5, and the analog signal processing unit 6. An A / D conversion circuit 7 that converts the output imaging signal into a digital signal is provided, and these are controlled by the system control unit 11. The imaging signal output from the A / D conversion circuit 7 is temporarily stored in the main memory 16. The main memory 16 stores the image pickup signal in a state where the image pickup signal obtained from the photoelectric conversion element is arranged at the pixel position corresponding to each photoelectric conversion element of the solid-state image pickup element 5.

デジタルカメラの電気制御系は、メインメモリ16と、メインメモリ16に接続されたメモリ制御部15と、A/D変換回路7からのデジタルの撮像信号に補間演算やガンマ補正演算,RGB/YC変換処理等を行ってカラー画像データを生成するデジタル信号処理部(DSP)17と、デジタル信号処理部17で生成されたカラー画像データをJPEG形式に圧縮したり圧縮画像データを伸張したりする圧縮伸張処理部18と、着脱自在の記録媒体21が接続される外部メモリ制御部20と、カメラ背面等に搭載された液晶表示部23が接続される表示制御部22とを備え、これらは、制御バス24及びデータバス25によって相互に接続され、システム制御部11からの指令によって制御される。   The electric control system of the digital camera includes a main memory 16, a memory control unit 15 connected to the main memory 16, and digital imaging signals from the A / D conversion circuit 7 for interpolation calculation, gamma correction calculation, and RGB / YC conversion. A digital signal processing unit (DSP) 17 that generates color image data by performing processing and the like, and compression / decompression that compresses the color image data generated by the digital signal processing unit 17 into a JPEG format or decompresses the compressed image data A processing unit 18, an external memory control unit 20 to which a detachable recording medium 21 is connected, and a display control unit 22 to which a liquid crystal display unit 23 mounted on the back of the camera or the like is connected include a control bus 24 and the data bus 25 are connected to each other and controlled by a command from the system control unit 11.

図3は、図2に示すデジタル信号処理部の内部構成を示す図である。
図3に示すように、デジタル信号処理部17には、メインメモリ16上の各画素位置にR信号,G信号,B信号,W信号を補間して生成するRGBW補間部17aと、各画素位置にある信号から各画素位置に輝度信号と色差信号を生成する輝度・色差信号生成部17bと、色差信号を補正する色差信号補正手段を構成する補正係数演算部17c及び乗算器17dとが含まれている。
FIG. 3 is a diagram showing an internal configuration of the digital signal processing unit shown in FIG.
As shown in FIG. 3, the digital signal processing unit 17 includes an RGBW interpolation unit 17a that interpolates and generates R, G, B, and W signals at each pixel position on the main memory 16, and each pixel position. A luminance / color difference signal generation unit 17b that generates a luminance signal and a color difference signal at each pixel position from the signals in the signal, a correction coefficient calculation unit 17c that constitutes a color difference signal correction unit that corrects the color difference signal, and a multiplier 17d. ing.

RGBW補間部17aは、A/D変換回路7から出力されたR信号、G信号、B信号、及びW信号から、メモリ上の光電変換素子51R,51G,51B,51Wの各々に対応する画素位置に、該画素位置に対応する光電変換素子から得られる信号と、該光電変換素子の周囲の該光電変換素子とは異なる種類の光電変換素子から得られる信号とを用いて、R信号と、G信号と、B信号と、W信号とを生成する処理を行う。   The RGBW interpolation unit 17a uses the R signal, the G signal, the B signal, and the W signal output from the A / D conversion circuit 7, and the pixel position corresponding to each of the photoelectric conversion elements 51R, 51G, 51B, and 51W on the memory. In addition, using a signal obtained from a photoelectric conversion element corresponding to the pixel position and a signal obtained from a photoelectric conversion element of a different type from the photoelectric conversion element around the photoelectric conversion element, an R signal, and G A process of generating a signal, a B signal, and a W signal is performed.

例えば、光電変換素子51Rに対応する画素位置には、該光電変換素子51Rから得られたR信号と、該光電変換素子51Rの周囲にある光電変換素子51G,51Bから得られたG信号とB信号を用いてR信号、G信号、及びB信号を生成し、該光電変換素子51Rの周囲にある光電変換素子51Wから得られたW信号を用いてW信号を生成する。同光電変換素子51Gに対応する画素位置には、該光電変換素子51Gから得られたG信号と、該光電変換素子51Gの周囲にある光電変換素子51R,51Bから得られたR信号とB信号を用いてR信号、G信号、及びB信号を生成し、該光電変換素子51Gの周囲にある光電変換素子51Wから得られたW信号を用いてW信号を生成する。光電変換素子51Bに対応する画素位置には、該光電変換素子51Bから得られたB信号と、該光電変換素子51Bの周囲にある光電変換素子51R,51Gから得られたR信号とG信号を用いてR信号、G信号、及びB信号を生成し、該光電変換素子51Bの周囲にある光電変換素子51Wから得られたW信号を用いてW信号を生成する。光電変換素子51Wに対応する画素位置には、該光電変換素子51Wから得られたW信号を用いてW信号を生成し、該光電変換素子51Wの周囲にある光電変換素子51R,51G,51Bから得られたR信号とG信号とB信号を用いてR信号、G信号、及びB信号を生成する。   For example, at the pixel position corresponding to the photoelectric conversion element 51R, the R signal obtained from the photoelectric conversion element 51R, the G signal obtained from the photoelectric conversion elements 51G and 51B around the photoelectric conversion element 51R, and B An R signal, a G signal, and a B signal are generated using the signal, and a W signal is generated using the W signal obtained from the photoelectric conversion element 51W around the photoelectric conversion element 51R. At the pixel position corresponding to the photoelectric conversion element 51G, a G signal obtained from the photoelectric conversion element 51G, and an R signal and a B signal obtained from the photoelectric conversion elements 51R and 51B around the photoelectric conversion element 51G. Are used to generate an R signal, a G signal, and a B signal, and a W signal is generated using a W signal obtained from the photoelectric conversion element 51W around the photoelectric conversion element 51G. At the pixel position corresponding to the photoelectric conversion element 51B, the B signal obtained from the photoelectric conversion element 51B and the R signal and G signal obtained from the photoelectric conversion elements 51R and 51G around the photoelectric conversion element 51B are provided. The R signal, the G signal, and the B signal are generated using the W signal, and the W signal is generated using the W signal obtained from the photoelectric conversion element 51W around the photoelectric conversion element 51B. At the pixel position corresponding to the photoelectric conversion element 51W, a W signal is generated using the W signal obtained from the photoelectric conversion element 51W, and the photoelectric conversion elements 51R, 51G, and 51B around the photoelectric conversion element 51W are generated. The R signal, G signal, and B signal are generated using the obtained R signal, G signal, and B signal.

この処理により、メモリ上の光電変換素子51R,51G,51B,51Wの各々に対応する画素位置には、R信号、G信号、B信号、及びW信号の4つの信号が生成されることになる。   By this process, four signals of R signal, G signal, B signal, and W signal are generated at pixel positions corresponding to the photoelectric conversion elements 51R, 51G, 51B, 51W on the memory. .

尚、以上の説明では、各画素位置に生成すべきR,G,B信号を、光電変換素子51R、光電変換素子51G、及び光電変換素子51Bから得られた色信号を用いて生成するものとしたが、色信号と共に、光電変換素子51Wから得られるW信号も用いて、これを生成しても良い。以下、この場合のR,G,B信号の生成方法について説明する。   In the above description, the R, G, and B signals to be generated at each pixel position are generated using the color signals obtained from the photoelectric conversion element 51R, the photoelectric conversion element 51G, and the photoelectric conversion element 51B. However, this may be generated using the W signal obtained from the photoelectric conversion element 51W together with the color signal. Hereinafter, a method for generating the R, G, and B signals in this case will be described.

図4は、A/D変換回路7から出力された撮像信号の配列を示した図である。図4において、内部に“R”を記した円は光電変換素子51Rから得られたR信号を示し、内部に“G”を記した円は光電変換素子51Gから得られたG信号を示し、内部に“B”を記した円は光電変換素子51Bから得られたB信号を示し、内部に“W”を記した円は光電変換素子51Wから得られたW信号を示している。各信号の存在する位置は、その信号の出力元の光電変換素子の位置に対応しており、この位置を画素位置とする。各画素位置の上にはその位置を示す番号(01〜32)を付してある。   FIG. 4 is a diagram showing the arrangement of the imaging signals output from the A / D conversion circuit 7. In FIG. 4, a circle with “R” inside indicates an R signal obtained from the photoelectric conversion element 51 </ b> R, a circle with “G” inside indicates a G signal obtained from the photoelectric conversion element 51 </ b> G, A circle with “B” inside shows the B signal obtained from the photoelectric conversion element 51B, and a circle with “W” inside shows the W signal obtained from the photoelectric conversion element 51W. The position where each signal exists corresponds to the position of the photoelectric conversion element from which the signal is output, and this position is defined as a pixel position. A number (01 to 32) indicating the position is given above each pixel position.

図4に示すように、A/D変換回路7から出力された撮像信号は、水平方向(X方向)と垂直方向(Y方向)に45度の角度で交差するZ方向に見たときに、W信号とG信号とをZ方向に交互に配列した第1の列と、R信号とB信号とをW信号を挟んでZ方向に交互に配列した第2の列とを、Z方向に直交する直交方向に交互に配列した構成となっている。又、Y方向に見たときに、G信号とR信号をY方向に交互に配列した第3の列と、G信号とB信号をY方向に交互に配列した第4の列とを、W信号をY方向に配列した第5の列を挟んでX方向に交互に配列した構成となっている。   As shown in FIG. 4, when the imaging signal output from the A / D conversion circuit 7 is viewed in the Z direction that intersects the horizontal direction (X direction) and the vertical direction (Y direction) at an angle of 45 degrees, A first column in which W signals and G signals are alternately arranged in the Z direction, and a second column in which R signals and B signals are alternately arranged in the Z direction across the W signal are orthogonal to the Z direction. The configuration is arranged alternately in the orthogonal direction. Further, when viewed in the Y direction, a third column in which G signals and R signals are alternately arranged in the Y direction and a fourth column in which G signals and B signals are alternately arranged in the Y direction are represented by W In this configuration, the signals are alternately arranged in the X direction across the fifth column in which the signals are arranged in the Y direction.

RGBW補間部17aは、A/D変換回路7から出力された多数の撮像信号の各々がどの方向に相関性を持っているかを、例えばその撮像信号に隣接するW信号を利用することで判断し、Z方向又はこれに直交する方向に相関性のある撮像信号については、その撮像信号が存在する画素位置に補間すべき色信号を、その画素位置の周囲にある該補間すべき色信号と同種類の色信号及びW信号を用いて推定する輝度利用推定処理と、その画素位置に補間すべき色信号を、その画素位置の周囲にある該補間すべき色信号と同種類の色信号を用いて推定する輝度非利用推定処理とにより、R信号、G信号、及びB信号の各々が存在する画素位置に、そこに存在していない色信号を補間する。   The RGBW interpolation unit 17a determines in which direction each of a large number of imaging signals output from the A / D conversion circuit 7 is correlated by using, for example, a W signal adjacent to the imaging signal. For an imaging signal correlated in the Z direction or a direction orthogonal thereto, the color signal to be interpolated at the pixel position where the imaging signal exists is the same as the color signal to be interpolated around the pixel position. Luminance utilization estimation processing that estimates using a type of color signal and W signal, and a color signal to be interpolated at the pixel position uses the same type of color signal as the color signal to be interpolated around the pixel position The luminance signal non-use estimation process estimated in this way interpolates the color signal that does not exist at the pixel position where each of the R signal, G signal, and B signal exists.

又、RGBW補間部17aは、X方向又はY方向に相関性のある撮像信号については、その撮像信号が存在する画素位置に補間すべき色信号を、その画素位置の周囲にある該補間すべき色信号と同種類の色信号及びG信号を用いて推定するG利用推定処理と、その画素位置に補間すべき色信号を、その画素位置の周囲にある該補間すべき色信号と同種類の色信号を用いて推定するG非利用推定処理とにより、R信号、G信号、及びB信号の各々が存在する画素位置に、そこに存在していない色信号を補間する。   Further, the RGBW interpolation unit 17a should interpolate the color signal to be interpolated at the pixel position where the image signal exists for the image signal correlated in the X direction or the Y direction. G use estimation processing that estimates using the same kind of color signal and G signal as the color signal, and the same color signal as the color signal to be interpolated around the pixel position. The color signal that does not exist is interpolated at the pixel position where each of the R signal, the G signal, and the B signal exists by the G non-use estimation process that estimates using the color signal.

例えば、図4に示すG10がどの方向に相関性を持っているかを判断する場合、RGBW補間部17aは、Y方向:(W05+W13)/2−(W06+W14)/2、X方向:(W05+W06)/2−(W13+W14)/2、Z方向:(W06−W13)、Z方向に直交する方向:(W05−W14)の4つの演算を行い、演算結果のうち絶対値が最小となった方向を、G10に相関性のある方向と判断する。   For example, when determining in which direction G10 shown in FIG. 4 is correlated, the RGBW interpolation unit 17a performs the Y direction: (W05 + W13) / 2− (W06 + W14) / 2, the X direction: (W05 + W06) / 2- (W13 + W14) / 2, Z direction: (W06-W13), direction orthogonal to the Z direction: (W05-W14), and the direction in which the absolute value is the smallest among the calculation results, It is determined that the direction has a correlation with G10.

以下の説明では、便宜上、画素位置**に存在するR信号のことを「R**」と記載し、画素位置**に存在するG信号のことを「G**」と記載し、画素位置**に存在するB信号のことを「B**」と記載し、画素位置**に存在するW信号のことを「W**」と記載する。   In the following description, for convenience, the R signal existing at the pixel position ** is described as “R **”, the G signal existing at the pixel position ** is described as “G **”, and the pixel The B signal existing at the position ** is described as “B **”, and the W signal existing at the pixel position ** is described as “W **”.

以下、例えばZ方向に相関性のある撮像信号の存在する画素位置に色信号を補間する方法を具体的に説明する。   Hereinafter, a method for interpolating a color signal at a pixel position where an imaging signal having a correlation in the Z direction, for example, will be described in detail.

(a)B信号の存在する画素位置にR信号を補間する場合
この場合、RGBW補間部17aは、輝度非利用推定処理によって補間すべきR信号を推定する。即ち、RGBW補間部17aは、B信号の存在する画素位置に補間すべきR信号を、当該画素位置に存在するB信号を含む第2の列に含まれる当該画素位置近傍のR信号を用いて推定する。
例えば、画素位置11にR信号(R11)を補間する場合、同時化処理部12は、B11を含む第2の列に含まれるB11近傍のR信号(R04、R18)の平均を求める以下の式(1)により、R11を推定する。R11=(R04+R18)/2・・・(1)
(A) When R signal is interpolated at a pixel position where B signal exists In this case, the RGBW interpolation unit 17a estimates an R signal to be interpolated by luminance non-use estimation processing. That is, the RGBW interpolation unit 17a uses the R signal to be interpolated at the pixel position where the B signal exists, and the R signal near the pixel position included in the second column including the B signal present at the pixel position. presume.
For example, when the R signal (R11) is interpolated at the pixel position 11, the synchronization processing unit 12 calculates the average of the R signals (R04, R18) in the vicinity of B11 included in the second column including B11. R11 is estimated from (1). R11 = (R04 + R18) / 2 (1)

(b)R信号の存在する画素位置にB信号を補間する場合
この場合、RGBW補間部17aは、輝度非利用推定処理によって補間すべきB信号を推定する。即ち、RGBW補間部17aは、R信号の存在する画素位置に補間すべきB信号を、当該画素位置に存在するR信号を含む第2の列に含まれる当該画素位置近傍のB信号を用いて推定する。
例えば、画素位置18にB信号(B18)を補間する場合、同時化処理部12は、R18を含む第2の列に含まれるR18近傍のB信号(B11、B25)の平均を求める以下の式(2)により、B18を推定する。B18=(B11+B25)/2・・・(2)
(B) When interpolating the B signal at the pixel position where the R signal exists In this case, the RGBW interpolation unit 17a estimates the B signal to be interpolated by the luminance non-use estimation process. That is, the RGBW interpolation unit 17a uses the B signal to be interpolated at the pixel position where the R signal exists, and the B signal near the pixel position included in the second column including the R signal existing at the pixel position. presume.
For example, when the B signal (B18) is interpolated at the pixel position 18, the synchronization processing unit 12 calculates the average of the B signals (B11, B25) in the vicinity of R18 included in the second column including R18. B18 is estimated from (2). B18 = (B11 + B25) / 2 (2)

(c)B信号の存在する画素位置とR信号の存在する画素位置とにそれぞれG信号を補間する場合
この場合、RGBW補間部17aは、輝度利用推定処理によって補間すべきG信号を推定する。即ち、B信号の存在する画素位置とR信号の存在する画素位置とにそれぞれ補間すべきG信号を、その画素位置に存在する色信号を含む第2の列に含まれる当該画素位置近傍のW信号と、当該第2の列の隣の第1の列にある当該画素位置近傍のW信号と、当該第1の列の当該W信号の近傍にあるG信号とを用いて推定する。
(C) When G Signal is Interpolated to the Pixel Position where the B Signal is Present and the Pixel Position where the R Signal is Present In this case, the RGBW interpolation unit 17a estimates the G signal to be interpolated by the luminance use estimation process. That is, the G signal to be interpolated to the pixel position where the B signal exists and the pixel position where the R signal exists respectively, and the W near the pixel position included in the second column including the color signal existing at the pixel position. The estimation is performed using the signal, the W signal in the vicinity of the pixel position in the first column adjacent to the second column, and the G signal in the vicinity of the W signal in the first column.

例えば、画素位置11にG信号(G11)を補間する場合、Z方向に相関性のある被写体において、画像の局所的な領域におけるW信号とG信号との差は等しいという前提、すなわち、G11−W11={(G06−W06)+(G15−W15)}/2の関係が成立するという前提に基づいて、G11=W11+{(G06−W06)+(G15−W15)}/2・・・(3)の演算により、G11を推定する。   For example, when the G signal (G11) is interpolated at the pixel position 11, it is assumed that the difference between the W signal and the G signal in the local region of the image is equal in an object correlated in the Z direction, that is, G11−. Based on the premise that the relationship of W11 = {(G06-W06) + (G15-W15)} / 2 is established, G11 = W11 + {(G06-W06) + (G15-W15)} / 2. G11 is estimated by the calculation of 3).

具体的に、RGBW補間部17aは、まず、画素位置11に、画素位置11に存在するB11を含む第2の列に含まれる画素位置11近傍の輝度信号(W07、W14)を用いて輝度信号(W11)を補間し、画素位置11のZ方向に直交する方向の隣の画素位置である画素位置06に、画素位置06に存在するW06を含む第1の列に含まれる画素位置06近傍のG信号(G03,G10)を用いてG06を補間し、画素位置11のZ方向に直交する方向の隣の画素位置である画素位置15に、画素位置15に存在するW15を含む第1の列に含まれる画素位置15近傍のG信号(G12,G19)を用いてG15を補間する。   Specifically, the RGBW interpolation unit 17a first uses the luminance signal (W07, W14) in the vicinity of the pixel position 11 included in the second column including B11 existing at the pixel position 11 at the pixel position 11. (W11) is interpolated, and the pixel position 06 which is the pixel position adjacent to the pixel position 11 in the direction orthogonal to the Z direction is near the pixel position 06 included in the first column including W06 existing at the pixel position 06. G06 is interpolated using the G signal (G03, G10), and the first column including W15 present at the pixel position 15 at the pixel position 15 that is the pixel position adjacent to the pixel position 11 in the direction orthogonal to the Z direction. G15 is interpolated using G signals (G12, G19) in the vicinity of the pixel position 15 included in.

W11については、例えばW07とW14の平均を演算することで求め、G06については、例えばG03とG10の平均を演算することで求め、G15については、例えばG12とG19の平均を演算することで求める。W11、G06、G15を求めた後は、式(3)に従い、W11+{(G06−W06)+(G15−W15)}/2の演算を行うことで、G11を推定する。同様の方法で、R信号の存在する画素位置にもG信号を補間する。   W11 is obtained, for example, by calculating the average of W07 and W14, G06 is obtained, for example, by calculating the average of G03 and G10, and G15 is obtained, for example, by calculating the average of G12 and G19. . After obtaining W11, G06, and G15, G11 is estimated by performing an operation of W11 + {(G06-W06) + (G15−W15)} / 2 according to the equation (3). In the same way, the G signal is also interpolated at the pixel position where the R signal exists.

尚、式(3)は、画素位置11と画素位置06及び15との相関に基づいてG11を推定するものとなっているが、画素位置11と画素位置06との相関又は画素位置11と画素位置15との相関に基づいてG11を推定することも可能である。この場合、上記式(3)を次の式(4)又は(5)のように変形して用いれば良い。G11=W11+(G06−W06)・・・(4)、G11=W11+(G15−W15)・・・(5)。   Note that Equation (3) estimates G11 based on the correlation between the pixel position 11 and the pixel positions 06 and 15, but the correlation between the pixel position 11 and the pixel position 06 or the pixel position 11 and the pixel. It is also possible to estimate G11 based on the correlation with the position 15. In this case, the above equation (3) may be used as modified as the following equation (4) or (5). G11 = W11 + (G06-W06) (4), G11 = W11 + (G15-W15) (5).

(d)G信号の存在する画素位置にR信号とB信号を補間する場合
この場合、RGBW補間部17aは、輝度利用推定処理によって補間すべきR信号及びB信号を推定する。即ち、G信号の存在する画素位置に補間すべきR信号を、その画素位置に存在するG信号を含む第1の列に含まれる当該画素位置近傍のW信号と、当該第1の列の隣の第2の列にある当該画素位置近傍のR信号と、当該第2の列の当該R信号の近傍にあるW信号とを用いて推定する。又、G信号の存在する画素位置に補間すべきB信号を、その画素位置に存在するG信号を含む第1の列に含まれる当該画素位置近傍のW信号と、当該第1の列の隣の第2の列にある当該画素位置近傍のB信号と、当該第2の列の当該B信号の近傍にあるW信号とを用いて推定する。
(D) When R signal and B signal are interpolated at the pixel position where the G signal exists In this case, the RGBW interpolation unit 17a estimates the R signal and the B signal to be interpolated by the luminance use estimation process. That is, the R signal to be interpolated at the pixel position where the G signal exists, the W signal in the vicinity of the pixel position included in the first column including the G signal existing at the pixel position, and the first column. The estimation is performed using the R signal in the vicinity of the pixel position in the second column and the W signal in the vicinity of the R signal in the second column. Also, the B signal to be interpolated at the pixel position where the G signal exists, the W signal near the pixel position included in the first column including the G signal present at the pixel position, and the first column adjacent to the W signal. Is estimated using the B signal in the vicinity of the pixel position in the second column and the W signal in the vicinity of the B signal in the second column.

例えば、画素位置19にR信号(R19)を補間する場合、Z方向に相関性のある被写体において、画像の局所的な領域におけるW信号とR信号との差は等しいという前提、すなわち、R19−W19={(R18−W18)+(R20−W20)}/2の関係が成立するという前提に基づいて、R19=W19+{(R18−W18)+(R20−W20)}/2・・・(6)の演算により、R19を推定する。
同様に、B19−W19={(B11−W11)+(B27−W27)}/2の関係が成立するという前提に基づいて、B19=W19+{(B11−W11)+(B27−W27)}/2・・・(7)の演算により、B19を推定する。
For example, when the R signal (R19) is interpolated at the pixel position 19, in the subject having a correlation in the Z direction, the difference between the W signal and the R signal in the local region of the image is equal, that is, R19−. Based on the premise that the relationship of W19 = {(R18−W18) + (R20−W20)} / 2 is established, R19 = W19 + {(R18−W18) + (R20−W20)} / 2. R19 is estimated by the calculation of 6).
Similarly, based on the premise that the relationship B19−W19 = {(B11−W11) + (B27−W27)} / 2 is established, B19 = W19 + {(B11−W11) + (B27−W27)} / 2... B19 is estimated by the calculation of (7).

具体的に、RGBW補間部17aは、画素位置19に、画素位置19に存在するG19を含む第1の列に含まれる画素位置19近傍の輝度信号(W15、W22)を用いて輝度信号(W19)を補間し、画素位置19のX方向の隣の画素位置である画素位置18に、画素位置18に存在するR18を含む第2の列に含まれる画素位置18近傍のW信号(W14,W21)を用いてW18を補間し、画素位置19のX方向の隣の画素位置である画素位置20に、画素位置20に存在するR20を含む第2の列に含まれる画素位置20近傍のW信号(W16,W23)を用いてW20を補間する。
又、RGBW補間部17aは、画素位置19のY方向の隣の画素位置である画素位置11に、画素位置11に存在するB11を含む第2の列に含まれる画素位置11近傍のW信号(W07,W14)を用いてW11を補間し、画素位置19のY方向の隣の画素位置である画素位置27に、画素位置27に存在するB27を含む第2の列に含まれる画素位置27近傍のW信号(W23,W30)を用いてW27を補間する。
Specifically, the RGBW interpolation unit 17a uses the luminance signal (W19, W22) near the pixel position 19 included in the first column including G19 existing at the pixel position 19 as the luminance signal (W19). ) To the pixel position 18 that is the pixel position adjacent to the pixel position 19 in the X direction, and the W signal (W14, W21) in the vicinity of the pixel position 18 included in the second column including R18 existing at the pixel position 18. ) Is used to interpolate W18, and the W signal in the vicinity of the pixel position 20 included in the second column including R20 existing at the pixel position 20 is added to the pixel position 20 that is the pixel position adjacent to the pixel position 19 in the X direction W20 is interpolated using (W16, W23).
In addition, the RGBW interpolation unit 17a sets the W signal (in the vicinity of the pixel position 11 included in the second column including B11 existing at the pixel position 11) to the pixel position 11 which is the pixel position adjacent to the pixel position 19 in the Y direction. W11 is interpolated using W07, W14), and the pixel position 27 that is adjacent to the pixel position 19 in the Y direction is adjacent to the pixel position 27 included in the second column including B27 that exists at the pixel position 27. W27 is interpolated using the W signal (W23, W30).

W19については、例えばW15とW22の平均を演算することで求め、W18については、例えばW14とW21の平均を演算することで求め、W20については、例えばW16とW23の平均を演算することで求め、W11については、例えばW07とW14の平均を演算することで求め、W27については、例えばW23とW30の平均を演算することで求める。W18,W19,W20,W11,W27を求めた後は、式(6)にしたがってW19+{(R18−W18)+(R20−W20)}/2の演算を行うことで、R19を推定し、式(7)にしたがってW19+{(B11−W11)+(B27−W27)}/2の演算を行うことで、B19を推定する。   For example, W19 is obtained by calculating the average of W15 and W22, W18 is obtained by calculating the average of W14 and W21, and W20 is obtained by calculating the average of W16 and W23, for example. , W11 is obtained, for example, by calculating the average of W07 and W14, and W27 is obtained, for example, by calculating the average of W23 and W30. After obtaining W18, W19, W20, W11, and W27, R19 is estimated by calculating W19 + {(R18−W18) + (R20−W20)} / 2 according to Equation (6), B19 is estimated by calculating W19 + {(B11-W11) + (B27-W27)} / 2 according to (7).

尚、式(6)は、画素位置19と画素位置18及び20との相関に基づいてR19を推定するものとなっているが、画素位置19と画素位置18との相関又は画素位置19と画素位置20との相関に基づいてR19を推定することも可能である。この場合、上記式(6)を次の式(8)又は(9)のように変形して用いれば良い。R19=W19+(R18−W18)・・・(8)、R19=W19+(R20−W20)・・・(9)。   Note that Equation (6) estimates R19 based on the correlation between the pixel position 19 and the pixel positions 18 and 20, but the correlation between the pixel position 19 and the pixel position 18 or the pixel position 19 and the pixel. It is also possible to estimate R19 based on the correlation with the position 20. In this case, the above formula (6) may be used as modified as the following formula (8) or (9). R19 = W19 + (R18−W18) (8), R19 = W19 + (R20−W20) (9).

又、式(7)は、画素位置19と画素位置11及び27との相関に基づいてB19を推定するものとなっているが、画素位置19と画素位置11との相関又は画素位置19と画素位置27との相関に基づいてB19を推定することも可能である。この場合、上記式(7)を次の式(10)又は(11)のように変形して用いれば良い。B19=W19+(B11−W11)・・・(10)、B19=W19+(B27−W27)・・・(11)。   Expression (7) estimates B19 based on the correlation between the pixel position 19 and the pixel positions 11 and 27, but the correlation between the pixel position 19 and the pixel position 11 or the pixel position 19 and the pixel. It is also possible to estimate B19 based on the correlation with the position 27. In this case, the above equation (7) may be used as modified as the following equation (10) or (11). B19 = W19 + (B11−W11) (10), B19 = W19 + (B27−W27) (11).

上記(a)〜(d)の処理を行うことで、光電変換素子51R,51G,51Bに対応する各画素位置にR信号、G信号、B信号を生成することができる。光電変換素子51Wに対応する画素位置には、それ以外の画素位置に生成したR信号、G信号、B信号を用いて、R信号、G信号、B信号を補間生成すれば良い。   By performing the processes (a) to (d), it is possible to generate an R signal, a G signal, and a B signal at each pixel position corresponding to the photoelectric conversion elements 51R, 51G, and 51B. The R signal, the G signal, and the B signal may be generated by interpolation using the R signal, the G signal, and the B signal generated at other pixel positions at the pixel position corresponding to the photoelectric conversion element 51W.

輝度・色差信号生成部17bは、RGBW補間部17aで生成された各画素位置にあるR信号、G信号、及びB信号から、その画素位置に対応する輝度信号Yと色差信号Cを生成する。輝度信号Yと色差信号Cの生成方法は公知の方法を採用することができる。   The luminance / color difference signal generation unit 17b generates a luminance signal Y and a color difference signal C corresponding to the pixel position from the R signal, G signal, and B signal at each pixel position generated by the RGBW interpolation unit 17a. As a method for generating the luminance signal Y and the color difference signal C, a known method can be adopted.

乗算器17dは、輝度・色差信号生成部17bで生成された任意の画素位置にある色差信号に、該任意の画素位置にある4つの撮像信号に基づいて算出した補正係数Kを乗じて、色差信号の補正を行う。   The multiplier 17d multiplies the color difference signal at the arbitrary pixel position generated by the luminance / color difference signal generation unit 17b by the correction coefficient K calculated based on the four imaging signals at the arbitrary pixel position, thereby obtaining the color difference. Correct the signal.

補正係数演算部17cは、上記補正係数Kを下記式(a)によって求める。   The correction coefficient calculator 17c calculates the correction coefficient K by the following equation (a).

K=W/(α×R+β×G+γ×B+δ)・・・(a)   K = W / (α × R + β × G + γ × B + δ) (a)

式(a)中のWは、上記任意の画素位置に生成されたW信号である。式(a)中のRは、該任意の画素位置に生成されたR信号である。式(a)中のGは、該任意の画素位置に生成されたG信号である。式(a)中のBは、該任意の画素位置に生成されたB信号である。式(a)中のα,β,γ,δは、光電変換素子51Wの分光特性を、光電変換素子51R,51G,51Bの各々の分光特性の積和で近似したときに、その近似した分光特性と光電変換素子51Wの分光特性との誤差が最小となるように予め決めておく係数である。   W in the equation (a) is a W signal generated at the arbitrary pixel position. R in the formula (a) is an R signal generated at the arbitrary pixel position. G in the formula (a) is a G signal generated at the arbitrary pixel position. B in the equation (a) is a B signal generated at the arbitrary pixel position. Α, β, γ, and δ in the formula (a) are approximated spectral spectra when the spectral characteristics of the photoelectric conversion element 51W are approximated by the product sum of the spectral characteristics of the photoelectric conversion elements 51R, 51G, and 51B. This coefficient is determined in advance so that an error between the characteristic and the spectral characteristic of the photoelectric conversion element 51W is minimized.

例えば、光電変換素子51R,51G,51B,51Wの各々に同一光量の光を当て、この光によって光電変換素子51R,51G,51Bの各々から得られたR信号,G信号,B信号を式(a)の右辺の分母に代入して得られる信号が、この光によって光電変換素子51Wから得られたW信号に最も近づくように、α,β,γ,δの値を決めておけば良い。   For example, the same amount of light is applied to each of the photoelectric conversion elements 51R, 51G, 51B, and 51W, and the R signal, G signal, and B signal obtained from each of the photoelectric conversion elements 51R, 51G, and 51B by this light are expressed by the formula ( The values of α, β, γ, and δ may be determined so that the signal obtained by substituting into the denominator on the right side of a) is closest to the W signal obtained from the photoelectric conversion element 51W by this light.

このように、上記α,β,γ,δの値が最適化されていれば、補正係数Kの値はほぼ“1”となる。ここで、補正係数Kを信号成分とノイズ成分に分離して表現すると下記式(b)のようになる。   Thus, if the values of α, β, γ, and δ are optimized, the value of the correction coefficient K is almost “1”. Here, when the correction coefficient K is expressed separately as a signal component and a noise component, the following equation (b) is obtained.

K=(Ws+Wn)/(α×Rs+β×Gs+γ×Bs+α×Rn+β×Gn+γ×Bn+δ)・・・(b)   K = (Ws + Wn) / (α × Rs + β × Gs + γ × Bs + α × Rn + β × Gn + γ × Bn + δ) (b)

式(b)中のXs(X=W,R,G,B)は、X信号に含まれる信号成分である。式(b)中のXn(X=W,R,G,B)は、X信号に含まれるノイズ成分である。   Xs (X = W, R, G, B) in the formula (b) is a signal component included in the X signal. Xn (X = W, R, G, B) in the formula (b) is a noise component included in the X signal.

W信号は、R信号やG信号やB信号と比べて感度が高いため、各信号の信号レベルを同じにしたときに、R信号,G信号,B信号に含まれるノイズ成分が、W信号に含まれるノイズ成分よりも大きくなる。R信号,G信号,B信号に含まれるノイズ成分が大きくなればなるほど、式(b)から分かるように、補正係数Kの値はより小さくなる。又、R信号,G信号,B信号に含まれる信号成分が小さくなると、R信号,G信号,B信号に含まれるノイズ成分が相対的に大きくなるので、この場合も補正係数Kの値は“1”よりも小さくなる。   Since the W signal has higher sensitivity than the R signal, the G signal, and the B signal, when the signal level of each signal is the same, noise components included in the R signal, the G signal, and the B signal are converted into the W signal. It becomes larger than the noise component contained. As the noise component contained in the R signal, G signal, and B signal increases, the value of the correction coefficient K decreases as can be seen from the equation (b). In addition, when the signal component included in the R signal, G signal, and B signal is reduced, the noise component included in the R signal, G signal, and B signal is relatively increased. In this case, the value of the correction coefficient K is “ Less than 1 ″.

一般に、色差信号は、その生成元であるR信号,G信号,B信号に含まれるノイズ成分が大きくなると、それに比例して色ノイズが大きくなる傾向にある。つまり、補正係数Kは、色差信号に含まれる色ノイズの大きさの度合いを表す指標と捉えることもできる。   In general, when a noise component included in an R signal, a G signal, and a B signal as a generation source of a color difference signal increases, color noise tends to increase in proportion thereto. That is, the correction coefficient K can also be regarded as an index representing the degree of the magnitude of color noise included in the color difference signal.

例えば、K=1/2であれば、W信号のノイズをゼロと考えても、R信号,G信号,B信号のノイズ成分は、R信号,G信号,B信号の信号成分とほぼ同じレベルと考えられ、色ノイズが大きいと言うことができる。このため、乗算器17dにおいて、輝度・色差信号生成部17bによって求められた色差信号Cに補正係数Kを乗じて、色差信号C’を生成することで、色ノイズを抑制することが可能となる。   For example, if K = 1/2, even if the noise of the W signal is considered to be zero, the noise components of the R signal, the G signal, and the B signal are almost the same level as the signal components of the R signal, the G signal, and the B signal. It can be said that the color noise is large. Therefore, the color noise can be suppressed by generating the color difference signal C ′ by multiplying the color difference signal C obtained by the luminance / color difference signal generation unit 17b by the correction coefficient K in the multiplier 17d. .

以上のように構成されたデジタルカメラの動作を説明する。
撮影が行われると、固体撮像素子5の各光電変換素子から撮像信号が出力され、この撮像信号がデジタル信号に変換された後、メインメモリ16に一時的に記憶される。その後、メインメモリ16上の固体撮像素子5の各光電変換素子に対応する画素位置に、その周囲の画素位置にある撮像信号を用いて撮像信号の補間が行われ、各光電変換素子に対応する画素位置に、R信号,G信号,B信号,W信号が生成される。
The operation of the digital camera configured as described above will be described.
When shooting is performed, an image pickup signal is output from each photoelectric conversion element of the solid-state image pickup element 5, and after this image pickup signal is converted into a digital signal, it is temporarily stored in the main memory 16. After that, the image signal is interpolated using the image signal at the surrounding pixel position at the pixel position corresponding to each photoelectric conversion element of the solid-state image sensor 5 on the main memory 16 to correspond to each photoelectric conversion element. An R signal, a G signal, a B signal, and a W signal are generated at the pixel position.

次に、各画素位置にあるR信号,G信号,B信号から輝度信号Yと色差信号Cが生成されると共に、該各画素位置にあるR信号,G信号,B信号,W信号から該各画素位置に対応する補正係数Kが算出される。そして、該各画素位置に生成した色差信号Cに、該各画素位置に対応する補正係数Kが乗じられて色差信号C’が生成される。   Next, a luminance signal Y and a color difference signal C are generated from the R signal, G signal, and B signal at each pixel position, and each of the R signal, G signal, B signal, and W signal at each pixel position is generated. A correction coefficient K corresponding to the pixel position is calculated. Then, the color difference signal C ′ is generated by multiplying the color difference signal C generated at each pixel position by the correction coefficient K corresponding to each pixel position.

各光電変換素子に対応する画素位置に輝度信号Yと色差信号C’が生成されてカラー画像データの生成が完了すると、該カラー画像データが圧縮されて記録媒体21に記録される。   When the luminance signal Y and the color difference signal C ′ are generated at the pixel position corresponding to each photoelectric conversion element and the generation of the color image data is completed, the color image data is compressed and recorded on the recording medium 21.

以上のように、本実施形態のデジタルカメラによれば、各画素位置に生成したR信号,G信号,B信号,W信号に応じて、その画素位置に生成する色差信号を補正しているため、被写体の場所毎に最適な補正を行うことができる。例えば、明るい部分と暗い部分とが混在する被写体であっても、明るい部分と暗い部分とで最適な色差信号の補正が自動的になされるため、複雑な処理を行うことなく、画質の劣化を防止することができる。又、このような色差信号の補正方法によれば、色相は変化しないため、この点からも画質の劣化を防止することができる。   As described above, according to the digital camera of the present embodiment, the color difference signal generated at the pixel position is corrected according to the R signal, G signal, B signal, and W signal generated at each pixel position. Optimal correction can be performed for each location of the subject. For example, even for a subject with a mixture of bright and dark areas, the optimum color difference signal correction is automatically performed between the bright and dark areas, so image quality can be reduced without complicated processing. Can be prevented. Further, according to such a color difference signal correction method, since the hue does not change, the deterioration of the image quality can be prevented from this point.

尚、以上の説明では、補正係数演算部17cが、上記式(a)に代入するR,G,Bのデータとして、RGBW補間部17aによって任意の画素位置に生成されたR信号、G信号、B信号を用いる例を説明したが、これに限らない。補正係数演算部17cは、RGBW補間部17aが行う方法とは別の方法で各画素位置にR信号,G信号,B信号を生成し、これらを上記式(a)に代入するR,G,Bのデータとして用いても良い。   In the above description, the R, G, and B signals generated by the RGBW interpolation unit 17a as R, G, and B data to be substituted into the equation (a) by the correction coefficient calculation unit 17c, Although the example using B signal was demonstrated, it is not restricted to this. The correction coefficient calculation unit 17c generates R, G, and B signals at each pixel position by a method different from the method performed by the RGBW interpolation unit 17a, and substitutes them into the above equation (a). It may be used as B data.

RGBW補間部17aが行う方法とは別の方法としては、例えば、RGBW補間部17aが行う方法をより簡素化した方法を採用することができる。例えば、RGBW補間部17aが、R信号、G信号、B信号、及びW信号を用いて信号の生成を行う場合には、R,G,B信号のみを用いて生成したデータを使って、補正係数Kを求めても良い。   As a method different from the method performed by the RGBW interpolation unit 17a, for example, a method obtained by simplifying the method performed by the RGBW interpolation unit 17a can be employed. For example, when the RGBW interpolation unit 17a generates a signal using the R signal, the G signal, the B signal, and the W signal, the correction is performed using the data generated using only the R, G, and B signals. The coefficient K may be obtained.

以下、本実施形態のデジタルカメラの別の実施形態を列挙する。   Hereinafter, other embodiments of the digital camera of this embodiment will be listed.

補正係数演算部17cは、算出した補正係数Kが“1”よりも大きかった場合、補正係数Kの値を“1”にクリップすることが好ましい。
補正係数Kが“1”よりも大きいということは、色ノイズは少ない状態であるといえる。このため、この状態で色差信号Cに補正係数Kを乗じてしまうと、逆に色ノイズが強調されてしまう。したがって、このような場合には、補正係数Kを“1”にすることで、誤った補正が行われないようにすることができ、画質劣化を防ぐことが可能となる。
The correction coefficient calculator 17c preferably clips the value of the correction coefficient K to “1” when the calculated correction coefficient K is larger than “1”.
If the correction coefficient K is larger than “1”, it can be said that the color noise is low. For this reason, if the color difference signal C is multiplied by the correction coefficient K in this state, the color noise is emphasized. Therefore, in such a case, by setting the correction coefficient K to “1”, it is possible to prevent erroneous correction and to prevent image quality deterioration.

補正係数演算部17cは、式(1)のように、Wと(α×R+β×G+γ×B+δ)との比でKを決めるのではなく、Wと(α×R+β×G+γ×B+δ)との大きさを比較し、その比較結果に応じて補正係数Kを決めるようにしても良い。例えば、式(1)の右辺の分子に係数Aを掛けた値(A×W)と、式(1)の右辺の分母に係数Bを掛けた値(B×(α×R+β×G+γ×B+δ))とを、係数A,Bを変化させながら比較し、その大小関係に応じて、補正係数Kを決めれば良い。   The correction coefficient calculation unit 17c does not determine K based on the ratio of W and (α × R + β × G + γ × B + δ) as in equation (1), but instead of W and (α × R + β × G + γ × B + δ). The magnitudes may be compared, and the correction coefficient K may be determined according to the comparison result. For example, a value (A × W) obtained by multiplying the numerator on the right side of Equation (1) by a coefficient A (A × W) and a value obtained by multiplying the denominator on the right side of Equation (1) by a coefficient B (B × (α × R + β × G + γ × B + δ). )) May be compared while changing the coefficients A and B, and the correction coefficient K may be determined according to the magnitude relationship.

この場合、デジタルカメラ内には、次の(a)〜(i)の各条件のときの補正係数Kを記憶しておく。
(a)A=1,B=1で(A×W)≧(B×(α×R+β×G+γ×B+δ))であれば、補正係数K=1
(b)A=4,B=3で(A×W)=(B×(α×R+β×G+γ×B+δ))であれば、補正係数K=0.75
(c)A=2,B=1で(A×W)=(B×(α×R+β×G+γ×B+δ))であれば、補正係数K=0.5
(d)A=4,B=1で(A×W)=(B×(α×R+β×G+γ×B+δ))であれば、補正係数K=0.25
(e)A=10,B=1で(A×W)≦(B×(α×R+β×G+γ×B+δ))であれば、補正係数K=0.1
(f)A=1,B=1で(A×W)<(B×(α×R+β×G+γ×B+δ))、且つ、A=4,B=3で(A×W)>(B×(α×R+β×G+γ×B+δ))であれば、補正係数K=0.88
(g)A=4,B=3で(A×W)<(B×(α×R+β×G+γ×B+δ))、且つ、A=2,B=1で(A×W)>(B×(α×R+β×G+γ×B+δ))であれば、補正係数K=0.63
(h)A=2,B=1で(A×W)<(B×(α×R+β×G+γ×B+δ))、且つ、A=4,B=1で(A×W)>(B×(α×R+β×G+γ×B+δ))であれば、補正係数K=0.38
(i)A=4,B=1で(A×W)<(B×(α×R+β×G+γ×B+δ))、且つ、A=10,B=1で(A×W)>(B×(α×R+β×G+γ×B+δ))であれば、補正係数K=0.18
In this case, the correction coefficient K for each of the following conditions (a) to (i) is stored in the digital camera.
(A) If A = 1, B = 1 and (A × W) ≧ (B × (α × R + β × G + γ × B + δ)), the correction coefficient K = 1
(B) If A = 4, B = 3 and (A × W) = (B × (α × R + β × G + γ × B + δ)), the correction coefficient K = 0.75.
(C) If A = 2, B = 1 and (A × W) = (B × (α × R + β × G + γ × B + δ)), the correction coefficient K = 0.5.
(D) If A = 4, B = 1 and (A × W) = (B × (α × R + β × G + γ × B + δ)), the correction coefficient K = 0.25.
(E) If A = 10, B = 1, and (A × W) ≦ (B × (α × R + β × G + γ × B + δ)), the correction coefficient K = 0.1
(F) When A = 1 and B = 1, (A × W) <(B × (α × R + β × G + γ × B + δ)), and when A = 4 and B = 3, (A × W)> (B × (Α × R + β × G + γ × B + δ)), the correction coefficient K = 0.88
(G) A = 4, B = 3, (A × W) <(B × (α × R + β × G + γ × B + δ)), and A = 2, B = 1, (A × W)> (B × (Α × R + β × G + γ × B + δ)), the correction coefficient K = 0.63
(H) When A = 2 and B = 1, (A × W) <(B × (α × R + β × G + γ × B + δ)), and when A = 4 and B = 1, (A × W)> (B × (Α × R + β × G + γ × B + δ)), the correction coefficient K = 0.38.
(I) When A = 4 and B = 1, (A × W) <(B × (α × R + β × G + γ × B + δ)), and when A = 10 and B = 1, (A × W)> (B × (Α × R + β × G + γ × B + δ)), the correction coefficient K = 0.18

補正係数演算部17cは、各画素位置の撮像信号を用いて(A×W)と(B×(α×R+β×G+γ×B+δ))を比較し、その比較結果に対応する補正係数Kをメモリから読み出して乗算器17dに入力すれば良い。   The correction coefficient calculation unit 17c compares (A × W) and (B × (α × R + β × G + γ × B + δ)) using the imaging signal at each pixel position, and stores the correction coefficient K corresponding to the comparison result in the memory. Is read out and input to the multiplier 17d.

Wと(α×R+β×G+γ×B+δ)との比を求める場合には割り算器が必要となるが、割り算器は回路量が膨大となる。上記のように、(A×W)と(B×(α×R+β×G+γ×B+δ))の大小の比較のみで補正係数Kを決定する方法を採用すれば、割り算器が不要となり、小型化や低コスト化が可能となる。   In order to obtain the ratio of W to (α × R + β × G + γ × B + δ), a divider is required, but the divider requires an enormous amount of circuit. As described above, if the method of determining the correction coefficient K only by comparing the magnitudes of (A × W) and (B × (α × R + β × G + γ × B + δ)) is adopted, a divider is unnecessary and the size is reduced. And cost reduction.

固体撮像素子5の光電変換素子の配列は、図1に示したものに限らず、光電変換素子51Rと、光電変換素子51Gと、光電変換素子51Bと、光電変換素子51Wとが一次元状又は二次元状に配設された構成であれば良い。   The arrangement of the photoelectric conversion elements of the solid-state imaging element 5 is not limited to that shown in FIG. 1, but the photoelectric conversion element 51R, the photoelectric conversion element 51G, the photoelectric conversion element 51B, and the photoelectric conversion element 51W are one-dimensional or Any configuration that is two-dimensionally arranged may be used.

図5は、固体撮像素子5の光電変換素子の配列の変形例を示す図である。
図5に示すように、半導体基板上に光電変換素子51R,51G,51B,51Wからなる多数の光電変換素子を正方格子状に配列し、光電変換素子51R,51G,51BからなるRGB光電変換素子群と、光電変換素子51WからなるW光電変換素子群とを、それぞれの各光電変換素子に異なる色をつけたときに市松模様を構成するように、それぞれを互いに市松状に配置した構成としても良い。
FIG. 5 is a diagram illustrating a modification of the arrangement of the photoelectric conversion elements of the solid-state imaging device 5.
As shown in FIG. 5, a large number of photoelectric conversion elements composed of photoelectric conversion elements 51R, 51G, 51B, 51W are arranged in a square lattice pattern on a semiconductor substrate, and an RGB photoelectric conversion element composed of photoelectric conversion elements 51R, 51G, 51B. A group and a W photoelectric conversion element group made up of photoelectric conversion elements 51W may be arranged in a checkered pattern so that a checkered pattern is formed when different colors are applied to the respective photoelectric conversion elements. good.

図1及び図5に示すRGB光電変換素子群の上方に設けるフィルタは、原色フィルタではなく、補色フィルタであっても良い。
補色フィルタを用いた場合でも、固体撮像素子の各光電変換素子に対応する画素位置に、R成分の信号、G成分の信号、B成分の信号、及び輝度成分の信号を生成することは可能であるため、本発明を適用することができる。
The filter provided above the RGB photoelectric conversion element group shown in FIGS. 1 and 5 may be a complementary color filter instead of a primary color filter.
Even when the complementary color filter is used, it is possible to generate an R component signal, a G component signal, a B component signal, and a luminance component signal at the pixel position corresponding to each photoelectric conversion element of the solid-state imaging device. Therefore, the present invention can be applied.

図1に示す例では、固体撮像素子5に含まれる全ての光電変換素子の数と同じ数の画素データからなるカラー画像データを生成しているが、RGB光電変換素子群を構成する光電変換素子の数と同じ数の画素データからなるカラー画像データを生成する撮影モードがあっても良い。   In the example shown in FIG. 1, color image data composed of the same number of pixel data as the number of all the photoelectric conversion elements included in the solid-state imaging element 5 is generated, but the photoelectric conversion elements constituting the RGB photoelectric conversion element group There may be a photographing mode for generating color image data composed of the same number of pixel data as the number of pixel data.

この場合、RGBW補間部17aは、A/D変換回路7から出力されたR信号、G信号、B信号、及びW信号から、メモリ上の光電変換素子51R,51G,51Bの各々に対応する画素位置に、R信号と、G信号と、B信号と、W信号とを生成する処理を行う。   In this case, the RGBW interpolation unit 17a uses the R signal, the G signal, the B signal, and the W signal output from the A / D conversion circuit 7 to correspond to each of the photoelectric conversion elements 51R, 51G, and 51B on the memory. Processing for generating an R signal, a G signal, a B signal, and a W signal is performed at the position.

画素位置に生成すべきW信号は、該画素位置に隣接する光電変換素子51Wから得られたW信号をそのまま利用すれば良い。画素位置に生成すべきR,G,B信号は、R信号、G信号、B信号、及びW信号のうち少なくともR信号、G信号、及びB信号を用いて生成すれば良い。   As the W signal to be generated at the pixel position, the W signal obtained from the photoelectric conversion element 51W adjacent to the pixel position may be used as it is. The R, G, and B signals to be generated at the pixel position may be generated using at least the R signal, the G signal, and the B signal among the R signal, the G signal, the B signal, and the W signal.

そして、光電変換素子51R,51G,51Bの各々に対応する画素位置に、上記生成した信号から色差信号と輝度信号を生成し、色差信号を上述した方法で補正して、カラー画像データを完成させれば良い。   Then, color difference signals and luminance signals are generated from the generated signals at pixel positions corresponding to the photoelectric conversion elements 51R, 51G, and 51B, and the color difference signals are corrected by the above-described method to complete color image data. Just do it.

つまり、最終的に輝度信号と色差信号を生成する画素位置が、特許請求の範囲のカラー画像データを構成する画素データの座標位置に相当する。
固体撮像素子5に含まれる全ての光電変換素子の数と同じ数の画素データからなるカラー画像データを生成する場合には、固体撮像素子5に含まれる全ての光電変換素子に対応する画素位置が上記座標位置になり、RGB光電変換素子群を構成する光電変換素子の数と同じ数の画素データからなるカラー画像データを生成する場合には、RGB光電変換素子群を構成する光電変換素子に対応する画素位置が上記座標位置になる。
That is, the pixel position where the luminance signal and the color difference signal are finally generated corresponds to the coordinate position of the pixel data constituting the color image data in the claims.
When generating color image data composed of the same number of pixel data as the number of all the photoelectric conversion elements included in the solid-state image sensor 5, the pixel positions corresponding to all the photoelectric conversion elements included in the solid-state image sensor 5 When generating color image data consisting of the same number of pixel data as the number of photoelectric conversion elements constituting the RGB photoelectric conversion element group at the coordinate position, it corresponds to the photoelectric conversion element constituting the RGB photoelectric conversion element group The pixel position to perform becomes the coordinate position.

本発明の実施形態である撮像装置に搭載される撮像素子の部分平面模式図Schematic diagram of a partial plane of an image sensor mounted on an image pickup apparatus according to an embodiment of the present invention 図1に示す固体撮像素子を搭載する撮像装置の一例であるデジタルカメラの概略構成を示す図The figure which shows schematic structure of the digital camera which is an example of the imaging device carrying the solid-state image sensor shown in FIG. 図2に示すデジタル信号処理部の内部構成を示す図The figure which shows the internal structure of the digital signal processing part shown in FIG. A/D変換回路7から出力された撮像信号の配列を示した図The figure which showed the arrangement | sequence of the imaging signal output from the A / D conversion circuit 7 図1に示す固体撮像素子5の光電変換素子の配列の変形例を示す図The figure which shows the modification of the arrangement | sequence of the photoelectric conversion element of the solid-state image sensor 5 shown in FIG.

符号の説明Explanation of symbols

5 固体撮像素子
17a RGBW補間部
17b 輝度・色差信号生成部
17c 補正係数演算部
17d 乗算器
51R,51G,51B,51W 光電変換素子
5 Solid-state image sensor 17a RGBW interpolation unit 17b Luminance / color difference signal generation unit 17c Correction coefficient calculation unit 17d Multipliers 51R, 51G, 51B, 51W Photoelectric conversion elements

Claims (5)

第一の色の光を検出する第一の光電変換素子と、第二の色の光を検出する第二の光電変換素子と、第三の色の光を検出する第三の光電変換素子と、光の輝度成分を検出する第四の光電変換素子とを含む固体撮像素子を有する撮像装置であって、
生成すべきカラー画像データを構成する画素データの座標位置に、前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、前記第三の光電変換素子から得られる信号、及び前記第四の光電変換素子から得られる信号のうち、少なくとも前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、及び前記第三の光電変換素子から得られる信号を用いて、R(赤色)成分の信号と、G(緑色)成分の信号と、B(青色)成分の信号とを生成するRGB信号生成手段と、
前記座標位置に生成された前記R成分の信号、前記G成分の信号、及び前記B成分の信号から前記座標位置に対応する色差信号を生成する色差信号生成手段と、
前記座標位置に、前記第四の光電変換素子から得られる信号を用いて、前記座標位置に対応する色差信号の補正用の輝度成分の信号を生成する補正用輝度信号生成手段と、
前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、前記第三の光電変換素子から得られる信号、及び前記第四の光電変換素子から得られる信号のうち、少なくとも前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、及び前記第三の光電変換素子から得られる信号と、前記座標位置に生成された前記補正用の輝度成分の信号とに基づいて、前記座標位置に対応する色差信号を補正する色差信号補正手段とを備え
前記色差信号補正手段が、前記RGB信号生成手段によって前記座標位置に生成された前記R成分の信号、前記G成分の信号、及び前記B成分の信号と、前記座標位置に生成された前記補正用の輝度成分の信号とを用いて、前記座標位置に対応する色差信号を補正する撮像装置。
A first photoelectric conversion element that detects light of a first color, a second photoelectric conversion element that detects light of a second color, and a third photoelectric conversion element that detects light of a third color; An imaging device having a solid-state imaging device including a fourth photoelectric conversion element for detecting a luminance component of light,
Obtained from the signal obtained from the first photoelectric conversion element, the signal obtained from the second photoelectric conversion element, and the third photoelectric conversion element at the coordinate position of the pixel data constituting the color image data to be generated Of the signal and the signal obtained from the fourth photoelectric conversion element, at least the signal obtained from the first photoelectric conversion element, the signal obtained from the second photoelectric conversion element, and the third photoelectric conversion element RGB signal generation means for generating an R (red) component signal, a G (green) component signal, and a B (blue) component signal using the signal obtained from
Color difference signal generating means for generating a color difference signal corresponding to the coordinate position from the R component signal, the G component signal, and the B component signal generated at the coordinate position;
A correction luminance signal generating means for generating a signal of a luminance component for correction of a color difference signal corresponding to the coordinate position using a signal obtained from the fourth photoelectric conversion element at the coordinate position;
Of the signal obtained from the first photoelectric conversion element, the signal obtained from the second photoelectric conversion element, the signal obtained from the third photoelectric conversion element, and the signal obtained from the fourth photoelectric conversion element , At least a signal obtained from the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, and a signal obtained from the third photoelectric conversion element, and the correction generated at the coordinate position based on the signal of the luminance component of a color difference signal correcting means for correcting the color difference signal corresponding to the coordinate position,
The color difference signal correcting unit is configured to generate the R component signal, the G component signal, and the B component signal generated at the coordinate position by the RGB signal generating unit, and the correction signal generated at the coordinate position. An image pickup apparatus that corrects a color difference signal corresponding to the coordinate position using a signal of a luminance component .
第一の色の光を検出する第一の光電変換素子と、第二の色の光を検出する第二の光電変換素子と、第三の色の光を検出する第三の光電変換素子と、光の輝度成分を検出する第四の光電変換素子とを含む固体撮像素子を有する撮像装置であって、
生成すべきカラー画像データを構成する画素データの座標位置に、前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、前記第三の光電変換素子から得られる信号、及び前記第四の光電変換素子から得られる信号のうち、少なくとも前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、及び前記第三の光電変換素子から得られる信号を用いて、R(赤色)成分の信号と、G(緑色)成分の信号と、B(青色)成分の信号とを生成するRGB信号生成手段と、
前記座標位置に生成された前記R成分の信号、前記G成分の信号、及び前記B成分の信号から前記座標位置に対応する色差信号を生成する色差信号生成手段と、
前記座標位置に、前記第四の光電変換素子から得られる信号を用いて、前記座標位置に対応する色差信号の補正用の輝度成分の信号を生成する補正用輝度信号生成手段と、
前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、前記第三の光電変換素子から得られる信号、及び前記第四の光電変換素子から得られる信号のうち、少なくとも前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、及び前記第三の光電変換素子から得られる信号と、前記座標位置に生成された前記補正用の輝度成分の信号とに基づいて、前記座標位置に対応する色差信号を補正する色差信号補正手段とを備え、
前記色差信号補正手段が、前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、前記第三の光電変換素子から得られる信号、及び前記第四の光電変換素子から得られる信号のうち、少なくとも前記第一の光電変換素子から得られる信号、前記第二の光電変換素子から得られる信号、及び前記第三の光電変換素子から得られる信号を用いて、前記RGB信号生成手段とは別の方法で、前記座標位置に前記色差信号の補正用の前記R成分の信号、前記G成分の信号、及び前記B成分の信号を生成し、前記補正用の前記R成分の信号、前記G成分の信号、及び前記B成分の信号と、前記補正用の輝度成分の信号とを用いて、前記色差信号を補正する撮像装置。
A first photoelectric conversion element that detects light of a first color, a second photoelectric conversion element that detects light of a second color, and a third photoelectric conversion element that detects light of a third color; An imaging device having a solid-state imaging device including a fourth photoelectric conversion element for detecting a luminance component of light,
Obtained from the signal obtained from the first photoelectric conversion element, the signal obtained from the second photoelectric conversion element, and the third photoelectric conversion element at the coordinate position of the pixel data constituting the color image data to be generated Of the signal and the signal obtained from the fourth photoelectric conversion element, at least the signal obtained from the first photoelectric conversion element, the signal obtained from the second photoelectric conversion element, and the third photoelectric conversion element RGB signal generation means for generating an R (red) component signal, a G (green) component signal, and a B (blue) component signal using the signal obtained from
Color difference signal generating means for generating a color difference signal corresponding to the coordinate position from the R component signal, the G component signal, and the B component signal generated at the coordinate position;
A correction luminance signal generating means for generating a signal of a luminance component for correction of a color difference signal corresponding to the coordinate position using a signal obtained from the fourth photoelectric conversion element at the coordinate position;
Of the signal obtained from the first photoelectric conversion element, the signal obtained from the second photoelectric conversion element, the signal obtained from the third photoelectric conversion element, and the signal obtained from the fourth photoelectric conversion element , At least a signal obtained from the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, and a signal obtained from the third photoelectric conversion element, and the correction generated at the coordinate position Color difference signal correcting means for correcting the color difference signal corresponding to the coordinate position based on the luminance component signal of
The color difference signal correcting means is a signal obtained from the first photoelectric conversion element, a signal obtained from the second photoelectric conversion element, a signal obtained from the third photoelectric conversion element, and the fourth photoelectric conversion. Among the signals obtained from the element, at least the signal obtained from the first photoelectric conversion element, the signal obtained from the second photoelectric conversion element, and the signal obtained from the third photoelectric conversion element, The R component signal for correcting the color difference signal, the G component signal, and the B component signal are generated at the coordinate position by a method different from the RGB signal generating means, and the R component signal for correction is generated. An imaging device that corrects the color difference signal using a component signal, the G component signal, the B component signal, and the correction luminance component signal .
請求項記載の撮像装置であって、
前記色差信号補正手段が、前記色差信号に補正係数を乗じて補正後の色差信号を求める乗算手段と、前記補正係数を演算する補正係数演算手段とで構成され、
前記補正係数演算手段が、前記補正係数をKとして、次の式、
K=W/(α×R+β×G+γ×B+δ)
W:前記座標位置に生成された前記補正用の輝度成分の信号
R:前記座標位置に生成された前記R成分の信号
G:前記座標位置に生成された前記G成分の信号
B:前記座標位置に生成された前記B成分の信号
α,β,γ,δ:前記第四の光電変換素子の分光特性を、前記第一の光電変換素子、前記第二の光電変換素子、及び前記第三の光電変換素子の各々の分光特性の積和で近似したときに、その近似した分光特性と前記第四の光電変換素子の分光特性との誤差が最小となるように決められた係数
により前記補正係数を演算する撮像装置。
The imaging apparatus according to claim 1 ,
The color difference signal correcting means includes a multiplying means for multiplying the color difference signal by a correction coefficient to obtain a corrected color difference signal, and a correction coefficient calculating means for calculating the correction coefficient.
The correction coefficient calculating means uses the correction coefficient as K and the following equation:
K = W / (α × R + β × G + γ × B + δ)
W: Signal of the correction luminance component generated at the coordinate position R: Signal of the R component generated at the coordinate position G: Signal of the G component generated at the coordinate position B: The coordinate position The B component signals α, β, γ, and δ generated in the above are the spectral characteristics of the fourth photoelectric conversion element, the first photoelectric conversion element, the second photoelectric conversion element, and the third photoelectric conversion element. When approximated by the product sum of each spectral characteristic of the photoelectric conversion element, the correction coefficient is determined by a coefficient determined so that an error between the approximated spectral characteristic and the spectral characteristic of the fourth photoelectric conversion element is minimized. An imaging device for calculating
請求項記載の撮像装置であって、
前記色差信号補正手段が、前記色差信号に補正係数を乗じて補正後の色差信号を求める乗算手段と、前記補正係数を演算する補正係数演算手段とで構成され、
前記補正係数演算手段が、前記補正係数をKとして、次の式、
K=W/(α×R+β×G+γ×B+δ)
W:前記座標位置に生成された前記補正用の輝度成分の信号
R:前記座標位置に生成された前記補正用のR成分の信号
G:前記座標位置に生成された前記補正用のG成分の信号
B:前記座標位置に生成された前記補正用のB成分の信号
α,β,γ,δ:前記第四の光電変換素子の分光特性を、前記第一の光電変換素子、前記第二の光電変換素子、及び前記第三の光電変換素子の各々の分光特性の積和で近似したときに、その近似した分光特性と前記第四の光電変換素子の分光特性との誤差が最小となるように決められた係数
により前記補正係数を演算する撮像装置。
The imaging apparatus according to claim 2 ,
The color difference signal correcting means includes a multiplying means for multiplying the color difference signal by a correction coefficient to obtain a corrected color difference signal, and a correction coefficient calculating means for calculating the correction coefficient.
The correction coefficient calculating means uses the correction coefficient as K and the following equation:
K = W / (α × R + β × G + γ × B + δ)
W: A signal of the correction luminance component generated at the coordinate position R: A signal of the correction R component generated at the coordinate position G: A signal of the G component for correction generated at the coordinate position Signal B: B component signal for correction generated at the coordinate position α, β, γ, δ: Spectral characteristics of the fourth photoelectric conversion element, the first photoelectric conversion element, the second photoelectric conversion element When approximated by the product sum of the spectral characteristics of the photoelectric conversion element and the third photoelectric conversion element, an error between the approximated spectral characteristic and the spectral characteristic of the fourth photoelectric conversion element is minimized. An imaging device that calculates the correction coefficient according to a coefficient determined by
請求項又は記載の撮像装置であって、
前記補正係数演算手段が、演算の結果、前記Kの値が“1”より大きかった場合、前記Kの値を“1”にクリップする撮像装置。
The imaging apparatus according to claim 3 or 4 ,
An imaging apparatus that clips the value of K to “1” when the value of K is greater than “1” as a result of the calculation by the correction coefficient calculation means.
JP2008098353A 2008-04-04 2008-04-04 Imaging device Expired - Fee Related JP5033702B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2008098353A JP5033702B2 (en) 2008-04-04 2008-04-04 Imaging device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2008098353A JP5033702B2 (en) 2008-04-04 2008-04-04 Imaging device

Publications (3)

Publication Number Publication Date
JP2009253616A JP2009253616A (en) 2009-10-29
JP2009253616A5 JP2009253616A5 (en) 2011-04-21
JP5033702B2 true JP5033702B2 (en) 2012-09-26

Family

ID=41313863

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2008098353A Expired - Fee Related JP5033702B2 (en) 2008-04-04 2008-04-04 Imaging device

Country Status (1)

Country Link
JP (1) JP5033702B2 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5673186B2 (en) * 2011-02-16 2015-02-18 株式会社Jvcケンウッド Imaging apparatus and interpolation processing method of imaging apparatus
KR101980931B1 (en) 2012-01-24 2019-05-21 소니 주식회사 Image processing device, image processing method, and recording medium
JP2015162706A (en) 2014-02-26 2015-09-07 ソニー株式会社 Image processing device and method, and program
JP6258842B2 (en) * 2014-12-10 2018-01-10 株式会社Soken Image processing apparatus and lane boundary line recognition system

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006064564A1 (en) * 2004-12-16 2006-06-22 Fujitsu Limited Imaging device, imaging element, and image processing method
JP2007074635A (en) * 2005-09-09 2007-03-22 Matsushita Electric Ind Co Ltd Image input apparatus and solid imaging element
JP4501855B2 (en) * 2005-12-22 2010-07-14 ソニー株式会社 Image signal processing apparatus, imaging apparatus, image signal processing method, and computer program
JP4144630B2 (en) * 2006-04-14 2008-09-03 ソニー株式会社 Imaging device
JP4967432B2 (en) * 2006-04-14 2012-07-04 ソニー株式会社 Imaging apparatus and camera system

Also Published As

Publication number Publication date
JP2009253616A (en) 2009-10-29

Similar Documents

Publication Publication Date Title
US8988591B2 (en) Solid-state imaging device, camera module, and focus adjustment method of camera module
JP2013093914A (en) Image input device
KR20040057927A (en) Image synthesis method and image pickup apparatus
JP2007259344A (en) Imaging apparatus and image processing method
JP2006270622A (en) Imaging apparatus and image processing method
WO2018098981A1 (en) Control method, control device, electronic device and computer-readable storage medium
JP2011113182A (en) Image processing apparatus
JP2010199650A (en) Image processing apparatus and method
KR20110093079A (en) Wide dynamic range hardware apparatus and photographing apparatus
JP6282123B2 (en) Image processing apparatus, image processing method, and program
JP5033702B2 (en) Imaging device
US10326947B2 (en) Image processing device, imaging device, image processing method, and program
US9565411B2 (en) Image processing device, image processing method, and non-transitory computer-readable recording medium for recording image processing program
JP2008177794A (en) Color shading correcting apparatus of imaging apparatus, imaging apparatus, and color shading correcting method of imaging apparatus
JP4777038B2 (en) Imaging apparatus and image processing method
JP5631769B2 (en) Image processing device
JP2006148881A5 (en)
JPWO2006059365A1 (en) Image processing device, non-imaging color signal calculation device, and image processing method
JP4940661B2 (en) Image processing apparatus, image processing method, and imaging apparatus
JP2008219230A (en) Imaging apparatus, and image processing method
JP4735820B2 (en) Signal processing method for imaging apparatus and imaging apparatus
JP2006279389A (en) Solid-state imaging apparatus and signal processing method thereof
JP5482428B2 (en) Imaging apparatus, camera shake correction method, and program
TWI405144B (en) Image correcting method, image correcting unit and image capture apparatus using the same
JP2012222704A (en) Imaging apparatus

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20110308

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20110308

RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20111216

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20120524

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20120605

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20120702

R150 Certificate of patent or registration of utility model

Ref document number: 5033702

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

Free format text: JAPANESE INTERMEDIATE CODE: R150

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20150706

Year of fee payment: 3

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

LAPS Cancellation because of no payment of annual fees