JP3984666B2 - Imaging device - Google Patents

Description

【００７０】
また、現在作られている全てのサンプルポイントグループに含ませることができない場合は、同サンプルポイントを基準とする新たなサンプルポイントグループを作成する。
【００７１】
全てのサンプルポイントにおいて、ステップＳ１７〜ステップＳ１９の処理が終了したら、ステップＳ１７で作られた一つあるいは複数個のサンプルポイントグループの中から、本当の白のグループを探すわけであるが、実際にはどれが白であるかわからない。
【００７２】
そこで、次に、まず、図２のステップＳ２４において、サンプルポイントグループを輝度の高い順にソートする。
次に、輝度の高いものは白であるという仮定をして、上記複数のサンプルポイントグループの輝度情報GpＢrightness を用いて、輝度の大きいものからＮ番目までを白である確率が高いグループとしてそのサンプルポイントグループの重み付けを行う。重み付けは以下の式で行う（ステップＳ２５、ステップＳ２６）。
【００７３】

【００７４】
一方、ステップＳ１５で求めて座標に表したホワイト点(X1,Y1)(X2,Y2)が所定範囲内にないものとステップＳ１６で判断されたならば、図１のステップＳ２７に進み、図５のように同座標点を所定範囲枠上に移動し、範囲枠上の座標(X',Y') から、次式を用いて、色フィルタ出力Ma1'、Ma2'、Ye1'、Ye2'を新たに求め直す。
Ma1'=(1+Y')*Gr1
Ma2'=(1+Y')*Gr2
Ye1'=(1+X')*Cy1
Ye2'=(1+X')*Cy2
【００７５】
ただし、座標点の移動の仕方は図５のように、座標点に対して最も近い所定範囲枠上の座標に移動するものとする。この座標移動処理を色クリップ処理と呼ぶ。この色クリップ処理されたサンプルポイントの総数を、次のステップＳ２８でカウントし、SN2 とする。次に、ステップＳ２９に進み、色クリップによって求められる積分値をMa1'、Ma2'、Gr1'、Gr2'、Cy1'、Cy2'、Ye1'、Ye2'とする。
【００７６】
重み付けが終わったら、次に、ステップＳ３０に進み、ステップＳ１７〜ステップＳ１９及びステップＳ２５〜ステップＳ２６で得られた全てのサンプルポイントグループに対して、以下の式を用いて重み付け、白判別範囲内の積分値(SMa1,SMa2,SGr1,SGr2,SCy1,SCy2,SYe1,SYe2) を求める。
【００７７】
SMa1=SMa1+GpMa1*GpWeight
SMa2=SMa2+GpMa2*GpWeight
SGr1=SGr1+GpGr1*GpWeight
SGr2=SGr2+GpGr2*GpWeight
SCy1=SCy1+GpCy1*GpWeight
SCy2=SCy2+GpCy2*GpWeight
SYe1=SYe1+GpYe1*GpWeight
SYe2=SYe2+GpYe2*GpWeight
【００７８】
次に、ステップＳ３１において、同時に白とみなされたサンプルポイント数SN1 を求める。
SN1=SN1+GpNumber
【００７９】
次に、ステップＳ３２において、全てのサンプルポイントグループに実行したたか否かを判断し、実行していなければステップＳ３０に戻る。また、全てに実行した場合にはステップＳ３３に進む。
【００８０】
ステップＳ３３においては、サンプル数(SN1+SN2) が０、または所定以上得られているか否かを判断し、所定以上得られていなければ、ステップＳ３４に進み、ホワイトバランス調整データとして、所定の色温度（例えば5000K ）のホワイトバランス調整値を用いる。
【００８１】
また、ステップＳ３３の判断の結果、サンプル数(SN1+SN2) が所定以上得られている場合には、ステップＳ３５に進み、白撮影時の平均輝度レベルとしての、
SY=(SMa1+SMa2+SGr1+SGr2+SCy1+SCy2+SYe1+SYe2)/8
を求め、この平均輝度レベルに対する各色の積分値の比率を求める。
gmg1=SY/(2*SMa1) gmg2=SY/(2*SMa2)
gg1=SY/(2*SGr1) gg2=SY/(2*SGr2)
gcy1=SY/(2*SCy1) gcy2=SY/(2*SCy2)
gye1=SY/(2*SYe1) gye2=SY/(2*SYe2)
【００８２】
次に、ステップＳ３６に進み、クリップ時の平均輝度レベルとしての
SY'=(SMa1'+SMa2'+SGr1'+SGr2'+SCy1'+SCy2'+SYe1'+SYe2'/8
を求め、この平均輝度レベルに対する各色の積分値の比率を求める。
gmg1'=SY'/(2*SMa1') gmg2'=SY'/(2*SMa2')
gg1'=SY'/(2*SGr1') gg2'=SY'/(2*SGr2')
gcy1'=SY'/(2*SCy1') gcy2'=ST'/(2*SCy2')
gye1'=SY'/(2*SYe1') gye2'=SY'/(2*SYe2')
【００８３】
次に、ステップＳ３７に進み、ステップＳ３５およびステップＳ３６で得られた、各色の積分値の比率と、白判別範囲内の積分サンプル数SN1 と、色クリップによる積分サンプル数SN2 とにより、ホワイトバランス調整データ(wmg1,wg1,wcy1,wye1,wmg2,wg2,wcy2,wye2) を作る。
【００８４】
例えば、

のような処理を施す。
【００８５】
これらの値をホワイトバランス調整データとして記憶する。
上記処理で得られたホワイトバランス調整データを、撮像素子から得られた画素毎の信号値にかけることで、ホワイトバランスをとることができる。
【００８６】
以上説明したように、本実施形態においては撮像素子からの信号値でホワイトバランスのための座標を作り、その座標において白領域に入ったサンプル点に対し、白いものは相対的に輝度値が高いという仮定のもとに重み付けを行う。
【００８７】
その後、そのサンプル点で積分を行う処理をするので、画像中に白が少ない場合であっても、他の色に影響されてホワイトバランスが大きくくずれることを回避することが可能となる。
【００８８】
（第２の実施形態）
次に、本発明の第２の実施形態を説明する。上述した第１の実施形態では、電子カメラの信号読みだしはフレーム読みだしであったが、この第２の実施形態においては電子カメラの信号をフィールド読みだしで行っている。
【００８９】
本実施形態における撮像素子はフィールド読み出しを行うため、図９に示すように、撮像素子からの出力は垂直方向２画素が加算されて読み出される。この図９において、
Wb=Ma+Cy
Gr=G+Ye
Gb=Cy+G
Wr=Ye+Ma
である。
【００９０】
本実施形態の場合、色フィルタが(Ma,g,Cy,Ye)から(Wb,Gr,Gb,Wr) に変わったものと考えると、上述したフレーム読み出しとほぼ同じ処理を行うことになり、ハード構成は、上述した第１の実施形態における図３と同様であるので、図示を省略する。
【００９１】
本実施形態において、まず、オプティカルブラック領域における各フィルタ出力の積分値を用いて黒レベルの調整を行う。
次に、ホワイトバランス処理を行い、白点におけるWb、Gr、Gb、Wrの出力ゲインが同じになるようにする。
【００９２】
輝度信号（Ｙ）を作るには、色フィルタによるモアレを除去するために水平ローパスフィルタがかけられた後、アパチャー補正を行う。ここでは、水平、垂直にそれぞれハイパスフィルタがかけられ、ゲインを調整、アパチャー信号の振幅の小さいものにはそのＣＰＵが大きくなるように、また振幅の大きいものにはその振幅が小さくなるようにPeak To Peak処理が施され、水平、垂直が足され、さらにゲインをかけて、輝度成分（Ｙｎ）に加えられる。
【００９３】
色差信号(R-Y,B-Y) を作るには、まず同時化回路で４色の信号を同時化し、撮像面領域に４つの色信号が割り当てられるよう画素補間を行う。
次に、４×３のマトリクス回路を用いてWb信号、Gr信号、Gb信号、Wr信号をＲＧＢ信号に変換する。このとき、マトリクス回路は、ホワイトバランス情報から被写体の色温度を判別して、その色温度における色再現の最もよくなるようなマトリクス回路が用いられる。
【００９４】
次に、ＲＧＢ信号をR-Y 、B-Y 信号に変換し、モアレ低減のためのローパスフィルタをかけ、Ｙ信号を参考に飽和の色消し処理を行う。
このようにして作られた、輝度信号（Ｙ）と色差信号(R-Y,B-Y) は、ＲＧＢ信号に変換され、ガンマ補正処理を行った後、記録される。
【００９５】
次に、本実施形態の主眼点である、ホワイトバランス処理について詳細に説明する。ＣＣＤ補色フィルタが図９（ａ）のように配列している場合、図の８画素配列を単位ブロックとして、この単位ブロックの繰り返しで全体の配列が成り立っている。ここで、フィールド読み出しを行うため、１フィールドでは、図９（ｂ）のような信号が読み出される。
【００９６】
次に、図７、図８を参照しながら第２の実施形態の処理手順を説明する。
まず、最初のステップＳ６１において、１サンプルごとにサンプルデータを抽出する。
【００９７】
次に、ステップＳ６２において、上記ステップＳ６１で抽出したサンプルデータがＷＢサンプル点として有効か否かを判断する。上記判断の結果、上記抽出したサンプルデータがＷＢサンプル点として有効である場合にはステップＳ６３に進み、有効でない場合にはステップＳ６１からやり直す。
【００９８】
ステップＳ６３においては、サンプルポイントがＷＢデータとして計算できる明るさがあるか否かを調べ、サンプルポイントの４画素全て(Wb,Gr,Gb,Wr) が、次の式を満たす場合は次のステップＳ６４に進み、一つでも満たさない場合、次のサンプルポイントへ移行しステップ１からやり直す。
DarkThreshold<PixelValue<BrightThreshold
【００９９】
次に、ステップＳ６４において、サンプルポイントにおいて、以下の式の値を求める。
X1=((Wr-Gb)-(Wb-Gr))/(Wr+Gb) …（６）
Y1=((Wr-Gb)+(Wb-Gr))/(Wb+Gr) …（７）
【０１００】
次に、ステップＳ６５に進み、ＷＢバランス設定用のデータとして有効か否かを判断する。すなわち、図５に示したように、((Wr-Gb)-(Wb-Gr))/(Wr+Gb) をＸ軸、((Wr-Gb)+(Wb-Gr))/(Wb+Gr) をＹ軸にとり、上記式（６）、（７）が、Ｘ、Ｙ軸上に設けた所定の領域内にある場合、つまり、以下の条件式、
a<X1<b, c<Y1<d …（８）
を満たす場合、同サンプルポイントをＷＢ設定するためのデータとして有効とみなし、ステップＳ６６へ進む。また、上記式（８）を満たさない場合はステップＳ７６へ進む。
【０１０１】
ステップＳ６６においては、同サンプルポイントにおいて、以下の式を用いて、ＷＢ座標におけるサンプルポイントのＸＹ軸値の平均値、及び、輝度値を求める。
AverageX=X1
AverageY=Y1
AverageBrightness=(Wb+Gr+Gb+Wr)/2
【０１０２】
次に、ステップＳ６７に進み、同サンプルポイントが最初のＷＢ設定有効データか否かを判断する。そして、上記判断の結果、同サンプルポイントがＷＢ設定有効データとみなされた最初のサンプルポイントである場合は、ステップＳ６８に進む。
【０１０３】
ステップＳ６８においては、図６のような値をパラメータとして持つサンプルポイントグループを作り、ステップＳ６６で求めた値、及び、同ポイントの画素値を代入する。
GpX=AverageX
GpY=AverageY
GpBrightness=AverageBrightness
GpWb=Wb
GpGr=Gr
GpGb=Gb
GpWr=Wr
GpNumber=1
GpWeight=1
【０１０４】
また、同サンプルポイントがＷＢ設定有効データとみなされた最初でないサンプルポイントである場合は、ステップＳ６９に進み、サンプルポイントグループに含まれるか否かを判断する。
【０１０５】
上記判断は、既存のサンプルポイントグループのGpX 、GpY 、GpBrightnessの値を用いて、以下の条件式を満たすときに、同サンプルポイントはそのサンプルポイントグループに含まれると判断する。この場合、ステップＳ７０に進み、サンプルポイントグループに含ませるようにする。
【０１０６】
すなわち、

【０１０７】
また、全てのサンプルポイントグループに含ませることができない場合は、ステップＳ７１に進み、サンプルポイントグループが終わりか否かを判断する。上記判断の結果、終わりでない場合にはステップＳ６９に戻り、また、終わりの場合にはステップＳ６８に進み、上述したように、同サンプルポイントを基準とする新たなサンプルポイントグループを作成する。
【０１０８】
ステップＳ６８、あるいはステップＳ７０の処理が終了したら、次に、ステップＳ７２に進み、全サンプルデータ抽出を完了したか否かを判断する。そして、上記判断の結果、終了していない場合にはステップＳ６１に戻り、上述した処理を繰り返し行う。
【０１０９】
全サンプルデータ抽出を完了した場合には、次に、図８のステップＳ７３〜７５の処理を行う。
これは、全てのサンプルポイントにおいて、ステップＳ６７〜ステップＳ７１の処理が終了したら、ステップ６７〜ステップＳ７１で作られた一つあるいは複数個のサンプルポイントグループの中から、本当の白のグループを探す処理を行う。
【０１１０】
しかし、本当の白のグループは、実際にはどれが白であるか分からない。そこで、輝度の高いものは白であるという仮定をして、上記複数のサンプルポイントグループの輝度情報GpBrightnessを用いて、輝度の大きいものからＮ番目までを本当の白グループとして、そのサンプルポイントの重み付けを行う。すなわち、ステップＳ７３〜ステップＳ７５の処理を順次行う。
【０１１１】
重み付けの処理は以下の式で行う。
GpWeight=GpWeight*2
ただし、Ｎは１以上であり、例えば、次式で求められる。
【０１１２】

【０１１３】
一方、ステップＳ６４で求め座標に表したホワイト点(X1,Y1) 、(X2,Y2) が所定範囲内にないものとステップＳ６５で判断されたならば、同座標点を所定範囲枠上に移動し、Wb' 、Wr' を新たに求め直す（ステップＳ７６）。
【０１１４】
次に、ステップＳ７７において、上記色クリップされたサンプルポイントの総数をSN2 とした後、ステップＳ７８において色クリップによって求められる積分値をWb' 、Gr' 、Gb' 、Wr' を各色毎に積分値に加算する処理を行う。
【０１１５】
重み付けが終わったら、次に、ステップＳ７９に進み、ステップＳ６７〜ステップＳ７１及びステップＳ７４〜ステップＳ７５で得られた全てのサンプルポイントグループに対して、以下の式を用いて、白判別範囲内の積分値(SWb,SGr,SGb,SWr) を求める。
SWb=SWb+GpWb*GpWeight
SGr=SGr+GpGr*GpWeight
SGb=SGb+GpGb*GpWeight
SWr=SWr+GpWr*GpWeight
【０１１６】
次に、ステップＳ８０に進み、同時に白とみなされたサンプルポイント数SN1 を求める。
SN1=SN1+GpNumber
【０１１７】
次に、ステップＳ８１において、全てのサンプルポイントグループに対して実行したか否かを判断し、実行していない場合にはステップＳ７９に戻って上述した動作を繰り返し行う。
【０１１８】
また、全てのサンプルポイントグループに対して実行している場合には、ステップＳ８２に進み、サンプル数(SN1+SN2) が０、または所定以上得られているか否かを判断する。
【０１１９】
ステップＳ８２の判断の結果、サンプル数(SN1+SN2) が所定以上得られていなければ、ステップＳ８３に進み、ホワイトバランス調整データとして、所定の色温度（例えば5000K)のホワイトバランス調整値を用いる。
【０１２０】
一方、ステップＳ８２の判断の結果、サンプル数(SN1+SN2) が所定以上得られていれば、ステップＳ８４に進み、白撮影時の平均輝度レベルとしての、
SY=(SWb+SGr+SGb+SWr)/4
を求め、この平均輝度レベルに対する各色の積分値の比率を求める。
gwb=SY/(2*SWb)
ggr=SY/(2*SGr)
ggb=SY/(2*SGb)
gwr=SY/(2*SWr)
【０１２１】
次に、ステップＳ８５に進み、クリップ時の平均輝度レベルとしての、
SY'=(SWb'+SGr'+SGb'+SWr') /4
を求め、この平均輝度レベルに対する各色の積分値の比率を求める。
gwb'=SY'/(2*SWb')
ggr'=SY'/(2*SGr')
ggb'=SY'/(2*SGb')
gwr'=SY'/(2*SWr')
【０１２２】
次に、ステップＳ８６に進み、ステップＳ８４およびステップＳ８５で得られた、各色の積分値の比率と、白判別範囲内の積分サンプル数SN1 と色クリップによる積分サンプル数SN2 より、ホワイトバランス調整データ(wwb,wgr,wgb,wwr) を作る。
【０１２３】
例えば、

のような処理を施す。そして、これらの値をホワイトバランス調整データとして記憶する。
【０１２４】
上記処理で得られたホワイトバランス調整データを、撮像素子から得られた画素毎の信号値にかけることでホワイトバランスをとることができる。
【０１２５】
（第３の実施形態）
図１０は、本発明の第３の実施形態のホワイトバランスの処理手順の一部を説明するためのフローチャートの１部分であり、図１におけるステップＳ２５〜ステップＳ２６に相当する部分である。この部分は、ホワイトバランスグループを作成した後、グループに重み付けをする部分であるが、本実施形態は、この重み付けに撮影時の照度情報を用いることを特徴とする。
【０１２６】
上記照度情報とは、例えばＥＶ値であり、ステップＳ９１〜ステップＳ９５の処理を行う。すなわち、まず、カメラ側から情報としてＥＶ値をもらい、ＥＶ値が大きい場合は外で撮影（太陽光）したものである場合が大きいので、図１１（ａ）のように白座標範囲を限定する。
【０１２７】
また、逆にＥＶ値が小さいのは夕日、もしくは人工光と限定し、図１１（ｂ）のように白座標範囲を限定する。ここで、サンプルポイントグループすべてに対し、以下のように重み付けを行う。つまり、限定座標範囲にあるホワイトバランスグループの重みを増す。
【０１２８】

【０１２９】
このような重み付けを行うことで、白の輝度の低い場合にも、より好ましいホワイトバランスを行うことができる。また、限定座標範囲が、本来は白でないサンプル点に入り、サンプルポイントグループがたくさんできていても、重み付けによる白グループ選択を行っているため、ホワイトバランスが大きくくずれることはない。
【０１３０】
また、本実施形態における重み付けを、第２の実施形態におけるステップＳ７４〜ステップＳ７５に用いてもよい。また、第１の実施形態及び第２の実施形態の重み付けの手法と組み合わせて用いてもよい。
【０１３１】
【発明の効果】
本発明は上述したように、白判別に、従来の座標とは異なる座標軸を用いることによって、より正確な色判別を行い、また、輝度信号Ｙまたは撮像素子の露光量を用いて、白判別内の信号値に重み付けを行うことにより、例えば、画像中に白が占める割合が低くても、他の色の影響を少なくすることができるため、ホワイトバランスが大きく崩れることが無く、より正確なホワイトバランス信号を得ることが可能になる。
【０１３２】
また、本発明に係わる第２の発明によれば、撮像素子をフィールド読み出し駆動した場合において、第１の発明と同様の効果が得られ、良好なホワイトバランス調整値を得ることができる。
【図面の簡単な説明】
【図１】本発明の第１の実施形態の処理を示すフローチャートである。
【図２】第１の実施形態の処理を示すフローチャートである。
【図３】本発明を適用した電子カメラの信号処理の様子を説明するためのブロック図である。
【図４】撮像素子の色フィルタパターンの一例を示す図である。
【図５】ホワイトバランスの概念を示す図である。
【図６】サンプルポイントグループの各パラメータを示す図である。
【図７】第２の実施形態の処理を示すフローチャートである。
【図８】第２の実施形態の処理を示すフローチャートである。
【図９】撮像素子のフィールド読みだしの様子を示す図である。
【図１０】第３の実施形態の処理を示すフローチャートである。
【図１１】第３の実施形態におけるホワイトバランス座標の重み付け領域を示す図である。
【図１２】従来の電子カメラの一例を示すブロック図である。
【図１３】図１２の電子カメラの信号処理回路の構成を示すブロック図である。
【図１４】ホワイトバランス制御回路の構成を示すブロック図である。
【図１５】白判別範囲を示す図である。
【図１６】色差信号と輝度信号との関係を示す図である。
【符号の説明】
３１ 積分器
３２ 黒レベル調整回路
３３ ホワイトバランス検出回路
３４ ゲイン調整回路
３５ ローパスフィルタ
３６ａ、３６ｂ ハイパスフィルタ
３７ａ、３７ｂ ゲイン調整回路
３８ａ、３８ｂ 振幅調整回路
３９ａ、３９ｂ ベースクリップ回路
４０ 加算回路
４１ ゲイン調整回路
４２ 加算回路
５０ 同時化回路
５１ 画素補間回路
５２ 第１のマトリクス回路
５３ 第２のマトリクス回路
５４ ローパスフィルタ
５５ 色処理回路
５６ カラーデコーダ
５７ ガンマ処理回路[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an imaging apparatus, and is particularly suitable for use in an imaging apparatus that captures a still image.
[0002]
[Prior art]
FIG. 12 is a block diagram showing an outline of a conventional digital electronic camera.
In FIG. 12, reference numeral 300 denotes a digital electronic camera, and 308 denotes a recording medium such as a memory card.
[0003]
In the digital electronic camera 300, reference numeral 301 denotes an imaging lens capable of adjusting the focal length and angle of view, 302 denotes a diaphragm / shutter having both an aperture function and a shutter function, and 304 denotes a mechanical system driving circuit for driving the imaging lens 301 and the diaphragm / shutter 302. It is.
[0004]
101 is an image sensor that converts reflected light from an object into an electrical signal, 103 is a timing signal generation circuit (hereinafter referred to as TG) that generates a timing signal necessary for operating the image sensor, and 102 is a timing signal generation circuit. An image sensor driving circuit that amplifies the above signal to a level at which the image signal can be driven, 104 is a pre-processing circuit (preprocess means) including a CDS circuit and an AGC circuit for removing output noise of the image sensor 101, and 105 is A / D converter.
[0005]
An imaging signal processing circuit 303 includes an imaging signal processing circuit for processing a digital signal A / D converted by the A / D converter 105, 108 is a system control CPU, 116 is a display for assisting operation and a state of the camera An operation display unit 118 represents a switch for the external photographer to control the start of shooting of the camera, and can be operated in two steps, the first stroke SW1 and the second stroke SW2.
[0006]
Reference numeral 307 denotes a recording medium I / F for connecting the digital electronic camera 300 and the recording medium 308. Reference numeral 305 denotes a device-supplied distance measuring circuit that measures the distance between the imaging device and the subject for focus adjustment of the optical system 301, and reference numeral 306 denotes a photometric circuit that measures the brightness of the subject.
[0007]
FIG. 13 is a block diagram showing the internal configuration of the image signal processing circuit 303 and its periphery.
In FIG. 13, reference numeral 101 denotes an image pickup device (CCD) having a complementary color filter. The picked-up signal is input to the A / D converter 105 via the preprocess means 104, and the digital signal is output from the A / D converter 105. Is input to the imaging signal processing circuit 303.
[0008]
The signal input to the imaging signal processing circuit 303 is temporarily stored in the memory 410 or directly input to the 401 enhancer / C SEP circuit. Then, the enhancer / C SEP circuit 401 and the delay unit 402 are used to separate the luminance signal and the color signal, and the luminance signal is enhanced and compensated for the high frequency part.
[0009]
The color signals output from the enhancer / C SEP circuit 401 are signals (Wr, Wb, Gr, Gb) obtained by adding CCD color filters (Ma (also referred to as Mg), G, Cy, Ye), respectively. The signal is converted into an RGB signal in CMTX 403.
[0010]
Next, filter processing is performed on the luminance signal Y and the color signal by the first low-pass filter 404. Thereafter, it is supplied to a white balance adjustment circuit (WB) 405. The white balance adjustment circuit (WB) 405 controls each gain of the color signals RGB according to the AWB 408.
[0011]
Reference numeral 406 denotes a gamma circuit that performs gamma processing on the luminance signal Y and the color signals RGB. The CY MTX 407 is a matrix circuit that converts RGB signals that have undergone gamma processing into color difference signals (RY) and (BY).
[0012]
The AWB 408 integrates the RGB signal or the output (RY) signal, the BY signal and the Y signal of the CY MTX 407 for a certain period, and the white balance adjustment circuit (WB) 405 converts the R signal and the B signal according to the integration value. It is a control circuit for generating a control signal for controlling the gain and performing white balance control.
[0013]
The second low-pass filter 409 is a filter that performs filter processing on the color difference signal RGB. Then, after the luminance signal Y and the color difference signals (RY) and (BY) are stored in the memory 410 for one image, the compression circuit 411 performs data compression processing. Reference numeral 412 denotes a pixel addition circuit that performs pixel addition in the vertical direction when the CCD is frame-read.
[0014]
FIG. 14 is a detailed drawing of the control circuit (AWB) 408. The color difference signal (RY, BY) is input, and the input of a large amplitude exceeding a predetermined value with respect to the amplitude of the color difference signal (RY, BY). The clip circuit is limited.
[0015]
Then, the color difference signal (RY), the color difference signal (BY), and the luminance signal Y that have passed through the clipping circuit are divided into, for example, a plurality of blocks and integrated for each divided block.
[0016]
In the coordinates formed by the color difference signals (RY) and (BY), the change range when the white color temperature changes is provided as a white discrimination range (as shown in FIG. 15), and the integrated value for each block is white on the same coordinate. The system control CPU 108 performs control to determine whether or not it is within the determination range, and to determine the gains of the R signal and the B signal using the integral value within the white determination range, and with the control value corresponding to this gain, R Adjust the gain of each / B.
[0017]
[Problems to be solved by the invention]
The white balance control according to the conventional example converges to a control state with the passage of time in order to adapt during moving image shooting. However, in order to apply the white balance control based on the conventional example in still image shooting, the image sensor is once exposed before image shooting (main exposure).
[0018]
Then, a color difference signal is generated from the exposed signal using a signal processing circuit, and the color temperature situation is detected from the position on the coordinates of the color difference signals (RY) and (BY) by the conventional white balance detection, and the adjustment value at the same color temperature is detected. After the exposure, the main exposure is performed and the gain adjustment is performed using the previously obtained adjustment value to obtain the white balance.
[0019]
In this way, it is necessary to detect the exact color temperature with a single white balance detection, but as before, coordinates are created with the color difference signals (RY) and (BY), and the white discrimination range is set on the same coordinates. When shooting a color with low brightness and when shooting white at a certain color temperature (especially the color temperature within the white discrimination range and far from the intersection of the RY and BY axes) Since both of them fall within the white discrimination range, it is difficult to specify the color temperature and the accurate color temperature cannot be detected.
[0020]
For example, as shown in FIG. 16, when the color temperature change is between α and β, white is surrounded by three points a, b, and o, as shown in FIG. Exists in the designated area. On the other hand, it is assumed that a certain color (color a) exists in a region surrounded by three points c, d, and o. As in the prior art, when white is discriminated based on the level (e) of the color difference signal (RY), (a color) in the region (F) is also included, and there is a problem that hinders color temperature detection.
Furthermore, for example, when the ratio of white in the image is low, there is a problem that the white balance is largely lost due to the influence of the other colors included in the white region.
[0021]
In view of the above-described problems, the present invention has an object to enable accurate color temperature detection independent of the luminance signal level.
[0022]
[Means for Solving the Problems]
An image pickup apparatus according to the present invention includes an image pickup element having a color filter composed of four types of Cy, Ma, Ye, and Gr having different spectral characteristics, and a plurality of pixels of the color filter set on the image pickup region of the image pickup element. A YeCy difference signal generating means for generating a YeCy difference signal representing a difference between the Ye signal and the Cy signal of the color filter at each of a plurality of sample points, and the Ma signal and the Gr signal of the color filter at the sample point MaGr difference signal generating means for generating a difference between the first and second ratio signals, a first ratio signal indicating the ratio of the YeCy difference signal and the Cy signal, and a ratio of the MaGr difference signal and the Gr signal. Coordinates formed by a ratio signal generating means for generating a ratio signal of 2, a component of the first ratio signal and a component of the second ratio signal White range determining means for determining whether or not the first ratio signal and the second ratio signal obtained for each of the sample points are included in the white range, and based on the determination of the white range determining means Among the plurality of sample points determined to be in the white range, based on illuminance information at the time of shooting, the color signals of the plurality of sample points included in the first region in the white range and the white range A weighting unit that performs different weighting on the color signals of a plurality of sample points included in a second region different from the first region and a white range based on the determination by the white range determination unit are determined to be not within the white range. In addition, with respect to the color signal of the sample point, moving means for moving the coordinates on the frame of the white range, and the number of the sample points in the white range and the sample whose coordinates have moved. Obtained from the image sensor based on the hybrid ratio according to the number of data points, the color signal determined to be within the white range and weighted, and the color signal whose coordinates have been moved by the moving means. And gain control value control means for obtaining a gain control value for controlling the level difference between the Cy signal, Ma signal, Ye signal, and Gr signal corresponding to the white region of the subject.
[0023]
Another feature of the present invention is that an image sensor having R, G, and B color filters having different spectral characteristics and a plurality of pixels of the color filter set on the image pickup area of the image sensor are respectively provided. First signal generation means for generating a first signal representing a difference between color signals respectively output from an image sensor corresponding to a predetermined two-color filter among the color filters at each of a plurality of sample points. Second signal generation for generating a second signal different from the first signal obtained by calculating the R, G, and B color signals respectively output from the imaging device at each of the sample points Means, a third signal generating means for generating a third signal from the color signal output from the image sensor, a first ratio signal representing a ratio of the first signal to the third signal, Second trust above White on the coordinates formed by the ratio signal generating means for generating the second ratio signal representing the ratio between the first ratio signal and the third signal, the component of the first ratio signal and the component of the second ratio signal A white range determining means for determining whether or not a first ratio signal and a second ratio signal obtained for each sample point fall within the range, and the white range based on the determination of the white range determining means Among the plurality of sample points determined to be within, based on illuminance information at the time of shooting, color signals of a plurality of sample points included in the first region within the white range, and the first signal within the white range The weighting unit that performs different weighting on the color signals of the plurality of sample points included in the second region different from the one region, and the above-described white range determined based on the determination of the white range determination unit Sun The hybrid ratio according to the moving means for moving the coordinates on the frame of the white range and the number of the sample points in the white range and the number of the sample points whose coordinates have moved with respect to the color signal of the white point And an R signal corresponding to the white area of the subject obtained from the image sensor based on the color signal determined to be within the white range and weighted and the color signal whose coordinates have been moved by the moving means, Gain control value control means for obtaining a gain control value for controlling the level difference between the G signal and the B signal to be suppressed.
[0024]
Another feature of the present invention is that an image sensor having color filters composed of four types of Cy, Ma, Ye, and G having different spectral characteristics and two pixels in the vertical direction from the image sensor are Wb = Ma + Cy. , Gr = G + Ye, Gb = Cy + G, Wr = Ye + Ma, and addition reading means for reading and each of a plurality of sample points each including the Wb, Gr, Gb, Wr signals on the addition read signal The first signal generating means for generating a signal represented by ((Wr−Gb) − (Wb−Gr)) from the signal output read by the addition reading means, and the addition at the sample point Second signal generating means for generating a signal represented by ((Wr−Gb) + (Wb−Gr)) from the signal output read by the reading means. Generating a first ratio signal representing a ratio of the first signal to the Wr + Gb signal and a second ratio signal representing a ratio of the second signal to the Wb + Gr signal for each sample point; And a first ratio signal and a second ratio signal obtained for each sample point within a white range on coordinates formed by the first ratio signal component and the second ratio signal component. Among the plurality of sample points determined to be within the white range based on the determination of the white range determination unit, based on the illuminance information at the time of shooting, The color signals of the plurality of sample points included in the first area within the white range are different from the color signals of the plurality of sample points included in the second area different from the first area within the white range. Weighting means for performing weighting, and moving means for moving coordinates on the frame of the white range with respect to the color signal of the sample point determined not to be within the white range based on the determination of the white range determination unit A mixture ratio according to the number of the sample points in the white range and the number of the sample points whose coordinates have moved, a color signal determined to be in the white range and weighted, and the movement Based on the color signal whose coordinates are moved by the means, a gain control value for controlling the level difference of the Cy signal, Ma signal, Ye signal, and G signal corresponding to the white area of the subject obtained from the image sensor is suppressed. Gain control value control means for obtaining
[0034]
[Action]
In order to solve the above problems, in the present invention, the gradients of the color difference signals (RY) and (BY) with respect to the luminance signal Y, that is, (RY) / Y and (BY) / Y depend on the level of the luminance signal Y. Since this is a value determined from the color temperature and the shooting color, the coordinates of (RY) / Y, (BY) / Y are used to determine the white discrimination range, and the (RY) / Y, (BY) Weighting is performed using / Y and the luminance signal Y or the exposure amount of the image sensor, and the color temperature is detected so that accurate detection independent of the luminance signal level can be performed.
[0035]
Also, by using the luminance signal Y or the exposure amount of the image sensor to weight the signal value in the white discrimination, the influence of other colors can be reduced even if the proportion of white in the image is low. Thus, a more accurate white balance signal can be obtained.
[0036]
According to another feature of the present invention, instead of the gradient (RY) / Y, (BY) / Y of the color difference signal, (Mg-G) / (Mg + G), (Ye-Cy) / ( (Ye + Cy) or (Mg-G) / (Mg + G + Cy + Ye), (Ye-Cy) / (Mg + G + Ye + Cy) Mg, G, Cy, Ye) Since the color temperature can be directly detected by simple processing from the readout signal of the image sensor with a complementary color filter, exposure and signal processing for color temperature detection before the main exposure can be omitted. This enables simple and accurate color temperature detection.
[0037]
Here, by setting the molecules to (Mg-G) and (Ye-Cy), the color temperature change direction is almost perpendicular to the (Mg-G) axis and almost parallel to the (Ye-Cy) axis. The determination range can be easily set.
For an image sensor with a (R, G, B) pure color filter, create coordinates using (RB) / (R + G + B) and (R + BG) / (R + G + B). By detecting the color temperature, exposure and signal processing for color temperature detection before the main exposure can be omitted as with an image sensor having a complementary color filter. Color temperature can be detected.
[0038]
Further, according to another feature of the present invention, (Mg-G) / G or (Mg-G) / Mg and (Ye- By detecting the color temperature using (Cy) / Cy or (Ye-Cy) / Ye, the color temperature is directly detected from the readout signal of the image sensor with the (Mg, G, Cy, Ye) color filter. Since detection is possible, exposure and signal processing for color temperature detection before the main exposure can be omitted, and simple and accurate color temperature detection is possible.
[0039]
Even when shooting a subject that contains only a small amount of white, white coordinates can be identified by giving coordinates within the white discrimination range in the form of (Mg-G) / G = a, (Ye-Cy) / Cy = b. The image sensor output can be replaced with simple processing in the form of Mg '= (1 + a) * G and Ye' = (1 + b) * Cy within the range. White balance adjustment that does not deviate significantly is possible.
[0040]
For an image sensor with a (R, G, B) pure color filter, an image with a complementary color filter can be obtained by creating coordinates using (RB) / G and (R + BG) / G to detect the color temperature. Like the device, exposure and signal processing for color temperature detection before the main exposure can be omitted, and accurate color temperature detection can be performed with simple processing from the image sensor output signal, and a small amount of white can be detected. Even when shooting a subject that only contains R ′ = (1 in the white discrimination range by giving coordinates in the white discrimination range in the form of (RB) / G = a, (R + BG) / G = b + a + b) * G / 2, B '= (1-a + b) * G / 2 can replace the image sensor output with simple processing, and white balance is maintained even under the above shooting conditions. This makes it possible to adjust the white balance without significant deviation.
[0041]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of an imaging apparatus of the present invention will be described with reference to the drawings.
1 and 2 are flowcharts for explaining the operation of the image pickup apparatus according to the present embodiment, and FIG. 3 is a block diagram showing the signal processing flow of the electronic camera according to the first embodiment of the image pickup apparatus of the present invention. It is.
[0042]
Since the image sensor in the present embodiment performs frame readout, output signals M, G, C, and Y from the image sensor come out according to the filter attached to the image sensor.
First, each filter output in the optical black region is integrated by the integrator 31, and the integrated value is given to the black level adjusting circuit 32 to adjust the black level.
[0043]
Next, white balance detection is performed in the white balance detection circuit 33, and the output from which white balance is detected is given to the gain adjustment circuit 34 so that the output gains of Ma, Gr, Cy, and Ye at the white point are the same. Gain is adjusted.
[0044]
On the other hand, the luminance signal (Y) is subjected to aperture correction after passing through the low-pass filter 35 in order to remove moire caused by the color filter. Here, the high-pass filters 36a and 36b are respectively passed in the horizontal and vertical directions.
[0045]
Thereafter, the gain is adjusted by the gain adjusting circuits 37a and 37b. Thereafter, in the amplitude adjustment circuits 38a and 38b, processing is performed to increase the amplitude of those with a small aperture signal, and Peak To reduce the amplitude of those with a large amplitude. Peak processing is performed.
[0046]
Next, the base clip (BC) circuits 39a and 39b are subjected to a process of cutting a signal having a low noise level, and then the adder circuit 40 adds horizontal and vertical. Further, after gain adjustment is performed in the gain adjustment circuit 41, it is added to the luminance component (Yn) in the addition circuit 42.
[0047]
In order to generate the color difference signals (RY, BY), first, the four-color signals M, G, C, Y are synchronized by the synchronization circuit 50.
Next, the pixel interpolation circuit 51 performs pixel interpolation so that the four color signals M, G, C, and Y are assigned to the entire area of the imaging surface.
[0048]
Next, the 4 × 3 first matrix circuit 52 is used to convert the MGYS signal into an RGB signal. At this time, the first matrix circuit 52 uses a matrix that determines the color temperature of the subject from the white balance information and provides the best color reproduction at the color temperature.
[0049]
Next, in the second matrix circuit 53, the RGB signal is converted into the (RY) signal and the (BY) signal, and the low-pass filter process for reducing moire is performed through the low-pass filter 54. Next, the color processing circuit 55 performs saturation achromatic processing with reference to the Y signal.
[0050]
The luminance signal (Y) and the color difference signals (RY, BY) produced in this way are converted into RGB signals by the color decoder 56, and then subjected to gamma correction processing in the gamma processing circuit 57, and then recorded. Is done.
[0051]
Next, white balance processing, which is the main point of the present embodiment, will be described in detail. When the CCD complementary color filters are arranged as shown in FIG. 4, the whole arrangement is formed by repeating this unit block with the 8-pixel arrangement of FIG. 4 as a unit block.
[0052]
These eight pixel outputs are Ma1, Ma2, Gr1, Gr2, Cy1, Cy2, Ye1, and Ye2 in accordance with the respective color filters. The output of the same color filter is divided into two colors (Ma1 and Ma2, Gr1 and Gr2, Cy1 and Cy2, Ye1 and Ye2). This is because they are different.
[0053]
Hereinafter, the white balance processing will be described in detail with reference to FIGS.
First, in the first step S11, sample data is extracted for each sample.
[0054]
Next, in step S12, it is determined whether or not the sample data extracted in step S11 is valid as a WB sample point. As a result of the determination, if the extracted sample data is valid as a WB sample point, the process proceeds to step S13, and if not valid, the process restarts from step S11.
[0055]
Step S13 determines whether or not all of the extracted 8 sample point pixels have brightness that can be calculated as WB data, and is performed using the following equation.
DarkThreshold <PixelValue <BrightThreshold
here,
DarkThreshold: A predetermined value
BrightThreshold:-"-
PixelValue: 8 pixel sample points
[0056]
As a result of the determination in step S13, if all of the eight pixels of the sample point satisfy the following expression, the process proceeds to step S14. Try again.
[0057]
In step S14, an edge threshold value is determined. This checks whether all eight pixels of the sample point satisfy the following expression.
EdgeThreshold> | Ma1-Ma2 |
EdgeThreshold> | Gr1-Gr2 |
EdgeThreshold> | Cy1-Cy2 |
EdgeThreshold> ｜ Ye1-Ye2 ｜
here,
EdgeThreshol: A predetermined value.
[0058]
If it is determined in step S14 that all eight pixels are satisfied, the process proceeds to step S15. If even one of the pixels is not satisfied, it is determined that the sample point includes an edge, and the process proceeds to the next sample point. Redo from step S11.
[0059]
In step S15, at each sample point, (X1, Y1, X2, Y2) is determined as the values of the following equations (1) to (4).
X1 = (Ye1-Cy1) / Cy1 (1)
Y1 = (Ma1-Gr1) / Gr1 (2)
X2 = (Ye2-Cy2) / Cy2 (3)
Y2 = (Ma2-Gr2) / Gr2 (4)
For an image sensor with a (R, G, B) pure color filter, create coordinates using (RB) / (R + G + B) and (R + BG) / (R + G + B). By detecting the color temperature, exposure and signal processing for color temperature detection before the main exposure can be omitted as with an image sensor having a complementary color filter. Color temperature can be detected.
For an image sensor with a (R, G, B) pure color filter, an image with a complementary color filter can be obtained by creating coordinates using (RB) / G and (R + BG) / G to detect the color temperature. Like the device, exposure and signal processing for color temperature detection before the main exposure can be omitted, and accurate color temperature detection can be performed with simple processing from the image sensor output signal, and a small amount of white can be detected. Even when shooting a subject that only contains R ′ = (1 in the white discrimination range by giving coordinates in the white discrimination range in the form of (RB) / G = a, (R + BG) / G = b + a + b) * G / 2, B '= (1-a + b) * G / 2 can replace the image sensor output with simple processing, and white balance is maintained even under the above shooting conditions. This makes it possible to adjust the white balance without significant deviation.
[0060]
Next, in step S16, as shown in FIG. 5, taking (Ye-Cy) / Cy as the X axis and (Ma-Gr) / Gr as the Y axis, equations (1) to (4) It is determined whether or not it is within a predetermined white area provided in the.
[0061]
If the result of this determination is within a predetermined white area, that is, the following conditional expression:
a <X1 <b, c <Y1 <d and a <X2 <b, c <Y2 <d (5)
If the condition is satisfied, the sample point is regarded as valid as data for setting the white balance WB, and the process proceeds to step S17. If the above equation (5) is not satisfied, the data is regarded as invalid and the process proceeds to step S27.
[0062]
In step S17, first, the average value and the luminance value of the XY axis values of the sample points in the WB coordinates are obtained at the sample points using the following equations.
AverageX = (X1 + X2) / 2
AverageY = (Y1 + Y2) / 2
AverageBrightness = (Ma1 + Ma2 + Gr1 + Gr2 + Cy1 + Cy2 + Ye1 + Ye2) / 8
[0063]
Next, in step S18, it is determined whether or not the sample data is valid data for the first white balance WB setting. If the sample point is the first sample point regarded as valid data for setting, the process proceeds to step S19, and if not, the process proceeds to step S21.
[0064]
In step S19, a sample point group having values as parameters as shown in FIG. 6 is created, and the value obtained in step S17 and the pixel value of the same point are substituted. That is, the following calculation is performed.
GpX = AverageX
GpY = AverageY
GpBrightness = AverageBrightness
GpMa1 = Ma1
GpMa2 = Ma2
GpGr1 = Gr1
GpGr2 = Gr2
GpCy1 = Cy1
GpCy2 = Cy2
GpYe1 = Ye1
GpYe2 = Ye2
GpNumber = 1
GpWeight = 1
[0065]
Then, it progresses to step S20 and it is judged whether extraction of all the sample data was completed. If extraction of all sample data has not been completed, the process returns to step S11 and the above-described processing is repeated.
[0066]
On the other hand, as a result of the determination in step S18, if the sample point is not the first sample point regarded as the set valid data, the process proceeds to step S21 to determine whether or not it is included in the sample point group.
[0067]
As a result of the above determination, if it is included, the process proceeds to step S22, and processing for inclusion in the sample point group is performed.
If the result of determination in step S21 is that the sample point group is not included, the process proceeds to step S23 to determine whether or not the sample point group has ended. If not, the process returns to step S21 and the sample point group has ended. Then, the process proceeds to step S19.
[0068]
If the processing of step S21 to step S22 described above is described using mathematical expressions, if the sample point is not the first sample point considered as WB setting valid data, GpX, GpY, GpBrightness of the existing sample point group When the following conditional expression is satisfied, the sample point is determined to be included in the sample point group, and is included in the sample point group.
[0069]

[0070]
Also, if it cannot be included in all the currently created sample point groups, a new sample point group based on the sample points is created.
[0071]
When the processing from step S17 to step S19 is completed for all the sample points, a true white group is searched from one or a plurality of sample point groups created in step S17. Do not know which is white.
[0072]
Therefore, first, in step S24 of FIG. 2, the sample point groups are sorted in descending order of luminance.
Next, assuming that the one with high luminance is white, the luminance information GpBrightness of the plurality of sample point groups is used, and the samples from the one with the highest luminance to the Nth are grouped as having a high probability of white. Weight point groups. Weighting is performed by the following formula (step S25, step S26).
[0073]

[0074]
On the other hand, if it is determined in step S16 that the white point (X1, Y1) (X2, Y2) obtained in step S15 and expressed in coordinates is not within the predetermined range, the process proceeds to step S27 in FIG. Move the same coordinate point on the predetermined range frame as follows, and from the coordinates (X ', Y') on the range frame, use the following formula to change the color filter output Ma1 ', Ma2', Ye1 ', Ye2' Ask for a new one.
Ma1 '= (1 + Y') * Gr1
Ma2 '= (1 + Y') * Gr2
Ye1 '= (1 + X') * Cy1
Ye2 '= (1 + X') * Cy2
[0075]
However, it is assumed that the coordinate point is moved to a coordinate on a predetermined range frame closest to the coordinate point as shown in FIG. This coordinate movement processing is called color clip processing. In step S28, the total number of sample points subjected to the color clip processing is counted as SN2. Next, the process proceeds to step S29, and the integration values obtained by the color clip are set to Ma1 ′, Ma2 ′, Gr1 ′, Gr2 ′, Cy1 ′, Cy2 ′, Ye1 ′, Ye2 ′.
[0076]
When the weighting is completed, the process proceeds to step S30. All sample point groups obtained in steps S17 to S19 and steps S25 to S26 are weighted using the following formula, and within the white discrimination range. Find the integral values (SMa1, SMa2, SGr1, SGr2, SCy1, SCy2, SYe1, SYe2).
[0077]
SMa1 = SMa1 + GpMa1 * GpWeight
SMa2 = SMa2 + GpMa2 * GpWeight
SGr1 = SGr1 + GpGr1 * GpWeight
SGr2 = SGr2 + GpGr2 * GpWeight
SCy1 = SCy1 + GpCy1 * GpWeight
SCy2 = SCy2 + GpCy2 * GpWeight
SYe1 = SYe1 + GpYe1 * GpWeight
SYe2 = SYe2 + GpYe2 * GpWeight
[0078]
Next, in step S31, the number of sample points SN1 considered to be white at the same time is obtained.
SN1 = SN1 + GpNumber
[0079]
Next, in step S32, it is determined whether or not the processing has been executed for all the sample point groups. If not, the processing returns to step S30. If it has been executed for all, the process proceeds to step S33.
[0080]
In step S33, it is determined whether or not the number of samples (SN1 + SN2) is 0 or more than a predetermined value. If not, the process proceeds to step S34, where a predetermined color is used as white balance adjustment data. The white balance adjustment value of temperature (for example, 5000K) is used.
[0081]
If the number of samples (SN1 + SN2) is greater than or equal to the predetermined value as a result of the determination in step S33, the process proceeds to step S35, where the average luminance level during white shooting is
SY = (SMa1 + SMa2 + SGr1 + SGr2 + SCy1 + SCy2 + SYe1 + SYe2) / 8
And the ratio of the integrated value of each color to the average luminance level is obtained.
gmg1 = SY / (2 * SMa1) gmg2 = SY / (2 * SMa2)
gg1 = SY / (2 * SGr1) gg2 = SY / (2 * SGr2)
gcy1 = SY / (2 * SCy1) gcy2 = SY / (2 * SCy2)
gye1 = SY / (2 * SYe1) gye2 = SY / (2 * SYe2)
[0082]
Next, the process proceeds to step S36, where the average luminance level at the time of clipping is
SY '= (SMa1' + SMa2 '+ SGr1' + SGr2 '+ SCy1' + SCy2 '+ SYe1' + SYe2 '/ 8
And the ratio of the integrated value of each color to the average luminance level is obtained.
gmg1 '= SY' / (2 * SMa1 ') gmg2' = SY '/ (2 * SMa2')
gg1 '= SY' / (2 * SGr1 ') gg2' = SY '/ (2 * SGr2')
gcy1 '= SY' / (2 * SCy1 ') gcy2' = ST '/ (2 * SCy2')
gye1 '= SY' / (2 * SYe1 ') gye2' = SY '/ (2 * SYe2')
[0083]
Next, the process proceeds to step S37, and white balance adjustment is performed by using the ratio of the integrated values of the respective colors obtained in steps S35 and S36, the integrated sample number SN1 within the white discrimination range, and the integrated sample number SN2 by the color clip. Create data (wmg1, wg1, wcy1, wye1, wmg2, wg2, wcy2, wye2).
[0084]
For example,

Apply the following process.
[0085]
These values are stored as white balance adjustment data.
By applying the white balance adjustment data obtained by the above processing to the signal value for each pixel obtained from the image sensor, white balance can be obtained.
[0086]
As described above, in the present embodiment, coordinates for white balance are created using signal values from the image sensor, and white ones have relatively higher luminance values than sample points that have entered the white region at those coordinates. Weighting is performed based on the assumption.
[0087]
Thereafter, the integration processing is performed at the sample points, so that even when there are few whites in the image, it is possible to avoid the white balance from being greatly affected by other colors.
[0088]
(Second Embodiment)
Next, a second embodiment of the present invention will be described. In the first embodiment described above, the signal reading of the electronic camera is frame reading, but in the second embodiment, the signal of the electronic camera is read by field reading.
[0089]
Since the image sensor in the present embodiment performs field readout, the output from the image sensor is read by adding two pixels in the vertical direction as shown in FIG. In FIG. 9,
Wb = Ma + Cy
Gr = G + Ye
Gb = Cy + G
Wr = Ye + Ma
It is.
[0090]
In the case of the present embodiment, assuming that the color filter is changed from (Ma, g, Cy, Ye) to (Wb, Gr, Gb, Wr), the same processing as the above-described frame reading is performed. Since the hardware configuration is the same as that in FIG. 3 in the first embodiment described above, the illustration is omitted.
[0091]
In the present embodiment, first, the black level is adjusted using the integrated value of each filter output in the optical black region.
Next, white balance processing is performed so that the output gains of Wb, Gr, Gb, and Wr at the white point are the same.
[0092]
In order to create the luminance signal (Y), aperture correction is performed after a horizontal low-pass filter is applied to remove moire due to the color filter. Here, a high-pass filter is applied horizontally and vertically to adjust the gain, so that the CPU becomes larger when the amplitude of the aperture signal is small, and the peak becomes small when the amplitude is large. To Peak processing is performed, horizontal and vertical are added, gain is further applied, and added to the luminance component (Yn).
[0093]
In order to generate the color difference signals (RY, BY), first, the signals of the four colors are synchronized by the synchronization circuit, and pixel interpolation is performed so that the four color signals are allocated to the imaging surface area.
Next, the Wb signal, Gr signal, Gb signal, and Wr signal are converted into RGB signals using a 4 × 3 matrix circuit. At this time, the matrix circuit is used such that the color temperature of the subject is determined from the white balance information and the color reproduction at the color temperature is the best.
[0094]
Next, the RGB signals are converted into RY and BY signals, a low-pass filter for moire reduction is applied, and saturation achromatic processing is performed with reference to the Y signals.
The luminance signal (Y) and the color difference signals (RY, BY) produced in this way are converted into RGB signals, recorded after being subjected to gamma correction processing.
[0095]
Next, white balance processing, which is the main point of the present embodiment, will be described in detail. When the CCD complementary color filters are arranged as shown in FIG. 9A, the whole arrangement is formed by repeating this unit block with the 8-pixel arrangement shown in the figure as a unit block. Here, in order to perform field reading, a signal as shown in FIG. 9B is read in one field.
[0096]
Next, the processing procedure of the second embodiment will be described with reference to FIGS.
First, in the first step S61, sample data is extracted for each sample.
[0097]
Next, in step S62, it is determined whether or not the sample data extracted in step S61 is valid as a WB sample point. As a result of the determination, if the extracted sample data is valid as a WB sample point, the process proceeds to step S63, and if not valid, the process restarts from step S61.
[0098]
In step S63, it is checked whether the sample point has brightness that can be calculated as WB data. If all four pixels (Wb, Gr, Gb, Wr) of the sample point satisfy the following formula, the next step is performed. Proceeding to S64, if even one is not satisfied, the process moves to the next sample point and starts again from step 1.
DarkThreshold <PixelValue <BrightThreshold
[0099]
Next, in step S64, the value of the following equation is obtained at the sample point.
X1 = ((Wr-Gb)-(Wb-Gr)) / (Wr + Gb) (6)
Y1 = ((Wr-Gb) + (Wb-Gr)) / (Wb + Gr) (7)
[0100]
In step S65, it is determined whether the data is valid as WB balance setting data. That is, as shown in FIG. 5, ((Wr−Gb) − (Wb−Gr)) / (Wr + Gb) is represented by the X axis, ((Wr−Gb) + (Wb−Gr)) / (Wb + Gr) is taken on the Y axis, and the above equations (6) and (7) are within a predetermined region provided on the X and Y axes, that is, the following conditional expression:
a <X1 <b, c <Y1 <d (8)
If the condition is satisfied, the sample point is regarded as valid as data for setting the WB, and the process proceeds to step S66. If the above equation (8) is not satisfied, the process proceeds to step S76.
[0101]
In step S66, the average value and the luminance value of the XY axis values of the sample points in the WB coordinates are obtained at the sample points using the following equations.
AverageX = X1
AverageY = Y1
AverageBrightness = (Wb + Gr + Gb + Wr) / 2
[0102]
In step S67, it is determined whether the sample point is the first WB setting valid data. As a result of the above determination, if the sample point is the first sample point regarded as the WB setting valid data, the process proceeds to step S68.
[0103]
In step S68, a sample point group having values as shown in FIG. 6 as parameters is created, and the value obtained in step S66 and the pixel value of the same point are substituted.
GpX = AverageX
GpY = AverageY
GpBrightness = AverageBrightness
GpWb = Wb
GpGr = Gr
GpGb = Gb
GpWr = Wr
GpNumber = 1
GpWeight = 1
[0104]
If the sample point is not the first sample point regarded as the WB setting valid data, the process proceeds to step S69 to determine whether or not the sample point is included in the sample point group.
[0105]
The above determination uses the values of GpX, GpY, and GpBrightness of an existing sample point group and determines that the sample point is included in the sample point group when the following conditional expression is satisfied. In this case, the process proceeds to step S70 and is included in the sample point group.
[0106]
That is,

[0107]
If the sample point group cannot be included, the process proceeds to step S71 to determine whether the sample point group is over. As a result of the determination, if it is not the end, the process returns to step S69, and if it is the end, the process proceeds to step S68 to create a new sample point group based on the sample point as described above.
[0108]
When the process of step S68 or step S70 is completed, the process proceeds to step S72, where it is determined whether all sample data extraction has been completed. If the result of the determination is that the processing has not ended, the process returns to step S61 and the above-described processing is repeated.
[0109]
If the extraction of all sample data has been completed, the processing of steps S73 to S75 in FIG. 8 is performed next.
This is a process of searching for a true white group from one or a plurality of sample point groups created in step 67 to step S71 when the process of step S67 to step S71 is completed for all sample points. I do.
[0110]
But the real white group doesn't really know which one is white. Therefore, assuming that the one with high luminance is white, the luminance information GpBrightness of the plurality of sample point groups is used, and from the one with the highest luminance to the Nth are regarded as the real white group, and the weight of the sample points is set. I do. That is, the processing from step S73 to step S75 is sequentially performed.
[0111]
The weighting process is performed by the following formula.
GpWeight = GpWeight * 2
However, N is 1 or more, and is obtained by the following equation, for example.
[0112]

[0113]
On the other hand, if it is determined in step S65 that the white points (X1, Y1) and (X2, Y2) represented in the coordinates obtained in step S64 are not within the predetermined range, the coordinate points are moved onto the predetermined range frame. Then, Wb ′ and Wr ′ are newly obtained again (step S76).
[0114]
Next, in step S77, the total number of sample points clipped in the above color is set to SN2, and in step S78, the integral values obtained by the color clip are set as integral values for each color. The process to add to is performed.
[0115]
After the weighting is completed, the process proceeds to step S79, and the integration within the white discrimination range is performed for all sample point groups obtained in steps S67 to S71 and S74 to S75 using the following expression. The values (SWb, SGr, SGb, SWr) are obtained.
SWb = SWb + GpWb * GpWeight
SGr = SGr + GpGr * GpWeight
SGb = SGb + GpGb * GpWeight
SWr = SWr + GpWr * GpWeight
[0116]
Next, the process proceeds to step S80, and the number of sample points SN1 regarded as white at the same time is obtained.
SN1 = SN1 + GpNumber
[0117]
Next, in step S81, it is determined whether or not the processing has been executed for all the sample point groups. If not, the processing returns to step S79 and the above-described operation is repeated.
[0118]
If it is executed for all sample point groups, the process proceeds to step S82, and it is determined whether or not the number of samples (SN1 + SN2) is 0 or greater than a predetermined value.
[0119]
If it is determined in step S82 that the number of samples (SN1 + SN2) is not greater than a predetermined value, the process proceeds to step S83, and a white balance adjustment value of a predetermined color temperature (for example, 5000K) is used as white balance adjustment data.
[0120]
On the other hand, if it is determined in step S82 that the number of samples (SN1 + SN2) is equal to or greater than a predetermined value, the process proceeds to step S84, where the average luminance level during white photography is
SY = (SWb + SGr + SGb + SWr) / 4
And the ratio of the integrated value of each color to the average luminance level is obtained.
gwb = SY / (2 * SWb)
ggr = SY / (2 * SGr)
ggb = SY / (2 * SGb)
gwr = SY / (2 * SWr)
[0121]
Next, the process proceeds to step S85, where the average luminance level at the time of clipping is
SY '= (SWb' + SGr '+ SGb' + SWr ') / 4
And the ratio of the integrated value of each color to the average luminance level is obtained.
gwb '= SY' / (2 * SWb ')
ggr '= SY' / (2 * SGr ')
ggb '= SY' / (2 * SGb ')
gwr '= SY' / (2 * SWr ')
[0122]
Next, the process proceeds to step S86, and white balance adjustment data (from the ratio of the integral values of the respective colors obtained in steps S84 and S85, the integral sample number SN1 within the white discrimination range, and the integral sample number SN2 by the color clip) wwb, wgr, wgb, wwr).
[0123]
For example,

Apply the following process. These values are stored as white balance adjustment data.
[0124]
The white balance can be obtained by applying the white balance adjustment data obtained by the above processing to the signal value for each pixel obtained from the image sensor.
[0125]
(Third embodiment)
FIG. 10 is a part of a flowchart for explaining a part of the white balance processing procedure according to the third embodiment of the present invention, and corresponds to steps S25 to S26 in FIG. This part is a part for weighting the group after the white balance group is created, but this embodiment is characterized in that illuminance information at the time of photographing is used for this weighting.
[0126]
The illuminance information is, for example, an EV value, and the processes of steps S91 to S95 are performed. That is, first, an EV value is obtained as information from the camera side, and when the EV value is large, it is often taken outside (sunlight), so the white coordinate range is limited as shown in FIG. .
[0127]
On the contrary, the EV value is limited to sunset or artificial light, and the white coordinate range is limited as shown in FIG. Here, weighting is performed on all sample point groups as follows. That is, the weight of the white balance group in the limited coordinate range is increased.
[0128]

[0129]
By performing such weighting, more preferable white balance can be performed even when white brightness is low. Even if the limited coordinate range enters sample points that are not originally white and there are many sample point groups, white group selection by weighting is not performed, so that the white balance is not greatly changed.
[0130]
Moreover, you may use the weighting in this embodiment for step S74-step S75 in 2nd Embodiment. Moreover, you may use in combination with the weighting method of 1st Embodiment and 2nd Embodiment.
[0131]
【The invention's effect】
As described above, the present invention performs more accurate color discrimination by using a coordinate axis different from conventional coordinates for white discrimination, and also uses the luminance signal Y or the exposure amount of the image sensor to detect the white discrimination. By weighting the signal value of, for example, even if the proportion of white in the image is low, the influence of other colors can be reduced, so that the white balance is not greatly disturbed and more accurate white A balance signal can be obtained.
[0132]
Further, according to the second aspect of the present invention, when the image sensor is driven for field readout, the same effect as the first aspect can be obtained, and a good white balance adjustment value can be obtained.
[Brief description of the drawings]
FIG. 1 is a flowchart showing processing of a first embodiment of the present invention.
FIG. 2 is a flowchart showing processing of the first embodiment.
FIG. 3 is a block diagram for explaining a state of signal processing of an electronic camera to which the present invention is applied.
FIG. 4 is a diagram illustrating an example of a color filter pattern of an image sensor.
FIG. 5 is a diagram illustrating a concept of white balance.
FIG. 6 is a diagram illustrating parameters of a sample point group.
FIG. 7 is a flowchart showing processing of the second embodiment.
FIG. 8 is a flowchart showing processing of the second embodiment.
FIG. 9 is a diagram illustrating a state of field reading of the image sensor.
FIG. 10 is a flowchart showing processing of the third embodiment.
FIG. 11 is a diagram showing a weighting area of white balance coordinates in the third embodiment.
FIG. 12 is a block diagram illustrating an example of a conventional electronic camera.
13 is a block diagram showing a configuration of a signal processing circuit of the electronic camera of FIG.
FIG. 14 is a block diagram showing a configuration of a white balance control circuit.
FIG. 15 is a diagram illustrating a white discrimination range.
FIG. 16 is a diagram illustrating a relationship between a color difference signal and a luminance signal.
[Explanation of symbols]
31 integrator
32 Black level adjustment circuit
33 White balance detection circuit
34 Gain adjustment circuit
35 Low-pass filter
36a, 36b High pass filter
37a, 37b Gain adjustment circuit
38a, 38b Amplitude adjustment circuit
39a, 39b Base clip circuit
40 Adder circuit
41 Gain adjustment circuit
42 Adder circuit
50 Synchronization circuit
51 pixel interpolation circuit
52 First matrix circuit
53 Second matrix circuit
54 Low-pass filter
55 color processing circuit
56 color decoder
57 Gamma processing circuit

Claims (9)

An imaging device having a color filter composed of four types of Cy, Ma, Ye, and Gr having different spectral characteristics;
A YeCy difference that generates a YeCy difference signal representing a difference between the Ye signal and the Cy signal of the color filter at each of a plurality of sample points each including a plurality of pixels of the color filter set on the imaging region of the imaging element. Signal generating means;
MaGr difference signal generating means for generating a MaGr difference signal representing a difference between the Ma signal and the Gr signal of the color filter at the sample point;
A ratio signal generating means for generating a first ratio signal representing a ratio between the YeCy difference signal and the Cy signal, and a second ratio signal representing a ratio between the MaGr difference signal and the Gr signal;
The first ratio signal and the second ratio signal obtained for each sample point are within the white range on the coordinates formed by the first ratio signal component and the second ratio signal component. White range determining means for determining whether or not,
Among the plurality of sample points determined to be within the white range based on the determination by the white range determination means, a plurality of samples included in the first region within the white range based on illuminance information at the time of shooting Weighting means for performing different weighting on the color signal of a point and the color signals of a plurality of sample points included in a second region different from the first region in the white range;
Moving means for moving coordinates on the frame of the white range with respect to the color signal of the sample point determined not to be within the white range based on the determination of the white range determination unit;
The hybrid ratio according to the number of sample points in the white range and the number of sample points whose coordinates have moved, the color signal determined to be within the white range and weighted, and the moving means Based on the color signal whose coordinates are moved, a gain control value is obtained for controlling the level difference among the Cy signal, Ma signal, Ye signal, and Gr signal corresponding to the white area of the subject obtained from the image sensor. An imaging apparatus comprising: a gain control value control unit.   The weighting means performs weighting based on the luminance by obtaining luminance at the sample point and multiplying each color signal of the sample point by a predetermined number based on the luminance. Imaging device.   The imaging apparatus according to claim 1, wherein the weighting unit acquires an EV value indicating an exposure amount of the imaging element at the time of shooting and performs weighting based on the EV value. An image sensor having R, G, and B color filters having different spectral characteristics;
At each of a plurality of sample points each including a plurality of pixels of the color filter set on the imaging region of the image sensor, colors output from the image sensor corresponding to a predetermined two-color filter among the color filters First signal generating means for generating a first signal representing a signal difference;
Second signal generating means for generating a second signal different from the first signal obtained by calculating the R, G, and B color signals respectively output from the image sensor at each of the sample points. When,
Third signal generating means for generating a third signal from the color signal output from the image sensor;
Ratio signal generating means for generating a first ratio signal representing a ratio between the first signal and the third signal and a second ratio signal representing a ratio between the second signal and the third signal. When,
The first ratio signal and the second ratio signal obtained for each sample point are within the white range on the coordinates formed by the first ratio signal component and the second ratio signal component. White range determining means for determining whether or not,
Among the plurality of sample points determined to be within the white range based on the determination by the white range determination means, a plurality of samples included in the first region within the white range based on illuminance information at the time of shooting Weighting means for performing different weighting on the color signal of a point and the color signals of a plurality of sample points included in a second region different from the first region in the white range;
Moving means for moving coordinates on the frame of the white range with respect to the color signal of the sample point determined not to be within the white range based on the determination of the white range determination unit;
The hybrid ratio according to the number of sample points in the white range and the number of sample points whose coordinates have moved, the color signal determined to be within the white range and weighted, and the moving means A gain control value for obtaining a gain control value for controlling the level difference between the R signal, the G signal, and the B signal corresponding to the white area of the subject obtained from the image sensor based on the color signal whose coordinates are moved And an imaging device.   The imaging apparatus according to claim 4, wherein the weighting unit obtains luminance information at the sample point and performs weighting based on the luminance information.   5. The imaging apparatus according to claim 4, wherein the weighting unit obtains an EV value indicating an exposure amount of the image sensor at the time of photographing, and performs weighting based on the EV value. The first signal generating means generates a first signal from the difference between the R signal and the B signal,
The second signal generating means generates a second signal by summing the R signal, B signal, and G signal,
The imaging apparatus according to claim 4, wherein the third signal generating unit generates the third signal from the R signal, the B signal, and the G signal. The first signal generating means generates a first signal from the difference between the R signal and the B signal,
The second signal generating means generates a second signal by summing the R signal, B signal, and G signal,
The imaging apparatus according to claim 4, wherein the third signal generating unit uses the G signal as the third signal. An imaging device having a color filter composed of four types of Cy, Ma, Ye, and G having different spectral characteristics;
Addition reading means for adding and reading two pixels in the vertical direction from the image sensor as Wb = Ma + Cy, Gr = G + Ye, Gb = Cy + G, Wr = Ye + Ma;
At each of a plurality of sample points each including the Wb, Gr, Gb, and Wr signals on the addition read signal, ((Wr−Gb) − (Wb−) from the signal output read by the addition reading means. First signal generating means for generating a signal represented by Gr));
Second signal generation means for generating a signal represented by ((Wr−Gb) + (Wb−Gr)) from the signal output read by the addition reading means at the sample point;
Ratio signal generating means for generating, for each sample point, a first ratio signal that represents the ratio between the first signal and the Wr + Gb signal, and a second ratio signal that represents the ratio between the second signal and the Wb + Gr signal. When,
The first ratio signal and the second ratio signal obtained for each sample point are within the white range on the coordinates formed by the first ratio signal component and the second ratio signal component. White range determining means for determining whether or not,
Among the plurality of sample points determined to be within the white range based on the determination by the white range determination means, a plurality of samples included in the first region within the white range based on illuminance information at the time of shooting Weighting means for performing different weighting on the color signal of a point and the color signals of a plurality of sample points included in a second region different from the first region in the white range;
Moving means for moving coordinates on the frame of the white range with respect to the color signal of the sample point determined not to be within the white range based on the determination of the white range determination unit;
The hybrid ratio according to the number of sample points in the white range and the number of sample points whose coordinates have moved, the color signal determined to be within the white range and weighted, and the moving means Based on the color signal whose coordinates have been moved, a gain control value for controlling the level difference of the Cy signal, Ma signal, Ye signal, and G signal corresponding to the white region of the subject obtained from the image sensor is obtained. An imaging apparatus comprising: a gain control value control unit.

Images