JP3631022B2 - Imaging apparatus and signal processing apparatus - Google Patents

Description

ｗｍｇ１＝ｐ ＊ｇｍｇ１＋（１−ｐ）＊ｇｍｇ２
ｗｃｙ１ ＝ｐ ＊ｇｃｙ ＋（１−ｐ）＊ｇｃｙ２
ｗｙｅ１ ＝ｐ ＊ｇｙｅ１ ＋（１−ｐ）＊ｇｙｅ２
（ＳＮ１／（ＳＮ１ ＋ＳＮ２ ））≦βの場合
ｗｍｇ１＝ｇｍｇ２
ｗｇ１ ＝ｇｇ２
ｗｃｙ１＝ｇｃｙ２
ｗｙｅ１＝ｇｙｅ２
（ただし、０＜β＜α＜１）
そして、これらの値をホワイトバランス調整データとして記憶する。
【０１０１】
以上のようにして、撮影条件と被写体条件により白判別範囲を変化させることにより、実際の色温度と大幅に異なる白判別範囲内の信号を検出することがなくなり、より正確なホワイトバランス信号を得ることが可能となる。
【０１０２】
また、（１）で定められた白判別範囲に入るサンプルポイントの数に応じて第１、第２の利得制御値を用いてホワイトバランス調整データを求める演算方法を変えているので、大きくホワイトバランスがくずれることがない。
【０１０３】
また、本実施例では図２に示すような補色センサーを用いる場合について説明したが、純色センサーの場合でも実施可能であり、その場合、例えば色評価値をＲ−Ｙ，Ｂ−Ｙで設定すれば良い。
【０１０４】
また、本実施例では、白判別領域の領域幅を自動で決定したが、図９に示すようにマニュアルで設定しても良い。例えば、白判別範囲を、太陽光の場合は図９の（ａ），蛍光燈の場合は図９の（ｂ），タングステン光の場合は図９の（ｃ），フラッシュ光の場合は図９の（ｄ）のように限定する。
【０１０５】
（第２の実施の形態）
第２の実施の形態におけるホワイトバランス処理は、上述の第１の実施例における白判別範囲を図５に示すように色温度別に定められた白判別領域に分割し、撮影条件と被写体条件により適切な白判別領域を選択し、その領域内に存在する色評価値を有するサンプルポイントおよび１画面をｍ個に分割して得られた領域に基づいてホワイトバランス制御する点を特徴としている。また、色温度別に補色純色変換マトリクスを備え、上記選択された白判別領域内の色評価値を平均化することにより色温度を算出し、複数のマトリクスの中から色温度に対応する上記マトリクスを選択する構成である。
【０１０６】
以下に、ホワイトバランスの処理の流れを説明するが、第２の利得制御値を求める手法は第１の実施例と同様であるため説明は省略する。
【０１０７】
（１）予め白判別範囲を図５に示すように約８０００Ｋから２０００Ｋまで、１０００Ｋ毎に６個の白判別領域に分割する。
【０１０８】
最適な色再現の画像を得るためには、被写体の色温度によって補色−純色変換用のマトリクスを変化させることは必須である。よって予め、最適な色再現を達成するために、照明光の色温度範囲内での必要な色変換マトリクス数を検討しておく。そして、個々のマトリクスで再生する色温度範囲に対応して白判別範囲を分割する。
【０１０９】
（２）第１の利得制御値を求めるために、図１２に示すサンプルポイントがホワイトバランスデータとして計算できる明るさがあるかどうかを調べる（第１の実施例と同じ）。
【０１１０】
（３）サンプルポイントにおいて、色評価値ｗＸ，ｗＹを求める。
【０１１１】
ｗＸ＝（Ｙｅ−Ｃｙ）／Ｙｌｏｗ （１）
ｗＹ＝（Ｍａ−Ｇ）／Ｙｌｏｗ （２）
但しＹｌｏｗ＝（Ｍａ＋Ｇ＋Ｃｙ＋Ｙｅ）／４
【０１１２】
（４）（３）における色評価値を、予め設定した白判別領域の中で該当する白判別領域の加算レジスタに加える。
【０１１３】
ｒｅｇ１Ｍａ〔Ｋ 〕＝ｒｅｇ１Ｍａ＋Ｍａ
ｒｅｇ１Ｇ 〔Ｋ 〕＝ｒｅｇ１Ｇ ＋Ｇ
ｒｅｇ１Ｃｙ〔Ｋ 〕＝ｒｅｇ１Ｃｙ＋Ｃｙ
ｒｅｇ１Ｙｅ〔Ｋ 〕＝ｒｅｇ１Ｙｅ＋Ｙｅ ０≦ｋ≦５
【０１１４】
（５）全てのサンプルポイントにおいて（４）の処理が終了したら、（４）において白と判別したサンプルの総数をＳＮ１に、白でないサンプルの総数をＳＮ２に代入する。
【０１１５】
（６）図５の６個の白判別領域の中から、被写体の撮影条件と被写体条件から適切な白判別領域を選択し、選択された白領域の加算レジスタから第１の利得制御値を求める。そして第１の実施の形態と同様に第１、第２の利得制御値に基づいてホワイトバランス調整データを作成し、ホワイトバランス補正する。
【０１１６】
第１の実施の形態と同様に、撮影条件及び被写体条件とは、明るさを示すＥＶ値，シャッタースピードＴＶ値，絞り値であるＦナンバー，被写体までの距離，フラッシュのＯＮ／ＯＦＦ情報，被写体への外光とフラッシュ光の影響度などである。
【０１１７】
撮影時においてフラッシュがＯＦＦの時、ＥＶ値がある程度以上大きいならば、屋外で撮影したと憶測し、第１の実施の形態と同様の白判別範囲に相当する白判別領域を限定する。限定の結果、白判別領域内に色評価値が存在するポイントの数が０になってしまったら白判別領域を１つ増やす。
【０１１８】
同時に、ＥＶ値がある程度大きい時は（約１３以上）、高色温度側の白判別領域は使用しない（６０００Ｋ付近）。これは、空の淡い青を白と誤判別するのを防ぐためである。
【０１１９】
また、ストロボ装置によるフラッシュＯＮの時には、フラッシュの被写体への影響度を、主被写体までの距離，ＥＶ値，フラッシュの発光量などから求め、その結果により白判別領域が選択される。第１の実施の形態と同様に、フラッシュ光の影響度が高い場合は白判別領域をフラッシュ光の色温度の白判別領域のみ用いる。また、フラッシュ光の影響度が低くなるにつれて白判別領域の選択範囲を広げていき、外光まで追従させる。
【０１２０】
また、絞りＦ値が開放でシャッタースピード値ＴＶがある程度長い時には、ほぼ屋内撮影と断定できるため第１の実施の形態と同様に高色温度側の白判別領域を徐々に使用しないようにする。
【０１２１】
（７）（６）において選択された白判別領域内の色評価価値の平均値により光源の色温度を算出し、その色温度に対応する色変換マトリクスを選択する。例えば、図５において光源の色温度が５１００Ｋと算出された場合は、この色温度に最も近い５０００Ｋの色温度に対応するマトリクス４を選択する。
【０１２２】
以上のようにして、白判別範囲を色温度別に複数の白判別領域に分割し、被写体の撮影条件と被写体条件により適切な白判別領域を選択するようにしているため、実際の色温度と大幅に異なるサンプルポイントを検出することがなくなり、より正確なホワイトバランス信号を得ることが可能となる。また、算出された光源の色温度選択に基づいて色変換マトリクスを選択することにより、例えば、光源の色温度が高い場合には赤色を強調するようなマトリクスを選択し、光源の色温度が低い場合には青色を強調するようなマトリクスを選択するため、その色温度において最適な色再現が実現できる。
【０１２３】
また、本実施の形態では図２に示すような補色センサーを用いる場合について説明したが、純色センサーの場合でも実施可能であり、その場合、例えば色評価値をＲ−Ｙ，Ｂ−Ｙで設定すれば良い。
【０１２４】
また第１の実施の形態と同様に、図５複数の白判別領域の選択をマニュアルで決定しても良い。
【０１２５】
（第３の実施の形態）
第３の実施の形態におけるホワイトバランス処理は、第１の実施の形態における白判別範囲を予め所定の色温度別の白判別領域に分割し、撮影条件と被写体条件により適切な白判別領域のそれぞれにおける加算レジスタの値を補正して重み付けを行う点が特徴である。また、第２の実施の形態と同様に、色温度別に補色純色変換マトリクスを備え、算出されたホワイトバランス係数より色温度値を抽出し、複数のマトリクスの中から色温度に対応するマトリクスを選択する手段を備えている。
【０１２６】
以下に、ホワイトバランスの処理の流れを説明するが、第２の利得制御値を求める手法は第１の実施の形態と同様であるため説明は省略する。
【０１２７】
（１）予め白判別範囲を図５に示すように、約８０００Ｋから２０００Ｋまで１０００Ｋ毎に６個の白判別領域に分割する。
【０１２８】
（２）第１の利得制御値を求めるために、サンプルポイントがホワイトバランスデータとして計算できる明るさがあるかどうかを調べる（第１の実施の形態と同じ）。
【０１２９】
（３）サンプルポイントにおいて、色評価値ｗＸ，ｗＹを求め、予め設定された６個の白判別領域のどれに当てはまるか判定する。
【０１３０】
ｗＸ＝（Ｙｅ−Ｃｙ）／Ｙｌｏｗ （１）
ｗＹ＝（Ｍａ−Ｇ）／Ｙｌｏｗ （２）
但しＹｌｏｗ＝（Ｍａ＋Ｇ＋Ｃｙ＋Ｙｅ）／４
【０１３１】
（４）（３）における色評価値を、予め設定された白判別領域の中で適切な領域の加算レジスタに加える。
【０１３２】
ｒｅｇ１Ｍａ〔Ｋ 〕＝ｒｅｇ１Ｍａ＋Ｍａ
ｒｅｇ１Ｇ 〔Ｋ 〕＝ｒｅｇ１Ｇ ＋Ｇ
ｒｅｇ１Ｃｙ〔Ｋ 〕＝ｒｅｇ１Ｃｙ＋Ｃｙ
ｒｅｇ１Ｙｅ〔Ｋ 〕＝ｒｅｇ１Ｙｅ＋Ｙｅ ０≦ｋ≦５
【０１３３】
（５）全てのサンプルポイントにおいて（４）の処理が終了したら、（４）において白判別領域に色評価値が存在すると判別されたサンプルポイントの総数をＳＮ１に、白判別領域に色評価値が存在しないと判別されたサンプルポイントの総数をＳＮ２に代入する。
【０１３４】
（６）上記６個の白判別領域において、撮影条件および／又は被写体条件から各白判別領域の加算レジスタへの補正ゲインを図６の（ａ）に示すような色温度−補正ゲイングラフより求め、ゲイン補正後の白判別領域の加算レジスタの平均値より第１の利得制御値を求める。そして第１の実施の形態と同様に第１、第２の利得制御値に基づいてホワイトバランス調整データを作成し、ホワイトバランス補正する。
【０１３５】
撮影時においてフラッシュがＯＦＦの時、ＥＶ値が例えば１３ならば、屋外で撮影したと憶測し、図６の（ａ）の右側部に示すような予め設定した色温度−補正ゲイングラフより、低色温度側の白判別領域の加算レジスタの値のゲインを下げる。例えばＥＶ＝１３のとき、２０００〜４０００Ｋ，７０００〜８０００Ｋの白判別領域のゲインは０であるので各加算レジスタの値が０となり４０００〜５０００Ｋの白判別領域の補正ゲインは０．２５であるのでその領域の加算レジスタの値に０．２５が掛けられる。そして、５０００Ｋ〜７０００Ｋの白判別領域の補正ゲインは１．０であるので各加算レジスタの値に１．０が掛けられる。そして、このようにそれぞれゲイン補正された加算レジスタの平均値より第１の利得制御値を求める。ここで作成したグラフは、入力を色温度、出力を補正ゲインとしたものを適当なＥＶ値分だけ用意する。本実施例の場合は、ＥＶ１３からＥＶ１０までの４本のグラフを用意している。
【０１３６】
同時に、図６の（ａ）の左側部のように高色温度側の白判別領域の加算レジスタの値も補正を行う。これは、空の淡い青を白と誤判別するのを防ぐためである。
【０１３７】
また、フラッシュＯＮの時には、フラッシュの被写体への影響度を、主被写体までの距離，ＥＶ値，フラッシュの発光量などから求め、それにより白判別領域のゲインを変化させる。図６の（ｂ）に示すように、フラッシュ光の影響度が高い場合は白判別領域の重みをフラッシュ光の色温度の周辺部のみとし、フラッシュ光の影響度が低くなるにつれて白判別領域の重み範囲を広げていき、外光まで追従させる。
【０１３８】
また、シャッタースピードによる重みも第１の実施の形態と同様に、シャッタースピードが遅ければ高色温度側の重みを減らすようにする。
【０１３９】
（７）（６）において各白領域の加算レジスタを補正し、補正後の各レジスタの平均値により（３）における式によって色評価値を算出し、算出された色評価値に対応する光源の色温度を図５のグラフから求め、それに対応する色変換マトリクスを第２の実施の形態と同様に選択する。
【０１４０】
以上のようにして、白判別範囲を色温度別に複数の白判別領域に分割し、被写体の撮影条件と被写体条件に応じて各白判別領域の加算レジスタに重み付けを施すようにしているため、より正確なホワイトバランス信号を得ることが可能となる。また、上述のように算出された光源の色温度に基づいてマトリクスを選択するため、その色温度において最適な色再現が実現できる。
【０１４１】
また、本実施の形態では図２に示すような補色センサーを用いる場合について説明したが、純色センサーの場合でも実施可能であり、その場合、例えば色評価値をＲ−Ｙ，Ｂ−Ｙで設定すれば良い。
【０１４２】
また第１の実施の形態と同様に、撮影時の照明条件をマニュアルで設定し、それに応じた重みを設定すれば、より効果的なホワイトバランスが可能となる。
【０１４３】
（第４の実施の形態）
図７は、本発明の第４の実施の形態の撮像装置が有する信号処理装置の構成を示すブロック図である。ここで、ホワイトバランス以降の処理は第１の実施の形態と同様であるので説明は省略する。本実施の形態は、撮像素子を加算読み出しした場合でも、非加算読み出し時のホワイトバランスブロックを使用できるように工夫したものである。
【０１４４】
撮像素子２０１からの出力信号は、プリプロセス回路２０２を経てＡ／Ｄ変換回路２０３によりデジタル信号となり、更にＯＢ回路２０４によって黒レベルが揃えられ、メモリ２０５に一旦記憶される。メモリ２０５から読み出された画像信号は、演算部２０６にて補色信号に復元され、ホワイトバランス回路２０７に送られて利得調整値が算出される。また画像信号は、ホワイトバランス回路２０７からの利得制御値によりホワイトバランスが調整されて出力される。
【０１４５】
上記演算部２０６の処理内容を説明すると、加算信号（Ｗｒ，Ｗｂ，Ｇｒ，Ｇｂ）は次式で表される。
【０１４６】
Ｗｒ＝Ｍａ＋Ｇ （４−１）
Ｗｂ＝Ｍａ＋Ｃｙ （４−２）
Ｇｂ＝Ｇ＋Ｃｙ （４−３）
Ｇｒ＝Ｇ＋Ｙｅ （４−４）
４−１，４−２式よりＧ−Ｃｙが得られ、４−３式よりＧ信号が得られるのは明瞭である。このようにして、Ｍａ，Ｇ，Ｃｙ，Ｙｅ信号を復元し、ホワイトバランス回路２０７の入力信号とする。そして、その出力を色分離色補間回路２０８に送る。
【０１４７】
このようにして、補色フィルタを備えた撮像素子２０１に対して非加算読み出しを行い、ホワイトバランス回路２０７の白判別範囲を例えば補色信号による利得制御値
（Ｍａ−Ｇ）／Ｙ
（Ｙｅ−Ｃｙ）／Ｙ
で設定した場合、上記撮像素子２０１を加算読み出した出力値（Ｗｒ，Ｗｂ，Ｇｒ，Ｇｂ）の演算により（Ｍａ，Ｇ，Ｃｙ，Ｙｅ）の信号に復元し、上記ホワイトバランス回路２０７の入力信号とすることで、加算読み出し，非加算読み出し時において、回路の共通化が可能となり、白判別範囲の設定値も一つで良くなる。
【０１４８】
（第５の実施の形態）
本発明の第５の実施の形態の撮像装置のホワイトバランス回路は第１〜第３の実施例の何れかを用いるが、ホワイトバランス装置の入力信号を撮像素子からの画像信号を色分離して帯域制限したものにした点が異なっている。
【０１４９】
画像に多くのエッジが含まれていると、▲１▼エッジ部におけるサンプルポイントの色評価値は白判別範囲の対象に入らず、ホワイトバランスを調整するためのサンプル数が減少してしまう、また▲２▼エッジ部を異なる色温度の白と誤判別するという影響があり、好ましくない。そこで、予め画像信号にローパスフィルタを施して帯域を制限し、エッジをなくしてからホワイトバランス装置に入力する。
【０１５０】
図８は本実施の形態の撮像装置が有する信号処理装置の構成を示すブロック図である。撮像素子３０１の出力信号は、プリプロセス回路３０２，Ａ／Ｄ変換回路３０３を通り、ＯＢ回路３０４により黒レベルが揃えられる。ＯＢ回路３０４の出力は一旦メモリ３０５に記憶され、色分離色補間回路３０６にて色分離と同時に水平及び垂直方向に軽いローパスフィルタがかけられる。
【０１５１】
色分離色補間回路３０６の出力は、ホワイトバランス回路３０７に送られる。そして、ホワイトバランス回路３０７から補正ゲインが返され、それにより各色の白レベルが合わせられると同時に、メモリ３０５から信号は輝度信号生成部に入力され、垂直ローパスフィルタ回路３１５の後に白レベルがゲイン補正回路３１６にて補正される。
【０１５２】
ホワイトバランス後の色処理部においては、マトリクス回路３０８でＲＧＢの信号に変換される。このＲＧＢ信号は、ガンマ補正回路３０９にてガンマ補正され、純色色差変換回路３１０によりＹ，Ｒ−Ｙ，Ｂ−Ｙの信号に変換される。その後、Ｃ＿ＳＵＰ回路３１１にて飽和領域の色消し処理が行われた色差信号は、垂直ローパスフィルタ回路３１２及び水平ローパスフィルタ回路３１３を通り、Ｙ補正回路３２０，色補正回路３１４に送られる。
【０１５３】
一方輝度処理部においては、水平ローパスフィルタ回路３１７で輝度段差除去された画像信号は、ＡＰＣ回路３１８によりエッジ強調が行われ、ガンマ補正回路３１９によってガンマ補正される。その後、Ｈ−ＬＰＦ回路３１３からの色差信号を用いてＹ補正回路３２０にて輝度信号の微妙な輝度調整が実施される。
【０１５４】
そして、色処理部からの色差信号と輝度処理部からの輝度信号は圧縮回路３２１でＪＰＥＧ画像などに圧縮が行われ、フラッシュメモリ等のメモリ３２２に記憶される。
【０１５５】
このような構成をとることで、画像のエッジ部などによるサンプル数の減少、色の誤判別を防ぐことが可能となる。
【０１５６】
（第６の実施の形態）
本発明の第６の実施の形態の信号処理装置に用いるホワイトバランス回路は、第１〜第５の実施の形態のホワイトバランス算出回路の何れかを用いるが、色変換に用いるマトリクスの求め方が異なる。本実施の形態では、図５に示すように、色温度の変化に対し予め数個のマトリクスを用意する。同図では、８０００Ｋから２０００Ｋまで１０００Ｋ刻みに７個のマトリクスを用意している。
【０１５７】
補色センサーを用いた撮像素子に対して非加算読み出しした場合、ホワイトバランス回路からの利得制御値（ｗｂＭａ，ｗｂＧ，ｗｂＣｙ，ｗｂＹｅ）が出力される。この値から、図５に対する色評価値のＸ軸値ｗＸを求める。
【０１５８】
ｗＸ＝（１／ｗｂＹｅ−１／ｗｂＣｙ） （６−１）
次に、ｗＸに対応する白判別領域を求め、その領域の色温度リミッタのマトリクス２つと、ｗＸとリミッタとの距離を用いて次式のような線形演算で新たなマトリクスを作成する。
【０１５９】
ｄｉｓ＝ｌｉｍＢ−ｌｉｍＡ
ｎｅｗＭＡＴ＝（（ｄｉｓ−（ｗＸ−ｌｉｍＡ））^＊ｍａｔＡ＋（ｄｉｓ（ｌｉｍＢ−ｗＸ））^＊ｍａｔＢ）／ｄｉｓ （６−２）
本実施の形態によれば、ホワイトバランス回路からの利得制御値を用いて色温度を表す色評価値を求め、その値から色変換マトリクスを線形演算で求めることを特徴としており、これにより、マトリクスを切り替えることによって生じる色再現の変化をなめらかにすることができる。
【０１６０】
【発明の効果】
以上説明したように、本発明によれば、設定した白判別範囲を撮影時の撮影条件と被写体条件により変化させて色温度を検出する構成としたため、輝度信号レベルに依存せず正確な検出が行え、実際の色温度と大幅に異なる白判別範囲内の信号を検出することがなくなり、より正確なホワイトバランス信号を得ることが可能となり、さらに、撮像素子から出力される色信号の読み出しにおいて、非加算読み出しを行った場合に、撮像素子の加算読み出し時の出力値を演算することによって加算読み出し時の出力値を復元してホワイトバランス回路の入力信号とする構成とすることにより、加算読み出しと非加算読み出しにおいて回路の共通化が可能となる。
【０１６１】
また、本発明によれば、予め色温度別に設定された複数の白判別領域に分割し、その複数の白判別領域毎に含まれる色評価値を有するサンプルポイントの蓄積電荷を色ごとに積分し、被写体の撮影条件と被写体条件により上記複数の白判別領域から適切な白判別領域を選択し、前記積分値を用いて利得制御値および色温度を求め、かつ、補色純色変換マトリクスを複数用意し、上記利得制御値と白判別範囲から色温度を求め、それに対応するマトリクスを選択する構成とすることにより、適切なホワイトバランス補正が可能となる。また、算出された色温度に基づいてマトリクスを選択するため、その色温度において最適な色再現が実現できる。
【０１６５】
また、本発明によれば、撮像素子の出力信号を各色毎に分割し、ローパスフィルタをかけてからホワイトバランス回路に入力する構成とすることにより、画像のエッジ部などによるサンプル数の減少、色の誤判別を防ぐことが可能となる。
【０１６６】
また、本発明によれば、ホワイトバランス回路からの利得制御値を用いて色温度を表す色評価値を求め、その値から色変換マトリクスを線形演算で求める構成とすることにより、マトリクスを切り替えることによって生じる色再現の変化をより一層なめらかにすることができる。
【０１６７】
上述した各実施の形態は、一例として、前述した実施形態の機能を実現するソフトウェアのプログラムコードを記録した記録媒体を、システムあるいは装置に供給し、そのシステムあるいは装置のコンピュータ（またはＣＰＵやＭＰＵ）が記憶媒体に格納されたプログラムコードを読み出し実行することによって達成できる。
【０１６８】
この場合、記憶媒体から読み出されたプログラムコード自体が前述した実施形態の機能を実現することになり、そのプログラムコードを記憶した記憶媒体は上述した各実施の形態を構成することになる。
【０１６９】
プログラムコードを供給するための記憶媒体としては、例えば、フロッピーディスク、ハードディスク、光ディスク、光磁気ディスク、ＣＤ−ＲＯＭ、ＣＤ−Ｒ、磁気テープ、不揮発性のメモリカード、ＲＯＭなどを用いることができる。
【０１７０】
また、コンピュータが読み出したプログラムコードを実行することにより、前述した実施形態の機能が実現されるだけでなく、そのプログラムコードの指示に基づき、コンピュータ上で稼動しているＯＳ（オペレーティングシステム）などが実際の処理の一部または全部を行い、その処理によって前述した実施形態の機能が実現される場合も含まれる。
【０１７１】
さらに、記憶媒体から読み出されたプログラムコードが、コンピュータに挿入された機能拡張ボードやコンピュータに接続された機能拡張ユニットに備わるメモリに書き込まれた後、そのプログラムコードの指示にもとづき、その機能拡張ボードや機能拡張ユニットに備わるＣＰＵなどが実際の処理の一部または全部を行い、その処理によって前述した実施形態の機能が実現される。上述した各実施の形態を上記記憶媒体に適用する場合、その記録媒体には、先に説明したフローチャートに対応するプログラムコードを格納することになるが、簡単に説明すると、上述した各実施の形態のカメラ制御システムに不可欠なモジュールを、記憶媒体に格納することになる。
【図面の簡単な説明】
【図１】第１〜第３の実施の形態の撮像装置が有する信号処理装置の構成を示すブロック図
【図２】色フィルタ配列例を示す図
【図３】第１の実施の形態における白判別範囲を示す説明図
【図４】第１の実施の形態における白判別範囲のシフトを示す説明図
【図５】第２、第３の実施の形態における白判別範囲の色温度による分割を示す説明図
【図６】第３の実施の形態における色温度−補正ゲイングラフと白判別範囲の重み付けを示す説明図
【図７】第４の実施の形態の撮像装置が有する信号処理装置の構成を示すブロック図
【図８】第５の実施の形態の撮像装置が有する信号処理装置の構成を示すブロック図
【図９】白判別範囲を示す説明図
【図１０】従来の撮像装置の信号処理系の構成を示すブロック図
【図１１】従来の白判別範囲を示す図
【図１２】本発明の実施の形態における１画面分の画素の構成を示す概念図[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image pickup apparatus or a signal processing apparatus having an image pickup element having a color filter, and in particular, improving white balance signal processing.
[0002]
[Prior art]
FIG. 10 is a signal processing block diagram of a conventional single-plate imaging apparatus. Here, a case will be described in which the image sensor 401 is provided with complementary color filters arranged as shown in FIG.
[0003]
The analog output signal from the image sensor 401 is converted into a digital signal by the A / D conversion means 403 through the preprocess circuit 402, the black level is adjusted by the OB circuit 404, and is temporarily stored in the memory 405. The digital signal here is a signal (Wr, Wb, Gr, Gb) obtained by performing general-purpose addition reading on the image sensor 401 and adding the signals of the color filters (Ma, G, Cy, Ye).
[0004]
The image signal read from the memory 405 is gain-adjusted by the pixel gain adjustment circuit 406 using the gain adjustment value from the white balance circuit 420. The gain-adjusted image signal is sent to the color separation color interpolation circuit 407, outputted as Wr, Wb, Gr, and Gb signals and sent to the subsequent color processing unit, and at the same time, the read signal array of the image sensor 401. The signals as they are are sent to the luminance processing unit.
[0005]
In the color processing unit, the output of the color separation color interpolation circuit 407 is sent to the vertical low-pass filter circuit 408 and converted into RGB color signals by the matrix 409. The RGB signal is band-limited by the horizontal low-pass filter circuit 410, gamma-corrected by the gamma conversion circuit 411, and further converted to Y, RY, BY color difference signals by the pure color / color difference conversion circuit 412. This color difference signal is sent to the white balance circuit 420 and the C_SUP circuit 413, and after saturation processing is performed in the C_SUP circuit 413, it is sent to the Y correction circuit 419 and the color correction circuit 414. The color correction circuit 414 performs fine color adjustment.
[0006]
On the other hand, in the luminance processing section, a filter for correcting a luminance step by a color filter is applied by a vertical low-pass filter circuit 1015 and a horizontal low-pass filter circuit 1016, edge enhancement is performed by an APC (luminance aperture correction) circuit 417, and further gamma is applied. The correction circuit 418 performs gamma correction. Thereafter, the Y correction circuit 419 performs delicate luminance adjustment of the luminance signal using the color difference signal from the C_SUP circuit 413.
[0007]
Then, the color difference signal from the color processing unit and the luminance signal from the luminance processing unit are temporarily stored in the memory 423. When the image sensor 401 is read out in the field, the addition circuit 421 generates one frame signal and the compression circuit 422 performs JPEG. Etc. and record it in flash memory.
[0008]
The white balance circuit 420 includes a clip circuit that inputs a color difference signal (R−Y, B−Y) and limits input of a large amplitude exceeding a predetermined value with respect to the amplitude of the signal. For example, the color difference signal and the luminance signal that have passed through are divided into a plurality of blocks for one screen and integrated for each of the divided blocks, and as shown in FIG. 11, the coordinates formed by (RY) and (BY) are used. The change range when the color temperature of white changes is set as the white discrimination range 501, it is determined whether the integrated value for each block is within the white discrimination range on the same coordinate, and the integral value within the white discrimination range The system control CPU performs control for determining the gains of the R signal and the B signal using the signal, obtains the gain control value of the output of the image sensor 401 from this gain, and adjusts the gain of the output value of the image sensor 401 using the control value. To do It is.
[0009]
[Problems to be solved by the invention]
As described above, a conventional white balance adjustment method using coordinates formed by (RY), (BY), etc., setting a white discrimination range on the coordinates, and using an integral value within the white discrimination range is used. For example, when shooting with high color temperature illumination, a low-saturation red subject is included in the same region as the low color temperature white in the white discrimination range. If the area is large, it is misclassified as low color temperature illumination. Similarly, when shooting with low color temperature illumination, a blue subject with low saturation in the white discrimination range is included in the same area as white with high color temperature. Misclassified as color temperature lighting.
[0010]
Conventionally, a color evaluation value that is not included in the white discrimination range is not used, or is replaced with a value of the edge of the white discrimination range that is closest to the color evaluation value, and is integrated as a new color evaluation value. Therefore, it is necessary to provide a new calculation circuit.
[0011]
Conventionally, when a complementary color sensor is used, there is a problem that color reproduction is deteriorated unless an optimal matrix is prepared depending on the color temperature when converting an image signal after white balance into a pure color signal by matrix calculation.
[0012]
Conventionally, since the color evaluation value differs depending on the readout method of the image sensor, it is necessary to set the white discrimination range separately. For example, in a complementary color sensor having a color filter array as shown in FIG. 2, the color evaluation value when adding and reading out is (R−Y, BY) or (((Wr−Gb) − (Wb−Gr). )) / (Wr + Gb), ((Wr−Gb) + (Wb−Gr)) / (Wb + Gr)), and the color evaluation value in the case of non-addition readout is ((Ye−Cy) / Y1, (Ma) Since -G) / Y1) is used, a plurality of color evaluation values exist, and a plurality of white discrimination ranges must be prepared accordingly.
[0013]
In addition, if the output of the image sensor is input to the white balance adjustment unit as it is, the number of sample points may decrease when the image contains many edges, and the edges may be misidentified as white. There was a problem that white detection was not performed.
[0014]
The present invention has been made by paying attention to the above problems, and can detect an accurate color temperature without depending on the luminance signal level, can perform appropriate white balance control, and An object of the present invention is to provide an imaging apparatus capable of realizing optimum color reproduction.
[0015]
[Means for Solving the Problems]
In order to achieve the above-described object, an image sensor having a plurality of types of color filters;
In the reading of the color signal output from the image sensor, a selection unit that selects addition reading and non-addition reading;
When addition reading is selected by the selection means, restoration means for restoring to a color signal output at the time of non-addition reading;
Color signal evaluation means for evaluating a color signal output from the image sensor;
Discriminating means for discriminating whether or not the color signal evaluated by the color signal evaluating means is included in a predetermined evaluation range;
White balance adjusting means for adjusting white balance based on a color signal determined to be included in a predetermined evaluation range by the determining means;
Evaluation range of the discrimination meansIs changed according to a predetermined conditionChange means andWith
The color signal evaluation unit evaluates the color signal restored by the restoration unit when addition reading is selected by the selection unit.It is characterized by that.
[0016]
According to the invention of claim 2, the color temperature calculation according to claim 1, wherein the color temperature of the imaging light source is calculated based on the color signal determined to be included in the predetermined evaluation range by the determination unit. And a matrix conversion means for color-converting the color signal adjusted by the white balance adjustment means using a matrix, and the matrix conversion means has a plurality of different matrices and is calculated by the color temperature calculation means. One of the plurality of matrices is selected according to the selected color temperature, and color conversion is performed using the selected matrix.
[0017]
According to a third aspect of the present invention, in the second aspect, the matrix conversion means performs color conversion from Ma, G, Cy, Ye signals to R, G, B signals.
[0019]
Claims4According to the first aspect of the present invention, in the first aspect of the present invention, it is further provided with a low-pass filter that is provided in a preceding stage of the color signal evaluation unit and limits the band of the color signal output from the image sensor.
[0020]
Claims5According to the invention according to claim 1, in claim 1, further, a color temperature calculation unit that calculates a color temperature of an imaging light source based on a color signal determined to be included in a predetermined evaluation range by the determination unit, and the white Matrix conversion means for color-converting the color signal adjusted by the balance adjustment means using a matrix, and creation means for creating a matrix of the matrix calculation means based on the color temperature calculated by the color temperature calculation means It is characterized by that.
[0021]
According to a sixth aspect of the present invention, in the first aspect, the changing means is based on at least one of the brightness of the light source, the degree of influence of external light and flash light on the subject, and the shutter speed. The evaluation range is changed accordingly.
[0028]
Claims7According to the invention according to the present invention, the color signal evaluation means for evaluating the color signal output from the image sensor having a plurality of types of color filters,
In the reading of the color signal output from the image sensor, a selection unit that selects addition reading and non-addition reading;
When addition reading is selected by the selection means, restoration means for restoring to a color signal output at the time of non-addition reading;
Discriminating means for discriminating whether or not the color signal evaluated by the color signal evaluating means is included in a predetermined evaluation range;
White balance adjusting means for adjusting white balance based on a color signal determined to be included in a predetermined evaluation range by the determining means;
Change means for changing the evaluation range of the discrimination means according to a predetermined condition.,
The color signal evaluation unit evaluates the color signal restored by the restoration unit when addition reading is selected by the selection unit.
It is characterized by that.
[0029]
Claims8According to the invention according to claim7The color temperature calculating means for calculating the color temperature of the imaging light source based on the color signal determined to be included in the predetermined evaluation range by the determining means, and the color signal adjusted by the white balance adjusting means. Matrix conversion means for performing color conversion using a matrix, wherein the matrix conversion means has a plurality of different matrices, and one of the plurality of matrices is selected according to the color temperature calculated by the color temperature calculation means. One is selected, and color conversion is performed using the selected matrix.
[0030]
Claims9According to the invention according to claim8The matrix conversion means performs color conversion from Ma, G, Cy, Ye signals to R, G, B signals.
[0032]
Claims10According to the invention according to claim7Further, a low-pass filter is provided in front of the color signal evaluation means and limits the band of the color signal output from the image sensor.
[0033]
Claims11According to the invention according to claim7The color temperature calculating means for calculating the color temperature of the imaging light source based on the color signal determined to be included in the predetermined evaluation range by the determining means, and the color signal adjusted by the white balance adjusting means. Matrix conversion means for performing color conversion using a matrix, and creation means for creating a matrix of the matrix calculation means based on the color temperature calculated by the color temperature calculation means.
[0034]
Claims12According to the invention according to claim7The changing means changes the evaluation range according to at least one of the brightness of a light source, the influence of external light and flash light on the subject, and the shutter speed.
[0054]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.
[0055]
(First embodiment)
FIG. 1 is a block diagram illustrating a configuration of a signal processing device included in the imaging device according to the first embodiment. Here, a case will be described in which the image sensor 101 is provided with complementary color filters arranged as shown in FIG.
[0056]
The output signal of the image sensor 101 enters the A / D conversion circuit 103 via the preprocess circuit 102, where it becomes a digital signal, the black level is aligned by the OB circuit 104, and is temporarily stored in the memory 105. The image signal read from the memory 105 is sent to the white balance circuit 106, and the gain of the color signal is adjusted by a gain adjustment value determined by a method described later and output. The output signal of the white balance circuit 106 is sent to the color separation color interpolation circuit 108, where four signals of Ma, G, Cy, and Ye are simultaneously output to the color processing unit, and the image pickup device 101 is provided to the luminance processing unit. The signal in the filter array is sent.
[0057]
In the color processing unit, a vertical band is limited by the vertical low-pass filter circuit 109 and converted into RGB color signals by the matrix circuit 110. The RGB signal is subjected to gamma correction by the gamma conversion circuit 111 and converted to Y, RY, BY signals by the pure color difference conversion circuit 112. Thereafter, the color difference signal that has been subjected to the achromatic processing of the saturation region in the C_SUP circuit 113 is sent to the Y correction circuit 120 and the color correction circuit 115 through the horizontal low-pass filter circuit 114.
[0058]
On the other hand, in the luminance processing section, a filter for correcting a luminance step by a color filter is applied by a vertical low-pass filter circuit 116 and a horizontal low-pass filter circuit 117, edge enhancement is performed by an APC circuit 118, and gamma correction is performed by a gamma correction circuit 119. Is done. Thereafter, the Y correction circuit 120 performs fine luminance adjustment of the luminance signal using the color difference signal from the H-LPF circuit 114.
[0059]
The color difference signal from the color processing unit and the luminance signal from the luminance processing unit are compressed by JPEG or the like in the compression circuit 121 and stored in a memory 122 such as a flash memory.
[0060]
Next, the operation of the white balance circuit 106 which is the main point of the present embodiment will be described.
[0061]
For example, when the CCD complementary color filters of the image sensor 101 are arranged as shown in FIG. 2, the whole is realized by repeating this unit block with the 8-pixel arrangement shown in FIG. Of these 8 pixels, the upper or lower 4 pixels (Ma, G, Cy, Ye) are used as white evaluation sample points, and the color evaluation value at each sample point.
(Ma-G) / Y, (Ye-Cy) / Y (where Y is a luminance signal)
, And a point included in a preset white discrimination range is discriminated as white and integrated.
[0062]
The white discrimination range is set by photographing a blank sheet in advance at a plurality of color temperatures existing in nature, obtaining the above color evaluation values at each color temperature, and, for example, (Ye-Cy) on the X axis as shown in FIG. / Y, plotting (Ma-G) / Y on the Y axis is set as the white discrimination range.
[0063]
Hereinafter, the processing flow of the white balance circuit 106 will be described.
[0064]
(1) The white discrimination range is corrected based on the shooting conditions of the subject and / or the subject conditions. This prevents the detection of a signal within a white discrimination range that is significantly different from the actual color temperature, and improves the white discrimination accuracy when the flash is turned on during shooting with daylight.
[0065]
Shooting conditions and / or subject conditions are: EV value indicating brightness, shutter speed TV value, F number as aperture value, distance to subject, flash ON / OFF information, external light to subject and flash light The degree of influence.
[0066]
When the flash is off at the time of shooting, if the EV value is larger than a certain level, it is assumed that the image was taken outdoors, and the limiter on the low color temperature side within the white discrimination range as shown on the right side of FIG. (X axis value X axis is the change direction of color temperature) is shifted to the high color temperature side (in many cases, outdoor light is around 5000K). Experience shows that when the EV value is 10 or more, the possibility of outdoor photography increases, and using this fact, the limiter is moved by the linear calculation of the following equation (1-1) using the EV value as an input.
[0067]
limit Low Temp = a1 * EV + b1 (1-1)
When the EV value is large to some extent (about 13 or more), the limiter on the high color temperature side is also moved to the low temperature side using the following equation (1-2) as shown on the left side of FIG. Shift (near 6000K). This is to prevent erroneous determination of light blue in the sky as white. In addition, since the shaded color temperature may exceed 6000K, it is preferable to set a limit of about 7000K in a normal case.
[0068]
limit High Temp = a2 * EV + b2 (1-2)
Next, in the case of flash photography, the influence of external light increases as the degree of influence of the flash light on the subject decreases. Therefore, the area of the white discrimination range is expanded to enable external light tracking. Therefore, the effect flash effect on the subject of the flash is obtained from the distance to the main subject, the EV value, the light emission amount of the flash, etc., and the following formula (1-3) is obtained as shown in FIG. Used to change the white discrimination range. This equation is a linear equation like the above-described equations 1-1, 1-2, etc. In this case, the input is the influence level and the output is the limiter value.
[0069]
limit Flash Temp = a3 * effectFlash + b3 (1-3)
Here, the degree of influence of the flash on the subject is calculated using a technique described in US Pat. No. 5,568,187 (OKINO Patent).
[0070]
Further, when the aperture F value is wide and the shutter speed value TV is long to some extent, it can be determined that the shooting is almost indoor, so the limiter on the high color temperature side is shifted to the color temperature of the fluorescent lamp as shown in FIG. . The limiter setting in this case also performs linear calculation of the following equation with the input being the TV value and the output being the limiter value (X-axis value).
limit Tv Temp = a4 * TV + b4 (1-4)
[0071]
Next, first and second gain control values for controlling the gain of each color signal are calculated.
[0072]
The first gain control value is calculated based on a sample point signal that is included in the white discrimination range defined in (1) above. Here, the sample point is described above, but FIG. As shown in FIG. 12, four pixels of Ma, G, Cy, and Ye are made into one set.
[0073]
The second gain control value is calculated based on a signal obtained by averaging signals in a small area (sample area) that is included in the white discrimination range determined in (1) above. Here, the small area indicates one area obtained by dividing one screen into m areas, and the small area signal indicates Ma, G, Cy, and Ye existing in the small area for each pixel. Shows the signal obtained by averaging.
[0074]
First, a method for calculating the first gain control value will be described.
[0075]
(2) First, FIG. Check whether the sample points shown in 12 satisfy all of the following expressions.
[0076]
Dark Threshold <(Ma + G + Cy + Ye) / 4
Bright Threshold> Ma
Bright Threshold> G
Bright Threshold> Cy
Bright Threshold> Ye
[0077]
(3) Color evaluation values wX and wY calculated based on the following formula are obtained at sample points that satisfy all the formulas (2).
[0078]
wX = (Ye-Cy) / Ylow (I)
wY = (Ma−G) / Ylow (II)
However, Ylow = (Ma + G + Cy + Ye)
[0079]
(4) It is determined whether or not the color evaluation value obtained at each sample point in (3) is included in the white determination range determined in (1). If the color evaluation value is included in the white discrimination range defined in (1), the output value output from the pixel at that sample point is added to the addition register for each color. This process is performed at every sample point on the screen.
[0080]
reg1Ma = reg1Ma + Ma
reg1G = reg1G + G
reg1Cy = reg1Cy + Cy
reg1Ye = reg1Ye + Ye
[0081]
(5) When the processing of (4) is completed for all the sample points, the total number of sample points determined to be white in SN4 is SN1, and the total number of sample points determined not to be white is SN2 to be described later. Is assigned to the expression
[0082]
(6) The first gain control value is obtained from the addition register of (4) as follows.
[0083]
gmg1 = aveReg1 / reg1Ma
(AveReg1 = (reg1Ma + reg1G + reg1Cy + reg1Ye) / 4)
gg1 = aveReg1 / reg1G
gcy1 = aveReg1 / reg1Cy
gye1 = aveReg1 / reg1Ye
[0084]
Next, a method for calculating the second gain control value will be described. The calculation of the second gain control value is performed in parallel with the calculation of the first gain control value.
[0085]
(7) First, FIG. As shown in FIG. 12, one screen is divided into m small areas, and an addition register for storing an addition value of Ma, G, Cy, Ye is performed for each divided area (sample area). .
[0086]
reg2Ma [m]
reg2G [m]
reg2Cy [m]
reg2Ye [m]
[0087]
(8) In each divided area (sample area), whether or not the image sensor output values of the pixels at the sample points in the sample area satisfy all of the following expressions, that is, the brightness at which the sample points can be calculated as white balance data Check if it is within the range.
[0088]
Dark Threshold <(Ma + G + Cy + Ye) / 4
Bright Threshold> Ma
Bright Threshold> G
Bright Threshold> Cy
Bright Threshold> Ye
[0089]
Within each region, the image sensor output value of the pixel is added to the addition register for each color only at sample points that satisfy these equations.
[0090]
reg2Ma [m] = reg2Ma [m] + Ma
reg2G [m] = reg2G [m] + G
reg2Cy [m] = reg2Cy [m] + Cy
reg2Ye [m] = reg2Ye [m] + Ye
[0091]
(9) Within each region, the values of the color addition registers obtained by the processing of (8) are averaged for each color addition register. The values obtained by averaging are substituted into the above (I) and (II), and the evaluation value of the color in the area (sample area) is obtained. It is determined whether the color evaluation value of each area obtained in this way is included in the white discrimination range defined in (1). Then, only the value of the addition register in the area determined to be included in the white discrimination range is added to the area average register described below.
[0092]
reg2Ma = reg2Ma + reg2Ma [m]
reg2G = reg2G + reg2G [m]
reg2Cy = reg2Cy + reg2Cy [m]
reg2Ye = reg2Ye + reg2Ye [m]
[0093]
(10) The second gain control value is obtained by the following equation based on the value of the area average register obtained in (9).
[0094]
gmg2 = aveReg2 / reg2Ma
(AveReg2 = (reg2Ma + reg2G + reg2Cy + reg2Ye) / 4)
gg2 = aveReg2 / reg2G
gcy2 = aveReg2 / reg2Cy
gye2 = aveReg2 / reg2Ye
[0095]
When none of the color evaluation values obtained in (9) is included in the white discrimination range defined in (1), any one of the color evaluation values in each area is (1). The area of each divided area (sample area) of the screen is expanded step by step until it is included in the white discrimination range defined in.
[0096]
(11) From the ratio of SN1 and SN2 obtained in (5), the first gain control value obtained in (6), and the second gain control value obtained in (10), white balance adjustment data ( wmg1, wg1, wcy1, and wye1).
[0097]
For example, the following processing is performed.
[0098]
If (SN1 / (SN1 + SN2)) ≧ α,
The first gain control value is directly used as white balance adjustment data.
wmg1 = gmg1
wg1 = gg1
wcy1 = gcy1
why1 = gye1
If β <(SN1 / (SN1 + SN2)) <α,
A linear function of input SN1 and output P is created with the following conditional expression.
[0099]
(conditions)
SN1 = α → P = 1
SN1 = β → P = 0
(Primary function formula)
p = a × SN1 + b
[0100]
[Outside 1]

wmg1 = p * gmg1 + (1-p) * gmg2
wcy1 = p * gcy + (1-p) * gcy2
why1 = p * gye1 + (1-p) * gye2
When (SN1 / (SN1 + SN2)) ≦ β
wmg1 = gmg2
wg1 = gg2
wcy1 = gcy2
where1 = gye2
(However, 0<β <α<1)
These values are stored as white balance adjustment data.
[0101]
As described above, by changing the white discrimination range according to the shooting conditions and the subject conditions, signals within the white discrimination range significantly different from the actual color temperature are not detected, and a more accurate white balance signal is obtained. It becomes possible.
[0102]
Also, since the calculation method for obtaining the white balance adjustment data using the first and second gain control values is changed according to the number of sample points that fall within the white discrimination range defined in (1), the white balance is greatly increased. There is no breakage.
[0103]
Further, in the present embodiment, the case where the complementary color sensor as shown in FIG. 2 is used has been described. However, the present invention can also be implemented in the case of a pure color sensor. It ’s fine.
[0104]
In this embodiment, the area width of the white discrimination area is automatically determined. However, it may be set manually as shown in FIG. For example, the white discrimination range is shown in FIG. 9 (a) for sunlight, FIG. 9 (b) for fluorescent light, FIG. 9 (c) for tungsten light, and FIG. 9 for flash light. It is limited as shown in (d).
[0105]
(Second Embodiment)
In the white balance processing in the second embodiment, the white discrimination range in the first embodiment described above is divided into white discrimination areas determined for each color temperature as shown in FIG. It is characterized in that white balance control is performed based on a sample point having a color evaluation value existing in the region and a region obtained by dividing one screen into m pieces. Also, a complementary color pure color conversion matrix is provided for each color temperature, and the color temperature is calculated by averaging the color evaluation values in the selected white discrimination region, and the matrix corresponding to the color temperature is selected from a plurality of matrices. This is the configuration to select.
[0106]
In the following, the flow of white balance processing will be described, but the method for obtaining the second gain control value is the same as in the first embodiment, so description thereof will be omitted.
[0107]
(1) The white discrimination range is divided into six white discrimination regions every 1000K from about 8000K to 2000K as shown in FIG.
[0108]
In order to obtain an image with optimum color reproduction, it is essential to change the matrix for complementary color-pure color conversion according to the color temperature of the subject. Therefore, in order to achieve optimal color reproduction, the number of necessary color conversion matrices within the color temperature range of illumination light is examined in advance. Then, the white discrimination range is divided corresponding to the color temperature range to be reproduced by each matrix.
[0109]
(2) In order to obtain the first gain control value, it is checked whether or not the sample point shown in FIG. 12 has brightness that can be calculated as white balance data (same as in the first embodiment).
[0110]
(3) Color evaluation values wX and wY are obtained at sample points.
[0111]
wX = (Ye-Cy) / Ylow (1)
wY = (Ma−G) / Ylow (2)
However, Ylow = (Ma + G + Cy + Ye) / 4
[0112]
(4) The color evaluation value in (3) is added to the addition register of the white discrimination area corresponding to the preset white discrimination area.
[0113]
reg1Ma [K] = reg1Ma + Ma
reg1G [K] = reg1G + G
reg1Cy [K] = reg1Cy + Cy
reg1Ye [K] = reg1Ye + Ye 0 ≦ k ≦ 5
[0114]
(5) When the processing of (4) is completed at all sample points, the total number of samples determined to be white in (4) is substituted for SN1, and the total number of non-white samples is substituted for SN2.
[0115]
(6) An appropriate white discrimination area is selected from the shooting conditions of the subject and the subject conditions from the six white discrimination areas shown in FIG. 5, and the first gain control value is obtained from the addition register of the selected white area. . Then, similarly to the first embodiment, white balance adjustment data is created based on the first and second gain control values, and white balance correction is performed.
[0116]
As in the first embodiment, the shooting conditions and the subject conditions are the EV value indicating brightness, the shutter speed TV value, the F number as the aperture value, the distance to the subject, the flash ON / OFF information, the subject This is the degree of influence of external light and flash light.
[0117]
When the flash is turned off at the time of shooting, if the EV value is larger than a certain level, it is assumed that the image is taken outdoors, and the white discrimination area corresponding to the white discrimination range similar to the first embodiment is limited. As a result of the limitation, if the number of points where the color evaluation value exists in the white discrimination area becomes 0, the white discrimination area is increased by one.
[0118]
At the same time, when the EV value is large to some extent (about 13 or more), the white discrimination region on the high color temperature side is not used (near 6000K). This is to prevent erroneous determination of light blue in the sky as white.
[0119]
When the flash is turned on by the strobe device, the degree of influence of the flash on the subject is obtained from the distance to the main subject, the EV value, the flash emission amount, and the like, and the white discrimination region is selected based on the result. Similar to the first embodiment, when the influence degree of the flash light is high, only the white determination area of the color temperature of the flash light is used as the white determination area. Further, as the influence of the flash light becomes lower, the selection range of the white discrimination area is expanded to follow the external light.
[0120]
Further, when the aperture F value is wide and the shutter speed value TV is long to some extent, it can be determined that the shooting is almost indoor, so that the white discrimination area on the high color temperature side is not gradually used as in the first embodiment.
[0121]
(7) The color temperature of the light source is calculated from the average value of the color evaluation values in the white discrimination region selected in (6), and a color conversion matrix corresponding to the color temperature is selected. For example, when the color temperature of the light source is calculated to be 5100K in FIG. 5, the matrix 4 corresponding to the color temperature of 5000K closest to this color temperature is selected.
[0122]
As described above, the white discrimination range is divided into a plurality of white discrimination regions for each color temperature, and an appropriate white discrimination region is selected according to the shooting conditions and subject conditions of the subject. Therefore, it is possible to obtain a more accurate white balance signal. Also, by selecting a color conversion matrix based on the calculated color temperature of the light source, for example, when the color temperature of the light source is high, a matrix that emphasizes red is selected, and the color temperature of the light source is low In this case, since a matrix that emphasizes blue is selected, optimal color reproduction can be realized at the color temperature.
[0123]
Further, in the present embodiment, the case where the complementary color sensor as shown in FIG. 2 is used has been described. However, the present invention can also be implemented in the case of a pure color sensor. In this case, for example, color evaluation values are set as RY and BY. Just do it.
[0124]
Similarly to the first embodiment, the selection of a plurality of white discrimination regions in FIG. 5 may be determined manually.
[0125]
(Third embodiment)
In the white balance processing in the third embodiment, the white discrimination range in the first embodiment is divided into white discrimination areas for each predetermined color temperature in advance, and each of the appropriate white discrimination areas is determined according to the shooting conditions and subject conditions. The feature is that weighting is performed by correcting the value of the addition register in FIG. Similarly to the second embodiment, a complementary color pure color conversion matrix is provided for each color temperature, color temperature values are extracted from the calculated white balance coefficient, and a matrix corresponding to the color temperature is selected from a plurality of matrices. Means to do.
[0126]
In the following, the flow of white balance processing will be described, but the method for obtaining the second gain control value is the same as in the first embodiment, and the description thereof will be omitted.
[0127]
(1) As shown in FIG. 5, the white discrimination range is divided into about 6 white discrimination areas every 1000K from about 8000K to 2000K.
[0128]
(2) In order to obtain the first gain control value, it is checked whether or not the sample point has brightness that can be calculated as white balance data (same as in the first embodiment).
[0129]
(3) At the sample points, the color evaluation values wX and wY are obtained, and it is determined which of the six preset white discrimination areas is applicable.
[0130]
wX = (Ye-Cy) / Ylow (1)
wY = (Ma−G) / Ylow (2)
However, Ylow = (Ma + G + Cy + Ye) / 4
[0131]
(4) The color evaluation value in (3) is added to an addition register in an appropriate area within a preset white discrimination area.
[0132]
reg1Ma [K] = reg1Ma + Ma
reg1G [K] = reg1G + G
reg1Cy [K] = reg1Cy + Cy
reg1Ye [K] = reg1Ye + Ye 0 ≦ k ≦ 5
[0133]
(5) When the processing of (4) is completed for all sample points, the total number of sample points determined to have a color evaluation value in the white determination area in (4) is SN1, and the color evaluation value is in the white determination area. The total number of sample points determined not to exist is substituted into SN2.
[0134]
(6) In the six white discrimination areas, the correction gain from the shooting condition and / or subject condition to the addition register of each white discrimination area is obtained from a color temperature-correction gain graph as shown in FIG. The first gain control value is obtained from the average value of the addition registers in the white discrimination region after gain correction. Then, similarly to the first embodiment, white balance adjustment data is created based on the first and second gain control values, and white balance correction is performed.
[0135]
When the flash is off at the time of shooting, if the EV value is 13, for example, it is assumed that the image was shot outdoors, and is lower than the preset color temperature-correction gain graph as shown on the right side of FIG. The gain of the value of the addition register in the white discrimination area on the color temperature side is lowered. For example, when EV = 13, the gains of the white discrimination regions of 2000 to 4000K and 7000 to 8000K are 0, so the value of each addition register is 0, and the correction gain of the white discrimination region of 4000 to 5000K is 0.25. The value of the addition register in that area is multiplied by 0.25. Since the correction gain of the white discrimination region from 5000K to 7000K is 1.0, the value of each addition register is multiplied by 1.0. Then, the first gain control value is obtained from the average value of the addition registers each gain-corrected in this way. The graph created here is prepared for the appropriate EV value with the input color temperature and the output correction gain. In this embodiment, four graphs from EV13 to EV10 are prepared.
[0136]
At the same time, the value of the addition register in the white discrimination region on the high color temperature side is corrected as shown on the left side of FIG. This is to prevent erroneous determination of light blue in the sky as white.
[0137]
When the flash is on, the influence of the flash on the subject is obtained from the distance to the main subject, the EV value, the flash emission amount, and the like, thereby changing the gain of the white discrimination region. As shown in FIG. 6B, when the influence degree of the flash light is high, the weight of the white discrimination area is set only to the peripheral part of the color temperature of the flash light, and the white discrimination area is increased as the influence degree of the flash light becomes low. Extend the weight range to follow outside light.
[0138]
In addition, as with the first embodiment, the weight on the high color temperature side is reduced if the shutter speed is slow, as in the first embodiment.
[0139]
(7) In step (6), the addition register for each white region is corrected, and a color evaluation value is calculated by the expression in (3) based on the average value of each corrected register. The light source corresponding to the calculated color evaluation value is calculated. The color temperature is obtained from the graph of FIG. 5, and the corresponding color conversion matrix is selected as in the second embodiment.
[0140]
As described above, the white discrimination range is divided into a plurality of white discrimination regions for each color temperature, and the addition register of each white discrimination region is weighted according to the shooting conditions of the subject and the subject conditions. An accurate white balance signal can be obtained. In addition, since the matrix is selected based on the color temperature of the light source calculated as described above, optimal color reproduction can be realized at the color temperature.
[0141]
Further, in the present embodiment, the case where the complementary color sensor as shown in FIG. 2 is used has been described. However, the present invention can also be implemented in the case of a pure color sensor. In this case, for example, color evaluation values are set as RY and BY. Just do it.
[0142]
Similarly to the first embodiment, more effective white balance can be achieved by manually setting illumination conditions at the time of shooting and setting weights accordingly.
[0143]
(Fourth embodiment)
FIG. 7 is a block diagram illustrating a configuration of a signal processing device included in the imaging device according to the fourth embodiment of the present invention. Here, the processing after the white balance is the same as that of the first embodiment, and the description thereof is omitted. The present embodiment is devised so that the white balance block at the time of non-addition readout can be used even when the image sensor is subjected to addition readout.
[0144]
An output signal from the image sensor 201 is converted into a digital signal by the A / D conversion circuit 203 through the preprocess circuit 202, and the black level is further adjusted by the OB circuit 204 and is temporarily stored in the memory 205. The image signal read from the memory 205 is restored to a complementary color signal by the arithmetic unit 206 and sent to the white balance circuit 207 to calculate a gain adjustment value. The image signal is output with the white balance adjusted by the gain control value from the white balance circuit 207.
[0145]
The processing content of the arithmetic unit 206 will be described. The addition signal (Wr, Wb, Gr, Gb) is expressed by the following equation.
[0146]
Wr = Ma + G (4-1)
Wb = Ma + Cy (4-2)
Gb = G + Cy (4-3)
Gr = G + Ye (4-4)
It is clear that G-Cy is obtained from equations 4-1 and 4-2, and a G signal is obtained from equation 4-3. In this way, the Ma, G, Cy, and Ye signals are restored and used as input signals to the white balance circuit 207. Then, the output is sent to the color separation color interpolation circuit 208.
[0147]
In this way, non-addition readout is performed on the image sensor 201 having the complementary color filter, and the white determination range of the white balance circuit 207 is set to, for example, the gain control value based on the complementary color signal.
(Ma-G) / Y
(Ye-Cy) / Y
Is set to (Ma, G, Cy, Ye) by restoring the output values (Wr, Wb, Gr, Gb) obtained by adding and reading the image sensor 201, and the input signal of the white balance circuit 207 is restored. By doing so, it is possible to share a circuit during addition reading and non-addition reading, and only one set value for the white discrimination range is required.
[0148]
(Fifth embodiment)
The white balance circuit of the image pickup apparatus according to the fifth embodiment of the present invention uses any one of the first to third examples. The input signal of the white balance apparatus is color-separated from the image signal from the image pickup device. The difference is that the band is limited.
[0149]
If the image contains many edges, (1) the color evaluation value of the sample point at the edge portion does not fall within the white discrimination range, and the number of samples for adjusting the white balance decreases. {Circle around (2)} The edge portion is erroneously distinguished from white having a different color temperature, which is not preferable. Therefore, a low-pass filter is applied to the image signal in advance to limit the band, and the edge is eliminated before inputting to the white balance device.
[0150]
FIG. 8 is a block diagram illustrating a configuration of a signal processing device included in the imaging device of the present embodiment. The output signal of the image sensor 301 passes through the preprocess circuit 302 and the A / D conversion circuit 303, and the black level is adjusted by the OB circuit 304. The output of the OB circuit 304 is temporarily stored in the memory 305, and a light low-pass filter is applied in the horizontal and vertical directions simultaneously with the color separation by the color separation color interpolation circuit 306.
[0151]
The output of the color separation color interpolation circuit 306 is sent to the white balance circuit 307. Then, the correction gain is returned from the white balance circuit 307, and thereby the white level of each color is adjusted. Correction is performed by the circuit 316.
[0152]
In the color processing unit after white balance, the matrix circuit 308 converts the signal into RGB signals. The RGB signals are gamma corrected by a gamma correction circuit 309 and converted into Y, RY, and BY signals by a pure color / color difference conversion circuit 310. After that, the color difference signal that has been subjected to the saturation region achromatic processing in the C_SUP circuit 311 passes through the vertical low-pass filter circuit 312 and the horizontal low-pass filter circuit 313 and is sent to the Y correction circuit 320 and the color correction circuit 314.
[0153]
On the other hand, in the luminance processing unit, the image signal from which the luminance level difference is removed by the horizontal low-pass filter circuit 317 is edge-enhanced by the APC circuit 318 and gamma-corrected by the gamma correction circuit 319. Thereafter, the Y correction circuit 320 performs fine luminance adjustment of the luminance signal using the color difference signal from the H-LPF circuit 313.
[0154]
The color difference signal from the color processing unit and the luminance signal from the luminance processing unit are compressed into a JPEG image or the like by the compression circuit 321 and stored in a memory 322 such as a flash memory.
[0155]
By adopting such a configuration, it is possible to prevent a decrease in the number of samples due to an edge portion of an image or the like, and erroneous color discrimination.
[0156]
(Sixth embodiment)
The white balance circuit used in the signal processing apparatus according to the sixth embodiment of the present invention uses any of the white balance calculation circuits according to the first to fifth embodiments. However, it is necessary to obtain a matrix used for color conversion. Different. In the present embodiment, as shown in FIG. 5, several matrices are prepared in advance for changes in color temperature. In the figure, seven matrices are prepared in increments of 1000K from 8000K to 2000K.
[0157]
When non-addition reading is performed on an image sensor using a complementary color sensor, gain control values (wbMa, wbG, wbCy, wbYe) are output from the white balance circuit. From this value, the X-axis value wX of the color evaluation value for FIG. 5 is obtained.
[0158]
wX = (1 / wbYe-1 / wbCy) (6-1)
Next, a white discriminating area corresponding to wX is obtained, and a new matrix is created by a linear calculation such as the following expression using two color temperature limiter matrices in that area and the distance between wX and the limiter.
[0159]
dis = limB-limA
newMAT = ((dis- (wX-limA))^*matA + (dis (limB-wX))^*matB) / dis (6-2)
According to the present embodiment, a color evaluation value representing a color temperature is obtained using a gain control value from a white balance circuit, and a color conversion matrix is obtained from the value by a linear operation. It is possible to smooth the change in color reproduction caused by switching between.
[0160]
【The invention's effect】
As described above, according to the present invention, since the color temperature is detected by changing the set white discrimination range according to the shooting condition and the subject condition at the time of shooting, accurate detection is possible regardless of the luminance signal level. It is not possible to detect a signal within a white discrimination range that is significantly different from the actual color temperature, it is possible to obtain a more accurate white balance signal, and in addition, in reading out the color signal output from the image sensor, When non-addition readout is performed, the output value at the time of addition readout of the image sensor is calculated to restore the output value at the time of addition readout to be the input signal of the white balance circuit. The circuit can be shared in non-addition readout.
[0161]
Further, according to the present invention, it is divided into a plurality of white discrimination regions set in advance for each color temperature, and the accumulated charge of sample points having color evaluation values included in each of the plurality of white discrimination regions is integrated for each color. The appropriate white discrimination region is selected from the plurality of white discrimination regions according to the shooting condition of the subject and the subject condition, the gain control value and the color temperature are obtained using the integral values, and a plurality of complementary color pure color conversion matrices are prepared. Thus, by obtaining a color temperature from the gain control value and the white discrimination range and selecting a matrix corresponding to the color temperature, an appropriate white balance correction can be performed. Further, since the matrix is selected based on the calculated color temperature, optimal color reproduction can be realized at the color temperature.
[0165]
In addition, according to the present invention, the output signal of the image sensor is divided for each color and applied to the white balance circuit after applying a low-pass filter, thereby reducing the number of samples due to the edge portion of the image, the color It is possible to prevent misidentification.
[0166]
Further, according to the present invention, the matrix is switched by obtaining a color evaluation value representing the color temperature using the gain control value from the white balance circuit, and obtaining the color conversion matrix by linear calculation from the value. The color reproduction change caused by the above can be made smoother.
[0167]
In each of the above-described embodiments, as an example, a recording medium that records a program code of software that realizes the functions of the above-described embodiments is supplied to the system or apparatus, and the computer of the system or apparatus (or CPU or MPU). Can be achieved by reading and executing the program code stored in the storage medium.
[0168]
In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes each of the above-described embodiments.
[0169]
As a storage medium for supplying the program code, for example, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.
[0170]
Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on the instruction of the program code. A case where part or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included.
[0171]
Further, after the program code read from the storage medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. The CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing. When each of the above-described embodiments is applied to the above-described storage medium, the program code corresponding to the above-described flowchart is stored in the recording medium. Modules indispensable for the camera control system are stored in a storage medium.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of a signal processing device included in an imaging device according to first to third embodiments.
FIG. 2 is a diagram showing an example of color filter arrangement
FIG. 3 is an explanatory diagram showing a white discrimination range according to the first embodiment.
FIG. 4 is an explanatory diagram showing a shift of a white discrimination range in the first embodiment.
FIG. 5 is an explanatory diagram showing division by color temperature of the white discrimination range in the second and third embodiments.
FIG. 6 is an explanatory diagram illustrating weighting of a color temperature-correction gain graph and a white discrimination range according to the third embodiment.
FIG. 7 is a block diagram illustrating a configuration of a signal processing device included in the imaging device according to the fourth embodiment.
FIG. 8 is a block diagram illustrating a configuration of a signal processing device included in the imaging device according to the fifth embodiment;
FIG. 9 is an explanatory diagram showing a white discrimination range.
FIG. 10 is a block diagram illustrating a configuration of a signal processing system of a conventional imaging device.
FIG. 11 is a diagram showing a conventional white discrimination range
FIG. 12 is a conceptual diagram showing the configuration of pixels for one screen in the embodiment of the present invention.

Claims (12)

An image sensor having a plurality of types of color filters;
In the reading of the color signal output from the image sensor, a selection unit that selects addition reading and non-addition reading;
When addition reading is selected by the selection means, restoration means for restoring to a color signal output at the time of non-addition reading;
Color signal evaluation means for evaluating a color signal output from the image sensor;
Discriminating means for discriminating whether or not the color signal evaluated by the color signal evaluating means is included in a predetermined evaluation range;
White balance adjusting means for adjusting white balance based on a color signal determined to be included in a predetermined evaluation range by the determining means;
Change means for changing the evaluation range of the discrimination means according to a predetermined condition ,
The color signal evaluation unit evaluates the color signal restored by the restoration unit when addition reading is selected by the selection unit. In Claim 1, the color temperature calculation means which calculates the color temperature of an imaging light source further based on the color signal discriminated as being contained in the predetermined evaluation range by the above-mentioned discrimination means,
Matrix conversion means for color-converting the color signal adjusted by the white balance adjustment means using a matrix,
The matrix conversion means has a plurality of different matrices, selects one of the plurality of matrices according to the color temperature calculated by the color temperature calculation means, and uses the selected matrix to select a color An imaging apparatus characterized by performing conversion. 3. The imaging apparatus according to claim 2, wherein the matrix conversion means performs color conversion from Ma, G, Cy, Ye signals to R, G, B signals. 2. The image pickup apparatus according to claim 1, further comprising a low-pass filter that is provided in a preceding stage of the color signal evaluation unit and limits a color signal output from the image pickup device. In Claim 1, the color temperature calculation means which calculates the color temperature of an imaging light source further based on the color signal discriminated as being contained in the predetermined evaluation range by the above-mentioned discrimination means,
Matrix conversion means for color-converting the color signal adjusted by the white balance adjustment means using a matrix;
An imaging apparatus comprising: a creation unit that creates a matrix of the matrix calculation unit based on the color temperature calculated by the color temperature calculation unit. 2. The evaluation means according to claim 1, wherein the changing means changes the evaluation range in accordance with at least one of brightness of a light source, influence of external light and flash light on a subject, and shutter speed. An imaging device. Color signal evaluation means for evaluating a color signal output from an image sensor having a plurality of types of color filters;
In the reading of the color signal output from the image sensor, a selection unit that selects addition reading and non-addition reading;
When addition reading is selected by the selection means, restoration means for restoring to a color signal output at the time of non-addition reading;
Discriminating means for discriminating whether or not the color signal evaluated by the color signal evaluating means is included in a predetermined evaluation range;
White balance adjusting means for adjusting white balance based on a color signal determined to be included in a predetermined evaluation range by the determining means;
Change means for changing the evaluation range of the discrimination means according to a predetermined condition ,
The color signal evaluation unit evaluates the color signal restored by the restoration unit when addition reading is selected by the selection unit. The color temperature calculation unit according to claim 7 , further calculating a color temperature of the imaging light source based on a color signal determined to be included in a predetermined evaluation range by the determination unit;
Matrix conversion means for color-converting the color signal adjusted by the white balance adjustment means using a matrix,
The matrix conversion means has a plurality of different matrices, selects one of the plurality of matrices according to the color temperature calculated by the color temperature calculation means, and uses the selected matrix to select a color A signal processing device that performs conversion. 9. A signal processing apparatus according to claim 8 , wherein said matrix conversion means performs color conversion from Ma, G, Cy, Ye signals to R, G, B signals. 8. The signal processing apparatus according to claim 7 , further comprising a low-pass filter that is provided in a preceding stage of the color signal evaluation unit and limits a color signal output from the image sensor. The color temperature calculation unit according to claim 7 , further calculating a color temperature of the imaging light source based on a color signal determined to be included in a predetermined evaluation range by the determination unit;
Matrix conversion means for color-converting the color signal adjusted by the white balance adjustment means using a matrix;
A signal processing apparatus comprising: a creation unit that creates a matrix of the matrix calculation unit based on the color temperature calculated by the color temperature calculation unit. 8. The evaluation means according to claim 7 , wherein the changing means changes the evaluation range in accordance with at least one of brightness of a light source, influence of external light and flash light on a subject, and shutter speed. Signal processing device.

Images