JP4025962B2 - White balance adjustment - Google Patents

Description

分布比は以下の式（２）にて算出する。
【数２】

ここで、Ｒ_ｉｊ，Ｇ_ｉｊ，Ｂ_ｉｊは各色成分毎の階調値であり、「ｉ」は帯域の番号に該当し「ｊ」は画素を区別するための便宜的な番号である。また、Ｉｒｉ，Ｉｇｉ，Ｉｂｉは各帯域毎、各成分毎の積分値であり、ＲｉｃとＢｉｃは各帯域毎の分布比である。
【００５５】
ここで、上記積分値は色成分の階調値の積算値であることから、各帯域毎にどれほどの値の階調値がどれほど分布しているかを反映した値となっている。また、この積分値を使用して分布比を算出することは帯域毎の色成分毎の平均階調値を使用して分布比を算出することと等価である。帯域毎の色成分毎の平均階調値は、帯域内に属する画素の所定色成分の階調値を足し合わせ、画素数で除したものであり、色成分毎の画素数は各帯域で同数であるところ、分布比にした場合には分母と分子で画素数が相殺されて結局分布比に寄与するのは階調値を足し合わせた成分のみとなるからである。
【００５６】
この分布比Ｒｉｃは「帯域毎のＧ成分の平均値／帯域毎のＲ成分の平均値」，分布比Ｂｉｃは「帯域毎のＧ成分の平均値／帯域毎のＢ成分の平均値」であり、この分布比はホワイトバランスを調整する性質を持った係数となる。すなわち、ある画素のＲＧＢ色成分それぞれがその画素の属する帯域の各色成分の平均階調値と等しい場合、分布比ＲｉｃとＲ成分の階調値とを乗じると分母が相殺してＧ成分の平均値と等しくなり、分布比ＢｉｃとＢ成分の階調値とを乗じても分母が相殺してＧ成分の平均値と等しくなる。従って、当該画素の全色成分が等しくなり無彩色になる。
【００５７】
このように、分布比Ｒｉｃと分布比Ｂｉｃとはそれぞれ画素のＲ成分とＢ成分とに作用し、その帯域のＧ成分の平均値を基準としつつ当該基準に近づけるようにＲ成分とＢ成分とを相対的に変化させるので、ホワイトバランスを調整するための係数であると言える。従って、上記ステップＳ２３０まででは、帯域毎のホワイトバランス補正係数を算出したと言える。本発明においては、所定帯域の分布比を参照して画像全体のホワイトバランスを調整するとともに、画像内容や光源等の差異にかかわらずより好ましい結果を得るための補正を加えている。
【００５８】
ステップＳ２３０にて分布比データを取得すると、上記色成分分布情報取得モジュール３３ｂはステップＳ２３５にて以下の式（３）に従って分布比の組みの平均値Ｒ’_ｎｃ，Ｂ’_ｎｃを算出する。
【数３】

ここで、ｎは帯域の組番号であり「１〜５」の整数である。
【００５９】
図１１は、組みの平均値Ｒ’_ｎｃについて２つの帯域を組みにする様子を説明する説明図である。同図に示す上部のグラフは上述のようにして１０個の帯域に分割されたヒストグラムを示している。上記式（３）においては、ｎ番目の帯域と（１１−ｎ）番目の帯域についての平均を算出していることから、同図に示すように両端の帯域（１番目と１０番目）が組みとなり、両端から２番目の帯域（２番目と９番目）が組みとなるようにして順次組み合わせられる。Ｂ’_ｎｃについての組み合わせの考え方も同様である。
【００６０】
図１２は、分布比を組みにして平均化する際の考え方を説明する説明図であり、同図はＹＵＶ色空間のＵ−Ｙ平面に写像されたＲＧＢ色空間の色域を示す。同図において、ＲＧＢ色空間の色域は、高輝度領域で負のＵ方向に偏り、低輝度領域で正のＵ方向に偏っている。Ｖ−Ｙ平面に写像した場合も同様で、高輝度領域で負のＶ方向に偏り、低輝度領域で正のＶ方向に偏ることとなる。すなわち、高輝度領域と低輝度領域とで逆の性質を有しており、この性質を上記分布比で考えると高輝度領域と低輝度領域とで大きな差が発生することとなるが、両者を平均すればその差を相殺することができ、色域形状に起因する分布比の偏りを除去した当該分布比を評価することができる。尚、この偏りを除去するように分布比を平均化する考え方は、ＲＧＢ色空間とＹＵＶ色空間との関係から発生したものであり、多くの画像機器についてのホワイトバランス調整にて採用することができる。
【００６１】
ステップ２３５にて分布比の組みの平均値が算出されると、ステップＳ２４０にてＲ’_ｎｃが「１」より小さいか否かを判別し、同ステップＳ２４０にてＲ’_ｎｃが「１」より小さいと判別されたときにはステップＳ２４５にてＲ’_ｎｃに対して当該Ｒ’_ｎｃの逆数を代入する。このステップＳ２４０，ステップＳ２４５の判別処理は、ステップＳ２４０での判別結果にかかわらず、組み番号１〜５の総てについて、また、Ｂ’_ｎｃについても同様に独立に実行される。ここでの処理はＲ’_ｎｃとＢ’_ｎｃとの総てについて、「１」より小さければ「１」より大きな数に変換する処理をしているものであり、次の評価関数ｅ_ｎによる判別のための変換であるとともに、評価関数による判別後は元の値に戻される。
【００６２】
（４−３）ホワイトバランス制御処理：
ステップＳ２５０では、評価関数ｅ_ｎに対して、各組のＲ’_ｎｃとＢ’_ｎｃとを代入し、上記ホワイトバランス制御モジュール３３ｃは当該評価関数ｅ_ｎが最小となる組み番号ｎのＲ’_ｎｃとＢ’_ｎｃとの値を第１段階のホワイトバランス補正係数Ｒ’_ｃおよびＢ’_ｃとして決定する。この第１段階のホワイトバランス補正係数も上述の分布比と同様の性質を有しており、上記図３に示すホワイトバランス制御マトリクス３４ｂ８の所定成分となり得る係数である。すなわち、画像内の任意の画素のＲＧＢ成分を成分とする行列にホワイトバランス制御マトリクス３４ｂ８を乗じると、その画像のホワイトバランスを制御することができる。
【００６３】
ここで、評価関数が最小のものを選択する処理は、Ｒ’_ｎｃとＢ’_ｎｃとのそれぞれと「１」との差が最小のものを選択している処理であり、補正量が最も小さいものを選択していることになる。すなわち、ホワイトバランス調整においてはカラーフェリアを防止することが重要であり、補正量の大きなものをホワイトバランス補正係数としてしまうと大きな補正によってカラーフェリアが発生することが多いが、補正量が小さいものを選択しておけば、カラーフェリアの発生確率を非常に小さくすることができる。本出願人は、上述の処理によって決定されたホワイトバランス補正係数が非常に効果的にホワイトバランス調整可能であることを確認しているが、ここでは、カラーフェリアを防止することができればよく、評価関数が２番目に「１」に近いものを選択する構成にするなど適宜変更可能である。
【００６４】
このとき、例えば、所定の値域内であれば２番目，３番目に「１」に近いものを選択するようにするなどの工夫を施せばカラーフェリアを防止可能である。また、本発明においては、画像の一部の輝度帯域に属するデータに基づいてホワイトバランス補正係数を算出しており、上述のように補正量の小さいものを選択しているため、上述の従来例のように画像全体のＲＧＢ積分値を反映させるホワイトバランス調整と比較して、非常にカラーフェリアが発生しにくい。
【００６５】
（４−３−１）光源種別に対応した補正：
上述のように第１段階のホワイトバランス補正係数でホワイトバランス制御を行うことも可能であるが、本実施形態においては光源の差異に起因したカラーフェリアの発生を防止するために、さらに図５のステップＳ２５５以降の処理を実行している。ステップＳ２５５では、Ｇ成分の階調値「１２８」（中央値）を基準としたホワイトバランス補正係数となるように以下の式にて上記第１段階のホワイトバランス補正係数Ｒ’_ｃおよびＢ’_ｃを変換する。
Ｒ＝128／Ｒ’_ｃ，Ｇ＝128，Ｂ＝128／Ｂ’_ｃ
【００６６】
ステップＳ２６０では、当該変換後の係数ＲＧＢを以下の式（４）に従ってＥｓｔｅｖｅｚ−Ｈｕｎｔ−Ｐｏｉｎｔｅｒの式にて生理的三原色の空間Ｒ’Ｇ’Ｂ’に変換する。
【数４】

【００６７】
ここで、マトリクスＡは上記ＲＯＭ３３に記憶されたマトリクスであり、ｓＲＧＢ座標を生理的三原色のＲＧＢ座標に変換する。また、マトリクスＡはマトリクスＢとマトリクスＣとを乗じて生成されたものであり、それぞれのマトリクスは上述した値であるとともにマトリクスＢはｓＲＧＢ座標をＸＹＺ座標に変換する変換マトリクスであり、マトリクスＣはＸＹＺ座標を生理的三原色のＲＧＢ座標に変換するマトリクスである。尚、上記式において斜体の文字はマトリクスを示している。ステップＳ２６０にて生理的三原色の空間データＲ’Ｇ’Ｂ’が得られると、ステップＳ２６５にて以下の式に従って最終的なホワイトバランス補正係数Ｒｃ，Ｂｃを決定する。
Ｒｃ＝Ｇ’／Ｒ’，Ｂｃ＝Ｇ’／Ｂ’
【００６８】
これらステップＳ２５５〜Ｓ２６５の処理は、ｓＲＧＢ座標を一旦生理的三原色のＲ’Ｇ’Ｂ’座標に変換し、当該変換後の空間でホワイトバランス補正係数を補正（ステップＳ２６５）していることに該当する。この処理によって人間の見た目に応じた良好なホワイトバランスに近づけることができ、光源種に依存しないホワイトバランス制御を実施することができる。また、最終的なホワイトバランス補正係数Ｒｃ，Ｂｃをホワイトバランス制御マトリクスＷに組み込むことによって、第２色空間の座標を補正してホワイトバランス調整を実行するようにしている。図１３，図１４は生理的三原色に変換してホワイトバランス補正係数を補正する様子を説明する説明図である。
【００６９】
図１３はｓＲＧＢの理想光源（ＣＩＥ標準の光源Ｄ６５）の分光分布を示す図であり、図１４は生理的三原色についての分光分布を示す図である。図において横軸は波長（ｎｍ）であり、縦軸は分光分布の相対値である。尚、図１３，図１４において４５０ｎｍ付近にピークがあるカーブはＢ成分、５４０ｎｍ付近にピークがあるカーブはＧ成分の分光分布であり、図１３の６１０ｎｍ付近，図１４の５８０ｎｍ付近にピークがあるカーブはＲ成分の分光分布である。上記ホワイトバランス補正係数はＲ成分とＢ成分とに対してそれぞれ乗ぜられるので、ホワイトバランス補正係数による調整は、上記図１３，１４における分光分布に対してはＢ成分とＲ成分のカーブを拡大したり縮小したりするように作用する。
【００７０】
図１４に示す分光分布は生理的三原色についての分布であることから、人間の色の感じ方はこの分布に非常に近く、この分布となるＲＧＢ空間でホワイトバランス補正係数の補正を行うと、人間の色の感じ方に非常に近い形でホワイトバランスを制御することができる。具体的には、図１３に示す分光分布においてはＲ成分のピーク波長およびピーク強度が図１４に示す分光分布のＲ成分のピーク波長およびピーク強度より大きい。従って、上記ステップＳ２６０における変換を実施せずにホワイトバランスの調整をしようとすると、Ｒ成分に対する補正が相対的に大きくなり、Ｒ成分を強調しすぎたり、弱めすぎたりしてしまう。そこで、上記ステップＳ２６０における変換を実施した後にホワイトバランス補正係数を補正することにより、より見た目に近い赤となるようにホワイトバランスを調整可能である。むろん、青に関しても同様で見た目に近い青となる。従って、白熱灯かでの撮影画像等であっても適切なホワイトバランスとなるよう調整されるようにホワイトバランス補正係数を決定可能である。
【００７１】
ステップＳ２６５にて最終的なホワイトバランス補正係数Ｒｃ，Ｂｃを決定したら、ステップＳ２７０において当該ホワイトバランス補正係数Ｒｃ，Ｂｃを１行１列目および３行３列目の成分とし、２行２列目の成分を「１」とし、他の成分を「０」としたマトリクスの左から上記逆変換マトリクスＡ^−１を乗じ、右から上記マトリクスＡを乗じてホワイトバランス制御マトリクスＷ（３４ｂ９）を生成し、ステップＳ２７５にて当該ホワイトバランス制御マトリクスＷを上記ＲＡＭ３４のワークエリア３４ｂに保存する。
【００７２】
尚、逆変換マトリクスＡ^−１は以下の式（５）にて表現される。
【数５】

上記式においても斜体の文字はマトリクスを示している。
【００７３】
このホワイトバランス制御マトリクスＷの算出／保存までがホワイトバランスの測定に該当し、ディジタルスチルカメラ１０の処理としてはこの後図２に示すフローチャートに戻ってステップＳ１０６以降の処理を実行する。そして、ステップＳ１１０においてＣＣＤＲａｗデータをｓＲＧＢデータに変換した画像データに対してホワイトバランス制御マトリクスＷを乗じることによって個々の撮像画像のホワイトバランスを制御する。
【００７４】
すなわち、ホワイトバランス制御マトリクスＷは、ｓＲＧＢ座標で表現されたカラー画像データの画像についてホワイトバランスを調整するように作用するマトリクスであり、入力および出力データはｓＲＧＢデータである。上記ホワイトバランス補正係数はｓＲＧＢに対する補正係数ではないため、上記マトリクスＡ等による空間の変換を行って入出力データがｓＲＧＢデータになるようにしてある。従って、ステップＳ２７０，Ｓ２７５およびＳ１１０の処理が上記マトリクス演算部３３ｅでの処理に該当する。図１５は、かかるホワイトバランス制御マトリクスＷによる演算を説明するための説明図であり、上部のマトリクス演算式において右辺のＲＧＢは入力ｓＲＧＢデータであり、左辺のＲ’’Ｇ’’Ｂ’’は出力ｓＲＧＢデータである。また、本実施形態の実際の処理においてホワイトバランス制御マトリクスＷは３個の３×３の行列を予め計算して得られる１個の３×３の行列であるが、ここでは便宜上３個のマトリクスに分けて説明する。
【００７５】
図１５においてマトリクス演算式の下方には３個のマトリクスによる各演算段階を示している。上述のように入力データ▲１▼はｓＲＧＢデータである。マトリクスＡはｓＲＧＢ座標系のＲＧＢ色成分を生理的三原色のＲＧＢ色成分に変換するマトリクスであるので、当該入力データ▲１▼にマトリクスＡを乗じると、▲２▼に示すように生理的三原色のＲＧＢ色成分に変換される。すなわち、この変換が上記第１色空間変換部３３ｅ１での処理である。この変換結果に対して上記ホワイトバランス補正係数Ｒｃ，Ｂｃを成分として含むマトリクスを乗じると、▲３▼に示すように生理的三原色のＲＧＢ色成分にてホワイトバランスが調整される。すなわち、ここでの処理が上記ホワイトバランス制御部３３ｅ２での処理である。
【００７６】
この調整後のＲＧＢデータに逆変換マトリクスＡ^−１を乗じると、▲４▼に示したようにホワイトバランス調整後の画像データがｓＲＧＢデータに変換される。すなわち、ここでの処理が上記第２色空間変換部３３ｅ３での処理である。本実施形態においてこの演算は一つのマトリクスで実現されており、１回の演算で第１色空間内のｓＲＧＢ座標を第２色空間としての生理的三原色のＲＧＢ色成分に変換し、当該第２色空間内で補正係数にてホワイトバランスを調整し、さらに補正後のＲＧＢ色成分を第１色空間内のｓＲＧＢ座標に戻している。従って、ディジタルスチルカメラの画像処理過程においてｓＲＧＢデータに１個の３×３の行列であるホワイトバランス制御マトリクスＷを乗じるとホワイトバランスが調整されたｓＲＧＢデータとなり、しかもこのデータは人間の目の特性に近い空間で補正されているので、光源に影響を受けずに確実にホワイトバランスを調整することができる。
【００７７】
以上説明したように、本発明においては第１色空間の座標を第２色空間の座標に変換し、同変換された第２色空間の座標を補正することによりホワイトバランスを制御し、上記補正後の座標を上記第１色空間の座標に変換する。従って、画像内の特定色や特定色に偏った光源に影響されることなくホワイトバランス調整を実施可能となる。
【図面の簡単な説明】
【図１】ディジタルスチルカメラの概略構成を示すブロック図である。
【図２】画像処理の主要な流れを示すフローチャートである。
【図３】ホワイトバランスの測定系及び制御系を示すブロック図である。
【図４】ホワイトバランス測定及び制御を示すフローチャートである。
【図５】ホワイトバランス測定及び制御を示すフローチャートである。
【図６】間引き処理を説明するための説明図である。
【図７】ＲＧＢＹｓ変換後の画素データを示す図である。
【図８】ヒストグラムの一例を示すグラフである。
【図９】帯域分割の様子を説明する説明図である。
【図１０】積分値と分布比とを算出する様子を説明する説明図である。
【図１１】２つの帯域を組みにする様子を説明する説明図である。
【図１２】分布比を組みにして平均化する際の考え方を説明する説明図である。
【図１３】ｓＲＧＢの理想光源の分光分布を示す図である。
【図１４】生理的三原色についての分光分布を示す図である。
【図１５】ホワイトバランス制御マトリクスによる演算を説明するための説明図である。
【符号の説明】
１０…ディジタルスチルカメラ
２０…光学部
２１…光学レンズ系
２２…オートフォーカス機構
２３…測距部
２４…オートフォーカスコントローラ
２５…コントローラ
２６…ＣＣＤ
２７…ストロボ
２８…オートゲインコントローラ（ＡＧＣ）
２９…Ａ／Ｄコンバータ
３０…制御部
３１…バス
３２…ＣＰＵ
３３…ＲＯＭ
３３ａ…輝度成分取得モジュール
３３ｂ…色成分分布情報取得モジュール
３３ｃ…ホワイトバランス制御モジュール
３３ｄ１…変換マトリクスＡ
３３ｄ２…逆変換マトリクスＡ^−１
３３ｅ…マトリクス演算部
３３ｅ１…第１色空間変換部
３３ｅ２…ホワイトバランス制御部
３３ｅ３…第２色空間変換部
３４…ＲＡＭ
３４ａ…画像エリア
３４ｂ…ワークエリア
３４ｂ１…間引き済データ
３４ｂ２…ＲＧＢＹｓデータ
３４ｂ３…ヒストグラムデータ
３４ｂ４…帯域データ
３４ｂ５…ＲＧＢ積分データ
３４ｂ６…分布比データ
３４ｂ７…補正係数Ｒ’_ｃ，Ｂ’_ｃ
３４ｂ８…ホワイトバランス制御マトリクス
３４ｂ９…ホワイトバランス制御マトリクス
３４ｃ…ビデオＲＡＭエリア
３５…操作パネル
３６…ＬＣＤパネル
３７ａ…Ｉ／Ｏ
３７ｂ…フラッシュメモリカード[0001]
BACKGROUND OF THE INVENTION
The present invention relates to white balance adjustment .
[0002]
[Prior art]
The human eye adjusts the color component by chromatic adaptation and can recognize a white object as white even if the light source changes. However, in an image device such as a digital camera, the image sensor itself does not have a chromatic adaptation function. Therefore, the color balance is conventionally adjusted by a technique called auto white balance. For example, when the color image data is composed of three components of R (red), G (green), and B (blue), the gradation values of the three components are integrated over the entire image, and the integration ratio is R : G: B = 1: 1: 1 The gradation value of each color component is corrected.
[0003]
[Problems to be solved by the invention]
In the above-described conventional digital camera, correction is performed so that the average color of the entire image becomes gray based on the hypothesis that adding all the color components of the subject results in an achromatic color. If there is a part with a large specific color or a captured image under a light source whose spectral distribution is biased to the specific color, a color feria in which white balance adjustment fails due to the influence of the specific color occurs.
The present invention has been made in view of the above problems, and an object of the present invention is to perform white balance adjustment capable of performing white balance adjustment without being affected by a specific color in the image or a light source biased to the specific color.
[0004]
[Means for Solving the Problems]
In order to achieve the above object, the invention according to claim 1 is a white balance adjustment device that adjusts the white balance of an image in which colors are expressed by coordinates in RGB space for a plurality of pixels, and each color component in the image A first distribution ratio calculating means for calculating a first distribution ratio indicating the balance of the first color space by coordinates of the first color space, and a physiological function in which the first distribution ratio is reflected on a cone sensitivity characteristic of the human eye by a predetermined conversion formula. Second distribution ratio calculation means for converting to a second distribution ratio indicated by the coordinates of the three primary color spaces, white balance correction coefficient calculation means for calculating a white balance correction coefficient based on the second distribution ratio, and the first The conversion formula for converting the coordinates of the color space to the coordinates of the color space of the physiological three primary colors, the conversion formula based on the white balance correction coefficient, and the coordinates of the color space of the physiological three primary colors are And white balance control matrix generating means for generating a white balance control matrix obtained by synthesizing the conversion formula for converting the coordinates of a color space, white for controlling the white balance of the image by converting the image by the white balance control matrix And a balance control means.
[0005]
According to a second aspect of the present invention, the first distribution ratio calculating means converts the gradation of each color component in the image into a value based on the central value of these gradations, thereby obtaining the first distribution ratio. This is a configuration to calculate .
[0016]
By the way, such a white balance adjustment device may exist alone, or may be used in a state of being incorporated in a certain device, but the idea of the invention is not limited to this and includes various aspects. It is a waste. Therefore, it can be changed as appropriate, such as software or hardware. As an embodiment of the idea of the invention, it corresponds to the case where the software of the white balance adjustment device is used, and the invention according to claims 6 and 7 has a configuration corresponding to each function for causing the white balance adjustment device to be implemented by a computer. . Needless to say, it is also effective as a program corresponding to claim 2 .
[0017]
Of course, the software recording medium may be a magnetic recording medium, a magneto-optical recording medium, or any recording medium to be developed in the future. In addition, the duplication stages such as the primary duplication product and the secondary duplication product are equivalent without any question. In addition, although it is not the above-mentioned medium, the present invention is not changed even when the communication method is used as a supply method. Further, even when a part is software and a part is realized by hardware, there is nothing completely different in the idea of the invention, and a part is stored on a recording medium, and it is appropriately changed as necessary. It may be in the form of being read.
[0018]
In addition, it is natural that such a white balance adjustment program can be applied as a method as a matter of course, as the process proceeds according to such control, an invention exists in the procedure. it can. That is, it is not necessarily limited to a substantial apparatus, and there is no difference that the method is effective. Of course, it is also effective as a method corresponding to the second aspect . Further, the present invention can be applied to a specific apparatus, and the present invention can be realized as a digital camera equipped with each means capable of performing white balance adjustment.
[0019]
【The invention's effect】
As described above, the inventions according to claims 1 to 3 provide a white balance adjustment device, a white balance adjustment program, and a white balance adjustment method capable of correcting color components in a space suitable for white balance adjustment. can do.
In the invention according to claim 2, the present invention can be applied to various image devices.
[0021]
DETAILED DESCRIPTION OF THE INVENTION
Here, embodiments of the present invention will be described in the following order.
(1) Schematic configuration of digital still camera:
(2) Outline of image processing:
(3) Configuration for white balance measurement and control:
(4) White balance measurement and control processing:
(4-1) Luminance component acquisition processing:
(4-2) Color component distribution information acquisition process:
(4-3) White balance control processing:
(4-3-1) Correction corresponding to light source type:
[0022]
(1) Schematic configuration of digital still camera:
FIG. 1 is a block diagram showing a schematic configuration of a digital still camera to which white balance control of the present invention is applied.
The digital still camera 10 includes an optical unit 20 and a control unit 30 having a CPU 32 as a core. The optical unit 20 includes an optical lens system 21, an autofocus mechanism 22, a distance measuring unit 23, and an autofocus controller 24, and the autofocus controller 24 controls the distance measuring unit 23 based on a control signal from the controller 25. The optical lens system 21 is driven by the autofocus mechanism 22 while the distance to the subject is measured. A subject image is formed by the optical lens system 21 on the image pickup surface of a CCD 26 as an image pickup element, and the CCD 26 is constituted by a single plate having 1800 × 1200 pixels. Since it is composed of a single plate, green (G), magenta (M), yellow (Y), and cyan (C) color filters are sequentially formed for 2 × 2 pixels. In addition, although the shutter is provided with the mechanical shutter with the electronic shutter, illustration is abbreviate | omitted about the mechanical shutter.
[0023]
In addition, the optical unit 20 is also provided with a strobe 27, which emits a predetermined amount of light and a predetermined number of times according to a drive signal from the controller 25. This realizes shooting such as red-eye prevention or slow sync. Note that an auto gain controller (AGC) 28 for amplifying an analog electric signal output from the CCD 26 is provided, and an A / D converter 29 for converting the amplified analog electric signal into a digital value and outputting the digital value. . Of course, the optical unit 20 described above is merely an example of a general configuration as the digital still camera 10, and it is needless to say that various changes can be made. For example, the optical lens system 21 may include a zoom lens, or may be an inexpensive one that does not have an autofocus function as a fixed focus.
[0024]
The control unit 30 includes a bus 31, and the CPU 32, the ROM 33, the RAM 34, and the A / D converter 29 are connected to the bus 31. The CPU 32 executes firmware written in the ROM 33 to execute control of the optical unit 20 and calculation of image processing. At this time, the RAM 34 functions as an image area 34a for storing image data or performs calculation processing. Function as a work area 34b. When the CPU 32 controls the optical unit 20, it outputs a control signal to the controller 25, and the controller 25 generates and outputs an appropriate control signal for each component circuit.
[0025]
The control unit 30 also has a function as a man-machine interface, and includes an operation panel 35 on which operation buttons and the like are arranged, and an LCD panel 36 for displaying captured images and operation instructions. The operation is monitored, and the corresponding control is executed while receiving the operation as appropriate. Further, the display on the LCD panel 36 is performed by the CPU 32 writing predetermined data in the video RAM area 34c allocated to the RAM 34, so that the display controller provided in the LCD panel 36 appropriately reads the data and displays it. . Then, image data that has been shot and generated through predetermined image processing is written to the flash memory card 37b, which is an external memory, via the I / O 37a, and is read out from the flash memory card 37b as necessary.
[0026]
The configuration of the control unit 30 is also a typical example of this type of digital still camera 10, and various changes can be made. For example, the type of external memory can be easily replaced with another type as appropriate, and image data can be output to a computer via a USB interface even without an interface with a removable memory. It may be. Another LCD display may be provided in association with the LCD panel 36 as a display unit, and the CPU 32 may be provided with a plurality of CPUs having different control targets.
[0027]
The image picked up by the optical unit 20 is output from the CCD 26 as an analog electric signal, then converted into digital data by the A / D converter 29 via the AGC 28, and further recorded in the image area 34 a of the RAM 34 via the bus 31. The Then, the CPU 32 executes predetermined image processing to convert it into JPEG image data and write it into the above-described flash memory card 37b.
[0028]
(2) Outline of image processing:
FIG. 2 shows the main flow of this image processing. Although detailed processing as described below is executed in detail, the processing is generally proceeded according to this flow. First, this outline will be described. In step S102, defective pixel interpolation is executed. A defective pixel is inevitable in the CCD 26 having about 2 million pixels, and if it is below a predetermined reference value, it is supplied as a non-defective product. For this reason, defective pixels within a non-defective range are compensated by using a 5 × 5 pixel median filter based on only the same color pixels. That is, if the defective pixel is a cyan pixel, the output values of 5 × 5 cyan pixels located around the defective pixel are arranged in order, and the median value is set as the value of the defective pixel. The defective pixel itself has already been detected by the optical unit 20.
[0029]
In the next step S104, white balance is measured. Depending on the type of light source at the time of shooting and the combination of the CCD 26 and the color filter, the white balance may be shifted. In this embodiment, the white balance is measured at step S104 for each shooting, and the result is reflected. In step S110, the white balance is controlled so as to reduce the deviation. When measuring the white balance, 113 × 75 pixel image data selected in the thinning process is used to measure the bias of the R component and the bias of the B component in the RGB color system based on the G component. The CCD 26 includes a complementary color filter, and performs control based on the fact that it is a complementary color system.
[0030]
In this embodiment, three types of images are generated. The first is a confirmation image, the second is a thumbnail image, and the third is a main image. The confirmation image is an image to be displayed on the LCD panel 36 immediately after shooting, and is composed of image data of 720 × 240 pixels. The thumbnail image is recorded as image data together with the main image, and is composed of image data of 160 × 120 pixels. The main image can be selected in three sizes according to the selection of the operator, and is either 720 × 480 pixels as an E-mail image, 1800 × 1200 pixels as a Print image, or 2160 × 1440 pixels with high resolution as Hypic2 It is.
[0031]
In FIG. 2, Steps S106 to S114 are displayed surrounded by broken lines, and at least these processes are performed for each of the three types of generated images. In step S106, data interpolation is performed. Although the CCD 26 is composed of a single plate and has 2 million pixels, the color information is not completely completed for all the pixels. That is, there is only cyan information for some pixels, only magenta information for some pixels, only yellow information for some pixels, and only green information for some pixels. This data is called mosaic data. Data interpolation is a process for completing the color information for each pixel by supplementing the information of other colors that are insufficient for each pixel. The lacking information is the average value of the color information included in the eight pixels surrounding the target pixel. For example, there are two yellow pixels (Y1, Y2), two magenta pixels (M1, M2), and four green pixels (G1-G4) around the cyan pixel (C1).
[0032]
Therefore, for the color information (C, M, Y, G) of this cyan pixel,
C = C1
M = (M1 + M2) / 2
Y = (Y1 + Y2) / 2
G = (G1 + G2 + G3 + G4) / 4
Calculate as
[0033]
In step S108, the color system is changed and converted to RGB image data. There are general formulas for conversion, but actually, a simple general formula is not used, but conversion is performed using a tuned lookup table or a determinant. After conversion into RGB image data, in step S110, as described above, control is performed to reduce the bias of the R component and the bias of the B component in the RGB color system. In the next step S112, tone curve correction is performed. Tone curve correction is also referred to as so-called γ correction. At this time, 10-bit image data is converted into 8-bit image data for each component, and further noise reduction and gradation characteristics in the white side region and the black side region are performed. The γ curve itself is also adjusted to maintain The tone curve correction itself uses a look-up table, and is converted with reference to a 10-bit → 8-bit look-up table created by tuning in advance.
[0034]
Thereafter, color conversion is performed from the RGB color system to the YUV color system in order to use the JPEG image compression technique. In order to distinguish between brightness and yellow, the brightness is hereinafter displayed as Ym, and the brightness directly derived from the mosaic color data is displayed as simple brightness Ys as described later. The color conversion from the RGB color system to the YUV color system is performed using a general formula or a determinant.
The general formula for conversion from mosaic color data to YUV color system is:
Ys = 0.25 × (C + M + Y + G)
U = C + M- (Y + G)
V = -C + M + Y-G
However, in practice, the tuning and white balance control as described above are performed, and this correspondence is not adopted.
[0035]
Steps S116 and S118 are processes performed only when the main image is created. Noise removal is performed by applying a so-called low-pass filter only for the UV component as the color component. In other words, the color change that is too steep acts on the pressing feeling. Further, edge enhancement is performed by calculating a nonlinear function value (for example, a coring function value) from an edge amount based on the luminance component only for the luminance component, and weighting the non-linear function value to the original luminance component. Therefore, when the luminance value changes between adjacent pixels, the edge amount is calculated, and if the absolute value of the edge amount is large, the amount of change is reflected in the luminance value. If the absolute value is small, the original luminance is calculated. The value is almost unchanged.
[0036]
The above is the outline of the image processing. However, the above procedure also shortens the calculation processing time by simultaneously performing a plurality of processes in the actual calculation processing. The shooting is completed by executing the above-described image processing for each of the three types of images described above, and finally writing the JPEG compressed image of the main image including the thumbnail image in the flash memory card 37b. In addition, this digital still camera can read image data from the flash memory card 37b and display it on the LCD panel 36, or can delete data and perform various settings. The description is omitted because it is applicable.
[0037]
(3) Configuration for white balance measurement and control:
Hereinafter, the white balance measurement in step S104 of the image processing and the white balance control reflecting the measurement result will be described in detail. FIG. 3 is a block diagram showing a white balance measurement system and control system realized by the configuration shown in FIG. The luminance component acquisition module 33a, the color component distribution information acquisition module 33b, and the white balance control module 33c in the figure are executed when the CPU 32 appropriately reads out program modules recorded in advance in the ROM 33.
[0038]
The white balance control module 33c further includes a matrix calculation unit 33e, and the matrix calculation unit 33e includes a first color space conversion unit 33e1, a white balance control unit 33e2, and a second color space conversion unit 33e3. Each module appropriately uses data recorded in the image area 34a and work area 34b of the RAM 34, and determines a matrix for writing intermediate data and controlling white balance as necessary. White balance control is realized by multiplying image data by this matrix.
[0039]
The luminance component acquisition module 33a acquires CCD raw data recorded in the image area 34a and acquires luminance components. Here, the luminance component acquisition module 33a performs a thinning process for reducing the amount of calculation, a process for converting GMYC Raw data to RGBYs data, a histogram process, and a band division process for the histogram. The thinned data 34 b 1, RGBYs data 34 b 2, histogram data 34 b 3, and band data 34 b 4 generated and used as appropriate are recorded in the work area 34 b of the RAM 34. Each processing will be described in detail later. A digital still camera equipped with a primary color filter does not require conversion from GMYC data to RGB data, but correction processing is required to approximate the CCD RGB data to the characteristics of sRGB data.
[0040]
The color component distribution information acquisition module 33b includes the RGB color component integration data 34b5 as the color component distribution information within the divided band and suitable as an index for adjusting the white balance, and a distribution ratio that is a ratio thereof. Data 34b6 is acquired. From the obtained distribution ratio data 34b6, the white balance control module 33c is a coefficient for use in white balance adjustment, and a correction coefficient R ′ _c for correcting the R component relative to the G component, and G A correction coefficient B ′ _c (34b7) for correcting the B component relative to the component is obtained. Then, the correction coefficient is converted into a coefficient in the color space of the physiological three primary colors and corrected in the color space to calculate the correction coefficients R _c and B _c , and the final correction is performed based on the correction coefficient. The white balance control matrix 34b9 is calculated.
[0041]
The ROM 33 carries out a conversion matrix A (33d1) for converting the RGB color components of the sRGB coordinate system into the RGB color components of the physiological three primary colors and the inverse conversion of the conversion based on the equation of Estevez-Hunt-Pointer. The inverse transformation matrix A ⁻¹ (33d2) to be recorded is recorded and used in the white balance control module 33c. Multiplying the white balance control matrix 34b9 created as described above and the CCD Raw data in the image area 34a converted into RGB color components, the RGB color component data for which the white balance of the image is controlled is obtained. Obtainable.
[0042]
In the present embodiment, the white balance is controlled by the white balance control matrix 34b9 as described above, and the actually stored matrix is a 3 × 3 matrix. This matrix is a correction coefficient obtained by an operation described later. It is obtained by multiplying R _c , B _c , the conversion matrix A, and the inverse conversion matrix A ^−1, and the calculation by this matrix is the first color space conversion means, white balance control means, and second color space conversion. It can be said that the means are realized at once.
[0043]
In this embodiment, the white balance correction coefficient is obtained in two stages. That is, once the correction coefficients R ′ _c and B ′ _c are obtained as the first stage, the correction coefficients R _c and B _c are further obtained as the second stage. By obtaining the correction coefficient in two stages in this way, white balance control fails due to the influence of a large monochromatic part in the image, or white balance control fails when a specific light source is used. Color feria and the like can be surely prevented, but each of the above steps can function as white balance control even if performed alone.
[0044]
Moreover, it is good also as a structure which adds each step | paragraph with respect to the white balance control of another method, and controls white balance. For example, even if the white balance control matrix 34b8 is calculated using the correction coefficients R ′ _c and B ′ _c calculated in the first stage, it functions as a white balance adjustment device. The white balance control according to the present invention is mainly for the second stage.
[0045]
(4) White balance measurement and control processing:
Hereinafter, a process performed for white balance measurement and control in the above configuration will be described in detail. 4 and 5 are flowcharts showing white balance measurement and control. FIG. 4 corresponds to the first stage, and FIG. 5 corresponds to the second stage and the stage of calculating the final white balance control matrix 34b9. 4 corresponds to the process of the luminance component acquisition module 33a, and steps S225 to S250 of FIG. 4 correspond to the process of the color component distribution information acquisition module 33b. Steps S250 to S275 and S110 correspond to the processing of the white balance control module 33c. Here, a series of processes will be described while being divided into processes for each module.
[0046]
(4-1) Luminance component acquisition processing:
When the luminance component acquisition module 33a acquires the CCD Raw data in the image area 34a in Step S200, the thinning process is performed on the CCD Raw data in Step S205. That is, the CCD Raw data is data having 1800 × 1200 pixels as described above, and it takes a long time to execute the subsequent processing for all the pixels, and the free storage capacity required for the RAM 34 is very large. Therefore, pixels are thinned out as follows.
[0047]
FIG. 6 is an explanatory diagram for explaining the thinning process. The square on the right side of the figure is a schematic diagram of CCD Raw data, which is GMYC mosaic data of 1800 × 1200 pixels. In the present embodiment, image data having a size 1/16 of the mosaic data is used. First, a 16 × 16 pixel group (Gr1) at the upper left of the mosaic data in the figure is acquired. The pixel group Gr1 includes 64 pieces of GMYC color component data, and an average of gradation values of 64 pieces of data is taken for each color component. These averages are set as color component data | G | ₁ , | M | ₁ , | Y | ₁ , | C | ₁ for one pixel.
[0048]
Thereafter, a 16 × 16 pixel group is sequentially obtained and averaged from the CCD Raw data in the same manner. As a result, the CCD Raw data is a gradation value data having a size of 1/16, that is, 113 × 75 pixels. This gradation value data is stored in the work area 34b as thinned data 34b1. Of course, in this thinning-out process, the compression ratio 1/16 is not an indispensable value, and can be appropriately changed according to the RAM 34, processing load, and the like. In addition, although averaging as described above is preferable because all information of the CCD Raw data can be taken into consideration, there are various other methods such as thinning out so that one pixel is left for each 16 pixels in the vertical and horizontal directions of the CCD Raw data. Can be adopted.
[0049]
In the present embodiment, white balance adjustment is considered in the RGB space, and the thinned data 34b1 after the thinning process in step S205 is converted into RGB components and simple luminance Ys in step S210. The conversion from the GMYC component to the RGB component uses a conversion equation similar to the conversion equation in step S108. In this conversion equation, the color conversion is performed with high accuracy by tuning as described above. It can be carried out. FIG. 7 shows the pixel data (RGBYs data 34b2) after the conversion in step S210. The pixel data after conversion is 113 × 75 pixels as shown in the figure, and is composed of RGBYs four-component gradation value data for each pixel.
[0050]
The luminance component acquisition module 33a acquires the luminance component of the pixel at this time point. However, in this embodiment, for the subsequent processing, in step S215, the simple luminance Ys of each pixel is converted into a histogram for each gradation value, and the histogram data 34b3. Is generated. FIG. 8 is a graph showing an example of this histogram. In the figure, since the gradation value pitch is fine, it is shown as a continuous curve, but the actual histogram is a discrete value. The following histogram graphs are similarly shown as curves, but are actually discrete values.
[0051]
When a histogram is created in step S215, the histogram is divided into 10 luminance bands in step S220. FIG. 9 is an explanatory diagram for explaining the state of band division in step S220. The table at the top of the figure shows the work of determining the band, and the graph at the bottom shows a histogram in which the band is divided. When dividing the band, as shown in the upper table, the histogram is accumulated from the gradation value of the simple luminance Ys “0”, and the accumulated value exceeds 847 (about 113 × 75/10). To move to the next band and repeat the integration.
[0052]
If such integration is repeated for 10 bands, the number of pixels belonging to each band becomes substantially equal. That is, the area of each band shown in the graph of FIG. 9 is substantially equal. The data after the band division is stored in the work area 34b as band data 34b4 that can determine which pixel belongs to which band. Here, since the histogram is a discrete value, the number of pixels belonging to each band is not necessarily exactly the same, but since there is a distribution ratio calculation process by the color component distribution information acquisition module 33b described later, the number of pixels The strictness of is not a problem. When the integrated value exceeds the above 847, the pixel of the gradation value may be included in only one band or both. Further, in this band division, it is only necessary to specify a suitable luminance band to be used later for white balance adjustment, and it is not essential that the number of divisions is 10, and the division method is as described above. Not limited to.
[0053]
(4-2) Color component distribution information acquisition process:
The color component distribution information acquisition module 33b acquires color distribution information from a band preferable for white balance control based on the band data 34b4. For this purpose, first in step S225, the RGB components of the pixels belonging to the same band are integrated for each of the divided bands, and are stored in the work area 34b as RGB integration data 34b5. In step S230, “integrated value of G component / integrated value of R component” and “integrated value of G component / integrated value of B component” are obtained for each band as the distribution ratio, and the above work area is obtained as distribution ratio data 34b6. 34b. FIG. 10 is an explanatory diagram for explaining how the integral value and the distribution ratio are calculated. Since it is possible to determine which pixel belongs to which band in the band data 34b4, the RGB components of the pixels belonging to each band can be easily known as shown in FIG.
[0054]
From this RGB component value, the integral value is calculated by the following equation (1),
[Expression 1]

The distribution ratio is calculated by the following formula (2).
[Expression 2]

Here, R _ij , G _ij , and B _ij are gradation values for each color component, “i” corresponds to a band number, and “j” is a convenient number for distinguishing pixels. Iri, Igi, and Ibi are integral values for each band and each component, and Ric and Bic are distribution ratios for each band.
[0055]
Here, since the integrated value is an integrated value of the gradation values of the color components, it is a value reflecting how many gradation values are distributed in each band. Also, calculating the distribution ratio using this integral value is equivalent to calculating the distribution ratio using the average gradation value for each color component in each band. The average gradation value for each color component in each band is the sum of the gradation values of the predetermined color components of the pixels belonging to the band and divided by the number of pixels. The number of pixels for each color component is the same in each band. However, in the case of the distribution ratio, the number of pixels is canceled out by the denominator and the numerator, and the only contribution to the distribution ratio is that only the component obtained by adding the gradation values.
[0056]
This distribution ratio Ric is “average value of G component for each band / average value of R component for each band”, and distribution ratio Bic is “average value of G component for each band / average value of B component for each band”. This distribution ratio is a coefficient having the property of adjusting the white balance. That is, when each RGB color component of a certain pixel is equal to the average gradation value of each color component in the band to which the pixel belongs, multiplying the distribution ratio Ric and the gradation value of the R component cancels the denominator and averages the G component. Even if the distribution ratio Bic and the gradation value of the B component are multiplied, the denominator cancels and becomes equal to the average value of the G component. Therefore, all the color components of the pixel are equal and become achromatic.
[0057]
As described above, the distribution ratio Ric and the distribution ratio Bic act on the R component and the B component of the pixel, respectively, and the R component and the B component are brought close to the reference while using the average value of the G component in the band as a reference. Can be said to be a coefficient for adjusting the white balance. Therefore, it can be said that the white balance correction coefficient for each band has been calculated up to step S230. In the present invention, the white balance of the entire image is adjusted with reference to the distribution ratio of the predetermined band, and correction is performed to obtain a more preferable result regardless of differences in image content, light source, and the like.
[0058]
When the distribution ratio data is acquired in step S230, the color component distribution information acquisition module 33b calculates average values R ′ _nc and B ′ _nc of the set of distribution ratios according to the following equation (3) in step S235.
[Equation 3]

Here, n is a set number of the band and is an integer of “1-5”.
[0059]
FIG. 11 is an explanatory diagram for explaining a state in which two bands are grouped for the average value R ′ _nc of the group. The upper graph shown in the figure shows a histogram divided into 10 bands as described above. In the above formula (3), since the average for the nth band and the (11-n) th band is calculated, as shown in FIG. Thus, the second bands (second and ninth) from both ends are sequentially combined. The concept of combination for B ′ _nc is the same.
[0060]
FIG. 12 is an explanatory diagram for explaining the concept when averaging by combining distribution ratios, and shows the color gamut of the RGB color space mapped on the UY plane of the YUV color space. In the figure, the color gamut of the RGB color space is biased in the negative U direction in the high luminance region, and is biased in the positive U direction in the low luminance region. The same applies to the case of mapping on the V-Y plane, which is biased in the negative V direction in the high luminance region and biased in the positive V direction in the low luminance region. That is, the high luminance region and the low luminance region have opposite properties, and considering this property with the above distribution ratio, a large difference occurs between the high luminance region and the low luminance region. If averaged, the difference can be canceled out, and the distribution ratio obtained by removing the distribution ratio bias due to the color gamut shape can be evaluated. Note that the idea of averaging the distribution ratio so as to eliminate this bias is generated from the relationship between the RGB color space and the YUV color space, and can be adopted in white balance adjustment for many image devices. it can.
[0061]
If the average value of the set of distribution ratio at step 235 is calculated, R _'nc is determined whether "1" or less, at the same step S240 R' in step S240 _nc is from "1" When it is determined that the value is smaller, the reciprocal of the R ′ _nc is substituted for R ′ _{nc in} step S245. The determination processing in steps S240 and S245 is performed independently for all of the combination numbers 1 to 5 and for B ′ _nc regardless of the determination result in step S240. For all the processing here is R _'nc and B' _nc, are those that the process of converting to a number greater than smaller than "1", "1", determination by the following evaluation function e _n And is returned to the original value after discrimination by the evaluation function.
[0062]
(4-3) White balance control processing:
In step S250, the evaluation function _{e n,} substituted and each pair of R _'nc and B' _nc, the above white balance control module 33c of the set number n to which the evaluation function _{e n} is the smallest R _'nc And B ′ _nc are determined as the first-stage white balance correction coefficients R ′ _c and B ′ _c . This first-stage white balance correction coefficient has the same properties as the distribution ratio described above, and is a coefficient that can be a predetermined component of the white balance control matrix 34b8 shown in FIG. That is, when the white balance control matrix 34b8 is multiplied by a matrix having RGB components of arbitrary pixels in the image as components, the white balance of the image can be controlled.
[0063]
Here, the process of selecting the smallest evaluation function is the process of selecting the smallest difference between R ′ _nc and B ′ _nc and “1”, and the correction amount is the smallest. You are selecting a thing. In other words, in white balance adjustment, it is important to prevent color failure, and if a large correction amount is used as a white balance correction coefficient, color correction often occurs due to large correction, but a correction amount is small. If selected, the occurrence probability of color feria can be made very small. The present applicant has confirmed that the white balance correction coefficient determined by the above-described processing is very effective in white balance adjustment. Here, it is only necessary to prevent color feria, and evaluation is performed. The function can be changed as appropriate, for example, by selecting a function that is second closest to “1”.
[0064]
At this time, for example, color feria can be prevented by taking measures such as selecting the second and third closest to “1” within a predetermined range. Further, in the present invention, the white balance correction coefficient is calculated based on data belonging to a part of the luminance band of the image, and the correction amount having a small correction amount is selected as described above. Compared with the white balance adjustment that reflects the RGB integral value of the entire image as described above, it is very difficult to generate color feria.
[0065]
(4-3-1) Correction corresponding to light source type:
As described above, it is possible to perform white balance control using the first-stage white balance correction coefficient. However, in the present embodiment, in order to prevent the occurrence of color failure due to the difference in the light source, it is further illustrated in FIG. The processing after step S255 is executed. In step S255, the first-stage white balance correction coefficients R ′ _c and B ′ _c are expressed by the following equation so that the white balance correction coefficient is based on the gradation value “128” (median value) of the G component. Convert.
R = 128 / R ′ _c , G = 128, B = 128 / B ′ _c
[0066]
In step S260, the converted coefficient RGB is converted into a physiological three primary color space R′G′B ′ by the equation of Estevez-Hunt-Pointer according to the following equation (4).
[Expression 4]

[0067]
Here, the matrix A is a matrix stored in the ROM 33 and converts the sRGB coordinates to the RGB coordinates of the physiological three primary colors. The matrix A is generated by multiplying the matrix B and the matrix C. Each matrix has the above-described values, and the matrix B is a conversion matrix that converts sRGB coordinates into XYZ coordinates. It is a matrix for converting XYZ coordinates to RGB coordinates of physiological three primary colors. In the above formula, italicized characters indicate a matrix. When the spatial data R′G′B ′ of the physiological three primary colors is obtained in step S260, final white balance correction coefficients Rc and Bc are determined according to the following formula in step S265.
Rc = G ′ / R ′, Bc = G ′ / B ′
[0068]
The processes in steps S255 to S265 correspond to temporarily converting sRGB coordinates to R′G′B ′ coordinates of physiological three primary colors and correcting the white balance correction coefficient in the converted space (step S265). To do. By this processing, it is possible to approach a good white balance according to human appearance, and it is possible to perform white balance control independent of the light source type. Further, by incorporating the final white balance correction coefficients Rc and Bc into the white balance control matrix W, the coordinates of the second color space are corrected and white balance adjustment is executed. FIG. 13 and FIG. 14 are explanatory diagrams for explaining how the white balance correction coefficient is corrected by converting to physiological three primary colors.
[0069]
FIG. 13 is a diagram showing a spectral distribution of an ideal light source of sRGB (CIE standard light source D65), and FIG. 14 is a diagram showing a spectral distribution of three physiological primary colors. In the figure, the horizontal axis is the wavelength (nm), and the vertical axis is the relative value of the spectral distribution. 13 and 14, the curve having a peak near 450 nm is the B component, and the curve having a peak near 540 nm is the spectral distribution of the G component, and has a peak near 610 nm in FIG. 13 and near 580 nm in FIG. 14. The curve is the spectral distribution of the R component. Since the white balance correction coefficient is multiplied by the R component and the B component, adjustment by the white balance correction coefficient enlarges the curve of the B component and the R component for the spectral distributions in FIGS. It works to shrink or shrink.
[0070]
Since the spectral distribution shown in FIG. 14 is a distribution for the three physiological primary colors, the human perception of color is very close to this distribution, and when the white balance correction coefficient is corrected in the RGB space that is this distribution, The white balance can be controlled in a way that is very close to how you feel the colors. Specifically, in the spectral distribution shown in FIG. 13, the peak wavelength and peak intensity of the R component are larger than the peak wavelength and peak intensity of the R component of the spectral distribution shown in FIG. Therefore, if the white balance is adjusted without performing the conversion in step S260, the correction for the R component becomes relatively large, and the R component is overemphasized or weakened. Therefore, it is possible to adjust the white balance so that the color looks more red by correcting the white balance correction coefficient after the conversion in step S260. Of course, the same is true for blue. Therefore, the white balance correction coefficient can be determined so that an appropriate white balance is adjusted even for an image taken with an incandescent lamp.
[0071]
When the final white balance correction coefficients Rc and Bc are determined in step S265, in step S270, the white balance correction coefficients Rc and Bc are used as components in the first row and first column and the third row and third column, and the second row and second column. A white balance control matrix W (34b9) is generated by multiplying the matrix with “1” and other components “0” by multiplying the inverse transformation matrix A- ¹ from the left and multiplying the matrix A from the right. In step S275, the white balance control matrix W is stored in the work area 34b of the RAM 34.
[0072]
The inverse transformation matrix A- ¹ is expressed by the following equation (5).
[Equation 5]

Also in the above formula, italicized characters indicate a matrix.
[0073]
The process up to the calculation / storage of the white balance control matrix W corresponds to the measurement of the white balance, and as the processing of the digital still camera 10, the process returns to the flowchart shown in FIG. In step S110, the white balance of each captured image is controlled by multiplying the image data obtained by converting the CCD Raw data into the sRGB data by the white balance control matrix W.
[0074]
That is, the white balance control matrix W is a matrix that operates to adjust the white balance of the image of the color image data expressed by the sRGB coordinates, and the input and output data are sRGB data. Since the white balance correction coefficient is not a correction coefficient for sRGB, the input / output data is converted to sRGB data by converting the space using the matrix A or the like. Therefore, the processing of steps S270, S275, and S110 corresponds to the processing in the matrix calculation unit 33e. FIG. 15 is an explanatory diagram for explaining the calculation by the white balance control matrix W. In the upper matrix calculation expression, RGB on the right side is input sRGB data, and R ″ G ″ B ″ on the left side is Output sRGB data. Further, in the actual processing of the present embodiment, the white balance control matrix W is one 3 × 3 matrix obtained by calculating three 3 × 3 matrices in advance. This will be explained separately.
[0075]
In FIG. 15, each calculation stage using three matrices is shown below the matrix calculation formula. As described above, the input data (1) is sRGB data. Since the matrix A is a matrix for converting the RGB color components of the sRGB coordinate system into the RGB color components of the physiological three primary colors, when the input data (1) is multiplied by the matrix A, the physiological three primary colors are represented as shown in (2). Converted to RGB color components. That is, this conversion is processing in the first color space conversion unit 33e1. When this conversion result is multiplied by a matrix including the white balance correction coefficients Rc and Bc as components, the white balance is adjusted with RGB color components of physiological three primary colors as shown in (3). That is, the process here is the process in the white balance control unit 33e2.
[0076]
When this adjusted RGB data is multiplied by the inverse conversion matrix A- ¹ , the image data after white balance adjustment is converted into sRGB data as shown in (4). That is, the process here is a process in the second color space conversion unit 33e3. In this embodiment, this calculation is realized by a single matrix, and the sRGB coordinates in the first color space are converted into RGB color components of physiological three primary colors as the second color space by one calculation, and the second The white balance is adjusted with the correction coefficient in the color space, and the corrected RGB color components are returned to the sRGB coordinates in the first color space. Accordingly, when the sRGB data is multiplied by a white balance control matrix W, which is a 3 × 3 matrix, in the image processing process of the digital still camera, the sRGB data with the adjusted white balance is obtained. Since it is corrected in a space close to, white balance can be adjusted reliably without being affected by the light source.
[0077]
As described above, in the present invention, the coordinates of the first color space are converted into the coordinates of the second color space, and the white balance is controlled by correcting the coordinates of the converted second color space. The later coordinates are converted into the coordinates of the first color space. Therefore, white balance adjustment can be performed without being affected by a specific color in the image or a light source biased to the specific color.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a schematic configuration of a digital still camera.
FIG. 2 is a flowchart showing a main flow of image processing.
FIG. 3 is a block diagram showing a white balance measurement system and a control system;
FIG. 4 is a flowchart showing white balance measurement and control.
FIG. 5 is a flowchart showing white balance measurement and control.
FIG. 6 is an explanatory diagram for explaining a thinning process;
FIG. 7 is a diagram illustrating pixel data after RGBYs conversion.
FIG. 8 is a graph showing an example of a histogram.
FIG. 9 is an explanatory diagram illustrating a state of band division.
FIG. 10 is an explanatory diagram for explaining how an integral value and a distribution ratio are calculated.
FIG. 11 is an explanatory diagram for explaining how two bands are combined.
FIG. 12 is an explanatory diagram for explaining a concept when averaging by combining distribution ratios;
FIG. 13 is a diagram showing a spectral distribution of an ideal light source of sRGB.
FIG. 14 is a diagram showing a spectral distribution for three physiological primary colors.
FIG. 15 is an explanatory diagram for explaining calculation by a white balance control matrix;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 ... Digital still camera 20 ... Optical part 21 ... Optical lens system 22 ... Autofocus mechanism 23 ... Distance measuring part 24 ... Autofocus controller 25 ... Controller 26 ... CCD
27 ... Strobe 28 ... Auto gain controller (AGC)
29 ... A / D converter 30 ... Control unit 31 ... Bus 32 ... CPU
33 ... ROM
33a ... Luminance component acquisition module 33b ... Color component distribution information acquisition module 33c ... White balance control module 33d1 ... Conversion matrix A
33d2 ... Inverse transformation matrix A ^-1
33e ... Matrix calculation unit 33e1 ... first color space conversion unit 33e2 ... white balance control unit 33e3 ... second color space conversion unit 34 ... RAM
34a ... image area 34b ... work area 34b1 ... thinned-data-34b2 ... RGBYs data 34B3 ... histogram data 34B4 ... band data 34b5 ... RGB integrated data 34B6 ... distribution ratio data 34B7 ... correction coefficient R _'c, B' _c
34b8 ... White balance control matrix 34b9 ... White balance control matrix 34c ... Video RAM area 35 ... Operation panel 36 ... LCD panel 37a ... I / O
37b ... Flash memory card

Claims (3)

A white balance adjustment device that adjusts the white balance of an image in which colors are expressed by coordinates in a first color space for a plurality of pixels,
First distribution ratio calculating means for calculating a first distribution ratio indicating a balance of gradation distribution of each color component in the image by coordinates of a first color space;
Second distribution ratio calculation means for converting the first distribution ratio into a second distribution ratio indicated by coordinates of a color space of physiological three primary colors reflecting the cone sensitivity characteristic of human eyes by a predetermined conversion formula;
White balance correction coefficient calculating means for calculating a white balance correction coefficient based on the second distribution ratio;
The conversion formula for converting the coordinates of the first color space to the coordinates of the color space of the physiological three primary colors, the conversion formula based on the white balance correction coefficient, and the coordinates of the color space of the physiological three primary colors are the coordinates of the first color space. A white balance control matrix generating means for generating a white balance control matrix obtained by synthesizing the conversion formula for conversion to
A white balance adjustment device comprising: white balance control means for controlling the white balance of the image by converting the image with the white balance control matrix. 2. The white balance adjustment device according to claim 1 , wherein the first distribution ratio is a coordinate representing a balance of gradation distribution of each color component with a value based on a median value of gradation of each color component. . A computer-readable white balance adjustment program for causing a computer to execute a function of adjusting a white balance of an image in which colors are expressed by coordinates in RGB space for a plurality of pixels,
A first distribution ratio calculation function for calculating a first distribution ratio indicating a balance of gradation distribution of each color component in the image by coordinates of a first color space;
A second distribution ratio calculation function for converting the first distribution ratio into a second distribution ratio indicated by the coordinates of the color space of the physiological three primary colors reflecting the cone sensitivity characteristics of the human eye by a predetermined conversion formula;
A white balance correction coefficient calculation function for calculating a white balance correction coefficient based on the second distribution ratio;
The conversion formula for converting the coordinates of the first color space to the coordinates of the color space of the physiological three primary colors, the conversion formula based on the white balance correction coefficient, and the coordinates of the color space of the physiological three primary colors are the coordinates of the first color space. A white balance control matrix generation function for generating a white balance control matrix that combines the conversion formula for conversion into
A white balance adjustment program for causing a computer to execute a white balance control function for controlling white balance of an image by converting the image by the white balance control matrix.

Images