JP4003037B2 - White balance adjustment device, white balance adjustment program, white balance adjustment method, and digital camera - Google Patents

Description

分布比は以下の式（２）にて算出する。
【数２】

ここで、Ｒ_ｉｊ，Ｇ_ｉｊ，Ｂ_ｉｊは各色成分毎の階調値であり、「ｉ」は帯域の番号に該当し「ｊ」は画素を区別するための便宜的な番号である。また、Ｉｒｉ，Ｉｇｉ，Ｉｂｉは各帯域毎、各成分毎の積分値であり、ＲｉｃとＢｉｃは各帯域毎の分布比である。
【００６５】
ここで、上記積分値は色成分の階調値の積算値であることから、各帯域毎にどれほどの値の階調値がどれほど分布しているかを反映した値となっている。また、この積分値を使用して分布比を算出することは帯域毎の色成分毎の平均階調値を使用して分布比を算出することと等価である。帯域毎の色成分毎の平均階調値は、帯域内に属する画素の所定色成分の階調値を足し合わせ、画素数で除したものであり、色成分毎の画素数は各帯域で同数であるところ、分布比にした場合には分母と分子で画素数が相殺されて結局分布比に寄与するのは階調値を足し合わせた成分のみとなるからである。
【００６６】
この分布比Ｒｉｃは「帯域毎のＧ成分の平均値／帯域毎のＲ成分の平均値」，分布比Ｂｉｃは「帯域毎のＧ成分の平均値／帯域毎のＢ成分の平均値」であり、この分布比はホワイトバランスを調整する性質を持った係数となる。すなわち、ある画素のＲＧＢ色成分それぞれがその画素の属する帯域の各色成分の平均階調値と等しい場合、分布比ＲｉｃとＲ成分の階調値とを乗じると分母が相殺してＧ成分の平均値と等しくなり、分布比ＢｉｃとＢ成分の階調値とを乗じても分母が相殺してＧ成分の平均値と等しくなる。従って、当該画素の全色成分が等しくなり無彩色になる。
【００６７】
このように、分布比Ｒｉｃと分布比Ｂｉｃとはそれぞれ画素のＲ成分とＢ成分とに作用し、その帯域のＧ成分の平均値を基準としつつ当該基準に近づけるようにＲ成分とＢ成分とを相対的に変化させるので、ホワイトバランスを調整するための係数であると言える。従って、上記ステップＳ２３０まででは、帯域毎のホワイトバランス補正係数を算出したと言える。本発明においては、所定帯域の分布比を参照して画像全体のホワイトバランスを調整するとともに、画像内容や光源等の差異にかかわらずより好ましい結果を得るための補正を加えている。
【００６８】
ステップＳ２３０にて分布比データを取得すると、上記色成分分布情報取得モジュール３３ｂはステップＳ２３５にて以下の式（３）に従って分布比の組みの平均値Ｒ’_ｎｃ，Ｂ’_ｎｃを算出する。
【数３】

ここで、ｎは帯域の組番号であり「１〜５」の整数である。
【００６９】
図１１は、組みの平均値Ｒ’_ｎｃについて２つの帯域を組みにする様子を説明する説明図である。同図に示す上部のグラフは上述のようにして１０個の帯域に分割されたヒストグラムを示している。上記式（３）においては、ｎ番目の帯域と（１１−ｎ）番目の帯域についての平均を算出していることから、同図に示すように両端の帯域（１番目と１０番目）が組みとなり、両端から２番目の帯域（２番目と９番目）が組みとなるようにして順次組み合わせられる。Ｂ’_ｎｃについての組み合わせの考え方も同様である。
【００７０】
図１２は、分布比を組みにして平均化する際の考え方を説明する説明図であり、同図はＹＵＶ色空間のＵ−Ｙ平面に写像されたＲＧＢ色空間の色域を示す。同図において、ＲＧＢ色空間の色域は、高輝度領域で負のＵ方向に偏り、低輝度領域で正のＵ方向に偏っている。Ｖ−Ｙ平面に写像した場合も同様で、高輝度領域で負のＶ方向に偏り、低輝度領域で正のＶ方向に偏ることとなる。すなわち、高輝度領域と低輝度領域とで逆の性質を有しており、この性質を上記分布比で考えると高輝度領域と低輝度領域とで大きな差が発生することとなるが、両者を平均すればその差を相殺することができ、色域形状に起因する分布比の偏りを除去した当該分布比を評価することができる。尚、この偏りを除去するように分布比を平均化する考え方は、ＲＧＢ色空間とＹＵＶ色空間との関係から発生したものであり、多くの画像機器についてのホワイトバランス調整にて採用することができる。
【００７１】
ステップ２３５にて分布比の組みの平均値が算出されると、ステップＳ２４０にてＲ’_ｎｃが「１」より小さいか否かを判別し、同ステップＳ２４０にてＲ’_ｎｃが「１」より小さいと判別されたときにはステップＳ２４５にてＲ’_ｎｃに対して当該Ｒ’_ｎｃの逆数を代入する。このステップＳ２４０，ステップＳ２４５の判別処理は、ステップＳ２４０での判別結果にかかわらず、組み番号１〜５の総てについて、また、Ｂ’_ｎｃについても同様に独立に実行される。ここでの処理はＲ’_ｎｃとＢ’_ｎｃとの総てについて、「１」より小さければ「１」より大きな数に変換する処理をしているものであり、次の評価関数ｅ_ｎによる判別のための変換であるとともに、評価関数による判別後は元の値に戻される。
【００７２】
（４−３）ホワイトバランス制御処理：
ステップＳ２５０では、評価関数ｅ_ｎに対して、各組のＲ’_ｎｃとＢ’_ｎｃとを代入し、上記ホワイトバランス制御モジュール３３ｃは当該評価関数ｅ_ｎが最小となる組み番号ｎのＲ’_ｎｃとＢ’_ｎｃとの値を第１段階のホワイトバランス補正係数Ｒ’_ｃおよびＢ’_ｃとして決定する。この第１段階のホワイトバランス補正係数も上述の分布比と同様の性質を有しており、上記図３に示すホワイトバランス制御マトリクス３４ｂ８の所定成分となり得る係数である。すなわち、画像内の任意の画素のＲＧＢ成分を成分とする行列にホワイトバランス制御マトリクス３４ｂ８を乗じると、その画像のホワイトバランスを制御することができる。
【００７３】
ここで、評価関数が最小のものを選択する処理は、Ｒ’_ｎｃとＢ’_ｎｃとのそれぞれと「１」との差が最小のものを選択している処理であり、補正量が最も小さいものを選択していることになる。すなわち、ホワイトバランス調整においてはカラーフェリアを防止することが重要であり、補正量の大きなものをホワイトバランス補正係数としてしまうと大きな補正によってカラーフェリアが発生することが多いが、補正量が小さいものを選択しておけば、カラーフェリアの発生確率を非常に小さくすることができる。また、補正量が小さいものを選択しているとともに、他の領域と比較して色成分分布の偏りが少ない領域を選択していると言うこともできる。この場合、上記輝度成分取得モジュール３３ａが上記画像分割手段に該当するし、色成分分布情報取得モジュール３３ｂにて評価関数が最小となるものを選択する処理が上記色成分分布の偏りが少ない領域を選択することに該当する。さらに、評価関数が最小となるものを選択する処理は、Ｇ成分に対してＲ成分およびＢ成分が他の領域と比較して比較的均等に分布している領域を選択している処理であると言うこともできる。
【００７４】
本出願人は、上述の処理によって決定されたホワイトバランス補正係数が非常に効果的にホワイトバランス調整可能であることを確認しているが、ここでは、カラーフェリアを防止することができればよく、評価関数が２番目に「１」に近いものを選択する構成にするなど適宜変更可能である。このとき、例えば、所定の値域内であれば２番目，３番目に「１」に近いものを選択するようにするなどの工夫を施せばカラーフェリアを防止可能である。また、本発明においては、画像の一部の輝度帯域に属するデータに基づいてホワイトバランス補正係数を算出しており、上述のように補正量の小さいものを選択しているため、上述の従来例のように画像全体のＲＧＢ積分値を反映させるホワイトバランス調整と比較して、非常にカラーフェリアが発生しにくい。
【００７５】
（４−３−１）光源種別に対応した補正：
上述のように第１段階のホワイトバランス補正係数でホワイトバランス制御を行うことも可能であるが、本実施形態においてはより見た目に近づけるために、さらに図５のステップＳ２５５以降の処理を実行している。ステップＳ２５５では、Ｇ成分の階調値「１２８」（中央値）を基準としたホワイトバランス補正係数となるように以下の式にて上記第１段階のホワイトバランス補正係数Ｒ’_ｃおよびＢ’_ｃを変換する。
Ｒ＝128／Ｒ’_ｃ，Ｇ＝128，Ｂ＝128／Ｂ’_ｃ
【００７６】
ステップＳ２６０では、当該変換後の係数ＲＧＢを以下の式（４）に従ってＥｓｔｅｖｅｚ−Ｈｕｎｔ−Ｐｏｉｎｔｅｒの式にて生理的三原色の空間Ｒ’Ｇ’Ｂ’に変換する。
【数４】

【００７７】
ここで、マトリクスＡは上記ＲＯＭ３３に記憶されたマトリクスであり、ｓＲＧＢ座標を生理的三原色のＲＧＢ座標に変換する。また、マトリクスＡはマトリクスＢとマトリクスＣとを乗じて生成されたものであり、それぞれのマトリクスは上述した値であるとともにマトリクスＢはｓＲＧＢ座標をＸＹＺ座標に変換する変換マトリクスであり、マトリクスＣはＸＹＺ座標を生理的三原色のＲＧＢ座標に変換するマトリクスである。尚、上記式において斜体の文字はマトリクスを示している。ステップＳ２６０にて生理的三原色の空間データＲ’Ｇ’Ｂ’が得られると、ステップＳ２６５にて以下の式に従って最終的なホワイトバランス補正係数Ｒｃ，Ｂｃを決定する。
Ｒｃ＝Ｇ’／Ｒ’，Ｂｃ＝Ｇ’／Ｂ’
【００７８】
これらステップＳ２５５〜Ｓ２６５の処理は、ｓＲＧＢ座標を一旦生理的三原色のＲ’Ｇ’Ｂ’座標に変換し、当該変換後の空間でホワイトバランス補正係数を補正（ステップＳ２６５）していることに該当する。この処理によって人間の見た目に応じた良好なホワイトバランスに近づけることができ、光源種に依存しないホワイトバランス制御を実施することができる。図１３，図１４は生理的三原色に変換してホワイトバランス補正係数を補正する様子を説明する説明図である。
【００７９】
図１３はｓＲＧＢの理想光源（ＣＩＥ標準の光源Ｄ６５）の分光分布を示す図であり、図１４は生理的三原色についての分光分布を示す図である。図において横軸は波長（ｎｍ）であり、縦軸は分光分布の相対値である。尚、図１３，図１４において４５０ｎｍ付近にピークがあるカーブはＢ成分、５４０ｎｍ付近にピークがあるカーブはＧ成分の分光分布であり、図１３の６１０ｎｍ付近，図１４の５８０ｎｍ付近にピークがあるカーブはＲ成分の分光分布である。上記ホワイトバランス補正係数はＲ成分とＢ成分とに対してそれぞれ乗ぜられるので、ホワイトバランス補正係数による調整は、上記図１３，１４における分光分布に対してはＢ成分とＲ成分のカーブを拡大したり縮小したりするように作用する。
【００８０】
図１４に示す分光分布は生理的三原色についての分布であることから、人間の色の感じ方はこの分布に非常に近く、この分布となるＲＧＢ空間でホワイトバランス補正係数の補正を行うと、人間の色の感じ方に非常に近い形でホワイトバランスを制御することができる。具体的には、図１３に示す分光分布においてはＲ成分のピーク波長およびピーク強度が図１４に示す分光分布のＲ成分のピーク波長およびピーク強度より大きい。従って、上記ステップＳ２６０における変換を実施せずにホワイトバランスの調整をしようとすると、Ｒ成分に対する補正が相対的に大きくなり、Ｒ成分を強調しすぎたり、弱めすぎたりしてしまう。そこで、上記ステップＳ２６０における変換を実施した後にホワイトバランス補正係数を補正することにより、より見た目に近い赤となるようにホワイトバランスを調整可能である。むろん、青に関しても同様で見た目に近い青となる。従って、白熱灯かでの撮影画像等であっても適切なホワイトバランスとなるよう調整されるようにホワイトバランス補正係数を決定可能である。
【００８１】
ステップＳ２６５にて最終的なホワイトバランス補正係数Ｒｃ，Ｂｃを決定したら、ステップＳ２７０において当該ホワイトバランス補正係数Ｒｃ，Ｂｃを１行１列目および３行３列目の成分とし、２行２列目の成分を「１」とし、他の成分を「０」としたマトリクスの左から上記逆変換マトリクスＡ^−１を乗じ、右から上記マトリクスＡを乗じてホワイトバランス制御マトリクスＷ（３４ｂ９）を生成し、ステップＳ２７５にて当該ホワイトバランス制御マトリクスＷを上記ＲＡＭ３４のワークエリア３４ｂに保存する。
【００８２】
尚、逆変換マトリクスＡ^−１は以下の式（５）にて表現される。
【数５】

上記式においても斜体の文字はマトリクスを示している。
【００８３】
このホワイトバランス制御マトリクスＷの算出／保存までがホワイトバランスの測定に該当し、ディジタルスチルカメラ１０の処理としてはこの後図２に示すフローチャートに戻ってステップＳ１０６以降の処理を実行する。そして、ステップＳ１１０においてＣＣＤＲａｗデータをｓＲＧＢデータに変換した画像データに対してホワイトバランス制御マトリクスＷを乗じることによって個々の撮像画像のホワイトバランスを制御する。
【００８４】
すなわち、ホワイトバランス制御マトリクスＷは、ｓＲＧＢ座標で表現されたカラー画像データの画像についてホワイトバランスを調整するように作用するマトリクスであり、入力および出力データはｓＲＧＢデータである。上記ホワイトバランス補正係数はｓＲＧＢに対する補正係数ではないため、上記マトリクスＡ等による空間の変換を行って入出力データがｓＲＧＢデータになるようにしてある。図１５は、かかるホワイトバランス制御マトリクスＷによる演算を説明するための説明図であり、上部のマトリクス演算式において右辺のＲＧＢは入力ｓＲＧＢデータであり、左辺のＲ’’Ｇ’’Ｂ’’は出力ｓＲＧＢデータである。また、本実施形態の実際の処理においてホワイトバランス制御マトリクスＷは３個の３×３の行列を予め計算して得られる１個の３×３の行列であるが、ここでは便宜上３個のマトリクスに分けて説明する。
【００８５】
図１５においてマトリクス演算式の下方には３個のマトリクスによる各演算段階を示している。上述のように入力データ▲１▼はｓＲＧＢデータである。マトリクスＡはｓＲＧＢ座標系のＲＧＢ色成分を生理的三原色のＲＧＢ色成分に変換するマトリクスであるので、当該入力データ▲１▼にマトリクスＡを乗じると、▲２▼に示すように生理的三原色のＲＧＢ色成分に変換される。この変換結果に対して上記ホワイトバランス補正係数Ｒｃ，Ｂｃを成分として含むマトリクスを乗じると、▲３▼に示すように生理的三原色のＲＧＢ色成分にてホワイトバランスが調整される。この調整後のＲＧＢデータに逆変換マトリクスＡ^−１を乗じると、▲４▼に示したようにホワイトバランス調整後の画像データがｓＲＧＢデータに変換される。従って、ディジタルスチルカメラの画像処理過程においてｓＲＧＢデータに１個の３×３の行列であるホワイトバランス制御マトリクスＷを乗じるとホワイトバランスが調整される。
【００８６】
以上説明したように、本発明においては画像の輝度成分を取得して帯域分割し、所定の帯域内にある画像の色成分分布情報に基づいてホワイトバランスを調整するためのホワイトバランス補正係数を算出する。従って、画像内の特定色や特定色に偏った光源に影響されることなくホワイトバランス調整を実施可能となる。
【図面の簡単な説明】
【図１】ディジタルスチルカメラの概略構成を示すブロック図である。
【図２】画像処理の主要な流れを示すフローチャートである。
【図３】ホワイトバランスの測定系及び制御系を示すブロック図である。
【図４】ホワイトバランス測定及び制御を示すフローチャートである。
【図５】ホワイトバランス測定及び制御を示すフローチャートである。
【図６】間引き処理を説明するための説明図である。
【図７】ＲＧＢＹｓ変換後の画素データを示す図である。
【図８】ヒストグラムの一例を示すグラフである。
【図９】帯域分割の様子を説明する説明図である。
【図１０】積分値と分布比とを算出する様子を説明する説明図である。
【図１１】２つの帯域を組みにする様子を説明する説明図である。
【図１２】分布比を組みにして平均化する際の考え方を説明する説明図である。
【図１３】ｓＲＧＢの理想光源の分光分布を示す図である。
【図１４】生理的三原色についての分光分布を示す図である。
【図１５】ホワイトバランス制御マトリクスによる演算を説明するための説明図である。
【符号の説明】
１０…ディジタルスチルカメラ
２０…光学部
２１…光学レンズ系
２２…オートフォーカス機構
２３…測距部
２４…オートフォーカスコントローラ
２５…コントローラ
２６…ＣＣＤ
２７…ストロボ
２８…オートゲインコントローラ（ＡＧＣ）
２９…Ａ／Ｄコンバータ
３０…制御部
３１…バス
３２…ＣＰＵ
３３…ＲＯＭ
３３ａ…輝度成分取得モジュール
３３ｂ…色成分分布情報取得モジュール
３３ｃ…ホワイトバランス制御モジュール
３３ｄ１…変換マトリクスＡ
３３ｄ２…逆変換マトリクスＡ^−１
３４…ＲＡＭ
３４ａ…画像エリア
３４ｂ…ワークエリア
３４ｂ１…間引き済データ
３４ｂ２…ＲＧＢＹｓデータ
３４ｂ３…ヒストグラムデータ
３４ｂ４…帯域データ
３４ｂ５…ＲＧＢ積分データ
３４ｂ６…分布比データ
３４ｂ７…補正係数Ｒ’_ｃ，Ｂ’_ｃ
３４ｂ８…ホワイトバランス制御マトリクス
３４ｂ９…ホワイトバランス制御マトリクス
３４ｃ…ビデオＲＡＭエリア
３５…操作パネル
３６…ＬＣＤパネル
３７ａ…Ｉ／Ｏ[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a white balance adjustment device, a white balance adjustment program, a white balance adjustment method, and a digital camera.
[0002]
[Prior art]
The human eye adjusts the color component by chromatic adaptation and can recognize a white object as white even if the light source changes. However, in an image device such as a digital camera, the image sensor itself does not have a chromatic adaptation function. Therefore, the color balance is conventionally adjusted by a technique called auto white balance. For example, when the color image data is composed of three components of R (red), G (green), and B (blue), the gradation values of the three components are integrated over the entire image, and the integration ratio is R : G: B = 1: 1: 1 The gradation value of each color component is corrected.
[0003]
[Problems to be solved by the invention]
In the above-described conventional digital camera, correction is performed so that the average color of the entire image becomes gray based on the hypothesis that adding all the color components of the subject results in an achromatic color. If there is a part with a large specific color or a captured image under a light source whose spectral distribution is biased to the specific color, a color feria in which white balance adjustment fails due to the influence of the specific color occurs.
The present invention has been made in view of the above problems. A white balance adjustment device, a white balance adjustment program, a white balance adjustment program capable of performing white balance adjustment without being affected by a specific color or a light source biased to a specific color in an image. An object is to provide a balance adjustment method and a digital camera.
[0004]
[Means for Solving the Problems]
  To achieve the above purpose, PaintingAn image dividing unit that divides an image into a plurality of reference regions composed of a plurality of parts, and a reference region that is divided by the image dividing unit, and the color component distribution in the region is biased compared to other reference regions Color component distribution information acquisition means for acquiring color component distribution information for a small predetermined area, and white balance control means for controlling the white balance by correcting the color components of the image based on the acquired color component distribution information. As constitution to haveAlso good.the aboveConstitution, The image is divided into a plurality of reference areas composed of a plurality of parts, color component distribution information for any one or combination of the reference areas is acquired, and the color components of the image are determined based on the color component distribution information. Correct and control the white balance. Here, the color component distribution information acquisition unit acquires the color component distribution information for a region where the deviation of the color component distribution in the region is small compared to other reference regions.
[0005]
That is, the white balance is adjusted based on the color components in the region where the color component distribution is less biased, and the overall balance is not adjusted based on the color components of the entire image. Therefore, even if there is a large portion of a single color in the image, this portion can be excluded, and the color feria can be prevented without being greatly affected by this single color portion. Here, the region where the color component distribution is less biased is compared with the other reference regions, and an area where the color component is less unevenly distributed, i.e., an achromatic region, is selected. I don't mean to get it. Further, the reference area does not need to be a continuous area, and may be composed of a plurality of parts extracted from the entire image. For example, a plurality of parts may be extracted from the entire image, or a part having a luminance within a predetermined range may be extracted, and the reference region can be configured with parts extracted based on various indices. The predetermined color may be any one of the color components when the image is composed of three color components, or may be a predetermined color expressed by a combination thereof.
[0006]
  Also, PaintingWhen the image is expressed by the first color component to the third color component, the other second color component and the third color component are compared with the other regions in the image with respect to the reference first color component. A color component distribution information acquisition unit that acquires color component distribution information of a relatively uniformly distributed region, and a white that controls white balance by correcting the color components of the image based on the acquired color component distribution information As a configuration comprising a balance control meansAlso good.
[0007]
  the aboveConstitutionIn FIG. 2, the image is expressed by the first to third color components, and the first color component is used as a reference in controlling the white balance. Then, color component distribution information is acquired for a region in which the other second color component and the third color component are distributed relatively evenly with respect to the reference first color component as compared with other regions in the image. Then, the color component of the image is corrected based on the color component distribution information of this region.
[0008]
That is, since the white balance is adjusted based on the color component distribution information in a region where the second color component and the third color component are not distributed with respect to the reference first color component, the presence of the specific color is greatly affected. Color feria can be prevented without being affected. Here, the first to third color components can be realized by various color components such as RGB components and CMY components, but are preferably RGB components close to the color distribution felt by the human cone. In addition, it is preferable that the G component that occupies most of the luminance component be the reference color.
[0009]
  The present inventionAre acquired by a luminance component acquisition unit that acquires a luminance component of an image and a color component distribution information acquisition unit that acquires color component distribution information of an image of a part of the image where the luminance component is in a predetermined band. White balance control means for controlling the white balance by correcting the color components of the image based on the color component distribution information is provided.
[0010]
  the aboveConstitutionThe luminance component acquisition unit acquires the luminance component of the image, and the color component distribution information acquisition unit acquires the color component distribution information of the image of the part where the luminance component is in a predetermined band. Then, the white balance control means corrects the color component of the image based on the acquired color component distribution information and controls the white balance.
[0011]
That is, since color component distribution information is used for an image of a part where the luminance component is within a predetermined band, color feria can be prevented without being greatly affected by a large specific color of the entire image. In all the above-mentioned inventions, since the color component distribution information is acquired, it is preferable to apply it to an image device that handles digital image data. However, it is of course preferable for an image device that handles analog data such as a television. The present invention can also be applied.
[0012]
  Here, the luminance component acquisition unit only needs to acquire the luminance component of the image, and various configurations can be employed. As an example of its configurationClaim 2According to the invention, the luminance component acquisition means is configured to divide the dot matrix image data into predetermined sections and acquire an average value of color components for each section as the luminance component. That is, in recent image devices, a very large amount of pixel data such as millions of pixels can be handled. In such image devices, the luminance component is referred to for all of the dot matrix image data, and all In order to process the color component distribution information, a very large amount of calculation time is required and a very large amount of buffer memory is required.
[0013]
Therefore, it is preferable to thin out the reference data as long as the overall white balance can be adjusted. In the present invention, the image data is divided into predetermined sections, and the average value of the color components for each section is acquired. In addition, when data as a luminance component is included in the image data, the data may be used. However, when the image data includes data of a color component, generally, the luminance component can be calculated from the color component data. . There are various formulas for calculating the luminance component, but if the average value of the color component is calculated as in the present invention, the calculation is very simple and can be performed easily. Of course, for example, the luminance component can be calculated as Y = 0.30R + 0.59G + 0.11B based on the RGB components.
[0014]
  Furthermore, the luminance component acquisition unit only needs to be able to provide an index for extracting the pixel to be referenced for white balance adjustment for each band.The present inventionThe luminance component acquisition means is configured to form a histogram of the luminance component of the image and divide the band into a predetermined luminance range. That is, when the luminance component is converted into a histogram, the distribution of the luminance component for each level, for example, each gradation value can be grasped, and if the distribution of the luminance component is known, the band can be easily divided. For example, it is possible to divide the band so that the number of pixels existing in each band becomes substantially equal.
[0015]
  As color component distribution information, various information that can be used as guidelines for adjusting white balance can be acquired.The present inventionThe color component distribution information acquisition means isDistribution ratio for each color component of the image for each divided bandIs obtained. In other words, when the image contains a lot of specific color components, the white balance tends to shift, so if the distribution ratio is obtained for each color component, it becomes easier to compare the distribution ratio of each color component, and the white balance can be adjusted easily. can do.
[0016]
Here, the color component distribution information is acquired for a predetermined reference area, or is acquired for an area in which the second color component and the third color component are distributed relatively evenly with respect to the standard first color component. If it is acquired for each predetermined band, a color failure can be prevented without being greatly influenced by a specific color of the entire image. Also in this case, various color components such as an RGB component and a CMY component can be adopted as the color component, but an RGB component close to the color distribution felt by a human cone is preferable. Furthermore, the distribution ratio may be any ratio that reflects the distribution of the color components, and various modes can be adopted. For example, the integrated value of the gradation value and level for each color component or an average value may be used. Further, the ratio may be any ratio regarding the color component distribution, and any color component may be a denominator or a numerator.
[0017]
  further,Claim 3In the invention according to the aspect of the invention, the color component distribution information acquisition unit may be configured such that when the image is expressed by a first color component to a third color component,As distribution ratio“1” is acquired for the first color component, the ratio between the distribution of the first color component and the distribution of the second color component is acquired for the second color component, and the distribution of the first color component and the first color component are acquired for the third color component. The ratio to the distribution of the three color components is obtained.
[0018]
That is, it is preferable to acquire color component distribution information that can be used for white balance adjustment as a distribution ratio, and if the ratio between the distribution of the first color component and the amount of the second color component is acquired, the first color component is obtained. The relative distribution of the second color component as a reference can be grasped, and at the same time, the relative correction amount of the second color component can be grasped. The same applies to the third color component, and if the ratio between the distribution of the first color component and the third color component is obtained, the relative distribution of the third color component based on the first color component can be grasped. can do.
[0019]
Accordingly, “1” is set for each of the first color components in the entire image, “ratio between the distribution of the first color component and the distribution of the second color component” for each of the second color components, and “ By multiplying the ratio of the distribution of the first color component and the distribution of the third color component ”, adjustment that reduces the deviation of the distribution of the color component, that is, adjustment of white balance can be performed.
[0020]
  further,The present inventionThe white balance control means averages the distribution ratio for at least two luminance bands.Turn intoThe configuration is as follows. That is, without referring to the color component distribution information for the entire image, refer to the color component distribution information for a predetermined reference region, a region where the distribution ratio of the color components is equal, or a region where the luminance component is within a predetermined band. In controlling the white balance, various guidelines can be adopted as to how the color component distribution information is acquired. As an example, if the distribution ratio is averaged over at least two luminance bands, the deviation of the color component distribution over the entire luminance range can be compensated, and this distribution ratio is made a coefficient for each color component. If the correction amount is determined in this way, white balance control can be performed while compensating for deviations in the color component distribution.
[0021]
As a more specific example, it is possible to compensate for a color gamut shift of an imaging device. That is, when the color gamut of the imaging device is considered in the Luv space, the color gamut broadens in the opposite direction in the high-luminance region and the low-luminance region. Therefore, by averaging the color component distribution ratio in the high luminance part and the low luminance part, it is possible to compensate for the color gamut shift along the luminance axis. Also, it is not necessary to average the distribution ratio only once. It is also possible to average the distribution ratio for many areas and adopt the most preferable average value to determine the correction amount for each color component. It is.
[0022]
  Furthermore, as a specific configuration example when averagingClaim 4In the invention according to claim 1, the white balance control means includesThe above histogram is divided into N bandsWhen divided, the distribution ratio of the band i and the distribution ratio of the band N + 1-i are averaged. That is, when the luminance band is divided into N (N is a positive integer) and a band number i (i is a positive integer) is assigned from the smaller luminance gradation value or level, the distribution ratio of the band i and the band If the distribution ratio of N + 1-i is averaged, the two bands at both ends of the luminance band are averaged, and the bands are sequentially averaged so that the second two bands are averaged from both ends of the luminance band. In addition, the color component distribution ratio is averaged at a portion where the color gamut of the image device is shifted to the substantially opposite side with respect to the L axis, and the shift of the color gamut can be compensated.
[0023]
  As described above, if the correction amount of the color component of the entire image is determined using the color component distribution ratio as a coefficient for each color component, white balance control can be performed while compensating for the deviation of the color component distribution. As an example of a configuration suitable for obtaining a preferable color component distribution ratio in controlling white balanceThe present inventionThen, the white balance control means is the distribution ratio.Is an averaged value ofThus, a value close to 1 is obtained as a coefficient for white balance correction.
[0024]
That is, the distribution ratio is a coefficient for adjusting the white balance. However, when a plurality of distribution ratios are calculated by considering the divided bands, the distribution ratio is very far from 1. If a product is used as the above coefficient, when it is multiplied as a coefficient of arbitrary image data, the color balance is changed greatly. Therefore, the possibility of incurring a color feria is very high. Therefore, if the distribution ratio is close to 1 and the correction amount by the coefficient is suppressed, there is almost no color failure.
[0025]
In addition, selecting a distribution ratio close to 1 is equivalent to selecting a small correction amount, but even if the correction amount is small, it functions well as white balance adjustment and is a preferable coefficient. Often. Specifically, if the one closest to 1 is acquired, it is often the most preferable coefficient, but of course, it is not essential to be closest to 1, and may be the second closest, or a predetermined value. Various configurations such as obtaining a value close to 1 within a range not exceeding the threshold value can be employed. Furthermore, even if the distribution ratio is averaged as described above, it is preferable to obtain a distribution ratio close to 1.
[0026]
  further,The present inventionThe white balance control means controls the white balance by multiplying each color component of dot matrix image data forming the image by the coefficient. That is, as described above, the distribution ratio is a coefficient for adjusting the white balance. Therefore, when the image is expressed by dot matrix image data, each color component of the image data is multiplied by the coefficient. White balance can be easily controlled. In addition, the occurrence of color feria can be prevented with a very high probability.
[0027]
  By the way, such a white balance adjustment device may exist alone, or may be used in a state of being incorporated in a certain device, but the idea of the invention is not limited to this and includes various aspects. It is a waste. Therefore, it can be changed as appropriate, such as software or hardware. As an implementation example of the idea of the invention, it corresponds to the case of becoming software of a white balance adjustment device.,UpAs a configuration corresponding to each function that allows the white balance adjustment device to be implemented by a computerAlso good. Claims 2 to 4Needless to say, it is also effective as a program corresponding to.
[0028]
Of course, the software recording medium may be a magnetic recording medium, a magneto-optical recording medium, or any recording medium to be developed in the future. In addition, the duplication stages such as the primary duplication product and the secondary duplication product are equivalent without any question. In addition, although it is not the above medium, the present invention is used even when the communication method is used as a supply method. Further, even when a part is software and a part is realized by hardware, there is nothing completely different in the idea of the invention, and a part is stored on a recording medium, and it is appropriately changed as necessary. It may be in the form of being read.
[0029]
  In addition, it is natural that such a white balance adjustment program can be applied as a method as a matter of course, as the process proceeds according to such control, an invention exists in the procedure. it can. For this reason,Claim 5The present invention has a configuration corresponding to the white balance adjustment method performed by the white balance adjustment apparatus. That is, it is not necessarily limited to a substantial apparatus, and there is no difference that the method is also effective. Of course, the aboveClaims 2 to 4It is also effective as a method corresponding to. Furthermore, the present invention can be applied to a specific device,Claim 6As described above, the present invention can also be realized as a digital camera equipped with each means capable of performing white balance adjustment.
[0030]
【The invention's effect】
  As explained above,singleA white balance adjustment device, a white balance adjustment program, and a white balance adjustment method capable of preventing color feria without being greatly affected by a color part can be provided.
  Also, SpecialIt is possible to provide a white balance adjustment device, a white balance adjustment program, and a white balance adjustment method capable of preventing color feria without being greatly affected by the presence of a constant color.
[0031]
  further,Claims 1, 5 and 6In the invention according to the present invention, the white balance adjustment device capable of preventing color failure without being greatly affected by a large specific color of the entire image., HoA weight balance adjusting method and a digital camera can be provided.
  further,Claim 2According to the invention, resources can be reduced.
[0032]
  further,Claim 1According to the invention, the luminance component can be easily divided into bands.
  further,Claim 1According to the invention, the white balance can be easily adjusted by comparing the distribution ratios of the respective color components of the image.
  further,Claim 3According to the invention, color component distribution information that can be directly used for white balance adjustment can be acquired.
  further,Claim 1According to the invention, it is possible to adjust the white balance while compensating for the deviation of the color component distribution for the entire luminance range.
[0033]
  further,Claim 4According to the invention, the color gamut shift can be easily compensated.
  further,Claim 1According to the invention, the white balance can be adjusted while the occurrence rate of color feria is very low.
  further,Claim 1According to the invention, white balance can be easily controlled by multiplying coefficients.
[0034]
DETAILED DESCRIPTION OF THE INVENTION
Here, embodiments of the present invention will be described in the following order.
(1) Schematic configuration of digital still camera:
(2) Outline of image processing:
(3) Configuration for white balance measurement and control:
(4) White balance measurement and control processing:
(4-1) Luminance component acquisition processing:
(4-2) Color component distribution information acquisition process:
(4-3) White balance control processing:
(4-3-1) Correction corresponding to light source type:
[0035]
(1) Schematic configuration of digital still camera:
FIG. 1 is a block diagram showing a schematic configuration of a digital still camera to which white balance control of the present invention is applied.
The digital still camera 10 includes an optical unit 20 and a control unit 30 having a CPU 32 as a core. The optical unit 20 includes an optical lens system 21, an autofocus mechanism 22, a distance measuring unit 23, and an autofocus controller 24, and the autofocus controller 24 controls the distance measuring unit 23 based on a control signal from the controller 25. The optical lens system 21 is driven by the autofocus mechanism 22 while the distance to the subject is measured. A subject image is formed by the optical lens system 21 on the image pickup surface of a CCD 26 as an image pickup element, and the CCD 26 is constituted by a single plate having 1800 × 1200 pixels. Since it is composed of a single plate, green (G), magenta (M), yellow (Y), and cyan (C) color filters are sequentially formed for 2 × 2 pixels. In addition, although the shutter is provided with the mechanical shutter with the electronic shutter, illustration is abbreviate | omitted about the mechanical shutter.
[0036]
In addition, the optical unit 20 is also provided with a strobe 27, which emits a predetermined amount of light and a predetermined number of times according to a drive signal from the controller 25. This realizes shooting such as red-eye prevention or slow sync. Note that an auto gain controller (AGC) 28 for amplifying an analog electric signal output from the CCD 26 is provided, and an A / D converter 29 for converting the amplified analog electric signal into a digital value and outputting the digital value. . Of course, the optical unit 20 described above is merely an example of a general configuration as the digital still camera 10, and it is needless to say that various changes can be made. For example, the optical lens system 21 may include a zoom lens, or may be an inexpensive one that does not have an autofocus function as a fixed focus.
[0037]
The control unit 30 includes a bus 31, and the CPU 32, the ROM 33, the RAM 34, and the A / D converter 29 are connected to the bus 31. The CPU 32 executes firmware written in the ROM 33 to execute control of the optical unit 20 and calculation of image processing. At this time, the RAM 34 functions as an image area 34a for storing image data or performs calculation processing. Function as a work area 34b. When the CPU 32 controls the optical unit 20, it outputs a control signal to the controller 25, and the controller 25 generates and outputs an appropriate control signal for each component circuit.
[0038]
The control unit 30 also has a function as a man-machine interface, and includes an operation panel 35 on which operation buttons and the like are arranged, and an LCD panel 36 for displaying captured images and operation instructions. The operation is monitored, and the corresponding control is executed while receiving the operation as appropriate. Further, the display on the LCD panel 36 is performed by the CPU 32 writing predetermined data in the video RAM area 34c allocated to the RAM 34, so that the display controller provided in the LCD panel 36 appropriately reads the data and displays it. . Then, image data that has been shot and generated through predetermined image processing is written to the flash memory card 37b, which is an external memory, via the I / O 37a, and is read out from the flash memory card 37b as necessary.
[0039]
The configuration of the control unit 30 is also a typical example of this type of digital still camera 10, and various changes can be made. For example, the type of external memory can be easily replaced with another type as appropriate, and image data can be output to a computer via a USB interface even without an interface with a removable memory. It may be. Another LCD display may be provided in association with the LCD panel 36 as a display unit, and the CPU 32 may be provided with a plurality of CPUs having different control targets.
[0040]
The image picked up by the optical unit 20 is output from the CCD 26 as an analog electric signal, then converted into digital data by the A / D converter 29 via the AGC 28, and further recorded in the image area 34 a of the RAM 34 via the bus 31. The Then, the CPU 32 executes predetermined image processing to convert it into JPEG image data and write it into the above-described flash memory card 37b.
[0041]
(2) Outline of image processing:
FIG. 2 shows the main flow of this image processing. Although detailed processing as described below is executed in detail, the processing is generally proceeded according to this flow. First, this outline will be described. In step S102, defective pixel interpolation is executed. A defective pixel is inevitable in the CCD 26 having about 2 million pixels, and if it is below a predetermined reference value, it is supplied as a non-defective product. For this reason, defective pixels within a non-defective range are compensated by using a 5 × 5 pixel median filter based on only the same color pixels. That is, if the defective pixel is a cyan pixel, the output values of 5 × 5 cyan pixels located around the defective pixel are arranged in order, and the median value is set as the value of the defective pixel. The defective pixel itself has already been detected by the optical unit 20.
[0042]
In the next step S104, white balance is measured. Depending on the type of light source at the time of shooting and the combination of the CCD 26 and the color filter, the white balance may be shifted. In this embodiment, the white balance is measured at step S104 for each shooting, and the result is reflected. In step S110, the white balance is controlled so as to reduce the deviation. When measuring the white balance, 113 × 75 pixel image data selected in the thinning process is used to measure the bias of the R component and the bias of the B component in the RGB color system based on the G component. The CCD 26 includes a complementary color filter, and performs control based on the fact that it is a complementary color system.
[0043]
In this embodiment, three types of images are generated. The first is a confirmation image, the second is a thumbnail image, and the third is a main image. The confirmation image is an image to be displayed on the LCD panel 36 immediately after shooting, and is composed of image data of 720 × 240 pixels. The thumbnail image is recorded as image data together with the main image, and is composed of image data of 160 × 120 pixels. The main image can be selected in three sizes according to the selection of the operator, and is either 720 × 480 pixels as an E-mail image, 1800 × 1200 pixels as a Print image, or 2160 × 1440 pixels with high resolution as Hypic2 It is.
[0044]
In FIG. 2, Steps S106 to S114 are displayed surrounded by broken lines, and at least these processes are performed for each of the three types of generated images. In step S106, data interpolation is performed. Although the CCD 26 is composed of a single plate and has 2 million pixels, the color information is not completely completed for all the pixels. That is, there is only cyan information for some pixels, only magenta information for some pixels, only yellow information for some pixels, and only green information for some pixels. This data is called mosaic data. Data interpolation is a process for completing the color information for each pixel by supplementing the information of other colors that are insufficient for each pixel. The lacking information is the average value of the color information included in the eight pixels surrounding the target pixel. For example, there are two yellow pixels (Y1, Y2), two magenta pixels (M1, M2), and four green pixels (G1-G4) around the cyan pixel (C1).
[0045]
Therefore, for the color information (C, M, Y, G) of this cyan pixel,
C = C1
M = (M1 + M2) / 2
Y = (Y1 + Y2) / 2
G = (G1 + G2 + G3 + G4) / 4
Calculate as
[0046]
In step S108, the color system is changed and converted to RGB image data. There are general formulas for conversion, but actually, a simple general formula is not used, but conversion is performed using a tuned lookup table or a determinant. After conversion into RGB image data, in step S110, as described above, control is performed to reduce the bias of the R component and the bias of the B component in the RGB color system. In the next step S112, tone curve correction is performed. Tone curve correction is also referred to as so-called γ correction. At this time, 10-bit image data is converted into 8-bit image data for each component, and further noise reduction and gradation characteristics in the white side region and the black side region are performed. The γ curve itself is also adjusted to maintain The tone curve correction itself uses a look-up table, and is converted with reference to a 10-bit → 8-bit look-up table created by tuning in advance.
[0047]
Thereafter, color conversion is performed from the RGB color system to the YUV color system in order to use the JPEG image compression technique. In order to distinguish between brightness and yellow, the brightness is hereinafter displayed as Ym, and the brightness directly derived from the mosaic color data is displayed as simple brightness Ys as described later. The color conversion from the RGB color system to the YUV color system is performed using a general formula or a determinant.
The general formula for conversion from mosaic color data to YUV color system is:
Ys = 0.25 × (C + M + Y + G)
U = C + M- (Y + G)
V = -C + M + Y-G
However, in practice, the tuning and white balance control as described above are performed, and this correspondence is not adopted.
[0048]
Steps S116 and S118 are processes performed only when the main image is created. Noise removal is performed by applying a so-called low-pass filter only for the UV component as the color component. In other words, the color change that is too steep acts on the pressing feeling. Further, edge enhancement is performed by calculating a nonlinear function value (for example, a coring function value) from an edge amount based on the luminance component only for the luminance component, and weighting the non-linear function value to the original luminance component. Therefore, when the luminance value changes between adjacent pixels, the edge amount is calculated, and if the absolute value of the edge amount is large, the amount of change is reflected in the luminance value. If the absolute value is small, the original luminance is calculated. The value is almost unchanged.
[0049]
The above is the outline of the image processing. However, the above procedure also shortens the calculation processing time by simultaneously performing a plurality of processes in the actual calculation processing. The shooting is completed by executing the above-described image processing for each of the three types of images described above, and finally writing the JPEG compressed image of the main image including the thumbnail image in the flash memory card 37b. In addition, this digital still camera can read image data from the flash memory card 37b and display it on the LCD panel 36, or can delete data and perform various settings. The description is omitted because it is applicable.
[0050]
(3) Configuration for white balance measurement and control:
Hereinafter, the white balance measurement in step S104 of the image processing and the white balance control reflecting the measurement result will be described in detail. FIG. 3 is a block diagram showing a white balance measurement system and control system realized by the configuration shown in FIG. The luminance component acquisition module 33a, the color component distribution information acquisition module 33b, and the white balance control module 33c in the figure are executed when the CPU 32 appropriately reads out program modules recorded in advance in the ROM 33. Each module appropriately uses data recorded in the image area 34a and work area 34b of the RAM 34, and determines a matrix for writing intermediate data and controlling white balance as necessary. White balance control is realized by multiplying image data by this matrix.
[0051]
The luminance component acquisition module 33a acquires CCD raw data recorded in the image area 34a and acquires luminance components. Here, the luminance component acquisition module 33a performs a thinning process for reducing the amount of calculation, a process for converting GMYC Raw data to RGBYs data, a histogram process, and a band division process for the histogram. The thinned data 34 b 1, RGBYs data 34 b 2, histogram data 34 b 3, and band data 34 b 4 generated and used as appropriate are recorded in the work area 34 b of the RAM 34. Each processing will be described in detail later. A digital still camera equipped with a primary color filter does not require conversion from GMYC data to RGB data, but correction processing is required to approximate the CCD RGB data to the characteristics of sRGB data.
[0052]
The color component distribution information acquisition module 33b includes the RGB color component integration data 34b5 as the color component distribution information within the divided band and suitable as an index for adjusting the white balance, and a distribution ratio that is a ratio thereof. Data 34b6 is acquired. The white balance control module 33c, based on the obtained distribution ratio data 34b6, is a coefficient used for white balance adjustment and a correction coefficient R 'for correcting the R component relative to the G component._c, And a correction coefficient B 'for correcting the B component relative to the G component_c(34b7) is obtained. Then, the correction coefficient is converted into a coefficient in a color space with physiological three primary colors and corrected in the color space to correct the correction coefficient R_c, B_cAnd a final white balance control matrix 34b9 is calculated based on the correction coefficient.
[0053]
The ROM 33 includes a conversion matrix A (33d1) for converting RGB color components of the sRGB coordinate system into RGB color components of physiological three primary colors based on an adaptation conversion equation such as an equation of Estevez-Hunt-Pointer and the conversion. Inverse transformation matrix A for performing inverse transformation^-1(33d2) is recorded and used by the white balance control module 33c. The white balance of the image can be controlled by multiplying the white balance control matrix 34b9 created as described above and the CCD Raw data in the image area 34a converted into RGB color components.
[0054]
In this embodiment, the white balance correction coefficient is obtained in two stages. That is, as the first stage, the correction coefficient R ′ is temporarily set._c, B '_cAfter obtaining the correction coefficient R as a second step_c, B_cSeeking. By obtaining the correction coefficient in two stages in this way, white balance control fails due to the influence of a large monochromatic part in the image, or white balance control fails when a specific light source is used. Color feria etc. can be prevented, but each of the above stages functions as white balance control even if implemented alone, and each stage is added to white balance control of another method to achieve white balance. It is good also as a structure to control. For example, the correction coefficient R ′ calculated in the first stage_c, B '_cEven if the configuration is such that the white balance control matrix 34b8 is calculated using the function, it functions as a white balance adjustment device.
[0055]
(4) White balance measurement and control processing:
Hereinafter, a process performed for white balance measurement and control in the above configuration will be described in detail. 4 and 5 are flowcharts showing white balance measurement and control. FIG. 4 corresponds to the first stage, and FIG. 5 corresponds to the second stage and the stage of calculating the final white balance control matrix 34b9. 4 corresponds to the process of the luminance component acquisition module 33a, and steps S225 to S250 of FIG. 4 correspond to the process of the color component distribution information acquisition module 33b. Steps S250 to S275 and S110 correspond to the processing of the white balance control module 33c. Here, a series of processes will be described while being divided into processes for each module.
[0056]
(4-1) Luminance component acquisition processing:
When the luminance component acquisition module 33a acquires the CCD Raw data in the image area 34a in Step S200, the thinning process is performed on the CCD Raw data in Step S205. That is, the CCD Raw data is data having 1800 × 1200 pixels as described above, and it takes a long time to execute the subsequent processing for all the pixels, and the free storage capacity required for the RAM 34 is very large. Therefore, pixels are thinned out as follows.
[0057]
FIG. 6 is an explanatory diagram for explaining the thinning process. The square on the right side of the figure is a schematic diagram of CCD Raw data, which is GMYC mosaic data of 1800 × 1200 pixels. In the present embodiment, image data having a size 1/16 of the mosaic data is used. First, a 16 × 16 pixel group (Gr1) at the upper left of the mosaic data in the figure is acquired. The pixel group Gr1 includes 64 pieces of GMYC color component data, and an average of gradation values of 64 pieces of data is taken for each color component. Then, the average of these is calculated as color component data | G | for one pixel.₁, | M |₁, | Y |₁, | C |₁And
[0058]
Thereafter, a 16 × 16 pixel group is sequentially obtained and averaged from the CCD Raw data in the same manner. As a result, the CCD Raw data is a gradation value data having a size of 1/16, that is, 113 × 75 pixels. This gradation value data is stored in the work area 34b as thinned data 34b1. Of course, in this thinning-out process, the compression ratio 1/16 is not an indispensable value, and can be appropriately changed according to the RAM 34, processing load, and the like. In addition, although averaging as described above is preferable because all information of the CCD Raw data can be taken into consideration, there are various other methods such as thinning out so that one pixel is left for each 16 pixels in the vertical and horizontal directions of the CCD Raw data. Can be adopted.
[0059]
In the present embodiment, white balance adjustment is considered in the RGB space, and the thinned data 34b1 after the thinning process in step S205 is converted into RGB components and simple luminance Ys in step S210. The conversion from the GMYC component to the RGB component uses a conversion equation similar to the conversion equation in step S108. In this conversion equation, the color conversion is performed with high accuracy by tuning as described above. It can be carried out. FIG. 7 shows the pixel data (RGBYs data 34b2) after the conversion in step S210. The pixel data after conversion is 113 × 75 pixels as shown in the figure, and is composed of RGBYs four-component gradation value data for each pixel.
[0060]
The luminance component acquisition module 33a acquires the luminance component of the pixel at this time point. However, in this embodiment, for the subsequent processing, in step S215, the simple luminance Ys of each pixel is converted into a histogram for each gradation value, and the histogram data 34b3. Is generated. FIG. 8 is a graph showing an example of this histogram. In the figure, since the gradation value pitch is fine, it is shown as a continuous curve, but the actual histogram is a discrete value. The following histogram graphs are similarly shown as curves, but are actually discrete values.
[0061]
When a histogram is created in step S215, the histogram is divided into 10 luminance bands in step S220. FIG. 9 is an explanatory diagram for explaining the state of band division in step S220. The table at the top of the figure shows the work of determining the band, and the graph at the bottom shows a histogram in which the band is divided. When dividing the band, as shown in the upper table, the histogram is accumulated from the gradation value of the simple luminance Ys “0”, and the accumulated value exceeds 847 (about 113 × 75/10). To move to the next band and repeat the integration.
[0062]
If such integration is repeated for 10 bands, the number of pixels belonging to each band becomes substantially equal. That is, the area of each band shown in the graph of FIG. 9 is substantially equal. The data after the band division is stored in the work area 34b as band data 34b4 that can determine which pixel belongs to which band. Here, since the histogram is a discrete value, the number of pixels belonging to each band is not necessarily exactly the same, but since there is a distribution ratio calculation process by the color component distribution information acquisition module 33b described later, the number of pixels The strictness of is not a problem. When the integrated value exceeds the above 847, the pixel of the gradation value may be included in only one band or both. Further, in this band division, it is only necessary to specify a suitable luminance band to be used later for white balance adjustment, and it is not essential that the number of divisions is 10, and the division method is as described above. Not limited to.
[0063]
(4-2) Color component distribution information acquisition process:
The color component distribution information acquisition module 33b acquires color distribution information from a band preferable for white balance control based on the band data 34b4. For this purpose, first in step S225, the RGB components of the pixels belonging to the same band are integrated for each of the divided bands, and are stored in the work area 34b as RGB integration data 34b5. In step S230, “integrated value of G component / integrated value of R component” and “integrated value of G component / integrated value of B component” are obtained for each band as the distribution ratio, and the above work area is obtained as distribution ratio data 34b6. 34b. FIG. 10 is an explanatory diagram for explaining how the integral value and the distribution ratio are calculated. Since it is possible to determine which pixel belongs to which band in the band data 34b4, the RGB components of the pixels belonging to each band can be easily known as shown in FIG.
[0064]
From this RGB component value, the integral value is calculated by the following equation (1),
[Expression 1]

The distribution ratio is calculated by the following formula (2).
[Expression 2]

Where R_ij, G_ij, B_ijIs a gradation value for each color component, “i” corresponds to a band number, and “j” is a convenient number for distinguishing pixels. Iri, Igi, and Ibi are integral values for each band and each component, and Ric and Bic are distribution ratios for each band.
[0065]
Here, since the integrated value is an integrated value of the gradation values of the color components, it is a value reflecting how many gradation values are distributed in each band. Also, calculating the distribution ratio using this integral value is equivalent to calculating the distribution ratio using the average gradation value for each color component in each band. The average gradation value for each color component in each band is the sum of the gradation values of the predetermined color components of the pixels belonging to the band and divided by the number of pixels. The number of pixels for each color component is the same in each band. However, in the case of the distribution ratio, the number of pixels is canceled out by the denominator and the numerator, and the only contribution to the distribution ratio is that only the component obtained by adding the gradation values.
[0066]
This distribution ratio Ric is “average value of G component for each band / average value of R component for each band”, and distribution ratio Bic is “average value of G component for each band / average value of B component for each band”. This distribution ratio is a coefficient having the property of adjusting the white balance. That is, when each RGB color component of a certain pixel is equal to the average gradation value of each color component in the band to which the pixel belongs, multiplying the distribution ratio Ric and the gradation value of the R component cancels the denominator and averages the G component Even if the distribution ratio Bic and the gradation value of the B component are multiplied, the denominator cancels and becomes equal to the average value of the G component. Therefore, all the color components of the pixel are equal and become achromatic.
[0067]
As described above, the distribution ratio Ric and the distribution ratio Bic act on the R component and the B component of the pixel, respectively, and the R component and the B component are brought close to the reference while using the average value of the G component in the band as a reference. Can be said to be a coefficient for adjusting the white balance. Therefore, it can be said that the white balance correction coefficient for each band has been calculated up to step S230. In the present invention, the white balance of the entire image is adjusted with reference to the distribution ratio of the predetermined band, and correction is performed to obtain a more preferable result regardless of differences in image content, light source, and the like.
[0068]
When the distribution ratio data is acquired in step S230, the color component distribution information acquisition module 33b acquires the average value R ′ of the set of distribution ratios according to the following equation (3) in step S235._nc, B '_ncIs calculated.
[Equation 3]

Here, n is a set number of the band and is an integer of “1-5”.
[0069]
FIG. 11 shows the average value R ′ of the set_ncIt is explanatory drawing explaining a mode that two bands are grouped. The upper graph shown in the figure shows a histogram divided into 10 bands as described above. In the above formula (3), since the average for the nth band and the (11−n) th band is calculated, the bands at both ends (first and tenth) are combined as shown in FIG. Thus, the second bands (second and ninth) from both ends are sequentially combined to form a pair. B ’_ncThe same applies to the combination concept.
[0070]
FIG. 12 is an explanatory diagram for explaining the concept when averaging by combining distribution ratios, and shows the color gamut of the RGB color space mapped onto the UY plane of the YUV color space. In the figure, the color gamut of the RGB color space is biased in the negative U direction in the high luminance region, and is biased in the positive U direction in the low luminance region. The same applies to the case of mapping on the V-Y plane, which is biased in the negative V direction in the high luminance region and biased in the positive V direction in the low luminance region. In other words, the high luminance region and the low luminance region have opposite properties. Considering this property with the above distribution ratio, a large difference occurs between the high luminance region and the low luminance region. If averaged, the difference can be canceled out, and the distribution ratio obtained by removing the distribution ratio bias due to the color gamut shape can be evaluated. Note that the idea of averaging the distribution ratio so as to remove this bias is generated from the relationship between the RGB color space and the YUV color space, and can be adopted in white balance adjustment for many image devices. it can.
[0071]
When the average value of the set of distribution ratios is calculated in step 235, R ′ is determined in step S240._ncIs smaller than “1”, and R ′ is determined in step S240._ncIs determined to be smaller than “1”, R ′ is determined in step S245._ncR '_ncSubstitute the inverse of. The determination processing in steps S240 and S245 is performed for all of the combination numbers 1 to 5 regardless of the determination result in step S240._ncAre also performed independently. Processing here is R '_ncAnd B ’_ncIs converted to a number larger than “1” if it is smaller than “1”, the following evaluation function e_nIs converted to the original value after the evaluation function.
[0072]
(4-3) White balance control processing:
In step S250, the evaluation function e_nFor each set of R '_ncAnd B ’_ncAnd the white balance control module 33c determines that the evaluation function e_nR 'of the set number n that minimizes_ncAnd B ’_ncAnd the value of the first stage white balance correction coefficient R ′_cAnd B ’_cDetermine as. This first-stage white balance correction coefficient has the same properties as the distribution ratio described above, and is a coefficient that can be a predetermined component of the white balance control matrix 34b8 shown in FIG. That is, when the white balance control matrix 34b8 is multiplied by a matrix having RGB components of arbitrary pixels in the image as components, the white balance of the image can be controlled.
[0073]
Here, the process of selecting the smallest evaluation function is R ′_ncAnd B ’_ncAre selected with the smallest difference between “1” and “1”, and the process with the smallest correction amount is selected. In other words, in white balance adjustment, it is important to prevent color failure, and if a large correction amount is used as a white balance correction coefficient, color correction often occurs due to large correction, but a correction amount is small. If selected, the occurrence probability of color feria can be made very small. It can also be said that a region with a small correction amount is selected, and a region with a small bias in color component distribution is selected compared to other regions. In this case, the luminance component acquisition module 33a corresponds to the image dividing means, and the color component distribution information acquisition module 33b selects an area with the smallest evaluation function in a region where the color component distribution is less biased. It corresponds to selecting. Further, the process of selecting the one having the smallest evaluation function is a process of selecting a region in which the R component and the B component are distributed relatively evenly with respect to the G component as compared with other regions. It can also be said.
[0074]
The present applicant has confirmed that the white balance correction coefficient determined by the above-described processing is very effective in white balance adjustment. Here, it is only necessary to prevent color feria, and evaluation is performed. The function can be changed as appropriate, for example, by selecting a function that is second closest to “1”. At this time, for example, color feria can be prevented by taking measures such as selecting the second and third closest to “1” within a predetermined range. Further, in the present invention, the white balance correction coefficient is calculated based on data belonging to a part of the luminance band of the image, and the correction amount having a small correction amount is selected as described above. Compared with the white balance adjustment that reflects the RGB integral value of the entire image as described above, it is very difficult to generate color feria.
[0075]
(4-3-1) Correction corresponding to light source type:
As described above, it is possible to perform white balance control with the first-stage white balance correction coefficient. However, in this embodiment, in order to make it look more visually, the processing after step S255 in FIG. 5 is further executed. Yes. In step S255, the first-stage white balance correction coefficient R ′ is expressed by the following equation so that the white balance correction coefficient is based on the G component gradation value “128” (median value)._cAnd B ’_cConvert.
R = 128 / R ′_c, G = 128, B = 128 / B ′_c
[0076]
In step S260, the converted coefficient RGB is converted into a physiological three primary color space R'G'B 'by the equation of Estevez-Hunt-Pointer according to the following equation (4).
[Expression 4]

[0077]
Here, the matrix A is a matrix stored in the ROM 33 and converts the sRGB coordinates to the RGB coordinates of the physiological three primary colors. The matrix A is generated by multiplying the matrix B and the matrix C. Each matrix has the above-described values, and the matrix B is a conversion matrix that converts sRGB coordinates into XYZ coordinates. It is a matrix for converting XYZ coordinates to RGB coordinates of physiological three primary colors. In the above formula, italicized characters indicate a matrix. When the physiological three primary color space data R'G'B 'is obtained in step S260, final white balance correction coefficients Rc and Bc are determined in step S265 according to the following equations.
Rc = G ′ / R ′, Bc = G ′ / B ′
[0078]
The processes in steps S255 to S265 correspond to temporarily converting sRGB coordinates to R′G′B ′ coordinates of physiological three primary colors and correcting the white balance correction coefficient in the converted space (step S265). To do. By this processing, it is possible to approach a good white balance according to human appearance, and it is possible to perform white balance control independent of the light source type. FIG. 13 and FIG. 14 are explanatory diagrams for explaining how the white balance correction coefficient is corrected by converting to physiological three primary colors.
[0079]
FIG. 13 is a diagram showing a spectral distribution of an ideal light source of sRGB (CIE standard light source D65), and FIG. 14 is a diagram showing a spectral distribution of three physiological primary colors. In the figure, the horizontal axis is the wavelength (nm), and the vertical axis is the relative value of the spectral distribution. 13 and 14, the curve having a peak near 450 nm is the B component, and the curve having a peak near 540 nm is the spectral distribution of the G component, and has a peak near 610 nm in FIG. 13 and near 580 nm in FIG. 14. The curve is the spectral distribution of the R component. Since the white balance correction coefficient is multiplied by the R component and the B component, adjustment by the white balance correction coefficient enlarges the curve of the B component and the R component for the spectral distributions in FIGS. It works to shrink or shrink.
[0080]
Since the spectral distribution shown in FIG. 14 is a distribution for the three physiological primary colors, the human perception of color is very close to this distribution, and when the white balance correction coefficient is corrected in the RGB space that is this distribution, The white balance can be controlled in a way that is very close to how you feel the colors. Specifically, in the spectral distribution shown in FIG. 13, the peak wavelength and peak intensity of the R component are larger than the peak wavelength and peak intensity of the R component of the spectral distribution shown in FIG. Therefore, if the white balance is adjusted without performing the conversion in step S260, the correction for the R component becomes relatively large, and the R component is overemphasized or weakened. Therefore, it is possible to adjust the white balance so that the color looks more red by correcting the white balance correction coefficient after the conversion in step S260. Of course, the same is true for blue. Therefore, the white balance correction coefficient can be determined so that an appropriate white balance is adjusted even for an image taken with an incandescent lamp.
[0081]
When the final white balance correction coefficients Rc and Bc are determined in step S265, in step S270, the white balance correction coefficients Rc and Bc are used as components in the first row and first column and the third row and third column, and the second row and second column. Inverse matrix A from the left of the matrix in which the component of “1” is “1” and the other components are “0”^-1Is multiplied by the matrix A from the right to generate a white balance control matrix W (34b9), and the white balance control matrix W is stored in the work area 34b of the RAM 34 in step S275.
[0082]
Inverse transformation matrix A^-1Is expressed by the following equation (5).
[Equation 5]

Also in the above formula, italicized characters indicate a matrix.
[0083]
The process up to the calculation / storage of the white balance control matrix W corresponds to the measurement of the white balance, and as the processing of the digital still camera 10, the process returns to the flowchart shown in FIG. In step S110, the white balance of each captured image is controlled by multiplying the image data obtained by converting the CCD Raw data into the sRGB data by the white balance control matrix W.
[0084]
That is, the white balance control matrix W is a matrix that operates to adjust the white balance of the image of the color image data expressed by the sRGB coordinates, and the input and output data are sRGB data. Since the white balance correction coefficient is not a correction coefficient for sRGB, the input / output data is converted to sRGB data by converting the space using the matrix A or the like. FIG. 15 is an explanatory diagram for explaining the calculation by the white balance control matrix W. In the upper matrix calculation expression, RGB on the right side is input sRGB data, and R ″ G ″ B ″ on the left side is Output sRGB data. Further, in the actual processing of the present embodiment, the white balance control matrix W is one 3 × 3 matrix obtained by calculating three 3 × 3 matrices in advance. This will be explained separately.
[0085]
In FIG. 15, each calculation stage using three matrices is shown below the matrix calculation expression. As described above, the input data (1) is sRGB data. Since the matrix A is a matrix for converting the RGB color components of the sRGB coordinate system into the RGB color components of the physiological three primary colors, when the input data (1) is multiplied by the matrix A, the physiological three primary colors are represented as shown in (2). Converted to RGB color components. When this conversion result is multiplied by a matrix including the white balance correction coefficients Rc and Bc as components, the white balance is adjusted with RGB color components of physiological three primary colors as shown in (3). This adjusted RGB data is converted into an inverse transformation matrix A.^-1Is multiplied, the image data after white balance adjustment is converted into sRGB data as shown in (4). Therefore, white balance is adjusted by multiplying the sRGB data by the white balance control matrix W which is one 3 × 3 matrix in the image processing process of the digital still camera.
[0086]
As described above, in the present invention, the luminance component of the image is acquired and divided into bands, and the white balance correction coefficient for adjusting the white balance is calculated based on the color component distribution information of the image within the predetermined band. To do. Therefore, white balance adjustment can be performed without being affected by a specific color in the image or a light source biased to the specific color.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a schematic configuration of a digital still camera.
FIG. 2 is a flowchart showing a main flow of image processing.
FIG. 3 is a block diagram showing a white balance measurement system and a control system;
FIG. 4 is a flowchart showing white balance measurement and control.
FIG. 5 is a flowchart showing white balance measurement and control.
FIG. 6 is an explanatory diagram for explaining a thinning process;
FIG. 7 is a diagram illustrating pixel data after RGBYs conversion.
FIG. 8 is a graph showing an example of a histogram.
FIG. 9 is an explanatory diagram illustrating a state of band division.
FIG. 10 is an explanatory diagram for explaining how to calculate an integral value and a distribution ratio.
FIG. 11 is an explanatory diagram for explaining how two bands are combined.
FIG. 12 is an explanatory diagram for explaining a concept when averaging by combining distribution ratios;
FIG. 13 is a diagram showing a spectral distribution of an ideal light source of sRGB.
FIG. 14 is a diagram showing a spectral distribution for three physiological primary colors.
FIG. 15 is an explanatory diagram for explaining calculation by a white balance control matrix;
[Explanation of symbols]
10. Digital still camera
20 ... Optical part
21 ... Optical lens system
22 ... Autofocus mechanism
23. Ranging section
24 ... Autofocus controller
25 ... Controller
26 ... CCD
27 Strobe
28 ... Auto gain controller (AGC)
29 ... A / D converter
30 ... Control unit
31 ... Bus
32 ... CPU
33 ... ROM
33a ... Luminance component acquisition module
33b ... Color component distribution information acquisition module
33c ... White balance control module
33d1 ... Conversion matrix A
33d2 ... Inverse transformation matrix A^-1
34 ... RAM
34a ... Image area
34b ... Work area
34b1 ... thinned data
34b2 ... RGBYs data
34b3 ... Histogram data
34b4: Band data
34b5 ... RGB integration data
34b6 ... Distribution ratio data
34b7 ... Correction coefficient R '_c, B '_c
34b8 ... White balance control matrix
34b9 ... White balance control matrix
34c ... Video RAM area
35 ... Control panel
36 ... LCD panel
37a ... I / O

Claims (6)

A luminance component acquisition unit that acquires a luminance component of an image, forms a histogram of the luminance component, and divides the band into a predetermined luminance range ;
Color component distribution information acquisition means for acquiring a distribution ratio for each color component of the image for each of the divided bands ;
The distribution ratio of at least two bands is averaged, and the average value close to 1 is set as a coefficient for white balance correction, and the white balance is controlled by multiplying each color component of the image by the coefficient. A white balance adjustment device comprising: a white balance control means. 2. The white balance according to claim 1 , wherein the luminance component acquisition unit divides dot matrix image data into predetermined sections and acquires an average value of color components for each section as the luminance component. Adjustment device. The color component distribution information acquisition unit acquires “1” for the first color component as the distribution ratio when the image is expressed by the first color component to the third color component, and the first for the second color component. The ratio of the distribution of the color component and the distribution of the second color component is acquired, and the ratio of the distribution of the first color component and the distribution of the third color component is acquired for the third color component. The white balance adjustment apparatus according to claim 1 or 2 . Said white balance control means, when dividing the histogram into N bands, the claims 1, characterized in that averaging the distribution ratio of the distribution ratio and the band N + 1-i of the band i 4. The white balance adjusting device according to any one of 3 . A luminance component acquisition step of acquiring a luminance component of an image, forming a histogram of the luminance component and dividing the band into a predetermined luminance range ;
A color component distribution information acquisition step for acquiring a distribution ratio for each color component of the image for each of the divided bands ;
The distribution ratio of at least two bands is averaged, and the average value close to 1 is set as a coefficient for white balance correction, and the white balance is controlled by multiplying each color component of the image by the coefficient. A white balance adjustment method comprising: a white balance control step. Color image data acquisition means for acquiring dot matrix-like color image data in which an image captured by a CCD element is expressed by gradation values of predetermined element colors;
Luminance component acquisition means for dividing the color image data into predetermined sections and using the average value of the color components for each section as a luminance component, forming the luminance component into a histogram, and dividing the band into a predetermined luminance range by N. When,
Color component distribution information acquisition means for acquiring a distribution ratio for each color component of the color image data for each band of the luminance component, and averaging the distribution ratio of the band i and the distribution ratio of the band N + 1-i;
A value close to 1 with the averaged distribution ratio is acquired as a coefficient for white balance correction, and the white balance is controlled by multiplying the color component to be corrected in the color image data by the coefficient. A digital camera.

Images