JP2004186876A - Digital camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a digital camera capable of properly composing a highlight part even when the images whose highlight scene is photographed with different sensitivities. <P>SOLUTION: OB processing sections 70A, 70B, LMTX 72A, 72B, WB adjustment sections 74A, 74B, gamma correction sections 76A, 76B apply black correction processing, color correction processing, white balance adjustment, and gamma correction to image data high, image data low photographed at a high sensitivity and a low sensitivity. A histogram calculation section 78 calculates the histogram of the image data high. A gain calculation section 80 calculates the adjustment gain for dynamic range adjustment of a CCD on the basis of highlight data of the histogram and a luminance level of a photographed image to calculate the gains h_gain and l_gain on the basis of the adjustment gain. A multiplier 82A multiplies the h_gain with the image data high, a multiplier 82B multiplies the l_gain with the image data low, and an adder 84 summates the products of the both. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

【００６１】
但し、ｉは、分割エリアの各々を示す添え字であり、上記の場合、０〜６３の値を取り得る。Ｎは、分割エリア数を示し、上記の場合は８×８＝６４である。ΔＥＶｉｓｏは、所定の感度（例えばＩＳＯ２００）を基準としてＥＶ値補正量であり、感度が変更されても、ＥＶ’が一定となるように調整するための値である。Ｗｉは、各分割エリアごとの重み係数であり、例えば図７に示すように中央重点測光方式の重み係数を用いる。
【００６２】
上記のように算出したＥＶ’に対し、更に、次式に示すように撮影モードに応じた露出補正ΔＥＶを行って撮影ＥＶ値を求める。求めた撮影ＥＶ値は、メモリ３９に記憶される。
【００６３】
ＥＶ＝ＥＶ’−ΔＥＶ …（２）
なお、ΔＥＶは、例えば、人物モードの場合にはΔＥＶ＝０、風景モード、夜景モードの場合にはΔＥＶ＝０．３とする。
【００６４】
上記のようにして求めた撮影ＥＶ値に基づいて撮影時の絞り値とシャッタスピードを最終的に決定する。
【００６５】
そして、シャッタボタンの全押し時に前記決定した絞り値になるように絞り駆動部４４を介して絞り１２を駆動し、また、決定したシャッタスピードとなるように電子シャッタによって電荷の蓄積時間を制御する。
【００６６】
次に、本実施の形態の作用として、デジタルカメラ１１で実行される処理ルーチンについて図８に示すフローチャートを参照して説明する。なお、図８に示す処理ルーチンは、シャッタボタン２が全押しされた場合に実行される。
【００６７】
まず、シャッタボタン２が半押しされると、ＣＰＵ３８は、高感度の画像信号の取り込みをＴＧ２２及びＣＣＤ駆動回路１６を介してＣＣＤ１４に指示する。これにより、ＣＣＤ１４から高感度の画像データがメモリ２４に一旦取り込まれ、ＣＰＵ３８によって前述したオートフォーカス制御及び自動露光制御が行われる。これにより、撮影ＥＶ値、シャッタスピード、絞り値が決定され、駆動部４２及び駆動部４４が制御される。
【００６８】
そして、シャッタボタン２が全押しされると、ステップ１００において、ＣＰＵ３８は、高感度の画像信号の取り込みをＴＧ２２及びＣＣＤ駆動回路１６を介してＣＣＤ１４に指示する。これにより、ＣＣＤ１４から高感度の画像データがメモリ２４に一旦取り込まれる。
【００６９】
次に、ステップ１０２では、ホワイトバランスを調整するためのゲイン値Ｒｇ，Ｇｇ，Ｂｇの算出が行われる。
【００７０】
次に、ホワイトバランス調整用のゲイン値の算出について、図９に示すフローチャートを参照して説明する。
【００７１】
なお、ストロボ発光しない撮影の場合には、図９に示したホワイトバランス制御は、撮影モードが夜景モード以外の場合に実行され、その他の撮影モードの場合には通常のホワイトバランス制御によりゲイン値Ｒｇ，Ｇｇ，Ｂｇが算出される。
【００７２】
また、ストロボ発光する撮影の場合には、予め定めた撮影モード及びストロボモードの場合で、かつストロボ発光すると判断した場合に図９に示すホワイトバランス制御が実行される。
【００７３】
以下の表に、ストロボ発光する撮影の場合に、図９に示したホワイトバランス制御が実行される撮影モード及びストロボモードの組み合わせを示す。例えば撮影モード及びストロボモードが共にオートモードで、ＥＶ値が所定閾値以下の場合には、以下のホワイトバランス制御が実行され、その他の場合、例えば撮影モードが風景モードでストロボモードが強制発光モードの場合には、通常のホワイトバランス制御が行われる。
【００７４】
【表１】

【００７５】
まず、ＥＶ値や撮影モード及びストロボモードからストロボ発光するか否かが判断され、撮影モードが夜景モードの場合で、かつストロボ発光しないと判断された場合には、周知の方法によりデーライト光を光源としたストロボ発光しない場合に適したホワイトバランス制御を行うためのホワイトバランス補正値Ｒｇ、Ｇｇ、Ｂｇが設定され、ＷＢ調整部７４Ａ、７４Ｂに出力される。
【００７６】
また、ストロボ発光すると判断された場合は、撮影モード及びストロボモードが上記の表１の○で示される組み合わせであるか否かが判断される。そして、ストロボ発光すると判断し、かつ撮影モード及びストロボモードが上記の表１の○で示される組み合わせでないと判断された場合には、周知の方法により通常のストロボ発光時のホワイトバランス制御を行うためのホワイトバランス補正値Ｒｇ、Ｇｇ、Ｂｇが設定され、ＷＢ調整部７４Ａ、７４Ｂに出力される。
【００７７】
そして、撮影モードが夜景モード以外で、かつストロボ発光しないと判断した場合又は撮影モード及びストロボモードが上記の表１の○で示される組み合わせであるか場合には、ＣＰＵ３８は、シャッタボタンの半押し時に求めた撮影ＥＶ値をメモリ３９から読み込む（ステップ２００）。
【００７８】
シャッタボタンの全押し時に撮像された高感度の画像のＲ、Ｇ、Ｂ信号は一旦メモリ２４に格納されているが、この画像を複数のエリア（例えば８×８）に分割し、各分割エリアごとにＲ、Ｇ、Ｂ信号の色別の積算値を積算回路４８によって求める（ステップ２０２）。
【００７９】
Ｒ，Ｇ，Ｂ信号の積算値Ｒｔ，Ｇｔ，Ｂｔは、積算回路４８とＣＰＵ３８との間に設けられた乗算器（乗算手段）５０Ｒ、５０Ｇ、５０Ｂに出力され、乗算器５０Ｒ、５０Ｇ、５０Ｂによって予め定められた基準ＷＢ（ホワイトバランス）ゲイン値（基準制御値）Ｒｃ，Ｇｃ、Ｂｃが掛けられる（ステップ２０４）。なお、基準ＷＢゲイン値Ｒｃ，Ｇｃ、Ｂｃは、予めメモリ３９に記憶されている。
【００８０】
基準ＷＢゲイン値Ｒｃ，Ｇｃ，Ｂｃが掛けられたＲ，Ｇ，Ｂ信号の積算値Ｒｔ’，Ｇｔ’，Ｂｔ’は、ＣＰＵ３８に入力される。
【００８１】
この基準ＷＢゲイン値Ｒｃ，Ｇｃ，Ｂｃは、ストロボ発光しない撮影の場合には、天気が晴れの場合のように、光源がデーライト光の場合にホワイトバランスが適正となるように調整するためのゲイン値Ｒｄ，Ｇｄ，Ｂｄが設定される。また、ストロボ発光して撮影する場合には、ストロボ光が被写体に十分に到達した場合のように、光源としてストロボ光が支配的な場合にホワイトバランスが適正となるように調整するためのゲイン値Ｒｓｔ，Ｇｓｔ，Ｂｓｔが設定される。
【００８２】
そして、ＣＰＵ３８は、入力されたＲ信号の積算値Ｒｔ’とＧ信号の積算値Ｇｔ’との比Ｒｔ’／Ｇｔ’、及びＢ信号の積算値とＧ信号の積算値との比Ｂｔ’／Ｇｔ’を求める（ステップ２０６）。
【００８３】
上記のようにして各分割エリアごとに求められるＲｔ’／Ｇｔ’、Ｂｔ’／Ｇｔ’は、その分割エリアが、図１０に示すグラフ上に表された検出枠のうちの、どの検出枠内に入るかを判別するために使用される。なお、図１０における各検出枠は、環境光源などの色分布の範囲を規定するものである。
【００８４】
図１０に示すように、検出枠には、青空検出枠、日陰検出枠、昼光色（蛍光灯）検出枠、昼白色（蛍光灯）検出枠、白色（蛍光灯）検出枠、デーライト検出枠、電球色蛍光灯検出枠、電球（タングステン電球）検出枠がある。なお、デーライト検出枠は、ストロボ発光しない撮影の場合の検出枠であり、ストロボ撮影する場合には、ストロボ検出枠となる。
【００８５】
そして、ＣＰＵ３８は、Ｒｔ’／Ｇｔ’＝Ｘ’、Ｂｔ’／Ｇｔ’＝Ｙ’として、各分割エリアごとに求められたＲｔ’／Ｇｔ’、Ｂｔ’／Ｇｔ’で表される（Ｘ_ｉ’、Ｙ_ｉ’）が、これらの検出枠のうちのどの検出枠内に入るかを判別し、その個数をカウントする（ステップ２０８）。このとき、ストロボ発光しない撮影の場合には、日陰検出枠内の個数は、分割エリアのＥＶ値が所定値以下のものについてのみカウントし、青空検出枠内の個数は、ＥＶ値が所定値以上のものだけについてカウントする。
【００８６】
次に、各検出枠内に存在する（Ｘ_ｉ’、Ｙ_ｉ’）の平均値（Ｘａ_ｊ’Ｙａ_ｊ’）を求める（ステップ２１０）。ここで、ｊは、各検出枠の各々を示す添え字である。なお、平均値でなく、重心値でもよい。
【００８７】
次に、各検出枠に対応して設けられたメンバシップ関数により、色に関する光源らしさ（日陰らしさ等）を表す評価値Ｆｃ（）を算出する（ステップ２１２）。メンバシップ関数は、検出枠内の個数を変数として色に関する光源らしさを表す評価値Ｆｃ（）を出力する関数である。すなわち、評価値が高ければ、その検出枠に対応する光源（環境光源）の可能性が高くなる。
【００８８】
Ｆｃ（ＣＬＤ）は、図１１に示すように、日陰検出枠内に入る分割エリアの個数を変数とした日陰らしさを表すメンバシップ関数の値である。
【００８９】
Ｆｃ（ＳＫＹ）は、図１２に示すように、青空検出枠内に入る分割エリアの個数を変数とした青空らしさを表すメンバシップ関数の値である。
【００９０】
Ｆｃ（ＥＸＤ）、Ｆｃ（ＥＸＮ）、Ｆｃ（Ｗ）、Ｆｃ（ＥＸＬ）、Ｆｃ（ＴＮＧ）は、図１１に示したＦｃ（ＣＬＤ）と同様のメンバシップ関数である。ここで、Ｆｃ（ＥＸＤ）は、昼光色検出枠内に入る分割エリアの個数を変数とした昼光色の蛍光灯らしさを表すメンバシップ関数であり、Ｆｃ（ＥＸＮ）は、昼白色検出枠内に入る分割エリアの個数を変数とした昼白色の蛍光灯らしさを表すメンバシップ関数の値であり、Ｆｃ（Ｗ）は、白色検出枠内に入る分割エリアの個数を変数とした白色の蛍光灯らしさを表すメンバシップ関数であり、Ｆｃ（ＥＸＬ）は、電球色蛍光灯検出枠内に入る分割エリアの個数を変数とした電球色の蛍光灯らしさを表すメンバシップ関数であり、Ｆｃ（ＴＮＧ）は、電球検出枠内に入る分割エリアの個数を変数とした電球らしさを表すメンバシップ関数の値である。
【００９１】
Ｆｃ（ＤＡＹ）は、デーライト検出枠内に入る分割エリアの個数を変数とした晴れらしさを表すメンバシップ関数の値であり、このメンバシップ関数は、図１１に示す日陰らしさを表すメンバシップ関数と同様である。
【００９２】
Ｆｃ（ＴＹＰ）は、ストロボ検出枠内に入る分割エリアの個数を変数としたストロボらしさ、すなわちストロボ光が支配的であるか否かを表すメンバシップ関数の値であり、このメンバシップ関数は、図１１に示す日陰らしさを表すメンバシップ関数と同様である。
【００９３】
次に、各検出枠に対応して設けられたメンバシップ関数により、輝度に関する光源らしさ（屋内らしさ、屋外らしさ）を表す評価値Ｆｙ（）を算出する（ステップ２１４）。メンバシップ関数は、ステップ２００で取得したＥＶ値を変数として輝度に関する光源らしさを表す評価値を出力する関数である。
【００９４】
Ｆｙ（屋外らしさＤＡＹ）は、図１３に示すように、ＥＶ値を変数としたデーライトの屋外らしさを表すメンバシップ関数の値である。
【００９５】
Ｆｙ（屋外らしさＣＬＤ）は、図１４に示すように、ＥＶ値を変数とした日陰の屋外らしさを表すメンバシップ関数の値である。
【００９６】
Ｆｙ（屋内らしさＥＸＤ）、Ｆｙ（屋内らしさＥＸＮ）、Ｆｙ（屋内らしさＴＮＧ）は、図１５に示すように、それぞれＥＶ値を変数とした昼光色の蛍光灯の屋内らしさ又は昼白色の蛍光灯の屋内らしさ又はタングステン電球の屋内らしさを表すメンバシップ関数の値である。また、Ｆｙ（屋内らしさＷ）は、ＥＶ値を変数とした白色の蛍光灯の屋内らしさを表すメンバシップ関数の値であり、Ｆｙ（屋内らしさＥＸＮ）と同様である。また、Ｆｙ（屋内らしさＥＸＬ）は、ＥＶ値を変数とした電球色の蛍光灯の屋内らしさを表すメンバシップ関数の値であり、Ｆｙ（屋内らしさＴＮＧ）と同様である。
【００９７】
次に、色に関する光源らしさを表す評価値及び輝度に関する評価値を表す評価値から各検出枠について総合的な光源らしさの評価値Ｈ_ｊを求める（ステップ２１６）。なお、ストロボ発光しない場合で、撮影モードが風景モードの場合には、Ｈ_１〜Ｈ_２、Ｈ_７についてのみ行う。これは、風景モードの場合は、通常屋外で撮影されるが、屋外の場合に光源がタングステン電球等である場合は考えられないからである。
【００９８】
評価値Ｈ_１〜Ｈ_７は、以下の式によって求められる。
【００９９】
Ｈ_１（日陰らしさの評価値）＝Ｆｃ（ＣＬＤ）×Ｆｙ（屋外らしさ）×Ｆｃ（ＳＫＹ） …（３）
Ｈ_２（昼光色の蛍光灯らしさの評価値）＝Ｆｃ（ＥＸＤ）×Ｆｙ（屋内らしさＥＸＤ） …（４）
Ｈ_３（昼白色の蛍光灯らしさの評価値）＝Ｆｃ（ＥＸＮ）×Ｆｙ（屋内らしさＥＸＮ） …（５）
Ｈ_４（白色の蛍光灯らしさの評価値）＝Ｆｃ（Ｗ）×Ｆｙ（屋内らしさＷ） …（６）
Ｈ_５（電球色蛍光灯らしさの評価値）＝Ｆｃ（ＥＸＬ）×Ｆｙ（屋内らしさＥＸＬ） …（７）
Ｈ_６（電球らしさの評価値）＝Ｆｃ（ＴＮＧ）×Ｆｙ（屋内らしさＴＮＧ） …（８）
Ｈ_７（晴れらしさの評価値）＝Ｆｃ（ＤＡＹ）×Ｆｙ（屋外らしさＤＡＹ） …（９）
なお、ストロボ撮影する場合には、Ｈ_７は次式となる。
【０１００】
Ｈ_７（ストロボらしさの評価値）＝Ｆｃ（ＴＹＰ）×Ｆｙ（屋外らしさＤＡＹ） …（１０）
ここで、０≦Ｆｃ（）、Ｆｙ（）≦１であるため、０≦Ｈ_ｊ≦１である。なお、Ｆｃ（ＳＫＹ）は、図１２に示すように、青空検出枠内に入る分割エリアの個数が多い程、日陰らしさの評価値を下げる方向に作用する値をとる。
【０１０１】
また、Ｆｙ（屋内らしさ）は、図１５に示すように、ＥＶ値が高い程、それらの検出枠の評価値を下げる方向に作用する値をとる。このように、光源らしさの評価値Ｈ_ｊは、色分布だけでなく輝度も考慮して決定されるため、適正に光源を特定することが可能となる。例えば、蛍光灯が点灯した室内と室外とで同じ色の被写体を撮影した場合に、色分布のみで光源を特定した場合には、光源の特定を誤ってしまい、カラーフェリアが発生してしまう場合がある。
【０１０２】
しかしながら、本実施の形態では、輝度が低い場合、すなわち室内で撮影されたような場合には、Ｆｙ（室内らしさ）が高くなり、輝度が高い場合、すなわち室外で撮影されたような場合には、Ｆｙ（室内らしさ）が低くなるため、光源を正確に特定することができ、カラーフェリアが発生するのを防ぐことができる。
【０１０３】
そして、上記のように各検出枠に対応した光源らしさの評価値が算出されると、これらの評価値のうち予め定めた所定閾値以上の（例えば０．４以上）か否かを判別する（ステップ２１８）。そして、評価値が所定閾値以上のものが存在する場合には、評価値が所定閾値以上のものを環境光源の候補として選択する。
【０１０４】
次に、選択された環境光源の各々について、以下の式に従って環境光源又はストロボ光と環境光源とが混合された混合光源の色が白色となるようにホワイトバランス制御を行うための制御値（ゲイン値）Ｇ_Ｒ、Ｇ_Ｂ、Ｇ_Ｇを算出する（ステップ２２０）。
【０１０５】
Ｇ_Ｒ＝｛Σ（ｇｒ_ｊ×Ｈ_ｊ）／ΣＨ_ｊ｝×｛Σ（Ｇ_ｊ×Ｈ_ｊ）／ΣＨ_ｊ｝ …（１１）
Ｇ_Ｂ＝｛Σ（ｇｂ_ｊ×Ｈ_ｊ）／ΣＨ_ｊ｝×｛Σ（Ｇ_ｊ×Ｈ_ｊ）／ΣＨ_ｊ｝ …（１２）
Ｇ_Ｇ＝Σ（Ｇ_ｊ×Ｈ_ｊ）／ΣＨ_ｊ …（１３）
ここで、上記（１１）〜（１３）式は、評価値が所定閾値以上のものについて算出され、例えば所定閾値以上の評価値がＨ_２〜Ｈ_５である場合には、Ｈ_２〜Ｈ_５の光源についてｇｒ_ｊ，ｇｂ_ｊが算出され、算出されたｇｒ_ｊ，ｇｂ_ｊ、Ｈ_２〜Ｈ_５を用いて各Σ内の計算が行われる。また、ｇｒ_ｊ，ｇｂ_ｊは、各検出枠における光源の色を所定色とするために必要なゲインであり、次式で示される。
【０１０６】
ｇｒ_ｊ＝Ｘｃ／Ｘａ_ｊ’×Ｔｘ_ｊ …（１４）
ｇｂ_ｊ＝Ｙｃ／Ｙａ_ｊ’×Ｔｙ_ｊ …（１５）
ここで、Ｘｃ＝Ｒｃ／Ｇｃ、Ｙｃ＝Ｂｃ／Ｇｃである。また、Ｔｘ_ｊ、Ｔｙ_ｊは、基準ＷＢゲイン値が掛けられていない元の画像データのＲ，Ｇ，Ｂの積算値について求めたＸ_ｉ、Ｙ_ｉの平均値をＸａ_ｊ（＝Ｘａ_ｊ’／Ｘｃ）、Ｙａ_ｊ（＝Ｙａ_ｊ’／Ｙｃ）とした場合における、Ｘａ_ｊ、Ｙａ_ｊの目標値であり、所定色に対応する。すなわち、ｇｒ_ｊ，ｇｂ_ｊは、元の画像データのＲ，Ｇ，Ｂの積算値についてＸａ_ｊ、Ｙａ_ｊを求めた場合に、このＸａ_ｊ、Ｙａ_ｊで示される光源（環境光源又はストロボ光と環境光源との混合光源）の色を所定色とするために必要なゲインである。なお、Ｔｘ_ｊ、Ｔｙ_ｊは、例えば１である。
【０１０７】
従って、上記（１１）、（１２）式の第１項は、評価値Ｈ_ｊを重みとしたｇｒ_ｊ，ｇｂ_ｊの加重平均値をそれぞれ表している。また、第２項は、明るさを補正するための項であり、ストロボ光を発光しない撮影の場合におけるＧ_ｊは次式で表される。
【０１０８】
Ｇ_ｊ＝ｇ_ｊ×（露出補正ゲイン／１２８） …（１６）
ここで、ｇ_ｊは、光源が晴れの場合に適したホワイトバランス制御を行うために必要なＧ（グリーン）のゲインである。露出補正ゲインは、以下の表のようになる。
【０１０９】
【表２】

【０１１０】
このように明るさを補正するのは、ストロボ光が発光しない場合は周囲の光源だけであるため、明るさが足りなくなる場合があるためである。
【０１１１】
また、ストロボ光を発光する撮影の場合には、Ｇ_ｊはＧｓｔであり、固定値となるため、明るさは補正されない。
【０１１２】
なお、デーライト検出枠（又はストロボ検出枠）の評価値Ｈ_７が所定閾値以上であるとして選択されている場合には、ｇｒ_７＝Ｘｃ、ｇｂ_７＝Ｙｃとする。
【０１１３】
このようにして、環境光源又は混合光源の色を所定色にするための制御値Ｇ_ＲＲＧ_Ｂ、Ｇ_Ｇが算出されると、これらの制御値が、基準ＷＢゲイン値との間の値で、かつ環境光源又は混合光源に適した値となるように、環境光源に応じて調整される（ステップ２２２）。調整後の制御値は、ホワイトバランス補正値Ｒｇ、Ｇｇ、Ｂｇとして表され、次式により得られる。
【０１１４】
Ｒｇ＝（Ｇ_Ｒ−Ｒｄ）×Ｈ_ｍａｘ×Ｌ_ｊ＋Ｒｄ …（１７）
Ｇｇ＝（Ｇ_Ｇ−Ｇｄ）×Ｈ_ｍａｘ×Ｌ_ｊ＋Ｇｄ …（１８）
Ｂｇ＝（Ｇ_Ｂ−Ｂｄ）×Ｈ_ｍａｘ×Ｌ_ｊ＋Ｂｄ …（１９）
ここで、Ｈ_ｍａｘは、Ｈ_ｊの最大値であり、Ｈ_ｍａｘに対応する光源を撮影時の光源として特定することができる。例えばＨ_ｍａｘがＨ_２であれば、光源は、昼光色の蛍光灯であると判断することができる。また、Ｌ_ｊはローワードコレクション係数であり、各光源に対応して予め定められた値であり、各々の光源下での撮影においてホワイトバランスが適正となるように実験等によって決定された値である。ローワードコレクション係数は、予めメモリ３９に記憶されている。なお、Ｌ_ｊは光源に拘わらず固定値としてもよい。
【０１１５】
また、０≦Ｌ_ｊ≦１であり、Ｌ_ｊ＝０の場合は、Ｒｇ＝Ｒｃ、Ｇｇ＝Ｇｃ、Ｂｇ＝Ｂｃとなり、基準ＷＢゲイン値でのホワイトバランス制御が行われる。また、Ｌ_ｊ＝１及びＨ_ｍａｘ＝１の場合には、Ｒｇ＝Ｇ_Ｒ、Ｇｇ＝Ｇ_Ｇ、Ｂｇ＝Ｇ_Ｇとなる。このように、ホワイトバランス補正値Ｒｇ、Ｇｇ、Ｂｇは、基準ＷＢゲイン値から環境光源の色を所定色にするための制御値Ｇ_Ｒ、Ｇ_Ｂ、Ｇ_Ｇとの間の値で、かつＨ_ｍａｘで特定された環境光源に適したホワイトバランス制御を行うことができる値となる。
【０１１６】
このように算出されたホワイトバランス補正値Ｒｇ、Ｇｇ、Ｂｇは、ＷＢ調整部７４Ａ、７４Ｂへ出力され（ステップ２２４）、元の高感度及び低感度の画像データのＲ、Ｇ、Ｂの各信号がホワイトバランス調整される。補正後の信号をＲ’、Ｇ’、Ｂ’とすると、Ｒ’、Ｇ’、Ｂ’は、次式で表される。
【０１１７】
Ｒ’＝Ｒｇ×Ｒ …（２０）
Ｇ’＝Ｇｇ×Ｇ …（２１）
Ｂ’＝Ｂｇ×Ｂ …（２２）
一方、所定閾値以上の評価値Ｈ_ｊが１つもない場合には、ホワイトバランス補正値Ｒｇ、Ｇｇ、ＢｇをそれぞれＲｃ、Ｇｃ、Ｂｃに設定して（ステップ２２６）、ＷＢ調整部７４Ａ、７４Ｂへ出力する（ステップ２２４）。
【０１１８】
このように、本実施形態では、環境光源の色だけでなくＥＶ値をも考慮して環境光源を定めるため、環境光源を正確に定めることができる。また、環境光源の色は、本撮影での撮像画像の画像データから求めているため、撮像シーケンスを複雑にすることなく環境光源を正確に定めることができる。
【０１１９】
そして、ホワイトバランス補正値Ｒｇ、Ｇｇ、Ｂｇは、基準ＷＢゲイン値から環境光源の色を所定色にするための制御値Ｇ_Ｒ、Ｇ_Ｂ、Ｇ_Ｇとの間の値で、かつＨ_ｍａｘで特定された環境光源に適したホワイトバランス制御を行うことができる値となるため、各々の環境光源に対して適正にホワイトバランス制御することができ、カラーフェリアの発生を抑えることができる。
【０１２０】
ステップ１０４では、ＯＢ処理部７０Ａにより、高感度の画像データに対して黒補正処理が行われる。
【０１２１】
ステップ１０６では、ＬＭＴＸ７２Ａにより、高感度の画像データに対して３×３マトリクスによる色補正処理が行われる。
【０１２２】
ステップ１０８では、ＷＢ調整部７４により、ステップ１０２で算出されたホワイトバランス補正用のゲイン値Ｒｇ，Ｇｇ，Ｂｇが高感度のＲ，Ｇ，Ｂデータに各々乗算されホワイトバランスの調整が行われる。なお、Ｇのデータは、予め定めた１より大きい所定値（例えば１．３７５）を乗算するようにしてもよい。これは、例えば光源が電球のような赤みの多い光源の場合には、ＧよりもＲのゲインを下げたい場合があり、このような場合にはＧのゲインを１より大きい所定値とすることにより、相対的にＲのゲインを下げることができるからである。
【０１２３】
ステップ１１０では、高感度の画像データに対してガンマ補正処理が行われる。
【０１２４】
また、上記のステップ１０２〜１１０の処理と並行して、ステップ１１２において、低感度の画像信号処理が行われる。すなわち、ＣＰＵ３８が、低感度の画像信号の取り込みをＴＧ２２及びＣＣＤ駆動回路１６を介してＣＣＤ１４に指示する。これにより、ＣＣＤ１４から低感度の画像データがメモリ２４に取り込まれる。
【０１２５】
そして、高感度の画像データと同様に、低感度の画像データに対して、ＯＢ処理部７０Ｂ、ＬＭＴＸ７２Ｂ、ＷＢ調整部７４Ｂ、ガンマ補正部７６Ｂによってそれぞれ黒補正処理、色補正処理、ホワイトバランス調整、ガンマ補正が行われる。
【０１２６】
このように、高感度の画像データに基づいてホワイトバランス調整用のゲイン値を算出する処理等と並行して低感度の画像データの処理を行うため、処理の高速化を図ることができる。
【０１２７】
また、ステップ１０８の高感度の画像データに対するホワイトバランス調整処理及びステップ１１０のガンマ補正処理と並行して、ステップ１１４において、ヒストグラム算出部７８により画像データのヒストグラム算出処理が行われ、ステップ１１６において、算出したヒストグラムに基づいてダイナミックレンジを調整するためのゲインを算出するゲイン算出処理が行われる。
【０１２８】
ヒストグラム算出部７８では、高感度の画像データのうち例えばＧデータのヒストグラムを算出する。まず、１２ビットのＧデータを例えば６ビットシフトして６ビット換算のデータとしてヒストグラムを求める。このようなヒストグラムの一例を図１６に示す。
【０１２９】
そして、予め定めたハイライト側、すなわちＧデータの累積頻度１００％からＸ％（例えば０．１％）下がった位置のデータをハイライト側の最大値とし（図１６では‘５２’）、これを６ビットシフト（２^６倍）した値をハイライトデータとする。このハイライトデータは、ホワイトバランス調整用のゲインを掛けたときにハイライト側のデータが飽和するか否かを表す情報として用いられる。
【０１３０】
ゲイン算出部８０では、このハイライトデータと、撮影ＥＶ値とに基づいて、ＣＣＤ１４のダイナミックレンジを調整するための調整ゲインを算出する。そして、この調整ゲインに基づく高感度のＲ，Ｇ，Ｂデータに乗算するためのｈ＿ｇａｉｎ及び低感度のＲ，Ｇ，Ｂデータに乗算するためのｌ＿ｇａｉｎを算出する。
【０１３１】
まず、ストロボ発光しない撮影の場合には、図１７（Ａ）に示すようなハイライトデータと評価値１との関係を表すメンバシップ関数により、ハイライトデータに対応する評価値１を求め、図１７（Ｂ）に示すような撮影ＥＶ値と評価値２との対応関係を表すメンバシップ関数により、撮影ＥＶ値に対応する評価値２を求める。
【０１３２】
評価値１を算出するメンバシップ関数は、図１７（Ａ）に示すように、ホワイトバランス調整用のゲイン（例えば１．３７５）を掛けたときにハイライト側のデータが飽和する所定値（図１７（Ａ）では３０００）までは、ハイライトデータが大きくなるに従って評価値１も徐々に大きくなり、ハイライトデータが所定値以上の場合には一定値‘１’となるような関数である。
【０１３３】
また、評価値２を算出するメンバシップ関数は、図１７（Ｂ）に示すように、撮影ＥＶ値が晴天の場合のＥＶ値（図１７（Ｂ）では１３．５）までは、撮影ＥＶ値が大きくなるに従って評価値２も徐々に大きくなり、撮影ＥＶ値が晴天の場合のＥＶ値以上の場合には一定値‘１’となるような関数である。
【０１３４】
そして、評価値１に評価値２を乗算する。この評価値１に評価値２を乗算した値は、撮影シーンのハイコントラストの度合い、すなわち撮影シーンのハイコントラストらしさを表す評価値であり、値が大きいほどハイコントラストシーンらしく、値が小さいほどハイコントラストシーンらしくなく、ローコントラストシーンらしくなる。このように、ヒストグラムを用いてハイライト側の最大値を求め、これと被写体輝度とにより精度よくハイコントラストシーンの検出が可能となる。
【０１３５】
次に、ダイナミックレンジを調整するためのトータルゲインｐ（調整ゲイン）を図１７（Ｃ）に示すようなトータルゲインｐと評価値１×評価値２との関係を表すメンバシップ関数により求める。
【０１３６】
トータルゲインｐを算出するメンバシップ関数は、図１７（Ｃ）に示すように、評価値１×評価値２が大きくなるに従ってトータルゲインｐが徐々に小さくなるような関数である。トータルゲインｐは１未満の値であり、例えば図１７（Ｃ）に示すように、０．８〜０．９の範囲の値となる。
【０１３７】
なお、ストロボ発光する撮影の場合には、被写体の輝度とハイコントラストシーンとの関係が薄いと考えられるため、撮影ＥＶ値に基づく評価値２は算出せず、図１８（Ａ）に示すメンバシップ関数により、ハイライトデータに基づく評価値１を求め、図１８（Ｃ）に示すメンバシップ関数により評価値１に基づくトータルゲインｐを求める。
【０１３８】
そして、このトータルゲインｐを用いた高感度の画像データと低感度の画像データの合成式から高感度の画像データに乗算するｈ＿ｇａｉｎ及び低感度の画像データに乗算するｌ＿ｇａｉｎを算出する。
【０１３９】
合成式は、合成データをｄａｔａ、高感度のデータをｈｉｇｈ、低感度のデータをｌｏｗとして次式で表される。
【０１４０】
ｄａｔａ ＝ ［ｈｉｇｈ＋Ｍｉｎ（ｈｉｇｈ／ｔｈ，１）×ｌｏｗ］ × Ｍａｘ（１−Ｓ×ｈｉｇｈ／ｔｈ，ｐ） …（２３）
ここで、ｔｈは閾値であり、Ｓは、高感度の画像データと低感度の画像データとの飽和比をＳｈ：Ｓｌとした場合に、Ｓ＝１−｛Ｓｈ／（Ｓｈ＋Ｓｌ）｝で表される値である。例えば、飽和比Ｓｈ：Ｓｌを４：１とした場合、Ｓ＝０．２である。
【０１４１】
（２３）式より、合成データｄａｔａは、高感度の画像データｈｉｇｈに重みＭｉｎ（ｈｉｇｈ／ｔｈ，１）が乗算された低感度の画像データを加算し、これに重みＭａｘ（−０．２×ｈｉｇｈ／ｔｈ＋１，ｐ）が乗算されたものである。
【０１４２】
重みＭｉｎ（ｈｉｇｈ／ｔｈ，１）は、図１９に示すように、高感度の画像データｈｉｇｈがｔｈよりも小さい区間では、ｈｉｇｈが大きくなるに従って傾き（ｈｉｇｈ／ｔｈ）で大きくなる値を出力し、ｈｉｇｈがｔｈ以上の場合には‘１’を出力する関数である。
【０１４３】
また、重みＭａｘ（１−Ｓ×ｈｉｇｈ／ｔｈ＋１，ｐ）は、図２０に示すように、高感度の画像データｈｉｇｈがｔｈよりも小さい区間では、ｈｉｇｈが大きくなるに従って傾き（１−Ｓ×ｈｉｇｈ／ｔｈ）で小さくなる値を出力し、ｈｉｇｈがｔｈ以上の場合にはＳを出力する関数である。なお、図２０では、Ｓ＝０．２、ｐ＝０．８の場合について示した。
【０１４４】
図２１には、高感度の画像データｈｉｇｈ、低感度の画像データｌｏｗ、及び上記（２３）式の第１項までの合成画像データｄａｔａにおける輝度−階調値（階調値は８ビット換算値）の特性を示した。図２１に示すように、上記（２３）式の第１項は、高感度の画像データｈｉｇｈに図１９に示すような特性の重みＭｉｎ（ｈｉｇｈ／ｔｈ，１）が乗算された低感度の画像データｌｏｗが加えられた値となる。
【０１４５】
従って、輝度がＥ１の範囲（高感度の画像データｈｉｇｈが閾値ｔｈ以下の範囲）では、高感度の画像データｈｉｇｈに、ｈｉｇｈ／ｔｈの重みが乗算された低感度の画像データｌｏｗが加えられた値となり、輝度がＥ２の範囲では、高感度の画像データｈｉｇｈと低感度の画像データｌｏｗが１対１で加算された値となる。
【０１４６】
また、図２２には、高感度の画像データｈｉｇｈ、低感度の画像データｌｏｗ、及び上記（２３）式の合成画像データｄａｔａにおける輝度−階調値の特性を示した。合成画像データｄａｔａは、高感度の画像データｈｉｇｈに図１９に示すような特性の重みＭｉｎ（ｈｉｇｈ／ｔｈ，１）が乗算された低感度の画像データｌｏｗが加えられた値に、さらに図２０に示すような特性の重みＭａｘ（１−Ｓ×ｈｉｇｈ／ｔｈ＋１，ｐ）が乗算された値である。
【０１４７】
従って、輝度がＥ１の範囲では、合成データｄａｔａは、上記（２３）式の第１項の値に（１−Ｓ×ｈｉｇｈ／ｔｈ）の重みが乗算された値となり、輝度がＥ２の範囲では、上記（２３）式の第１項の値にトータルゲインｐが乗算された値となる。
【０１４８】
ここで、トータルゲインｐは、０．８〜０．９の範囲に設定されるため、図２２に示すように、合成データｄａｔａの特性を表すラインは、高感度の画像データｈｉｇｈの特性を表すラインよりも全体的に下側に下がると共に、上昇の仕方が緩やかとなる。すなわち、合成データｄａｔａのダイナミックレンジは、高感度の画像データｈｉｇｈと比較して広くなる。
【０１４９】
ダイナミックレンジは、トータルゲインｐが小さくなるに従って、すなわちハイコントラストシーンらしさが高いほど広くなり、トータルゲインｐが大きくするに従って、すなわちハイコントラストシーンらしさが低いほど狭くなる。
【０１５０】
これにより、ハイコントラストの度合いが高いときにダイナミックレンジが広くなるため、ハイコントラストシーンを撮影した場合において高感度と低感度の画像データを合成した場合でもハイライト部を適正に再現することができ、ロバスト性のある階調制御が可能となる。
【０１５１】
ところで、上記（２３）式は、以下のように変形することができる。
【０１５２】
ｄａｔａ ＝ （ｈｉｇｈ＋ｗｌ×ｌｏｗ） × ｗｔ
＝ ｗｔ×ｈｉｇｈ ＋ （ｗｌ×ｗｔ）×ｌｏｗ …（２４）
ここで、ｗｌ＝Ｍｉｎ（ｈｉｇｈ／ｔｈ，１）、ｗｔ＝ Ｍａｘ（１−Ｓ×ｈｉｇｈ／ｔｈ＋１，ｐ）である。
【０１５３】
上記（２４）式より、ｈ＿ｇａｉｎ＝ｗｔ、ｌ＿ｇａｉｎ＝ｗｌ・ｗｔとなる。従って、ｈ＿ｇａｉｎ、ｌ＿ｇａｉｎは、ｗｌ、ｗｔ、すなわちＭｉｎ（ｈｉｇｈ／ｔｈ，１）、Ｍａｘ（１−Ｓ×ｈｉｇｈ／ｔｈ＋１，ｐ）を求めることにより定める。そして、合成データは、ｈ＿ｇａｉｎで重み付けした高感度の画像データとｌ＿ｇａｉｎで重み付けした低感度の画像データとを加算することにより得られる。
【０１５４】
ステップ１１２では、乗算器８２Ａにより、ガンマ補正された高感度のＲ，Ｇ，Ｂデータにステップ１１６で算出したｈ＿ｇａｉｎを乗算して加算器８４に出力すると共に、乗算器８２Ｂにより、ガンマ補正された低感度のＲ，Ｇ，Ｂデータにステップ１１６で算出したｌ＿ｇａｉｎを乗算して加算器８４に出力する。
【０１５５】
ステップ１２０では、加算器８４により、ｈ＿ｇａｉｎが乗算された高感度のＲ，Ｇ，Ｂデータにｌ＿ｇａｉｎが乗算された低感度のＲ，Ｇ，Ｂデータが加算され、各色の合成データｄａｔａが生成される。
【０１５６】
ステップ１２２では、リミッタ８６により、各色の合成データｄａｔａが予め定めたビット数（例えば８ビット）で表現できる範囲に収まるように調整される。
【０１５７】
ステップ１２４では、同時化回路３２により、メモリ２４から読み出された点順次のＲ、Ｇ、Ｂ信号が同時式に変換され、ＹＣ信号作成回路３４へ出力される。
【０１５８】
ステップ１２６では、ＹＣ信号作成回路３４により、合成されたＲ、Ｇ、Ｂ信号から輝度信号Ｙとクロマ信号Ｃｒ、Ｃｂとが作成される。これらの輝度信号Ｙとクロマ信号Ｃｒ、Ｃｂ（ＹＣ信号）は、メモリ３６に格納される。
【０１５９】
ステップ１２８では、圧縮／伸長回路５４により、ＹＣ信号が所定のフォーマットに圧縮される。
【０１６０】
ステップ１３０では、記録部５６により、圧縮された画像データがメモリカードなどの記録媒体に記録される。
【０１６１】
このように、本実施の形態では、画像データのヒストグラムを用いてハイライト側の最大値を求め、これと被写体輝度とにより精度よくハイコントラストらしさを表す評価値を求め、この評価値からダイナミックレンジを調整するトータルゲインｐを求めて高感度と低感度の画像データの特性を変化させている。
【０１６２】
これにより、撮影シーンに応じて適正にダイナミックレンジをコントロールすることができる。例えばハイコントラストらしさの評価値が高いときにのみダイナミックレンジが広くなるため、ハイコントラストシーンを撮影した場合には、どのような撮影シーンにおいても同じように高感度と低感度の画像データを合成する場合と比較して、ハイライト部を適正に再現することができ、ロバスト性のある階調制御が可能となる。
【０１６３】
また、本実施の形態では、高感度の画像データに基づいてホワイトバランス調整用のゲイン値を算出する処理等と並行して低感度の画像データの処理を行うため、処理の高速化を図ることができる。
【０１６４】
なお、本実施の形態では、高感度の受光素子ＰＤ１と低感度の受光素子ＰＤ２の各々を設け、高感度信号及び低感度信号を得る例について説明したが、図２３に示されるように、１つの受光素子ＰＤの受光領域をチャネルストッパ９４により高感度の受光を行う受光面積が広い高感度受光領域９２と低感度の受光を行う受光面積が狭い低感度受光領域９０とに分割し、それぞれの領域により高感度信号及び低感度信号が得られるような構成としてもよい。なお、受光素子ＰＤにはチャネルストッパ９４が設けられているため、高感度で受光された信号と低感度で受光された信号とが混合されずに、双方の信号を別々に受光することができる。
【０１６５】
【発明の効果】
以上説明したように本発明によれば、異なる感度で撮影した画像を合成する場合において、ハイコントラストシーンを撮影した場合でもハイライト部を適正に再現することができる、という効果を有する。
【図面の簡単な説明】
【図１】本発明に係るデジタルカメラの背面図である。
【図２】デジタルカメラの上面に設けられたモードダイヤルの平面図である。
【図３】デジタルカメラの内部構成を示すブロック図である。
【図４】ＣＣＤの概略構成図である。
【図５】デジタル信号処理回路のブロック図である。
【図６】ＥＶ値の求め方について説明するための図である。
【図７】測光方式について説明するための図である。
【図８】デジタルカメラの処理の流れを示すフローチャートである。
【図９】ホワイトバランス調整用のゲイン算出処理の流れを示すフローチャートである。
【図１０】光源の色分布の範囲としての検出枠を示すグラフである。
【図１１】日陰らしさを表すメンバシップ関数を示すグラフである。
【図１２】青空のメンバシップ関数を示すグラフである。
【図１３】デーライトの屋外らしさを表すメンバシップ関数を示すグラフである。
【図１４】屋外らしさを表すメンバシップ関数を示すグラフである。
【図１５】屋内らしさを表すメンバシップ関数を示すグラフである。
【図１６】画像データのヒストグラムである。
【図１７】（Ａ）はハイライトデータと評価値１との関係を示す線図、（Ｂ）は被写体輝度と評価値２との関係を示す線図、（Ｃ）は評価値１×評価値２とトータルゲインｐとの関係を示す線図である。
【図１８】（Ａ）はハイライトデータと評価値１との関係を示す線図、（Ｂ）は被写体輝度と評価値２との関係を示す線図である。
【図１９】高感度の画像データと重みとの関係を示す線図である。
【図２０】高感度の画像データと重みとの関係を示す線図である。
【図２１】高感度画像、低感度画像、及び合成画像の輝度−階調特性を示す線図である。
【図２２】高感度画像、低感度画像、及び合成画像の輝度−階調特性を示す線図である。
【図２３】ＣＣＤの他の形態の概略構成図である。
【符号の説明】
１０ 撮影レンズ
１１ デジタルカメラ
１２ ＣＣＤ
１８ ＣＤＳ回路
２０ Ａ／Ｄ変換器
２６ デジタル信号処理回路
３０ 合成処理部
３２ 同時化回路
３４ ＹＣ信号作成回路
４０ カメラ操作部
５２ 液晶モニタ
５４ 伸長回路
５６ 記録部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a digital camera, and more particularly to a digital camera capable of expanding a dynamic range by synthesizing image signals having different sensitivities.
[0002]
[Prior art]
The dynamic range of an image sensor such as a CCD in a digital camera that is currently widely used is generally narrower than that of a photographic film. For this reason, when shooting a high-contrast subject, the amount of received light exceeds the dynamic range, the output of the image sensor is saturated, and subject information may be lost. In order to solve such a problem, there has been proposed a technique for expanding the dynamic range by synthesizing photographed images photographed with different sensitivities (see, for example, Patent Document 1).
[0003]
In this technique, a white balance correction value is calculated for each captured image captured with different sensitivities, and white balance adjustment is performed for each captured image.
[0004]
When shooting very bright subjects in a high-contrast shooting environment, combining images shot with different sensitivities in this way leads to an expansion of the dynamic range, and detail reproduction of bright areas (highlight areas). This is effective.
[0005]
[Patent Document 1]
JP 2001-94999 A (page 3, FIG. 9)
[0006]
[Problems to be solved by the invention]
However, in the above prior art, when shooting in a low-contrast shooting environment such as cloudy or indoors, the dynamic range is widened so that the gradation of the highlight portion becomes soft (smooth), and on the contrary, the highlight portion There is a problem that the reproduction of the image may deteriorate.
[0007]
The present invention has been made in consideration of the above facts, and in the case of synthesizing images shot at different sensitivities, a digital camera capable of appropriately reproducing the highlight portion even when shooting a high contrast scene is provided. The purpose is to provide.
[0008]
[Means for Solving the Problems]
In order to achieve the above object, the invention described in claim 1 is configured to capture an image of a subject with a first sensitivity and a second sensitivity lower than the first sensitivity, and with the first sensitivity. Based on at least one of the first picked-up image and the second picked-up image picked up with the second sensitivity, a luminance level detecting means for detecting the luminance level of the subject and a histogram of the first picked-up image are calculated. And calculating an evaluation value representing the likelihood of high contrast based on highlight data on a predetermined highlight side of the histogram and the luminance level, and based on the calculated evaluation value, dynamics of the image sensor Gain calculating means for calculating an adjustment gain for adjusting the range, and combining the first captured image and the second captured image based on the adjustment gain And forming means, characterized by comprising a.
[0009]
The imaging device captures an image of the subject with the first sensitivity and also with a second sensitivity lower than the first sensitivity. As a result, two captured images having different sensitivities between the first captured image captured with the first sensitivity and the second captured image captured with the second sensitivity are obtained.
[0010]
As described above, the imaging device capable of imaging with different sensitivities includes, for example, a plurality of first light receiving elements that receive light from the subject with the first sensitivity and output a signal corresponding to the received light amount, and the subject. And a plurality of second light receiving elements for receiving a signal corresponding to the received light amount. In this case, the second light receiving element may be provided so as to be positioned between the first light receiving elements, or may be provided on the first light receiving element provided with means for preventing mixing of received light amounts such as a channel stopper. May be.
[0011]
The brightness level detection means detects the brightness level of the subject based on at least one of the first captured image and the second captured image. The luminance level can be obtained from each color data of R, G, B of the captured image, for example, but may be detected by a dedicated sensor.
[0012]
The histogram calculation means calculates a histogram of the first captured image. The gain calculating means obtains an evaluation value representing the likelihood of high contrast based on highlight data on the highlight side in the histogram and the brightness level detected by the brightness level detecting means.
[0013]
Then, an adjustment gain for adjusting the dynamic range of the image sensor is calculated based on the obtained evaluation value.
[0014]
The synthesizing unit synthesizes the first captured image and the second captured image based on the adjustment gain.
[0015]
In this manner, a histogram is obtained from the first captured image, and an evaluation value representing the likelihood of high contrast is obtained based on the highlight data on the highlight side of the histogram and the luminance level detected by the luminance level detecting means. Therefore, it is possible to appropriately detect the high contrast likelihood of the shooting scene.
[0016]
In order to adjust the dynamic range based on the high contrast, the dynamic range is appropriately adjusted according to the shooting scene even when the first captured image and the second captured image having different sensitivities are combined. It is possible to adjust and perform gradation processing more appropriately.
[0017]
Specifically, when at least one of the highlight data and the luminance level is high, it is considered that the degree of high contrast becomes high. Therefore, as described in claim 2, the gain calculation means represents the high contrast likelihood. The adjustment gain is calculated by a membership function that acts so that the dynamic range of the synthesized image by the synthesizing means becomes wider as the evaluation value becomes higher.
[0018]
That is, the adjustment gain is determined so that the dynamic range becomes wider as the evaluation value representing the high contrast likelihood becomes higher, and the dynamic range becomes narrower as the evaluation value showing the high contrast likelihood becomes lower.
[0019]
As a result, even when a high-sensitivity and low-sensitivity image is combined when shooting a high-contrast scene, the high-sensitivity and low-sensitivity images are the same in any shooting scene as compared to when combining high-sensitivity images. The light part can be reproduced appropriately, and robust gradation control is possible.
[0020]
In addition, as described in claim 3, it is preferable that the processing by the histogram calculation unit and the gain calculation unit and the processing related to the second captured image are performed in parallel. As a result, the processing speed can be increased.
[0021]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings.
[0022]
FIG. 1 is a rear view of a digital camera according to the present invention, and FIG. 2 is a plan view of a mode dial provided on the upper surface of the camera.
[0023]
As shown in FIG. 2, the mode dial 1 is rotated so that the icons 1A to 1F on the dial are aligned with the mark M, so that the continuous shooting / bracketing mode 1A for shooting with multiple exposures, the aperture and the shutter speed are set. Manual shooting mode 1B that can be set, auto shooting mode 1C suitable for shooting various subjects, portrait mode 1D suitable for shooting a person, landscape mode 1E suitable for shooting a landscape, and night scene shooting It is possible to set a shooting mode such as night view mode 1F suitable for the case. In FIG. 2, the landscape mode 1E is set.
[0024]
In addition to these shooting modes, you can select a combination of aperture and shutter speed, P (program) mode, you can select an aperture, A (aperture priority) mode in which the shutter speed is automatically selected, and shutter speed It is possible to set a photographing mode such as an S (shutter speed priority) mode in which an aperture is automatically selected. In addition, for each shooting mode, auto mode that automatically fires the flash, red-eye reduction mode to reduce red-eye when a person is shot, flash forced flash mode that forces the flash to fire, flash light emission At the same time, it is possible to set a flash mode such as a slow sync mode for shooting a person and a night scene in a balanced manner by increasing the shutter speed.
[0025]
In the center of the mode dial 1, a shutter button 2 having a switch S1 that is turned on when half-pressed and a switch S2 that is turned on when fully pressed is provided.
[0026]
On the back of the digital camera, as shown in FIG. 1, a viewfinder eyepiece 3, a shift key 4, a display key 5, a shooting mode / playback mode switching lever 6, a cancel key 7, an execution key 8, and a multifunction cross key 9 And a liquid crystal monitor 52 are provided.
[0027]
FIG. 3 is a block diagram showing an internal configuration of the digital camera shown in FIG.
[0028]
The digital camera according to the present embodiment includes a photographic lens 10, a diaphragm 12 that adjusts the amount of light that passes through the photographic lens 10, and light from a subject that has passed through the photographic lens 10 and the diaphragm 12 with high sensitivity and low sensitivity. A CCD (Charge Coupled Device) 14 is provided that outputs R, G, and B color analog image signals indicating the subject image received by each light receiving element.
[0029]
Here, the structure of the CCD 14 according to the present embodiment will be described. As the CCD 14, a honeycomb CCD as shown in FIG. 4 can be adopted.
[0030]
As shown in FIG. 4, the CCD 14 has an image pickup unit assigned one by one for each color of one pixel and a predetermined arrangement pitch (horizontal arrangement pitch = Ph (μm), vertical arrangement pitch = Pv (μm)). And adjacent light receiving elements PD1 are arranged so as to bypass a plurality of light receiving elements PD1 that are two-dimensionally shifted in the vertical direction and the horizontal direction, and an opening AP formed in the front surface of the light receiving element PD1. In addition, a vertical transfer electrode VEL that takes out a signal (charge) from the light receiving element PD1 and transfers it in the vertical direction and a vertical transfer electrode VEL that is positioned at the lowest in the vertical direction are arranged on the lower side in the vertical direction, A horizontal transfer electrode HEL for transferring the transferred signal to the outside. In the example shown in the figure, the opening AP is formed in an octagonal honeycomb shape.
[0031]
Here, each of the vertical transfer electrode groups constituted by a plurality of vertical transfer electrodes VEL arranged in a straight line in the horizontal direction has one of the vertical transfer drive signals V1, V2,. Can be applied simultaneously. In the example shown in the figure, the vertical transfer drive signal V3 is applied to the first-stage vertical transfer electrode group, and the vertical transfer drive signal V4 is applied to the second-stage vertical transfer electrode group. The vertical transfer drive signal V5 for the transfer electrode group, the vertical transfer drive signal V6 for the fourth vertical transfer electrode group, and the vertical transfer drive signal V7 for the fifth vertical transfer electrode group are 6 The vertical transfer drive signal V8 for the vertical transfer electrode group at the stage, the vertical transfer drive signal V1 for the vertical transfer electrode group at the seventh stage, and the vertical transfer drive signal for the vertical transfer electrode group at the eighth stage. V2 can be applied to each.
[0032]
On the other hand, each light receiving element PD1 is configured to be electrically connected to one adjacent vertical transfer electrode VEL via a transfer gate TG. In the example shown in the figure, each light receiving element PD1 is configured to be connected to a vertical transfer electrode VEL adjacent to the lower right via a transfer gate TG.
[0033]
In the figure, the opening AP formed on the front surface of the light receiving element PD1 in which “R” is entered is covered with a color separation filter (color filter) that transmits red light, and “G” is entered. The opening AP formed on the front surface of the light receiving element PD1 is covered with a color separation filter that transmits green light, and the opening AP formed on the front surface of the light receiving element PD1 on which “B” is written is blue. It is covered with a color separation filter that transmits light. That is, the light receiving element PD1 in which “R” is written receives red light, the light receiving element PD1 in which “G” is written receives green light, and the light receiving element PD1 in which “B” is written receives blue light. Each analog signal corresponding to the amount of received light is output.
[0034]
The CCD 14 further includes a light receiving element PD2 that is less sensitive than the light receiving element PD1 described above. As shown in FIG. 4, the light receiving element PD2 is provided between the plurality of light receiving elements PD1. Similarly to the light receiving element PD1, the light receiving element PD2 is formed with an opening AP having a smaller area than the opening of the light receiving element PD1, and is electrically connected to one adjacent vertical transfer electrode VEL by the transfer gate TG. It is connected. Further, in the light receiving element PD2, any one of RGB color filters is mounted in the opening AP formed on the front surface thereof, similarly to the light receiving element PD1. Thus, since the light receiving area of the light receiving element PD2 is smaller than the light receiving area of the light receiving element PD1, an RGB signal having a lower sensitivity than the light receiving element PD1 can be obtained.
[0035]
The electrode to which the transfer gate TG of the light receiving element PD2 is connected is provided different from the electrode to which the transfer gate TG of the adjacent light receiving element PD1 is connected.
[0036]
The subject image formed on the light receiving surface of the CCD 14 via the photographing lens 10 and the diaphragm 12 is converted into signal charges of an amount corresponding to the amount of incident light by each of the light receiving elements PD1 and PD2. Each of the high-sensitivity or low-sensitivity signal charges accumulated in this way is read to the shift register by a read gate pulse applied from the CCD drive circuit 16, and sequentially as a voltage signal corresponding to the signal charge by the register transfer pulse. Read out. The CCD 14 has a so-called electronic shutter function that can sweep out the accumulated signal charge by a shutter gate pulse and thereby control the charge accumulation time (shutter speed).
[0037]
Although details will be described later, in the present embodiment, the charge of the light receiving element PD2 is read after the charge of the light receiving element PD1 is read first. That is, after reading out the high-sensitivity signal charge, the low-sensitivity signal charge is read out.
[0038]
The high-sensitivity or low-sensitivity voltage signal sequentially read from the CCD 14 is applied to a correlated double sampling circuit (CDS circuit) 18 where the R, G, B signals for each pixel are sampled and held, and A / It is added to the D converter 20. The A / D converter 20 converts a high-sensitivity R, G, B signal and a low-sensitivity R, G, B signal sequentially applied from the CDS circuit 18 into, for example, a 12-bit high-sensitivity digital R, G, B signal. (First captured image) and low-sensitivity digital R, G, B signals (second captured image) are converted and output. The CCD drive circuit 16, the CDS circuit 18, and the A / D converter 20 are driven in synchronization with a timing signal applied from a timing generation circuit (TG) 22.
[0039]
The high-sensitivity or low-sensitivity R, G, B signals output from the A / D converter 20 are temporarily stored in the memory 24, and then the R, G, B signals stored in the memory 24 are digital signal processing. Applied to circuit 26. The digital signal processing circuit 26 includes a synthesis processing unit 30, a synchronization circuit 32, a YC signal creation circuit 34, and a memory 36.
[0040]
As shown in FIG. 5, the synthesis processing unit 30 includes an OB processing unit 70A, 70B, LMTX 72A, 72B, a WB adjustment unit 74A, 74B, a gamma correction unit 76A, 76B, a histogram calculation unit 78, a gain calculation unit 80, and a multiplier. 82A and 82B, an adder 84, and a limiter 86. R, G, B data of a photographed image taken with high sensitivity is input to the OB processing unit 70A, and R, G, B data of a photographed image photographed with low sensitivity is input to the OB processing unit 70B. Is done.
[0041]
The OB processing units 70A and 70B perform processing for correcting the black level. That is, offset adjustment is performed by subtracting the data of the light shielding area of the CCD 14 from the original image data, and the black level is corrected.
[0042]
The LMTXs 72A and 72B are composed of, for example, a 3 × 3 matrix circuit, and perform color correction by adjusting hue or saturation.
[0043]
The WB adjusting units 74A and 74B adjust the white balance by multiplying the R, G, and B data by gain values Rg, Gg, and Bg for white balance correction specified by the CPU 38, respectively.
[0044]
The gamma correction units 76A and 76B change the input / output characteristics so that the R, G, and B data subjected to white balance adjustment have desired gamma characteristics.
[0045]
Although details will be described later, the histogram calculation unit 78 calculates a histogram of G data, for example.
[0046]
The gain calculation unit 80 is based on highlight data on the highlight side determined in advance in the histogram calculated by the histogram calculation unit 78 and a shooting EV value representing the luminance level of the shot image calculated by the CPU 38 to be described later. Thus, an adjustment gain for adjusting the dynamic range of the CCD 14 is calculated. Then, h_gain for multiplying the high-sensitivity R, G, B data based on the adjustment gain and l_gain for multiplying the low-sensitivity R, G, B data are calculated. The calculation of h_gain and l_gain will be described later.
[0047]
The multiplier 82A multiplies g_corrected high-sensitivity R, G, B data by h_gain and outputs the result to the adder 84. The multiplier 82B multiplies the low-sensitivity R, G, B data subjected to gamma correction by l_gain and outputs the result to the adder 84.
[0048]
The adder 84 adds the low-sensitivity R, G, B data multiplied by l_gain to the high-sensitivity R, G, B data multiplied by h_gain, synthesizes both, and outputs them to the limiter 86. The limiter 86 keeps R, G, B data within a range that can be expressed by a predetermined number of bits (for example, 8 bits).
[0049]
The synchronization circuit 32 converts the dot-sequential R, G, B signals read from the memory 24 into simultaneous equations and outputs the converted signals to the YC signal generation circuit 34.
[0050]
The YC signal creation circuit 34 creates a luminance signal Y and chroma signals Cr and Cb from the combined R, G, and B signals. These luminance signal Y and chroma signals Cr and Cb (YC signal) are stored in the memory 36.
[0051]
Here, by reading the YC signal in the memory 36 and outputting it to the liquid crystal monitor 52, a moving image or a still image can be displayed on the liquid crystal monitor 52. The YC signal after photographing is compressed into a predetermined format by the compression / decompression circuit 54 and then recorded on a recording medium such as a memory card by the recording unit 56. Further, in the reproduction mode, the image data recorded on the memory card or the like is decompressed by the compression / decompression circuit 54 and then output to the liquid crystal monitor 52 so that the reproduced image is displayed on the liquid crystal monitor 52. .
[0052]
The CCD 14 corresponds to the image sensor of the present invention, the histogram calculator 78 corresponds to the histogram calculator of the present invention, the gain calculator 80 corresponds to the gain calculator of the present invention, and the adder 86 corresponds to the present invention. It corresponds to the synthesis means.
[0053]
The CPU 38 controls each circuit based on inputs from the camera operation unit 40 including the mode dial 1 and the shutter button 2 shown in FIG. 1, and performs control such as autofocus, automatic exposure control, and auto white balance. Do. This autofocus control is, for example, contrast AF that moves the taking lens 10 so that the high-frequency component of the G signal is maximized, and the drive unit 42 so that the high-frequency component of the G signal is maximized when the shutter button 2 is half-pressed. Then, the photographing lens 10 is moved to the in-focus position.
[0054]
In the automatic exposure control, as shown in FIG. 6, the R, G, B signals are fetched up to four times at predetermined exposures (1) to (4), and these R, G, B signals are integrated. Based on this, the subject brightness (shooting EV value) is obtained. This is executed when the shutter button 2 is half-pressed. Note that the process for obtaining the photographing EV value corresponds to the luminance level detecting means of the present invention.
[0055]
The CPU 38 is connected to a memory 39 for storing various data such as a photographing EV value, a strobe 60 for emitting strobe light, and a drive unit 64 for driving a strobe light control sensor 62. Control these.
[0056]
The strobe 60 emits light when, for example, the photographing EV value is equal to or smaller than a predetermined value, that is, in a photographing situation where the subject is dark and sufficient contrast cannot be obtained, and the subject is irradiated with strobe light.
[0057]
The strobe light control sensor 62 detects strobe light emitted from the strobe 60 and outputs a detection voltage corresponding to the amount of light to the drive unit 64. The drive unit 64 uses the strobe 60 when the detection voltage output from the strobe dimming sensor 62 reaches a predetermined value, that is, when the subject is irradiated with strobe light to such an extent that sufficient contrast is obtained. Stop flash light emission. Thereby, the light intensity is adjusted to an appropriate light amount.
[0058]
Next, details of the measurement of the photographing EV value will be described.
[0059]
As shown in FIG. 7, one screen is divided into a plurality of areas (8 × 8), and luminance signals obtained from, for example, high-sensitivity R, G, and B signals are integrated for each divided area, and the integrated value is obtained. Based on this, the EV value (EVi) of each divided area is obtained. Subsequently, as shown in FIG. 7, the EV value of each divided area is weighted corresponding to the shooting mode, and the EV ′ value of the entire screen is calculated by the following equation.
[0060]
[Expression 1]

[0061]
However, i is a subscript indicating each of the divided areas, and can take a value of 0 to 63 in the above case. N indicates the number of divided areas. In the above case, 8 × 8 = 64. ΔEViso is an EV value correction amount based on a predetermined sensitivity (for example, ISO 200), and is a value for adjusting EV ′ to be constant even when the sensitivity is changed. Wi is a weighting factor for each divided area, and for example, as shown in FIG.
[0062]
The EV ′ calculated as described above is further subjected to exposure correction ΔEV corresponding to the shooting mode as shown in the following equation to obtain the shooting EV value. The obtained shooting EV value is stored in the memory 39.
[0063]
EV = EV′−ΔEV (2)
Note that ΔEV is, for example, ΔEV = 0 in the portrait mode, and ΔEV = 0.3 in the landscape mode and the night view mode.
[0064]
The aperture value and shutter speed at the time of shooting are finally determined based on the shooting EV value obtained as described above.
[0065]
Then, when the shutter button is fully pressed, the diaphragm 12 is driven through the diaphragm driving unit 44 so that the determined diaphragm value is obtained, and the charge accumulation time is controlled by the electronic shutter so that the determined shutter speed is achieved. .
[0066]
Next, as an operation of the present embodiment, a processing routine executed by the digital camera 11 will be described with reference to a flowchart shown in FIG. Note that the processing routine shown in FIG. 8 is executed when the shutter button 2 is fully pressed.
[0067]
First, when the shutter button 2 is half-pressed, the CPU 38 instructs the CCD 14 through the TG 22 and the CCD drive circuit 16 to take in a highly sensitive image signal. As a result, high-sensitivity image data is once taken from the CCD 14 into the memory 24, and the above-described autofocus control and automatic exposure control are performed by the CPU 38. Thereby, the photographing EV value, the shutter speed, and the aperture value are determined, and the drive unit 42 and the drive unit 44 are controlled.
[0068]
When the shutter button 2 is fully pressed, in step 100, the CPU 38 instructs the CCD 14 via the TG 22 and the CCD drive circuit 16 to capture a high-sensitivity image signal. As a result, high-sensitivity image data is temporarily captured from the CCD 14 into the memory 24.
[0069]
Next, in step 102, gain values Rg, Gg, and Bg for adjusting the white balance are calculated.
[0070]
Next, calculation of a gain value for white balance adjustment will be described with reference to a flowchart shown in FIG.
[0071]
In the case of shooting without strobe light emission, the white balance control shown in FIG. 9 is executed when the shooting mode is other than the night scene mode, and in other shooting modes, the gain value Rg is obtained by normal white balance control. , Gg, Bg are calculated.
[0072]
Further, in the case of shooting with strobe light emission, white balance control shown in FIG. 9 is executed in the case of a predetermined shooting mode and strobe mode and when it is determined that strobe light emission is performed.
[0073]
The following table shows combinations of shooting modes and flash modes in which the white balance control shown in FIG. 9 is executed in the case of shooting with strobe light emission. For example, when the shooting mode and the flash mode are both the auto mode and the EV value is equal to or less than a predetermined threshold, the following white balance control is executed. In other cases, for example, the shooting mode is the landscape mode and the flash mode is the forced flash mode. In this case, normal white balance control is performed.
[0074]
[Table 1]

[0075]
First, it is determined whether or not strobe light is emitted from the EV value, the shooting mode, and the strobe mode. When the shooting mode is the night scene mode and it is determined that the strobe light is not emitted, daylight light is emitted by a well-known method. White balance correction values Rg, Gg, and Bg for performing white balance control suitable for the case where the strobe is not emitted as a light source are set and output to the WB adjustment units 74A and 74B.
[0076]
If it is determined that the flash light is emitted, it is determined whether or not the shooting mode and the flash mode are combinations indicated by the circles in Table 1 above. If it is determined that the flash light is emitted and the shooting mode and the flash mode are determined not to be a combination indicated by a circle in Table 1, the white balance control during normal flash light emission is performed by a well-known method. White balance correction values Rg, Gg, and Bg are set and output to the WB adjustment units 74A and 74B.
[0077]
When it is determined that the shooting mode is other than the night scene mode and the flash is not emitted, or when the shooting mode and the flash mode are combinations indicated by the circles in Table 1, the CPU 38 presses the shutter button halfway. The photographing EV value obtained at times is read from the memory 39 (step 200).
[0078]
R, G, and B signals of a high-sensitivity image captured when the shutter button is fully pressed are temporarily stored in the memory 24. This image is divided into a plurality of areas (for example, 8 × 8), and each divided area is divided. For each of the R, G, and B signals, an integrated value for each color is obtained by the integrating circuit 48 (step 202).
[0079]
The integrated values Rt, Gt, Bt of the R, G, B signals are output to multipliers (multiplier means) 50R, 50G, 50B provided between the integrating circuit 48 and the CPU 38, and the multipliers 50R, 50G, 50B are output. Is multiplied by a predetermined reference WB (white balance) gain value (reference control value) Rc, Gc, Bc (step 204). The reference WB gain values Rc, Gc, Bc are stored in the memory 39 in advance.
[0080]
The integrated values Rt ′, Gt ′, Bt ′ of the R, G, B signals multiplied by the reference WB gain values Rc, Gc, Bc are input to the CPU 38.
[0081]
The reference WB gain values Rc, Gc, and Bc are used to adjust the white balance so that the white balance is appropriate when the light source is daylight light, such as when the weather is sunny, in the case of shooting without strobe light emission. Gain values Rd, Gd, and Bd are set. Also, when shooting with strobe light, a gain value for adjusting the white balance to be appropriate when the strobe light is dominant as a light source, such as when the strobe light reaches the subject sufficiently Rst, Gst, and Bst are set.
[0082]
Then, the CPU 38 has a ratio Rt ′ / Gt ′ between the integrated value Rt ′ of the input R signal and the integrated value Gt ′ of the G signal, and a ratio Bt ′ / Gt of the integrated value of the B signal and the integrated value of the G signal. Gt ′ is obtained (step 206).
[0083]
Rt ′ / Gt ′ and Bt ′ / Gt ′ obtained for each divided area as described above are the detection frames in which the divided area is detected among the detection frames represented on the graph shown in FIG. Used to determine whether to enter. Note that each detection frame in FIG. 10 defines a range of color distribution such as an environmental light source.
[0084]
As shown in FIG. 10, the detection frame includes a blue sky detection frame, a shade detection frame, a daylight color (fluorescent light) detection frame, a daylight white (fluorescent light) detection frame, a white (fluorescent light) detection frame, a daylight detection frame, There are a light bulb color fluorescent light detection frame and a light bulb (tungsten light bulb) detection frame. The daylight detection frame is a detection frame for shooting without flash emission, and becomes a strobe detection frame for shooting with flash.
[0085]
Then, the CPU 38 represents Rt ′ / Gt ′ = X ′, Bt ′ / Gt ′ = Y ′, and Rt ′ / Gt ′, Bt ′ / Gt ′ obtained for each divided area (X _i It is determined in which detection frame of these detection frames “′, Y _i ′” falls, and the number thereof is counted (step 208). At this time, in the case of shooting without strobe lighting, the number in the shade detection frame is counted only for the EV value of the divided area that is equal to or smaller than the predetermined value, and the number in the blue sky detection frame is equal to or greater than the predetermined value. Count only about things.
[0086]
Next, an average value (Xa _j 'Ya _j ') of (X _i ', Y _i ') existing in each detection frame is obtained (step 210). Here, j is a subscript indicating each detection frame. The center of gravity value may be used instead of the average value.
[0087]
Next, an evaluation value Fc () representing the light source-likeness (shade-likeness, etc.) relating to the color is calculated by the membership function provided corresponding to each detection frame (step 212). The membership function is a function that outputs an evaluation value Fc () representing the light source likeness with respect to the color using the number in the detection frame as a variable. That is, if the evaluation value is high, the possibility of a light source (environment light source) corresponding to the detection frame increases.
[0088]
As shown in FIG. 11, Fc (CLD) is a value of a membership function that represents the likelihood of being shaded with the number of divided areas that fall within the shade detection frame as a variable.
[0089]
As shown in FIG. 12, Fc (SKY) is a membership function value representing the likelihood of blue sky with the number of divided areas that fall within the blue sky detection frame as a variable.
[0090]
Fc (EXD), Fc (EXN), Fc (W), Fc (EXL), and Fc (TNG) are membership functions similar to Fc (CLD) shown in FIG. Here, Fc (EXD) is a membership function representing the likelihood of a fluorescent light of daylight color with the number of divided areas that fall within the daylight color detection frame as a variable, and Fc (EXN) is a division function that falls within the daylight white detection frame. This is a membership function value representing the likelihood of a daylight fluorescent lamp with the number of areas as a variable, and Fc (W) represents the likelihood of a white fluorescent lamp with the number of divided areas within the white detection frame as a variable. Fc (EXL) is a membership function that represents the likelihood of a fluorescent light of a light bulb color with the number of divided areas within the light bulb color fluorescent light detection frame as a variable, and Fc (TNG) is a light bulb This is a membership function value representing the light bulb likeness with the number of divided areas falling within the detection frame as a variable.
[0091]
Fc (DAY) is a value of a membership function that represents the clearness with the number of divided areas that fall within the daylight detection frame as a variable. This membership function is a membership function that represents the shadedness shown in FIG. It is the same.
[0092]
Fc (TYP) is a strobe characteristic with the number of divided areas falling within the strobe detection frame as a variable, that is, a membership function value indicating whether or not strobe light is dominant. This is the same as the membership function representing the shading shown in FIG.
[0093]
Next, an evaluation value Fy () representing the light source likelihood (indoorness, outdoorness) relating to the luminance is calculated by a membership function provided corresponding to each detection frame (step 214). The membership function is a function that outputs an evaluation value representing the light source-likeness related to the luminance using the EV value acquired in step 200 as a variable.
[0094]
As shown in FIG. 13, Fy (outdoor likelihood DAY) is a value of a membership function that represents the outdoorness of the daylight with the EV value as a variable.
[0095]
As shown in FIG. 14, Fy (outdoor likelihood CLD) is a value of a membership function that represents the outdoor appearance of shade with the EV value as a variable.
[0096]
As shown in FIG. 15, Fy (indoorness EXD), Fy (indoorness EXN), and Fy (indoorness TNG) are respectively the indoorness of daylight fluorescent lamps with the EV value as a variable or the whiteness of daylight fluorescent lamps. This is a membership function value representing the indoorness or the indoorness of a tungsten light bulb. Fy (indoorness W) is a value of a membership function representing the indoorness of a white fluorescent lamp with the EV value as a variable, and is the same as Fy (indoorness EXN). Also, Fy (indoorness EXL) is a membership function value representing the indoorness of a fluorescent light bulb with the EV value as a variable, and is the same as Fy (indoorness TNG).
[0097]
Next, a comprehensive light source evaluation value H _j is obtained for each detection frame from the evaluation value indicating the light source likelihood relating to the color and the evaluation value indicating the evaluation value relating to the luminance (step 216). In the case where the flash is not emitted and the shooting mode is the landscape mode, only H _{1 to} H ₂ and H ₇ are performed. This is because, in the landscape mode, the image is usually taken outdoors, but it is not considered that the light source is a tungsten bulb or the like in the outdoors.
[0098]
Evaluation values H _{1 to} H ₇ are obtained by the following equations.
[0099]
H ₁ (Evaluation value of shade) = Fc (CLD) × Fy (likeness of outdoors) × Fc (SKY) (3)
H ₂ (Evaluation value of the likelihood of daylight fluorescent lamp) = Fc (EXD) × Fy (Indoorness EXD) (4)
H ₃ (Evaluation value of the likelihood of daylight fluorescent light) = Fc (EXN) × Fy (Indoorness EXN) (5)
H ₄ (Evaluation value of white fluorescent lamp-likeness) = Fc (W) × Fy (indoor-likeness W) (6)
H ₅ (Evaluation value of the likelihood of a light bulb color fluorescent lamp) = Fc (EXL) × Fy (Individuality EXL) (7)
H ₆ (Evaluation value of light bulb-likeness) = Fc (TNG) × Fy (Indoor-likeness TNG) (8)
H ₇ (Evaluation value of sunnyness) = Fc (DAY) × Fy (Outdoor likeness DAY) (9)
In the case of flash photography is, H ₇ becomes the following equation.
[0100]
H ₇ (Evaluation value of strobe) = Fc (TYP) × Fy (Outdoor likelihood DAY) (10)
Here, since 0 ≦ Fc () and Fy () ≦ 1, 0 ≦ H _j ≦ 1. As shown in FIG. 12, Fc (SKY) takes a value that acts in the direction of lowering the evaluation value of shading as the number of divided areas in the blue sky detection frame increases.
[0101]
Further, as shown in FIG. 15, Fy (indoorness) takes a value that acts in the direction of lowering the evaluation value of the detection frames as the EV value is higher. As described above, the evaluation value H _j of the light source quality is determined in consideration of not only the color distribution but also the luminance, so that the light source can be appropriately specified. For example, when shooting a subject of the same color in a room where the fluorescent lamp is lit, and when the light source is specified only by the color distribution, the light source is incorrectly specified and a color failure occurs. There is.
[0102]
However, in this embodiment, when the luminance is low, that is, when the image is taken indoors, Fy (appearance of the room) is high, and when the luminance is high, that is, when the image is taken outdoor. , Fy (likeness in the room) is reduced, so that the light source can be specified accurately and the occurrence of color feria can be prevented.
[0103]
Then, when the evaluation value of the light source likelihood corresponding to each detection frame is calculated as described above, it is determined whether or not the evaluation value is equal to or more than a predetermined threshold value (for example, 0.4 or more) among these evaluation values ( Step 218). If there is an evaluation value that is equal to or greater than a predetermined threshold value, an evaluation value that is equal to or greater than the predetermined threshold value is selected as an environmental light source candidate.
[0104]
Next, for each selected environment light source, a control value (gain for performing white balance control so that the color of the environment light source or the mixed light source in which the strobe light and the environment light source are mixed is white according to the following formula: Value) G _R , G _B , _GG are calculated (step 220).
[0105]
G _R = {Σ (gr _j × H _j ) / ΣH _j } × {Σ (G _j × H _j ) / ΣH _j } (11)
G _B = {Σ (gb _j × H _j ) / ΣH _j } × {Σ (G _j × H _j ) / ΣH _j } (12)
G _G = Σ (G _j × H _j ) / ΣH _j (13)
Here, the (11) to (13), when the evaluation value is calculated for more than a predetermined threshold value, for example, evaluation values higher than a predetermined threshold is _H 2 to H _{5 is} H 2 to H ₅ Gr _j and gb _j are calculated for each of the light sources, and calculations in each Σ are performed using the calculated gr _j , gb _j and H _{2 to} H ₅ . Further, gr _j and gb _j are gains necessary for setting the color of the light source in each detection frame to a predetermined color, and are represented by the following equations.
[0106]
gr _j = Xc / Xa _j '× Tx _j (14)
_{gb j = Yc / Ya j '} × Ty j ... (15)
Here, Xc = Rc / Gc and Yc = Bc / Gc. Tx _j and Ty _j are the average values of X _i and Y _i obtained for the integrated values of R, G, and B of the original image data not multiplied by the reference WB gain value, Xa _j (= Xa _j ' / Xc), Ya _j (= Ya _j ′ / Yc), Xa _j , Ya _j target values, corresponding to a predetermined color. That is, gr _j and gb _j are the light sources (environment light source or strobe light) indicated by Xa _j and Ya _j when Xa _j and Ya _j are obtained for the integrated values of R, G, and B of the original image data. Is a gain necessary to set the color of the mixed light source of the light source and the environmental light source to a predetermined color. Tx _j and Ty _j are 1, for example.
[0107]
Accordingly, the first terms of the above equations (11) and (12) represent the weighted average values of gr _j and gb _j with the evaluation value H _j as the weight. The second term is a term for correcting the brightness, and G _j in the case of shooting that does not emit strobe light is expressed by the following equation.
[0108]
G _j = g _j × (exposure correction gain / 128) (16)
Here, g _j is a gain of G (green) necessary for performing white balance control suitable when the light source is sunny. The exposure compensation gain is as shown in the table below.
[0109]
[Table 2]

[0110]
The reason for correcting the brightness in this manner is that if the strobe light does not emit light, only the surrounding light source is used, and the brightness may be insufficient.
[0111]
Further, in the case of shooting that emits strobe light, G _j is Gst, which is a fixed value, and thus the brightness is not corrected.
[0112]
In the case where the evaluation value _{H 7} daylight detection frame (or flash detection frame) is selected as being equal to or greater than a predetermined threshold _value, gr 7 = Xc, and _gb 7 = Yc.
[0113]
In this way, the control value G _{RR G} B to the color of the environmental light or mixed light in a predetermined _color, the G _G is calculated, these control values, a value between the reference WB gain value And it adjusts according to an environmental light source so that it may become a value suitable for an environmental light source or a mixed light source (step 222). The adjusted control values are expressed as white balance correction values Rg, Gg, and Bg, and are obtained by the following equations.
[0114]
_{Rg = (G R -Rd) ×} H max × L j + Rd ... (17)
_{Gg = (G G -Gd) ×} H max × L j + Gd ... (18)
_{Bg = (G B -Bd) ×} H max × L j + Bd ... (19)
Here, H _max is the maximum value of H _j , and the light source corresponding to H _max can be specified as the light source at the time of shooting. For example, if the H _max is a H _2, the light source may be determined to be a fluorescent lamp daylight. L _j is a lowward correction coefficient, which is a predetermined value corresponding to each light source, and is a value determined by experiments or the like so that white balance is appropriate in shooting under each light source. is there. The lowward correction coefficient is stored in the memory 39 in advance. Note that L _j may be a fixed value regardless of the light source.
[0115]
When 0 ≦ L _j ≦ 1 and L _j = 0, Rg = Rc, Gg = Gc, and Bg = Bc, and white balance control is performed with the reference WB gain value. Further, when L _j = 1 and H _max = 1, Rg = G _R , Gg = G _G , and Bg = G _G. Thus, the white balance correction values Rg, Gg, and Bg are values between the control values G _R , G _B , and G _G for setting the color of the environmental light source to a predetermined color from the reference WB gain value, and H _This is a value that enables white balance control suitable for the environmental light source specified by _max .
[0116]
The white balance correction values Rg, Gg, and Bg calculated in this way are output to the WB adjustment units 74A and 74B (step 224), and the R, G, and B signals of the original high-sensitivity and low-sensitivity image data are output. The white balance is adjusted. Assuming that the corrected signals are R ′, G ′, and B ′, R ′, G ′, and B ′ are expressed by the following equations.
[0117]
R ′ = Rg × R (20)
G ′ = Gg × G (21)
B ′ = Bg × B (22)
On the other hand, if there is no evaluation value _Hj equal to or greater than the predetermined threshold, the white balance correction values Rg, Gg, and Bg are set to Rc, Gc, and Bc, respectively (step 226), and the WB adjustment units 74A and 74B are performed. Output (step 224).
[0118]
Thus, in this embodiment, since the environmental light source is determined in consideration of not only the color of the environmental light source but also the EV value, the environmental light source can be accurately determined. In addition, since the color of the environmental light source is obtained from the image data of the captured image in the actual photographing, the environmental light source can be accurately determined without complicating the imaging sequence.
[0119]
The white balance correction values Rg, Gg, and Bg are values between control values G _R , G _B , and G _G for setting the color of the environmental light source to a predetermined color from the reference WB gain value, and at H _max Since the value is such that white balance control suitable for the specified environmental light source can be performed, the white balance control can be appropriately performed for each environmental light source, and the occurrence of color failure can be suppressed.
[0120]
In step 104, black correction processing is performed on the highly sensitive image data by the OB processing unit 70A.
[0121]
In step 106, the LMTX 72A performs color correction processing using a 3 × 3 matrix on the highly sensitive image data.
[0122]
In step 108, the white balance adjustment is performed by the white balance correction gain values Rg, Gg, and Bg calculated in step 102 by the WB adjusting unit 74 and multiplied by the highly sensitive R, G, and B data, respectively. The G data may be multiplied by a predetermined value (for example, 1.375) larger than a predetermined one. For example, when the light source is a light source with a lot of redness such as a light bulb, there is a case where it is desired to lower the gain of R than G. In such a case, the gain of G is set to a predetermined value larger than 1. This is because the gain of R can be relatively lowered.
[0123]
In step 110, gamma correction processing is performed on the highly sensitive image data.
[0124]
In parallel with the processing in steps 102 to 110 described above, low-sensitivity image signal processing is performed in step 112. That is, the CPU 38 instructs the CCD 14 via the TG 22 and the CCD drive circuit 16 to capture a low-sensitivity image signal. As a result, low-sensitivity image data is taken into the memory 24 from the CCD 14.
[0125]
Similarly to the high-sensitivity image data, the black correction process, the color correction process, the white balance adjustment, and the low-sensitivity image data are respectively performed by the OB processing unit 70B, the LMTX 72B, the WB adjustment unit 74B, and the gamma correction unit 76B. Gamma correction is performed.
[0126]
As described above, since the processing of the low-sensitivity image data is performed in parallel with the processing for calculating the gain value for white balance adjustment based on the high-sensitivity image data, the processing speed can be increased.
[0127]
In parallel with the white balance adjustment processing for the high-sensitivity image data in step 108 and the gamma correction processing in step 110, the histogram calculation processing of the image data is performed by the histogram calculation unit 78 in step 114. A gain calculation process for calculating a gain for adjusting the dynamic range based on the calculated histogram is performed.
[0128]
The histogram calculation unit 78 calculates, for example, a histogram of G data out of high sensitivity image data. First, 12-bit G data is shifted by 6 bits, for example, and a histogram is obtained as 6-bit equivalent data. An example of such a histogram is shown in FIG.
[0129]
Then, the data on the predetermined highlight side, that is, the position where the cumulative frequency of G data is reduced by 100% (for example, 0.1%) is set as the maximum value on the highlight side ('52' in FIG. 16). A value obtained by shifting 6 by 6 bits ( ²⁶ times) is used as highlight data. This highlight data is used as information indicating whether or not highlight side data is saturated when a gain for white balance adjustment is applied.
[0130]
The gain calculation unit 80 calculates an adjustment gain for adjusting the dynamic range of the CCD 14 based on the highlight data and the shooting EV value. Then, h_gain for multiplying the high-sensitivity R, G, B data based on the adjustment gain and l_gain for multiplying the low-sensitivity R, G, B data are calculated.
[0131]
First, in the case of shooting without strobe light emission, an evaluation value 1 corresponding to highlight data is obtained by a membership function representing the relationship between highlight data and evaluation value 1 as shown in FIG. An evaluation value 2 corresponding to the photographing EV value is obtained by a membership function representing a correspondence relationship between the photographing EV value and the evaluation value 2 as shown in FIG.
[0132]
As shown in FIG. 17A, the membership function for calculating the evaluation value 1 is a predetermined value (FIG. 17) at which highlight side data is saturated when a gain for white balance adjustment (eg, 1.375) is applied. Up to 3000 in 17 (A), the evaluation value 1 gradually increases as the highlight data increases, and becomes a constant value “1” when the highlight data is greater than or equal to a predetermined value.
[0133]
In addition, as shown in FIG. 17B, the membership function for calculating the evaluation value 2 is the shooting EV value up to the EV value when the shooting EV value is fine (13.5 in FIG. 17B). As the value increases, the evaluation value 2 also gradually increases, and when the shooting EV value is equal to or higher than the EV value when the sky is clear, the function is a constant value “1”.
[0134]
Then, the evaluation value 1 is multiplied by the evaluation value 2. A value obtained by multiplying the evaluation value 1 by the evaluation value 2 is an evaluation value representing the degree of high contrast of the photographic scene, that is, the high contrast likeness of the photographic scene. It does not look like a contrast scene, but looks like a low contrast scene. In this way, the maximum value on the highlight side is obtained using the histogram, and the high contrast scene can be detected with high accuracy based on this and the subject luminance.
[0135]
Next, the total gain p (adjustment gain) for adjusting the dynamic range is obtained by a membership function representing the relationship between the total gain p and the evaluation value 1 × evaluation value 2 as shown in FIG.
[0136]
As shown in FIG. 17C, the membership function for calculating the total gain p is a function such that the total gain p gradually decreases as the evaluation value 1 × evaluation value 2 increases. The total gain p is a value less than 1, for example, a value in the range of 0.8 to 0.9 as shown in FIG.
[0137]
In the case of shooting with strobe light emission, it is considered that the relationship between the brightness of the subject and the high contrast scene is weak, so the evaluation value 2 based on the shooting EV value is not calculated, and the membership shown in FIG. The evaluation value 1 based on the highlight data is obtained by the function, and the total gain p based on the evaluation value 1 is obtained by the membership function shown in FIG.
[0138]
Then, h_gain for multiplying the high-sensitivity image data and l_gain for multiplying the low-sensitivity image data are calculated from the synthesis formula of the high-sensitivity image data and the low-sensitivity image data using the total gain p.
[0139]
The synthesis formula is expressed by the following formula, where synthesis data is data, high sensitivity data is high, and low sensitivity data is low.
[0140]
data = [high + Min (high / th, 1) × low] × Max (1−S × high / th, p) (23)
Here, th is a threshold value, and S is represented by S = 1− {Sh / (Sh + Sl)} when the saturation ratio between the high-sensitivity image data and the low-sensitivity image data is Sh: Sl. Value. For example, when the saturation ratio Sh: Sl is 4: 1, S = 0.2.
[0141]
From the equation (23), the synthesized data data is obtained by adding low sensitivity image data obtained by multiplying the high sensitivity image data high by the weight Min (high / th, 1) to the weight Max (−0.2 ×). high / th + 1, p) is multiplied.
[0142]
As shown in FIG. 19, the weight Min (high / th, 1) outputs a value that increases with an inclination (high / th) as the high increases in a section where the high-sensitivity image data high is smaller than th. , A function that outputs '1' when high is greater than or equal to th.
[0143]
Further, as shown in FIG. 20, the weight Max (1−S × high / th + 1, p) has a slope (1−S × high) as the height increases in the section where the high-sensitivity image data high is smaller than th. / Th) is a function that outputs a smaller value and outputs S when high is greater than or equal to th. FIG. 20 shows the case where S = 0.2 and p = 0.8.
[0144]
FIG. 21 shows high-sensitivity image data high, low-sensitivity image data low, and luminance-gradation values (gradation values are converted into 8-bit values) in the combined image data data up to the first term of the above equation (23). ) Characteristics. As shown in FIG. 21, the first term of the above equation (23) is a low-sensitivity image obtained by multiplying the high-sensitivity image data high by the characteristic weight Min (high / th, 1) as shown in FIG. The data low value is added.
[0145]
Therefore, in the range where the luminance is E1 (the range where the high-sensitivity image data high is equal to or less than the threshold th), the low-sensitivity image data low multiplied by the weight of high / th is added to the high-sensitivity image data high. In the range where the luminance is E2, the high-sensitivity image data high and the low-sensitivity image data low are added on a one-to-one basis.
[0146]
FIG. 22 shows the luminance-gradation value characteristics of the high-sensitivity image data high, the low-sensitivity image data low, and the composite image data data of the above equation (23). The composite image data data is a value obtained by adding low-sensitivity image data low obtained by multiplying high-sensitivity image data high by characteristic weight Min (high / th, 1) as shown in FIG. 19 to FIG. Is a value multiplied by the characteristic weight Max (1−S × high / th + 1, p).
[0147]
Therefore, in the range where the luminance is E1, the composite data data is a value obtained by multiplying the value of the first term of the above equation (23) by the weight of (1−S × high / th), and in the range where the luminance is E2. The value of the first term of the above equation (23) is multiplied by the total gain p.
[0148]
Here, since the total gain p is set in the range of 0.8 to 0.9, as shown in FIG. 22, the line representing the characteristics of the composite data data represents the characteristics of the high-sensitivity image data high. As it goes down below the line as a whole, the way of ascending becomes gradual. That is, the dynamic range of the composite data “data” is wider than that of the high-sensitivity image data “high”.
[0149]
The dynamic range becomes wider as the total gain p becomes smaller, that is, as the high-contrast scene likelihood becomes higher, and becomes narrower as the total gain p becomes larger, that is, as the high-contrast scene likelihood becomes lower.
[0150]
This widens the dynamic range when the degree of high contrast is high, so that when you shoot a high-contrast scene, you can properly reproduce the highlight area even if you combine high-sensitivity and low-sensitivity image data. In addition, robust gradation control is possible.
[0151]
By the way, the above equation (23) can be modified as follows.
[0152]
data = (high + wl × low) × wt
= Wt x high + (wl x wt) x low (24)
Here, wl = Min (high / th, 1) and wt = Max (1−S × high / th + 1, p).
[0153]
From the above equation (24), h_gain = wt and l_gain = wl · wt. Accordingly, h_gain and l_gain are determined by obtaining wl and wt, that is, Min (high / th, 1), Max (1-S × high / th + 1, p). The combined data is obtained by adding high-sensitivity image data weighted with h_gain and low-sensitivity image data weighted with l_gain.
[0154]
In step 112, the high-sensitivity R, G, B data subjected to gamma correction by the multiplier 82A is multiplied by h_gain calculated in step 116 and output to the adder 84, and gamma correction is performed by the multiplier 82B. The low-sensitivity R, G, B data is multiplied by l_gain calculated in step 116 and output to the adder 84.
[0155]
In step 120, the adder 84 adds the low-sensitivity R, G, B data multiplied by l_gain to the high-sensitivity R, G, B data multiplied by h_gain to generate combined data data of each color. The
[0156]
In step 122, the limiter 86 adjusts the combined data data of each color so as to fall within a range that can be expressed by a predetermined number of bits (for example, 8 bits).
[0157]
In step 124, the dot-sequential R, G, B signals read from the memory 24 are converted into simultaneous equations by the synchronization circuit 32 and output to the YC signal generation circuit 34.
[0158]
In step 126, the YC signal creation circuit 34 creates a luminance signal Y and chroma signals Cr, Cb from the synthesized R, G, B signals. These luminance signal Y and chroma signals Cr and Cb (YC signal) are stored in the memory 36.
[0159]
In step 128, the compression / decompression circuit 54 compresses the YC signal into a predetermined format.
[0160]
In Step 130, the recording unit 56 records the compressed image data on a recording medium such as a memory card.
[0161]
As described above, in the present embodiment, the maximum value on the highlight side is obtained using the histogram of the image data, and an evaluation value representing the high-contrast characteristic is obtained with accuracy from this and the subject luminance, and the dynamic range is obtained from this evaluation value. The characteristic of the high-sensitivity and low-sensitivity image data is changed by obtaining the total gain p for adjusting the image.
[0162]
Thereby, the dynamic range can be appropriately controlled according to the shooting scene. For example, the dynamic range is widened only when the evaluation value for high contrast is high, so when shooting a high-contrast scene, high-sensitivity and low-sensitivity image data is combined in any shooting scene. Compared to the case, the highlight portion can be reproduced appropriately, and robust gradation control is possible.
[0163]
In the present embodiment, the processing of low-sensitivity image data is performed in parallel with the processing of calculating the gain value for white balance adjustment based on the high-sensitivity image data, so that the processing speed is increased. Can do.
[0164]
In the present embodiment, an example in which each of the high sensitivity light receiving element PD1 and the low sensitivity light receiving element PD2 is provided to obtain a high sensitivity signal and a low sensitivity signal has been described. However, as shown in FIG. The light receiving regions of one light receiving element PD are divided into a high sensitivity light receiving region 92 having a large light receiving area for receiving high sensitivity light by a channel stopper 94 and a low sensitivity light receiving region 90 having a small light receiving area for receiving low sensitivity light. A configuration may be employed in which a high sensitivity signal and a low sensitivity signal are obtained depending on the region. Since the light receiving element PD is provided with the channel stopper 94, the signal received with high sensitivity and the signal received with low sensitivity are not mixed, and both signals can be received separately. .
[0165]
【The invention's effect】
As described above, according to the present invention, in the case of synthesizing images shot with different sensitivities, there is an effect that the highlight portion can be appropriately reproduced even when a high contrast scene is shot.
[Brief description of the drawings]
FIG. 1 is a rear view of a digital camera according to the present invention.
FIG. 2 is a plan view of a mode dial provided on the upper surface of the digital camera.
FIG. 3 is a block diagram showing an internal configuration of the digital camera.
FIG. 4 is a schematic configuration diagram of a CCD.
FIG. 5 is a block diagram of a digital signal processing circuit.
FIG. 6 is a diagram for explaining how to obtain an EV value;
FIG. 7 is a diagram for explaining a photometric method.
FIG. 8 is a flowchart showing a processing flow of the digital camera.
FIG. 9 is a flowchart showing a flow of gain calculation processing for white balance adjustment.
FIG. 10 is a graph showing a detection frame as a color distribution range of a light source.
FIG. 11 is a graph showing a membership function representing shading.
FIG. 12 is a graph showing a blue sky membership function.
FIG. 13 is a graph showing a membership function representing the outdoorness of daylight.
FIG. 14 is a graph showing a membership function representing outdoorness.
FIG. 15 is a graph showing a membership function representing indoorness.
FIG. 16 is a histogram of image data.
17A is a diagram showing the relationship between highlight data and evaluation value 1, FIG. 17B is a diagram showing the relationship between subject brightness and evaluation value 2, and FIG. 17C is evaluation value 1 × evaluation; It is a diagram which shows the relationship between the value 2 and the total gain p.
18A is a diagram showing the relationship between highlight data and evaluation value 1, and FIG. 18B is a diagram showing the relationship between subject brightness and evaluation value 2.
FIG. 19 is a diagram showing the relationship between high-sensitivity image data and weights.
FIG. 20 is a diagram showing the relationship between high-sensitivity image data and weights.
FIG. 21 is a diagram illustrating luminance-gradation characteristics of a high-sensitivity image, a low-sensitivity image, and a composite image.
FIG. 22 is a diagram showing luminance-gradation characteristics of a high-sensitivity image, a low-sensitivity image, and a composite image.
FIG. 23 is a schematic configuration diagram of another embodiment of a CCD.
[Explanation of symbols]
10 Photography lens 11 Digital camera 12 CCD
18 CDS circuit 20 A / D converter 26 digital signal processing circuit 30 synthesis processing unit 32 synchronization circuit 34 YC signal creation circuit 40 camera operation unit 52 liquid crystal monitor 54 expansion circuit 56 recording unit

Claims (3)

An image sensor for imaging a subject with a first sensitivity and a second sensitivity lower than the first sensitivity;
A luminance level detection means for detecting a luminance level of the subject based on at least one of a first captured image captured with the first sensitivity and a second captured image captured with the second sensitivity;
Histogram calculating means for calculating a histogram of the first captured image;
An evaluation value representing high contrast is determined based on highlight data on the highlight side of the histogram and the luminance level, and the dynamic range of the image sensor is adjusted based on the calculated evaluation value. A gain calculating means for calculating an adjustment gain;
Combining means for combining the first captured image and the second captured image based on the adjustment gain;
Digital camera equipped with. The gain calculating means calculates the adjustment gain by a membership function that acts so that a dynamic range of a synthesized image by the synthesizing means becomes wider as an evaluation value representing the high contrast likelihood becomes higher. Item 1. A digital camera according to Item 1. 3. The digital camera according to claim 1, wherein the processing by the histogram calculation unit and the gain calculation unit and the processing related to the second captured image are performed in parallel.

Images