JP3760846B2 - Subject extraction device and photographing device - Google Patents

Description

【００６６】
【数２】

【００６７】
図１６は、「ＣＩＥ １９７６ ＵＣＳ色度図」の簡略図である。図１６において領域ＲＨは予め計測等によって設定される肌色領域である。このため、上記数１及び数２の式によって求められる色度ｕ’，ｖ’を次の条件式にあてはめれば各画素が肌色画素であるか否かを判定することができる。
【００６８】
【数３】

【００６９】
【数４】

【００７０】
すなわち、上記数１の式によって算出される色度ｕ’が上記数３の条件式を満たし、かつ、上記数２の式によって算出される色度ｖ’が上記数４の条件式を満たす場合には、当該画素は肌色画素であることが判明する。そしてＵＣＳ色度図を利用して肌色画素を特定することにより、輝度値の影響を受けることがなく、正確に肌色画素の特定を行うことが可能になるとともに、ＲＧＢ値から色度ｕ’，ｖ’への変換が比較的簡単な演算であるため、効率的に演算処理を行うことも可能である。
【００７１】
なお、図１６においては一例として肌色領域ＲＨが四角形の領域として定義される場合を例示したが、例えばＵＣＳ色度図のｕ’ｖ’空間において台形や段円形の肌色領域を設定するようにしてもよい。
【００７２】
そして、この肌色画素抽出処理を画素ごとに行う場合、上述したように合焦評価領域の画像成分がぼけた状態にあるときには、画像サイズが小さく画素数が比較的少ないので、迅速に肌色画素の抽出処理を行うことが可能である。これに対し、合焦評価領域の画像成分が合焦状態に近い状態にあるときには、画像サイズが大きく画素数が比較的多いので、サンプル数が増加し、正確に被写体抽出を行うことが可能になる。
【００７３】
次に、ステップＳ２０６に進み、被写体抽出部４５は肌色ブロック抽出処理を行う。ここではステップＳ２０３のブロック分割処理によって分割された個々のブロックごとに肌色画素が所定割合以上含まれているか否かを判断し、所定割合以上肌色画素が含まれている場合に当該ブロックを肌色ブロックとして特定する。これに対し、１個のブロック中に含まれる肌色画素数が所定割合未満である場合には、そのブロックは肌色ブロックとしては抽出されず、主被写体を構成するものでないと判断される。
【００７４】
このように肌色ブロックを抽出し、以後の処理をブロック単位で実行していくことにより、画素単位で処理を進めていく場合に比べて処理速度の高速化を図ることができる。特にデジタルカメラ１の場合にはシャッタボタン８が半押し状態とされると、シャッタチャンスを逃さないためにも迅速に主被写体を合焦状態に導くことが求められるため、上記のようなブロック単位ごとの処理とすることは重要な意義を有する。また同時に、所定割合の肌色画素が存在するブロックを肌色ブロックとするため、肌色ブロックの信頼性を高めることにもなる。
【００７５】
また、合焦評価領域の画像成分がぼけた状態にあるときには、ブロック分割数は比較的少なく、その結果肌色ブロックとして特定されるブロックも少なくなるので、以後の処理を効率的に行うことが可能である。ただし、その反面、合焦評価領域の画像成分がぼけた状態にある場合には、後の形状判定処理において正確な主被写体の検出処理を行うことが困難になるとともに、被写体倍率が小さい（すなわち、画像中に主被写体が小さく写っている）と、肌色ブロックとして認識されず、主被写体を良好に抽出することができなくなる場合がある。
【００７６】
これに対し、合焦評価領域の画像成分が合焦状態に近い状態にあるときには、ブロック分割数は比較的多く、その結果肌色ブロックとして特定されるブロック数も多くなるので、以後の形状判定等の処理においてより正確に主被写体を抽出することが可能になる。また、被写体倍率が小さい（すなわち、画像中に主被写体が小さく写っている）場合でも、ブロック数が多いので、主被写体を複数のブロックがカバーすることとなるので、肌色ブロックを良好に認識することができ、主被写体を正確に抽出することが可能になる。
【００７７】
次に、ステップＳ２０７に進み、被写体抽出部４５は肌色ブロックとして特定されたブロックのグループ化処理を行う。グループ化とは、縦方向又は横方向に連続して分布する肌色ブロックを連結して肌色ブロックの集合体を形成させる処理である。このため、１個の肌色ブロックが他の肌色ブロックに接することなく孤立して存在する場合には、当該肌色ブロックはグループ化処理によって肌色ブロックの集合体形成から排除される。換言すれば、このグループ化処理により、主被写体としてありえないような小さな肌色ブロックは、主被写体を構成しないものとして取り扱われる。
【００７８】
また、ここでは、グループ化処理によって形成される肌色ブロックの集合体のグループ情報も生成される。グループ情報とは、肌色ブロックの集合体ごとに生成され、集合体を構成する縦方向及び横方向のブロック数とその集合体の重心位置とを含む情報である。このグループ情報は、後に行われる形状判定処理（ステップＳ２０８）及び顔の特定処理（ステップＳ２０９）において用いられる。
【００７９】
次に、ステップＳ２０８に進み、被写体抽出部４５はステップＳ２０７のグループ化処理によって形成された肌色ブロック集合体の形状判定処理を行う。一般に、画像から肌色グループ集合体を求めると、人物の肌部分だけが抽出される訳ではなく、事実上肌色を示す壁や樹木肌等の人物の肌以外の物体も肌色グループ集合体を形成し得る。また人物の手や足も肌色グループ集合体を形成し得る。そこで、この実施の形態では、被写体抽出部４５においてさらに形状判定処理を行うことで、人物の顔形状に近い形状分布をしている肌色グループ集合体を主被写体として認識するように実現されている。
【００８０】
ここでの形状判定処理は、ステップＳ２０４で設定された縦横比の範囲内に肌色ブロックの集合体が含まれるか否かを判断することにより行われる。具体的には、ステップＳ２０７で生成されたグループ情報に含まれる縦方向及び横方向のブロック数を取得し、肌色ブロック集合体の縦方向及び横方向の縦横比（Ｍ：１）を求める。そしてステップＳ２０４で設定された縦横比がＫ：１である場合、Ｍ≦Ｋの関係が成立すれば、当該肌色ブロック集合体は人物の顔部分を撮影した画像成分であると判断する。
【００８１】
なお、Ｍ≦Ｋの関係だけでは、横方向一列状に分布する肌色ブロック集合体も人物の顔部分であると判断されるので、そのような事態を避けるために、Ｍの値に下限値Ｌを設定してもよい。すなわち、Ｌ≦Ｍ≦Ｋの関係が成立すれば当該肌色ブロック集合体は人物の顔部分を撮影した画像成分であると判断するようにしてもよい。
【００８２】
図１７は、形状判定処理（ステップＳ２０８）において主被写体であると認識される領域を示す図である。なお、図１７では、主被写体であると認識される領域を黒塗りで示している。図１７に示すように、被写体抽出部４５が形状判定処理を行うことにより、人物の顔部分が含まれるブロックが主被写体の領域であるものとして特定されることになる。
【００８３】
次に、ステップＳ２０９に進み、被写体抽出部４５は形状判定処理によって複数の主被写体が特定された場合に、そのうちから一の主被写体を特定するために顔の特定処理、すなわち主被写体の特定処理を行う。被写体抽出部４５には予め画面に対する重み付け係数が設定されており、主被写体として特定された肌色ブロック集合体の重心位置に基づいて当該肌色ブロック集合体についての重み付け係数を特定する。
【００８４】
図１８は画面に対する重み付け係数の一例を示す図である。図１８に示すように画面Ｇ５が複数の領域に分割されており、各領域にその領域の重み付け係数が定義されている。なお、画面Ｇ５のサイズは、画素間引き数に応じて変化し、画素間引き処理後の画像Ｇ１，Ｇ２（図１１参照）の画像サイズと一致するように調整される。そして、肌色ブロック集合体の重心位置が画面Ｇ５のどの位置に位置するかを特定し、その特定結果に基づいて重み付け係数を特定する。例えば、図１７において、主被写体として特定された黒塗りの肌色ブロック集合体について考えると、その重心位置は図１８において位置Ｐ１にあるため、重み付け係数は「１０」と特定される。そして、被写体抽出部４５は、当該肌色ブロック集合体の面積（例えば、肌色ブロック集合体を構成するブロック数）と、特定された重み付け係数の積を算出し、肌色ブロック集合体の主被写体適合値（主被写体らしさを示す値）とする。そして、複数の肌色ブロック集合体のそれぞれについて主被写体適合値を算出すると、各主被写体適合値を比較し、最大値を示す肌色ブロック集合体を主被写体となる人物の顔部分であると認定する。
【００８５】
このように、複数の主被写体が特定された場合には、画面の中央寄りに主被写体がフレーミングされるであろうとの前提にたって、画面上の位置に対する重み付け係数を用いて主被写体適合値を求めることにより、一の主被写体を正確に特定することができる。また、小さく写っている物体よりも大きく写っている物体の方が主被写体である可能性が高いため、重み付け係数を用いた計算を行う際には、肌色ブロック集合体の面積を考慮に入れることにより、正確に主被写体を特定することが可能になる。
【００８６】
なお、最大値を示す主被写体適合値から所定範囲内の値を示す複数の主被写体適合値を採用してもよい。この場合、主被写体領域として複数の領域が特定されることになるので、いわゆるマルチエリアの合焦制御を適用することになる。
【００８７】
以上のようにして主被写体が特定されると、被写体抽出部４５は全体制御部１８に対して主被写体領域を指示することにより、主被写体領域が液晶表示部３３にも表示されることになる（ステップＳ２１０）。これにより、ユーザは、画像のどの部分が主被写体と認識され、合焦制御がなされるかを容易に把握することができる。また、被写体抽出部４５は、上記のような処理を行うことによって特定した主被写体領域を合焦評価領域設定部４１に指示する。
【００８８】
図７のフローチャートに戻り、ステップＳ１０４の被写体抽出処理が終了すると、ステップＳ１０５に進み、合焦評価領域設定部４１は合焦評価部４２が評価値を算出するための合焦評価領域を被写体抽出部４５によって指示された被写体領域に設定変更する。この結果、合焦評価部４２において次回の評価値算出が行われる際には、主被写体の画像成分について評価値算出を行うことが可能になる。
【００８９】
次に合焦制御部４３によって所定ピッチでのフォーカシングレンズ１２のレンズ駆動が行われ（ステップＳ１０６）、そのレンズ位置での画像取り込みが行われて（ステップＳ１０７）、合焦評価部４２による評価値算出処理が行われる（ステップＳ１０８）。ステップＳ１０８において合焦評価部４２は、ステップＳ１０５の合焦評価領域の変更設定処理で設定された合焦評価領域の画像成分に基づいて評価値を算出する。
【００９０】
以後、評価値が最大値（又は最小値）を示すまでステップＳ１０６〜Ｓ１０８の処理を繰り返し、評価値が最大値（又は最小値）を示すレンズ位置を特定することができた場合、すなわちステップＳ１０９において合焦評価領域の合焦状態が実現された場合に、図８のフローチャートに進む。
【００９１】
そして、合焦評価領域の合焦状態が実現された段階で再度被写体抽出処理を実行する（ステップＳ１１０）。ここでの被写体抽出処理（ステップＳ１１０）も、図９を参照して説明した上記ステップＳ１０４の被写体抽出処理と同様である。
【００９２】
ただし、ステップＳ１０４の段階では、合焦評価領域の画像成分がぼけた状態で主被写体の抽出処理が行われるため、画素間引き数が大きく、ブロック分割数も少ない状態で被写体抽出処理が行われる。そしてまた、形状判定基準も厳しい状態ではないため、効率的に主被写体であると推定される領域は特定されるが、その精度は低い。そのため、合焦評価領域の画像成分がぼけた状態で主被写体の抽出処理が行われ、それによって特定された主被写体領域が合焦状態と判定されたとしても、その合焦評価領域には実際の主被写体が含まれていない可能性がある。
【００９３】
そのため、この実施の形態ではステップＳ１１０において再度被写体抽出処理を行うこととしている。そしてステップＳ１１０の段階では、合焦評価領域の画像成分が合焦状態であると判断された状態であるため、被写体抽出処理において、画素間引き数が小さく、かつ、ブロック分割数が多い状態で被写体抽出処理が行われる。そしてまた、形状判定基準も厳しい状態となるため、主被写体領域を高精度に特定することが可能である。
【００９４】
そして被写体抽出処理（ステップＳ１１０）が行われて、信頼性の高い主被写体領域が特定されると、被写体抽出部４５は前回同様に合焦評価領域設定部４１に対して特定された主被写体領域を指示する。
【００９５】
これにより、合焦評価領域設定部４１は、ステップＳ１０８において採用されていた合焦評価領域と、ステップＳ１１０の被写体抽出処理において特定された被写体領域とが一致するか否かを判断する（ステップＳ１１１）。つまり、この処理は、ステップＳ１０６〜Ｓ１０８の繰り返し処理で評価対象とされていた合焦評価領域が正確な主被写体領域であったのかどうかを確認するための処理である。その結果、合焦評価領域と、特定された主被写体領域とが一致しない場合には、ステップＳ１０５（図７）に戻り、合焦評価領域設定部４１は合焦評価領域を、ステップＳ１１０の被写体抽出処理で特定された主被写体領域に設定変更する。よって、合焦評価部４２において次回の評価値算出が行われる際には、信頼性の高い主被写体領域の画像成分について評価値算出を行うことが可能になり、ステップＳ１０６以降の処理が行われる。その一方、合焦評価領域と、特定された主被写体領域とが一致する場合には、ステップＳ１１２（図８）に進む。
【００９６】
合焦評価領域設定部４１が、合焦評価領域と、特定された主被写体領域とが一致すると判断した場合には、全体制御部１８に対して本撮影動作の許可を行う（ステップＳ１１２）。
【００９７】
全体制御部１８は、本撮影動作の許可が与えられると、まず、ユーザによってシャッタボタン８が全押し状態とされたか否かを判断する。デジタルカメラ１ではシャッタボタン８の全押し状態が本撮影動作の条件となっているため、シャッタボタン８が全押し状態でない場合には、フレーミングが変更された場合に対処するために、ステップＳ１０２（図７）に戻ってシャッタボタン８が半押し状態とされた以降の処理を繰り返す。
【００９８】
また、シャッタボタン８が全押し状態とされた場合には、全体制御部１８が撮影機能部２０を制御して本撮影動作を行い（ステップＳ１１４）、画像処理部３２において所定の画像処理（ステップＳ１１５）を行った後、記録メディア９への記録処理（ステップＳ１１６）が行われる。
【００９９】
以上で、シャッタボタン８が半押し状態とされてから本撮影動作が完了するまでのデジタルカメラ１の処理手順が終了することとなり、本撮影動作においては、主被写体が画像のどの部分に存在する場合であってもその主被写体を合焦状態として撮影することが可能である。
【０１００】
そして、この実施の形態のデジタルカメラ１においては、被写体抽出装置としても機能する主被写体合焦制御部４０が、合焦評価領域の評価結果に基づいて主被写体（特定被写体）の抽出処理を行う際の処理内容を変更し、その処理内容に基づいて画像のうちから主被写体の抽出処理を行うように構成されているため、合焦評価領域の画像成分がぼけている場合には、処理効率を重視した処理内容を設定することができ、また、合焦評価領域の画像成分が合焦状態に近い状態にある場合には、被写体抽出の信頼性を重視した処理内容を設定することができる。
【０１０１】
＜２．第２の実施の形態＞
次に、第２の実施の形態について説明する。なお、この実施の形態においても、装置構成については第１の実施の形態で説明したものと同様である（図２等参照）。
【０１０２】
上記第１の実施の形態では、合焦制御部４３が所定ピッチでフォーカシングレンズ１２を段階的に駆動させつつ、合焦評価部４２において各レンズ位置での評価値が算出されるが、被写体抽出処理が実行されるのは、フォーカシングレンズ１２を段階的に駆動させる前と合焦評価領域が合焦状態に達した後である。そのため、フォーカシングレンズ１２を段階的に駆動させる前に実行される被写体抽出処理によって実際には主被写体と全く異なる被写体が主被写体であると特定されていた場合には、その被写体が含まれる合焦評価領域を合焦状態に導くための動作が無駄になる。
【０１０３】
このため、この実施の形態では、フォーカシングレンズ１２を段階的に移動させる最中にも、被写体抽出処理を実行することにより、無駄な動作を省いてさらに効率的に主被写体を正確に合焦状態に導く形態について説明する。
【０１０４】
図１９及び図２０は、第２の実施の形態におけるデジタルカメラ１の処理手順を示すフローチャートであり、特にシャッタボタン８が半押し状態とされてから本撮影動作が行われるまでの全体的な処理手順を示している。
【０１０５】
まず、ユーザによってシャッタボタン８が半押し状態にされると（ステップＳ３０１）、全体制御部１８が主被写体合焦制御部４０と撮影機能部２０とを機能させる。
【０１０６】
そして合焦制御部４３がレンズ駆動部５０を制御してフォーカシングレンズ１２を最も遠側に移動させる（ステップＳ３０２）。なお、ステップＳ３０２の処理が次回以降実行される場合には、合焦制御部４３がフォーカシングレンズ１２を近側方向に所定ピッチ分移動させる制御動作を行うことになる。
【０１０７】
次に、合焦評価領域の設定処理が行われる（ステップＳ３０３）。合焦評価領域設定部４１は、被写体抽出部４５から主被写体領域が指定されている場合はその指定された主被写体領域を合焦評価領域に設定する。ただし、シャッタボタン８が半押し状態にされてから最初にステップＳ３０３の処理が行われる際には、未だ被写体抽出処理（ステップＳ３０８）は実行されていない。このため、未だ被写体抽出部４５から主被写体領域が指定されていない場合、合焦評価領域設定部４１は画像の中央部分に予め設定された初期状態の合焦評価領域ＦＲ１（図３参照）を評価値算出対象の合焦評価領域として設定する。これに対し、既に被写体抽出処理が何回か実行されている場合には、合焦評価領域設定部４１は前回指定された最新の主被写体領域を評価値算出対象の合焦評価領域として設定する。そして、合焦評価領域設定部４１は合焦評価領域を合焦評価部４２に対して指示する。
【０１０８】
なお、撮影モードとして例えばスナップモードやポートレートモード等のような人物を撮影するためのモードが選択設定されている場合には、シャッタボタン８が半押し状態とされたときに、被写体抽出処理を１回行っておくようにしてもよい。そのような処理手順とすれば、ステップＳ３０３の合焦評価領域の設定処理を行う際には、常に主被写体と推定される領域を合焦評価領域として設定することが可能である。
【０１０９】
そして、全体制御部１８はフォーカシングレンズ１２が現在のレンズ位置にあるときに撮影機能部２０における撮影動作を行わせる。そして、合焦評価部４２は画像メモリ３１に格納された画像を取り込み（ステップＳ３０４）、評価値の算出処理を行う（ステップＳ３０５）。このとき、合焦評価部４２は、ステップＳ３０３において合焦評価領域設定部４１から指定された合焦評価領域の画像成分に基づいて評価値の算出を行い、その結果算出された評価値は合焦制御部４３及び処理内容変更４４に与えられる。
【０１１０】
そして処理内容変更部４４は、被写体抽出処理を実行するか否かの判定を行う（ステップＳ３０６）。この実施の形態では、フォーカシングレンズ１２を所定ピッチで段階的に移動させる際に、各レンズ位置で被写体抽出処理を実行することができるように構成されるが、所定ピッチでのレンズ駆動ごとに毎回被写体抽出処理を実行するように構成すると、主被写体を合焦状態に導くまでに長時間を要することになる。そのため、この実施の形態の処理内容変更部４４は、被写体抽出部４５における処理内容を変更する機能だけでなく、被写体抽出部４５にける被写体抽出処理の実行頻度を調整する機能をも備える。
【０１１１】
処理内容変更部４４が被写体抽出処理の実行頻度を調整する方法には、いくつかの方法がある。まず、第１に、所定ピッチでのフォーカシングレンズ１２のレンズ駆動が所定回数行われるごとに被写体抽出処理を１回実行する方法がある。これにより、レンズ駆動が行われる度に毎回被写体抽出処理が行われることを回避することができるので、効率的な合焦動作を実現することができる。
【０１１２】
また、第２に、合焦評価部４２で求められる評価値を参酌して合焦評価領域の画像成分が合焦状態に近づくにつれて被写体抽出処理の実行頻度を増加させる方法がある。例えば、評価値が合焦位置で最大値を示す場合には、評価値が第１の閾値未満の場合はレンズ駆動が１０回行われるごとに１回被写体抽出処理を実行し、第１の閾値以上第２の閾値未満の場合はレンズ駆動が５回行われるごとに１回被写体抽出処理を実行し、第２の閾値以上の場合はレンズ駆動が２回行われるごとに１回被写体抽出処理を実行するというように、合焦評価部４２における評価結果に基づいて被写体抽出処理の実行頻度を調整するのである。このように構成することにより、合焦評価領域の画像成分が合焦状態に近づくにつれて、主被写体を抽出するための処理が増加する。合焦評価領域の画像成分が合焦状態に近い場合には比較的高精度に主被写体の特定が行われるので、この第２の方法は、主被写体を合焦状態に導くためには効果的な方法となる。
【０１１３】
ただし、処理内容変更部４４が被写体抽出処理の実行頻度を調整する方法は、上記第１又は第２の方法に限定されるものではなく、他の方法を採用してもよい。また、主被写体合焦制御部４０における処理速度が十分に速い場合には、レンズ駆動後の毎回の画像取り込み時に被写体抽出処理を実行するようにしてもよい。
【０１１４】
そして、処理内容変更部４４がレンズ駆動回数をカウントして被写体抽出処理を実行すると判定した場合には、ステップＳ３０８の被写体抽出処理に進む。これに対し、被写体抽出処理を実行しないと判断した場合には、被写体抽出処理（ステップＳ３０８）をスキップさせて合焦判定を行う処理、すなわちステップＳ３１０（図２０のフローチャート）に進むことになる。
【０１１５】
この実施の形態においても、被写体抽出処理（ステップＳ３０８）の詳細は、図９に示すフローチャートと同様である。したがって、合焦評価領域がぼけた状態にあるときは処理効率を重視した被写体抽出処理が行われ、合焦評価領域が合焦状態に近づくにつれて被写体抽出処理によって特定される主被写体の信頼性が向上することになる。
【０１１６】
そして被写体抽出処理（ステップＳ３０８）において主被写体の領域が特定されるとその主被写体領域が被写体抽出部４５から合焦評価領域設定部４１に伝えられる。
【０１１７】
ステップＳ３０９に進み、合焦評価領域設定部４１は、それまでの合焦評価領域と、被写体抽出処理によって特定された主被写体領域とを比較し、両者が一致するか否かを判断する。この処理は、それまで評価対象とされていた合焦評価領域が正確な主被写体領域であったのかどうかを確認するための処理である。そして、一致しない場合には、ステップＳ３０３に戻り、合焦評価領域設定部４１は以後の合焦評価領域を、ステップＳ３０８の被写体抽出処理で特定された主被写体領域に設定変更する。よって、合焦評価部４２において次回の評価値算出が行われる際には、前回よりも信頼性の高い主被写体領域の画像成分について評価値算出を行うことが可能になる。その一方、合焦評価領域と、特定された主被写体領域とが一致する場合には、ステップＳ３１０（図２０）に進む。
【０１１８】
そして合焦制御部４３は、図４に示すような評価値が最大値（又は最小値）を示すレンズ位置ＰＸを特定することができたか否か、すなわち、合焦評価領域の合焦状態が実現されたか否かを判断する。ここで、未だ合焦評価領域の合焦状態が実現されていないと判断された場合には、ステップＳ３０２（図１９）に戻って所定ピッチでのレンズ駆動、評価値算出処理等を繰り返す。その結果、最終的に合焦評価領域の合焦状態が実現された場合には、ステップＳ３１１に進み、全体制御部１８に対して本撮影動作の許可を行う。
【０１１９】
全体制御部１８は、本撮影動作の許可が与えられると、まず、ユーザによってシャッタボタン８が全押し状態とされたか否かを判断する（ステップＳ３１２）。シャッタボタン８が全押し状態でない場合には、フレーミングが変更された場合に対処するために、ステップＳ３０２（図１９）に戻ってシャッタボタン８が半押し状態とされた以降の処理を繰り返す。なお、このとき、合焦制御部４３は合焦制御を最初から行うことが必要になるので、フォーカシングレンズ１２を最も遠側の位置に移動させることになる。
【０１２０】
これに対し、シャッタボタン８が全押し状態とされた場合には、全体制御部１８が撮影機能部２０を制御して本撮影動作を行い（ステップＳ３１３）、画像処理部３２において所定の画像処理（ステップＳ３１４）を行った後、記録メディア９への記録処理（ステップＳ３１５）が行われる。
【０１２１】
以上で、第２の実施の形態のデジタルカメラ１において、シャッタボタン８が半押し状態とされてから本撮影動作が完了するまでの処理手順が終了することとなり、本撮影動作においては、主被写体が画像のどの部分に存在する場合であってもその主被写体を合焦状態として撮影することが可能である。
【０１２２】
そして、この実施の形態のデジタルカメラ１においても、被写体抽出装置としても機能する主被写体合焦制御部４０が、合焦評価領域の評価結果に基づいて主被写体（特定被写体）の抽出処理を行う際の処理内容を変更し、その処理内容に基づいて画像のうちから主被写体の抽出処理を行うように構成されているため、合焦評価領域の画像成分がぼけている場合には、処理効率を重視した処理内容を設定することができ、また、合焦評価領域の画像成分が合焦状態に近い状態にある場合には、被写体抽出の信頼性を重視した処理内容を設定することができる。その結果、効率的かつ正確に主被写体を抽出して主被写体の合焦状態を実現することが可能になる。
【０１２３】
また、この実施の形態のデジタルカメラ１では、合焦動作を行うためのレンズ駆動段階でも被写体抽出処理が実行されるように構成されているので、主被写体でない部分を評価対象として合焦動作を進められることが回避されるので、さらに効率的な合焦動作が実現される。
【０１２４】
さらに、合焦評価領域の画像成分が合焦状態に近づくにつれて、被写体抽出処理の実行頻度が増加するように構成することで、より正確に主被写体を抽出することができ、その信頼性の高い主被写体を迅速に合焦状態に導くことが可能になる。
【０１２５】
＜３．変形例＞
以上、この発明の実施の形態について説明したが、この発明は上記説明した内容のものに限定されるものではない。
【０１２６】
例えば、上記説明におけるデジタルカメラには、デジタルスチルカメラとデジタルビデオカメラの双方が含まれる。また、撮影装置の一例としてデジタルカメラ１を例示して述べたが、この発明はデジタルカメラ以外の撮影装置に対しても適用することが可能である。
【０１２７】
また、上記説明においては、いわゆる山登り方式等と呼ばれるような合焦制御を行う場合について述べたが、アクティブ方式や位相差方式等と呼ばれる自動焦点検出方式、さらに複数の測距エリアで測距を行う自動焦点検出方式等の制御方式であっても、この発明を適用することは可能である。
【０１２８】
また、上記説明においては、合焦動作を開始させるタイミングがシャッタボタン８の半押し後である場合について述べたが、それに限定されるものではなく、例えば、ライブビュー画像の表示を行っている状態においても上述した合焦動作を行うようにしてもよい。
【０１２９】
また、上記説明においては、画素間引き数、ブロック分割数、及び形状判定基準と、合焦評価領域のぼけ量とは連続的な関係を示すものであったが、ぼけ量の変化に対して、画素間引き数、ブロック分割数、及び形状判定基準のそれぞれが段階的に変化するようにしてもよい。
【０１３０】
また、上記説明においては、主被写体（すなわち特定被写体）が人物の顔部分である場合について述べた。これは、一般的に写真撮影を行う場合、人物を被写体として撮影することが多いことに鑑みたものであるが、この発明の本質としては、主被写体は人物の顔部分であることに限定されるものではない。このため、主被写体を構成する色成分や形状等に関する情報をユーザが自由に設定入力することができるようにすれば、ユーザが意図する主被写体を適切に特定することが可能になる。また、撮影モードとしてスナップモードやポートレートモード等のような人物を被写体とするモードが選択されている場合に、上述したような人物の顔部分を主被写体として抽出するための処理を行うように構成することも可能である。また逆に、人物の顔部分の抽出によって被写体が風景であることを判別したり、主被写体の肌色ブロック集合体の面積によってスナップやポートレート等であることを判別したりすることができるので、オートシーン判別を行うことが可能である。
【０１３１】
さらに、上記説明においては、被写体抽出処理が主被写体を合焦状態に導くために行われる場合について説明したが、被写体抽出処理の結果によって画像中から主被写体領域が特定されることに鑑みれば、被写体抽出処理を自動合焦制御のために実行するだけでなく、主被写体領域の画像成分に対して特別な画像処理を施すために実行することも可能である。つまり、上述した被写体抽出技術は、主被写体を合焦状態に導くために適用されるものに限定されるものではない。
【０１３２】
【発明の効果】
以上説明したように、請求項１に記載の発明によれば、合焦評価領域に含まれる被写体像が合焦状態に近づくにつれて、特定被写体の抽出処理における、画素間引き処理の間引き数を減少させていくように構成されるため、合焦評価領域の画像成分がぼけている場合には、処理効率を重視した処理内容を設定することができ、また、合焦評価領域の画像成分が合焦状態に近い状態にある場合には、特定被写体抽出の信頼性を重視した処理内容を設定することができる。また、合焦評価領域のぼけ量に応じて特定被写体の抽出処理の処理内容が変化させることができる。そして、合焦評価領域が合焦状態に近づけば特定被写体を抽出するためのサンプル数が増加するので、高精度に特定被写体の抽出を行うことができる。逆に、合焦評価領域がぼけている場合はサンプル数が少ないので、迅速に特定被写体の抽出処理を行うことができる。その結果、効率的かつ正確に特定被写体を抽出することが可能になる。
【０１３３】
請求項２に記載の発明によれば、合焦評価領域に含まれる被写体像が合焦状態に近づくにつれて、特定被写体の抽出処理における、ブロック分割処理のブロック分割数を増加させていくように構成されるため、合焦評価領域の画像成分がぼけている場合には、処理効率を重視した処理内容を設定することができ、また、合焦評価領域の画像成分が合焦状態に近い状態にある場合には、特定被写体抽出の信頼性を重視した処理内容を設定することができる。また、合焦評価領域のぼけ量に応じて特定被写体の抽出処理の処理内容が変化させることができる。そして、合焦評価領域が合焦状態に近づけば特定被写体を抽出するためのブロック数が増加するので、高精度に特定被写体の抽出を行うことができる。逆に、合焦評価領域がぼけている場合はブロック数が少ないので、迅速に特定被写体の抽出処理を行うことができる。その結果、効率的かつ正確に特定被写体を抽出することが可能になる。
【０１３５】
請求項３に記載の発明によれば、合焦評価領域に含まれる被写体像が合焦状態に近づくにつれて、特定被写体の抽出処理の実行頻度を増加させるため、合焦評価領域の画像成分がぼけている場合には、処理効率を重視した処理内容を設定することができ、また、合焦評価領域の画像成分が合焦状態に近い状態にある場合には、特定被写体抽出の信頼性を重視した処理内容を設定することができる。つまり、合焦状態に近づくにつれて、高精度な特定被写体の抽出が可能になる。その結果、効率的かつ正確に特定被写体を抽出することが可能になる。
【０１３６】
請求項４に記載の発明によれば、画像の肌色判別を行うことによって特定被写体を抽出するため、人物が被写体である場合に、その人物像を特定被写体として抽出することが可能である。
【０１３７】
請求項５に記載の発明によれば、特定被写体の抽出処理によって抽出される特定被写体の領域と合焦評価領域とが一致しない場合に、合焦評価領域を特定被写体の領域に設定変更するように構成されるため、特定被写体の合焦状態を適切に評価することが可能である。
【０１３８】
請求項６に記載の発明によれば、合焦評価領域に含まれる被写体像が合焦状態に近づくにつれて、画素間引き処理の間引き数を減少させるように、特定被写体の抽出処理を行う際の処理内容を変更し、その変更された処理内容に基づいて画像のうちから特定被写体の抽出処理を行い、それによって抽出される特定被写体の領域を合焦評価領域として設定するように構成されるため、合焦評価領域の画像成分がぼけている場合には、処理効率を重視した処理内容を設定することができ、また、合焦評価領域の画像成分が合焦状態に近い状態にある場合には、特定被写体抽出の信頼性を重視した処理内容を設定することができる。その結果、効率的かつ正確に特定被写体を特定してその特定被写体を合焦状態に導くことができる。
【０１３９】
請求項７に記載の発明によれば、合焦評価領域に含まれる被写体像が合焦状態に近づくにつれて、ブロック分割処理のブロック分割数を増加させていくように、特定被写体の抽出処理を行う際の処理内容を変更し、その変更された処理内容に基づいて画像のうちから特定被写体の抽出処理を行い、それによって抽出される特定被写体の領域を合焦評価領域として設定するように構成されるため、合焦評価領域の画像成分がぼけている場合には、処理効率を重視した処理内容を設定することができ、また、合焦評価領域の画像成分が合焦状態に近い状態にある場合には、特定被写体抽出の信頼性を重視した処理内容を設定することができる。その結果、効率的かつ正確に特定被写体を特定してその特定被写体を合焦状態に導くことができる。
【０１４１】
請求項８に記載の発明によれば、合焦評価領域に含まれる被写体像が合焦状態に近づくにつれて、特定被写体の抽出処理の実行頻度を増加させていくように、特定被写体の抽出処理を行う際の処理内容を変更し、その変更された処理内容に基づいて画像のうちから特定被写体の抽出処理を行い、それによって抽出される特定被写体の領域を合焦評価領域として設定するように構成されるため、合焦評価領域の画像成分がぼけている場合には、処理効率を重視した処理内容を設定することができ、また、合焦評価領域の画像成分が合焦状態に近い状態にある場合には、特定被写体抽出の信頼性を重視した処理内容を設定することができる。その結果、効率的かつ正確に特定被写体を特定してその特定被写体を合焦状態に導くことができる。
請求項９に記載の発明によれば、特定被写体を抽出した際に、特定被写体の領域と合焦評価領域とが一致する場合に、本撮影を行うための撮影動作を許可するため、特定被写体の合焦状態が評価された状態で適切に本撮影動作を行うことが可能になる。
【図面の簡単な説明】
【図１】撮影装置の一例としてのデジタルカメラを示す斜視図である。
【図２】デジタルカメラの内部構成を示すブロック図である。
【図３】デジタルカメラに予め設定される合焦評価領域の初期状態を示す図である。
【図４】レンズ位置に対する評価値変化の一例を示す図である。
【図５】固定された合焦評価領域で合焦制御を行った場合の撮影画像を示す図である。
【図６】ユーザが撮影を意図する画像を示す図である。
【図７】第１の実施の形態におけるデジタルカメラの処理手順を示すフローチャートである。
【図８】第１の実施の形態におけるデジタルカメラの処理手順を示すフローチャートである。
【図９】被写体抽出処理の詳細な処理手順を示すフローチャートである。
【図１０】ぼけ量と画素間引き数との関係を示す図である。
【図１１】画素間引き処理の概念を示す図である。
【図１２】ぼけ量とブロック分割数との関係を示す図である。
【図１３】ぼけ量が大きい場合のブロック分割処理の概念を示す図である。
【図１４】ぼけ量が小さい場合のブロック分割処理の概念を示す図である。
【図１５】ぼけ量と、形状判定基準である縦横比との関係を示す図である。
【図１６】ＣＩＥ １９７６ ＵＣＳ色度図の簡略図である。
【図１７】形状判定処理において主被写体であると認識される領域を示す図である。
【図１８】画面に対する重み付け係数の一例を示す図である。
【図１９】第２の実施の形態におけるデジタルカメラの処理手順を示すフローチャートである。
【図２０】第２の実施の形態におけるデジタルカメラの処理手順を示すフローチャートである。
【符号の説明】
１ デジタルカメラ（撮影装置）
８ シャッタボタン
１１ 撮影レンズ
１２ フォーカシングレンズ
１８ 全体制御部
２０ 撮影機能部
３０ ＣＣＤ撮像素子
３１ 画像メモリ
４０ 主被写体合焦制御部（被写体抽出装置）
４１ 合焦評価領域設定部
４２ 合焦評価部
４３ 合焦制御部
４４ 処理内容変更部
４５ 被写体抽出部
５０ レンズ駆動部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a subject extraction technique for extracting a specific subject from an image.
[0002]
[Prior art]
When automatic focusing control (autofocus control) is performed in a conventional camera or the like, in which region of the screen the focus detection is performed is determined in advance. For example, automatic focus detection is performed in the focus evaluation area set in the center of the image, or an area with the closest subject distance is specified from among a plurality of focus evaluation areas, and automatic focus detection is performed in that area. .
[0003]
In the technical field of image processing, a technique for detecting a human image from an image has been proposed. In the human image extraction process using such a technique, since the main object is always to detect human images with high accuracy, no matter what the target image is, the human image is always detected with high accuracy. Processing for detection is performed.
[0004]
[Problems to be solved by the invention]
However, in the conventional automatic focusing control, only automatic focus detection is performed in a predetermined area in the screen. Therefore, when the main subject intended by the user is not included in the area, the user As a result, focus detection is performed in an area different from the intention of the user, and as a result, there is a problem that the main subject that the user wanted to shoot is shot with a blurred image.
[0005]
It is also possible to apply a human image detection technique proposed in the field of image processing to detect a person from an image, specify the human image as a main subject, and perform focus detection. If the same person image extraction process is performed even for an image, there is a problem that the processing time becomes long. On the other hand, if a rough person extraction process is performed with an emphasis on processing efficiency, there is a problem that accurate person detection cannot be performed.
[0006]
In particular, in a photographing apparatus such as an electronic camera, when the main subject is detected and the photographing operation is performed in a state where the main subject is properly focused, the process of extracting the main subject is made efficient. It is also desirable to improve accuracy at the same time.
[0007]
Accordingly, the present invention has been made in view of the above problems, and has an object to provide a subject extraction device capable of improving the processing efficiency and accurately extracting the main subject. Another object of the present invention is to provide a photographing apparatus to which such subject extraction technology is applied.
[0008]
[Means for Solving the Problems]
  In order to achieve the above object, the invention described in claim 1 is a subject extraction device that detects a specific subject from an image, and is a focus evaluation unit that evaluates a focus state of a focus evaluation region in the image. And focusing control means for controlling the subject image included in the focusing evaluation area to be in focus based on the evaluation result of the focusing evaluation means, and based on the evaluation result of the focusing evaluation area. Processing content changing means for changing the processing content when performing the extraction processing of the specific subject; and subject extraction means for extracting the specific subject from the image based on the processing content.The subject extraction unit is configured to perform pixel thinning processing in the extraction process of the specific subject, and the processing content change unit is controlled by the focus control unit as the subject image approaches a focused state. The thinning-out number in the pixel thinning-out process is reduced..
[0009]
  The invention described in claim 2Detect a specific subject from an imageSubject extraction deviceBecause,Focus evaluation means for evaluating the focus state of the focus evaluation area in the image, and control so that the subject image included in the focus evaluation area is in focus based on the evaluation result of the focus evaluation means. A focus control means for performing processing content change means for changing the processing content when performing the extraction processing of the specific subject based on the evaluation result of the focus evaluation area, and the image based on the processing content. Subject extraction means for performing extraction processing of the specific subject, wherein the subject extraction means is configured to perform block division processing in the extraction processing of the specific subject, and the processing content changing means is configured to perform the focusing. As the subject image approaches the in-focus state under the control of the control means, the number of block divisions in the block division processing is increased.It is characterized by that.
[0011]
  Claim3The subject extraction device for detecting a specific subject from an image includes a focus evaluation unit that evaluates a focus state of a focus evaluation region in the image, and an evaluation result of the focus evaluation unit. A focus control means for controlling the subject image included in the focus evaluation area based on a focus state, and a process for performing the extraction process of the specific subject based on the evaluation result of the focus evaluation area Processing content changing means for changing the content; and subject extracting means for extracting the specific subject from the image based on the processing content, wherein the processing content changing means is the focus control means As the subject image approaches the in-focus state by the control, the execution frequency of the specific subject extraction process is increased.
[0012]
  Claim4The invention described in claims 1 to3In the subject extraction device according to any one of the above, the subject extraction unit extracts the specific subject by performing skin color discrimination of the image in color determination processing.
[0013]
  Claim5The invention described in claims 1 to4In the subject extraction device according to any one of the above, if the region of the specific subject extracted by the subject extraction unit and the focus evaluation region do not match, the focus evaluation region is set as the region of the specific subject Means for changing.
[0014]
  Claim6The invention described in (1) is a photographing apparatus for photographing an image through a photographing lens, the focusing evaluation means for evaluating a focusing state of a focusing evaluation area in the image, and the evaluation result of the focusing evaluation means. A focus control means for controlling the subject image included in the focus evaluation area to be in a focused state by driving the photographing lens based on the focus lens, and a specific subject based on the evaluation result of the focus evaluation area The processing content changing means for changing the processing content when performing the extraction processing, the subject extracting means for extracting the specific subject from the image based on the processing content, and the subject extracting means Means for setting the area of the specific subject as the focus evaluation area, and the subject extraction means is configured to perform pixel thinning processing in the extraction processing of the specific subject, Serial processing content changing means, as the object image under the control of the focusing control means approaches the focus state, is characterized by gradually reducing the thinning number in the pixel thinning process.
[0015]
  Claim7The invention described in (1) is a photographing apparatus for photographing an image through a photographing lens, the focusing evaluation means for evaluating a focusing state of a focusing evaluation area in the image, and the evaluation result of the focusing evaluation means. A focus control means for controlling the subject image included in the focus evaluation area to be in a focused state by driving the photographing lens based on the focus lens, and a specific subject based on the evaluation result of the focus evaluation area The processing content changing means for changing the processing content when performing the extraction processing, the subject extracting means for extracting the specific subject from the image based on the processing content, and the subject extracting means Means for setting the area of the specific subject as the focus evaluation area, and the subject extraction means is configured to perform block division processing in the extraction processing of the specific subject. The processing content changing means, as the object image under the control of the focusing control means approaches the focus state, it is characterized by gradually increasing the block division number in the block division processing.
[0017]
  Claim8The invention described in (1) is a photographing apparatus for photographing an image through a photographing lens, the focusing evaluation means for evaluating a focusing state of a focusing evaluation area in the image, and the evaluation result of the focusing evaluation means. A focus control means for controlling the subject image included in the focus evaluation area to be in a focused state by driving the photographing lens based on the focus lens, and a specific subject based on the evaluation result of the focus evaluation area The processing content changing means for changing the processing content when performing the extraction processing, the subject extracting means for extracting the specific subject from the image based on the processing content, and the subject extracting means Means for setting the area of the specific subject as the focus evaluation area, and the processing content changing means, as the subject image approaches a focused state under the control of the focus control means, It is characterized by increasing the frequency of execution of extraction processing of the serial specific object.
  Claim9The invention described in claim6Thru8In the photographing apparatus according to any one of the above, when the subject extraction unit extracts the specific subject, the photographing operation for performing the main photographing when the region of the specific subject matches the focus evaluation region A means for authorizing.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Hereinafter, a digital camera will be described as an example.
[0019]
<1. First Embodiment>
First, the first embodiment will be described. FIG. 1 is a perspective view showing a digital camera 1 as an example of a photographing apparatus according to the present invention. As shown in FIG. 1, a photographing lens 11 and a finder window 2 are provided on the front side of the digital camera 1. Inside the photographic lens 11, a CCD imaging device 30 is provided as an imaging means for photoelectrically converting a subject image incident through the photographic lens 11 and generating image data (data composed of an array of pixel data for each pixel). It has been.
[0020]
The photographing lens 11 includes a focusing lens that can move along the optical axis direction. By driving the focusing lens, the in-focus state of the image formed on the CCD image sensor 30 can be varied. It is configured as follows.
[0021]
A shutter button 8, a camera status indicator 13, and a setting key 14 are arranged on the upper surface side of the digital camera 1. The shutter button 8 is a button that is pressed by the user when photographing a subject, and is configured to detect two stages of a half-pressed state and a fully-pressed state. The camera status indicator 13 is composed of, for example, a segment display type liquid crystal display, and is provided for showing the user the current setting contents in the digital camera 1. The setting key 14 is a key switch for selecting and setting the shooting mode according to the subject and what kind of focusing control is performed during the automatic focusing control.
[0022]
Further, a mounting portion 15 for mounting a recording medium 9 for recording image data is formed on the side surface portion of the digital camera 1, and a replaceable recording medium 9 such as a memory card can be mounted. .
[0023]
Although not shown in FIG. 1, a liquid crystal display unit for displaying an image taken by the CCD image sensor 30 is provided on the back side of the digital camera 1.
[0024]
FIG. 2 is a block diagram showing the internal configuration of the digital camera 1. In FIG. 2, solid arrows indicate the flow of control signals, and hatched arrows indicate the flow of image signals.
[0025]
As shown in FIG. 2, the digital camera 1 includes a shooting function unit 20 that realizes a shooting function when shooting a subject, and a main subject (specific subject) is detected during focus control, and the main subject is in focus. A main subject focusing control unit 40 that performs control such that the focusing lens 12 included in the photographing lens 11 is driven to change the focusing state of an image formed on the CCD image pickup device 30. And an operation unit 16 including the shutter button 8 and the setting key 14 described above, and an overall control unit 18 that controls the photographing function unit 20 and the main subject focusing control unit 40 according to the operation content of the operation unit 16. Composed.
[0026]
The imaging function unit 20 includes an imaging lens 11, a CCD imaging device 30, an image memory 31 for temporarily storing image data from the CCD imaging device 30, and image compression obtained from the image data obtained from the image memory 31. An image processing unit 32 that performs predetermined image processing, a liquid crystal display unit 33 that displays a display image subjected to image processing for image display in the image processing unit 32, and image processing for image recording in the image processing unit 32 Is provided with a recording medium 9 for recording a photographed image subjected to.
[0027]
Under the control of the overall control unit 18, the shooting function unit 20 controls the live view operation for displaying the live view image on the liquid crystal display unit 33 before the main shooting, and the recording medium 9 when the shutter button 8 is fully pressed. And a main photographing operation for photographing an image for recording. Even if the liquid crystal display unit 33 is in the off state, the photographing operation by the CCD image pickup device 30 is performed at least during focusing control.
[0028]
Further, the main subject focusing control unit 40 fetches image data from the focusing evaluation region setting unit 41 and the image memory 31 for setting a focusing evaluation region for evaluating the focusing state in the entire image, and the focusing evaluation region. A focus evaluation unit 42 for obtaining an evaluation value for evaluating the in-focus state from the partial image data in the focus evaluation area specified by the setting unit 41, and a lens driving unit based on the evaluation value obtained by the focus evaluation unit 42 50, a driving signal is given to the focus evaluation area 43 to bring the subject image in the focus evaluation area into a focused state, and the blur amount of the subject image in the focus evaluation area is determined based on the evaluation value obtained by the focus evaluation section 42. The processing content change unit 44 that changes the setting state in the subject extraction unit 45 in accordance with the amount of blur obtained, the image data is taken from the image memory 31, and the processing content set from the processing content change unit 44 is set. Subject extracting unit 45 for extracting process in the main subject (specific object) you are, configured with a.
[0029]
For example, when the shutter button 8 is half-pressed, the overall control unit 18 causes the main subject focus control unit 40 to function so that the main subject is in an appropriate focus state, and when the shutter button 8 is fully pressed. The photographing function unit 20 is controlled to perform the main photographing operation. In addition, the overall control unit 18 causes the imaging function unit 20 to sequentially capture images as a live view operation before the actual imaging, and displays the sequentially updated images on the liquid crystal display unit 33, or the main subject focusing control unit 40. Control to give to.
[0030]
The lens driving unit 50 moves the focusing lens 12 at a predetermined pitch from the far side to the near side based on the drive signal given from the focusing control unit 43 during the focusing control operation, and forms an image on the CCD image pickup device 30. Change the in-focus state of the image. When the lens position at which the main subject is in focus is specified by the main subject focusing control unit 40 and the lens position is designated by the focusing control unit 43, the lens driving unit 50 designates the lens position. The focusing lens 12 is moved to the lens position, and an image with the main subject in focus is formed on the CCD image sensor 30.
[0031]
FIG. 3 is a diagram showing an initial state of a focus evaluation area preset in the digital camera 1. As shown in FIG. 3, in the focus evaluation area setting unit 41, as the initial state of the focus evaluation area, a focus evaluation area FR1 is set at a substantially central portion of the image G1 acquired from the image memory 31, When the main subject focus control unit 40 functions to perform the first focus evaluation, the focus evaluation region setting unit 41 sets the position and size of the focus evaluation region FR1 shown in FIG. Instruct.
[0032]
The focus evaluation unit 42 acquires the image G1 from the image memory 31, and extracts an image component for the focus evaluation region FR1 therefrom. Then, an evaluation value for evaluating the focus state of the image component in the focus evaluation area FR1 is obtained. There are several methods for calculating the evaluation value. For example, as a first method, there is a method in which an image component included in the focus evaluation region FR1 is passed through a bandpass filter, and the intensity of a high-frequency component of an image signal extracted thereby is used as an evaluation value. As a method, there is a method in which a difference value of luminance values between adjacent pixels of an image component included in the focus evaluation area FR1 is obtained, and a value obtained by accumulating the difference values in the focus evaluation area FR1 is used as an evaluation value. . Further, as a third method, edge extraction processing using a differential filter or the like is performed on the image component included in the focus evaluation area FR1, and the width of the edge extracted as a result (blur width) is used as the evaluation value. There is also a method.
[0033]
In the first and second methods, when the subject is photographed, the level of the high-frequency component is small when the image is blurred, and the cumulative value of the luminance difference between adjacent pixels is small. This is a method for detecting that the level of the high-frequency component increases as it comes in close (approaching the in-focus state), and the cumulative value of the luminance difference between adjacent pixels also increases.
[0034]
For this reason, when the focusing lens 12 is gradually moved from the far side to the near side, the evaluation values obtained by the first and second methods change as shown in FIG. When the evaluation value is obtained from the image captured at each lens position while moving the focusing lens 12, the evaluation value changes as shown in FIG. 4, and the evaluation value shows the maximum value at the lens position PX. Therefore, if the focusing lens 12 is moved to the lens position PX, it is found that the image component included in the focusing evaluation area FR1 is in a focused state, and the focusing control unit 43 moves the focusing lens 12 to the lens position PX. To achieve the in-focus state.
[0035]
On the other hand, the third method is a method of detecting that when the subject is photographed, the edge width increases when the image is blurred, but the edge width decreases as the focus is achieved. Therefore, when the focusing lens 12 is gradually moved from the far side to the near side, the evaluation value obtained by the third method shows the minimum value at the in-focus position, and the focus control unit 43 sets the minimum value. The focusing lens 12 is moved to a lens position indicating the in-focus state.
[0036]
Note that the calculation method of the evaluation value applied in the focus evaluation unit 42 may employ any of the first to third methods, and may employ other calculation methods.
[0037]
By the way, if the focus evaluation area FR1 is set to an area that is always fixed with respect to the image G1, as described above, the focus evaluation area FR1 does not include the main subject intended by the user. Thus, focus detection is performed in a region different from the user's intention, and as a result, the main subject that the user wanted to photograph is photographed as a blurred image.
[0038]
FIG. 5 is a diagram showing a captured image in the case where the person who is the main subject is present at the edge of the image and the focus control is performed in the fixed focus evaluation area. As shown in FIG. 5, when the person who is the main subject is present at the edge of the image, if the focus control is performed in the focus evaluation area at the center of the image, the background portion is in focus, and the main subject The phenomenon of blurring occurs.
[0039]
On the other hand, FIG. 6 is a diagram showing an image that the user intends to shoot when the person who is the main subject is present at the edge of the image. As shown in FIG. 6, even when a person who is the main subject is present at the end of the image, it is desired that the person who is the main subject be photographed in a focused state.
[0040]
Therefore, in the digital camera 1 according to this embodiment, the subject extraction unit 45 acquires an image from the image memory 31 and performs a process of extracting the main subject from the image. Then, the focus evaluation area setting unit 41 changes and sets the focus evaluation area for the image portion determined to be the main subject, and the focus evaluation section 42 sets the evaluation value for the focus evaluation area including the main subject. Perform the calculation. That is, the focus evaluation area is corrected from the initial state area to the image area recognized as the main subject, and focus control is performed so that the subject image of the main subject is in focus.
[0041]
Thus, when the main photographing operation is performed, an appropriate image in which the main subject is in focus is photographed.
[0042]
The subject extraction unit 45 performs subject extraction processing that extracts the skin color component of the image and recognizes the region in which the skin color component is distributed as the main subject region. For this reason, when the main subject is blurred, for example, the color of the background portion affects the face portion of the person, and it is difficult to perform accurate subject extraction processing. Therefore, this embodiment is configured to perform subject extraction processing according to the amount of blur of the image or main subject. In other words, when the image or main subject is blurred, a relatively coarse subject extraction process is performed to efficiently extract the main subject, and when the main subject approaches the in-focus state, a precise subject extraction process is performed. Thus, highly accurate subject extraction processing is performed.
[0043]
The focus evaluation unit 42 gives an evaluation value calculated at each lens position to the processing content changing unit 44, and the processing content changing unit 44 sets the processing content of the subject extraction processing in the subject extraction unit 45 in accordance with the blur amount of the image or the main subject. To decide. Then, the subject extraction unit 45 performs the main subject extraction process from the image obtained from the image memory 31 based on the processing content determined according to the blur amount.
[0044]
As a result, the main subject can be extracted efficiently when the image or the main subject is blurred, and the main subject can be extracted with high accuracy when the main subject approaches the in-focus state.
[0045]
The processing procedure of the digital camera 1 in this embodiment will be described below. 7 to 9 are flowcharts showing the processing procedure of the digital camera 1 in the first embodiment. In particular, in FIGS. 7 and 8, the main photographing operation is performed after the shutter button 8 is pressed halfway. FIG. 9 shows a detailed processing procedure of the subject extraction processing.
[0046]
First, when the user presses the shutter button 8 halfway (step S101), the overall control unit 18 causes the main subject focusing control unit 40 and the photographing function unit 20 to function. At this time, the overall control unit 18 moves the focusing lens 12 to the farthest side by the main subject focusing control unit 40 and controls the photographing function unit 20 to take an image while the focusing lens 12 has moved to the farthest side. To do.
[0047]
Then, the focus evaluation unit 42 takes in the image stored in the image memory 31 (step S102) and performs an evaluation value calculation process (step S103). At this time, the focus evaluation area setting unit 41 determines that the focus evaluation area FR1 in the initial state set in advance in the center portion of the image G1 should be the focus evaluation area to be evaluated. To instruct. The focus evaluation unit 42 calculates an evaluation value based on an image component included in the focus evaluation region FR1 (see FIG. 3) set almost at the center of the screen, and the calculated evaluation value is the focus control unit. 43 and processing content change 44.
[0048]
Then, the processing content change unit 44 and the subject extraction unit 45 function to perform subject extraction processing (step S104).
[0049]
Reference is made to the flowchart of FIG. When the process proceeds to the subject extraction process (step S104), first, the process content changing unit 44 calculates the amount of blur in the focus evaluation area from the evaluation value (step S201). The blur amount is a value that indicates a large value when the image component in the focus evaluation area is blurred, and gradually decreases as the image component in the focus evaluation area approaches the focus state. When calculating the evaluation value, if the above-described first and second methods are adopted, for example, the reciprocal of the evaluation value may be defined as the blur amount, and other values may be defined as the blur amount. Also good. When the third method described above is adopted as the evaluation value calculation method, the calculated evaluation value itself may be used as the blur amount.
[0050]
The subject extraction unit 45 is configured to perform pixel thinning processing and block division processing as pre-processing of the main subject extraction processing, and to perform color determination processing and shape determination processing as actual main subject extraction processing. ing. Then, the process content changing unit 44 changes the pixel thinning number in the pixel thinning process, the block division number in the block division process, and the shape determination reference in the shape determination process according to the amount of blur in the focus evaluation area.
[0051]
First, the processing content changing unit 44 sets a pixel thinning number according to the amount of blur in the focus evaluation area (step S202). FIG. 10 is a diagram illustrating the relationship between the blur amount and the pixel thinning number. As shown in FIG. 10, the relationship between the blur amount and the pixel thinning number is set so that the pixel thinning number increases in proportion to the blur amount. In other words, when the amount of blur is small and the image component in the focus evaluation area is relatively close to the in-focus state, the pixel thinning number is set to a small value, the blur amount is large, and the image component in the focus evaluation area is blurred. In this state, the pixel thinning number is set to a large value.
[0052]
In step S202, the pixel extraction number set by the processing content changing unit 44 is further instructed to the subject extraction unit 45, and the subject extraction unit 45 is obtained from the image memory 31 based on the instructed pixel thinning number. Pixel thinning processing is performed on the image.
[0053]
FIG. 11 is a diagram illustrating the concept of the pixel thinning process. As shown in FIG. 11, when the amount of blur is large, the pixel thinning-out number is set to a large value, so that the number of pixels thinned out from the image G1 increases and an image G2 having a small image size is obtained. On the other hand, when the amount of blur is small (that is, when the state is close to the in-focus state), the pixel thinning-out number is set to a small value, so that the number of pixels thinned out from the image G1 is reduced. An image G3 having a relatively large size is obtained.
[0054]
As described above, the subject extraction unit 45 blurs the image components in the focus evaluation area by changing the image size of the image to be processed before performing the main subject extraction process such as the skin color determination process. It is possible to efficiently extract the main subject when it is in the state, while it is possible to accurately extract the main subject when the image components in the focus evaluation area are close to the focus state. become.
[0055]
Next, the processing content changing unit 44 sets the number of block divisions according to the amount of blur in the focus evaluation area (step S203). FIG. 12 is a diagram illustrating the relationship between the blur amount and the number of block divisions. As shown in FIG. 12, the relationship between the amount of blur and the number of block divisions is set so that the pixel thinning number decreases in proportion to the amount of blur. That is, when the amount of blur is small and the image component in the focus evaluation area is relatively close to the focus state, the block division number is set to a large value, the blur amount is large, and the image component in the focus evaluation area is blurred. If it is in the state, the block division number is set to a small value.
[0056]
In step S203, the subject extraction unit 45 is further instructed by the number of block divisions set by the processing content changing unit 44. Based on the designated number of block divisions, the subject extraction unit 45 performs image size reduction by pixel thinning processing. The block division process is executed on the changed image.
[0057]
13 and 14 are diagrams showing the concept of block division processing. FIG. 13 shows a case where the amount of blur is large, and FIG. 14 shows a case where the amount of blur is small, that is, the image component in the focus evaluation area is in focus. It shows a case close to the state.
[0058]
As shown in FIGS. 13 and 14, when the blur amount is large, the subject extraction unit 45 performs block division processing so that a relatively small number of blocks are generated, and when the blur amount is small (that is, in-focus state). When the state is close to the state), the subject extraction unit 45 performs block division processing so that a relatively large number of blocks are generated.
[0059]
In this manner, the subject extraction unit 45 divides the image to be processed into blocks before performing the processing for extracting the main subject such as the skin color determination processing, so that the processing after the grouping processing (step S207) described later is performed. Thus, it is possible to perform processing in units of blocks in order to improve processing efficiency. In particular, when the image component in the focus evaluation area is blurred, it is possible to efficiently extract the main subject by reducing the number of block divisions, while the image component in the focus evaluation area is When the state is close to the in-focus state, the main subject can be accurately extracted by increasing the number of block divisions.
[0060]
Next, the processing content changing unit 44 sets a shape determination criterion according to the amount of blur in the focus evaluation area (step S204). For example, in this embodiment, the subject extraction unit 45 is configured to determine whether or not the flesh color block aggregate has a shape similar to a human face shape. Specifically, it sets the aspect ratio of the reference shape that is the shape determination reference, determines whether the collection of skin color blocks is within the range of the aspect ratio, and if so, the skin color An aggregate of blocks is specified as the main subject. Therefore, the processing content changing unit 44 sets an aspect ratio as a shape determination reference according to the amount of blur in the focus evaluation area, and instructs the subject extracting unit 45 to set it. Here, the aspect ratio means the ratio of the distribution length in the vertical direction when the distribution length in the horizontal direction of the flesh color block aggregate is 1.
[0061]
FIG. 15 is a diagram illustrating the relationship between the blur amount and the aspect ratio. As shown in FIG. 15, the relationship between the blur amount and the aspect ratio is set so that the aspect ratio increases in proportion to the blur amount. That is, when the amount of blur is small and the image component in the focus evaluation area is relatively close to the focus state, the distribution length in the vertical direction when the horizontal distribution length of the block aggregate is set to 1 The ratio is set to be small, but when the amount of blur is large and the image component in the focus evaluation area is in a blurred state, the distribution length in the horizontal direction of the block aggregate is set to 1. The ratio of the distribution length in the vertical direction is set to be large.
[0062]
When the face shape of a person is specified as the main subject, the minimum aspect ratio is preferably about vertical: horizontal = 1.5: 1, and the maximum value is vertical: horizontal = 2: Preferably, it is about 1. By setting in this way, even when the image component in the focus evaluation area is relatively blurred, the subject extraction unit 45 can perform a certain degree of face shape determination with low accuracy and focus. When the image component of the evaluation region is in a state close to the in-focus state, the face shape can be determined with high accuracy.
[0063]
As described above, changing the aspect ratio in accordance with the amount of blur means changing the reference shape for recognizing the face shape, and reducing the aspect ratio is equivalent to tightening the shape criterion. To do. When the image component in the focus evaluation area is in a state close to the focus state, the main subject can be specified with high accuracy by setting the shape determination criterion strictly.
[0064]
In step S205, the subject extraction unit 45 performs a skin color pixel extraction process for extracting skin color pixels from the pixel-thinned image. In other words, this skin color pixel extraction process is a color determination process. In the skin color pixel extraction process, chromaticities u ′ and v ′ that are not affected by luminance are calculated from the RGB values of the pixel data, and the chromaticities u ′ and v ′ are within a specific area of the “CIE 1976 UCS chromaticity diagram”. The flesh color pixel is extracted depending on whether or not it exists. In addition, when calculating chromaticity u 'and v' from the RGB value of a certain pixel, it can calculate with the following formulas 1 and 2.
[0065]
[Expression 1]

[0066]
[Expression 2]

[0067]
FIG. 16 is a simplified diagram of the “CIE 1976 UCS Chromaticity Diagram”. In FIG. 16, a region RH is a skin color region set in advance by measurement or the like. Therefore, whether or not each pixel is a flesh color pixel can be determined by applying the chromaticities u ′ and v ′ obtained by the above formulas 1 and 2 to the following conditional formula.
[0068]
[Equation 3]

[0069]
[Expression 4]

[0070]
That is, when the chromaticity u ′ calculated by the formula 1 satisfies the conditional formula 3 and the chromaticity v ′ calculated by the formula 2 satisfies the conditional formula 4 Is found to be a skin color pixel. By specifying the skin color pixel using the UCS chromaticity diagram, it becomes possible to specify the skin color pixel accurately without being affected by the luminance value, and the chromaticity u ′, Since the conversion to v ′ is a relatively simple calculation, it is possible to efficiently perform the calculation process.
[0071]
In FIG. 16, the skin color region RH is defined as a rectangular region as an example. For example, a trapezoidal or stepped skin color region is set in the u′v ′ space of the UCS chromaticity diagram. Also good.
[0072]
When this skin color pixel extraction process is performed for each pixel, when the image component in the focus evaluation area is blurred as described above, the image size is small and the number of pixels is relatively small. Extraction processing can be performed. On the other hand, when the image component in the focus evaluation area is close to the focus state, the image size is large and the number of pixels is relatively large, so the number of samples is increased and the subject can be accurately extracted. Become.
[0073]
In step S206, the subject extraction unit 45 performs a flesh color block extraction process. Here, it is determined whether or not the flesh color pixels are included in a predetermined ratio or more for each block divided by the block division processing in step S203. As specified. On the other hand, when the number of skin color pixels included in one block is less than a predetermined ratio, it is determined that the block is not extracted as a skin color block and does not constitute a main subject.
[0074]
By extracting the flesh color block in this way and executing the subsequent processing in units of blocks, the processing speed can be increased as compared with the case where the processing is performed in units of pixels. In particular, in the case of the digital camera 1, when the shutter button 8 is half-pressed, it is required to quickly bring the main subject into the focused state in order not to miss the photo opportunity. Each process has an important significance. At the same time, since a block in which a predetermined proportion of flesh-color pixels exists is used as a flesh-color block, the reliability of the flesh-color block is also improved.
[0075]
In addition, when the image component in the focus evaluation area is in a blurred state, the number of block divisions is relatively small, and as a result, the number of blocks specified as flesh-colored blocks is also small, so that subsequent processing can be performed efficiently. It is. However, if the image component in the focus evaluation area is blurred, it becomes difficult to perform accurate main subject detection processing in the subsequent shape determination processing, and the subject magnification is small (that is, If the main subject appears small in the image), it may not be recognized as a skin color block, and the main subject may not be extracted well.
[0076]
On the other hand, when the image component of the focus evaluation area is in a state close to the focus state, the number of block divisions is relatively large, and as a result, the number of blocks specified as flesh-color blocks also increases. In this process, the main subject can be extracted more accurately. Even when the subject magnification is small (that is, the main subject appears small in the image), since the number of blocks is large, the main subject is covered by a plurality of blocks. And the main subject can be extracted accurately.
[0077]
Next, proceeding to step S207, the subject extraction unit 45 performs a grouping process on the blocks identified as skin color blocks. Grouping is a process in which flesh color blocks distributed continuously in the vertical direction or the horizontal direction are connected to form an aggregate of flesh color blocks. For this reason, when one flesh-color block exists in isolation without touching another flesh-color block, the flesh-color block is excluded from the formation of the flesh-color block aggregate by the grouping process. In other words, by this grouping process, a small flesh color block that cannot be a main subject is treated as not constituting the main subject.
[0078]
Further, here, group information of a collection of flesh color blocks formed by the grouping process is also generated. The group information is information that is generated for each aggregate of skin color blocks and includes the number of blocks in the vertical and horizontal directions constituting the aggregate and the barycentric position of the aggregate. This group information is used in a shape determination process (step S208) and a face identification process (step S209) performed later.
[0079]
Next, proceeding to step S208, the subject extraction unit 45 performs a shape determination process of the flesh color block aggregate formed by the grouping process of step S207. In general, when a skin color group aggregate is obtained from an image, only the skin portion of the person is not extracted. obtain. Human hands and feet can also form a flesh-colored group aggregate. Thus, in this embodiment, the subject extraction unit 45 further performs shape determination processing so that a flesh color group aggregate having a shape distribution close to a human face shape is recognized as a main subject. .
[0080]
The shape determination process here is performed by determining whether or not a flesh color block aggregate is included in the range of the aspect ratio set in step S204. Specifically, the numbers of vertical and horizontal blocks included in the group information generated in step S207 are acquired, and the vertical and horizontal aspect ratios (M: 1) of the flesh color block aggregate are obtained. When the aspect ratio set in step S204 is K: 1, if the relationship of M ≦ K is established, it is determined that the flesh color block aggregate is an image component obtained by photographing a human face portion.
[0081]
Note that the skin color block aggregate distributed in a row in the horizontal direction is also determined to be a human face part only by the relationship of M ≦ K. Therefore, in order to avoid such a situation, the lower limit L is added to the value of M. May be set. That is, if the relationship of L ≦ M ≦ K is established, it may be determined that the flesh color block aggregate is an image component obtained by photographing a human face portion.
[0082]
FIG. 17 is a diagram showing an area recognized as the main subject in the shape determination process (step S208). In FIG. 17, the area recognized as the main subject is shown in black. As illustrated in FIG. 17, the subject extraction unit 45 performs shape determination processing, so that a block including a person's face is identified as a main subject region.
[0083]
Next, proceeding to step S209, when a plurality of main subjects are specified by the shape determination process, the subject extraction unit 45 performs face specification processing, that is, main subject specification processing, in order to specify one of the main subjects. I do. A weighting coefficient for the screen is set in advance in the subject extraction unit 45, and the weighting coefficient for the flesh color block aggregate is identified based on the barycentric position of the flesh color block aggregate identified as the main subject.
[0084]
FIG. 18 is a diagram showing an example of weighting coefficients for the screen. As shown in FIG. 18, the screen G5 is divided into a plurality of areas, and a weighting coefficient for each area is defined for each area. Note that the size of the screen G5 changes according to the pixel thinning number and is adjusted to match the image size of the images G1 and G2 (see FIG. 11) after the pixel thinning processing. Then, the position of the center of gravity of the flesh color block aggregate is specified on the screen G5, and the weighting coefficient is specified based on the specification result. For example, in FIG. 17, considering a black-skinned flesh color block aggregate specified as the main subject, the center of gravity is located at position P1 in FIG. 18, and thus the weighting coefficient is specified as “10”. Then, the subject extraction unit 45 calculates the product of the area of the skin color block aggregate (for example, the number of blocks constituting the skin color block aggregate) and the specified weighting coefficient, and the main subject fitness value of the skin color block aggregate (Value indicating the likelihood of being a main subject). Then, when the main subject suitability value is calculated for each of the plurality of skin color block aggregates, the main subject suitability values are compared, and the skin color block aggregate showing the maximum value is recognized as the face portion of the person who is the main subject. .
[0085]
Thus, when a plurality of main subjects are identified, the main subject fitness value is calculated using the weighting coefficient for the position on the screen on the assumption that the main subject will be framed closer to the center of the screen. By determining, one main subject can be specified accurately. Also, the object that appears larger than the object that appears larger is more likely to be the main subject, so the area of the flesh color block aggregate must be taken into account when performing calculations using the weighting coefficient. Thus, the main subject can be accurately specified.
[0086]
It should be noted that a plurality of main subject adaptation values indicating values within a predetermined range from the main subject adaptation value indicating the maximum value may be employed. In this case, since a plurality of areas are specified as the main subject area, so-called multi-area focusing control is applied.
[0087]
When the main subject is specified as described above, the subject extraction unit 45 instructs the main control unit 18 to specify the main subject region, so that the main subject region is also displayed on the liquid crystal display unit 33. (Step S210). Accordingly, the user can easily grasp which part of the image is recognized as the main subject and the focus control is performed. In addition, the subject extraction unit 45 instructs the focus evaluation region setting unit 41 on the main subject region identified by performing the processing as described above.
[0088]
Returning to the flowchart of FIG. 7, when the subject extraction process in step S104 ends, the process proceeds to step S105, where the focus evaluation area setting unit 41 extracts the focus evaluation area for the focus evaluation unit 42 to calculate the evaluation value. The setting is changed to the subject area designated by the unit 45. As a result, when the next evaluation value calculation is performed in the focus evaluation unit 42, the evaluation value can be calculated for the image component of the main subject.
[0089]
Next, the focus control unit 43 drives the focusing lens 12 at a predetermined pitch (step S106), the image is captured at the lens position (step S107), and the evaluation value by the focus evaluation unit 42 is obtained. A calculation process is performed (step S108). In step S108, the focus evaluation unit 42 calculates an evaluation value based on the image component of the focus evaluation area set in the focus evaluation area change setting process in step S105.
[0090]
Thereafter, the processing of steps S106 to S108 is repeated until the evaluation value shows the maximum value (or minimum value), and when the lens position where the evaluation value shows the maximum value (or minimum value) can be specified, that is, step S109. When the in-focus state of the in-focus evaluation area is realized, the process proceeds to the flowchart of FIG.
[0091]
Then, the subject extraction process is executed again at the stage where the in-focus state of the in-focus evaluation area is realized (step S110). The subject extraction process (step S110) here is also the same as the subject extraction process in step S104 described with reference to FIG.
[0092]
However, in the stage of step S104, the extraction process of the main subject is performed in a state where the image component in the focus evaluation area is blurred, and thus the subject extraction process is performed with a large number of pixel thinning and a small number of block divisions. Also, since the shape criterion is not strict, an area that is efficiently estimated to be the main subject is identified, but its accuracy is low. Therefore, even if the main subject extraction process is performed in a state where the image component of the focus evaluation area is blurred, and the main subject area specified thereby is determined to be in focus, the focus evaluation area does not The main subject may not be included.
[0093]
Therefore, in this embodiment, subject extraction processing is performed again in step S110. In step S110, since it is determined that the image component in the focus evaluation area is in the focused state, the subject extraction process has a small pixel thinning number and a large number of block divisions. An extraction process is performed. In addition, since the shape determination criterion is also strict, the main subject area can be specified with high accuracy.
[0094]
When subject extraction processing (step S110) is performed and a highly reliable main subject region is specified, the subject extraction unit 45 determines the main subject region specified for the focus evaluation region setting unit 41 as in the previous time. Instruct.
[0095]
Thereby, the focus evaluation area setting unit 41 determines whether or not the focus evaluation area used in step S108 matches the subject area specified in the subject extraction process in step S110 (step S111). ). That is, this process is a process for confirming whether or not the focus evaluation area that has been the object of evaluation in the repetitive processes of steps S106 to S108 is an accurate main subject area. As a result, when the focus evaluation area does not match the identified main subject area, the process returns to step S105 (FIG. 7), and the focus evaluation area setting unit 41 sets the focus evaluation area as the subject in step S110. The setting is changed to the main subject area specified by the extraction process. Therefore, when the next evaluation value calculation is performed in the focus evaluation unit 42, it is possible to perform the evaluation value calculation for the image component of the main subject region with high reliability, and the processing after step S106 is performed. . On the other hand, if the focus evaluation area matches the identified main subject area, the process proceeds to step S112 (FIG. 8).
[0096]
When the focus evaluation area setting unit 41 determines that the focus evaluation area matches the identified main subject area, the main control section 18 is permitted to perform the main photographing operation (step S112).
[0097]
The overall control unit 18 first determines whether or not the shutter button 8 has been fully pressed by the user when permission for the main photographing operation is given. Since the digital camera 1 is fully depressed when the shutter button 8 is fully depressed, if the shutter button 8 is not fully depressed, in order to cope with a change in framing, step S102 ( Returning to FIG. 7), the processing after the shutter button 8 is half-pressed is repeated.
[0098]
When the shutter button 8 is fully pressed, the overall control unit 18 controls the shooting function unit 20 to perform the actual shooting operation (step S114), and the image processing unit 32 performs predetermined image processing (step S114). After performing S115), the recording process to the recording medium 9 (step S116) is performed.
[0099]
Thus, the processing procedure of the digital camera 1 from when the shutter button 8 is half-pressed to when the main photographing operation is completed is completed. In the main photographing operation, the main subject exists in any part of the image. Even in this case, it is possible to photograph the main subject in a focused state.
[0100]
In the digital camera 1 of this embodiment, the main subject focus control unit 40 that also functions as a subject extraction device performs a main subject (specific subject) extraction process based on the evaluation result of the focus evaluation area. If the image component in the focus evaluation area is blurred, the processing efficiency is improved. Can be set, and when the image components in the focus evaluation area are in a state close to the in-focus state, the processing content can be set with an emphasis on the reliability of subject extraction. .
[0101]
<2. Second Embodiment>
Next, a second embodiment will be described. In this embodiment, the apparatus configuration is the same as that described in the first embodiment (see FIG. 2 and the like).
[0102]
In the first embodiment, the focus control unit 43 drives the focusing lens 12 stepwise at a predetermined pitch, and the focus evaluation unit 42 calculates the evaluation value at each lens position. The processing is executed before the focusing lens 12 is driven stepwise and after the focusing evaluation area reaches the in-focus state. For this reason, if the subject extraction process executed before the focusing lens 12 is driven stepwise has actually identified a subject that is completely different from the main subject as the main subject, the focus including that subject is included. The operation for leading the evaluation area to the in-focus state is wasted.
[0103]
For this reason, in this embodiment, the subject extraction process is executed even while the focusing lens 12 is moved stepwise, thereby eliminating unnecessary operations and more efficiently focusing the main subject accurately. A form to be led to will be described.
[0104]
19 and 20 are flowcharts showing the processing procedure of the digital camera 1 according to the second embodiment, and in particular, the overall processing from when the shutter button 8 is half-pressed until the actual photographing operation is performed. The procedure is shown.
[0105]
First, when the shutter button 8 is pressed halfway by the user (step S301), the overall control unit 18 causes the main subject focusing control unit 40 and the imaging function unit 20 to function.
[0106]
Then, the focusing control unit 43 controls the lens driving unit 50 to move the focusing lens 12 to the farthest side (step S302). When the process of step S302 is executed after the next time, the focusing control unit 43 performs a control operation for moving the focusing lens 12 by a predetermined pitch in the near side direction.
[0107]
Next, a focus evaluation area setting process is performed (step S303). When the main subject region is designated by the subject extraction unit 45, the focus evaluation region setting unit 41 sets the designated main subject region as the focus evaluation region. However, when the process of step S303 is performed for the first time after the shutter button 8 is half-pressed, the subject extraction process (step S308) is not yet executed. For this reason, when the main subject area has not yet been designated by the subject extraction unit 45, the focus evaluation area setting unit 41 uses the initial focus evaluation area FR1 (see FIG. 3) set in advance in the center of the image. It is set as a focus evaluation area for evaluation value calculation. On the other hand, when the subject extraction process has already been performed several times, the focus evaluation area setting unit 41 sets the latest main subject area designated last time as the focus evaluation area to be evaluated. . Then, the focus evaluation area setting unit 41 instructs the focus evaluation part 42 about the focus evaluation area.
[0108]
If a mode for shooting a person such as a snap mode or a portrait mode is selected and set as the shooting mode, subject extraction processing is performed when the shutter button 8 is half-pressed. You may make it go once. With such a processing procedure, when performing the focus evaluation area setting process in step S303, it is possible to always set the area estimated as the main subject as the focus evaluation area.
[0109]
Then, the overall control unit 18 causes the photographing function unit 20 to perform a photographing operation when the focusing lens 12 is at the current lens position. Then, the focus evaluation unit 42 takes in the image stored in the image memory 31 (step S304) and performs evaluation value calculation processing (step S305). At this time, the focus evaluation unit 42 calculates an evaluation value based on the image component of the focus evaluation region specified from the focus evaluation region setting unit 41 in step S303, and the evaluation value calculated as a result is the focus value. This is given to the focus control unit 43 and the processing content change 44.
[0110]
Then, the processing content changing unit 44 determines whether or not to execute subject extraction processing (step S306). In this embodiment, when the focusing lens 12 is moved stepwise at a predetermined pitch, the subject extraction process can be executed at each lens position, but every time the lens is driven at the predetermined pitch. If the subject extraction processing is executed, it takes a long time to lead the main subject to the focused state. Therefore, the processing content changing unit 44 of this embodiment has not only a function of changing the processing content in the subject extraction unit 45 but also a function of adjusting the execution frequency of the subject extraction processing in the subject extraction unit 45.
[0111]
There are several methods for the processing content changing unit 44 to adjust the execution frequency of the subject extraction processing. First, there is a method in which the subject extraction process is executed once every time the lens driving of the focusing lens 12 at a predetermined pitch is performed a predetermined number of times. Accordingly, it is possible to avoid subject extraction processing from being performed every time lens driving is performed, and thus an efficient focusing operation can be realized.
[0112]
Second, there is a method of increasing the execution frequency of the subject extraction process as the image component in the focus evaluation area approaches the focus state by taking into account the evaluation value obtained by the focus evaluation unit 42. For example, when the evaluation value indicates the maximum value at the in-focus position, when the evaluation value is less than the first threshold value, the subject extraction process is executed once every time the lens is driven ten times, and the first threshold value is set. When the value is less than the second threshold, the subject extraction process is executed once every time the lens is driven five times. When the value is equal to or greater than the second threshold, the subject extraction process is performed once every time the lens is driven twice. The execution frequency of the subject extraction process is adjusted on the basis of the evaluation result in the focus evaluation unit 42 so as to execute. With this configuration, the processing for extracting the main subject increases as the image component in the focus evaluation area approaches the focused state. When the image component in the focus evaluation area is close to the focus state, the main subject is identified with relatively high accuracy, so this second method is effective for leading the main subject to the focus state. Method.
[0113]
However, the method by which the processing content changing unit 44 adjusts the execution frequency of the subject extraction processing is not limited to the first or second method, and other methods may be adopted. Further, when the processing speed in the main subject focusing control unit 40 is sufficiently high, the subject extraction process may be executed at the time of image capture every time after driving the lens.
[0114]
If the processing content changing unit 44 determines that the subject extraction process is to be executed by counting the number of lens driving operations, the process proceeds to the subject extraction process in step S308. On the other hand, if it is determined that the subject extraction process is not to be executed, the process proceeds to a process of skipping the subject extraction process (step S308) and performing the focus determination, that is, step S310 (the flowchart of FIG. 20).
[0115]
Also in this embodiment, the details of the subject extraction process (step S308) are the same as those in the flowchart shown in FIG. Accordingly, when the focus evaluation area is in a blurred state, subject extraction processing is performed with emphasis on processing efficiency, and the reliability of the main subject specified by the subject extraction processing as the focus evaluation region approaches the focus state is improved. Will improve.
[0116]
When the main subject region is specified in the subject extraction process (step S308), the main subject region is transmitted from the subject extraction unit 45 to the focus evaluation region setting unit 41.
[0117]
In step S309, the focus evaluation area setting unit 41 compares the focus evaluation area so far with the main subject area specified by the subject extraction process, and determines whether or not they match. This process is a process for confirming whether or not the focus evaluation area that has been the object of evaluation so far is an accurate main subject area. If they do not match, the process returns to step S303, and the focus evaluation area setting unit 41 changes the setting of the subsequent focus evaluation area to the main subject area specified by the subject extraction process in step S308. Therefore, when the next evaluation value calculation is performed in the focus evaluation unit 42, it is possible to calculate the evaluation value for the image component of the main subject region with higher reliability than the previous time. On the other hand, if the focus evaluation area matches the identified main subject area, the process proceeds to step S310 (FIG. 20).
[0118]
Then, the focus control unit 43 determines whether or not the lens position PX having the maximum (or minimum) evaluation value as shown in FIG. 4 has been identified, that is, the focus state of the focus evaluation area is determined. Determine whether it has been realized. Here, if it is determined that the in-focus state of the in-focus evaluation region has not yet been realized, the process returns to step S302 (FIG. 19) to repeat lens driving at a predetermined pitch, evaluation value calculation processing, and the like. As a result, when the in-focus state of the in-focus evaluation area is finally realized, the process proceeds to step S311 to permit the main control unit 18 to perform the main photographing operation.
[0119]
When the main photographing operation is permitted, the overall control unit 18 first determines whether or not the shutter button 8 has been fully pressed by the user (step S312). If the shutter button 8 is not fully pressed, the processing after the shutter button 8 is half-pressed is repeated by returning to step S302 (FIG. 19) in order to cope with the case where the framing is changed. At this time, since the focus control unit 43 needs to perform focus control from the beginning, the focusing lens 12 is moved to the farthest position.
[0120]
On the other hand, when the shutter button 8 is fully pressed, the overall control unit 18 controls the imaging function unit 20 to perform the actual imaging operation (step S313), and the image processing unit 32 performs predetermined image processing. After performing (Step S314), the recording process to the recording medium 9 (Step S315) is performed.
[0121]
As described above, in the digital camera 1 according to the second embodiment, the processing procedure from when the shutter button 8 is pressed halfway down until the main shooting operation is completed is completed. In any part of the image, the main subject can be photographed in a focused state.
[0122]
Also in the digital camera 1 of this embodiment, the main subject focus control unit 40 that also functions as a subject extraction device performs a main subject (specific subject) extraction process based on the evaluation result of the focus evaluation area. If the image component in the focus evaluation area is blurred, the processing efficiency is improved. Can be set, and when the image components in the focus evaluation area are in a state close to the in-focus state, the processing content can be set with an emphasis on the reliability of subject extraction. . As a result, it is possible to efficiently and accurately extract the main subject and realize the focused state of the main subject.
[0123]
Further, in the digital camera 1 of this embodiment, since the subject extraction process is executed even in the lens driving stage for performing the focusing operation, the focusing operation is performed with a portion that is not the main subject as an evaluation target. Since the progress is avoided, a more efficient focusing operation is realized.
[0124]
Furthermore, the main subject can be extracted more accurately by configuring the frequency of subject extraction processing to increase as the image component in the focus evaluation area approaches the focused state, and its reliability is high. The main subject can be quickly brought into focus.
[0125]
<3. Modification>
Although the embodiments of the present invention have been described above, the present invention is not limited to the contents described above.
[0126]
For example, the digital camera in the above description includes both a digital still camera and a digital video camera. Although the digital camera 1 has been described as an example of the photographing apparatus, the present invention can also be applied to photographing apparatuses other than the digital camera.
[0127]
In the above description, the focus control such as the so-called hill-climbing method is described. However, the auto focus detection method called the active method or the phase difference method, and the distance measurement is performed in a plurality of distance measuring areas. The present invention can be applied even to a control method such as an automatic focus detection method.
[0128]
In the above description, the case where the timing for starting the focusing operation is after halfway pressing of the shutter button 8 is described. However, the present invention is not limited to this, for example, a state in which a live view image is displayed Also, the above-described focusing operation may be performed.
[0129]
In the above description, the pixel thinning number, the number of block divisions, and the shape determination criterion, and the blur amount in the focus evaluation area indicate a continuous relationship. Each of the pixel thinning number, the block division number, and the shape determination criterion may be changed in a stepwise manner.
[0130]
In the above description, the case where the main subject (that is, the specific subject) is a human face portion has been described. This is because, in general, when taking a picture, a person is often taken as a subject. However, the essence of the present invention is that the main subject is limited to the face of a person. It is not something. For this reason, if the user can freely set and input information on the color components, shapes, etc. constituting the main subject, the main subject intended by the user can be appropriately specified. In addition, when a mode in which a person is a subject, such as a snap mode or a portrait mode, is selected as a shooting mode, the process for extracting the face portion of a person as a main subject as described above is performed. It is also possible to configure. Conversely, it is possible to determine that the subject is a landscape by extracting the face portion of the person, or to determine that it is a snap or a portrait by the area of the skin color block aggregate of the main subject, Auto scene discrimination can be performed.
[0131]
Furthermore, in the above description, the case where the subject extraction process is performed to guide the main subject to the in-focus state has been described, but in view of the fact that the main subject region is specified from the image according to the result of the subject extraction process, The subject extraction processing can be executed not only for automatic focusing control but also for performing special image processing on the image components of the main subject region. That is, the subject extraction technique described above is not limited to the one applied to guide the main subject to the focused state.
[0132]
【The invention's effect】
  As described above, according to the first aspect of the present invention, the focus evaluation areaAs the subject image included in theSpecific subject extraction processingDecrease the number of thinning out pixels inTherefore, when the image component in the focus evaluation area is blurred, it is possible to set the processing content that emphasizes the processing efficiency, and the image component in the focus evaluation area is in the in-focus state. If it is in a close state, it is possible to set processing contents that place importance on the reliability of the extraction of the specific subject.Further, the processing content of the specific subject extraction processing can be changed according to the amount of blur in the focus evaluation area. Then, when the focus evaluation area approaches the focused state, the number of samples for extracting the specific subject increases, so that the specific subject can be extracted with high accuracy. On the other hand, when the focus evaluation area is blurred, the number of samples is small, so that a specific subject can be extracted quickly.As a result, it is possible to extract a specific subject efficiently and accurately.
[0133]
  According to invention of Claim 2,Since the subject image included in the focus evaluation area is configured to increase the number of block divisions in the block division process in the extraction process of the specific subject as the subject image approaches the in-focus state, the image component of the focus evaluation area When the image is blurred, it is possible to set the processing content with an emphasis on the processing efficiency, and when the image component in the focus evaluation area is close to the focus state, the reliability of the extraction of the specific subject It is possible to set processing contents that place emphasis on. Further, the processing content of the specific subject extraction processing can be changed according to the amount of blur in the focus evaluation area. Then, when the focus evaluation area approaches the focused state, the number of blocks for extracting the specific subject increases, so that the specific subject can be extracted with high accuracy. On the other hand, when the focus evaluation area is blurred, the number of blocks is small, so that a specific subject can be extracted quickly. As a result, it is possible to extract a specific subject efficiently and accurately.
[0135]
  Claim3According to the invention described in the above, when the subject image included in the focus evaluation area approaches the in-focus state, the frequency of executing the extraction process of the specific subject is increased, so that the image component in the focus evaluation area is blurred Can set processing content that emphasizes processing efficiency, and when the image component in the focus evaluation area is in a state close to the in-focus state, processing content that emphasizes the reliability of extraction of a specific subject Can be set. That is, as the in-focus state is approached, it is possible to extract a specific subject with high accuracy. As a result, it is possible to extract a specific subject efficiently and accurately.
[0136]
  Claim4Since the specific subject is extracted by performing skin color discrimination of the image, it is possible to extract the person image as the specific subject when the person is the subject.
[0137]
  Claim5According to the invention described above, when the specific subject region extracted by the specific subject extraction process and the focus evaluation region do not match, the focus evaluation region is configured to be changed to the specific subject region. Therefore, it is possible to appropriately evaluate the focused state of the specific subject.
[0138]
  Claim6According to the invention described in the above, the processing content when performing the extraction processing of the specific subject is changed so that the thinning-out number of the pixel thinning processing is decreased as the subject image included in the focusing evaluation area approaches the in-focus state. In addition, since the specific subject is extracted from the image based on the changed processing content and the region of the specific subject extracted thereby is set as the focus evaluation region, the focus evaluation is performed. When the image component of the area is blurred, the processing content with an emphasis on processing efficiency can be set, and when the image component of the focus evaluation area is close to the focused state, the specific subject It is possible to set processing contents that emphasize extraction reliability. As a result, the specific subject can be identified efficiently and accurately, and the specific subject can be brought into a focused state.
[0139]
  Claim7According to the invention described in the above, the process when performing the extraction process of the specific subject so that the number of block divisions in the block division process increases as the subject image included in the focus evaluation area approaches the in-focus state. Since the content is changed, the specific subject is extracted from the image based on the changed processing content, and the region of the specific subject extracted thereby is set as the focus evaluation region. When the image component in the focus evaluation area is blurred, it is possible to set processing details that emphasize processing efficiency, and when the image component in the focus evaluation area is close to the focus state Therefore, it is possible to set processing contents that place importance on the reliability of extraction of a specific subject. As a result, the specific subject can be identified efficiently and accurately, and the specific subject can be brought into a focused state.
[0141]
  Claim8According to the invention described in the above, when the subject extraction process is performed so that the frequency of the subject extraction process increases as the subject image included in the focus evaluation area approaches the in-focus state. Since the processing content is changed, the specific subject is extracted from the image based on the changed processing content, and the region of the specific subject extracted thereby is set as the focus evaluation region When the image component in the focus evaluation area is blurred, it is possible to set the processing content with an emphasis on processing efficiency, and when the image component in the focus evaluation area is in a state close to the focus state. Can set processing contents that place importance on the reliability of extraction of a specific subject. As a result, the specific subject can be identified efficiently and accurately, and the specific subject can be brought into a focused state.
  Claim9According to the above-described invention, when a specific subject is extracted, if the region of the specific subject and the focus evaluation region match, the shooting operation for performing the main shooting is permitted. It is possible to appropriately perform the main photographing operation in a state where the state is evaluated.
[Brief description of the drawings]
FIG. 1 is a perspective view showing a digital camera as an example of a photographing apparatus.
FIG. 2 is a block diagram showing an internal configuration of the digital camera.
FIG. 3 is a diagram illustrating an initial state of a focus evaluation area set in advance in the digital camera.
FIG. 4 is a diagram illustrating an example of an evaluation value change with respect to a lens position.
FIG. 5 is a diagram showing a captured image when focus control is performed in a fixed focus evaluation area.
FIG. 6 is a diagram illustrating an image that the user intends to shoot.
FIG. 7 is a flowchart illustrating a processing procedure of the digital camera according to the first embodiment.
FIG. 8 is a flowchart illustrating a processing procedure of the digital camera according to the first embodiment.
FIG. 9 is a flowchart showing a detailed processing procedure of subject extraction processing;
FIG. 10 is a diagram illustrating a relationship between a blur amount and a pixel thinning number.
FIG. 11 is a diagram illustrating a concept of pixel thinning processing.
FIG. 12 is a diagram illustrating a relationship between a blur amount and the number of block divisions.
FIG. 13 is a diagram illustrating a concept of block division processing when the amount of blur is large.
FIG. 14 is a diagram illustrating a concept of block division processing when the amount of blur is small.
FIG. 15 is a diagram illustrating a relationship between a blur amount and an aspect ratio as a shape determination criterion.
FIG. 16 is a simplified diagram of a CIE 1976 UCS chromaticity diagram.
FIG. 17 is a diagram illustrating an area recognized as a main subject in the shape determination process.
FIG. 18 is a diagram illustrating an example of a weighting coefficient for a screen.
FIG. 19 is a flowchart illustrating a processing procedure of the digital camera according to the second embodiment.
FIG. 20 is a flowchart illustrating a processing procedure of the digital camera according to the second embodiment.
[Explanation of symbols]
1 Digital camera (photographing device)
8 Shutter button
11 Shooting lens
12 Focusing lens
18 Overall control unit
20 Shooting function section
30 CCD image sensor
31 Image memory
40 Main subject focus control unit (Subject extraction device)
41 Focus evaluation area setting section
42 Focus evaluation section
43 Focus control unit
44 Processing content change section
45 Subject extraction unit
50 Lens drive unit

Claims (9)

A subject extraction device that detects a specific subject from an image,
In-focus evaluation means for evaluating the in-focus state of the in-focus evaluation area in the image;
Focus control means for controlling the subject image included in the focus evaluation area to be in focus based on the evaluation result of the focus evaluation means;
Processing content changing means for changing the processing content when performing extraction processing of the specific subject based on the evaluation result of the focus evaluation area;
Subject extraction means for extracting the specific subject from the image based on the processing content;
With
The subject extraction means is configured to perform pixel thinning processing in the extraction processing of the specific subject,
The subject extraction device according to claim 1, wherein the processing content changing means reduces the thinning-out number in the pixel thinning-out process as the subject image approaches the in-focus state under the control of the focusing control means. A subject extraction device that detects a specific subject from an image,
In-focus evaluation means for evaluating the in-focus state of the in-focus evaluation area in the image;
Focus control means for controlling the subject image included in the focus evaluation area to be in focus based on the evaluation result of the focus evaluation means;
Processing content changing means for changing the processing content when performing extraction processing of the specific subject based on the evaluation result of the focus evaluation area;
Subject extraction means for extracting the specific subject from the image based on the processing content;
With
The subject extraction means is configured to perform block division processing in the extraction processing of the specific subject,
The subject extraction device characterized in that the processing content changing means increases the number of block divisions in the block division processing as the subject image approaches the in-focus state under the control of the focusing control means. A subject extraction device that detects a specific subject from an image,
In-focus evaluation means for evaluating the in-focus state of the in-focus evaluation area in the image;
Focus control means for controlling the subject image included in the focus evaluation area to be in focus based on the evaluation result of the focus evaluation means;
Processing content changing means for changing the processing content when performing extraction processing of the specific subject based on the evaluation result of the focus evaluation area;
Subject extraction means for extracting the specific subject from the image based on the processing content;
Equipped with a,
Pre Symbol processing content changing means, as the object image under the control of the focusing control means approaches the focus state, the object extracting apparatus characterized by increasing the frequency of execution of extraction processing of the specific object. Subject extraction apparatus odor according to any one of claims 1 to 3 Te,
The subject extraction device, wherein the subject extraction unit extracts the specific subject by performing skin color discrimination of the image in color determination processing . The subject extraction device according to any one of claims 1 to 4,
Means for changing the setting of the focus evaluation area to the area of the specific subject when the area of the specific subject extracted by the subject extraction means does not match the focus evaluation area;
Further comprising Ru subject extraction device. A photographing device for photographing an image through a photographing lens ,
In-focus evaluation means for evaluating the in-focus state of the in-focus evaluation area in the image;
Focus control means for controlling the subject image included in the focus evaluation area to be in focus by driving the photographing lens based on the evaluation result of the focus evaluation means;
Processing content changing means for changing processing content when performing extraction processing of a specific subject based on the evaluation result of the focus evaluation area;
Subject extraction means for extracting the specific subject from the image based on the processing content;
Means for setting the area of the specific subject extracted by the subject extraction means as the focus evaluation area;
With
The subject extraction means is configured to perform pixel thinning processing in the extraction processing of the specific subject,
The imaging apparatus according to claim 1, wherein the processing content changing unit decreases a thinning number in the pixel thinning process as the subject image approaches a focused state by the control of the focusing control unit . A photographing device for photographing an image through a photographing lens,
In-focus evaluation means for evaluating the in-focus state of the in-focus evaluation area in the image;
Focus control means for controlling the subject image included in the focus evaluation area to be in focus by driving the photographing lens based on the evaluation result of the focus evaluation means;
Processing content changing means for changing processing content when performing extraction processing of a specific subject based on the evaluation result of the focus evaluation area;
Subject extraction means for extracting the specific subject from the image based on the processing content;
Means for setting the area of the specific subject extracted by the subject extraction means as the focus evaluation area;
With
The subject extraction means is configured to perform block division processing in the extraction processing of the specific subject,
The processing apparatus according to claim 1, wherein the processing content changing unit increases the number of block divisions in the block division processing as the subject image approaches the in-focus state under the control of the focusing control unit. A photographing device for photographing an image through a photographing lens,
In-focus evaluation means for evaluating the in-focus state of the in-focus evaluation area in the image;
Focus control means for controlling the subject image included in the focus evaluation area to be in focus by driving the photographing lens based on the evaluation result of the focus evaluation means;
Processing content changing means for changing processing content when performing extraction processing of a specific subject based on the evaluation result of the focus evaluation area;
Subject extraction means for extracting the specific subject from the image based on the processing content;
Means for setting the area of the specific subject extracted by the subject extraction means as the focus evaluation area;
Equipped with a,
Pre Symbol processing content changing means, wherein as the object image under the control of the focus control means approaches the focus state, the photographing apparatus characterized by increasing the frequency of execution of extraction processing of the specific object. Te imaging apparatus smell as claimed in any one of claims 6 to 8,
Means for permitting a photographing operation for performing actual photographing when the subject extraction means extracts the specific subject and the region of the specific subject and the focus evaluation region match;
An imaging device further comprising:

Images