JP4153108B2 - Image processing method, image processing apparatus, and recording medium - Google Patents

Description

但し、ｉは各顔候補領域を識別するための符号、Ｎは顔候補領域の総数、Ｍ_i は顔候補領域ｉの濃度、Ｐ_i は顔候補領域ｉの重み点数、Ｓ_i は顔候補領域ｉの面積である。
【００９３】
（１）式及び（２）式より明らかなように、顔領域濃度Ｍfaceは各顔候補領域の濃度Ｍの加重平均値であり、（１）式では各顔候補領域の重み点数Ｐに基づいて各顔候補領域を重み付けしており、（２）式では重み点数Ｐ及び面積Ｓに基づいて各顔候補領域を重み付けしている。
【００９４】
上記の顔領域抽出・濃度演算処理を行うと、オートセットアップエンジン４４は、更に、イメージプロセッサ４０で実行される各種の画像処理の処理条件を演算するが、顔領域抽出・濃度演算処理の処理結果は一部の画像処理の処理条件の演算に利用される。例えば先のステップ１４６で抽出された顔領域は、イメージプロセッサ４０で実行される顔領域又はその一部のみを対象とした画像処理（例えば顔領域に対するシャープネス補正や赤目補正等）の演算に利用され、前記画像処理が顔領域のみを対象として行われるように処理条件が設定される。また、先のステップ１４８で演算された顔領域濃度Ｍfaceは、例えばイメージプロセッサ４０で実行される画像全体を対象とした画像処理（例えば色・濃度補正等）に利用され、例えば顔領域濃度Ｍfaceが所定濃度になるように濃度補正条件等の処理条件が演算される。
【００９５】
先にも説明したように、顔領域の抽出及び顔領域濃度Ｍfaceはマッチング用パターンとの一致度に基づいて設定した重み点数を用いて行っているので、顔候補領域抽出処理によって抽出された顔候補領域の中に、実際には顔領域でない顔候補領域が誤抽出により混在していたとしても、実際には顔領域でない顔候補領域が顔領域として抽出される確率が大幅に低減されると共に、実際には顔領域でない顔候補領域の濃度によって顔領域濃度が大幅に変化することも防止することができる。従って、顔領域の抽出結果又は顔領域濃度Ｍfaceを利用して処理条件が演算される各画像処理に対しても適正な処理条件が得られ、ファインスキャン画像データを対象としてイメージプロセッサ４０で実行される各画像処理についても適正な処理結果が得られる。
【００９６】
〔第２実施形態〕
次に本発明の第２実施形態について説明する。なお、本第２実施形態は第１実施形態と同一の構成であるので、各部分に同一の符号を付して構成の説明を省略し、以下、本第２実施形態に係る顔領域抽出・濃度演算処理について、図９のフローチャートを参照し、第１実施形態と異なる部分についてのみ説明する。
【００９７】
なお、本第２実施形態に係る顔領域抽出・濃度演算処理は、請求項３の発明に係る画像処理方法が適用された処理であり、オートセットアップエンジン４４のＣＰＵ４６により、顔領域抽出・濃度補正プログラムが実行されることにより実現される。顔領域抽出・濃度補正プログラムは、当初は、情報記憶媒体７２（図１参照）に記憶されており、パーソナルコンピュータ４２に接続された情報読出装置（図示省略）に情報記憶媒体７２が装填され、情報記憶媒体７２から画像処理装置１４へのプログラムの移入が指示されると、情報読出装置によって情報記憶媒体７２から顔領域抽出・濃度補正プログラム等が読み出され、記憶内容を書換え可能なＲＯＭ５０に記憶される。そして顔領域抽出・濃度補正処理を実行すべきタイミングが到来すると、ＲＯＭ５０から顔領域抽出・濃度補正プログラムが読み出され、顔領域抽出・濃度補正プログラムがＣＰＵ４６によって実行される。これにより、オートセットアップエンジン４４は請求項７の発明に係る画像処理装置として機能する。このように、本第２実施形態に係る情報記憶媒体７２は請求項９に記載の記録媒体に対応している。
【００９８】
本第２実施形態に係る顔領域抽出・濃度演算処理では、顔候補領域抽出処理を行い（ステップ１００）、天地情報の有無を判定し（ステップ１０２）、天地情報が有る場合にはステップ２００へ移行し、顔候補領域抽出処理によって抽出された顔候補領域の中から単一の顔候補領域のデータを取り込む。ステップ２０２では、ステップ２００でデータを取り込んだ顔候補領域に対し、高濃度方向に凸の濃度パターンの領域及び低濃度方向（高輝度方向）に凸の濃度パターンの領域が存在しているか否かを探索する。なお、ステップ２０２は請求項７に記載の第２の抽出手段に対応している。
【００９９】
人物の顔に相当する顔領域のうち濃度が所定値以上の領域（高濃度領域）は、例として図１０（Ａ）に示すように、眼部及びその周囲に相当する領域（眼部領域）に存在しており、眼部領域内における濃度変化は、図１０（Ｂ）に示すように急峻な傾きで高濃度方向に凸の変化パターンとなっている。なお、眼部領域には眼球、瞼、睫毛、眉毛等が近接した位置に存在していることから、眼部領域における実際の濃度変化はより複雑であるが、人物の顔のうち眼部及びその周辺は窪んでいるので、眼部領域内のおおよその濃度変化は、図１０（Ａ）に示すような変化パターンを示す。
【０１００】
また、人物の顔に相当する顔領域のうち輝度が所定値以上の領域（高輝度領域）は、例として図１１（Ａ）に示すように、頬部に相当する領域（頬部領域）や鼻に相当する領域、額に相当する領域に存在しており、例えば頬部領域内における輝度変化は、図１１（Ｂ）に示すように緩やかな傾きで高輝度方向に凸の変化パターンとなっている。従って、処理対象の顔候補領域が人物の顔に相当する顔領域である場合には、ステップ２０２の処理により、高濃度方向に凸の濃度パターンの領域として眼部領域が抽出され、低濃度方向に凸の濃度パターンの領域として頬部領域が抽出されることになる。
【０１０１】
次のステップ２０４では、ステップ２０２の探索により濃度パターンが発見されたか否か判定する。判定が否定された場合には、処理対象の顔候補領域は顔領域でない可能性が非常に高いので、ステップ２０６で顔領域としての整合度に０を代入してステップ２１０へ移行する。一方、ステップ２０４の判定が肯定された場合にはステップ２０８へ移行し、天地情報が表す天地方向を基準として、ステップ２０２で発見・抽出した濃度パターン領域の顔領域内における位置及び面積（濃度パターン領域についての濃度ヒストグラムの形状や、濃度パターン領域内の濃度の変化の仕方（例えば濃度変化の傾きや濃度パターンにおける山の高さとすそ野の広さとの比等）も含めてもよい）に基づいて、抽出した全ての濃度パターン領域について眼部及び頬部に相当する領域としての整合度を判定する。
【０１０２】
すなわち、処理対象の顔候補領域が顔領域である場合、眼部領域に相当する高濃度方向に凸の濃度パターン領域が例えば図４（Ｂ）にハッチングで示すブロック対内に各々存在しており、前記濃度パターン領域と処理対象の顔候補領域との面積比も、眼部領域としての面積比に相当する所定の数値範囲内である可能性が高い。従って、高濃度方向に凸の濃度パターン領域の眼部に相当する領域としての整合度は、例えば天地情報が表す天地方向を基準にして眼部領域が存在しているべき範囲（図４（Ｂ）にハッチングで示すブロック対）を設定し、設定した範囲と前記濃度パターン領域の位置との一致度を求めると共に、前記濃度パターン領域と処理対象の顔候補領域との面積比を、眼部領域としての面積比に相当する所定の数値範囲と比較して一致度を求め、双方の一致度に基づき２次元のマップ等を用いて判定することができる。
【０１０３】
また、処理対象の顔候補領域が顔領域である場合、頬部領域に相当する低濃度方向（高輝度方向）に凸の濃度パターン領域が、例えば図４（Ｂ）にハッチングで示すブロックの下方に隣接するブロック対内に各々存在しており、前記濃度パターン領域と処理対象の顔候補領域との面積比も、頬部領域としての面積比に相当する所定の数値範囲内である可能性が高い。従って、低濃度方向に凸の濃度パターン領域の頬部に相当する領域としての整合度についても、例えば天地情報が表す天地方向を基準にして頬部領域が存在しているべき範囲（図４（Ｂ）にハッチングで示すブロックの下方に隣接するブロック対）を設定し、設定した範囲と前記濃度パターン領域の位置との一致度を求めると共に、前記濃度パターン領域と処理対象の顔候補領域との面積比を、頬部領域としての面積比に相当する所定の数値範囲と比較して一致度を求め、双方の一致度に基づき２次元のマップ等を用いて判定することができる。
【０１０４】
そしてステップ２０９では、抽出した各濃度パターン領域に対して判定した整合度に基づいて、処理対象の顔候補領域に対して顔領域としての整合度を演算・設定し、ステップ２１０へ移行する。なお、顔領域の整合度としては、例えば各濃度パターン領域毎の整合度の総合計等を用いることができる。
【０１０５】
なお、上記では抽出した濃度パターン領域に対して眼部又は頬部に相当する領域としての整合度を判定しているが、特に低濃度方向に凸の濃度パターン領域に対しては、鼻や額に相当する領域としての整合度も併せて判定するようにしてもよい。但し、眼部及び頬部が顔領域の略左右対称な位置に一対存在しているのに対し、鼻や額は顔領域の左右方向に沿った略中央に１個のみ存在しているので、特定の濃度パターン領域について鼻や額に相当する領域としての整合度が高くなったとしても、該濃度パターン領域が実際には鼻や額に相当する領域ではなく、偶然に整合度が高くなる場合もあり、眼部や頬部と比較して整合度判定の信頼性は若干低い。このため、顔候補領域に対する顔領域としての整合度の設定にあたっては、鼻や額に相当する領域としての整合度は、低い重みで顔領域としての整合度に反映されるように設定することが望ましい。
【０１０６】
次のステップ２１０では全ての顔候補領域について処理（整合度の判定）を行ったか否か判定する。判定が否定された場合にはステップ２００に戻り、各顔候補領域に対してステップ２００〜２０８の処理・判定を各々行う。これにより、各顔候補領域に対して顔領域としての整合度が各々判定・設定されることになる。ステップ２１０の判定が肯定されるとステップ２４４へ移行する。
【０１０７】
一方、ステップ１０２の判定が否定された場合（天地情報が無かった場合）にはステップ２２０へ移行し、ステップ１００の顔候補領域抽出処理によって抽出された顔候補領域の中から単一の顔候補領域のデータを取り込んだ後に、次のステップ２２２において、先のステップ２０２と同様に、高濃度方向に凸の濃度パターンの領域及び低濃度方向（高輝度方向）に凸の濃度パターンの領域が存在しているか否かを探索する。このステップ２２２も請求項７に記載の第２の抽出手段に対応している。
【０１０８】
ステップ２２４では、ステップ２２２の探索により濃度パターンが発見されたか否か判定する。判定が否定された場合には、処理対象の顔候補領域は顔領域でない可能性が非常に高いので、ステップ２２６で顔領域としての整合度に０を代入してステップ２３８へ移行する。
【０１０９】
一方、ステップ２２４の判定が肯定された場合にはステップ２２８へ移行して変数ｎに１を代入し、次のステップ２３０では、第ｎ方向（この場合は第１方向）を基準（天地方向）として、ステップ２２２で発見・抽出した濃度パターン領域の顔領域内における位置及び面積に基づいて、抽出した全ての濃度パターン領域について、眼部に相当する領域としての整合度及び頬部に相当する領域としての整合度を判定する。そして次のステップ２３１では、抽出した各濃度パターン領域に対して判定した整合度に基づいて、先のステップ２０９と同様にして処理対象の顔候補領域に対して顔領域としての整合度を演算・設定する。
【０１１０】
ステップ２３２では変数ｎの値が４になったか否か判定し、判定が否定された場合にはステップ２３４で変数ｎの値を１だけインクリメントしてステップ２３０へ戻る。これにより、ステップ２３０、２３１では、第１〜第４方向を各々天地方向と仮定して、処理対象の顔候補領域に対し、顔領域としての整合度が各々判定されることになる。
【０１１１】
各方向について整合度が各々判定されると、ステップ２３２の判定が肯定されてステップ２３６へ移行し、第１〜第４方向のうち整合度が最大となった方向を記憶する。次のステップ２３８では、全ての顔候補領域に対して処理を行ったか否か判定する。判定が否定された場合にはステップ２２０に戻り、ステップ２２０〜２３８を繰り返す。これにより、全ての顔候補領域に対し、第１〜第４方向について顔領域としての整合度が各々判定される。
【０１１２】
ステップ２３８の判定が肯定されるとステップ２４０へ移行し、先のステップ２３６で各顔候補領域について各々記憶した整合度が最大の方向に基づいて、第１実施形態で説明したステップ１３８と同様にして画像の天地方向を判定する。次のステップ２４２では、判定した天地方向に対応する所定方向についての各顔候補領域毎の顔領域としての整合度を各顔候補領域に設定し、ステップ２４４へ移行する。
【０１１３】
そしてステップ２４４では、各顔候補領域に対して設定した整合度に応じて、各顔候補領域に対して重み点数を設定する。このステップ２４４は、ステップ２０８、２０９、２３０、２３１、２４２と共に請求項７に記載の評価手段に対応している。ステップ２４４における重み点数の設定についても、例として図６（Ａ）や図６（Ｂ）に示すように、整合度が高くなるに伴って重み点数Ｐが高くなるように変換特性が定められたマップを用い、該マップを用いて整合度を重み点数Ｐに変換することで行うことができる。なお、次のステップ１４４以降の処理は第１実施形態と同様であるので説明を省略する。
【０１１４】
上記では、眼部及び頬部に特有の濃度パターンが生じている濃度パターン領域を抽出し、抽出した濃度パターン領域の、顔候補領域内における位置及び面積比に基づいて、抽出した濃度パターン領域の眼部に相当する領域としての整合度及び頬部に相当する領域としての整合度を判定して顔候補領域に対する顔領域としての整合度を判定し、顔候補領域に対する重み点数を前記整合度に基づいて設定しているので、顔候補領域に対し、人物の顔の内部構造に基づき、顔領域としての確度を精度良く評価することができる。また、上記では眼部や頬部に相当する領域の外縁にエッジが存在していない場合であっても候補領域に対する評価の精度が低下することを防止できると共に、二値化に基づく領域分割のように、閾値を変更しながら繰り返し処理を行う必要が無いので処理も簡略化される。
【０１１５】
なお、上記では背景候補領域の面積比が第１の所定値以上か否か、及び背景候補領域の偏在度合いが第２の所定値以上か否か、の少なくとも一方の判定を満足する顔候補領域に対し、重み点数が低下するように修正していたが、これに限定されるものではなく、顔候補領域から顔領域を抽出する処理を行う場合には、該顔領域の抽出において、前記少なくとも一方の判定を満足する顔候補領域に対しては、例として図８（Ｂ）に示すように顔領域判定用の閾値が高くなるように閾値を変更してもよい。これにより、背景領域である可能性が比較的高い顔候補領域が顔領域として抽出されにくくすることができる。なお閾値ＴＨ_F の変更量は一定値としてもよいし、背景候補領域の面積比と第１の所定値との差、及び背景候補領域の偏在度合いと第２の所定値との差に応じて閾値ＴＨ_F の変更量を変化させてもよい。
【０１１６】
また、各顔候補領域に設定する重み点数Ｐ、顔領域判定用の閾値ＴＨ_F 、或いは顔領域濃度Ｍfaceにおいて各顔候補領域の濃度Ｍに付与する重みは、顔領域抽出・濃度演算処理の処理結果を利用して行われる画像処理の種類に応じて変更するようにしてもよい。
【０１１７】
例えば顔領域抽出・濃度演算処理による顔領域の抽出結果を利用して、イメージプロセッサ４０において、抽出された顔領域に対してのみ局所的にエッジ強調フィルタをかけて顔領域のシャープネスを強調するシャープネス強調処理が行われる場合、シャープネス強調の程度やフィルタの種類にも依存するが、実際には顔領域でない領域にもシャープネスの強調が行われたとしても視覚上は悪影響が小さい（目立たない）ことがある。このような場合には、顔領域判定用の閾値ＴＨ_F の値を通常よりも小さくし（すなわち顔候補領域の選択の基準を変更し）、より多くの顔候補領域が顔領域と判定されるようにしてもよい。顔領域判定用の閾値ＴＨ_F の値を低くするに従って、実際の顔領域に対応する顔候補領域が顔領域でないと誤判定される確率が低くなるので、上記により、画像中の顔領域に対して漏れなくシャープネス強調処理を施すことができる。
【０１１８】
また、顔領域判定用の閾値ＴＨ_F の値を変更することに代えて、第１実施形態に記載した一致度や第２実施形態に記載した整合度に対し、重み点数Ｐとして通常よりも大きな値を設定する（すなわち各顔候補領域に対する評価の基準を変更する）ことで、より多くの顔候補領域が顔領域と判定されるようにすることも可能である。特にシャープネス強調処理として、重み点数Ｐが大きくなるに従ってシャープネスの強調度合いを強くする処理が行われる場合には、重み点数Ｐを上記のように設定することでシャープネスの強調度合いを強めにコントロールすることも可能となる。
【０１１９】
また例えば、顔領域抽出・濃度演算処理による顔領域の抽出結果及び顔領域濃度Ｍfaceを利用して、抽出された顔領域に対してのみ顔領域濃度Ｍfaceに基づき局所的に濃度を補正する濃度補正処理が行われる場合、濃度補正の程度にも依存するが、実際には顔領域でない領域にも濃度補正が行われたとしても視覚上は悪影響が小さい（目立たない）ことがある。このような場合には、顔領域判定用の閾値ＴＨ_F の値を通常よりも小さくし、より多くの顔候補領域が顔領域と判定されるようにしてもよい。顔領域判定用の閾値ＴＨ_F の値を低くするに従って、実際の顔領域に対応する顔候補領域が顔領域でないと誤判定される確率が低くなるので、上記により、画像中の顔領域に対して漏れなく濃度補正処理を施すことができる。
【０１２０】
上記の説明は、顔領域の抽出において、実際には顔領域でない領域を誤って顔領域として抽出した場合にも影響が小さい画像処理が行われる場合であるが、逆に実際には顔領域でない領域を誤って顔領域として抽出した場合に多大な影響を受ける画像処理が行われる場合には、例えば顔領域判定用の閾値ＴＨ_F の値を通常よりも大きくしたり、一致度や整合度に対し重み点数Ｐとして通常よりも小さな値を設定することで、顔領域としての確度がより高い顔候補領域のみが顔領域として抽出されるようにすることも可能である。
【０１２１】
また、顔領域濃度についても、例えば次の（３）式に示すように、先の（１）式((２）式でもよい）で求まる顔領域濃度Ｍfaceと、他の画像特徴量Ｄ（例えば画像全体の平均濃度、非顔候補領域の平均濃度等）との加重平均値Ｍface’（但し、α_F は顔領域濃度Ｍfaceに対する重み係数、α₀ は画像特徴量Ｄに対する重み係数）を顔領域濃度として演算する場合、演算した顔領域濃度を利用して行われる画像処理の種類に応じて重み係数α_F ，α₀ の値を変更する（すなわち各顔候補領域に対する重み付けの基準を相対的に変更する）ことで、各顔候補領域の濃度Ｍに付与する重みを変更するようにしてもよい。
【０１２２】
Ｍface’＝α_F ・Ｍface＋α₀ ・Ｄ …（３）
また、顔領域抽出・濃度演算処理の処理結果を利用して行われる画像処理として、顔領域抽出・濃度演算処理の処理結果に対する要求が異なる複数種の画像処理（例えば、抽出された顔領域の中に実際には顔領域でない領域が混在していないことが望ましい画像処理と、抽出された顔領域の中に画像中の全ての顔領域が含まれていることが望ましい画像処理等）が各々行われる場合には、それぞれの画像処理に対応して顔領域の抽出や顔領域濃度の演算を複数回行ってもよい。本実施形態では、一致度や整合度を各顔候補領域の顔領域としての信頼度（確度）として用いることができ、上述したように、各顔候補領域に対する重み点数設定の基準、顔領域判定の基準（閾値ＴＨ_F ）、各顔候補領域に対する重み付けの基準の少なくとも１つを変更することで、顔領域抽出結果や顔領域濃度演算結果としてそれぞれの画像処理が要求する結果を各々得ることができるので、前記複数種の画像処理が各々行われる場合にも、非常に複雑かつ時間がかかる顔候補領域抽出処理を、前記複数種の画像処理に対応して処理条件を変更しながら画像処理の種類数と同数回繰り返す必要はなく、顔領域抽出・濃度演算処理の処理時間を短縮することができ、画像処理装置１４の性能向上を実現できる。
【０１２３】
また、上記ではプレスキャン画像データに基づきオートセットアップエンジン４４によって顔領域抽出・濃度演算処理を含む処理条件の演算を行い、ファインスキャン画像データに対する実際の画像処理はイメージプロセッサ４０で行う場合を説明したが、これに限定されるものでなく、単一の画像データに対して処理条件の演算、演算した処理条件での画像処理を順に行うようにしてもよく、これらの一連の処理を単一の処理部で行うようにしてもよい。
【０１２４】
更に、上記では各顔候補領域に対して設定した重み点数に基づき、顔領域の抽出及び顔領域濃度の演算を各々行っていたが、これに限定されるものではなく、何れか一方のみを行うようにしてもよい。
【０１２５】
また、上記では写真フィルムに記録された画像を読み取ることで得られた画像データを処理対象としていたが、これに限定されるものではなく、紙等の他の記録材料に記録された画像を読み取ることで得られた画像データや、デジタルカメラによる撮像によって得られた画像データ、或いはコンピュータによって生成された画像データを処理対象としてもよい。また、本発明は写真フィルムに記録されたフィルム画像を面露光により印画紙に露光記録する際の露光条件の決定に利用してもよいことは言うまでもない。
【０１２７】
【発明の効果】
以上説明したように請求項１、６、８記載の発明は、画像中の人物の顔に相当すると推定される候補領域を所定数の小領域に分割すると共に、小領域内における濃度又は輝度の変化の頻度及び変化の大きさに関連する特徴量を、各小領域毎に複数の方向について各々求め、人物の顔に相当する領域を所定数の小領域に分割したときの各小領域毎の前記特徴量の関係を表すパターンと照合して一致度を演算し、複数の方向についての一致度の演算結果に基づいて画像の天地方向を判定すると共に、判定した天地方向についての一致度の演算結果に基づいて候補領域の人物の顔に相当する領域としての確度を評価するので、画像の天地方向が不明であったとしても、人物の顔の内部構造に基づき、画像の天地方向を精度良く判定できると共に、人物の顔に相当する領域を簡易な処理により精度良く抽出することができる、という優れた効果を有する。
【０１２８】
請求項２記載の発明は、請求項１の発明において、人物の顔の眼部対を構成する個々の眼部に相当する領域が異なる小領域内に位置するように分割対象領域を分割するので、上記効果に加え、候補領域に対する人物の顔に相当する領域としての確度の評価をより精度良く行うことができる、という効果を有する。
【０１２９】
請求項３及び請求項７記載の発明は、人物の顔に相当すると推定される候補領域内の濃度又は輝度の分布に基づいて、人物の顔の特定の部分に特有の濃度又は輝度のパターンが生じている領域を抽出し、抽出した領域の候補領域内における位置、候補領域との面積比、濃度又は輝度のヒストグラムの形状の少なくとも１つに基づいて、抽出した領域の特定の部分に相当する領域としての整合度を演算することを複数の方向について各々行い、複数の方向についての整合度の演算結果に基づいて画像の天地方向を判定すると共に、判定した天地方向についての整合度の演算結果に基づいて候補領域の人物の顔に相当する領域としての確度を評価するので、画像の天地方向が不明であったとしても、人物の顔の内部構造に基づき、画像の天地方向を精度良く判定できると共に、人物の顔に相当する領域を簡易な処理により精度良く抽出することができる、という優れた効果を有する。
【０１３０】
請求項４記載の発明は、請求項３の発明において、高濃度方向又は低輝度方向に凸の濃度又は輝度のパターンが生じている領域を抽出し、抽出した領域の眼部に対応する領域としての整合性を判定するか、又は低濃度方向又は高輝度方向に凸の濃度又は輝度のパターンが生じている領域を抽出し、抽出した領域の頬部に対応する領域としての整合性を判定するので、上記効果に加え、候補領域に対する人物の顔に相当する領域としての確度の評価をより精度良く行うことができる、という効果を有する。
【０１３１】
請求項５記載の発明は、請求項１又は請求項２の発明において、画像中の主要部に相当すると推定される候補領域の明度が所定値以上の場合に、候補領域との明度の差が所定範囲内の画素から成る背景候補領域を、候補領域外の範囲内で探索し、抽出された背景候補領域の候補領域に対する面積比が所定値以上の場合、又は抽出された背景候補領域が画像中の周縁部に偏在している場合に、候補領域に対する主要部に相当する領域としての確度の評価を低くするので、画像中の背景に相当する領域が主要部に相当する領域として誤抽出されることを抑制できる、という効果を有する。
【図面の簡単な説明】
【図１】本実施形態に係る画像処理システムの概略構成図である。
【図２】第１実施形態に係る顔領域抽出・濃度演算処理の内容を示すフローチャートである。
【図３】背景領域判定処理の内容を示すフローチャートである。
【図４】（Ａ）は顔候補領域の分割、（Ｂ）は分割によって得られた所定数のブロックと照合するためのマッチング用パターンの一例を各々示す概念図である。
【図５】濃度変化値（エッジ強度）を求めるための微分フィルタの一例を示す概念図である。
【図６】（Ａ）及び（Ｂ）はマッチング用パターンとの一致度に応じて顔候補領域に重み点数を設定するためのマップを各々示す線図である。
【図７】背景領域が顔候補領域として誤抽出される可能性が高い画像の一例を示すイメージ図である。
【図８】（Ａ）は背景領域である可能性が比較的高い顔候補領域に対する重み点数を修正するためのマップを示す線図、（Ｂ）は前記顔候補領域が存在していた場合の顔領域判定用の閾値の変更を説明する線図である。
【図９】第２実施形態に係る顔領域抽出・濃度演算処理の内容を示すフローチャートである。
【図１０】（Ａ）は顔領域内の濃度が所定値以上の高濃度領域の分布の一例を示すイメージ図、（Ｂ）は顔領域のうち眼部に相当する領域内における濃度変化の一例を示す線図である。
【図１１】（Ａ）は顔領域内の輝度が所定値以上の高輝度領域の分布の一例を示すイメージ図、（Ｂ）は顔領域のうち頬部に相当する領域内における輝度変化の一例を示す線図である。
【符号の説明】
１０ 画像処理システム
１４ 画像処理装置
４０ イメージプロセッサ
４４ オートセットアップエンジン
７２ 情報記憶媒体[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing method, an image processing apparatus, and a recording medium, and in particular, an image processing method for extracting an area estimated to correspond to a human face in an image, an image processing apparatus to which the image processing method can be applied, And a recording medium on which a program for causing the computer to execute the image processing method is recorded.
[0002]
[Prior art]
The most noticeable part when viewing a portrait is the face of a person. For example, when an original image recorded on a photographic film or the like is recorded on a recording material such as photographic paper by exposure (recorded by surface exposure or scanning exposure) It is desirable to control the exposure so that the color and density of the person's face are appropriate, but in order to realize this exposure control, the color and density of the area corresponding to the person's face in the original image Must be detected accurately. In addition, in various image processing developed for the purpose of improving the image quality of an image represented by the image data obtained by reading the image, an area corresponding to a human face in the image or Some image processing (for example, local density correction or red-eye correction) is performed only on a part of the image. In order to perform this processing, the position of an area corresponding to a human face in the image, It is necessary to detect the size accurately.
[0003]
For this reason, conventionally, various methods for extracting a region estimated to correspond to a main part such as a human face in an image have been proposed. For example, in Japanese Patent Laid-Open No. 8-184925, based on image data, a shape pattern peculiar to each part of a person existing in an image (for example, head contour, face contour, face internal structure, trunk contour, etc.) The shape of the person's face is determined according to the size and orientation of the detected shape pattern, and the positional relationship between the predetermined portion of the person represented by the detected shape pattern and the person's face. Set a highly consistent area (candidate area) as the corresponding area, search for another shape pattern different from the detected shape pattern, and evaluate the consistency of the previously set candidate area as a human face However, a face area extraction method for extracting an area (face area) estimated to correspond to a human face has been disclosed.
[0004]
[Problems to be solved by the invention]
In the technique described in the above publication, the image to be processed is divided into a white area and a black area by binarization. For example, in setting a candidate area based on the internal structure of the face, each black area obtained by the division On the other hand, a black region estimated to correspond to the eye part of the person's face is extracted by determining the consistency of the shape as the area corresponding to the eye part of the person's face, and the extracted black region Candidate areas are set based on this. However, since the density range of the image to be processed is not constant and the density of the so-called black eye part of the eye part is not constant, the threshold value used for binarization is used in order to accurately extract the area corresponding to the eye part. While changing the above, it is necessary to repeat the above-described processing including region division by binarization a plurality of times, and to evaluate the region extracted by each processing. Therefore, there is a problem that it takes a lot of time for processing.
[0005]
In binarization, for example, when an area corresponding to an eye part of a person's face and an area corresponding to a relatively high density part such as a hair part are adjacent to each other on an image, black corresponding to an eye part is obtained. The region may be combined with a black region corresponding to another high density portion. In this case, there is a problem that it is difficult to correctly extract the region corresponding to the eye portion. This problem occurs even when the image is divided based on the edge extracted from the image (the portion where the density or luminance changes by a predetermined value or more), which is described in the above publication as an alternative method of binarization. If the change in density or luminance at the boundary between the area corresponding to the part and the area corresponding to the other high density part is small, it can occur similarly.
[0006]
As described above, in the conventional process using the internal structure of the face, when the area corresponding to the specific part inside the face to be extracted cannot be correctly extracted, the area corresponding to the human face is accurately extracted. There was a problem that it was not possible.
[0007]
By the way, when a scene including a main part such as a person is photographed, the area corresponding to the main part in the image is brighter than the area corresponding to the background, including when shooting with a strobe light. There are many. For this reason, even in the region corresponding to the main part, the low-luminance region is often excluded and the high-luminance region is extracted, but with this, the image to be processed is a person or the like under backlight illumination conditions. In the case of an image representing a scene including a main part, a high-luminance background part in the image may be erroneously extracted as an area corresponding to the main part. In this case, assuming that the exposure amount when the image is exposed and recorded on the recording material is controlled based on the color and density of the extracted area, the recorded image has an inappropriate image quality in which the area corresponding to the main part is crushed in black. Become.
[0008]
  The present invention has been made in consideration of the above facts, and based on the internal structure of a person's face, an image processing method and an image processing apparatus capable of accurately extracting a region corresponding to a person's face by simple processing And obtaining a recording mediumEyesIs.
[0010]
[Means for Solving the Problems]
  the aboveTo achieve the object, the image processing method according to the first aspect of the present invention extracts a candidate area estimated to correspond to a human face in the image represented by the image data based on the image data, and The extracted candidate area is divided into a predetermined number of small areas, and the feature amount related to the frequency and magnitude of the change in density or brightness in the small area,For each small areaEach in multiple directionsFor each small areaFor each of the plurality of directionsA pattern representing the relationship between the feature amounts of each small region when the obtained feature amount is divided into the predetermined number of small regions from a region corresponding to a person's faceEachCollationThe degree of coincidence is calculated, and the vertical direction of the image is determined based on the calculation result of the degree of coincidence for the plurality of directions, and based on the calculation result of the degree of coincidence for the determined vertical direction.The accuracy of the candidate area as an area corresponding to a person's face is evaluated.
[0011]
  In the first aspect of the present invention, first, based on the image data, a candidate area estimated to correspond to a human face in the image represented by the image data is extracted. In extracting the candidate region, any conventionally known algorithm can be applied. Next, the extracted candidate area is divided into a predetermined number of small areas, and the feature amount related to the frequency and magnitude of change in density or luminance in the small area is calculated.,For each small areaEach in multiple directionsAsk. Note that, as the feature amount related to the frequency of change in density or luminance and the magnitude of the change, for example, the amount of change in density or luminance between adjacent pixels along a predetermined direction may be integrated. It can be obtained by performing every time.
[0012]
  Within the area corresponding to the face of a person in the image, there are roughly constant areas where the density or brightness changes in a complex manner and areas where the density or brightness changes slowly or is almost uniform. Each exists in the position. For example, the eyeball, eyelashes, eyelashes, eyebrows, etc. are located close to each other in the partial area corresponding to the eye area and the periphery of the eye area, and the density and brightness are complex around the pupil and iris in the eyeball. Therefore, the density or luminance changes frequently and greatly (particularly, in the direction of arrangement of the eyeball, eyelashes, eyelashes, eyebrows, etc., that is, in the vertical direction of the face, the frequency of change in density or luminance And the magnitude of change is large). On the other hand, in the area corresponding to the face of the person, in the partial area corresponding to the cheek and its periphery, changes in density and luminance are gradual depending on the illumination conditions. Therefore, if the candidate area is an area corresponding to a person's face, the feature amount obtained for each small area by dividing the candidate area into a predetermined number of small areas is any part of the person's face. The value varies greatly depending on whether it corresponds to.Further, the feature amount of each small area obtained for each of a plurality of directions is the degree of coincidence with a pattern representing the relationship between the feature quantities for each small area when the area corresponding to the face of a person is divided into small areas. However, the value varies greatly depending on whether or not the direction of the feature value matches the vertical direction of the image.
[0013]
  In the invention of claim 1, for each small areaEach in multiple directionsA pattern representing a relationship between the feature quantities for each small area when the area corresponding to the human face is divided into a predetermined number of small areas.EachCollationThe degree of coincidence is calculated, and the vertical direction of the image is determined based on the calculation result of the degree of coincidence for a plurality of directions, and based on the calculation result of the degree of coincidence for the determined vertical direction.The accuracy of the candidate area as the area corresponding to the person's face is evaluated (for example, for each small areaEach in multiple directionsThe calculated feature valueThe degree of coincidence calculated fromOut ofDegree of coincidence about the determined top and bottom directionBut,Matches the pattern or has a high degree of matchIs a value indicating thatSo that the evaluation is higher), Can accurately determine the top and bottom direction of the image based on the internal structure of the person's face,The accuracy of the candidate area as an area corresponding to the person's face can be accurately evaluated based on the internal structure of the person's face. And based on the evaluation result with respect to a candidate area | region, the area | region equivalent to a person's face can be extracted accurately.
[0014]
  According to the first aspect of the present invention, it is not necessary to extract a specific part inside the face or to repeat the process while changing the threshold as in binarization. By not extracting correctly, it can also prevent that the precision of evaluation with respect to a candidate area | region falls. Therefore, according to the invention of claim 1,Even if the top and bottom direction of the image is unknown,Based on the internal structure of the human face,In addition to accurately determining the vertical direction of the image,An area corresponding to a human face can be extracted with a simple process with high accuracy.
[0016]
By the way, the eyes of a person's face are present at positions that are substantially symmetrical when viewed from the front of the face. For example, when photographing a scene including a person, the left and right eyes are present in the image. When there is an area corresponding to a person's face in the image, as in most cases of shooting, the area corresponding to the eye pair of the person's face exists in the image with a very high probability. is doing. And in the area | region equivalent to the eye part in an image, and its periphery, it has the characteristic that a density | concentration or a brightness | luminance changes frequently and largely as demonstrated previously.
[0017]
Therefore, in the invention according to claim 2, in the invention according to claim 1, the region to be divided is such that the regions corresponding to the individual eye parts constituting the eye pair of the person's face are located in different small regions. To split. This can be realized by adjusting at least one of the size of the small region (area ratio with the candidate region), the number (predetermined number described in claim 1), and the arrangement in the candidate region.
[0018]
If the candidate area is an area corresponding to a person's face, a small area pair corresponding to the eye pair of the person's face is present in the predetermined number of small areas with a very high probability, and the eye part is Since it exists in a peculiar position (a position that is substantially bilaterally symmetric when viewed from the front of the face), it is possible to easily determine a small region pair that is estimated to correspond to the eye pair. Therefore, according to the second aspect of the present invention, it is possible to more accurately evaluate the accuracy of a candidate region as a region corresponding to a person's face from a feature value value of a small region pair estimated to correspond to an eye pair. It can be carried out.
[0019]
  Also, ContractThe image processing method according to the third aspect of the present invention extracts a candidate area estimated to correspond to a human face in an image represented by the image data based on the image data, and the density in the extracted candidate area Alternatively, based on the luminance distribution, an area where a specific density or luminance pattern is generated in a specific part of the person's face is extracted, and the position of the extracted area in the candidate area and the area ratio with the candidate area Matching the extracted region as the region corresponding to the particular portion based on at least one of the shape of the density or luminance histogramEvery timeTheCalculationDoThis is performed for each of a plurality of directions, and the vertical direction of the image is determined based on the calculation result of the degree of matching for the plurality of directions, and based on the calculation result of the degree of matching for the determined vertical direction.The accuracy of the candidate area as an area corresponding to a person's face is evaluated.
[0020]
In the third aspect of the invention, similar to the first aspect of the invention, based on the image data, after extracting a candidate area estimated to correspond to a human face in the image represented by the image data, the extracted candidates Based on the density or luminance distribution in the area, an area in which a specific density or luminance pattern is generated in a specific portion of the human face is extracted.
[0021]
Since a person's face has irregularities, in an image representing a scene in which the person's face is illuminated under general lighting conditions, in a partial area corresponding to a depression (for example, an eye) of the person's face, The density distribution or luminance distribution is convex in the high density direction (low luminance direction) with the vicinity of the bottom as a vertex. As described above, the density or luminance frequently and greatly changes in the partial area corresponding to the eye part and the periphery thereof, but the density distribution (luminance as described above) Distribution). On the other hand, in a partial region corresponding to a convex portion (for example, cheek portion) of a person's face, a density distribution or luminance distribution convex in the low density direction (high luminance direction) with the vicinity of the vertex of the convex portion as the vertex.
[0022]
  In the invention of claim 3, for example, an area where a specific density or luminance pattern is generated in a specific part of the human face as described above is extracted, and the position of the extracted area in the candidate area, the candidate area, As a region corresponding to a specific part of the extracted region based on at least one of the shape of the histogram of area ratio, density or luminanceEvery timeTheCalculationDoThis is performed for each of a plurality of directions, and the top / bottom direction of the image is determined based on the calculation result of the matching degree for the plurality of directions, and based on the calculation result of the matching degree for the determined top / bottom direction.Since we evaluate the accuracy of the candidate area as the area corresponding to the person's face,Based on the internal structure of the person's face, it can accurately determine the vertical direction of the image,The accuracy of the candidate area as an area corresponding to the person's face can be accurately evaluated based on the internal structure of the person's face. And based on the evaluation result with respect to a candidate area | region, the area | region equivalent to a person's face can be extracted accurately.
[0023]
  The invention of claim 3 does not divide the area at the position where the edge exists unlike the area division based on the edge, so that the edge exists at the outer edge of the area corresponding to the specific part of the human face. It is not necessary, and even if there is no edge at the outer edge of the area, it is possible to prevent the evaluation accuracy for the candidate area from being lowered. Further, unlike the region division based on binarization, it is not necessary to perform the repeated processing while changing the threshold value, so that the processing is simplified. Therefore, according to the invention of claim 3, as in the invention of claim 1,Even if the top and bottom direction of the image is unknown,Based on the internal structure of the human face,In addition to accurately determining the vertical direction of the image,An area corresponding to a human face can be extracted with a simple process with high accuracy.
[0024]
In the invention of claim 3, as the density or luminance pattern peculiar to a specific part of the person's face, a pattern of density or luminance in any part of the person's face can be adopted. As described in 4, a region in which a convex density or luminance pattern is generated in a high density direction or low luminance direction peculiar to the eye part of a person's face is extracted, and the region corresponding to the eye part of the extracted region is extracted. Determine the consistency as a region, or extract a region where a pattern of convex density or luminance in the low density direction or high luminance direction peculiar to the cheek part of a person's face is generated, and the cheek of the extracted region It is preferable to determine consistency as a region corresponding to a part.
[0025]
It should be noted that the determination of the consistency of the region corresponding to the eye portion with respect to the region where the convex density or luminance pattern is generated in the high density direction or the low luminance direction is performed by determining the position of the extracted region in the candidate region, the candidate region At least one of the shape of the histogram of the area ratio, density or brightness of the eye, the position of the eye part on the face of the person, the area ratio of the whole face of the person to the eye part, the density or brightness of the eye part of the person face This can be done by matching with at least one of the shapes of the histogram. Similarly, regarding the determination of consistency as a region corresponding to the cheek portion with respect to a region where a convex density or luminance pattern occurs in the low density direction or the high luminance direction, the position of the extracted region in the candidate region, At least one of the shape of the area ratio, density or luminance histogram with the candidate area, the position of the cheek in the person's face, the area ratio of the whole person's face to the cheek, the density in the cheek of the person's face or This can be done by collating with at least one of the shapes of the luminance histogram.
[0026]
Since the above-described eye part and cheek part are present at positions that are substantially symmetrical when viewed from the front of the person's face, it corresponds to a region having a density or luminance pattern corresponding to the eye part or a cheek part. If a pair of regions having a density or luminance pattern to be extracted is extracted at substantially symmetrical positions, it can be determined that the consistency of the extracted region pair as a region corresponding to the eye part or cheek part is high. . Therefore, it is possible to more accurately evaluate the accuracy of the candidate region as a region corresponding to a person's face.
[0027]
By the way, when the image to be processed is an image representing a scene in which a main part such as a person exists under backlight illumination conditions, the high-luminance background part in the image is mistaken as an area corresponding to the main part. In some cases, the present inventor further examined the case where erroneous extraction occurs as described above. As a result, for example, when a high-luminance background area corresponding to the sky in the scene forms a single area on the image (not divided into a plurality of areas), the probability of erroneous extraction is low. On the other hand, for example, it has been found that there is a high probability of erroneous extraction when a high-luminance background region is divided into a plurality of regions on the image by an object existing before the background portion in the scene.
[0028]
  Based on the above, ContractOrigin of claim 5Tomorrow,In the invention of claim 1 or claim 2,When the brightness of the candidate area is equal to or greater than a predetermined value, a background candidate area consisting of pixels whose brightness difference from the candidate area is within a predetermined range is searched within the range outside the candidate area, and the background candidate area is extracted. And when the area ratio of the extracted background candidate area to the candidate area is a predetermined value or more, or when the extracted background candidate area is unevenly distributed in the peripheral portion of the image, the main part for the candidate area Lower the evaluation of accuracy as the area corresponding toIt is characterized by.
[0029]
  In the invention of claim 5,WithoutWhen the brightness of the candidate area is equal to or greater than a predetermined value, a background candidate area composed of pixels whose brightness difference from the candidate area is within a predetermined range is searched for within a range outside the candidate area. Thus, when the extracted candidate area is actually a part of the area corresponding to the background portion in the image, another candidate area corresponding to the background portion existing outside the candidate area is included. Parts will be extracted.
[0030]
Then, when a background candidate region composed of pixels whose brightness difference from the candidate region is within a predetermined range is extracted and the area ratio of the extracted background candidate region to the candidate region is a predetermined value or more, or the extracted background candidate When the area is unevenly distributed in the peripheral part in the image, the evaluation of the accuracy as the area corresponding to the main part with respect to the candidate area is lowered. When the area ratio of the background candidate area to the candidate area is greater than or equal to a predetermined value (when areas with the same brightness as the candidate area are distributed over a wide area in the image), or the background candidate area is at the periphery of the image If it is unevenly distributed, the background candidate area and the previously extracted candidate area are likely to be areas corresponding to the background portion. Therefore, by reducing the evaluation of the accuracy of the candidate region as the region corresponding to the main part as described above, the region corresponding to the background in the image is prevented from being erroneously extracted as the region corresponding to the main part. be able to.
[0031]
In addition, in an image or the like representing a scene in which the main part is illuminated by a strobe or the like, even if a region (background candidate region) having the same brightness as the main part exists in the image, the area of the background candidate region is Since it is small and has a low possibility of being unevenly distributed in the peripheral part in the image, according to the invention of claim 5, it corresponds to the main part for the candidate area estimated to correspond to the main part in the image as described above. It is possible to prevent the evaluation of the accuracy as the area to be performed from being unduly lowered.
[0032]
  An image processing apparatus according to a sixth aspect of the present invention is based on image data, and includes first extraction means for extracting a candidate area estimated to correspond to a human face in an image represented by the image data; The candidate area extracted by one extraction means is divided into a predetermined number of small areas, and the feature quantity related to the frequency of change in density or luminance in the small area and the magnitude of the change is obtained.,For each small areaEach in multiple directionsCalculating means for obtaining, and for each small region by the calculating meansFor each of the plurality of directionsA pattern representing a relationship between the feature quantities for each small area when the area corresponding to a human face is divided into the predetermined number of small areas.EachCollationThe degree of coincidence is calculated, and the vertical direction of the image is determined based on the calculation result of the degree of coincidence for the plurality of directions, and based on the calculation result of the degree of coincidence for the determined vertical direction.And an evaluation unit that evaluates the accuracy of the candidate region as a region corresponding to a person's face.Even if the top and bottom direction of the image is unknown,Based on the internal structure of the human face,In addition to accurately determining the vertical direction of the image,An area corresponding to a human face can be extracted with a simple process with high accuracy.
[0033]
  An image processing apparatus according to a seventh aspect of the invention is based on image data, and includes first extraction means for extracting a candidate area estimated to correspond to a human face in an image represented by the image data; Second extraction means for extracting a region in which a specific density or luminance pattern is generated in a specific part of a person's face based on the density or luminance distribution in the candidate area extracted by one extraction means; The aboveBy the second extraction meansExtractionIsOf the areaAbovePosition within the candidate area,AboveMatching the extracted region as the region corresponding to the specific portion based on at least one of the shape of the histogram of the area ratio, density or luminance with the candidate regionEvery timeTheCalculationDoThis is performed for each of a plurality of directions, and the vertical direction of the image is determined based on the calculation result of the degree of matching for the plurality of directions, and based on the calculation result of the degree of matching for the determined vertical direction.And an evaluation unit that evaluates the accuracy of the candidate region as a region corresponding to a person's face.Even if the top and bottom direction of the image is unknown,Based on the internal structure of the human face,In addition to accurately determining the vertical direction of the image,An area corresponding to a human face can be extracted with a simple process with high accuracy.
[0034]
  The recording medium according to the invention of claim 8 isFirst extraction means for extracting a candidate area estimated to correspond to a human face in the image represented by the image data based on the image data, and the candidate area extracted by the first extraction means A calculation unit that divides into a predetermined number of small areas and obtains a feature quantity related to the frequency of change in density or luminance in the small area and the magnitude of the change in each of the plurality of directions; and The feature amount obtained for each of the plurality of directions by the calculation means for each of the plurality of directions, and the feature amount for each small region when the region corresponding to the face of a person is divided into the predetermined number of small regions. A matching degree is calculated by collating each with a pattern representing a relationship, and the vertical direction of the image is determined based on the calculation result of the matching degree for the plurality of directions, and the matching for the determined vertical direction Of the result in to function as an evaluation means for evaluating the accuracy of a region corresponding to a person's face of the candidate area basedThe program is recorded.
[0035]
  The recording medium according to the invention of claim 8 includesComputeraboveFirst extraction means, calculation means, and evaluation meansClaims a program for functioning as a computer, ie a computer6 related to the inventionImage processingFunction as a deviceSince a program for recording is recorded, the computer reads out and executes the program recorded in the recording medium, and, 6Like the invention ofEven if the top and bottom direction of the image is unknown,Based on the internal structure of the human face,In addition to accurately determining the vertical direction of the image,An area corresponding to a human face can be extracted with a simple process with high accuracy.
[0036]
  A recording medium according to the invention of claim 9 is provided.First extraction means for extracting a candidate area estimated to correspond to the face of a person in the image represented by the image data based on the image data; in the candidate area extracted by the first extraction means; A second extracting unit that extracts a region where a specific density or luminance pattern is generated in a specific portion of a person's face based on the density or luminance distribution, and the second extracting unit extracts A region corresponding to the specific portion of the extracted region based on at least one of a position in the candidate region, an area ratio with the candidate region, a density or luminance histogram shape, and the like. The degree is calculated for each of a plurality of four directions, and the top and bottom direction of the image is determined based on the calculation result of the degree of matching for the plurality of directions. About for matching degree on the basis of the calculation result to function as an evaluation means for evaluating the accuracy of a region corresponding to a person's face in the candidate regionThe program is recorded.
[0037]
  The recording medium according to the invention of claim 9 includesComputeraboveFirst extraction means, second extraction means, and evaluation meansClaims a program for functioning as a computer, ie a computer7 related to the inventionImage processingFunction as a deviceSince a program for recording is recorded, the computer reads out and executes the program recorded on the recording medium, and, 7Like the invention ofEven if the top and bottom direction of the image is unknown,Based on the internal structure of the human face,In addition to accurately determining the vertical direction of the image,An area corresponding to a human face can be extracted with a simple process with high accuracy.
[0038]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings.
[0039]
[First Embodiment]
FIG. 1 shows an image processing system 10 to which the present invention is applied. The image processing system 10 includes a scanner 12, an image processing device 14, and a printer 16 connected in series.
[0040]
The scanner 12 is a negative image visualized by photographing a subject and developing the film image recorded on a photographic photosensitive material (hereinafter simply referred to as a photographic film) such as a photographic film (for example, a negative film or a reversal film). Image or positive image) is output, and image data obtained by the reading is output, and light emitted from the light source 20 and light amount unevenness reduced by the light diffusion box 22 is set on the film carrier 24. The light that has been applied to the photographic film 26 such as a negative film or a reversal film and has passed through the photographic film 26 passes through the lens 28 onto the light receiving surface of the CCD sensor 30 (which may be an area sensor or a line sensor). It is configured to form an image.
[0041]
The film carrier 24 conveys the photographic film 26 so that the portions where the film images on the photographic film 26 are recorded are sequentially positioned on the optical axis of the light emitted from the light source 20. As a result, the film images recorded on the photographic film 26 are sequentially read by the CCD sensor 30, and a signal corresponding to the film image is output from the CCD sensor 30. The signal output from the CCD sensor 30 is converted into digital image data by the A / D converter 32 and input to the image processing device 14.
[0042]
The line scanner correction unit 36 of the image processing apparatus 14 performs dark correction / darkness to reduce the dark output level of the corresponding cell for each pixel from the input scan data (R, G, B data input from the scanner 12). Density conversion for logarithmically converting the corrected data into data representing density values, shading correction for correcting data after density conversion in accordance with unevenness in the amount of light illuminating the photographic film 26, and data of the data subjected to the shading correction Among these, each process of defective pixel correction that is newly generated by interpolating data of cells (so-called defective pixels) that do not output a signal corresponding to the amount of incident light from data of surrounding pixels is sequentially performed. The output end of the line scanner correction unit 36 is connected to the input end of the I / O controller 38, and the data subjected to the above processes in the line scanner correction unit 36 is input to the I / O controller 38 as scan data. .
[0043]
The input end of the I / O controller 38 is also connected to the data output end of the image processor 40, and image data subjected to image processing (details will be described later) is input from the image processor 40. The input end of the I / O controller 38 is also connected to the personal computer 42. The personal computer 42 has an expansion slot (not shown). In this expansion slot, image data is written by a digital camera card in which image data is written by a digital still camera or the like, or a CD-R writing device (not shown). A driver (not shown) for reading / writing data and a communication control device for communicating with other information processing devices are connected to the information storage medium such as a CD-R. When file image data (image data read from a digital camera card or CD-R or image data received from another information processing device) is input from the outside via an expansion slot, the input file image data is I / O input to the controller 38.
[0044]
The output end of the I / O controller 38 is connected to the data input end of the image processor 40, the auto setup engine 44, and the personal computer 42, and is further connected to the printer 16 via the I / F circuit 54. The I / O controller 38 selectively outputs the input image data to each device connected to the output terminal.
[0045]
In this embodiment, each film image recorded on the photographic film 26 is read twice by the scanner 12 at different resolutions. In the first reading at a relatively low resolution (hereinafter referred to as pre-scan), even if the density of the film image is very low (for example, a negative image of an underexposure in a negative film), the CCD sensor 30 saturates the accumulated charge. The entire surface of the photographic film 26 is read under the reading conditions determined so as not to occur (the amount of light in each of the R, G, and B wavelength ranges of the light irradiated on the photographic film 26 and the charge accumulation time of the CCD sensor 30). . Data obtained by this prescan (prescan data) is input from the I / O controller 38 to the auto setup engine 44.
[0046]
The auto setup engine 44 includes a CPU 46, a RAM 48 (for example, DRAM), a ROM 50 (for example, a ROM capable of rewriting stored contents), and an input / output port 52, which are connected to each other via a bus. The auto setup engine 44 determines the frame position of the film image based on the prescan data input from the I / O controller 38, and obtains data corresponding to the film image recording area on the photographic film 26 (prescan image data). Extract. Further, based on the pre-scan image data, the size of the film image is determined and the image feature amount such as density is calculated, and the scanner 12 performs re-scanning at a relatively high resolution on the pre-scanned photographic film 26. Reading conditions for reading (hereinafter referred to as fine scan) are determined. Then, the frame position and reading conditions are output to the scanner 12.
[0047]
Further, the auto setup engine 44 is an image including extraction of a main part (for example, a region corresponding to a human face (face region)) in the image based on the pre-scan image data (or the file image data with a reduced resolution). A feature amount calculation is performed, and various image processing conditions for the fine scan image data (or file image data) obtained by the fine scan performed by the scanner 12 are automatically determined by the calculation (setup calculation). The processed conditions are output to the image processor 40.
[0048]
The personal computer 42 is connected to a display, a keyboard, and a mouse (all not shown). The personal computer 42 captures low-resolution image data from the auto setup engine 44, captures the image processing conditions determined by the auto setup engine 44, and targets high-resolution image data based on the captured processing conditions. Image processing equivalent to the image processing performed by the image processor 40 is performed on the low-resolution image data to generate simulation image data.
[0049]
Then, the generated simulation image data is converted into a signal for displaying the image on the display, and the simulation image is displayed on the display based on the signal. When the displayed simulation image is verified by an operator, such as image quality, and information for instructing correction of processing conditions is input as a verification result via the keyboard, the information is output to the auto setup engine 44. . As a result, the auto setup engine 44 performs processing such as recalculation of processing conditions for image processing.
[0050]
On the other hand, fine scan image data (or high resolution file image data) input to the I / O controller 38 by performing fine scan on the film image by the scanner 12 is transferred from the I / O controller 38 to the image processor 40. Is input. The image processor 40 enhances sharpness while suppressing graining, color / density correction processing including gradation conversion and color conversion, pixel density conversion processing, hypertone processing for compressing the gradation of the ultra-low frequency luminance component of the image, and graininess. Each image processing circuit that performs various image processing such as hyper sharpness processing is provided, and various image processing is performed on the input image data according to the processing conditions determined and notified for each image by the auto setup engine 44. I do.
[0051]
In addition to the above, image processing that can be executed by the image processor 40 includes, for example, sharpness correction or soft focus processing for the entire image or a part thereof (for example, an area corresponding to a human face), and image processing for intentionally changing the tone. (Image processing for finishing an output image in a monotone, image processing for finishing an output image in a portrait, image processing for finishing an output image in a sepia tone, etc.) and image processing for processing an image (for example, a person existing in an original image) Image processing for thinning on the main image, image processing for correcting red-eye, etc.) and images taken with LF (film with lens) for images caused by distortion and lateral chromatic aberration of the LF lens LF aberration correction processing that corrects geometric distortion and color misregistration, and brightness reduction at the periphery of an image due to peripheral light reduction of the LF lens Corrects deterioration in image quality of output images due to LF lens characteristics, such as peripheral dimming correction processing to correct or out-of-focus blur correction processing to correct reduction in image sharpness due to LF lens characteristics And various LF aberration correction processes.
[0052]
When image data that has been subjected to image processing by the image processor 40 is used for recording an image on photographic paper, the image data that has been subjected to image processing by the image processor 40 is transferred from the I / O controller 38 to an I / F circuit. The image data is output to the printer 16 via 54. When the image data after image processing is output to the outside as an image file, the image data is output from the I / O controller 38 to the personal computer 42. As a result, the personal computer 42 outputs the image data input from the I / O controller 38 for output to the outside to the outside (the driver, the communication control device, etc.) as an image file via the expansion slot.
[0053]
The printer 16 includes an image memory 58, R, G, and B laser light sources 60, and a laser driver 62 that controls the operation of the laser light sources 60. The recording image data input from the image processing device 14 is temporarily stored in the image memory 58, read out, and used for modulation of R, G, and B laser beams emitted from the laser light source 60. The laser light emitted from the laser light source 60 is scanned on the photographic paper 68 through the polygon mirror 64 and the fθ lens 66, and an image is exposed and recorded on the photographic paper 68. The photographic paper 68 on which the image is recorded by exposure is sent to the processor unit 18 and subjected to color development, bleach-fixing, washing with water, and drying. Thereby, the image exposed and recorded on the photographic paper 68 is visualized.
[0054]
Next, as an operation of the present embodiment, prescan data is input from the scanner 12 to the image processing apparatus 14, and a face is performed after the auto setup engine 44 performs processing such as extraction of image data from the prescan data. The region extraction / density calculation processing will be described.
[0055]
The face area extraction / density calculation process according to the first embodiment is a process to which the image processing method according to the invention of claim 1 is applied, and the face area extraction / density correction program is executed by the CPU 46 of the auto setup engine 44. This is realized by being executed. The face area extraction / density correction program is initially stored in the information storage medium 72 (see FIG. 1) together with a program for causing the CPU 46 to execute other processes. In FIG. 1, the information storage medium 72 is shown as a floppy disk. However, the information storage medium 72 may be composed of another information storage medium such as a CD-ROM or a memory card. When the information reading medium (not shown) connected to the personal computer 42 is loaded with the information storage medium 72 and an instruction to transfer (install) the program from the information storage medium 72 to the image processing apparatus 14 is given, the information reading apparatus A face area extraction / density correction program or the like is read from the information storage medium 72 and stored in a ROM 50 that can rewrite the stored contents.
[0056]
When the timing for executing the face area extraction / density correction process arrives, the face area extraction / density correction program is read from the ROM 50, and the face area extraction / density correction program is executed by the CPU 46. Thus, the auto setup engine 44 functions as an image processing apparatus according to the invention of claim 6. Thus, the information storage medium 72 storing the face area extraction / density correction program and the like corresponds to the recording medium according to the eighth aspect.
[0057]
The face area extraction / density correction process will be described below with reference to the flowchart of FIG. In step 100, based on the image data to be processed, an area estimated to correspond to a human face in the image (a face candidate area extracting process for extracting a face candidate area) is performed as a main part in the image represented by the image data. As an extraction method for performing this face candidate region extraction process, a face candidate region extraction method for determining a region estimated to correspond to a human face in an image and extracting the region as a face candidate region, or an image There is a background removal method that determines a region (background region) that is estimated to correspond to the background in the middle, and extracts a region other than the background region as a face candidate region. The face candidate region extraction process can be performed by adopting at least one of the face candidate extraction method and the background removal method.
[0058]
[Example of face candidate area extraction method 1]
Based on the data (image data) obtained by dividing the image into a large number of measurement points and separating each measurement point into three colors R, G, and B, each measurement point has a skin color range on the color coordinates. A region containing a cluster (group) of measurement points determined to be within the skin color range is extracted as a face candidate region (Japanese Patent Laid-Open No. 52-156624, Japanese Patent Laid-Open No. (See JP-A-52-156625, JP-A-53-12330, JP-A-53-145620, JP-A-53-145621, JP-A-53-145622, etc.).
[0059]
[Example 2 of face candidate area extraction method]
Based on the image data, a histogram for the hue value (and saturation value) is obtained, the obtained histogram is divided for each mountain, and it is determined to which of the divided mountains each measurement point belongs. Are divided into groups corresponding to the divided mountains, the image is divided into a plurality of regions for each group, a region corresponding to a human face is estimated from the plurality of regions, and the estimated region is extracted as a face candidate region (See JP-A-4-346333).
[0060]
[Example 3 of face candidate area extraction method]
Based on the image data, a shape pattern peculiar to each part of the person existing in the image (for example, a shape pattern representing the contour of the head, the contour of the face, etc .: the internal structure of the face is used in the processing described later, so here 1) is detected and corresponds to a human face according to the size and orientation of the detected shape pattern and the positional relationship between the predetermined portion of the person represented by the detected shape pattern and the human face. Set the estimated area. Also, another shape pattern different from the detected shape pattern is searched, the consistency of the previously set region as a human face is obtained, and face candidate regions are extracted (Japanese Patent Laid-Open Nos. 8-122944 and No. 8-183925, Japanese Patent Laid-Open No. 9-138471, etc.).
[0061]
[Example 4 of face candidate area extraction method]
Based on the image data, the amount of change in density or brightness at each location in the image is obtained for each direction, a reference point is set, and a search range and a search at each location within the search range are set for the reference point. A search direction pattern indicating a change direction of density or luminance to be set is set in accordance with the contour shape of the face area, and a change amount of density or luminance that exists in the search range and along the direction indicated by the search direction pattern is set. Searching for a location that is equal to or greater than a predetermined value, and when a location that satisfies the search condition is detected, repeatedly setting that location as the next reference point, and connecting multiple locations in the image set in sequence as the reference point A face candidate region is extracted by extracting a line as a contour line representing the contour of the face region (see JP-A-913871).
[0062]
[Example 1 of background removal method]
Based on the image data, it is determined whether or not each measurement point is included in a range of a specific color clearly belonging to the background on the color coordinates (for example, sky, sea blue, lawn, tree green, etc.) Then, an area where a cluster (group) of measurement points determined to be within the specific color range exists is determined as a background area, and the remaining area is removed as a non-background area (an area corresponding to a human face is Region that is likely to be included: this is also extracted as a face candidate region of the present invention.
[0063]
[Example 2 of background removal method]
Based on the image data, the image is divided into a plurality of areas in the same manner as in the second example of the main part extraction method, and then the feature amount as the area corresponding to the background for each area (the straight line portion included in the outline). Ratio, line symmetry, number of irregularities, contact rate with the outer edge of the image, density contrast in the area, presence / absence of density change pattern in the area, etc.) and whether each area is a background area based on the obtained feature quantity Thus, the area determined as the background portion is removed, and the remaining area is extracted as a non-background area (face candidate area) (see JP-A-8-122944, JP-A-8-183925, etc.).
[0064]
Note that the above extraction method is merely an example, and it goes without saying that any method can be applied as long as it is an extraction method that extracts an area estimated to correspond to a human face from an image. Further, in step 100, the face candidate area extraction process may be performed a plurality of times by applying each of a plurality of types of extraction methods, or the face candidate area extraction process may be performed a plurality of times by changing processing conditions in a single type of extraction method. May be. Step 100 corresponds to the first extraction means described in claim 6 (and claim 7).
[0065]
In the next step 102, it is determined whether or not there is top / bottom information indicating the top / bottom direction of the image represented by the image data to be processed. For example, when the image data to be processed is image data obtained by reading an image recorded on a photographic film on which a magnetic layer is formed, among various information magnetically recorded on the magnetic layer, It often includes top-and-bottom information that is magnetically recorded when an image is recorded. In such a case, the top-and-bottom information is acquired by magnetically reading the information magnetically recorded on the magnetic layer, the determination at step 102 is affirmed, and the routine proceeds to step 104.
[0066]
In step 104, data of a single face candidate area is fetched from the face candidate areas extracted by the face candidate area extraction process in step 100, and in step 106, the face candidate area to be processed in which the data is fetched is selected. As an example, it is divided into a predetermined number of blocks as shown in FIG. As an example, in FIG. 4A, a square region (the outermost edge of FIG. 4A) normalized so that the length of one side coincides with the length of the face candidate region (usually the vertical direction of the image). A candidate for a face along a dividing line (line indicated by a broken line) when the square area is divided into 5 × 5 square blocks (the small area described in claim 1). The area is divided.
[0067]
It should be noted that the division pattern of the face candidate area shown in FIG. 4A (the number of divisions into blocks and the shape and arrangement of the blocks) is, as is apparent from the figure, the individual eyes that constitute the eye pair of the human face. The areas corresponding to the sections are determined to be located in different blocks. Therefore, dividing the face candidate area according to the division pattern shown in FIG. 4A corresponds to the division described in claim 2. The division pattern is not limited to the above, and the number of blocks and the shape and arrangement of the individual blocks can be changed as appropriate without departing from the present invention.
[0068]
In step 108, for each block of the face candidate area, the edge intensity integrated value is obtained by calculating and integrating the edge intensity (density change amount) in a predetermined direction corresponding to the vertical direction represented by the vertical information. Note that the edge intensity integrated value corresponds to the “feature amount related to the frequency and magnitude of changes in density or luminance within a small region” according to the present invention, and step 108 is the calculation according to claim 6. Corresponds to the means. In the first embodiment, since the face area is extracted using the fact that the density in the area corresponding to the eye part of the person's face changes frequently and greatly, the change in the density in the area corresponding to the eye part is changed. The direction in which the frequency and the magnitude of the change are remarkably increased, that is, the direction that coincides with the top and bottom direction is set as a predetermined direction corresponding to the top and bottom direction.
[0069]
FIG. 5 shows density changes along a direction from a specific pixel toward each of the eight pixels existing in the vicinity of the specific pixel (total of eight directions: shown as eight arrows having different directions in FIG. 5). A differential filter for computing the value is shown. For the calculation of the edge intensity integrated value for each block, for example, a differential filter corresponding to a predetermined direction corresponding to the top-and-bottom direction is selected from the above eight differential filters, and all of the face candidate regions are selected using the selected differential filter. It can be obtained by calculating the amount of change in density (edge intensity) along the predetermined direction for each pixel and integrating the calculation results for each block.
[0070]
In the next step 110, the integrated edge intensity value for each block calculated in step 108 is used for each matching pattern preset in order to evaluate the accuracy of the face candidate area as an area corresponding to a human face. The degree of coincidence with the matching pattern is obtained by collating with the edge intensity integrated value (standard value) for each block. The matching pattern divides an area (face area) corresponding to a human face in an image in the same manner as the division pattern for the face candidate area (see FIG. 4B), and obtains an edge intensity integrated value for each block. This is performed for a large number of face areas of a large number of images, and for each block, the average value of the integrated edge intensity values for each block obtained for each of the large number of face areas of the large number of images is set as a standard value. It has been.
[0071]
As described above, the standard value of the edge intensity integrated value for each block in the matching pattern is as shown in FIG. 4 (B) for a pair of blocks (see also FIG. 4 (A)) corresponding to a pair of eyes of a person's face. As shown in FIG. 4B, the block corresponding to a pair of cheeks and eyebrows (see also FIG. 4A) of the person's face is very high as described in FIG. The value becomes a low value and reflects the frequency of change in density (or luminance) and the magnitude of the change in each part of the human face.
[0072]
In the first embodiment, out of 5 × 5 blocks, five blocks corresponding to a pair of eyes, a pair of cheeks, and an eyebrow in a human face (enclosed by a solid line in FIG. 4B). For the block indicated by (5), collation is performed by comparing the calculated value of the edge intensity integrated value with the standard value of the edge intensity integrated value set in the matching pattern. Then, a physical quantity whose value changes in accordance with the difference of the calculated value with respect to the standard value of the edge intensity integrated value for the five blocks (for example, when the deviation of the standard value of the edge intensity integrated value of each block is used as a reference) The ratio of the deviation of the calculated value of the edge intensity integrated value of each block is calculated and set as the degree of coincidence. The degree of coincidence may be set by comparing and collating the edge strength integrated values of all blocks.
[0073]
In step 112, it is determined whether or not the processing in steps 104 to 110 has been performed on all the face candidate regions extracted in step 100. If the determination is negative, the process returns to step 104, and steps 104 to 112 are repeated until the determination is positive. As a result, the degree of coincidence with the matching pattern is calculated and set for all face candidate regions. When the determination at step 112 is affirmed, the routine proceeds to step 142.
[0074]
On the other hand, if there is no top-and-bottom information, the determination in step 102 is negative and the process proceeds to step 120, where data of a single face candidate region is extracted from the face candidate regions extracted by the face candidate region extraction process in step 100. Then, in the next step 122, the face candidate area in which the data has been captured is divided into a predetermined number of blocks in the same manner as in the previous step 106. This step 122 also corresponds to the division described in claim 2. In step 124, since the vertical direction of the image to be processed is unknown, predetermined four directions parallel to the four sides forming the outer edge of the image to be processed (hereinafter referred to as the first direction to the fourth direction for convenience). The edge strength integrated value for each of the first to fourth directions is obtained by calculating and integrating the edge strength (density change amount). This step 124 also corresponds to the calculation means described in claim 6.
[0075]
In step 126, 1 is substituted for the variable n, and in the next step 128, the edge intensity integrated value for each block in the n-th direction (in this case, the first direction) is assumed, and the n-th direction is assumed to be the vertical direction. Matching is performed with the matching pattern, and the degree of coincidence is calculated in the same manner as in Step 110 above. In step 130, it is determined whether or not the value of the variable n has become 4. If the determination is negative, the value of the variable n is incremented by 1 in step 132 and the process returns to step 128. Thus, in step 128, the degree of coincidence with the matching pattern is calculated assuming that the first to fourth directions are the vertical directions.
[0076]
Note that the direction in which the degree of coincidence is calculated is not limited to the first to fourth directions. For example, the degree of coincidence may be calculated in eight directions indicated by eight arrows in FIG. The same applies to the second embodiment.
[0077]
When the degree of coincidence is calculated for each direction, the determination in step 130 is affirmed, the process proceeds to step 134, and the direction in which the degree of coincidence is maximum among the first to fourth directions is stored. In the next step 136, it is determined whether or not all face candidate areas have been processed. If the determination is negative, the process returns to step 120 and steps 120 to 136 are repeated. As a result, the degree of coincidence with the matching pattern is calculated and set for each of the face candidate regions in the first to fourth directions.
[0078]
If the determination in step 136 is affirmative, the process proceeds to step 138, and the top-to-bottom direction of the image is determined based on the direction with the highest degree of coincidence stored for each face candidate region in the previous step 134. In this determination, the number of face candidate areas with the highest matching degree direction may be calculated for each direction, and the direction with the largest number of face candidate areas may be determined as the vertical direction. In addition, based on the degree of coincidence of the specific face candidate region in the first to fourth directions, weight points are added in each direction so that the weight point increases as the degree of coincidence increases. The direction in which the integrated value of the weight points added to each direction is the largest may be determined as the vertical direction.
[0079]
When the top / bottom direction is determined as described above, in the next step 140, the degree of coincidence of each face candidate region in a predetermined direction corresponding to the determined top / bottom direction is set in each face candidate region, and the process proceeds to step 142. .
[0080]
In step 142, a weight score is set for each face candidate area in accordance with the matching degree set for each face candidate area. The weight score corresponds to an evaluation value for evaluating the accuracy of the face candidate area as an area corresponding to a person's face, and step 142 includes steps 110, 128, and 140 together with the evaluation means according to claim 6. It corresponds.
[0081]
For example, as shown in FIG. 6A, the weight score is obtained by using a map in which conversion characteristics are determined so that the weight score P increases as the matching score increases. It can be set by converting to weight point number P. Note that the conversion characteristic shown in FIG. 6A is merely an example. For example, in the conversion characteristic shown in FIG. 6A, there is a dead area where the weight point P does not change with respect to the change in matching degree. It may be a conversion characteristic having no insensitive area (see FIG. 6B as an example). Also, a conversion characteristic in which the weight point number P changes nonlinearly with respect to the change in the degree of coincidence may be used, as long as the conversion characteristic increases as the degree of coincidence increases.
[0082]
In the above, the edge intensity integrated value obtained for each block is collated with a matching pattern representing the edge intensity integrated value for each block in the actual face area to obtain the degree of coincidence, and as the face area for the face candidate area Since the weight score corresponding to the accuracy evaluation result is set based on the degree of coincidence, the accuracy as the face region is accurately evaluated for each face candidate region based on the internal structure of the person's face. Can do. Also, it is not necessary to extract a specific part inside the face or to repeat the process while changing the threshold as in binarization, so that the process is simplified and the specific part inside the face is not correctly extracted. It is also possible to prevent the evaluation accuracy for the face candidate area from being lowered.
[0083]
In the next step 144, background area determination processing is performed. The background area determination process will be described with reference to the flowchart of FIG. In step 160, data of a single face candidate area to be determined is fetched from the face candidate areas extracted by the face candidate area extraction process (step 100), and in the next step 162, the brightness M of the face candidate area to be determined. Is calculated. The lightness M may be an average value (absolute value) of lightness in the face candidate area to be determined, or the average lightness in the face candidate area to be determined with respect to the entire screen average value of the lightness of the processing target image. It may be a ratio (relative value).
[0084]
In step 164, it is determined whether or not the brightness obtained in step 162 is a predetermined value or more. If the determination is negative, the process proceeds to step 176 without performing any processing. If the determination is affirmative, the process proceeds to step 166, and out of the processing target image outside the face candidate region to be determined. From the corresponding range, an area (background candidate area) is extracted that includes only pixels whose brightness difference from the face candidate area is within a predetermined range. Note that step 166 corresponds to “searching for a background candidate region including pixels whose brightness difference from the candidate region is within a predetermined range” according to claim 5.
[0085]
In the next step 168, the total area of the extracted background candidate regions is calculated, and the ratio (area ratio) of the total area of the background candidate regions to the area of the face candidate region to be determined is calculated. In the next step 170, the degree of uneven distribution of the extracted background candidate region in the peripheral portion of the image is calculated. As the uneven distribution degree, for example, a value obtained by integrating the distance between the screen center of the processing target image and each pixel constituting the background candidate region can be used.
[0086]
In step 172, it is determined whether or not at least one of whether the area ratio of the background candidate region is equal to or greater than a first predetermined value and whether the degree of uneven distribution of the background candidate region is equal to or greater than a second predetermined value is satisfied. . For example, as shown in FIG. 7, the image to be processed has a plurality of background regions (regions corresponding to the sky in FIG. 7) having a high brightness and a large area depending on the regions corresponding to the objects (trees in FIG. 7) present in the foreground. If the image is divided into two regions, a part of the background region may be erroneously extracted as a face candidate region as shown by hatching in FIG.
[0087]
However, in such an image, a region having the same brightness as that of the face candidate region exists around the erroneously extracted face candidate region. In many cases, this region has a large area and a peripheral portion of the image. Is unevenly distributed. Therefore, when the determination in step 172 is affirmed, it is estimated that there is a high possibility that the determination target face candidate area is a part of the background area. Therefore, if the determination in step 172 is negative, the process proceeds to step 176 without performing any processing. If the determination is affirmative, the process proceeds to step 174, where the area ratio and the degree of uneven distribution of the background candidate region Accordingly, the weight score is corrected so that the weight score for the face candidate region to be determined decreases.
[0088]
The correction of the weight point number P can be performed using, for example, a map shown in FIG. This map has a conversion characteristic represented by a straight line passing through the origin and having a slope of less than 1 on the coordinates with the initial (initial) weight points on the horizontal axis and the corrected weight points on the vertical axis. is doing. By converting the weight point P using the map as described above (downward correction), it is possible to reduce the degree of the adverse effect on the post-processing of the face candidate area to be judged that is likely not actually a face area. Can do. The correction of the number of weight points in step 174 corresponds to “reducing the evaluation of accuracy as a region corresponding to the main part of the candidate region” according to claim 5.
[0089]
In the next step 176, it is determined whether or not processing has been performed for all face candidate regions. If the determination is negative, the process returns to step 160, and steps 160 to 176 are repeated for all face candidate regions. If the determination in step 176 is affirmed, the background area determination process is terminated, and the process proceeds to step 146 in the flowchart of FIG. Note that the weight point P finally set for each face candidate area through the above-described processing corresponds to an evaluation value representing the final evaluation of the accuracy of each face candidate area as an area corresponding to a human face. ing.
[0090]
In step 146, the weight point P of each face candidate area is set to the threshold TH for face area determination._FAnd the weight point P is the threshold value TH._FThe above face candidate areas are extracted (selected) as face areas. For face candidate areas that have a high degree of coincidence with the matching pattern and a low background area ratio and low degree of uneven distribution, the final weight score P is high, so that the face candidate has a high probability of being a face area. An area is extracted as a face area.
[0091]
In the next step 148, the face area density Mface of the image to be processed is calculated according to the following expression (1) or (2), and the face area extraction / density calculation process is terminated.
[0092]
[Expression 1]

Where i is a code for identifying each face candidate area, N is the total number of face candidate areas, M_iIs the density of the face candidate region i, P_iIs the weight score of the face candidate area i, S_iIs the area of the face candidate region i.
[0093]
As is clear from the expressions (1) and (2), the face area density Mface is a weighted average value of the density M of each face candidate area, and in the expression (1), based on the weight point P of each face candidate area. Each face candidate area is weighted, and each face candidate area is weighted based on the number of weight points P and area S in equation (2).
[0094]
When the face area extraction / density calculation process is performed, the auto setup engine 44 further calculates various image processing conditions executed by the image processor 40. The processing result of the face area extraction / density calculation process is as follows. Is used to calculate processing conditions for some image processing. For example, the face area extracted in the previous step 146 is used for calculation of image processing (for example, sharpness correction or red-eye correction for the face area, etc.) targeting only the face area executed by the image processor 40 or a part thereof. The processing conditions are set so that the image processing is performed only on the face area. The face area density Mface calculated in the previous step 148 is used for image processing (for example, color / density correction) for the entire image executed by the image processor 40, for example. Processing conditions such as density correction conditions are calculated so as to obtain a predetermined density.
[0095]
As described above, since the face area extraction and the face area density Mface are performed using the weight points set based on the degree of matching with the matching pattern, the face extracted by the face candidate area extraction process Even if face candidate areas that are not actually face areas are mixed due to erroneous extraction in the candidate areas, the probability that face candidate areas that are not actually face areas will be extracted as face areas is greatly reduced. In fact, it is possible to prevent the face area density from significantly changing depending on the density of the face candidate area that is not a face area. Therefore, an appropriate processing condition is obtained for each image processing in which the processing condition is calculated using the face region extraction result or the face region density Mface, and is executed by the image processor 40 on the fine scan image data. An appropriate processing result can be obtained for each image processing.
[0096]
[Second Embodiment]
Next, a second embodiment of the present invention will be described. Since the second embodiment has the same configuration as that of the first embodiment, the same reference numerals are given to the respective parts, and the description of the configuration is omitted. Hereinafter, the face area extraction / decoding according to the second embodiment will be described. Concerning the density calculation process, only the parts different from the first embodiment will be described with reference to the flowchart of FIG.
[0097]
The face area extraction / density calculation process according to the second embodiment is a process to which the image processing method according to the invention of claim 3 is applied, and the face area extraction / density correction is performed by the CPU 46 of the auto setup engine 44. This is realized by executing the program. The face area extraction / density correction program is initially stored in the information storage medium 72 (see FIG. 1), and the information storage medium 72 is loaded into an information reading device (not shown) connected to the personal computer 42. When an instruction to transfer a program from the information storage medium 72 to the image processing apparatus 14 is given, a face area extraction / density correction program or the like is read from the information storage medium 72 by the information reading apparatus, and the stored contents can be rewritten in the ROM 50. Remembered. When it is time to execute the face area extraction / density correction process, the face area extraction / density correction program is read from the ROM 50, and the face area extraction / density correction program is executed by the CPU 46. Thus, the auto setup engine 44 functions as an image processing apparatus according to the invention of claim 7. As described above, the information storage medium 72 according to the second embodiment corresponds to the recording medium according to the ninth aspect.
[0098]
In the face area extraction / density calculation process according to the second embodiment, a face candidate area extraction process is performed (step 100), the presence / absence of top / bottom information is determined (step 102), and if there is top / bottom information, the process proceeds to step 200. Then, the data of a single face candidate area is captured from the face candidate areas extracted by the face candidate area extraction process. In step 202, whether or not there is a density pattern area convex in the high density direction and a density pattern area convex in the low density direction (high luminance direction) with respect to the face candidate area in which the data is captured in step 200. Explore. Step 202 corresponds to the second extraction means described in claim 7.
[0099]
Of the face area corresponding to a human face, an area having a density equal to or higher than a predetermined value (high density area) is an area corresponding to the eye and its surroundings (eye area) as shown in FIG. 10A as an example. As shown in FIG. 10B, the density change in the eye region is a change pattern convex in the high density direction with a steep slope. Since the eyeball area, eyelashes, eyelashes, eyebrows, etc. are present in the eye area, the actual density change in the eye area is more complicated. Since the periphery is depressed, the approximate density change in the eye region shows a change pattern as shown in FIG.
[0100]
In addition, as shown in FIG. 11A, for example, an area having a luminance equal to or higher than a predetermined value (high luminance area) in a face area corresponding to a person's face is an area corresponding to a cheek (cheek area) or The luminance change in the region corresponding to the nose and the region corresponding to the forehead, for example, the luminance change in the cheek region is a change pattern convex in the high luminance direction with a gentle inclination as shown in FIG. ing. Accordingly, if the face candidate area to be processed is a face area corresponding to a human face, the eye area is extracted as a density pattern area convex in the high density direction by the process of step 202, and the low density direction Thus, the cheek region is extracted as the region of the convex density pattern.
[0101]
In the next step 204, it is determined whether or not a density pattern has been found by the search in step 202. If the determination is negative, it is very likely that the face candidate area to be processed is not a face area, and therefore, in step 206, 0 is substituted for the degree of matching as a face area, and the process proceeds to step 210. On the other hand, if the determination in step 204 is affirmed, the process proceeds to step 208, and the position and area (density pattern) in the face area of the density pattern area found and extracted in step 202 on the basis of the top and bottom direction indicated by the top and bottom information. Based on the shape of the density histogram for the area and how the density changes in the density pattern area (for example, the slope of the density change or the ratio of the height of the mountain to the width of the base in the density pattern). The degree of matching is determined as a region corresponding to the eye part and cheek part for all the extracted density pattern regions.
[0102]
That is, when the face candidate area to be processed is a face area, a density pattern area convex in the high density direction corresponding to the eye area exists, for example, in each block pair indicated by hatching in FIG. The area ratio between the density pattern area and the face candidate area to be processed is also likely to be within a predetermined numerical range corresponding to the area ratio as the eye area. Accordingly, the degree of matching as a region corresponding to the eye part of the density pattern region convex in the high density direction is, for example, a range in which the eye region should exist with reference to the top / bottom direction represented by the top / bottom information (FIG. 4B ) Is set to the block pair indicated by hatching), the degree of coincidence between the set range and the position of the density pattern area is obtained, and the area ratio between the density pattern area and the face candidate area to be processed is determined as the eye area. The degree of coincidence can be obtained by comparison with a predetermined numerical range corresponding to the area ratio, and a determination can be made using a two-dimensional map or the like based on the degree of coincidence.
[0103]
When the face candidate area to be processed is a face area, a density pattern area convex in the low density direction (high luminance direction) corresponding to the cheek area is, for example, below the block indicated by hatching in FIG. The area ratio between the density pattern area and the face candidate area to be processed is also likely to be within a predetermined numerical range corresponding to the area ratio as the cheek area. . Therefore, for the degree of matching as a region corresponding to the cheek portion of the density pattern region convex in the low density direction, for example, a range in which the cheek region should exist with reference to the top and bottom direction represented by the top and bottom information (FIG. 4 ( B) is set to a block adjacent to the lower part of the block indicated by hatching, and the degree of coincidence between the set range and the position of the density pattern area is obtained, and the density pattern area and the face candidate area to be processed are The degree of coincidence can be obtained by comparing the area ratio with a predetermined numerical range corresponding to the area ratio as the cheek region, and can be determined using a two-dimensional map or the like based on the degree of coincidence.
[0104]
In step 209, the degree of matching as a face area is calculated and set for the face candidate area to be processed based on the degree of matching determined for each extracted density pattern area, and the process proceeds to step 210. As the matching degree of the face area, for example, a total sum of matching degrees for each density pattern area can be used.
[0105]
In the above description, the degree of matching is determined as an area corresponding to the eye or cheek for the extracted density pattern area. However, the nose and the forehead are particularly suitable for the density pattern area convex in the low density direction. It is also possible to determine the matching degree as a region corresponding to. However, while there are a pair of eyes and cheeks at substantially symmetrical positions in the face area, there is only one nose and forehead at substantially the center along the left and right direction of the face area. Even if the degree of matching as a region corresponding to the nose or forehead is high for a specific density pattern region, the density pattern region is not actually a region corresponding to the nose or forehead, and the matching degree is increased by chance. Therefore, the reliability of the matching degree determination is slightly lower than that of the eye or cheek. For this reason, in setting the degree of matching as a face area with respect to the face candidate area, the degree of matching as an area corresponding to the nose or forehead may be set to be reflected in the degree of matching as a face area with a low weight. desirable.
[0106]
In the next step 210, it is determined whether or not processing (determination of matching degree) has been performed for all face candidate regions. If the determination is negative, the process returns to step 200, and the processing / determination of steps 200 to 208 is performed for each face candidate region. As a result, the degree of matching as a face area is determined and set for each face candidate area. If the determination at step 210 is affirmative, the routine proceeds to step 244.
[0107]
On the other hand, when the determination in step 102 is negative (when there is no top and bottom information), the process proceeds to step 220, where a single face candidate is selected from the face candidate areas extracted by the face candidate area extraction process in step 100. After capturing the area data, in the next step 222, as in the previous step 202, there is a density pattern area convex in the high density direction and a density pattern area convex in the low density direction (high luminance direction). Search whether or not. This step 222 also corresponds to the second extraction means described in claim 7.
[0108]
In step 224, it is determined whether or not a density pattern has been found by the search in step 222. If the determination is negative, it is very likely that the face candidate area to be processed is not a face area, and therefore, in step 226, 0 is substituted for the degree of matching as a face area, and the process proceeds to step 238.
[0109]
On the other hand, if the determination in step 224 is affirmed, the process proceeds to step 228 and 1 is substituted for the variable n. In the next step 230, the nth direction (in this case, the first direction) is used as a reference (top and bottom direction). Based on the position and area of the density pattern area found and extracted in step 222 in the face area, for all the extracted density pattern areas, the degree of matching as the area corresponding to the eye part and the area corresponding to the cheek part The degree of matching is determined. In the next step 231, based on the degree of matching determined for each extracted density pattern area, the degree of matching as a face area is calculated for the face candidate area to be processed in the same manner as in the previous step 209. Set.
[0110]
In step 232, it is determined whether or not the value of the variable n has become 4. If the determination is negative, the value of the variable n is incremented by 1 in step 234 and the process returns to step 230. Thus, in steps 230 and 231, assuming that the first to fourth directions are vertical directions, the degree of matching as a face area is determined for each face candidate area to be processed.
[0111]
When the degree of matching is determined for each direction, the determination in step 232 is affirmed, the process proceeds to step 236, and the direction in which the degree of matching is the maximum among the first to fourth directions is stored. In the next step 238, it is determined whether or not all face candidate areas have been processed. When determination is denied, it returns to step 220 and repeats steps 220-238. As a result, the degree of matching as a face area is determined for each of the face candidate areas in the first to fourth directions.
[0112]
If the determination in step 238 is affirmative, the process proceeds to step 240, and in the same manner as step 138 described in the first embodiment, based on the direction in which the degree of matching stored for each face candidate region in the previous step 236 is the maximum. To determine the vertical direction of the image. In the next step 242, the matching degree as the face area for each face candidate area in the predetermined direction corresponding to the determined vertical direction is set in each face candidate area, and the process proceeds to step 244.
[0113]
In step 244, a weight score is set for each face candidate area according to the matching degree set for each face candidate area. This step 244 corresponds to the evaluation means described in claim 7 together with steps 208, 209, 230, 231, and 242. Regarding the setting of the weight points in step 244, as shown in FIG. 6A and FIG. 6B as an example, the conversion characteristics are determined so that the weight points P increase as the degree of matching increases. This can be done by using a map and converting the degree of matching into the number of weight points P using the map. In addition, since the process after the following step 144 is the same as that of 1st Embodiment, description is abbreviate | omitted.
[0114]
In the above, the density pattern region where the density pattern peculiar to the eye part and the cheek part is generated is extracted, and the extracted density pattern region is extracted based on the position and area ratio of the extracted density pattern region in the face candidate region. The degree of matching as the area corresponding to the eye part and the degree of matching as the area corresponding to the cheek part are determined to determine the degree of matching as the face area with respect to the face candidate area, and the weight score for the face candidate area is set as the degree of matching. Therefore, the accuracy as the face area can be accurately evaluated based on the internal structure of the person's face with respect to the face candidate area. Further, in the above, even when there is no edge at the outer edge of the area corresponding to the eye part or cheek part, it is possible to prevent the accuracy of the evaluation for the candidate area from being lowered, and the area division based on binarization can be performed. Thus, since it is not necessary to repeat the process while changing the threshold value, the process is also simplified.
[0115]
In the above, the face candidate area satisfying at least one of the determination of whether the area ratio of the background candidate area is equal to or greater than the first predetermined value and whether the uneven distribution degree of the background candidate area is equal to or greater than the second predetermined value. On the other hand, the weight points are corrected so as to decrease. However, the present invention is not limited to this, and when performing a process of extracting a face area from a face candidate area, For a face candidate region that satisfies one determination, the threshold value may be changed so that the threshold value for determining the face region is increased as shown in FIG. 8B as an example. Thereby, it is possible to make it difficult to extract a face candidate area that is relatively likely to be a background area as a face area. Threshold TH_FThe change amount of the threshold value TH may be a constant value, or the threshold TH according to the difference between the area ratio of the background candidate region and the first predetermined value, and the difference between the degree of uneven distribution of the background candidate region and the second predetermined value._FThe amount of change may be changed.
[0116]
Further, the number of weight points P set for each face candidate area, the threshold TH for face area determination_FAlternatively, the weight given to the density M of each face candidate area in the face area density Mface may be changed according to the type of image processing performed using the processing result of the face area extraction / density calculation process. .
[0117]
For example, using the result of face area extraction by face area extraction / density calculation processing, the image processor 40 applies an edge enhancement filter only to the extracted face area to sharpen the sharpness of the face area. When enhancement processing is performed, it depends on the degree of sharpness enhancement and the type of filter, but even if sharpness enhancement is actually performed on areas that are not facial areas, the visual impact is small (not noticeable). There is. In such a case, the threshold TH for determining the face area_FMay be made smaller than usual (that is, the criteria for selecting a face candidate area is changed), and more face candidate areas may be determined as face areas. Threshold value TH for face area determination_FAs the value of is decreased, the probability that the face candidate area corresponding to the actual face area is erroneously determined not to be a face area decreases, so that sharpness enhancement processing is performed on the face area in the image without omission as described above. be able to.
[0118]
Further, the threshold TH for face area determination_FInstead of changing the value of, a larger value than usual is set as the weight point P for the degree of matching described in the first embodiment and the degree of matching described in the second embodiment (that is, each face candidate region) It is also possible to determine more face candidate areas as face areas by changing the evaluation criteria for. In particular, when sharpness enhancement processing is performed to increase the sharpness enhancement degree as the weight point P increases, the sharpness enhancement degree is controlled to be stronger by setting the weight point P as described above. Is also possible.
[0119]
Further, for example, by using the face area extraction result and the face area density Mface by the face area extraction / density calculation processing, the density correction for locally correcting the density only for the extracted face area based on the face area density Mface. When processing is performed, although depending on the degree of density correction, even if density correction is actually performed on an area that is not a face area, there is a case where an adverse visual effect is small (not noticeable). In such a case, the threshold TH for determining the face area_FMay be made smaller than usual so that more face candidate areas are determined as face areas. Threshold value TH for face area determination_FSince the probability that a face candidate area corresponding to an actual face area is erroneously determined not to be a face area decreases as the value of is reduced, density correction processing is performed without omission on the face area in the image. be able to.
[0120]
In the above description, in the face area extraction, an image process that has a small influence is performed even when an area that is not actually a face area is mistakenly extracted as a face area. When image processing that is greatly affected when an area is mistakenly extracted as a face area is performed, for example, a threshold TH for determining a face area_FBy setting a larger value than normal, or by setting a smaller value than normal for the degree of coincidence and consistency, a face candidate area with higher accuracy as a face area is extracted as a face area. It is also possible to do so.
[0121]
As for the face area density, for example, as shown in the following expression (3), the face area density Mface obtained by the above expression (1) (or expression (2) may be used) and other image feature amount D (for example, Weighted average value Mface ′ (where α is the average density of the entire image, average density of the non-face candidate area, etc.)_FIs a weighting factor for the face area density Mface, α₀Is a weighting coefficient for the image feature amount D) as a face area density, the weighting coefficient α is determined according to the type of image processing performed using the calculated face area density._F, Α₀The weight given to the density M of each face candidate area may be changed by changing the value of (i.e., relatively changing the weighting criterion for each face candidate area).
[0122]
Mface ’= α_F・ Mface + α₀・ D (3)
In addition, as image processing performed using the processing result of the face area extraction / density calculation process, there are a plurality of types of image processing (for example, extracted face areas that have different requirements for the processing result of the face area extraction / density calculation process). Image processing in which it is desirable that no non-face area is actually mixed, and image processing in which all the face areas in the image are preferably included in the extracted face area) When performed, face area extraction and face area density calculation may be performed a plurality of times corresponding to each image processing. In the present embodiment, the degree of matching and the degree of matching can be used as the reliability (accuracy) of each face candidate area as the face area. As described above, the criterion for setting the weight score for each face candidate area, the face area determination Criteria (threshold TH_F) By changing at least one of the weighting criteria for each face candidate area, the results required by the respective image processing can be obtained as face area extraction results and face area density calculation results. Even when each of the image processes is performed, the extremely complex and time-consuming face candidate region extraction process is repeated the same number of times as the number of types of image processes while changing the processing conditions corresponding to the plurality of types of image processes. This is not necessary, and the processing time of face area extraction / density calculation processing can be shortened, and the performance of the image processing apparatus 14 can be improved.
[0123]
In the above description, the processing conditions including face area extraction / density calculation processing are calculated by the auto setup engine 44 based on the pre-scan image data, and the actual image processing for the fine scan image data is performed by the image processor 40. However, the present invention is not limited to this. Processing conditions may be calculated for a single image data, and image processing under the calculated processing conditions may be performed in order. You may make it perform in a process part.
[0124]
Further, in the above, the extraction of the face area and the calculation of the face area density are performed based on the weight points set for each face candidate area, but the present invention is not limited to this, and only one of them is performed. You may do it.
[0125]
In the above, image data obtained by reading an image recorded on a photographic film is a processing target. However, the present invention is not limited to this, and an image recorded on another recording material such as paper is read. Image data obtained in this way, image data obtained by imaging with a digital camera, or image data generated by a computer may be processed. Further, it goes without saying that the present invention may be used for determining exposure conditions when a film image recorded on a photographic film is exposed and recorded on photographic paper by surface exposure.
[0127]
【The invention's effect】
  Claim 1 as described above, 6,8The described invention divides a candidate area estimated to correspond to a human face in an image into a predetermined number of small areas, and features related to the frequency and magnitude of changes in density or luminance in the small areas Amount,For each small areaEach in multiple directionsThe pattern corresponding to the feature amount relationship for each small area when the area corresponding to the human face is divided into a predetermined number of small areasThe degree of coincidence is calculated, and the vertical direction of the image is determined based on the calculation result of the degree of coincidence for a plurality of directions, and based on the calculation result of the degree of coincidence for the determined vertical direction.Since we evaluate the accuracy of the candidate area as the area corresponding to the person's face,Even if the top and bottom direction of the image is unknown,Based on the internal structure of the human face,In addition to accurately determining the vertical direction of the image,It has an excellent effect that a region corresponding to a human face can be extracted with high accuracy by simple processing.
[0128]
  In the invention of claim 2, in the invention of claim 1, the division target area is divided so that the areas corresponding to the individual eye parts constituting the eye pair of the person's face are located in different small areas. In addition to the above-described effects, there is an effect that it is possible to more accurately evaluate the accuracy of the candidate region as a region corresponding to a person's face.
[0129]
  According to the third and seventh aspects of the present invention, the density or luminance pattern specific to a specific part of the human face is based on the density or luminance distribution in the candidate area estimated to correspond to the human face. Extracts the generated region and corresponds to a specific portion of the extracted region based on at least one of the position of the extracted region in the candidate region, the area ratio with the candidate region, the density or luminance histogram shape Alignment as a regionEvery timeTheCalculationDoThis is performed for each of a plurality of directions, and the top / bottom direction of the image is determined based on the calculation result of the matching degree for the plurality of directions, and based on the calculation result of the matching degree for the determined top / bottom direction.Since we evaluate the accuracy of the candidate area as the area corresponding to the person's face,Even if the top and bottom direction of the image is unknown,Based on the internal structure of the human face,In addition to accurately determining the vertical direction of the image,It has an excellent effect that a region corresponding to a human face can be extracted with high accuracy by simple processing.
[0130]
According to a fourth aspect of the present invention, in the third aspect of the invention, a region where a convex density or luminance pattern is generated in the high density direction or the low luminance direction is extracted, and the region corresponding to the eye part of the extracted region is obtained. The region where convex density or luminance pattern is generated in the low density direction or high luminance direction is extracted, and the consistency as the region corresponding to the cheek part of the extracted region is determined. Therefore, in addition to the above effect, there is an effect that the accuracy of the candidate region as the region corresponding to the person's face can be evaluated with higher accuracy.
[0131]
  The invention according to claim 5In the invention of claim 1 or claim 2,When the brightness of the candidate area estimated to correspond to the main part in the image is greater than or equal to a predetermined value, a background candidate area composed of pixels whose brightness difference with the candidate area is within a predetermined range is set within a range outside the candidate area. Corresponds to the main part for the candidate area when the area ratio of the searched and extracted background candidate area to the candidate area is greater than or equal to a predetermined value, or when the extracted background candidate area is unevenly distributed in the periphery of the image Since the evaluation of the accuracy as a region to be performed is lowered, it can be prevented that the region corresponding to the background in the image is erroneously extracted as the region corresponding to the main part.EffectHas fruit.
[Brief description of the drawings]
FIG. 1 is a schematic configuration diagram of an image processing system according to an embodiment.
FIG. 2 is a flowchart showing the contents of face area extraction / density calculation processing according to the first embodiment;
FIG. 3 is a flowchart showing the contents of background area determination processing;
4A is a conceptual diagram showing an example of a matching pattern for collating with a predetermined number of blocks obtained by the division, and FIG.
FIG. 5 is a conceptual diagram illustrating an example of a differential filter for obtaining a density change value (edge strength).
FIGS. 6A and 6B are diagrams each showing a map for setting the number of weight points in the face candidate region in accordance with the degree of matching with the matching pattern.
FIG. 7 is an image diagram illustrating an example of an image with a high possibility that a background area is erroneously extracted as a face candidate area;
8A is a diagram showing a map for correcting the number of weight points for a face candidate area that is relatively likely to be a background area, and FIG. 8B is a diagram in the case where the face candidate area exists. It is a diagram explaining the change of the threshold value for face area determination.
FIG. 9 is a flowchart showing the contents of face area extraction / density calculation processing according to the second embodiment.
FIG. 10A is an image diagram showing an example of a distribution of a high density region where the density in the face region is equal to or higher than a predetermined value, and FIG. FIG.
FIG. 11A is an image diagram showing an example of a distribution of a high-luminance region in which the luminance in the face region is equal to or higher than a predetermined value, and FIG. FIG.
[Explanation of symbols]
10 Image processing system
14 Image processing device
40 Image processor
44 Auto Setup Engine
72 Information storage media

Claims (9)

Based on the image data, a candidate area estimated to correspond to a human face in the image represented by the image data is extracted,
With dividing the candidate regions the extracted predetermined number of small regions, the feature quantities relating to the magnitude of the frequency and the change of the change in density or brightness in the small region, determined each plurality of directions in each small area ,
A pattern representing the relationship between the feature quantities for each small area when the area corresponding to the face of a person is divided into the predetermined number of small areas for each of the small areas for each of the plurality of directions. And collating with each other to calculate the degree of coincidence, determining the vertical direction of the image based on the calculation result of the degree of coincidence for the plurality of directions, and based on the calculation result of the degree of coincidence for the determined vertical direction An image processing method for evaluating the accuracy of a candidate area as an area corresponding to a human face.   2. The division target area is divided into the predetermined number of small areas so that areas corresponding to individual eye parts constituting a pair of human face eyes are located in different small areas. The image processing method as described. Based on the image data, a candidate area estimated to correspond to a human face in the image represented by the image data is extracted,
Based on the density or luminance distribution in the extracted candidate area, extract a region where a specific density or luminance pattern is generated in a specific part of the person's face,
Matching the extracted region as a region corresponding to the specific portion based on at least one of a position in the candidate region, an area ratio with the candidate region, a shape of density or luminance histogram The degree is calculated for each of a plurality of directions, and the vertical direction of the image is determined based on the calculation result of the matching degree for the plurality of directions, and based on the calculation result of the matching degree for the determined vertical direction. an image processing method of assessing the accuracy of a region corresponding to a person's face in the candidate region Te. Extracts areas with convex density or luminance patterns in the high density direction or low luminance direction peculiar to the eye part of a person's face, and determines the consistency of the extracted area as the area corresponding to the eye part Or,
Alternatively, a region where a pattern having a convex density or luminance in the low density direction or high luminance direction peculiar to the cheek portion of a person's face is extracted, and the consistency of the extracted region as the region corresponding to the cheek portion is extracted. The image processing method according to claim 3, wherein the determination is performed. When the brightness of the candidate area is greater than or equal to a predetermined value, a background candidate area consisting of pixels whose brightness difference with the candidate area is within a predetermined range is searched within a range outside the candidate area,
When the background candidate area is extracted and the area ratio of the extracted background candidate area to the candidate area is a predetermined value or more, or when the extracted background candidate area is unevenly distributed in the peripheral portion of the image, The image processing method according to claim 1, wherein the evaluation of the accuracy as an area corresponding to a main part with respect to the candidate area is lowered. First extraction means for extracting a candidate area estimated to correspond to a human face in an image represented by the image data based on the image data;
While dividing the candidate region extracted by the first extracting means into a predetermined number of small regions, the feature quantities relating to the magnitude of the frequency and the change of the change in density or brightness in the small area, each small area per Computing means for each of a plurality of directions ,
The feature amount obtained for each of the plurality of directions for each of the small regions by the calculation means, and the feature amount for each small region when the region corresponding to the face of a person is divided into the predetermined number of small regions The degree of coincidence is calculated by collating each pattern with the pattern representing the relationship between the plurality of directions, and the top and bottom direction of the image is determined based on the result of the degree of coincidence calculation for the plurality of directions. Evaluation means for evaluating the accuracy of the candidate area as an area corresponding to a person's face based on the result ;
An image processing apparatus. First extraction means for extracting a candidate area estimated to correspond to a human face in an image represented by the image data based on the image data;
Second extraction for extracting a region in which a specific density or luminance pattern is generated in a specific part of a human face based on the density or luminance distribution in the candidate region extracted by the first extraction unit Means,
Said extracted region by the second extraction means, the position in the candidate area, the area ratio of the candidate region based on at least one of the shape of the histogram of the density or brightness, the specific of the extracted area And calculating the degree of matching as a region corresponding to the portion for each of a plurality of directions, and determining the top and bottom direction of the image based on the calculation result of the degree of matching for the plurality of directions, and the determined top and bottom direction and evaluating means for evaluating the accuracy of a region corresponding to a person's face of the candidate region based on alignment of the calculation results for,
An image processing apparatus. Computer
First extraction means for extracting a candidate area estimated to correspond to a human face in an image represented by the image data based on the image data;
The candidate area extracted by the first extracting means is divided into a predetermined number of small areas, and the feature quantity related to the frequency of change in density or luminance in the small area and the magnitude of the change are determined for each small area. Computing means for each of a plurality of directions,
And the feature amount obtained for each of the plurality of directions for each of the small areas by the computing means, the area corresponding to the face of a person is divided into the predetermined number of small areas, The degree of coincidence is calculated by comparing each of the patterns representing the relationship between the feature amounts, and the vertical direction of the image is determined based on the calculation result of the degree of coincidence for the plurality of directions. Evaluation means for evaluating the accuracy of the candidate area as an area corresponding to the face of a person based on the calculation result of
A recording medium on which a program for causing the program to function is recorded. Computer
First extraction means for extracting a candidate area estimated to correspond to a human face in an image represented by the image data based on the image data;
Second extraction for extracting a region in which a specific density or luminance pattern is generated in a specific part of a human face based on the density or luminance distribution in the candidate region extracted by the first extraction unit means,
And, based on at least one of the position of the region extracted by the second extracting means in the candidate region, the area ratio with the candidate region, the shape of the histogram of density or luminance, the extracted region The degree of matching as a region corresponding to the specific part is calculated for each of a plurality of four directions, and the vertical direction of the image is determined based on the calculation result of the degree of matching for the plurality of directions. Evaluation means for evaluating the accuracy of the candidate area as an area corresponding to the face of a person based on the calculation result of the degree of matching for the vertical direction
A recording medium on which a program for causing the program to function is recorded.

Images