JP2004038480A - Method and apparatus for recognizing image, computer program, and computer readable recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect an image of interest from an image file using a method that imposes little processing load. <P>SOLUTION: For each predetermined block, spatial frequency information and chromaticity information are obtained from image data recorded by compressed recording and are used to search for an image of interest in the image data, whereby information including ac component information for each image data block is obtained, without advanced calculations, and used to search for the image of interest in the image file. Pixels of interest can be detected using the method that imposes little processing load. <P>COPYRIGHT: (C)2004,JPO

Description

【００３４】
本実施の形態においては、処理の簡便さを考慮に入れた下記式（４）である色度分布範囲を肌色の色度範囲とした。この範囲を表したのが図２０である。
【００３５】
【数２】

【００３６】
本実施の形態においては、画像における周波数成分の特徴を検出する単位として８＊８画素単位のブロックで行っている関係で、構造的論理的な簡単さより色度判定においても８＊８画素単位にて実行する。
【００３７】
図７は、本実施の形態で用いている色度検出ポイントを図示したものである。これによると「８＊８画素」単位のブロックの四隅の色度の全てが色度範囲に入っているか否かを確認し、全てが範囲に入っている時は、そのブロックを適合色度と判定している。
【００３８】
図７においては、上段の左から２番目と下段の左から１，２，３ブロックが該当する。上段の一番左のブロックは４ポイントのうち左上の式度は非肌色ピクセルと判定されるので、これを含むブロックは肌色の範囲外と判定される。同じように上段の右側１，２ブロックと下段の一番右のブロックが範囲外となる。
【００３９】
図８は、「８＊８画素」単位のブロック全体の平均色度による判定である。このブロック内の平均色度の求め方としては、８＊８ブロック全ての画素値の平均値を取る方法の他に、解凍中の逆ＤＣＴを行う前の色度データ（Ｃｂ，Ｃｒ）の中のＤＣ成分から求めることも可能である。この方式の利点としては、ブロック全体の色調にて判定できるので、検出点の少ないものに比べて精度が高い期待ができる。ここで、自然画における色度のみの検出についての内容を見ることにする。
【００４０】
図９は、図７と同じ考えの中ではあるが、全体画像における検出間隔を等分化するためのものである。
【００４１】
図１０は、一般的なポートレート写真であり、図１４は人物の肌色色度と同様な色度範囲を有する枯木の林の写真である。図１０と図１４に対して、それぞれの画素に色度の適合だけで検出を行った結果を図１１と図１５に示す。
【００４２】
図１１のポートレートでの検出結果としては人物の肌色部分をよく検出しているが、その他に柵や背景の中で、ごみのような細かい部分においても適合色度を満たすものが検出されていることがわかる。このため、色度のみでは注目画像を特定できないことがわかる。
【００４３】
図１４においては、人物の肌色を検出する目的にもかかわらず同じ色度を持つ枯れ木の林が全面検出されている。このように、画素レベルでの色度判定を行った場合、注目画像を特定することは不可能である。
【００４４】
検出をブロックレベルにすることにより、特定のまとまりを持った状態が対象になるので、外来ノイズの影響は受けずらくなる。しかしながら、８＊８画素のブロックが適正なまとまりの大きさとは言えず、色度によるブロック検出においても縦方向及び横方向に隣接したブロックの連続検出という、制約を付けた検出を行うことで更に精度を上げることになる。
【００４５】
ここにおいて、人の肌色であってもプリントにおいて顔を認識できるデータ量を満たさないものにおいても適応外としてはじいても良いと言う概念でノイズと判定する連続範囲を設定する。
【００４６】
この部分を表したのが、図６のステップＳ６０３以降の処理である。すなわち、ステップＳ６０３においては、画像に対して長手方向にブロックごとに色度検出を行い、連続検出ブロック数の多い順に候補を策定する。
【００４７】
次に、ステップＳ６０４において、その連続量が、注目画像としての適応する連続量に入っているか否かを比較する。この比較の結果、該当する連続ブロックがある場合はステップＳ６０５に進み、短い方向のブロック連続検出設定を満たすデータが画像に存在するか否かを検索を行う。
【００４８】
次に、ステップＳ６０６において、検出データが有るか否かを判断し、検出データが有る場合にはステップＳ６０８に進んで、この過程で残ったものの中から長手方向の連続ブロック量が大きいデータから順に候補番号を付ける。
【００４９】
また、ステップＳ６０６の判断の結果、検出データが無い場合にはステップＳ６０７に進み、「目的領域無し」をセットして処理を終了する。
【００５０】
ここで、話は少し戻るが、連続ブロックにて色度判定を施した場合の効果については、図１２と図１６で示す。
図１２においては、図１０のポートレート画像に対して検出を行った結果である。図１２において、検出候補の優先順位が高い方からカラーコード（１＝茶、２＝赤、３＝橙、４＝黄、５＝緑、６＝青、７＝紫、８＝灰）順に配置され、それ以外で検出されているのは色度のみ適性範囲に入っているものである。連続ブロック検出により画素レベルの色度検出と比べるとかなりの背景などの非該当候補を削除できていることが判る。
【００５１】
図１６においては、図１４の枯木の林に対して検出を行った結果で、連続ブロック検出においても注目画像以外を検出してしまうことがわかる。
【００５２】
次に、ＶＧＡ（ｖｉｄｅｏ　ｇｒａｐｈｉｃｓ　ａｒｒａｙ）サイズ（６４０＊４８０画素）の複数の画像サンプルを用いて人物肌と枯れ木の林の部分において、検出された適合色度連続ブロックにおける周波数特性を算出した。
【００５３】
図１８は、画像内に撮影されている人物肌の連続ブロック検出されたブロックのＤＣＴデータを周波数の低い順に並べたものを、周波数の低い方から１０個単位で加算し、連続ブロック数で除したもので、連続検出されたブロックの１個あたりの平均周波数成分をまとめたものである。
【００５４】
したがって、図面において横軸は、ＡＣ成分６３個の周波数成分をまとめたもので、１０個単位のまとまりが６グループと最も周波数の高いデータは３個分のデータとなる。縦軸は、各周波数成分の要素を加算した値である。
【００５５】
これにより、値が大きいほどそのブロックにおいて、該当周波数成分が高いことがわかる。また、検出した連続ブロック数ごとに色分けしたデータ線で表されている。例えば“Ｂ２”は連続ブロックが２個検出されているデータの平均した値を表し、“Ｂ１５”は連続ブロックが１５個検出されているデータの平均した値を表している。以下同じで、“Ｂ２〜Ｂ１５”までの複数画像からの平均的な人物肌色部分の連続検出値ごとの空間周波数特性を表している。
【００５６】
検出結果を見ると、
１）低い周波数成分の値が大きく低い周波数成分の下から３グループ以降は、連続ブロック数に係わり無く５０以下となっている。
２）連続ブロックの連続値が大きいほど周波数特性が低くなっている。
【００５７】
これらの結果から言えることは、人物の肌色部分の周波数特性は比較的低い周波数で構成されていることと、検出された連続ブロックの値が大きいことは、被写体の撮影された大きさが大きいことを示していて、この連続ブロックとしての平均値を出すことによって周波数成分が下がっていることがわかる。
【００５８】
連続ブロックの連続値により、同じ注目画像の色度を持っているものでも、その連続ブロックを１つの代表値にすること（例えば、Ｂ６のブロックの時は検出した６個のブロックの値を、各々周波数の低い順に１０個単位のグループとして加算したものをグループごとに加算した後、その連続値である６で除して平均を出している。）により、空間周波数特性の値が変わるので、連続検出値により適当な該当周波数特性が違うことが判る。
【００５９】
図１９は、人物の肌色色度と同様な色度範囲を有する枯木の林の写真を複数用意して、検出を行った結果を図１８と同じように表したものである。
【００６０】
検出結果を見ると、
１）人物の肌の空間周波数特性と比べると高い周波数成分にデータ多くあることが確認できる。
２）一番低い周波数成分のグループは人物の肌の結果と大きくは違わない。
【００６１】
これらのことから、連続ブロックにおける周波数成分を検出することで、同じ色度を持った検出物体を周波数特性により区別することが可能であることがわかる。
【００６２】
図４は、本実施の形態において使用したもので、注目画像である人物肌の空間周波数特性を表したものである。上の段がＶＧＡ（６４０＊４８０）画像における周波数特性の適正範囲である。
【００６３】
連続ブロック値を２〜８個のグループ（〜Ｌ８）と９〜２０個のグループ（Ｌ９〜２０）と２１個以上のグループ（Ｌ２１〜）の３グループにまとめて、グループごとに周波数の適正範囲を設定したものである。周波数の適正範囲も先に示した１０個単位の７グループによる周波数特性を用いた。これは、処理の簡略化と検出精度のバランスで行ったもので、これに縛られる必要は無い。
【００６４】
図６の説明に戻る。上述したように、色度により検出された長手方向の連続量が大きいデータから順に注目画像の候補番号１〜ｎ（本実施の形態においてはｎ＝８）を付ける（ステップＳ６０８）。ｎ以降の検出したものについては候補番号を付けられない。
【００６５】
次に、ステップＳ６０９に進み、上記候補１〜ｎに対して、図４で示した連続ブロック数に対する空間周波数特性適正範囲判定表の範囲に適合するか逐次比較する。この結果、適合する候補が存在しない場合は注目画像が存在しないと判断する。
【００６６】
また、適合する候補が存在する場合においては、以下に説明を行う。
図２２に、そのフローチャートを示す。
最初のステップＳ２２０１において、候補の数を確認する（１〜ｍ）。
次に、ステップＳ２２０２に進み、候補グループを形成する。この場合、候補に隣接する色度適合ブロックを候補グループとする。
【００６７】
次に、ステップＳ２２０３に進み、候補グループが複数であるか否かをを判断する。この判断の結果、候補グループの中に複数の候補が含まれた場合は、ステップＳ２２０４に進み、候補番号の若い方の番号を用いたグループとする。
【００６８】
そして、検出された各グループに対して、どちらのグループが補正対象となる注目画像としての重みが大きいかを判断するために、グループ内の確からしさをポイント換算で、比較を行い、よりポイントが高いグループが最終注目画像と設定される。
【００６９】
ポイントの方法としては、候補が“ｍ”個存在する場合、候補１のポイントは“ｍ”。候補２のポイントは“ｍ−１”以下同様に候補ｍのポイントは“１”となる。
【００７０】
このようにして、候補グループ間の優位性を判断した結果の実例を図２３に示す。検出した候補グループは２グループあり、そのうち右のグループのポイントが左の候補グループのポイントを上回ったので、最終候補となっている。
【００７１】
また、ポイント数の絶対値は、対象となる候補グループの注目画像としての信頼度を表しているので、このポイントにより注目画像に対する補正強度を決定する。補正強度決定方法としては、ポイントによる閾値を設け、閾値の上下関係で強度の指定を行う。
【００７２】
先の、図１０と図１４に対する結果を図１３と図１７に示す。
図１３においては、注目画像である人物の顔の肌を検出している。また、図１７においては、各候補が周波数特性に適合せず候補部分が黒塗りの状態で表している。これは、注目画像が検出されなかった状態を表し、注目画像に重みを置いた画像補正の対象にならないことを示し意している。
【００７３】
こうして注目画像を検出することができる。通常の画像補正は、画像全体のバランスに亘って補正が行われるので、逆光などで本来注目したい画像の画質を落としてしまう場合が存在しているが、本実施の形態による注目画像検出により、補正項目として輝度の最適化のための露出、及び好ましい肌色のための色バランスや彩度補正を注目画像のデータを基に補正を行うことで、より高品質な画像を得ることができる。
【００７４】
図２４に、一般画像補正を行った結果と、本実施の形態の注目画像検出を利用して画像補正を行った結果の一例を示す。図２４に示したように、本実施の形態の注目画像検出を利用して画像補正を行った場合は、人物などの注目画像をより良くプリントすることができる。
【００７５】
本実施の形態においては、プリントのための最適画像処理用に注目画像を検出する方法を示しているが、表示用などにも使用できることは言うまでも無い。
【００７６】
また、本実施の形態においては、検出画像の周波数成分特性を見るために周波数情報を１０個単位で加算して周波数成分の６３個を７グループとして、画像の特性を判断したが、グループ化という発想をなくし、６３個全ての周波数をそのまま利用しても良いことは言うまでも無い。
【００７７】
更に、画像の長手方向からの連続量の検出後短い方向の検出を行ったが、この順序も逆になっても可能であり、この他、検出ブロックを一列のグループとして検出する方法以外にも色度で検出したグループにおける全ての方向に隣接したブロックグループという、とらえ方で空間周波数特性を確認する方法など、色度と周波数特性を組み合わせた検出方法はいくらでもあり、これら一連の検出方法は本発明に含まれることは言うまでも無い。
【００７８】
本実施の形態においては、図４のように、連続検出値を３グループに分け周波数特性の適正範囲との比較を行い周波数特性の合否を判定したが、連続検出を３グループ化したのは、実施形態を簡単化するためで、連続値ごとに適正範囲を設定しても良いし、連続値には相関関係が有るので、テーブル方式ではなく理論式による方法を用いても良い。また、周波数特性も７グループ値を使用したが、６３個の周波数のすべてにて行っても良いし、更には特定の周波数に注目して判定しても良い。
【００７９】
本実施の形態においては、検出の目的になっている注目画像は人物の肌の領域に設定して説明しているが、周波数成分、もしくは周波数成分と色度により検出可能なものは、人物の肌色に限らず、空、海、木々の緑なども存在する。
【００８０】
本実施の形態においては、８＊８ブロック単位データの周波数成分を周波数の低い順から１０個単位でまとめた値を用いて、その１０個の和によるグループ（最も高い周波数グループは３個の和）の特性から周波数特性を代表させているが、Ｊｐｅｇ　ファイルの場合、ＤＣ成分１個に対し、ＡＣ成分６３個の構成で、周波数特性を表しているので、１０個の集合体として特性を見なくても良い。
【００８１】
また、６３個の個々の特性より判断しても良いし、もっとグループ化しても良い。また、特定の周波数成分のみの利用により特性を導き出しても良い。このように、周波数特性を利用した特性を導くのにＡＣ成分の利用方法はいくらでもある。
【００８２】
更に、本実施の形態では８＊８ブロックの連結と言う概念で縦方向と横方向について注目画像を検出するために色度該当ブロックの連続性において、候補を抽出しているが、この時のブロック集合体の判定方法も、この方法に限られてものではないことは言うまでも無い。
【００８３】
本実施の形態では連続検出した色度ブロックに対して検出した連続値により、端のブロックを削除した値を特性利用しているが、周波数成分による適合から色度ブロックの境界を設定したり（図２１）、予め特定以上の周波数特性のあるブロックを、色度検索を行う前に除外してから行うようにしたりなど、ブロックの集合体を決定するための色度と周波数成分による分離の仕方は、複数の方法と組み合わせがあるが、本願特許の範囲に包含される。
【００８４】
上記図２１について説明する。図２１の左側は元画像であり、このＪｐｅｇ　ファイル画像の圧縮単位である８＊８画素ブロックの周波成分における高周波成分の総データ値が閾値を超えるか超えないかで判定したのが右側の画像になる。明るい部分が高周波成分を持つ領域で、暗い部分が高周波成分の少ない領域である。この領域を境に設けた色度判定による注目画像検出も可能である。
【００８５】
また、本実施の形態は、画像圧縮ファイルとして、“Ｊｐｅｇファイル”を利用した方法を開示したが、“Ｊｐｅｇ２０００　ｆｉｌｅ“など、周波数成分への変換を利用した他のファイルに対しても同様な考え方で、注目画像の検出を簡単な処理で実現できることは、言うまでもない。
【００８６】
（本発明の他の実施の形態）
なお、以上に説明した本実施形態の画像認識装置は、コンピュータのＣＰＵあるいはＭＰＵ、ＲＡＭ、ＲＯＭなどで構成されるものであり、ＲＡＭやＲＯＭに記憶されたプログラムが動作することによって実現できる。
【００８７】
したがって、コンピュータが上記機能を果たすように動作させるプログラムを、例えばＣＤ−ＲＯＭのような記録媒体に記録し、コンピュータに読み込ませることによって実現できるものである。上記プログラムを記録する記録媒体としては、ＣＤ−ＲＯＭ以外に、フレキシブルディスク、ハードディスク、磁気テープ、光磁気ディスク、不揮発性メモリカード等を用いることができる。
【００８８】
また、コンピュータが供給されたプログラムを実行することにより上述の実施形態の機能が実現されるだけでなく、そのプログラムがコンピュータにおいて稼働しているＯＳ（オペレーティングシステム）あるいは他のアプリケーションソフト等と共同して上述の実施形態の機能が実現される場合や、供給されたプログラムの処理の全てあるいは一部がコンピュータの機能拡張ボードや機能拡張ユニットにより行われて上述の実施形態の機能が実現される場合も、かかるプログラムは本発明の実施形態に含まれる。
【００８９】
また、本発明をネットワーク環境で利用するべく、全部あるいは一部のプログラムが他のコンピュータで実行されるようになっていても良い。例えば、画面入力処理は、遠隔端末コンピュータで行われ、各種判断、ログ記録等は他のセンターコンピュータ等で行われるようにしても良い。
【００９０】
【発明の効果】
上述したように、本発明によれば、圧縮記録された画像データから所定のブロックごとに空間周波数情報と色度情報とを取得して、上記圧縮記録された画像データにおける注目画像検索のために利用するようにしたので、高度な計算をすることなく画像データブロックごとの交流成分情報を含む情報を取得して、画像ファイルの中の注目画像を検索するために利用することができる。これにより、処理負荷の少ない方法で注目画素を検出することができる。
【００９１】
また、本発明のその他の特徴によれば、デジタルカメラから直接プリントする場合などのように、パーソナルコンピュータと比べ処理能力が低い組み込み式の機器においても製品として使用可能な範囲の処理で、印刷する圧縮画像ファイルに補正の対象となる注目画像の有無及びその値の適正度を検出することができ、必要に応じて注目画像を重視した画像補正を施すことができる。
【図面の簡単な説明】
【図１】本発明の実施の形態に係わるＪｐｅｇ画像解凍時に必要なデータを取得する流れを示す概念図である。
【図２】実施の形態の画像データをＪｐｅｇ形式へ変換する処理過程の流れを示す概念図である。
【図３】実施の形態のＪｐｅｇの画像圧縮単位である８＊８ブロックを例にしたＪｐｅｇ形式へ変換する処理過程を示す図である。
【図４】実施の形態のＪｐｅｇファイル画像圧縮単位である８＊８ブロックのＡＣ成分特性を利用した判別テーブルを示す図である。
【図５】実施の形態の他にある肌色のＲＧ色度分布例を示す図である。
【図６】実施の形態のＪｐｅｇ画像解凍からの注目画像検出フローチャートである。
【図７】実施の形態のＪｐｅｇファイル画像圧縮単位である８＊８ブロックにおける、色度検出方法を示す図である。
【図８】実施の形態のＪｐｅｇファイル画像圧縮単位である８＊８ブロックでのＤＣ成分を利用した色度検出方法を示す図である。
【図９】実施の形態の色度検出において、３ビット間引きを利用して検出をした場合の８＊８ブロックにおける検出状況を示す図である。
【図１０】実施の形態の検出用Ｊｐｅｇ画像サンプルの第１の例を示す図である。
【図１１】第１の画像サンプルを色度のみによる検出を行った結果のＢＭＰ　ファイルの一例を示す図である。
【図１２】第１の画像サンプルを８＊８ブロック単位の色度検出を元に配置と連続Ｂｌｏｃｋ検出を行った結果のＢＭＰ　ファイルの一例を示す図である。
【図１３】実施の形態の注目画像検出により、第１の画像サンプルを８＊８ブロック単位の色度検出を元に配置と連続ブロックとＡＣ成分による検出を行った結果のＢＭＰファイルの一例を示す図である。
【図１４】実施の形態の検出用Ｊｐｅｇ画像サンプルの第２の例を示す図である。
【図１５】第２の画像サンプルを色度のみによる検出を行った結果のＢＭＰファイルの一例を示す図である。
【図１６】第２の画像サンプルを８＊８ブロック単位の色度検出を元に配置と連続Ｂｌｏｃｋ検出を行った結果のＢＭＰ　ファイルの一例を示す図である。
【図１７】実施の形態の注目画像検出により、第２の画像サンプルを８＊８ブロック単位の色度検出を元に配置と連続ブロックとＡＣ成分による検出を行った結果のＢＭＰ　ファイルの一例を示す図である。
【図１８】実施の形態の人物肌検出において、人物肌検出データの連続色度検出値におけるＡＣ成分の周波数特性を示す図である。
【図１９】実施の形態の人物肌検出において、枯れ林の検出データの連続色度検出値におけるＡＣ成分の周波数特性の表を示す図である。
【図２０】実施の形態の肌色のＲＧ色度分布を示す図である。
【図２１】周波数特性による境界作成のための検出方法の一例を示す図である。
【図２２】実施の形態の候補グループの判定手順を示すフローチャートである。
【図２３】実施の形態の候補グループ判定の検出結果画像の一例を示す図である。
【図２４】実施の形態の注目画像検出を利用した画像補正の比較結果の一例を示す図である。
【符号の説明】
１　エントロピー復号化手段
２　符号テーブル
３　逆量子化手段
４　量子化テーブル
５　逆ＤＣＴ手段[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image recognition method, an image recognition device, a computer program, and a computer-readable recording medium, and is particularly suitable for use in recognizing a Jpeg file image in a compressed image data format.
[0002]
[Prior art]
A JPEG file image photographed by a digital camera or the like may be printed from a PC printer or a direct printer, or may be printed by a DPE. At this time, if the photographed image data is of high quality, it is sufficient to print the image faithfully, so that no problem occurs.
[0003]
However, depending on the captured image data, there is color coverage, insufficient contrast, improper exposure, and the like, and it is necessary to perform image correction in order to obtain a high-quality print result. In particular, in the case of an image obtained by photographing a person, generally, if the print is performed so that the color of the face of the person becomes appropriate, the impression given to the person who saw the photograph is improved, and the quality of the photograph is improved.
[0004]
In the case of silver halide photography, it is preferable to change the exposure at the time of printing for each original image in order to obtain a good quality photo. It is convenient to focus on the color of a person's face. This is because since the human face is known to be flesh-colored, it is possible to determine the exposure amount so that the color of the human face in the printed photograph becomes flesh-colored.
[0005]
Further, as a method of recognizing an image from an image file of digital data, for example, “Japanese Patent Application Laid-Open No. 8-161497”, “Japanese Patent Application Laid-Open No. 2000-48036”, and “Japanese Patent Application Laid-Open No. 11-238067” are known.
[0006]
These methods detect the degree of similarity and the degree of coincidence with a designated image. In the case of "Japanese Patent Laid-Open No. Hei 8-161497", a rough match in a block unit by a DC component is obtained, and then the candidate image area is determined. In this method, a decompression process is performed on the data to obtain a fine match as uncompressed data.
[0007]
In the case of Japanese Patent Application Laid-Open No. 2000-48036, the image processing apparatus inputs and creates search data and determines the similarity between the data and a plurality of image data. Further, in the case of Japanese Patent Application Laid-Open No. 11-238067, a compressed image is created by performing a wavelet transform on an image to be searched. Further, the specified image is also subjected to wavelet transform, and the similarity is determined by comparing each feature data.
[0008]
In addition, when printing an image captured by a digital camera, an application or printer driver application analyzes the captured data using a histogram, etc., and performs uniform image correction such as contrast, white balance, exposure correction, and sharpness. Are known.
[0009]
[Problems to be solved by the invention]
When printing a Jpeg file image photographed by a digital camera or the like, a Jpeg file image such that a noticeable image of a person or the like, such as a print of a silver halide photograph, can be corrected as necessary so that it can be printed better. It is necessary to decide how to find the image of interest in.
[0010]
Further, there is a demand for a method in which the detection processing can be made as light as possible so that it can be used even with a device having a low data processing capability such as direct printing for directly printing from a digital camera to a printer.
SUMMARY OF THE INVENTION In view of the above problems, an object of the present invention is to enable a target image in an image file to be detected by a method with a small processing load.
[0011]
[Means for Solving the Problems]
The image recognition method of the present invention acquires spatial frequency information and chromaticity information for each predetermined block from compressed and recorded image data, and uses the spatial frequency information and chromaticity information for searching for a target image in the compressed and recorded image data. It is characterized by having.
According to another feature of the present invention, in a process of decompressing compressed image data obtained by compressing a plurality of pixels as one compression unit, a two-dimensional spatial frequency is calculated for each of the plurality of partial pixel regions as the compression unit. Data including information is extracted, and it is detected whether or not the feature value of the AC frequency component of the extracted partial region falls within a predetermined range of the feature value of the image of interest. Or not.
Another feature of the present invention is to use a process of restoring compressed data containing an AC component in units of a plurality of pixels into uncompressed data by using orthogonal transform to a two-dimensional spatial frequency domain. An image recognition method, comprising: extracting data including an AC frequency component for a plurality of partial pixel regions, which are units of compression, when restoring the compressed data to uncompressed data; Whether each of the extracted partial regions is included in the target image target portion based on whether or not the feature amount of the AC frequency component of the partial region extracted by the process is within a predetermined range of the target image feature amount. And a judging step of judging whether or not the judgment is made.
Another feature of the present invention is to use a process of restoring compressed data containing an AC component in units of a plurality of pixels into uncompressed data by using orthogonal transform to a two-dimensional spatial frequency domain. An image recognition method, comprising: extracting data including an AC frequency component for a plurality of partial pixel regions, which are units of compression, when restoring the compressed data to uncompressed data; Based on whether or not each value of the AC frequency component and the chromaticity feature amount of the extracted partial region fall within a predetermined range of the target image feature amount, the partial region enters the target image target portion. And a determining step of determining whether or not there is a
Another feature of the present invention is to use a process of restoring compressed data containing an AC component in units of a plurality of pixels into uncompressed data by using orthogonal transform to a two-dimensional spatial frequency domain. An image recognition method, comprising: extracting data including an AC frequency component for a plurality of partial pixel regions, which are units of compression, when restoring the compressed data to uncompressed data; The image region dividing step of dividing the image region into a plurality of partial regions based on the characteristic amount of the AC frequency component extracted by the above-described process, and the chromaticity characteristic amount in the range of each of the partial regions divided by the image region dividing step are focused on. A determining step of determining whether or not each partial region is included in the target image target portion based on whether or not the partial range is within a predetermined range of the image feature amount. It is characterized in that.
[0012]
The image recognition apparatus of the present invention obtains spatial frequency information and chromaticity information for each predetermined block from compressed and recorded image data, and uses the acquired spatial frequency information and chromaticity information for a target image search in the compressed and recorded image data. It is characterized by having.
Another feature of the present invention is that, in the process of decompressing compressed image data that has been compressed using a plurality of pixels as one compression unit, a two-dimensional space is provided for each of the plurality of partial pixel regions as the compression unit. Data including frequency information is extracted, and it is detected whether or not the feature amount of the AC frequency component of the extracted partial region falls within a predetermined range of the feature amount of the image of interest. It is characterized in that it is determined whether or not it is a target.
Another feature of the present invention is to use a process of restoring compressed data containing an AC component in units of a plurality of pixels into uncompressed data by using orthogonal transform to a two-dimensional spatial frequency domain. An image recognition device, comprising: a data extraction unit for extracting data including an AC frequency component for a plurality of partial pixel regions that are units of compression when restoring the compressed data to uncompressed data; Whether each of the extracted partial regions is included in the target image target portion based on whether or not the feature amount of the AC frequency component of the partial region extracted by the process is within a predetermined range of the target image feature amount. Determining means for determining whether or not the determination is made.
Another feature of the present invention is to use a process of restoring compressed data containing an AC component in units of a plurality of pixels into uncompressed data by using orthogonal transform to a two-dimensional spatial frequency domain. An image recognition device, comprising: a data extraction unit for extracting data including an AC frequency component for a plurality of partial pixel regions that are units of compression when restoring the compressed data to uncompressed data; Based on whether or not each value of the AC frequency component and the chromaticity feature amount of the extracted partial region fall within a predetermined range of the target image feature amount, the partial region enters the target image target portion. And determining means for determining whether or not there is an error.
Another feature of the present invention is to use a process of restoring compressed data containing an AC component in units of a plurality of pixels into uncompressed data by using orthogonal transform to a two-dimensional spatial frequency domain. An image recognition device, comprising: a data extraction unit for extracting data including an AC frequency component for a plurality of partial pixel regions that are units of compression when restoring the compressed data to uncompressed data; The image region dividing unit that divides the image region into a plurality of partial regions based on the feature amount of the AC frequency component extracted by the method, and the feature amount of chromaticity in the range of each partial region divided by the image region dividing unit are focused on. Determining means for determining whether or not each partial region is included in the target image target portion, based on whether or not the image feature amount falls within a predetermined range. It is characterized in that.
[0013]
The computer program of the present invention can execute an image recognition method that uses a process of restoring compressed data containing an AC component in units of a plurality of pixels into uncompressed data using an orthogonal transform to a two-dimensional spatial frequency domain. A data extraction step of extracting data including an AC frequency component for a plurality of partial pixel regions, which are units of compression, when the compressed data is decompressed into uncompressed data. Whether each of the extracted partial regions is included in the target image target portion based on whether or not the feature amount of the AC frequency component of the partial region extracted by the process is within a predetermined range of the target image feature amount. And a determining step of determining whether or not the processing is performed by a computer.
Another feature of the present invention is to use a process of restoring compressed data containing an AC component in units of a plurality of pixels into non-compressed data using orthogonal transform into a two-dimensional spatial frequency domain. A computer program capable of executing an image recognition method, comprising: a data extracting step of extracting data including an AC frequency component for a plurality of partial pixel regions, which are units of compression, when restoring the compressed data to uncompressed data. Based on whether or not each value of the AC frequency component and the feature value of the chromaticity of the partial region extracted in the data extraction step are within a predetermined range of the feature value of the image of interest. And a determining step of determining whether the image is included in the image target portion.
Another feature of the present invention is to use a process of restoring compressed data containing an AC component in units of a plurality of pixels into uncompressed data by using orthogonal transform to a two-dimensional spatial frequency domain. A computer program capable of executing an image recognition method, comprising: a data extracting step of extracting data including an AC frequency component for a plurality of partial pixel regions, which are units of compression, when restoring the compressed data to uncompressed data. And an image area dividing step of dividing the image area into a plurality of partial areas by the feature amount of the AC frequency component extracted in the data extracting step; and chromaticity in a range of each partial area divided in the image area dividing step. Are included in the target image target portion based on whether the feature amount of the target image is within a predetermined range of the target image feature amount. It is characterized in that to execute a determining step to a computer or dolphin not.
[0014]
A computer-readable recording medium according to the present invention is characterized by recording any of the computer programs described above.
[0015]
BEST MODE FOR CARRYING OUT THE INVENTION
Next, embodiments of an image recognition method, an image recognition device, a computer program, and a computer-readable recording medium of the present invention will be described with reference to the accompanying drawings. First, when printing image data in the Jpeg file format, which is a data compression recording format of a general digital camera, before decompressing the data into uncompressed data, the luminance and chromaticity bases are set for each block (8 * 8 pixels). DCT (Discrete Cosine Transform) data is also obtained and used for the image of interest search. By doing so, it is possible to acquire and use AC component information and the like for each image data block without performing advanced calculations.
[0016]
"First Embodiment"
First, information omission and encoding / decoding of the “Jpeg file” of the most common image compression file are described.
[0017]
First, encoding is generally performed. In a digital camera, a digital video or the like, a still image is generally stored in a Jpeg file. In this case, a signal input from a CCD or the like, which is a light receiving element of an input device, is A / D-converted, and then taken into a frame memory to convert RGB or CMY filter information into luminance and chromaticity information. Then, it is divided into 8 * 8 (64) square pixel blocks.
[0018]
(1) in FIG. 3 shows an example of the data of one block out of the division of the bit map of the luminance data into 8 * 8 blocks. FIG. 3B shows an example in which pixel values of 0 to 255 are level-shifted and converted to signals of -128 to 127. Further, (3) in FIG. 3 shows an example in which DCT coefficients are obtained by DCT (discrete cosine transform).
[0019]
Also, (4) in FIG. 3 is a quantization table in which omission of high-frequency components in consideration of visual characteristics is increased, and using this table, the DCT coefficient as a result of (3) in FIG. An example of quantization is shown.
[0020]
(5) in FIG. 3 shows the result of the quantization. This value is entropy-encoded and represented by a Huffman code to generate compressed data as an encoded signal.
[0021]
Next, in the decoding, the reverse process of the above-described encoding is performed. That is, the coded signal is decoded, and the value of the quantized DCT coefficient is decoded. Next, DCT coefficients are obtained by multiplying by a quantization table in order to perform inverse quantization. Thereafter, the level-shifted image is restored by performing the inverse DCT, and the image of one block is decoded by further adding the inverse level shift value 128.
[0022]
In the above description, the process of combining the data divided into the luminance information and the chromaticity information and converting the data into an RGB image has been omitted. However, as shown in FIG. (Y) and two chromaticity components (Cb, Cr) are converted, and each of them is encoded and combined to generate compressed image data.
[0023]
As a method of printing a Jpeg image which is a compressed image data file as described above, a compressed image data from an input device is taken into a personal computer (hereinafter, referred to as a PC) via a USB or a storage medium and the image is read. If you want to send the data to the printer after developing and applying image correction as needed, or input the image data from the input device directly to the printer, decompress the image in the printer, and correct the image if necessary There are several options, such as printing after adding.
[0024]
In any case, in order to print a good image, it is necessary to determine whether the photographed image data is a good photographed image or an image that needs to be corrected, and determine whether the photographed image data is a good quality image to be printed faithfully. It is necessary to further divide the source of printing after approaching a good quality image by performing correction.
[0025]
The following are considered as good images.
1) Good white balance.
2) The contrast is appropriate.
3) A necessary part of gradation is assigned. That is, the exposure setting is good.
4) Saturation is appropriate.
5) The finish is like a silver halide photograph.
6) The image of interest such as a person is corrected mainly.
[0026]
At present, even in commercially available PC printers and direct printers that do not pass through a PC, the above items 1) to 5) are carried out to varying degrees. In addition, the reason why the correction of the target image in the above 6) is not performed is that a large amount of processing is required for the detection and that the method has not been established.
[0027]
In particular, it is said that implementation is difficult in a direct printer or the like having a weak processing capability, but the present invention solves this problem. As a means for this, a method of detecting the presence of an image of interest in the Jpeg image file, confirming the necessity of correction for the detected image, etc., and then transferring the image to the entire image correction is used.
[0028]
FIG. 1 is a block diagram showing a process of decompressing a Jpeg file and information obtained at that time.
In the process of converting a Jpeg file into RGB bitmap data, first, entropy decoding is performed by the entropy decoding means 1 using the code table 2. Next, in the inverse quantization means 3, the quantization table 4 used for the inverse quantization is stored as data in addition to performing the inverse quantization.
[0029]
The dequantized data is frequency-converted as data in block units, and this data is obtained as data for obtaining image frequency characteristics. Thereafter, the inverse DCT means 5 performs an inverse DCT process and an inverse level shift and performs Ycc-RGB conversion to develop the data into normal RGB bitmap data.
[0030]
Next, FIG. 6 shows a flowchart of person detection, which is the most important target image detection in this image.
In the first step S601, DCT data of 8 * 8 pixels in block units is obtained, and at the same time, the image file is developed into RGB bitmap data.
Next, the process proceeds to step S602, and a search is performed in the RGB bitmap data to determine whether each block of 8 * 8 pixels corresponds to the chromaticity of the skin color of a person as the target image.
[0031]
In this case, the search range may cover the entire screen, but in the present embodiment, it is difficult to imagine that the target image is located at the end of the image, and therefore, the search range is several blocks from the end of the image. Is excluded from the chromaticity search target and is searched.
[0032]
There are a plurality of chromaticity search methods. What is known is:
1) A color in which the ratio of B (blue) / G (green) falls within the range of 0.7 to 0.8 and the ratio of R (red) / G (green) falls within the range of 1.4 to 1.8. Things with degrees.
2) As shown in the conceptual diagram of FIG. 5, the flesh color can be represented by a probability ellipse. The following equations (1) to (3) are obtained.
[0033]
(Equation 1)

[0034]
In the present embodiment, the chromaticity distribution range represented by the following equation (4) taking the simplicity of processing into consideration is defined as the chromaticity range of the skin color. FIG. 20 shows this range.
[0035]
(Equation 2)

[0036]
In the present embodiment, since the feature of the frequency component in the image is detected in units of 8 * 8 pixels, the chromaticity determination is performed in 8 * 8 pixel units due to structural and logical simplicity. Run.
[0037]
FIG. 7 illustrates chromaticity detection points used in the present embodiment. According to this, it is checked whether or not all the chromaticities at the four corners of the block of “8 * 8 pixels” are within the chromaticity range. Has been determined.
[0038]
In FIG. 7, the second block from the left in the upper row and the 1,2,3 blocks from the left in the lower row correspond to this. Since the leftmost block in the upper row is determined as a non-skin color pixel in the upper left of the four points, a block including this is determined to be out of the range of the flesh color. Similarly, the upper right block and the lower right block are out of the range.
[0039]
FIG. 8 shows the determination based on the average chromaticity of the entire block of “8 * 8 pixels”. As a method of calculating the average chromaticity in this block, in addition to the method of obtaining the average value of the pixel values of all 8 * 8 blocks, the chromaticity data (Cb, Cr) before the inverse DCT during decompression is performed. Can also be determined from the DC component of An advantage of this method is that it is possible to judge by the color tone of the entire block, so that higher accuracy can be expected as compared with the one having a small number of detection points. Here, the content of detection of only chromaticity in a natural image will be described.
[0040]
FIG. 9 is for the same idea as in FIG. 7, but for equally dividing the detection interval in the whole image.
[0041]
FIG. 10 is a general portrait photograph, and FIG. 14 is a photograph of a forest of a dead tree having a chromaticity range similar to that of a person. FIGS. 11 and 15 show the results of detection performed only by matching the chromaticity of each pixel with respect to FIGS. 10 and 14. FIG.
[0042]
As the detection result in the portrait of FIG. 11, the flesh color portion of the person is well detected, but in addition, a fence or a background that satisfies the matching chromaticity even in a fine portion such as dust is detected. You can see that there is. For this reason, it turns out that an attention image cannot be specified only by chromaticity.
[0043]
In FIG. 14, a forest of a dead tree having the same chromaticity is entirely detected regardless of the purpose of detecting the skin color of a person. As described above, when the chromaticity determination is performed at the pixel level, it is impossible to specify the target image.
[0044]
When the detection is performed at the block level, a state having a specific unit is targeted, and thus the influence of external noise is hardly received. However, a block of 8 * 8 pixels cannot be said to be an appropriate unit size, and the block detection based on chromaticity is further performed by performing a restricted detection of continuous detection of blocks adjacent in the vertical and horizontal directions. It will increase accuracy.
[0045]
Here, a continuous range for determining noise is set based on the concept that even a human skin color that does not satisfy the amount of data that can recognize a face in a print may be rejected as non-adaptive.
[0046]
This processing is illustrated in the processing after step S603 in FIG. That is, in step S603, chromaticity detection is performed for each block in the longitudinal direction of the image, and candidates are formulated in descending order of the number of consecutively detected blocks.
[0047]
Next, in step S604, a comparison is made as to whether or not the continuous amount falls within the adaptive continuous amount as the target image. As a result of the comparison, if there is a corresponding continuous block, the process proceeds to step S605, and a search is performed to determine whether data that satisfies the block continuous detection setting in the short direction exists in the image.
[0048]
Next, in step S606, it is determined whether or not there is detected data. If there is detected data, the process proceeds to step S608, and data remaining in this process is sequentially sorted from data having the largest continuous block amount in the longitudinal direction. Assign a candidate number.
[0049]
If the result of determination in step S606 is that there is no detected data, processing proceeds to step S607, where "no destination area" is set, and the process ends.
[0050]
Here, the story will return a little, but the effect of performing the chromaticity determination in the continuous block is shown in FIGS. 12 and 16.
FIG. 12 shows the result of detecting the portrait image of FIG. In FIG. 12, color codes (1 = brown, 2 = red, 3 = orange, 4 = yellow, 5 = green, 6 = blue, 7 = purple, 8 = gray) are arranged in descending order of detection candidate priority. In other cases, only the chromaticity is within the appropriate range. It can be seen that a considerable number of non-applicable candidates, such as backgrounds, can be deleted by continuous block detection as compared with pixel-level chromaticity detection.
[0051]
In FIG. 16, it is understood from the result of the detection performed on the dead tree forest of FIG. 14 that the non-target image is also detected in the continuous block detection.
[0052]
Next, a plurality of image samples of a VGA (video graphics array) size (640 * 480 pixels) were used to calculate the frequency characteristics of the detected adaptive chromaticity continuous block in the human skin and the dead tree forest.
[0053]
FIG. 18 shows DCT data of blocks in which continuous blocks of human skin photographed in an image are detected, arranged in order of lower frequency, added in units of 10 from the lower frequency, and divided by the number of continuous blocks. It is a summary of the average frequency component per block of consecutively detected blocks.
[0054]
Therefore, in the drawing, the horizontal axis is a summary of 63 frequency components of the AC component, and the data having the highest frequency is data of three groups with six groups of ten components. The vertical axis is a value obtained by adding the components of each frequency component.
[0055]
This indicates that the larger the value is, the higher the corresponding frequency component is in the block. In addition, data lines are color-coded for each number of detected continuous blocks. For example, “B2” represents an average value of data in which two consecutive blocks are detected, and “B15” represents an average value of data in which 15 consecutive blocks are detected. Hereinafter, the same applies to the spatial frequency characteristics for each successive detection value of the average human skin color portion from a plurality of images “B2 to B15”.
[0056]
Looking at the detection results,
1) The value of the low frequency component is large and the value of the low frequency component after the third group is 50 or less regardless of the number of continuous blocks.
2) The larger the continuous value of the continuous block, the lower the frequency characteristic.
[0057]
It can be said from these results that the frequency characteristics of the flesh-colored part of the person are composed of relatively low frequencies, and that the value of the detected continuous block is large means that the size of the photographed object is large. It can be understood that the frequency component is lowered by calculating the average value of the continuous block.
[0058]
By using the continuous values of the continuous blocks, even those having the same chromaticity of the target image, the continuous block is made one representative value (for example, in the case of the block B6, the detected values of the six blocks are The values of the spatial frequency characteristics are changed by adding the values obtained as a group of 10 units in ascending order of the frequency, adding the values for each group, and dividing the result by 6 which is the continuous value to obtain an average.) It can be seen that the appropriate corresponding frequency characteristic differs depending on the continuous detection value.
[0059]
FIG. 19 shows a result of preparing a plurality of photographs of a forest of a dead tree having a chromaticity range similar to the skin color chromaticity of a person and performing detection in the same manner as in FIG. 18.
[0060]
Looking at the detection results,
1) It can be confirmed that there is much data in high frequency components as compared with the spatial frequency characteristics of the human skin.
2) The group of the lowest frequency component is not significantly different from the result of human skin.
[0061]
From these facts, it is understood that by detecting the frequency components in the continuous blocks, it is possible to distinguish the detection objects having the same chromaticity by the frequency characteristics.
[0062]
FIG. 4 shows the spatial frequency characteristics of the human skin, which is the image of interest, used in the present embodiment. The upper row shows the appropriate range of the frequency characteristic in the VGA (640 * 480) image.
[0063]
The continuous block values are grouped into three groups of 2 to 8 groups (to L8), 9 to 20 groups (L9 to 20), and 21 or more groups (L21 to), and the appropriate frequency range for each group Is set. For the appropriate range of the frequency, the frequency characteristics of the seven groups of ten shown above were used. This is performed in a balance between the simplification of the processing and the detection accuracy, and there is no need to be limited to this.
[0064]
Returning to the description of FIG. As described above, candidate numbers 1 to n (n = 8 in the present embodiment) of the target image are assigned in order from the data having the largest amount of continuity in the longitudinal direction detected by the chromaticity (step S608). Candidate numbers cannot be assigned to detected items after n.
[0065]
Next, the process proceeds to step S609, and the candidates 1 to n are successively compared with each other to determine whether or not the candidates match the range of the spatial frequency characteristic appropriate range determination table for the number of continuous blocks shown in FIG. As a result, when there is no matching candidate, it is determined that the target image does not exist.
[0066]
In the case where there is a suitable candidate, the description will be given below.
FIG. 22 shows the flowchart.
In the first step S2201, the number of candidates is confirmed (1 to m).
Next, the process proceeds to step S2202 to form a candidate group. In this case, a chromaticity matching block adjacent to the candidate is set as a candidate group.
[0067]
Next, the process proceeds to step S2203, and it is determined whether or not there are a plurality of candidate groups. As a result of this determination, when a plurality of candidates are included in the candidate group, the process proceeds to step S2204, and the group is set using the smaller candidate number.
[0068]
Then, for each detected group, in order to determine which group has a larger weight as the target image to be corrected, the likelihood within the group is compared in terms of points, and a comparison is made. The higher group is set as the final target image.
[0069]
As a point method, when there are “m” candidates, the point of candidate 1 is “m”. The point of candidate 2 is “m−1” and the same, and the point of candidate m is “1”.
[0070]
FIG. 23 shows an example of the result of determining the superiority between the candidate groups in this manner. There are two detected candidate groups, of which the point of the right group exceeds the point of the left candidate group, and thus is the final candidate.
[0071]
Since the absolute value of the number of points represents the reliability of the target candidate group as the target image, the correction strength for the target image is determined based on the points. As a method of determining the correction intensity, a threshold value based on points is provided, and the intensity is designated based on the upper and lower relationship of the threshold value.
[0072]
The results for FIGS. 10 and 14 are shown in FIGS. 13 and 17.
In FIG. 13, the skin of the face of the person, which is the image of interest, is detected. In FIG. 17, each candidate does not conform to the frequency characteristics, and the candidate portion is shown in black. This indicates a state in which the target image has not been detected, and indicates that the target image is not to be subjected to image correction with a weight.
[0073]
Thus, the target image can be detected. In normal image correction, correction is performed over the balance of the entire image.Therefore, there is a case where the image quality of the image that the user originally wants to pay attention to is deteriorated due to backlight or the like. As a correction item, exposure for optimizing luminance, and color balance and saturation correction for a preferable skin color are corrected based on the data of the target image, so that a higher quality image can be obtained.
[0074]
FIG. 24 illustrates an example of a result of performing the general image correction and a result of performing the image correction using the target image detection according to the present embodiment. As shown in FIG. 24, when image correction is performed using the target image detection of the present embodiment, a target image of a person or the like can be printed better.
[0075]
In the present embodiment, a method of detecting an image of interest for optimal image processing for printing is described, but it is needless to say that the method can also be used for display.
[0076]
Further, in the present embodiment, in order to see the frequency component characteristics of the detected image, the frequency information is added in units of 10 and 63 frequency components are grouped into 7 groups to determine the image characteristics. It goes without saying that it is possible to eliminate the idea and use all 63 frequencies as they are.
[0077]
Furthermore, although the detection in the short direction was performed after the detection of the continuous amount from the longitudinal direction of the image, it is also possible to reverse the order, and in addition to the method of detecting the detection blocks as a group of one row, There are a number of detection methods that combine chromaticity and frequency characteristics, such as a method of checking spatial frequency characteristics in terms of a block group adjacent in all directions in a group detected by chromaticity. It goes without saying that it is included in the invention.
[0078]
In the present embodiment, as shown in FIG. 4, continuous detection values are divided into three groups and comparison with an appropriate range of the frequency characteristic is performed to determine whether the frequency characteristic is acceptable or not. In order to simplify the embodiment, an appropriate range may be set for each continuous value, and since continuous values have a correlation, a method based on a theoretical formula may be used instead of a table method. In addition, although the frequency characteristics use the seven group values, the determination may be performed at all 63 frequencies, or the determination may be performed by focusing on a specific frequency.
[0079]
In the present embodiment, the target image to be detected is described as being set in the skin area of the person, but the frequency component, or the image that can be detected by the frequency component and the chromaticity, is Not only the skin color, but also the sky, the sea, and the greenery of trees.
[0080]
In the present embodiment, a value obtained by summing the frequency components of 8 * 8 block unit data in units of 10 from the lowest frequency is used, and a group of the 10 sums (the highest frequency group is the sum of 3 sums) ), The frequency characteristics are represented. However, in the case of a Jpeg file, the frequency characteristics are represented by a configuration of 63 AC components for one DC component. You don't have to.
[0081]
Further, the judgment may be made based on the 63 individual characteristics, or more groups may be used. Alternatively, the characteristic may be derived by using only a specific frequency component. As described above, there are many methods of using the AC component to derive the characteristics using the frequency characteristics.
[0082]
Furthermore, in the present embodiment, candidates are extracted in the continuity of the chromaticity-corresponding block in order to detect the target image in the vertical direction and the horizontal direction based on the concept of connecting 8 * 8 blocks. It goes without saying that the method of determining the block aggregate is not limited to this method.
[0083]
In the present embodiment, a characteristic value is used in which a block at an end is deleted based on a continuous value detected for a continuously detected chromaticity block, but a boundary of the chromaticity block is set based on adaptation by a frequency component ( FIG. 21), a method for separating blocks having a frequency characteristic higher than a predetermined value before performing a chromaticity search and then performing the chromaticity search, for example, by using a chromaticity and a frequency component to determine a set of blocks. Although there are a plurality of methods and combinations, they are included in the scope of the present patent.
[0084]
FIG. 21 will be described. The left side of FIG. 21 is the original image, and the right side image is determined based on whether the total data value of the high frequency components in the frequency component of the 8 * 8 pixel block, which is the compression unit of the Jpeg file image, exceeds or does not exceed the threshold value. become. A bright portion is a region having a high-frequency component, and a dark portion is a region having a small high-frequency component. It is also possible to detect an image of interest by chromaticity determination provided at this region.
[0085]
Although the present embodiment discloses a method using a “Jpeg file” as an image compression file, the same concept is applied to other files using conversion to frequency components, such as “Jpeg2000 file”. It goes without saying that the detection of the target image can be realized by simple processing.
[0086]
(Another embodiment of the present invention)
The above-described image recognition apparatus according to the present embodiment includes a computer CPU, an MPU, a RAM, a ROM, and the like, and can be realized by operating a program stored in the RAM or the ROM.
[0087]
Therefore, the present invention can be realized by recording a program that causes a computer to perform the above functions on a recording medium such as a CD-ROM, and reading the program into the computer. As a recording medium for recording the above program, a flexible disk, a hard disk, a magnetic tape, a magneto-optical disk, a nonvolatile memory card, and the like can be used in addition to the CD-ROM.
[0088]
Further, the functions of the above-described embodiments are realized by the computer executing the supplied program, and the program cooperates with the operating system (OS) or other application software running on the computer. When the functions of the above-described embodiments are realized, or when all or a part of the processing of the supplied program is performed by a function expansion board or a function expansion unit of a computer to realize the functions of the above-described embodiments. Such a program is also included in the embodiment of the present invention.
[0089]
Further, in order to use the present invention in a network environment, all or some of the programs may be executed on another computer. For example, the screen input processing may be performed by a remote terminal computer, and various determinations and log recording may be performed by another center computer or the like.
[0090]
【The invention's effect】
As described above, according to the present invention, the spatial frequency information and the chromaticity information are obtained for each predetermined block from the compressed and recorded image data, so that the image of interest can be searched for in the compressed and recorded image data. Since this is used, information including AC component information for each image data block can be obtained without performing advanced calculations, and can be used to search for a target image in an image file. Thus, the target pixel can be detected by a method with a small processing load.
[0091]
Further, according to another feature of the present invention, such as when printing directly from a digital camera, printing is performed in a range of processing that can be used as a product even in a built-in device having lower processing capability than a personal computer. The presence / absence of a target image to be corrected in the compressed image file and the appropriateness of the value can be detected, and an image correction emphasizing the target image can be performed as needed.
[Brief description of the drawings]
FIG. 1 is a conceptual diagram showing a flow of acquiring necessary data when decompressing a Jpeg image according to an embodiment of the present invention.
FIG. 2 is a conceptual diagram showing a flow of a process of converting image data into a Jpeg format according to the embodiment.
FIG. 3 is a diagram illustrating a process of converting an 8 * 8 block, which is an image compression unit of a Jpeg, into a Jpeg format according to the embodiment;
FIG. 4 is a diagram illustrating a discrimination table using AC component characteristics of 8 * 8 blocks, which is a JPEG file image compression unit, according to the embodiment;
FIG. 5 is a diagram illustrating an example of an RG chromaticity distribution of a flesh color according to another embodiment.
FIG. 6 is a flowchart for detecting an image of interest from decompression of a Jpeg image according to the embodiment.
FIG. 7 is a diagram illustrating a chromaticity detection method in an 8 * 8 block that is a JPEG file image compression unit according to the embodiment;
FIG. 8 is a diagram illustrating a chromaticity detection method using a DC component in an 8 * 8 block, which is a JPEG file image compression unit, according to an embodiment.
FIG. 9 is a diagram illustrating a detection state in an 8 * 8 block when detection is performed using 3-bit thinning in chromaticity detection according to the embodiment.
FIG. 10 is a diagram illustrating a first example of a detection Jpeg image sample according to the embodiment;
FIG. 11 is a diagram illustrating an example of a BMP file obtained as a result of detecting a first image sample based on only chromaticity.
FIG. 12 is a diagram illustrating an example of a BMP file as a result of arranging first image samples based on chromaticity detection in units of 8 * 8 blocks and performing continuous block detection.
FIG. 13 shows an example of a BMP file as a result of arranging first image samples based on chromaticity detection in units of 8 * 8 blocks and performing detection using continuous blocks and AC components by detecting an image of interest according to the embodiment; FIG.
FIG. 14 is a diagram illustrating a second example of a detection Jpeg image sample according to the embodiment;
FIG. 15 is a diagram illustrating an example of a BMP file obtained as a result of detecting a second image sample based on only chromaticity.
FIG. 16 is a diagram illustrating an example of a BMP file obtained as a result of arranging and detecting continuous blocks based on chromaticity detection in units of 8 * 8 blocks of a second image sample.
FIG. 17 shows an example of a BMP file as a result of arranging second image samples based on chromaticity detection in units of 8 * 8 blocks and performing detection using continuous blocks and AC components by detecting an image of interest according to the embodiment; FIG.
FIG. 18 is a diagram illustrating frequency characteristics of AC components in continuous chromaticity detection values of human skin detection data in human skin detection according to the embodiment.
FIG. 19 is a diagram showing a table of frequency characteristics of AC components in continuous chromaticity detection values of detection data of dead forest in human skin detection according to the embodiment.
FIG. 20 is a diagram illustrating an RG chromaticity distribution of flesh color according to the embodiment.
FIG. 21 is a diagram illustrating an example of a detection method for creating a boundary based on frequency characteristics.
FIG. 22 is a flowchart illustrating a procedure for determining a candidate group according to the embodiment;
FIG. 23 is a diagram illustrating an example of a detection result image of candidate group determination according to the embodiment.
FIG. 24 is a diagram illustrating an example of a comparison result of image correction using target image detection according to the embodiment;
[Explanation of symbols]
1 Entropy decoding means
2 Code table
3 Inverse quantization means
4 Quantization table
5 Inverse DCT means

Claims (32)

An image characterized in that spatial frequency information and chromaticity information are obtained for each predetermined block from the compressed and recorded image data, and the acquired spatial frequency information and chromaticity information are used for a target image search in the compressed and recorded image data. Recognition method. In the process of decompressing compressed image data obtained by compressing a plurality of pixels as one compression unit, data including two-dimensional spatial frequency information is extracted for each of the plurality of partial pixel regions which are the compression units, and the extracted By detecting whether or not the feature amount of the AC frequency component of the partial region falls within a predetermined range of the feature amount of the image of interest, it is determined whether or not each partial region is a target image of interest. An image recognition method characterized by the following. An image recognition method using a process of restoring compressed data including an AC component in units of a plurality of pixels using two-dimensional spatial frequency domain and transforming the compressed data into uncompressed data,
When decompressing the compressed data to non-compressed data, for a plurality of partial pixel regions that are units of compression, a data extraction step of extracting data including an AC frequency component,
Based on whether or not the feature amount of the AC frequency component of the partial region extracted in the data extraction step falls within a predetermined range of the feature amount of the image of interest, each of the extracted partial regions is converted into a target image portion. A determining step of determining whether the image is included in the image. An image recognition method using a process of restoring compressed data including an AC component in units of a plurality of pixels using two-dimensional spatial frequency domain and transforming the compressed data into uncompressed data,
When decompressing the compressed data to non-compressed data, for a plurality of partial pixel regions that are units of compression, a data extraction step of extracting data including an AC frequency component,
Based on whether each value of the AC frequency component and the chromaticity feature amount of the partial region extracted in the data extraction step are within a predetermined range of the feature amount of the image of interest, the partial region is A determining step of determining whether or not the image is included in a portion. An image recognition method using a process of restoring compressed data including an AC component in units of a plurality of pixels using two-dimensional spatial frequency domain and transforming the compressed data into uncompressed data,
When decompressing the compressed data to non-compressed data, for a plurality of partial pixel regions that are units of compression, a data extraction step of extracting data including an AC frequency component,
An image area dividing step of dividing the image area into a plurality of partial areas by the feature amount of the AC frequency component extracted in the data extracting step,
Based on whether or not the chromaticity feature amount in the range of each partial region divided by the image region dividing step falls within a predetermined range of the target image feature amount, each partial region is set as a target image target portion. A determining step of determining whether or not the image is included. 6. The image recognition method according to claim 1, wherein the compressed image data is Jpeg @ file data. 7. The information according to claim 2, wherein arrangement information of the extracted partial region in the entire image is added to the information for determining whether or not the target image is included in the target image target portion. The image recognition method according to the paragraph. The continuity information adjacent to the extracted partial area is added in the vertical and horizontal directions to the information for determining whether or not the target image is included in the target image target part. 8. The image recognition method according to claim 7. The DC component of the extracted partial region is added to the information for determining whether or not the target image is included in the target image target portion, according to any one of claims 2 to 7, wherein the DC component of the extracted partial region is added. Image recognition method. In the determination of the target image target portion, when a plurality of target image target portion candidates are detected in the setting range, the target image target portion candidate having the largest adjacent continuation of the detected partial region is noted. The image recognition method according to claim 7, wherein it is determined that the image corresponds to the image target portion. In the partial region that is consecutively adjacent to the partial region that includes the target image target partial candidate that has the largest number of consecutive adjacent partial regions, the target image target partial candidate whose chromaticity falls within the appropriate range is included. The image recognition method according to claim 10, wherein the target image is set as a target image. 12. The method according to claim 11, wherein in the detected image area of interest, the likelihood of detection of the image of interest detected with the content ratio of the second or subsequent longest partial area is determined. Image recognition method. 13. The image recognition method according to claim 12, wherein the image correction strength is determined using the likelihood of the detected image of interest. 14. The apparatus according to claim 11, wherein brightness information in the detected image area of interest is created, and exposure correction for the entire image is performed based on the created brightness information. Image recognition method as described. An image characterized in that spatial frequency information and chromaticity information are obtained for each predetermined block from the compressed and recorded image data, and the acquired spatial frequency information and chromaticity information are used for a target image search in the compressed and recorded image data. Recognition device. In the process of decompressing compressed image data obtained by compressing a plurality of pixels as one compression unit, data including two-dimensional spatial frequency information is extracted for each of the plurality of partial pixel regions which are the compression units, and the extracted By detecting whether or not the feature amount of the AC frequency component of the partial region falls within a predetermined range of the feature amount of the image of interest, it is determined whether or not each partial region is a target image of interest. An image recognition device characterized by the above-mentioned. An image recognition device using a process of restoring compressed data containing an AC component in units of a plurality of pixels using two-dimensional spatial frequency domain and transforming the compressed data into uncompressed data,
When decompressing the compressed data to uncompressed data, for a plurality of partial pixel regions that are units of compression, data extraction means for extracting data including an AC frequency component,
Based on whether or not the feature value of the AC frequency component of the partial region extracted by the data extracting means falls within a predetermined range of the feature value of the image of interest, each of the extracted partial regions is set to Determining means for determining whether the image is included in the image. An image recognition device using a process of restoring compressed data containing an AC component in units of a plurality of pixels using two-dimensional spatial frequency domain and transforming the compressed data into uncompressed data,
When decompressing the compressed data to uncompressed data, for a plurality of partial pixel regions that are units of compression, data extraction means for extracting data including an AC frequency component,
Based on whether or not each value of the AC frequency component and the chromaticity feature amount of the partial region extracted by the data extraction means are within a predetermined range of the feature amount of the image of interest, the partial region is An image recognition device comprising: a determination unit configured to determine whether a part is included. An image recognition device using a process of restoring compressed data containing an AC component in units of a plurality of pixels using two-dimensional spatial frequency domain and transforming the compressed data into uncompressed data,
When decompressing the compressed data to uncompressed data, for a plurality of partial pixel regions that are units of compression, data extraction means for extracting data including an AC frequency component,
Image area dividing means for dividing the image area into a plurality of partial areas by the feature amount of the AC frequency component extracted by the data extracting means,
Based on whether or not the chromaticity feature amount in the range of each partial region divided by the image region dividing means is within a predetermined range of the image feature amount of interest, each partial region is set as a target image object portion. An image recognition device comprising: a determination unit configured to determine whether or not the image is included. 20. The image recognition apparatus according to claim 16, wherein the compressed image data is Jpeg @ file data. 21. The information according to claim 16, wherein arrangement information of the extracted partial region in the entire image is added to the information for determining whether or not the extracted partial region is included in the target image target portion. An image recognition device according to the item. 17. The continuity information adjacent to the extracted partial region is added in the vertical and horizontal directions to the information for determining whether or not the target image is included in the target image portion. 22. The image recognition device according to any one of 21. The DC component of the extracted partial region is added to the information for determining whether or not the target image is included in the target image portion, according to any one of claims 16 to 21, wherein the DC component of the extracted partial region is added. Image recognition device. In the determination of the target image target portion, when a plurality of target image target portion candidates are detected in the setting range, the target image target portion candidate having the largest adjacent continuation of the detected partial region is noted. The image recognition apparatus according to any one of claims 21 to 23, wherein the apparatus is determined to correspond to an image target portion. In the partial region that is consecutively adjacent to the partial region that includes the target image target partial candidate that has the largest number of consecutive adjacent partial regions, the target image target partial candidate whose chromaticity falls within the appropriate range is included. 25. The image recognition apparatus according to claim 24, wherein the image is a target image. 26. The method according to claim 25, wherein in the detected image area of interest, the likelihood of detection of the image of interest detected with the content rate of the second or subsequent partial region is determined. Image recognition device. 25. The image recognition apparatus according to claim 24, wherein the image correction strength is determined using the likelihood of the detected image of interest. 28. The apparatus according to claim 25, wherein brightness information in the detected image area of interest is created, and exposure correction for the entire image is performed based on the created brightness information. The image recognition device according to claim 1. A computer program capable of executing an image recognition method using a process of restoring compressed data containing an AC component in units of a plurality of pixels into uncompressed data by using an orthogonal transform to a two-dimensional spatial frequency domain,
When decompressing the compressed data to non-compressed data, for a plurality of partial pixel regions that are units of compression, a data extraction step of extracting data including an AC frequency component,
Based on whether or not the feature amount of the AC frequency component of the partial region extracted in the data extraction step falls within a predetermined range of the feature amount of the image of interest, each of the extracted partial regions is converted into a target image portion. A computer program for causing a computer to execute a determining step of determining whether or not the information is included in the program. A computer program capable of executing an image recognition method using a process of restoring compressed data containing an AC component in units of a plurality of pixels into uncompressed data by using an orthogonal transform to a two-dimensional spatial frequency domain,
When decompressing the compressed data to non-compressed data, for a plurality of partial pixel regions that are units of compression, a data extraction step of extracting data including an AC frequency component,
Based on whether each value of the AC frequency component and the chromaticity feature amount of the partial region extracted in the data extraction step are within a predetermined range of the feature amount of the image of interest, the partial region is A computer program for causing a computer to execute a determination step of determining whether or not a part is included. A computer program capable of executing an image recognition method using a process of restoring compressed data containing an AC component in units of a plurality of pixels into uncompressed data by using an orthogonal transform to a two-dimensional spatial frequency domain,
When decompressing the compressed data to non-compressed data, for a plurality of partial pixel regions that are units of compression, a data extraction step of extracting data including an AC frequency component,
An image area dividing step of dividing the image area into a plurality of partial areas by the feature amount of the AC frequency component extracted in the data extracting step,
Based on whether or not the chromaticity feature amount in the range of each partial region divided by the image region dividing step falls within a predetermined range of the target image feature amount, each partial region is set as a target image target portion. A computer program for causing a computer to execute a determining step of determining whether or not the information is included. A computer-readable recording medium on which the computer program according to any one of claims 29 to 31 is recorded.

Images