JP4073156B2 - Image search device - Google Patents

Description

【００４２】
次に、本発明の画像検索方法に従って、上述の記憶されているデータに基づいて画像を検索する動作の流れを、図５に基づいて説明する。
まず、始めに、ステップ５０１で検索条件として、検索用指定パターン２０を読み込む。次のステップ５０２において、読み込まれた検索用指定パターン２０を前述と同様にして数値化し、特性値に変換して、テーブル化する。この手順では、特定図形の位置、サイズ、向き等の特性値や色などを直接入力してもよい。次に、ステップ５０３で、原画像データベース（ＤＢ）から検索対象として原画像を読み込む。続いて、ステップ５０４でテーブル化した検索用指定パターン２０と比較照合する。
【００４３】
この照合は、検索用指定パターンテーブルの各列，各行を順次行う。例えば、検索用指定パターンの複数の特性値等がテーブルの１行に並べられている場合、まず、ステップ５０４で，テーブルの第１行の各列に配列されている第１の指定パターンの各特性値をデータベースから読み出された原画像に付属情報として付属している特性値を比較照合して、ステップ５０５で一致不一致の判断を行い、一致するものがあればステップ５０６で検索結果として取り出す。
【００４４】
一方、ステップ５０５での判断が不一致であれば、ステップ５０６をバイパスして、ステップ５０７に進み、次の行に移る。次に、ステップ５０８で検索用指定パターンテーブルに次の行があるかどうか判断し、次の行が存在していれば、ステップ５０４に戻り、次の行の比較照合を行い、ステップ５０４〜ステップ５０８を繰り返す。ステップ５０８の判断で次の行が存在していなければ、検索対象の一つの原画像の照合を終了し、ステップ５０９に進み、次の検索対象となる原画像に移る。次に、ステップ５１０で次の原画像があるかどうか判断し、次の原画像が存在していれば、ステップ５０３に戻り、次の原画像について検索を行う。ステップ５０８の判断で次の原画像が存在していなければ、検索対象の全ての原画像の検索を終了する。
このようにして、本発明の検索方法は実施される。
【００４５】
ここで、検索用指定パターンの複数の特性値などがテーブルの１列に並べられている場合、上記照合は、列単位で行われ、一つの検索用指定パターンがある行と列との塊を単位として配列されている場合には、その一塊を単位として行われる。
なお、ここでは、簡単に「ステップ５０５で一致するものがあれば、ステップ５０６で取り出す」と説明しているが、実際には、完全に一致することは少ないので、後述するように一致度を定義して、ステップ５０４で一致度を算出し、ステップ５０５で、算出した一致度が、所定の閾値を超えたものを、一致したものとするのがよい。
【００４６】
なお、ユーザが入力・指定した検索用指定パターン２０は、例えば、表２に示すような形で、テーブル化されたデータとして、図示されていない記憶部に記憶される。また、ここでは、楕円だけについて例示したが、他の図形についても同様なデータが作成される。そして、このデータを基にして、上述のように、この検索用指定パターンに適合する原画像の検索が行われる。
【００４７】
【表２】

【００４８】
以下に、上述の、一致度の計算式の一例として、楕円の場合を示す。
【数１】

ここで、Ｆ｛Ｘ｝値は入力値Ｘが小であるほど大となる特性とする。
【００４９】
ここで、Ｉは１〜付属情報に存在する個数Ｉmax 、Ｊは１〜検索時のパターン個数Ｊmax 、ｋ１〜ｋ３は係数である。
また、検索パターンの番号Ｊに関する点数：
Ｐ（Ｊ）＝max （Ｄ（Ｉ，Ｊ) ）
全画面に置ける点数：
Ｐtotal （Ａ：楕円）＝Σ（Ｐ（Ｊ））
Ｊ＝１〜Ｊmax
である。
【００５０】
上記説明では、一例として、楕円形の場合について説明したが、この他の図形についても、ステップ５０４で同様の一致度計算を行う。
例えば、楕円に関する一致度をＰtotal （Ａ）、３角形に関する一致度をＰtotal （Ｂ）、４角形に関する一致度をＰtotal （Ｃ）と表わすとすると、図２のケースでは、コマの一致度：Ｐtotal （ＡＬＬ）＝Ｐtotal （Ａ）＋Ｐtotal （Ｂ）＋Ｐtotal （Ｃ）となる。
【００５１】
なお、本発明にいう特定図形は、上記幾何学図形に限定されるものではなく、上述したように、その他の多角形，星型（☆）のようなキャラクタ，直線，曲線およびその組み合わせなど、多様な図形を用いることを含むものである。
また、計算結果を基に、一致度のポイント順に、検索された画像をサムネイル表示するようにしてもよい。
【００５２】
検索時の指定パターンには、ＡＮＤ条件モードとＯＲ条件のモードを設定することが可能である。例えば、以下のようにする。
（１）ＡＮＤ条件モード時：関数Ｆ｛Ｘ｝は、入力Ｘが所定値以下の場合、負の値となる特性を持たせる。
例えば、検索指定パターンの位置に指定した図形が存在すれば正の値、存在しない場合は負の値となる。外れた場合、全体の一致度点数は低下する。
【００５３】
（２）ＯＲ条件モード時：関数Ｆ｛Ｘ｝は、０以上、または、正の値となる特性を持たせる。
例えば、検索指定パターンの位置に指定した図形が存在すれば正の値、存在しない場合は０となる。外れた場合でも、全体の一致度点数は低下しない。
【００５４】
なお、上述した例では、全ての検索用指定パターン２０の中心位置等の位置、サイズ、向き等の複数の特性値を検索条件として、読み出された原画像の付属情報と比較照合しているが、本発明はこれに限定されず、検索用指定パターン自体の存在の有無のみを検索条件にしても良いし、特性値の内の位置、サイズ、向きおよび色の少なくとも一つの要素や項目を検索条件としても良いし、特性値の他指定パターンの色（色味）を検索条件としても良いし、複数の検索用指定パターンの少なくとも一つを用いても良い。
【００５５】
ただし、検索条件として検索用指定パターン自体の有無が指定される場合や指定される要素や項目や指定パターンが少ない場合には、検索に要する時間は短くなり、ヒットして取り出される原画像の数は増加し、その中に目的とする原画像が含まれている可能性は高いが、検索の精度は低下する。一方、指定される要素や項目や指定パターンが多ければ多いほど検索の精度は向上し、目的の原画像のみがヒットして取り出される可能性があるが、検索時間が長くかかり、要素や項目や指定パターンが正確でないと、ヒットする原画像がないという結果になる虞もある。このため、本発明の検索においては、検索の確実性および精度を考慮して、指定する要素や項目や指定パターンの数を適宜選択すれば良い。
【００５６】
この時、検索指示のユーザインターフェースとして、特定図形などの特定の情報の存在の有無のチェックのみか、または、さらにその位置、大きさ、向き等の特性値や色などをそれぞれ利用するか、また、これらの特性値や色などの種々の項目や要素の関連付けとして、これらの項目や要素をそれぞれ単独で使うか、２つ以上組み合わせて使うかというモードの設定をユーザが指定できるようにしておくのが好ましい。
例えば、上述した数１に示す数式において、係数ｋ１およびｋ２を０として、一致度計算から除外するようにしても良い。
【００５７】
本実施例に示した画像検索方法によれば、タブレットに大まかな図形を描くとか、キーボードから図形の名称や記号を入力するとか、これらを音声入力するといった簡単な指示方法により、指示に適合する画像を検索することが可能になるという効果が得られる。本発明においては、簡単な指示方法により指示された図形や名称（言語）や記号や音声による指定は、特定図形に自動変換され、所要の特性値に数値化され、色味など変えられる。
【００５８】
次に、本発明の他の実施例を説明する。
前述の実施例では、特定の情報として特定図形を用いて検索対象となる特定図形の位置や大きさや向きや色等を付属情報としたが、本実施例では、特定の情報として特定の主要被写体（以下、特定被写体という、ここでは、人物を例に挙げる）とし、その位置や大きさや向きや色などを付属情報として検索を行う点を特徴とするものである。
【００５９】
図６に、検索時の特定被写体指定例６０を示す。ここでは、被写体を人物と設定した後、例えば、楕円形などの単純な図形６１を用いて、その存在の有無、またはその位置や大きさや向きや色の少なくとも一つを指定する。
図７に、上述の指定に対応する検索結果の一例を示す。この例では、人物の位置や大きさのみに基づいて、検索が行われた結果、目的とする画像７０が検索された例を示している。
【００６０】
本実施例に示した画像検索方法によれば、検索対象が人物である場合に、その人物自体、またはその位置や大きさや向きの少なくとも一つを、タブレットで大まかに示すとか、キーボードから人物自体の有無、または人物の位置や大きさや向きの少なくとも一つを入力するといった簡単な指示により、指示に適合する画像を検索することが可能になるという効果が得られる。
【００６１】
なお、人物の位置や大きさに加えて、この人物の服装を色で表現した情報を付加することにより、人物の胴体部分を検索する際に、その色味をチェックするようにすれば、検索の効率が大幅に向上する可能性がある。また、複数の人物の特徴を組み合わせてもよい。
【００６２】
応用分野によっては、特定図形または特定被写体について、位置がほぼ固定的である場合があるが、そのような場合には、大きさまたは大きさと向き、さらには色を付属情報として記録すればよい。
また、特定図形または特定被写体について、大きさがほぼ固定的である場合には、位置または位置と向き、さらには色を付属情報として記録すればよい。
【００６３】
本発明のさらに他の実施例を説明する。
前述の各実施例では、静止画像を検索対象としていたが、本実施例では、動画像を検索対象として、特定図形または特定被写体（人物）の当初の位置や移動方向を付属情報とし、これらを指定して検索を行う点を特徴とするものである。
【００６４】
図８に、検索時の特定被写体指定例８０を示す。ここでは、特定被写体を人物と設定した後、その特定被写体８１の移動方向（ベクトル）８２を矢印で指定する。
図９に、上述の指定に対応する検索結果の一例を示す。この例では、人物の当初の位置や大きさのみに基づいて、検索が行われた結果、目的とする動画像９０ａ，９０ｂが検索された例を示している。
【００６５】
本実施例においては、付属情報の登録を行う際に、動画像のフレーム内で抽出した指定図形（特定図形）または指定被写体（特定被写体）が、所定フレーム後にどのように移動しているかの追跡を行い、そのベクトル情報を付属情報に含めて記憶するようにする。検索を行う際には、指定の図形およびその動きベクトル（大きさおよび方向の少なくとも一方）に基づいて、付属情報とのマッチングを行う。
【００６６】
他の方法としては、付属情報をフレーム単位で作成する方法が可能である。すなわち、検索を行う際には、フレーム単位で指定図形（特定図形）または被写体を検出した場合、そのフレームから所定時間後（あるいは、その中間フレーム）のフレームに該当する位置（この位置は、指定した動きベクトルから求める）に指定図形が存在するか否かを、付属情報とのマッチングによりチェックするという方法である。
【００６７】
本実施例に係る方法においては、プログラムは入力のシーンの指定情報を、シンプルな形状とその動きに解読し、データベース内の各画像の付属情報との一致度を計算する。計算結果は、ポイント順に、該当する動画像のコマ表示により出力（サムネイル表示）する。
【００６８】
上記実施例に示した画像検索方法によれば、検索対象が動画像中の人物である場合に、その当初の位置や移動方向を、タブレットで大まかに示すとか、キーボードから入力するとか、音声入力するといった簡単な指示により、指示に適合する動画像を検索することが可能になるという効果が得られ、例えば、ビデオ編集における大まかなシーン記述での画像検索が容易になる。
【００６９】
本発明のさらに他の実施例を説明する。
前述の各実施例では、音声情報を含まない画像を検索対象としていたが、本実施例では、動画像などの音声情報を含む画像を検索対象として、音声情報の概略を付属情報に含めるようにして、検索を行う際には、音声情報の概略を指定して検索を行うというものである。
【００７０】
例えば、人物を被写体とする動画像において、動きの情報に加えて音声情報を取り込み、音声変換により、その内容をデータベースに記憶する。これにより、検索時のキーワードとして、例えば、被写体：人物、音声情報：学校と指定することにより、シーンとして、人物が存在し、かつ、学校という単語に関連するシーンが検索される。なお、動画像に限らず、静止画像に音声情報を付加したものを検索対象としても良い。
【００７１】
本実施例に示す画像検索方法によれば、検索対象が音声情報を有する場合に、その情報を利用することにより、指示に適合する画像を検索することが可能になるという効果が得られ、被写体の動きのみならず、その発言内容などからの検索が可能になり、ビデオ編集の際などに有効である。なお、もちろん、これ（音声情報）に加えて、特定図形または特定の主要被写体（人物）の各種の情報を追加指定するようにしても良い。
【００７２】
本発明のさらに他の実施例を説明する。
本実施例においては、画像中の特定の人物を人物同定処理を用いて同定することを特徴とするものである。すなわち、予め、特定の人物の顔画像と人物名を付属情報として登録しておき、検索を行う際には、上述の登録されている人物の内から必要な人物を顔画像または人物名により指定して、検索を行うというものである。
【００７３】
具体的には、顧客が予め名前と顔画像を組み合わせて、Ｎ人分の情報を登録する。ここで、画像中の特定の人物を人物同定処理を用いて同定する際には、例えば、電子情報通信学会論文誌Ｖol．Ｊ80−Ｄ-II Ｎo.8,pp.2218-2224小杉等による「粗密濃淡モザイク特徴をキーとした類似顔画像の検索」、同 pp.2225-2231 相馬等による「取得過程の極端に異なる顔画像の照合」等に記載されている技術が適用可能である。
【００７４】
ラボに持ち込まれた画像に対して、上記処理により、顔抽出および人物同定を行い、付属情報を作成する。すなわち、各コマに誰が写っているかを表わす付属情報を、画像データに付加して記憶する。もちろん、ここで、人物の位置や大きさの情報を含めてもよい。顧客が、人物名を指定して検索を依頼したときには、付属情報から該当する人物が写っている画像コマを取り出す。
【００７５】
本実施例に示した画像検索方法によれば、検索対象となる人物の顔画像と人物名が対応付けられて付属情報化されるので、その情報を利用することにより、指示に適合する画像を検索することが可能になるという効果が得られ、例えば、家族の名前で、思い出のある写真を容易に検索することが可能になる。なお、対象画像は、静止画像だけでなく、動画像も対象とすることができる。
【００７６】
また、音声情報を有する画像の場合には、検索時に、これらの情報を組み合わせて用いることも可能である。
さらに、上述のラボと顧客のパソコン（パーソナルコンピュータ）を通信回線で接続しておけば、上述の処理を、オンラインにより、より迅速に行うことが可能になる。
【００７７】
本発明のさらに他の実施例を説明する。
前述の各実施例では、撮影画像などを検索対象としていたが、本実施例では、パーソナルコンピュータ（ＰＣ）などで作成されたコンピュータグラフィックス（ＣＧ）画像を検索対象とし、単純な図形や大まかな色や特定の物体によりシーンを指定して、ＣＧ画像の作成・編集データを用いて検索を行う点を特徴とするものである。
【００７８】
本実施例においては、このＣＧ画像を作成した時の編集データを特定の情報とするとともに付属情報として当該ＣＧ画像に付属させてデータベースに記憶させる。すなわち、データベース機関では、ＣＧ画像と編集データをセットで蓄積しておく。ここで、編集データは、ＣＧ画像中の単純な図形や大まかな色や人物や動物や自動車や建物などの特定の物体を作成した時のそれらの図形や色や物体を表すＣＧコマンドであるのが好ましい。
本実施例における検索においては、ユーザ（顧客）が、単純な図形や大まかな色や人物や動物や自動車や建物などの特定の物体によりシーンを指定する。この指定は、手書き、描画ソフトウェアやＣＧソフトウェア等によって指定するのが良い。
【００７９】
次に、データベース機関では、データベース処理装置において、ユーザからのシーン指定データをＣＧコマンドに変換する。ここで、ユーザがシーン指定した図形の形状や、位置、サイズ、向き等の特性値や、色などを所定パターンに分類しておくのが好ましい。
続いて、変換したＣＧコマンドと、データベース内のＣＧ画像に付属情報として付随しているＣＧ編集データとの間で、一致度や類似度を計算し、付属情報とのマッチングを行う。なお、計算結果は、ポイント順に、該当するＣＧ画像のコマ表示により出力（サムネイル表示）するようにしても良い。
【００８０】
本実施例に示す画像検索方法によれば、検索対象となるＣＧ画像の作成・編集データを付属情報として用いることにより、ＣＧ画像検索処理での演算量を低減でき、ユーザの指示に適合するＣＧ画像を迅速かつ精度良く検索することが可能になるという効果が得られる。なお、対象ＣＧ画像は、静止画像だけでなく、動画像も対象とすることができる。ＣＧ動画像を検索対象とする場合には、前述と同様に、特定の図形や物体と、その移動ベクトル（その大きさおよび方向の少なくとも一つ）を指定するのが良い。また、音声情報を有する画像の場合には、検索時に、これらの情報を組み合わせて用いることも可能である。
【００８１】
本発明のさらに他の実施例を説明する。
本実施例では、メッセージ情報を備える画像を検索対象とし、画像検索用に画像に付随するメッセージ情報をチェックし、検索を行う。
本実施例においては、前述した音声情報を持つ画像を検索する場合と同様に、画像に付随するメッセージ情報を特定の情報とするとともに付属情報として当該画像に付属させてデータベースに記憶させる。
【００８２】
ここで、画像に付属する付属情報としては、メッセージ情報として画像に合成したテキストデータ、またはマークや撮影画像やＣＧ画像等の画像データを用いるのが好ましいが、前述した音声情報のように、画像中ではなく画像に付随させたテキストデータや画像データ、例えば、プリント画像の裏面側に、白縁のあるプリント画像ではその白縁の部分に記録（合成）したテキストデータや画像データを用いても良い。
例えば、テキストデータとしては、ワードプロセッサ等の印刷文字データやサイン等の手書き文字データなどを用いることもできるし、画像から文字抽出を行って得られた文字データを用いることもできる。
【００８３】
本実施例における検索においては、ユーザ（顧客）が、メッセージ情報を、文字自体、その内容、筆跡の形状（サイン等の場合）などで指定したり、メッセージ情報として登録されている参照用画像を選択することにより、指定し、マッチング処理を行う。
例えば、文字自体を検索するモードでは、前述の特定図形を用いた検索での特定図形の特性値や色等の指定と同様に、文字自体の有無、文字の位置、サイズ、向き、色等の少なくとも一つを用いて指定し、一致度を計算して一致した画像を選び出すマッチング処理を行えばよい。また、文字検索モードでも、文字の内容を指定する場合には、指定された漢字などの文字の内容を解釈して、マッチング処理を行えば良い。また、サイン等の手書き文字の場合には、筆跡の形状を指定して、マッチング処理を行えば良い。
【００８４】
さらに、参照用画像を用いて検索する場合には、ユーザが、メッセージ情報として登録されたマークや撮影画像やＣＧ画像等の複数の参照用画像の中から参照用画像のエリアを指定したり、予め用意された図柄パターンを選択する等によって、一つ以上の参照用画像を選択して指定し、画像マッチングを行えば良い。
なお、マッチング処理における一致度の計算結果は、ポイント順に、該当する画像のコマ表示により出力（サムネイル表示）するようにしても良い。
【００８５】
本実施例に示す画像検索方法によれば、検索対象となる画像に付随するメッセージ情報として、文字などのテキストデータや参照用の画像データを用い、文字自体の特性値や色、または、その内容や筆跡の形状などの指定するなどの簡単な指定方法により、予め登録されている参照用画像をエリア指定や図柄パターンの選択等による指定などの簡単な指定方法により、指定に適合する画像を検索することが可能になるという効果が得られる。
【００８６】
上記各実施例において、検索結果の画像を表示する際には、例えば、背景などの動きのない部分を除いて、検索条件指定に合致するエリアを優先して表示するようにするとよい。あるいは、検索条件指定に合致するエリアを他のエリアとは異なる、例えば、輝度強調，ブリンク（点滅）などの特別な画像処理結果の強調表示により表示することも有効である。
【００８７】
上述のような表示方法を採用すれば、検索した画像を、モニタなどに転送する際の転送データ容量を大幅に削減できるので、表示の応答速度が向上し、使い勝手がよくなるという効果がある。また、表示画面内の目的とする画像の見やすさが向上し、やはり使い勝手がよくなる効果がある。
【００８８】
また、表示を行う際に、表示装置の表示能力をチェックして、その能力に応じた画像データに加工して表示させるようにすることも好ましいことである。またさらに、データベース上で、検索したエリアをエリア単位で管理するようにしておき、アクセス頻度に応じて、検索時の参照優先順位を設定することや、データ記憶時の解像度を設定することも有効である。
【００８９】
この方法では、例えば、顧客が携帯用端末からデータベースのあるセンタ（データベース機関）にアクセスしたような場合に、顧客の携帯用端末の表示能力に関する情報をＩＤと共に転送するようにして、センタからのデータ転送の際に、必要に応じて、画像圧縮や２値化、線画化、トリミング（主要被写体のみの切り出し）などの画像データ加工を行う。
【００９０】
上述のような画像表示時の処理を効率的の行うことにより、データ通信量の低減が可能になると共に、携帯用端末における画像表示までの時間が短縮され、さらに、表示画像の見やすさが向上するなどの効果が得られる。
【００９１】
上述の、各種の付属情報を具備した画像データの管理を行う場合には、画像を被写体毎に領域分割して、各分割エリア毎にラベルを付けて登録することが好ましい。そして、ラベルを付けた部分のみを切り出してファイル化するようにすれば、データ容量の削減に大きな効果があるばかりでなく、検索の際の速度の向上にも、大きな効果がある。また、例えば、上述のラベルを付けた部分の画像のみを、高解像度で記憶し、その他の領域は画像圧縮して記憶するなどの方法を用いることも可能である。
【００９２】
各領域の付属情報としては、画像ファイル名，位置（中心，輪郭画素），色などの特性、および、通算アクセス回数／ヒット回数などが挙げられる。これらを用いて、例えば、アクセス頻度に応じて、記憶手段内での記憶階層の制御を行うようにすれば、検索の際の速度の向上にさらに大きな効果がある。また、利用頻度の高いデータは、個別に保存するようにして、より利用しやすくすることも可能である。
【００９３】
以下、上述のような、本発明に係る画像検索方法を応用した、画像処理方法を説明する。第１の方法は、ラボにおける最初のプリント処理（いわゆる、同時プリント）や先の再プリント処理（これらを代表して先のプリントという）時に、プリント画像中の特定図形または被写体等の特定の情報を抽出して、抽出された特定情報やその関連情報（特性値や色）を付属情報として、プリントのための画像データおよび画像処理のパラメータと共にデータベース等の記憶手段に記憶させておくというものである。
【００９４】
この画像が含まれるフィルムの再プリントまたは後の再プリント（これらを代表して後のプリントという）の依頼があった場合には、顧客が指定した画像コマから特定図形または被写体等の特定の情報を抽出して、先に記憶させた付属情報との一致度の大きいもの（所定の閾値を超えるもの）の画像の先のプリント処理時の画像処理条件（処理パラメータ）により、後のプリントの処理を行うようにする。
【００９５】
この画像処理方法によれば、後のプリント時に顧客が持参したフィルム中の画像から、先のプリント時の画像処理条件を検索することが可能になり、顧客に満足を与えられるような、再現性に優れたプリントを行うことが可能になるという効果が得られる。なお、上記処理は、写真フィルムだけではなく、ＤＳＣ（デジタルスチルカメラ）などの撮影画像記録媒体などのようにプリント用画像データが記録されたスマートメディアやＭＯやフォトＣＤ等の画像記録媒体でも、同様に適用できる。
【００９６】
また、顧客が、再プリントの注文時に、写真フィルムまたは撮影画像記録媒体の撮影画像データではなく、同時プリント時の写真プリントのみを持ち込んだ場合には、そのプリントをスキャナで読み取り、画像検索用の特定図形等の特定の情報の抽出処理を実行して、抽出された特定の情報やその関連情報を検索条件として、データベース内の同時プリント時に記録した写真フィルムからの読取画像データを検索して類似度の計算（マッチング処理）を行い、該当するコマを取り出し、プリント処理においては、該当コマに対応する同時プリント時の読取画像データと処理パラメータとを読み出し、これらを用いて、プリント処理を行ってもよい。
【００９７】
第２の方法は、１本のフィルムストリップ（いわゆる、１件の写真フィルム）内、または１枚の撮影画像記録媒体内、もしくはユーザ（顧客）が一時に注文した１本または複数の写真フィルムまたは１枚または複数の撮影画像記録媒体（いわゆる、１回の注文）内での焼き付けのバラツキ（変動）を防止可能とするための方法である。すなわち、１件または１回の注文（これらを代表して１注文という）の写真フィルムや撮影記録媒体内で、特定図形または被写体等の特定の情報が類似した構図（各図形の配置やサイズや向きや色）である場合は、１注文に属するこれらのコマ間で、仕上がりの色調，濃度などを揃えるように画像処理を調整するのが良い。
【００９８】
本実施例によれば、画像の構図に基づいて類似画像を判別するので、色や濃度の影響を受けにくくなり、これにより、１注文の写真フィルムや撮影画像記録媒体内で極端に色調，濃度などが異なるプリントを作製することがなくなり、顧客に不満を持たせることがなくなるという効果が得られる。
【００９９】
以上のような本発明に係る画像検索方法および画像処理方法は、それぞれ図１０に示すような画像検索装置およびこれを用いるプリントシステムによって実施することができる。
同図に示すプリントシステム１００は、上述した本発明の画像検索方法を実施する画像検索装置（以下、単に検索装置という）１０２と、検索装置１０２に接続され、検索装置１０２と共に、本発明の画像処理方法を実施するフォトプリンタ１０４と、検索装置１０２にインターネット等の通信網によってオンライン接続され、本発明に係る画像検索方法および画像処理方法の実施およびプリント出力の依頼を行うことができるユーザ端末１０６とを有する。
【０１００】
まず、画像検索装置１０２は、画像の画像データと共に、画像に存在およびまたは付随する特定情報自体または特定情報の関連情報などを画像データの付属情報として蓄積記憶する画像データベース（以下、単にデータベースまたはＤＢという）１０８と、データベース１０８に記憶するために特定情報自体または特定情報の関連情報を付属データとして取得する付属情報取得部（以下、単に取得部という）１１０と、検索条件として付属情報の少なくとも一部を指定する付属情報指定部（以下、単に指定部という）１１２と、この指定部１１２によって指定された検索条件に基づいてデータベース１０８を検索して、選び出された画像データに付属している付属条件と、検索条件として用いた付属情報の少なくとも一部との一致度を求めて、所定値以上の一致度を持つ画像を読み出す画像検索部（以下、単に検索部という）１１４と、検索部１１４による検索結果や読み出された画像を表示する表示部１１６とを有する。
【０１０１】
また、フォトプリンタ１０４は、ネガやリバーサルなどのフィルムＦからフィルム画像の画像データを読み取る、あるいはデジタルカメラによる撮影画像の画像データ記録媒体Ｒから画像データを読み取る、もしくはＣＧ画像作成装置１１７などの画像データ供給源から画像データを受け取る画像入力部１１８と、画像入力部１１８から受け取った画像データを分析して良質のプリント作成のために画像データを処理する画像処理条件を設定し、設定された画像処理条件または検索装置１０２から受け取った画像処理条件に従って、画像入力部１１８または検索装置１０２から受け取った画像データを画像処理する画像処理部１２０と、画像処理部１２０によって画像処理された画像データを用いて写真感光材料などの記録材料に画像を記録（潜像焼付、現像定着）してプリントＰとして出力するプリント出力部１２２とを有する。なお、図示例では、画像処理部１２０は、画像データを取得部１１０に転送し、取得部１１０で得られた付属情報を受け取り、画像データに付属させてＤＢ１０８に転送し、蓄積記憶させる機能も有する。
【０１０２】
ユーザ端末１０６は、画像を表示可能な表示部１２４と、検索条件や画像や記号や言語やプリントの注文情報などを入力する入力部１２６と、画像や検索条件や注文情報を検索装置１０２に転送するインターフェース（Ｉ／Ｆ）１２８とを有する。なお、検索装置１０２の検索部１１４には、ユーザ端末１０６のＩ／Ｆ１２８から画像や種々の情報を受信するＩ／Ｆ１１５を有する。
【０１０３】
検索装置１０２においては、取得部１１０は、特定の幾何学図形、特定の被写体、メッセージ情報、特定の人物の顔画像からなる人物情報などの特定情報の概略をフリーハンドで入力するタブレット１３０と、特定の幾何学図形、特定の被写体、メッセージ情報、特定の人物の顔画像と人物名とからなる人物情報、ＣＧ画像の編集データなどの特定情報の概略や音声情報からなる特定情報の意味などをキーボードやマウスによるＧＵＩ（グラフィカルユーザインターフェース）を使って入力し、必要に応じてメッセージ情報などではその意味を認識して入力する入力部１３２、画像に付随する音声情報または画像に存在する特定情報の音声情報を入力し、その意味を認識して文字認識された特定情報に変換する音声認識部１３４と、これらのタブレット１３０や入力部１３２や音声認識部１３４から入力された特定情報の概略を用いて、フォトプリンタ１０４の画像処理部１２０から受け取った画像の画像データから特定情報を抽出する抽出部１３６と、抽出部１３６で抽出された特定情報から特定情報自体やその関連情報、例えば認識されたメッセージ情報および音声情報等の特定情報、特定の幾何学図形、特定の被写体、特定の人物の顔画像と人物名とからなる人物情報等の特定情報の位置、大きさ、向き、色、存在の有無、動画像の場合の特定情報の移動方向（移動ベクトルの方向）等の関連情報、ＣＧ画像の編集データ等の特定情報自体などや、音声認識部１３４や入力部で認識されたメッセージ情報および音声情報等の特定情報自体を、ＤＢ１０８に蓄積すべき付属情報として設定し、設定した付属情報を画像データに関係付け（て画像処理部１２０に転送す）る付属情報設定部（以下、単に設定部という）１３８とを有する。
【０１０４】
従って、フォトプリンタ１０４の画像処理部１２０は、取得部１１０の設定部１３８から伝送された付属情報を画像の画像データに付属させて、ＤＢ１０８に転送し、その状態で蓄積記憶させる。ここで、画像処理部１２０から抽出部１３６および／またはＤＢ１０８に転送される画像データは、画像処理前のものでも良いし、画像処理後のものでも良いし、両画像データは、同じでも異なっていても良い。なお、画像処理前の画像データであれば、画像入力部１２０から転送しても良い。
なお、ＣＧ画像の場合には、画像データに付属情報として付属させる編集データをＣＧ画像作成装置１１７から直接設定部１３８に転送しても良いし、画像データおよびこれに付属する編集データを直接ＣＧ画像作成装置１１７からＤＢ１０８に転送しても良い。
さらに、フォトプリンタ１０４からではなく、別のフォトプリンタや、画像読取装置（スキャナ）や、画像処理装置や、画像データ記録媒体のドライバなどの別の画像データ供給源から取得部１１０で付属情報を取得するための画像データやＤＢ１０８に蓄積するための画像データを受け取っても良い。また、同じ画像の画像データであれば、これらの２種の画像データの一方をフォトプリンタ１０４から、他方を別の画像データ供給源から受け取っても良い。
【０１０５】
検索装置１０２の指定部１１２は、取得部１１０で用いるタブレット１３０や入力部１３２や音声認識部１３４などやＣＧ作成装置１１７などを用いて、検索条件として付属情報の少なくとも一部、例えば、上記音声情報やメッセージ情報等の特定情報自体の概略、特定情報の存在の有無、位置、大きさ、向き、または移動ベクトルの方向、色等の関連情報の少なくとも１つを指定するものである。なお、指定部１１２で用いるタブレット１３０や入力部１３２や音声認識部１３４やＣＧ作成装置１１７などは、取得部１１０などで用いるものと同じであっても良いし、異なるものであっても良いし、さらには、取得部１１０と、指定部１１２を個別に設けるのではなく、共用し、１つで画像データの付属情報の取得と検索条件として付属情報の少なくとも１つの指定とを行っても良い。
【０１０６】
検索部１１４による検索の結果、ＤＢ１０８から読み出された画像データは、画像として表示部１１６に表示されるが、表示画像は、選び出された画像の確認ができれば良いので、特に高精細である必要はなく、ＤＢ１０８から読み出された画像データをそのまま表示しても良いのはもちろんであるが、高精細な画像が表示可能な表示部１１６であれば、フォトプリンタ１０４の画像処理部１２０で処理した後の高精細な画像を表示するようにしても良い。すなわち、表示部１１６を検索装置１０２とフォトプリンタ１０４とで共用しても良い。さらに、検索部１１４で選び出された画像データを確認するために、これを直接フォトプリンタ１０４のプリント出力部１２２に転送して、プリント出力するようにしてもよいし、別のプリンタや簡易型プリンタに転送して出力するようにしても良い。
【０１０７】
ところで、本発明の画像処理方法においては、フィルムＦや画像データ記録媒体Ｒがなくても、プリント（ハードコピー画像）Ｐがあれば、再注文プリントが可能である。このプリントＰから読み取られる画像データは、特定図形や特定被写体などの特定情報が、検索に必要な程度の画素密度（解像度）有していれば良いので、フォトプリンタ１０４の画像入力部１１８に備えられている反射原稿用スキャナで読み取っても良いし、全く別のスキャナや、簡便なスキャナでよみとって、得られた画像データを取得部１１０の抽出部１３６に直接転送しても良い。この時には、抽出部１３６で抽出された特定情報および／またはその関連情報をそのまままたは設定部１３８で付属情報に変換して、直接検索部１１４に転送しても良い。なお、この場合には、ＤＢ１０８から読み出された画像データに対しても、特定情報の抽出処理が行われるので、検索部１１４にＤＢ１０８から読み出された画像データを直接取得部１１０の抽出部１３６に転送しても良い。
さらに、１注文内の複数の画像中の特定図形や特定被写体等の特定情報の一致度を検索部１１４で算出する場合にも抽出部１３６による特定情報の抽出結果を直接検索部１１４に転送するのが良い。
【０１０８】
なお、上記各実施例はいずれも本発明の一例を示したものであり、本発明はこれらに限定されるべきものではなく、本発明の要旨を逸脱しない範囲において、改良や設計の変更をしても良いことは言うまでもない。
【０１０９】
【発明の効果】
以上、詳細に説明したように、本発明によれば、原画像中に存在する被写体（人物や物体）の形状（特定の幾何学図形など）や特定の被写体自体等や原画像に合成されたまたは付随するメッセージ情報や音声情報などをキーとして、画像検索を効率的に行う画像検索装置を実現することが可能である。
【０１１０】
より具体的には、本発明に係る画像検索方法によれば、第１に、タブレットや画像表示装置に大まかな図形を描くとか、キーボードから図形の名称や記号を入力するとか、これらを音声入力するといった簡単な指示方法により、指示に適合する画像を検索することが可能になるという効果が得られる。なお、検索対象が人物である場合にも、同様に、その人物自体、またはその位置や大きさや向きの少なくとも一つを、タブレットや画像表示装置で大まかに示すとか、キーボードから人物自体の有無、または人物の位置や大きさや向きの少なくとも一つを入力するといった簡単な指示により、指示に適合する画像を検索することが可能になるという効果が得られる。
【０１１１】
第２には、検索対象が動画像中の人物である場合に、その当初の位置や移動方向を、タブレットなどで大まかに示すとか、キーボードから入力するとか、音声入力するといった簡単な指示により、指示に適合する動画像を検索することが可能になるという効果が得られ、例えば、ビデオ編集における大まかなシーン記述での画像検索が容易になる。
【０１１２】
第３には、検索対象が音声情報を有する場合に、その情報を利用することにより、指示に適合する画像を検索することが可能になるという効果が得られる。なお、対象画像は、静止画像だけでなく、動画像も対象とすることができ、被写体の動きのみならず、その発言内容などからの検索が可能になり、ビデオ編集の際などに有効である。
【０１１３】
第４には、検索対象となる人物の顔画像と人物名が対応付けられて付属情報化して、その情報を利用することにより、指示に適合する画像を検索することが可能になるという効果が得られる。例えば、家族の名前で、思い出のある写真を容易に検索することが可能になる。なお、対象画像は、静止画像だけでなく、動画像も対象とすることができるし、音声情報を有する画像も対象とすることができる。
【０１１４】
また、第５には、検索対象が音声情報を有する画像の場合には、検索時に、これらの情報を組み合わせて用いることも可能である。なお、この場合にも、対象画像は、静止画像だけでなく、動画像も対象とすることができる。
【０１１５】
また、第６には、検索対象となるＣＧ画像の作成・編集データを付属情報として用いることにより、ＣＧ画像検索処理での演算量を低減でき、ユーザの指示に適合するＣＧ画像を迅速かつ精度良く検索することが可能になるという効果が得られる。なお、対象ＣＧ画像は、静止画像だけでなく、動画像も対象とすることができる。ＣＧ動画像を検索対象とする場合には、前述と同様に、特定の図形や物体と、その移動ベクトル（その大きさおよび方向の少なくとも一つ）を指定するのが良い。また、音声情報を有する画像の場合には、検索時に、これらの情報を組み合わせて用いることも可能である。
【０１１６】
また、第７には、検索対象となる画像に付随するメッセージ情報として、文字などのテキストデータや参照用の画像データを用い、文字自体の特性値や色、その内容や筆跡の形状などの指定するなどの簡単な指定方法により、予め登録されている参照用画像をエリア指定や図柄パターンの選択等による指定などの簡単な指定方法により、指定に適合する画像を検索することが可能になるという効果が得られる。
【０１１７】
なお、検索結果の画像を表示する際には、例えば、背景などの動きのない部分を除いて、検索条件指定に合致するエリアを優先して表示するようにしたり、あるいは、検索条件指定に合致するエリアを他のエリアとは異なる、例えば、輝度強調，ブリンク（点滅）などの特別な画像処理結果の強調表示により表示することも有効である。
【０１１８】
また、表示を行う際に、表示装置の表示能力をチェックして、その能力に応じた画像データに加工して表示させるようにすることも好ましいことである。またさらに、データベース上で、検索したエリアをエリア単位で管理するようにしておき、アクセス頻度に応じて、検索時の参照優先順位を設定することや、データ記憶時の解像度を設定することも有効である。
【０１１９】
上記画像検索方法を応用すれば、後のプリント時に顧客が持参した写真フィルムや撮影記録媒体中の画像から、先のプリント時の画像処理条件を検索することが可能になり、顧客に満足を与えられるような、再現性に優れたプリントを行うことが可能になるという効果が得られる。
【０１２０】
また、上記画像検索方法を応用すれば、画像の構図に基づいて類似画像を判別することができるので、色や濃度の影響を受けにくくなり、これにより、１注文の写真フィルムや撮影記録媒体内で極端に色調，濃度などが異なるプリントを作製することがなくなり、顧客に不満を持たせることがなくなるという効果が得られる。
【図面の簡単な説明】
【図１】 本発明の一実施例に係る画像検索方法を説明するための図で、原画像を示す図である。
【図２】 同、図１に示す原画像を検索する際に、ユーザが入力した検索用指定パターンを示す図である。
【図３】 同、図１に示した原画像中から抽出した特定図形を原画像に重ねて示す図である。
【図４】 同、準備動作としての、図１に示した原画像中から予め定められた特定図形を抽出する動作の概要を示す動作フロー図である。
【図５】 同、記憶手段に記憶されているデータに基づいて画像を検索する動作の流れを示す動作フロー図である。
【図６】 本発明の他の実施例に係る画像検索方法を説明するための図で、被写体指定例を示す図である。
【図７】 同、図６に示した指定に対応する被写体検索結果の一例を示す図である。
【図８】 本発明のさらに他の実施例に係る画像検索方法を説明するための図で、動画像検索時の被写体指定例を示す図である。
【図９】 同、図８に示した指定に対応する動画像検索結果の一例を示す図である。
【図１０】 本発明の画像検索方法および画像処理方法をそれぞれ実施する画像検索装置およびこれを用いるプリントシステムの一実施例のブロック図である。
【符号の説明】
１０ 原画像
１１ ３角形状の山
１２ 人物の顔
１３ 人物の胴体
１４ ビルディング
２０ 検索用指定パターン
２１ 山を象徴する３角形
２２，２３ 人物を象徴する２つの楕円
２４ ビルディングを示す縦型の４角形
３０ 原画像からの図形抽出結果
３１ 山１１に対応する図形
３２，３３ 人物１２，１３に対応する図形
３４ ビルディング１４に対応する図形
３５，３６ 誤検出図形
６０ 被写体指定例
６１ 人物指定図形
７０ 原画像からの図形抽出結果
８０ 被写体指定例
８１ 被写体
８２ 被写体移動方向
９０ａ，９０ｂ 動画像抽出結果
１００ プリントシステム
１０２ 画像検索装置
１０４ フォトプリンタ
１０６ ユーザ端末
１０８ 画像データベース（ＤＢ）
１１０ 付属情報取得部
１１２ 付属情報指定部
１１４ 画像検索部
１１５、１２８ インターフェース（Ｉ／Ｆ）
１１６、１２４ 表示部
１１７ コンピュータグラフィックス（ＣＧ）画像作成装置
１１８ 画像入力部
１２０ 画像処理部
１２２ プリント出力部
１２６、１３２ 入力部
１３０ タブレット
１３４ 音声認識部
１３６ 抽出部
１３８ 付属情報設定部[0001]
BACKGROUND OF THE INVENTION
  The present inventionImage search deviceMore specifically, specific information such as simple figures extracted from the image, subjects such as persons, objects, etc., audio information attached to the images, message information such as characters and marks, etc. , Store it in the database as ancillary information of the image and search for the image based on itImage search deviceAbout.
[0002]
[Prior art]
For example, the most noticeable part when appreciating a portrait is the face of a person. For this reason, when an original image recorded on a film or the like is printed on a copying material such as photographic paper, it is necessary to determine a printing exposure amount so that a person's face is reproduced with an appropriate color and density.
[0003]
For this reason, in portrait photography, for example, a color original image is divided into a large number of pixels by a method as disclosed in Japanese Patent Laid-Open No. 4-346332, and each pixel has three colors, R, G, and B. Based on the result, a histogram for hue and saturation is obtained, and an area where pixels having hue and saturation corresponding to the face of the person exist in the obtained histogram is determined. And the exposure amount is determined from the photometric data in this area.
[0004]
Japanese Patent Laid-Open No. 6-160993 discloses that a region in contact with the outer edge of an image is determined as a background region and is removed or extracted in order to improve the accuracy of extracting a region corresponding to a human face. A technique for making an area into a line figure and determining whether the extracted area corresponds to a human face based on the shape of the neighboring area located around the extracted area and the shape of the extracted area It is shown.
[0005]
However, in these techniques, for example, when an area of hue and saturation close to the skin color such as the ground or a tree exists in the original image, this area is erroneously determined as an area corresponding to a human face. there's a possibility that. In addition, when these areas are adjacent to an area corresponding to a person's face, the two cannot be properly separated, and the original image may not be divided into areas of an appropriate range.
[0006]
[Problems to be solved by the invention]
As described above, when an area other than a person's face is erroneously determined as an area corresponding to a person's face, or an area having a similar hue and saturation adjacent to an area corresponding to a person's face cannot be separated. In such a case, there is a problem that an exposure amount for properly printing the face of the original person cannot be obtained.
[0007]
For such a problem, it is considered effective to use not only the color (hue and saturation) of the site of interest but also information regarding the shape of the site. In other words, an area corresponding to a person's face is considered to have an oval or egg-shaped shape in general. Therefore, in addition to information about color, it is necessary to extract the fact that it has such a special shape. It is sufficient to satisfy the following conditions.
[0008]
As described above, information on the shape of the subject in the original image has extremely important meaning in image detection or image search. In this regard, it is possible to refer to the technique disclosed in Japanese Patent Application Laid-Open No. 9-138471 relating to the applicant's application. The purpose of this technique is to detect the edge of an image, extract the outline of a region corresponding to a person, and perform preferable color reproduction for the region corresponding to a person (particularly a face).
[0009]
In addition to this, the searcher wants to extract the image without having the searcher draw the target image, and even if there is no image close to the image that the searcher wants at hand or within the number that can be listed. As for providing an image search system for making it possible to search an image that has been searched, for example, the descriptions in JP-A-7-21198 and JP-A-8-249352 can be referred to.
[0010]
  The present invention is a further development of the idea of performing area detection using a specific shape in an original image as a key in the above-described prior art. The first object of the present invention is to Image search that efficiently performs image detection or image search (hereinafter collectively referred to as image search) using specific information such as the shape of a subject such as an object or voice information or message information accompanying the original image as a key.apparatusIs to provide.
[0011]
[Means for Solving the Problems]
  Above purposeIn order to achieve the present inventionFirst image retrieval apparatus according toIsEnteredExtract or recognize specific information that is present, associated with, or present and associated with this imageAn extractor to perform,The obtained specific information is used as it is or the related information of the specific information is attached to the image data of the image and stored in the database as attached information.And the attached information setting unit to beAt least some of the attached informationIs specified, the aboveThe database is searched under specified search conditions, the degree of coincidence with the attached information attached to the selected stored image data is obtained, and an image having a degree of coincidence equal to or greater than a predetermined value is read.An image search device having a search unit and a display unit for displaying the read image,
  The specific information existing in the image is M (M is 1 or more) predetermined specific geometric figures for the image, and the related information stored as the attached information is the image Is applied to the M kinds of specific geometric figures, and the position, size, and position corresponding to N (N is 0 or more) specific geometric figures obtained as a result of extraction. In the step of designating at least a part of the attached information as the search condition, the specific geometry is any one of: and size and orientation, position and size and color, or position and size and orientation and color. For L types of figures (L is 1 to M), the presence / absence of the specific geometric figure or the outline of at least one of the position, size, direction and color is designated.It is characterized by.
[0012]
  Also,According to the present inventionSecondImage searchapparatusIsAn extraction unit for extracting or recognizing specific information that is present in or associated with the image with respect to the input image, and that is associated with the image, and the specific information obtained as it is or related information of the specific information Is attached to the image data of the image and stored in the database as attached information, and when at least a part of the attached information is specified as a search condition, the specified search condition A search unit that searches a database, obtains a degree of coincidence with the attached information attached to the selected stored image data, reads an image having a degree of coincidence of a predetermined value or more, and displays the read image An image search device having a display unit
  The specific information existing in the image is an object defined by M (M is 1 or more) types of specific geometric figures for the image, and is stored as the associated information. The information is obtained by applying a process of extracting the M kinds of specific subjects to the image, and the positions and sizes corresponding to N (N is 0 or more) specific subjects obtained as a result of the extraction. In the step of designating at least a part of the attached information as the search condition, the position, size and orientation, position, size and color, or position, size and orientation and color are specified. For L types of subjects (L is 1 to M), the presence or absence of the specific subject or at least one outline of the position, size, orientation, and color is designated.It is characterized by that.
[0014]
Also, the image search method according to the present invention extracts a specific geometric figure predetermined for an image having a predetermined time or number of frames from an image having temporal continuity. Applying processing, the movement direction of the image corresponding to the arbitrary geometric figure is obtained based on the comparison of the positions of the extracted arbitrary geometric figures between frames, and the information on the geometric figure and the geometry Ancillary information consisting of the moving direction of the image corresponding to the figure is stored in association with the data of the corresponding image, and the geometricity is determined at an interval of a predetermined time or the number of frames for an image having temporal continuity. The outline of the direction of the figure and its motion vector is designated, the degree of coincidence with the attached information attached to the stored image data is obtained, and an image having a degree of coincidence greater than or equal to a predetermined value is read. .
[0015]
  In addition, the image search method according to the present invention provides a predetermined specific time or an interval of the number of frames for an image having temporal continuity.Defined by geometric figuresApplying a process of extracting a subject, and based on a comparison of positions of the extracted subject between frames, a moving direction of an image corresponding to the subject is obtained, and information on the subject and a moving direction of the image corresponding to the subject are obtained. Is attached to the data of the corresponding image and stored, and for the image having temporal continuity, an outline of the direction of the subject and its motion vector is given at intervals of a predetermined time or number of frames. The degree of coincidence with the attached information attached to the stored image data is obtained, and an image having a degree of coincidence of a predetermined value or more is read out.
[0016]
The image search method according to the present invention recognizes audio information for an image having audio information, stores the recognized audio information attached to data of the corresponding image, and stores the audio information in an image having audio information. On the other hand, the outline of the voice information is designated, the degree of coincidence with the voice recognition information attached to the stored image data is obtained, and an image having a degree of coincidence of a predetermined value or more is read.
[0017]
In the image search method according to the present invention, in addition to the audio information, the position and size corresponding to a specific geometric figure, or the position and size and orientation, or additional information including color as necessary is attached. The presence or absence of a specific geometric figure or at least one of position, size, orientation and color can be designated as the search condition. In addition to the audio information, position and size corresponding to a specific subject, or position and size and orientation, or additional information including color as necessary can be attached. The presence / absence of the subject, or at least one of the position, size, orientation, and color can be designated.
[0018]
In addition, the image search method according to the present invention registers in advance personal information consisting of a face image and a person name of a specific person A (A is 1 or more), and extracts the person and the person information at the time of image storage. Depending on the person identification process used, the position and size of the specific person, the position and size and orientation, or, if necessary, additional color information or additional information consisting only of the presence or absence of the person, the corresponding image data Specify the position, size, direction and / or color of a specific person B (B is 1 to A) or only the presence / absence of a person as a search condition, The degree of coincidence with the attached information of the stored image is obtained, and an image having a degree of coincidence of a predetermined value or more is read.
[0019]
Further, in the image search method according to the present invention, the position of the specific person is determined by the person identification process using the person information at a predetermined time or at an interval of the number of frames with respect to an image having continuity in time. Attached information that includes either size, position, size and orientation, or, if necessary, additional color, or only the presence or absence of a person, is attached to the corresponding image data and stored, and is continuous in time As a search condition for an image having a characteristic, the presence or absence of a specific person B (B is 1 to A) is specified, or at least one of position, size, orientation and color and an outline of the direction of the motion vector are designated. The degree of coincidence with the attached information of each stored image is obtained, and an image having a degree of coincidence of a predetermined value or more is read out.
[0020]
In addition, the image search method according to the present invention performs the position and size of a specific person, or the position and size and orientation, or as necessary, by performing person identification processing using the person information on an image having audio information. In addition, additional information including either color or only presence / absence of a person is attached to the corresponding image data together with the recognized audio information and stored, and a specific person is used as a search condition for an image having audio information. Presence / absence of B people (B is 1 to A) or at least one of position, size, orientation and color and outline of audio information are specified, and the degree of coincidence with attached information of each stored image is obtained, and predetermined An image having a degree of coincidence equal to or greater than a value is read.
[0021]
The image search method according to the present invention recognizes the message information for an image having message information, stores the recognized message information attached to data of the corresponding image, and stores the message information. The outline of the message information is designated for the image having, the degree of coincidence with the message recognition information attached to the stored image data is obtained, and the image having the degree of coincidence of a predetermined value or more is read. And
Here, the message information is preferably text data or accompanying image data.
[0022]
The image search method according to the present invention stores, for a computer graphics image, edit data used when creating the computer graphics image, attached to the image data of the computer graphics image, A scene of the computer graphics image according to the presence or absence of at least one of a specific geometric figure, a specific object and a specific color, or at least one of its position, size and orientation with respect to the computer graphics image The designated scene designation data is converted into a computer graphics command, and the degree of coincidence between the converted computer graphics command and the edited data attached to the selected stored image data is obtained. Reading images with a degree of match equal to or greater than a predetermined value The features.
[0023]
Here, the computer graphics image is a computer graphics image having continuity in time,
The attached information includes at least the specific geometric figure and the specific object at a predetermined time or frame interval with respect to the computer graphics image having temporal continuity in addition to the editing data. The specific geometric figure obtained by determining a moving direction of an image corresponding to at least one of the specific geometric figure and the specific object based on a comparison of positions between frames for one; At least one of the specific objects and the moving direction of the image;
As the search condition, with respect to the computer graphics image having temporal continuity, presence or absence of at least one of a specific geometric figure and a specific object at a predetermined time or an interval of the number of frames, or its presence Preferably, at least one of position, size and color and an outline of the direction of the motion vector are specified.
[0024]
In the image search method according to the present invention, the designation step includes a step of designating at least one of a symbol, a language, and a voice, and at least one of the designated symbol, language, and voice is the specific information or its It is preferable to have a step of automatically converting the information into attached information.
[0025]
In the image search method according to the present invention, it is preferable that an area matching the search condition in the image data is displayed with priority in time. It is also preferable to highlight an area that matches the search condition in the image data by image processing different from other areas.
[0026]
Furthermore, in the image search method according to the present invention, it is preferable to process the image data according to the display capability of the display terminal. In addition, it is preferable to manage the searched image areas in units of areas and set the reference priority for searching according to the access frequency. Further, it is preferable to manage the searched image area in units of areas and set the resolution at the time of data storage according to the access frequency.
[0030]
DETAILED DESCRIPTION OF THE INVENTION
  Image retrieval according to the present inventionapparatusIs described in detail below based on a preferred embodiment shown in the accompanying drawings.
[0031]
1 to 3 are views for explaining an image search method according to an embodiment of the present invention. FIG. 1 shows an original image 10 supplied from an image data source and stored in a storage means such as a database (not shown). FIG. 2 shows that when the original image 10 is searched, the user has input by handwriting using a pen or the like specified by an image specifying device (not shown), or an image input device or image display device. The search designation pattern 20 input or displayed / input using a mouse or a keyboard is shown in FIG. FIG. 3 shows a specific geometric figure (circle, ellipse, etc.) determined in advance from the original image 10 shown in FIG. 1 by a program such as pattern matching. A result 30 of extracting a figure such as a triangle or a quadrangle (hereinafter referred to as a specific figure) is shown.
Various specific shapes can be detected by using the edge tracking method disclosed in the above-mentioned Japanese Patent Application Laid-Open No. 9-138471.
[0032]
In the original image 10 shown in FIG. 1, 11 indicates a triangular mountain, 12 and 13 indicate a person's face and body, and 14 indicates a building, respectively. Further, in the search specifying pattern 20 shown in FIG. 2, the user searches for the original image 10 shown in FIG. 1 by using a triangle 21 symbolizing a mountain and two ellipses symbolizing a person (that is, a face). The fuselage) 22 and 23 and the vertical quadrilateral 24 indicating the building are arranged in correspondence with positions where they are supposed to exist.
[0033]
In FIG. 3, the graphic extracted by the program of the image search method according to the present embodiment from the original image 10 shown in FIG. 1 is indicated by a dotted line superimposed on the original image 10 indicated by a solid line (however, The dotted lines are partially offset for clarity.) Here, 31 corresponds to the triangular mountain 11 in the original image 10, 32 and 33 correspond to the same person's face and torso 12 and 13, and 34 corresponds to the same building 14. is there. In addition, the quadrangle 35 on the window portion of the building at the right end of FIG. 3 and the oval 36 below the false are detected by the program.
[0034]
  Here, the original image 10 as an object of the image search method of the present invention is not particularly limited as long as it is supplied as image data. For example, the original image 10 is input or acquired by an image input device or an image acquisition device. Or an image supplied from an image supply source, that is, by an image reader such as a scanner such as an image photographed on a photographic film such as a negative film or a positive film by a conventional camera, a photographic print or a printed image. Scanned image, digitalStillImages taken by cameras and video cameras, images taken by medical diagnostic imaging devices such as X-ray imaging devices, CR devices, CT devices, MRI devices, etc., computer graphics images (hereinafter referred to as CG images) ) And the like created by a computer or the like. In addition, as will be described later, continuous images (continuous shooting) taken with a conventional camera or a digital still camera (hereinafter referred to as continuous images),Moving images), moving images taken with a video camera (hereinafter referred to as video images), television broadcast moving images (hereinafter referred to as television images), such video images, television images, CG images, etc. It may be an image having various audio information, or an image with message information such as a character, mark, face image, character image, or the like.
[0035]
As the search specification pattern 20 shown in FIG. 2, a combination of different specific figures or a combination in which the size of the same specific figure is changed can be used. In addition to the user drawing on the image display device using a mouse or keyboard, or directly specifying the search specification pattern 20 in a specific shape using an image specification device such as a tablet, characters, symbols, It is possible to use an indirect designation method such as designation with a keyword such as a mark, or input by voice, recognizing the input voice and automatically converting it into a specific shape.
For example, in FIG. 2, instead of inputting a specific figure, the user encloses the approximate position on the tablet or the image display device, and inputs characters, symbols, or marks in the enclosed area. For example, if the characters “mountain” are entered, “mountain” is automatically converted into a triangle search and processed. Similarly, if characters “Bill” and “People” are input, the search is converted into a quadrangle search and a face / person search, respectively. In addition, when specifying with symbols and marks, sites and centers (database organizations) that have a large database of images with attached information, such as laboratories, and users who use the database can set symbols and marks. It is preferable to keep it. In this way, it is possible to perform a process of automatically converting the symbol designation or mark designation by the user into a specific figure or a specific main subject, person or object described later.
[0036]
Here, the specific figure used as the search designation pattern 20 may be any figure corresponding to specific information existing in the image, and as described above, such as a circle, an ellipse, a triangle, a quadrangle, and the like. A simple figure is preferable because it is easy to search. However, in the present invention, the invention is not limited to this. For example, a polygon such as a pentagon or more, a character such as a star or a star (☆), an irregular shape, Any figure may be used as long as it exists in the image, such as various figures such as straight lines, curved lines, and combinations thereof. The search specification pattern 20 is not limited to specifying a specific graphic itself, and may be at least one of the center position, size, orientation, vertex, color, and the like of the graphic.
In addition, the specific information used in the present invention is not limited to the above-described specific figure. For example, as described later, a specific subject existing in an image, a face image of a specific person existing in the image, and an image Messages consisting of personal information consisting of a specific person name attached to the text, audio information attached to the image, text data such as handwritten characters such as characters and signatures, and image data such as marks, photographed images and reference images such as CG images Information, edit data used when creating a CG image such as a CG command for drawing a specific figure, a specific color, or a specific object may be used.
[0037]
FIG. 4 is an operation flow diagram illustrating an outline of an example of an operation for extracting a predetermined specific figure from the original image 10 illustrated in FIG. 1 as a preparation operation in the image search method according to the present embodiment. An example will be described.
[0038]
As shown in FIG. 4, first, when the original image 10 is read as image data from an image data supply source (not shown) in step 401, a specific geometric figure scheduled in step 402 is next. A region having a shape similar to (specific figure) is extracted using the above-described pattern matching or edge tracking method. Subsequently, in step 403, the extracted areas are digitized as related information, and characteristic values of those areas as described later are calculated. In step 404, the obtained characteristic values are not shown as attached information. Is stored in the storage means. Here, in steps 402 to 404, various characteristic values such as the center position, size (size), and orientation of the specific figure in the original image 10 are recognized and sequentially stored in the database as attached information.
[0039]
As the various characteristic values used as the attached information, for example, in the case of a circle, if the diameter as the other size of the center position is an ellipse, the major axis as the other size of the center position is 3 if the minor axis is a triangle. If the coordinates of one vertex are a quadrangle, the X-axis width, the Y-axis width, etc., other than the center position, are adopted. Of course, in the image search method of the present invention, the way of taking the various characteristic values described above is not limited in any way. When the original image 10 is a color image, related information such as the color of a specific figure may be used as attached information.
Further, the attached information used in the present invention is not limited to various characteristic values such as the center position, size (size), and orientation of a specific figure, and related information such as color, and as described later, a specific subject, It may be related information such as the position, size, orientation, and color of a specific person, a specific figure in a moving image, a specific subject, information on a specific person, and a specific figure between multiple frames of the moving image, CG image creation editing data itself such as a specific subject, a moving direction of a specific person, audio information itself, message information itself, and a CG command may be used.
[0040]
Table 1 shows an example of the result of extracting the above-mentioned specific figure from the original image 10 shown in FIG. Table 1 shows that the program of the image search method according to the present embodiment uses the pattern matching or the like to create three oval regions, one trigonal region, and two tetragonal regions from the original image 10. It shows that it extracted. As described above, one of the elliptical areas and one of the quadrangular areas are erroneous detection results.
[0041]
[Table 1]

[0042]
Next, the flow of an operation for searching for an image based on the above stored data according to the image search method of the present invention will be described with reference to FIG.
First, in step 501, the search designated pattern 20 is read as a search condition. In the next step 502, the read designated pattern 20 for search is digitized in the same manner as described above, converted into a characteristic value, and tabulated. In this procedure, characteristic values such as the position, size, orientation, color, etc. of the specific figure may be directly input. Next, in step 503, an original image is read from the original image database (DB) as a search target. Subsequently, it is compared with the designated pattern for search 20 tabulated in step 504.
[0043]
This collation is performed sequentially for each column and each row of the designated pattern table for search. For example, when a plurality of characteristic values of the search designated pattern are arranged in one row of the table, first, in step 504, each of the first designated pattern arranged in each column of the first row of the table. The characteristic values attached to the original image read from the database as the attached information are compared and collated, and a match / mismatch is determined in step 505. If there is a match, the search result is extracted in step 506. .
[0044]
On the other hand, if the determination in step 505 is not coincident, step 506 is bypassed, the process proceeds to step 507, and the process proceeds to the next line. Next, in step 508, it is determined whether or not there is a next line in the designated pattern table for search. If there is a next line, the process returns to step 504 to perform comparison and collation of the next line. Repeat 508. If it is determined in step 508 that the next line does not exist, collation of one original image to be searched is terminated, and the process proceeds to step 509 to move to the next original image to be searched. Next, in step 510, it is determined whether or not there is a next original image. If the next original image exists, the process returns to step 503 to search for the next original image. If it is determined in step 508 that the next original image does not exist, the search of all original images to be searched is terminated.
In this way, the search method of the present invention is implemented.
[0045]
Here, when a plurality of characteristic values of the search specification pattern are arranged in one column of the table, the above collation is performed in units of columns, and a lump of rows and columns having one search specification pattern is obtained. When arranged as a unit, the lump is used as a unit.
Here, it is simply described that “if there is a match at step 505, it is taken out at step 506”. However, in reality, it is rare to match completely. It is preferable that the degree of coincidence is calculated in step 504, and that the degree of coincidence calculated in step 505 exceeds a predetermined threshold value.
[0046]
The search designation pattern 20 input / designated by the user is stored in a storage unit (not shown) as tabulated data in the form shown in Table 2, for example. Although only the ellipse is illustrated here, similar data is created for other figures. Then, based on this data, as described above, an original image that matches the designated pattern for search is searched.
[0047]
[Table 2]

[0048]
The case of an ellipse will be shown below as an example of the above-described coincidence calculation formula.
[Expression 1]

Here, it is assumed that the F {X} value becomes larger as the input value X is smaller.
[0049]
Here, I is 1 to the number Imax existing in the attached information, J is 1 to the number of patterns Jmax at the time of search, and k1 to k3 are coefficients.
In addition, the score for the search pattern number J:
P (J) = max (D (I, J))
Number of points that can be placed on the full screen:
Ptotal (A: ellipse) = Σ (P (J))
J = 1 ~ Jmax
It is.
[0050]
In the above description, the case of an ellipse has been described as an example, but the same degree of coincidence calculation is performed in step 504 for other figures.
For example, assuming that the degree of coincidence for an ellipse is Ptotal (A), the degree of coincidence for a triangle is Ptotal (B), and the degree of coincidence for a quadrangle is Ptotal (C), in the case of FIG. (ALL) = Ptotal (A) + Ptotal (B) + Ptotal (C).
[0051]
The specific figure referred to in the present invention is not limited to the above geometric figure, and as described above, other polygons, characters such as stars (☆), straight lines, curves and combinations thereof, This includes using various figures.
In addition, based on the calculation result, the searched images may be displayed as thumbnails in the order of matching points.
[0052]
It is possible to set an AND condition mode and an OR condition mode for the specified pattern at the time of retrieval. For example:
(1) In the AND condition mode: The function F {X} has a characteristic that becomes a negative value when the input X is equal to or less than a predetermined value.
For example, if a specified figure exists at the position of the search specification pattern, the value is positive, and if not, the value is negative. If it falls off, the overall score will decrease.
[0053]
(2) In OR condition mode: The function F {X} has a characteristic that is 0 or more or a positive value.
For example, if there is a figure specified at the position of the search designation pattern, the value is positive, and if it does not exist, it is 0. Even if it deviates, the overall matching score does not decrease.
[0054]
In the above-described example, a plurality of characteristic values such as the position, size, direction, and the like of the center position of all the search specifying patterns 20 are used as search conditions and compared with the attached information of the read original image. However, the present invention is not limited to this, and only the presence or absence of the designated pattern for search itself may be used as the search condition, and at least one element or item of the position, size, orientation, and color among the characteristic values is included. The search condition may be used, the color (color tone) of the specified pattern in addition to the characteristic value may be used as the search condition, or at least one of a plurality of search specified patterns may be used.
[0055]
However, if the specified search pattern itself is specified as a search condition, or if there are few specified elements, items, or specified patterns, the time required for the search will be shortened, and the number of original images that will be retrieved as a hit There is a high possibility that the target original image is included therein, but the accuracy of the search is reduced. On the other hand, the more elements, items, and specified patterns that are specified, the more accurate the search will be, and there is a possibility that only the target original image will be hit and extracted. If the specified pattern is not accurate, there may be a result that no original image is hit. For this reason, in the search of the present invention, the number of elements, items, and specified patterns to be specified may be appropriately selected in consideration of the certainty and accuracy of the search.
[0056]
At this time, as a search instruction user interface, whether only the presence or absence of specific information such as a specific figure is used, or further, the characteristic value and color such as the position, size, orientation, etc. are used, respectively. In order to associate various items and elements such as these characteristic values and colors, the user can specify a mode setting for using these items and elements individually or in combination of two or more. Is preferred.
For example, in the mathematical formula shown in the above equation 1, the coefficients k1 and k2 may be set to 0 and excluded from the coincidence calculation.
[0057]
According to the image retrieval method shown in the present embodiment, it is adapted to the instruction by a simple instruction method such as drawing a rough figure on the tablet, inputting the name or symbol of the figure from the keyboard, or inputting these by voice. The effect that it becomes possible to search an image is acquired. In the present invention, the designation by a figure, name (language), symbol or voice designated by a simple designation method is automatically converted into a specific figure, digitized to a required characteristic value, and the color is changed.
[0058]
Next, another embodiment of the present invention will be described.
In the above-described embodiment, the specific figure is used as the specific information, and the position, size, orientation, color, and the like of the specific figure to be searched are attached information. In this embodiment, the specific main subject is used as the specific information. (Hereinafter, a specific subject, a person is taken as an example here), and the feature is that the position, size, orientation, color, etc. are searched for as auxiliary information.
[0059]
FIG. 6 shows a specific subject designation example 60 at the time of retrieval. Here, after setting the subject as a person, for example, a simple figure 61 such as an ellipse is used to designate the presence / absence of the figure, or at least one of its position, size, orientation, and color.
FIG. 7 shows an example of a search result corresponding to the above-described designation. This example shows an example in which a target image 70 is searched as a result of searching based on only the position and size of a person.
[0060]
According to the image search method shown in the present embodiment, when the search target is a person, the person itself or at least one of its position, size, and orientation is roughly indicated on the tablet, or the person itself is displayed from the keyboard. An effect is obtained that it is possible to search for an image that matches the instruction by a simple instruction such as inputting the presence / absence of an image or at least one of the position, size, and orientation of a person.
[0061]
In addition to the position and size of the person, by adding information that expresses the person's clothes in color, when searching for the person's torso, the color will be checked. There is a possibility that the efficiency of Moreover, you may combine the characteristic of a several person.
[0062]
Depending on the application field, the position of the specific figure or the specific subject may be substantially fixed. In such a case, the size, the size and direction, and the color may be recorded as attached information.
In addition, when the size of the specific figure or the specific subject is substantially fixed, the position or position and orientation, and the color may be recorded as the attached information.
[0063]
Still another embodiment of the present invention will be described.
In each of the above-described embodiments, a still image is a search target. In this embodiment, a moving image is a search target, and the initial position and moving direction of a specific figure or a specific subject (person) are attached information. It is characterized in that a search is performed by designating.
[0064]
FIG. 8 shows a specific subject designation example 80 at the time of retrieval. Here, after setting the specific subject as a person, the moving direction (vector) 82 of the specific subject 81 is designated by an arrow.
FIG. 9 shows an example of search results corresponding to the above-described designation. In this example, the target moving images 90a and 90b are searched as a result of the search based on only the initial position and size of the person.
[0065]
In this embodiment, when registering attached information, tracking of how a specified figure (specific figure) or a specified subject (specific subject) extracted within a frame of a moving image has moved after a predetermined frame is performed. The vector information is included in the attached information and stored. When performing a search, matching with attached information is performed based on a specified figure and its motion vector (at least one of size and direction).
[0066]
As another method, a method of creating attached information in units of frames is possible. In other words, when searching, if a specified figure (specific figure) or subject is detected in units of frames, a position corresponding to a frame after a predetermined time (or its intermediate frame) from that frame (this position is specified) It is a method of checking whether or not the designated figure exists in the obtained motion vector by matching with attached information.
[0067]
In the method according to this embodiment, the program decodes the input scene designation information into a simple shape and its movement, and calculates the degree of coincidence with the attached information of each image in the database. The calculation results are output (thumbnail display) in frame order of the corresponding moving image in the order of points.
[0068]
According to the image search method shown in the above embodiment, when the search target is a person in the moving image, the initial position and moving direction are roughly indicated on the tablet, input from the keyboard, or voice input. The effect that it becomes possible to search for a moving image that matches the instruction can be obtained by a simple instruction such as, for example, and an image search with a rough scene description in video editing is facilitated.
[0069]
Still another embodiment of the present invention will be described.
In each of the embodiments described above, an image that does not include audio information is a search target. However, in this embodiment, an image that includes audio information such as a moving image is the search target, and an outline of the audio information is included in the attached information. Thus, when performing a search, an outline of audio information is designated and the search is performed.
[0070]
For example, in a moving image with a person as a subject, audio information is captured in addition to motion information, and the content is stored in a database by audio conversion. Thus, by specifying, for example, subject: person, voice information: school as keywords at the time of search, a scene in which a person exists as a scene and is related to the word school is searched. The search target is not limited to a moving image, but may be a still image added with audio information.
[0071]
According to the image search method shown in the present embodiment, when the search target has audio information, the information can be used to search for an image that matches the instruction. This makes it possible to search based on the content of the utterance as well as the movement of the video, which is effective when editing video. Of course, in addition to this (voice information), various types of information on a specific figure or a specific main subject (person) may be additionally specified.
[0072]
Still another embodiment of the present invention will be described.
This embodiment is characterized in that a specific person in an image is identified by using a person identification process. In other words, a face image and a person name of a specific person are registered in advance as attached information, and when performing a search, a necessary person is designated by the face image or person name from among the registered persons described above. Then, a search is performed.
[0073]
Specifically, the customer registers information for N people in advance by combining names and face images. Here, when identifying a specific person in the image using the person identification process, for example, the IEICE Transactions Vol. J80-D-II No.8, pp.2218-2224 “Searching for similar face images using coarse and dense mosaic features as key” by Kosugi et al., Pp.2225-2231 “Faces with extremely different acquisition processes” Techniques described in “image collation” and the like can be applied.
[0074]
With the above processing, face extraction and person identification are performed on the image brought into the laboratory, and attached information is created. That is, additional information indicating who is shown in each frame is added to the image data and stored. Of course, information on the position and size of the person may be included here. When a customer requests a search by specifying a person name, an image frame showing the person is taken out from the attached information.
[0075]
According to the image search method shown in the present embodiment, the face image of the person to be searched and the person name are associated with each other and converted into attached information. For example, it is possible to easily search for a memorable photo by the name of a family member. The target image can be a moving image as well as a still image.
[0076]
In the case of an image having audio information, it is also possible to use a combination of these pieces of information at the time of retrieval.
Furthermore, if the above-mentioned laboratory and customer's personal computer (personal computer) are connected by a communication line, the above-described processing can be performed more quickly online.
[0077]
Still another embodiment of the present invention will be described.
In each of the embodiments described above, a photographed image or the like is a search target. However, in this embodiment, a computer graphics (CG) image created by a personal computer (PC) or the like is a search target, and a simple figure or rough outline is used. A feature is that a scene is designated by a color or a specific object, and search is performed using CG image creation / edit data.
[0078]
In the present embodiment, the editing data when the CG image is created is used as specific information and is attached to the CG image as accessory information and stored in a database. That is, the database organization stores the CG image and editing data as a set. Here, the edit data is a simple graphic in the CG image, a rough color, or a CG command representing a graphic, color, or object when a specific object such as a person, animal, car, or building is created. Is preferred.
In the search in the present embodiment, the user (customer) designates a scene by a simple object, a rough color, a specific object such as a person, an animal, a car, or a building. This designation is preferably designated by handwriting, drawing software, CG software, or the like.
[0079]
Next, in the database organization, the scene designation data from the user is converted into a CG command in the database processing apparatus. Here, it is preferable that the shape of the figure designated by the user, the characteristic values such as position, size, and orientation, the color, and the like are classified into predetermined patterns.
Subsequently, the degree of coincidence and similarity are calculated between the converted CG command and the CG editing data attached to the CG image in the database as attached information, and matching with attached information is performed. Note that the calculation results may be output (thumbnail display) by frame display of the corresponding CG image in the order of points.
[0080]
According to the image search method shown in the present embodiment, the amount of calculation in the CG image search process can be reduced by using the creation / edit data of the CG image to be searched as attached information, and the CG conforming to the user's instruction. An effect is obtained that an image can be searched quickly and accurately. Note that the target CG image can be a moving image as well as a still image. When a CG moving image is set as a search target, a specific figure or object and its movement vector (at least one of its size and direction) are preferably designated as described above. In the case of an image having audio information, it is also possible to use a combination of these pieces of information at the time of retrieval.
[0081]
Still another embodiment of the present invention will be described.
In the present embodiment, an image having message information is set as a search target, and message information accompanying the image is checked for image search, and the search is performed.
In this embodiment, as in the case of searching for an image having audio information as described above, message information associated with the image is set as specific information and attached to the image as auxiliary information and stored in a database.
[0082]
Here, as the attached information attached to the image, it is preferable to use text data synthesized with the image as message information, or image data such as a mark, a photographed image, or a CG image. Text data or image data attached to the image instead of the inside, for example, text data or image data recorded (synthesized) in the white edge portion of a print image with a white edge on the back side of the print image may be used. good.
For example, as text data, print character data such as a word processor, handwritten character data such as a signature, or the like can be used, or character data obtained by extracting characters from an image can be used.
[0083]
In the search in the present embodiment, the user (customer) specifies message information by the character itself, its content, the shape of a handwriting (in the case of a signature, etc.), or the reference image registered as message information. By selecting, it designates and performs a matching process.
For example, in the mode of searching for a character itself, the presence / absence of the character itself, the position, size, orientation, color, etc. of the character itself, as well as the specification of the characteristic value and color of the specific graphic in the search using the specific graphic described above. A matching process may be performed in which at least one is designated, the matching degree is calculated, and a matching image is selected. Even in the character search mode, when specifying the content of a character, the content of the specified character such as kanji may be interpreted and the matching process may be performed. In the case of a handwritten character such as a signature, a matching process may be performed by specifying the shape of the handwriting.
[0084]
Furthermore, when searching using the reference image, the user designates an area of the reference image from a plurality of reference images such as marks, captured images, and CG images registered as message information. Image matching may be performed by selecting and specifying one or more reference images, for example, by selecting a design pattern prepared in advance.
Note that the calculation result of the matching degree in the matching process may be output (thumbnail display) by displaying the corresponding images in frame order in the order of points.
[0085]
According to the image search method shown in the present embodiment, text data such as characters or image data for reference is used as message information associated with an image to be searched, and the characteristic values and colors of the characters themselves, or their contents Search for images that meet the specification by a simple specification method such as area specification or design pattern selection by a simple specification method such as specifying the shape of a handwriting or handwriting. The effect that it becomes possible to obtain is obtained.
[0086]
In each of the embodiments described above, when displaying the search result image, for example, an area that matches the search condition specification may be displayed with priority, excluding a non-moving part such as a background. Alternatively, it is also effective to display an area that matches the search condition specification by using a special image processing result emphasis display different from other areas, for example, brightness emphasis, blink (flashing), and the like.
[0087]
If the display method as described above is adopted, the transfer data capacity when transferring the searched image to a monitor or the like can be greatly reduced, so that the display response speed is improved and the usability is improved. In addition, it is possible to improve the usability of the target image in the display screen and to improve usability.
[0088]
In addition, it is also preferable to check the display capability of the display device when processing the display, and to process and display the image data according to the capability. It is also effective to manage the searched areas in the database on a database basis, and set the reference priority for searching and set the resolution for storing data according to the access frequency. It is.
[0089]
In this method, for example, when a customer accesses a center (database organization) having a database from a portable terminal, information on the display capability of the customer's portable terminal is transferred together with the ID, At the time of data transfer, image data processing such as image compression, binarization, line drawing, trimming (cutting out only the main subject) is performed as necessary.
[0090]
Efficient processing at the time of image display as described above makes it possible to reduce the amount of data communication, shorten the time until image display on a portable terminal, and improve the visibility of the displayed image. The effect of doing etc. is acquired.
[0091]
When managing image data having various types of attached information as described above, it is preferable to divide an image into regions for each subject and register each divided area with a label. If only the labeled part is cut out and filed, it not only has a great effect on reducing the data capacity, but also has a great effect on improving the speed of the search. In addition, for example, it is possible to use a method in which only the image of the part with the above-described label is stored at a high resolution and the other regions are stored after being compressed.
[0092]
Attached information of each area includes characteristics such as image file name, position (center, contour pixel), color, and the total number of accesses / hits. Using these, for example, if the storage hierarchy in the storage means is controlled in accordance with the access frequency, there is a greater effect in improving the speed of the search. Also, frequently used data can be stored separately to make it easier to use.
[0093]
Hereinafter, an image processing method to which the above-described image search method according to the present invention is applied will be described. The first method is a method for specifying specific information such as a specific figure or a subject in a printed image at the time of initial print processing (so-called simultaneous printing) or previous reprint processing (representatively referred to as previous print) in a laboratory. And the extracted specific information and related information (characteristic values and colors) are stored as auxiliary information in a storage means such as a database together with image data for printing and image processing parameters. is there.
[0094]
When there is a request for reprinting of the film containing this image or subsequent reprinting (representatively referred to as later printing), specific information such as a specific figure or subject from the image frame designated by the customer , And the subsequent print processing according to the image processing conditions (processing parameters) at the time of the previous print processing of the image having a high degree of coincidence with the previously stored auxiliary information (those exceeding the predetermined threshold) To do.
[0095]
According to this image processing method, it becomes possible to retrieve image processing conditions at the time of the previous printing from the image in the film brought by the customer at the time of the subsequent printing, and reproducibility that gives satisfaction to the customer. The effect that it becomes possible to perform excellent printing is obtained. Note that the above processing is not limited to photographic film, but also on smart media in which image data for printing is recorded, such as a photographed image recording medium such as DSC (digital still camera), and image recording media such as MO and photo CD. The same applies.
[0096]
In addition, when a customer places only a photographic print at the time of simultaneous printing, not a photographic image data on a photographic film or a photographic image recording medium, when the customer places an order for reprinting, the customer scans the print with a scanner, Execute processing to extract specific information such as specific figures, and use the extracted specific information and related information as a search condition to search the scanned image data from photographic film recorded at the time of simultaneous printing in the database and similar The degree is calculated (matching process), the corresponding frame is taken out, and in the printing process, the read image data and processing parameters corresponding to the corresponding frame at the time of simultaneous printing are read out, and the print process is performed using them. Also good.
[0097]
The second method is the use of one or more photographic films in one film strip (so-called one photographic film) or in one photographic image recording medium or ordered by a user (customer) or This is a method for preventing variation (variation) in printing within one or a plurality of photographed image recording media (so-called one order). That is, in a photographic film or photographing recording medium of one or one order (representing these as one order), a specific figure or a composition with similar specific information such as a subject (the arrangement and size of each figure, (Orientation or color), it is preferable to adjust the image processing so that the finished color tone, density, etc. are aligned between these frames belonging to one order.
[0098]
According to the present embodiment, similar images are discriminated based on the composition of the image, so that they are not easily influenced by the color and density. Thus, there is no effect of producing different prints and the customer is not dissatisfied.
[0099]
The image search method and the image processing method according to the present invention as described above can be implemented by an image search apparatus as shown in FIG. 10 and a print system using the same.
A print system 100 shown in FIG. 1 is connected to an image search apparatus (hereinafter simply referred to as a search apparatus) 102 that performs the above-described image search method of the present invention, and the search apparatus 102 together with the image of the present invention. A photo printer 104 that executes the processing method and a user terminal 106 that is online connected to the search device 102 via a communication network such as the Internet, and that can execute the image search method and the image processing method according to the present invention and request print output. And have.
[0100]
First, the image search apparatus 102 stores an image database (hereinafter simply referred to as a database or DB) that stores and stores the image data of the image as well as specific information existing and associated with the image or related information of the specific information. 108), an auxiliary information acquisition unit (hereinafter simply referred to as an acquisition unit) 110 that acquires specific information itself or related information of specific information as auxiliary data for storage in the database 108, and at least one of the auxiliary information as a search condition Attached information specifying part (hereinafter simply referred to as a specifying part) 112 for specifying a copy, and the database 108 is searched based on the search condition specified by the specifying part 112 and attached to the selected image data. Find the degree of agreement between the attached conditions and at least some of the attached information used as search conditions Image retrieval unit for reading an image with a predetermined value or more matching degree (hereinafter, simply search unit hereinafter) having a 114, and a display unit 116 for displaying the search results Ya read image by the search unit 114.
[0101]
The photo printer 104 reads image data of a film image from a film F such as a negative or a reversal, reads image data from an image data recording medium R of a photographed image by a digital camera, or an image of a CG image creation device 117 or the like. An image input unit 118 that receives image data from a data supply source, and sets image processing conditions for analyzing the image data received from the image input unit 118 and processing the image data for producing a high-quality print. In accordance with the processing conditions or the image processing conditions received from the search device 102, the image processing unit 120 that performs image processing on the image data received from the image input unit 118 or the search device 102, and the image data that has been subjected to image processing by the image processing unit 120 are used. Record images on recording materials such as photographic materials And a print output section 122 which latent baking outputs developing and fixing) that the print P. In the illustrated example, the image processing unit 120 also has a function of transferring image data to the acquisition unit 110, receiving attached information obtained by the acquisition unit 110, transferring it to the DB 108 attached to the image data, and storing and storing it. Have.
[0102]
The user terminal 106 transfers a display unit 124 capable of displaying an image, an input unit 126 for inputting search conditions, images, symbols, languages, print order information, and the like, and the images, search conditions, and order information to the search device 102. Interface (I / F) 128. Note that the search unit 114 of the search device 102 has an I / F 115 that receives images and various information from the I / F 128 of the user terminal 106.
[0103]
In the search device 102, an acquisition unit 110 includes a tablet 130 that inputs freehand outlines of specific information such as specific geometric figures, specific subjects, message information, and personal information including a specific person's face image; Specific geometric figure, specific subject, message information, personal information consisting of face image and name of specific person, outline of specific information such as CG image editing data, meaning of specific information consisting of audio information, etc. Input using a GUI (graphical user interface) using a keyboard or mouse, and the input unit 132 for recognizing and inputting the meaning of the message information, etc. as necessary, voice information accompanying the image or specific information present in the image A voice recognition unit 134 for inputting voice information, recognizing its meaning, and converting the voice information into specific information that has been recognized; An extraction unit 136 that extracts specific information from image data of an image received from the image processing unit 120 of the photo printer 104 using an outline of the specific information input from the bullet 130, the input unit 132, or the voice recognition unit 134; Specific information itself and related information, for example, specific information such as recognized message information and voice information, a specific geometric figure, a specific subject, a face image and a person name of a specific person Related information such as the position, size, orientation, color, presence / absence of specific information such as person information, and the moving direction (direction of movement vector) of specific information in the case of a moving image, editing data of a CG image, etc. Specific information itself, and specific information itself such as message information and voice information recognized by the voice recognition unit 134 and the input unit should be stored in the DB 108 And to set, association (to transferred to the image processing unit 120 Te) Ru auxiliary information setting unit attached information set in the image data (hereinafter, simply setting unit hereinafter) and an 138.
[0104]
Therefore, the image processing unit 120 of the photo printer 104 attaches the attached information transmitted from the setting unit 138 of the acquisition unit 110 to the image data of the image, transfers it to the DB 108, and stores and stores it in that state. Here, the image data transferred from the image processing unit 120 to the extraction unit 136 and / or the DB 108 may be before image processing or after image processing, and both image data may be the same or different. May be. Note that image data before image processing may be transferred from the image input unit 120.
In the case of a CG image, edit data to be attached to the image data as attached information may be directly transferred from the CG image creating device 117 to the setting unit 138, or the image data and the edit data attached thereto may be directly transferred to the CG. It may be transferred from the image creation device 117 to the DB 108.
In addition, the acquisition unit 110 obtains attached information from another image data supply source such as another photo printer, an image reading device (scanner), an image processing device, or a driver of an image data recording medium, not from the photo printer 104. Image data to be acquired or image data to be stored in the DB 108 may be received. Further, as long as the image data of the same image, one of these two types of image data may be received from the photo printer 104 and the other may be received from another image data supply source.
[0105]
The specifying unit 112 of the search device 102 uses the tablet 130, the input unit 132, the voice recognition unit 134, or the like used by the acquisition unit 110, the CG creation device 117, or the like. It specifies at least one of related information such as an outline of specific information itself such as information and message information, presence / absence of specific information, position, size, direction, direction of movement vector, and color. Note that the tablet 130, the input unit 132, the voice recognition unit 134, the CG creation device 117, and the like used in the designation unit 112 may be the same as or different from those used in the acquisition unit 110 or the like. In addition, the acquisition unit 110 and the specification unit 112 may not be provided separately, but may be shared to acquire the auxiliary information of the image data and specify at least one of the auxiliary information as a search condition. .
[0106]
As a result of the search by the search unit 114, the image data read from the DB 108 is displayed as an image on the display unit 116. The display image is particularly high-definition as long as the selected image can be confirmed. Of course, the image data read from the DB 108 may be displayed as it is. However, if the display unit 116 can display a high-definition image, the image processing unit 120 of the photo printer 104 can display the image data. A high-definition image after processing may be displayed. That is, the display unit 116 may be shared by the search device 102 and the photo printer 104. Further, in order to confirm the image data selected by the search unit 114, the image data may be directly transferred to the print output unit 122 of the photo printer 104 to be printed out. It may be transferred to a printer and output.
[0107]
  By the way, in the image processing method of the present invention, even if there is no film F or image data recording medium R, if there is a print (hard copy image) P, reorder printing is possible. The image data read from the print P only needs to have specific information such as a specific figure or a specific subject having a pixel density (resolution) necessary for the search.Image input unit 118The image data may be read by a reflective document scanner included in the image reading apparatus, or may be directly transferred to the extraction unit 136 of the acquisition unit 110 by using a completely different scanner or a simple scanner. . At this time, the specific information extracted by the extraction unit 136 and / or its related information may be directly transferred to the search unit 114 directly or converted into attached information by the setting unit 138. In this case,Read from DB108Since the specific information extraction process is also performed on the image data, the image data read from the DB 108 may be directly transferred to the search unit 114 to the extraction unit 136 of the acquisition unit 110.
  Further, when the search unit 114 calculates the degree of coincidence of specific information such as specific figures and specific subjects in a plurality of images in one order, the extraction result of the specific information by the extraction unit 136 is directly transferred to the search unit 114. Is good.
[0108]
Each of the above-described embodiments shows an example of the present invention, and the present invention should not be limited to these. Modifications and design changes are made without departing from the scope of the present invention. Needless to say.
[0109]
【The invention's effect】
  As described above in detail, according to the present invention, the shape (a specific geometric figure or the like) of a subject (person or object) existing in the original image, the specific subject itself, or the like or the original image is synthesized. Image search for efficient image search using accompanying message information and voice information as keysapparatusCan be realized.
[0110]
More specifically, according to the image search method of the present invention, first, a rough figure is drawn on a tablet or an image display device, a figure name or symbol is input from a keyboard, or these are input by voice. With this simple instruction method, it is possible to retrieve an image that matches the instruction. Even when the search target is a person, similarly, the person itself or at least one of its position, size, and orientation is roughly indicated on a tablet or an image display device, the presence / absence of the person itself from the keyboard, Alternatively, it is possible to retrieve an image that matches the instruction by a simple instruction such as inputting at least one of the position, size, and orientation of the person.
[0111]
Second, when the search target is a person in a moving image, the initial position and moving direction is roughly indicated by a tablet or the like, input from a keyboard, or voice input, An effect is obtained that it is possible to search for a moving image that matches the instruction, and for example, an image search with a rough scene description in video editing becomes easy.
[0112]
Thirdly, when the search target has audio information, the use of the information makes it possible to search for an image that matches the instruction. The target image can be not only a still image but also a moving image, and it is possible to search not only from the movement of the subject but also from the content of the remarks, which is effective when editing video. .
[0113]
Fourth, there is an effect that it is possible to search for an image suitable for an instruction by using the information of the face image of the person to be searched and the person name associated with each other and using the information. can get. For example, it becomes possible to easily search for a memorable photo by the name of the family. Note that the target image can be not only a still image but also a moving image, and can also be an image having audio information.
[0114]
Fifth, when the search target is an image having audio information, these information can be used in combination at the time of the search. Also in this case, the target image can be a moving image as well as a still image.
[0115]
In addition, sixthly, by using creation / editing data of a CG image to be searched as attached information, the amount of calculation in the CG image search process can be reduced, and a CG image suitable for a user instruction can be quickly and accurately obtained. The effect that it becomes possible to search well is acquired. Note that the target CG image can be a moving image as well as a still image. When a CG moving image is set as a search target, a specific figure or object and its movement vector (at least one of its size and direction) are preferably designated as described above. In the case of an image having audio information, it is also possible to use a combination of these pieces of information at the time of retrieval.
[0116]
Seventh, text data such as characters or reference image data is used as message information associated with the image to be searched, and the characteristic values and colors of the characters themselves, the contents, the shape of the handwriting, etc. are specified. It is possible to search for images that match the designation by a simple designation method such as area designation or design by selecting a design pattern, etc. An effect is obtained.
[0117]
When displaying the search result image, for example, the area that matches the search condition specification is displayed with priority, except for the non-moving part such as the background, or the search condition specification is met. It is also effective to display the area to be displayed differently from other areas, for example, by highlighting special image processing results such as brightness enhancement and blinking (flashing).
[0118]
In addition, it is also preferable to check the display capability of the display device when processing the display, and to process and display the image data according to the capability. It is also effective to manage the searched areas in the database on a database basis, and set the reference priority for searching and set the resolution for storing data according to the access frequency. It is.
[0119]
By applying the above image search method, it becomes possible to search image processing conditions at the time of the previous printing from the images in the photographic film or shooting recording medium brought by the customer at the time of the subsequent printing, which gives satisfaction to the customer. As a result, it is possible to perform printing with excellent reproducibility.
[0120]
Also, by applying the above image search method, similar images can be discriminated based on the composition of the image, so that it is less affected by color and density. This eliminates the need to produce prints with extremely different color tones, densities, etc., and prevents the customer from being dissatisfied.
[Brief description of the drawings]
FIG. 1 is a diagram for explaining an image search method according to an embodiment of the present invention, and shows an original image.
2 is a diagram showing a search designation pattern input by a user when searching for the original image shown in FIG. 1; FIG.
3 is a diagram showing a specific figure extracted from the original image shown in FIG. 1 superimposed on the original image. FIG.
4 is an operation flow diagram showing an outline of an operation for extracting a predetermined specific figure from the original image shown in FIG. 1 as a preparatory operation. FIG.
FIG. 5 is an operation flowchart showing an operation flow for searching for an image based on data stored in a storage unit.
FIG. 6 is a diagram for explaining an image search method according to another embodiment of the present invention and is a diagram showing an example of subject designation.
7 is a diagram showing an example of a subject search result corresponding to the designation shown in FIG. 6; FIG.
FIG. 8 is a diagram for explaining an image search method according to still another embodiment of the present invention, and is a diagram showing an example of subject designation at the time of moving image search.
FIG. 9 is a diagram showing an example of a moving image search result corresponding to the designation shown in FIG. 8;
FIG. 10 is a block diagram of an embodiment of an image search apparatus that implements an image search method and an image processing method of the present invention and a print system that uses the image search apparatus.
[Explanation of symbols]
  10 Original image
  11 Triangular mountain
  12 Human face
  13 Human Torso
  14 Building
  20 Search specified pattern
  21 Triangle symbolizing mountain
  22, 23 Two ellipses symbolizing a person
  24 Vertical quadrilateral showing building
  30 Graphic extraction results from the original image
  31 Figure corresponding to mountain 11
  32, 33 Figures corresponding to persons 12, 13
  34 Figure corresponding to building 14
  35, 36 False detection figure
  60 Subject designation example
  61 person-designated figure
  70 Figure extraction result from original image
  80 Subject designation example
  81 subjects
  82 Subject movement direction
  90a, 90b Moving image extraction result
  100 printing system
  102 Image search device
  104 Photo printer
  106 User terminal
  108 Image Database (DB)
  110 Attached information acquisition unit
  112 Attached information specification part
  114 Image search unit
  115, 128 interface (I / F)
  116, 124 display unit
  117 Computer Graphics (CG) Image Creation Device
  118 Image input unit
  120 Image processing unit
  122 Print output section
  126, 132 input section
  130 tablets
  134 Voice recognition unit
  136 Extraction unit
  138 Attached information setting part

Claims (20)

An extraction unit that extracts or recognize specific information pertaining with the presence or associated with the image, or exist for an input image,
An attached information setting unit that stores the obtained specific information as it is or the related information of the specific information is attached to the image data of the image and stored in the database as attached information;
When at least a part of the attached information is specified as a search condition, the database is searched with the specified search condition, and matches with the attached information attached to the selected stored image data A search unit that obtains a degree and reads an image having a matching degree equal to or greater than a predetermined value ;
An image search device having a display unit for displaying a read image,
The specific information present in the image is M (M is 1 or more) types of predetermined specific geometric figures for the image,
The related information stored as the attached information is obtained by applying a process of extracting the M kinds of specific geometric figures to the image, and N (N is 0 or more) obtained as a result of extraction. One of position and size, position and size and orientation, position and size and color, or position, size, orientation and color corresponding to a specific geometric figure,
In the step of designating at least a part of the attached information as the search condition, the presence or absence of the specific geometric figure or the position for L types (L is 1 to M) of the specific geometric figure, An image search apparatus characterized in that an outline of at least one of a size, a direction, and a color is designated. An extractor for extracting or recognizing specific information that is present or associated with, or present and associated with, the input image;
An attached information setting unit that stores the obtained specific information as it is or related information of the specific information in the database as attached information attached to the image data of the image;
When at least a part of the attached information is specified as a search condition, the database is searched with the specified search condition and matches the attached information attached to the selected stored image data. A search unit that obtains a degree and reads out an image having a matching degree equal to or greater than a predetermined value;
An image search device having a display unit for displaying a read image,
The specific information present in the image is a subject defined by M (M is 1 or more) types of specific geometric figures that are predetermined for the image,
The related information stored as the attached information is obtained by applying a process of extracting the M kinds of specific subjects to the image, and N (N is 0 or more) specific results obtained as a result of extraction. Position and size, position and size and orientation, position and size and color, or position, size, orientation and color
In the step of designating at least a part of the attached information as the search condition, the presence / absence of the specific subject or the position, size, direction of the L type (L is 1 to M) of the specific subject And at least one outline of color is designated. The image is an image having continuity in time,
The specific information present in the image is a specific geometric figure predetermined for the image,
The related information stored as the attached information is a process of extracting the specific geometric figure for the image at a predetermined time or an interval of the number of frames with respect to the image having continuity in time. The geometric figure information obtained by obtaining the moving direction of the image corresponding to the geometric figure based on the comparison of the positions of the extracted geometric figure between frames, and the geometry The direction of movement of the image corresponding to the geometric figure,
The step of designating at least a part of the attached information as the search condition includes the step of changing the direction of the geometric figure and its motion vector at a predetermined time or an interval of the number of frames with respect to the temporally continuous image. The image search apparatus according to claim 1, wherein the outline is designated. The image is an image having continuity in time,
The specific information present in the image is a subject defined by a specific geometric figure predetermined for the image,
The related information stored as the attached information is extracted by applying a process of extracting the specific subject at a predetermined time or an interval of the number of frames to the temporally continuous image. Information on the subject and the moving direction of the image corresponding to the subject obtained by determining the moving direction of the image corresponding to the subject based on a comparison of the positions of the subject between frames;
As at least a part of the attached information as the search condition, by specifying an outline of the direction of the subject and its motion vector at a predetermined time or an interval of the number of frames with respect to the temporally continuous image. The image search device according to claim 2 . The image is an image having audio information,
The specific information accompanying the image is the audio information,
The attached information is the audio information recognized for the image having the audio information,
The step of specifying at least a part of the attached information as the search condition is to specify an outline of audio information for an image having the audio information,
The degree of matching, the image retrieval apparatus according to claim 1 or 2 which is the degree of coincidence with the speech recognition information that comes with the data of the image obtained by said memory. As the auxiliary information, in addition to the audio information, any one of the position and size corresponding to a specific geometric figure, the position and size and the direction, the position and size and the color, or the position, the size and the direction and the color Attached with related information,
The search condition, in addition to the audio information, according to claim 5 position, at least one of magnitude and direction, as well as the color corresponding to the specific presence or absence of geometrical figures or the specific geometrical figures, Image search device . As the attached information, in addition to the audio information, the related information of any one of a position and a size, a position and a size and a direction, a position and a size and a color, and a position, a size and a direction and a color corresponding to a specific subject. Attached,
The image search apparatus according to claim 5 , wherein the search condition is at least one of presence / absence of a specific subject or a position, size, direction, and color corresponding to the specific subject in addition to the audio information. . The specific information that is present and attached to the image is personal information including a face image of a specific person and a person name,
As the related information stored as the attached information, personal information including a face image and a person name of the specific person A (A is 1 or more) is registered in advance. Presence or absence of the specific person, position and size of the specific person, position and size and direction, position and size and color, and position and size and direction obtained by the person identification process using the person information And any one of the colors
The step of designating at least a part of the attached information as the search condition includes the presence / absence of a specific person B (B is 1 to A) or the position, size and orientation of the specific person as the search condition. Is to specify at least one of
The degree of matching, the image retrieval apparatus according to claim 1 or 2 which is the degree of coincidence with the accompanying information of each stored image. The image is an image having continuity in time,
The related information stored as the auxiliary information is obtained by the person identification process using the person information at a predetermined time or an interval of the number of frames for the temporally continuous image. Yes,
The step of designating at least a part of the attached information as the search condition includes the presence / absence of the specific person or at least one of the position, size and color of the specific person and the movement as the search condition. The image search apparatus according to claim 8 , wherein the vector direction is designated. The image is an image having audio information,
The attached information is the related information and recognized voice information obtained by the person identification process using the person information for the image having the voice information,
The step of designating at least a part of the attached information as the search condition includes the presence / absence of the specific person as the related information or the position of the specific person as the search condition for the image having the audio information. The image search device according to claim 8 , wherein at least one of size, direction, and color, or an outline of the audio information is designated. The image is an image having message information,
The specific information attached to the image is the message information,
The attached information is the message information recognized for an image having the message information;
The step of specifying at least a part of the attached information as the search condition is to specify an outline of message information for an image having the message information,
The degree of matching, the image retrieval apparatus according to claim 1 or 2 which is the degree of coincidence with the message identification information that comes with the data of the image obtained by said memory. The image search device according to claim 11 , wherein the message information is text data or accompanying image data synthesized in an image. The image is a computer graphics image;
The specific information accompanying the image is editing data used when creating the image,
The attached information is the editing data,
The step of designating at least a part of the attached information as the search condition includes the presence / absence of at least one of a specific geometric figure, a specific object and a specific color, or at least one of its position, size and orientation. Designating a scene of the computer graphics image by: converting the designated scene designation data into a computer graphics command;
3. The image according to claim 1, wherein the step of obtaining the degree of coincidence is a step of obtaining a degree of coincidence between the converted computer graphics command and the edit data attached to the selected stored image data. Search device . The computer graphics image is a computer graphics image having continuity in time,
The attached information includes at least the specific geometric figure and the specific object at a predetermined time or frame interval with respect to the computer graphics image having temporal continuity in addition to the editing data. The specific geometric figure obtained by determining a moving direction of an image corresponding to at least one of the specific geometric figure and the specific object based on a comparison of positions between frames for one; At least one of the specific objects and the moving direction of the image;
As the search condition, with respect to the computer graphics image having temporal continuity, presence or absence of at least one of a specific geometric figure and a specific object at a predetermined time or an interval of the number of frames, or its presence The image search device according to claim 13 , wherein an outline of at least one of position, size, and color and a direction of a motion vector thereof is designated. The designation step includes a step of designating by at least one of a symbol, a language, and a voice, and a step of automatically converting at least one of the designated symbol, language, and voice to the specific information or its auxiliary information. Item 15. The image search device according to any one of Items 1 to 14 . Image retrieval apparatus according to any one of claims 1 to 15, the area that matches the search conditions in the image data, and displays the temporal priority. The image search device according to any one of claims 1 to 15 , wherein an area that matches the search condition in the image data is highlighted by image processing different from other areas. Image retrieval apparatus according to any one of claims 1 to 17 for processing of the image data in accordance with the display capability of the display terminal. The image search device according to any one of claims 1 to 18 , wherein the searched image areas are managed in units of areas, and a reference priority for search is set according to access frequency. The image search device according to any one of claims 1 to 18 , wherein the searched image area is managed in area units, and the resolution at the time of data storage is set according to the access frequency.

Images