JP3745425B2 - Motion vector detection method and adaptive switching prefilter for motion vector detection - Google Patents

Description

と定義すれば、画素ｐの動きｍ１は、
【００３７】
【数２】

【００３８】
と表せる。ここでノイズの影響を避けるため、ｍthを定義し、
ｍ１≧ｍth
のときは、動領域として
Ｍ１＝１
とし、
ｍ１＜ｍth
のときは、静止領域として
Ｍ１＝０
とする。
【００３９】
同様にして、ｆ(t) 上の画素ｐと、これと同じ空間位置にあるｆ(t+2) 上、ｆ(t-1) 上およびｆ(t-2) 上の各画素との間で動き検出を行い、それぞれの動き検出の結果をＭ２、Ｍ３、Ｍ４とする。
【００４０】
次に、
【００４１】
【数３】
Ｍｐ＝（Ｍ1 and Ｍ2 and Ｍ3 and Ｍ4 ） …(3)
を調べ、
Ｍｐ＝０のときは、画素ｐは静止領域
とし、
Ｍｐ＝１のときは、画素ｐは動領域
と判定する。
【００４２】
以上の処理をｆ(t) 上の全ての画素に対して行い、ｆ(t) 上の画素毎の動領域と静止領域を求める。
【００４３】
次に、ｆ(t) 上の検出領域毎の静動判定を行うために、検出領域内の画素の内、
Ｍｐ＝１
となる個数をｍとする。静動判定のためのｎthを定義し、
ｍ＞ｎth
のときは、検出領域は動領域とし、
Ｍ＝１
とし、
ｍ≦ｎth
のときは、検出領域は静止領域とし、
Ｍ＝０
と判定する。同様にして、ｆ(t) 上の全ての検出領域に対して静動判定を行う。
【００４４】
（II）ＳＮ比について
次に、ｆ(t) のＭ＝０となる全静止領域の全ての画素に対して、ｆ(t+2) 上にあるｆ(t) の静止領域と同じ空間位置を占める画素との間でフレーム間の輝度レベルの差分の絶対値を求め、全静止領域の１画素当たりの輝度レベルの差分の絶対値の平均Ｎを求める。ＳＮ比はフィールド単位で決定される。上記（Ｉ）の検出領域の静動状態を表すＭが画面内でＭ＝０となる検出領域の個数を数え、その数をＣとする。ここでパンニング画像では、静止領域がないかあるいは少ないため、しきい値Ｃthを導入し、
Ｃ≦Ｃth
のときは、別途求めた前フィールドのグローバル・ベクトルＶｇ（次に述べる（III ）を参照）を参照し、Ｖｇ≠０あるいはＶｇ＝無しのときはＳＮ比が悪いとし、
ｓｎ＝１
とし、Ｖｇ＝０のときはＳＮ比が良いとし、
ｓｎ＝０
とする。
【００４５】
Ｃ＞Ｃthのときは、ＳＮ比を判定するためのＮthを導入し、
Ｎ＜Ｎth
のときはＳＮ比が良いとし、
ｓｎ＝０
とする。
【００４６】
Ｎ≧Ｎth
のときはＳＮ比が悪いとし、
ｓｎ＝１
とする。
【００４７】
（III ）グローバル・ベクトルについて
図５に示すように、前フィールドのグローバル・ベクトルを求める。グローバル・ベクトルはフィールド単位に決定される。動きベクトル検出で検出した既知の前フィールドの検出領域毎の動きベクトルＶ(t-1) を用いて、画面内で最も多く検出したベクトル値の内、上位から１，２，３番目のベクトル値とその個数の画面内の総検出領域数に対する割合を調べ、次のようにベクトル値の比較をし、グローバル・ベクトルの値を決める。
【００４８】
ここで、グローバル・ベクトル値をＶｇとし、その確からしさをＲｇとし、検出ベクトル値と個数の割合を上位からそれぞれｖ１，ｖ２，ｖ３とｎ１，ｎ２，ｎ３と表すことにする。また、ベクトル値の比較は、誤差αを許容して行い、グローバル・ベクトル値決定の基礎とする。誤差αを許容したベクトル値の比較は、例えば、ある値ＶｎとＶｍを比較するものとすれば、Ｖｎ＝（Ｖnx，Ｖny）とＶｍ＝（Ｖmx，Ｖmy）のベクトル値が、
【００４９】
【数４】

を満足した場合に、ＶｎとＶｍは、等しいとみなすこととする。
【００５０】
グローバル・ベクトル検出は以下の手順で行う。
【００５１】
１）誤差α＝０を許容して検出ベクトル値を相互に比較し、最多検出ベクトル値の基データｖ１.orgを導出する。
【００５２】
２）次に、誤差α＝３を許容して検出ベクトル値を相互に比較し、ｖ１.orgと等しいあるいは、等しいと見なされた検出ベクトル値の割合を調べ、ｖ１＝ｖ１.orgとし、その割合をｎ１とする。
【００５３】
３）さらに、誤差α＝３を許容して検出ベクトル値を相互に比較し、ｖ１に相当するベクトル値を除いた検出ベクトル値の中から、２番目に多い検出ベクトル値ｖ２とその割合ｎ２を同様に調べる。ｖ２の値は、２番目に多い検出ベクトルの中から最初に選択されたものをｖ２の代表値とする。
【００５４】
４）上記同様の手順に従い、誤差α＝３を許容して検出ベクトル値を相互に比較し、ｖ１，ｖ２に相当するベクトル値を除いた検出ベクトル値の中から、３番目に多い検出ベクトル値ｖ３とその割合ｎ３を調べる。ｖ３の値は、３番目に多い検出ベクトルの中から最初に選択されたものをｖ３の代表値とする。
【００５５】
５）上記で求めたｖ１，ｖ２，ｖ３，ｎ１，ｎ２，ｎ３を用いて以下の条件に従い、初期値をＲｇ＝０として、Ｒｇにポイントを順次加算していき、グローバル・ベクトルの信頼度Ｒｇを求める。
【００５６】
ここで、ベクトル値ｖ１，ｖ２，ｖ３の相互の比較は、誤差β＝２０％を許容して行い、２つのベクトル値が等しい、
【００５７】
【外１】

【００５８】
２つのベクトル値が誤差β＝２０％を越えて異なる場合は、記号≠で表す。誤差βを許容したベクトル値の比較は、例えば、ある値ＶｎとＶｍを比較するものとすれば、Ｖｎ＝（Ｖnx，Ｖny）とＶｍ＝（Ｖmx，Ｖmy）のベクトル値が、
【００５９】
【数５】

を満足した場合に、ＶｎとＶｍは、等しいとみなすこととする。
【００６０】
（ａ） もし ｎ１＞８０％
ならば Ｒg に１０ポイント加算する。
【００６１】
（ｂ） もし ｎ１＞５０％
ならば Ｒg に４ポイント加算する。
【００６２】
【外２】

【００６３】
上記のように信頼度Ｒｇにポイント制を導入し、最終的なＲｇの値が６以上となれば、ｖ１をグローバル・ベクトルとし、
Ｖｇ＝ｖ１
とする。
【００６４】
Ｒｇが６未満の場合は、Ｖｇはグローバル・ベクトルではないとし、
Ｖｇ＝無し
とする。
【００６５】
動きベクトル検出法（例えば、ブロック・マッチング法，勾配法）で検出した既知の前フィールドの検出領域毎の動きベクトルをＶ(t-1) とする。
【００６６】
続いて、本発明を適用した前置フィルタの構成要素である複数の特性の異なるフィルタを図６，図９，図１１，図１２，図１４を参照して、個々に説明する。
【００６７】
イ）前置フィルタＡについて
図６に示した前置フィルタＡは、フレーム信号にタップ数の少ない（例えば３タップ）空間ＬＰＦをかけて１ラインおきに走査線を間引くことにより、フィールド信号を生成するフィルタ（６０Ａ，６０Ｂ）を２個並列に含むものである。
【００６８】
図６，図９，図１１で用いているフレーム→フィールド・サブサンプリングのＬＰＦについて、その構成例を図７に示し、その動作例を図８に示して以下に説明する。
【００６９】
入力のフィールド信号ｆ(t) は、飛び越し走査のテレビジョン信号であるため、ｆ(t-1) とｆ(t) 、ｆ(t) とｆ(t+1) は、垂直方向に１走査線分ずれている。しかし、フレーム間離れたｆ(t-1) とｆ(t+1) は、垂直方向のずれはない。ｆ(t) を奇数フィールドとすると、ｆ(t-1) とｆ(t+1) は偶数フィールドとなる。ここでｆ(t-1) とｆ(t) で構成されるフレーム信号をＦ(t) とし、ｆ(t) とｆ(t+1) で構成されるフレーム信号をＦ(t+1) とする。 フレーム信号Ｆ(t) 、Ｆ(t+1) 空間ＬＰＦをかけたフレーム信号をそれぞれＧ(t) 、Ｇ(t+1) とする。
【００７０】
フレーム信号Ｇ(t) を構成するフィールド信号の内、奇数フィールドをｇｏ(t) とし、偶数フィールドをｇｅ(t) とし、フレーム信号Ｇ(t+1) を構成するフィールド信号の内、奇数フィールドをｇｏ(t+1) とし、偶数フィールドをｇｅ(t+1) とする。
【００７１】
図７で入力画像のｆ(t) が奇数フィールドの場合、フレーム→フィールド・サブサンプリングのＬＰＦから出力されるベクトル検出用輝度信号のｆ'(t)とｆ'(t+1)は、前記のｇｏ(t) とｇｏ(t+1) を出力することとする。入力画像のｆ(t) が偶数フィールドの場合、フレーム→フィールド・サブサンプリングのＬＰＦから出力されるベクトル検出用輝度信号のｆ'(t)とｆ'(t+1)は、前記のｇｅ(t) ｇｅ(t+1) を出力する。つまり、ｆ(t) が奇数フィールドの場合、
ｆ'(t) ＝ｇｏ(t)
ｆ'(t+1)＝ｇｏ(t+1)
となる。従って、同時に出力されるフィールド信号ｇｏ(t) とｇｏ(t+1) は、走査線の垂直方向の空間位置が同じであるため、出力信号ｆ'(t)は走査線が半減された順次走査信号と等価である。
【００７２】
ここで、静止領域では、入力信号のｆ(t-1) とｆ(t+1) は
ｆ(t-1) ＝ｆ(t+1)
で同じ値であるため、ｆ(t-1) とｆ(t) 、ｆ(t) とｆ(t+1) で構成されるフレーム信号Ｆ(t) とＦ(t+1) は、
Ｆ(t) ＝Ｆ(t+1)
となる。従って、Ｆ(t) およびＦ(t+1) に空間ＬＰＦをかけたフレーム信号Ｇ(t) びＧ(t+1) は、
Ｇ(t) ＝Ｇ(t+1)
となる。従って、Ｇ(t) とＧ(t+1) を構成するフィールド信号ｇｏ(t) とｇｅ(t) 、
およびｇｏ(t+1) とｇｅ(t+1) は、
ｇｏ(t) ＝ｇｏ(t+1)
ｇｅ(t) ＝ｇｅ(t+1)
となる。従って、静止領域では、飛び越し走査信号を用いるにも拘らずフレーム→フィールド・サブサンプリングのＬＰＦの出力のベクトル検出用輝度信号を用いることにより、飛び越し走査による影響を避けて、零ベクトルを検出できる。
【００７３】
このようなフィルタを用いないと、入力信号が飛び越し走査の場合、静止領域では、１フレーム分離れたフィールド信号を用いていたために、静動領域の判定を間違えて、動領域であるにも拘らず静止処理を行ったり、静止領域であるにも係わらず動処理を行ったりと、誤った処理による不都合が発生していた。本実施例によるフレーム→フィールド・サブサンプリングのＬＰＦの出力を用いて動きベクトル検出を行うことにより、静止領域では前述のように正しく静止を検出することができ、動領域ではｇｏ(t) とｇｏ(t+1) あるいはｇｅ(t) とｇｅ(t+1) は時間方向に２タップのＬＰＦフィルタをかけたフィールド信号と等価になるため、正しく動きを検出することができる。
【００７４】
すなわち、前置フィルタＡは、図６に示すように３フィールドのデータを用いて、１フィールド分離れていながら走査線の垂直方向の空間位置が一致する信号をベクトル検出用輝度信号として同時に出力するフィルタである。これにより、ノイズの除去とフィールド間での動きを検出することができる。すなわち、テレビ信号のような飛び越し走査の信号を用いてフィールド間の動きベクトル検出を行う場合、垂直方向の周波数成分を持つ静止領域では、前置フィルタＡの出力を用いると１フィールド離れたベクトル検出用輝度信号間の画像の差分値がほぼ零となるため、ブロック・マッチング法あるいは勾配法のベクトル検出方法と組み合わせると零ベクトルを検出することができる。
【００７５】
また、動領域では、時間方向に２タップの時間フィルタをかけたものと等価になるため、勾配法のベクトル検出方法と組み合わせるとフィールド間の動きを検出することできる。
【００７６】
ロ）前置フィルタＢについて
図９の前置フィルタＢは、１フレーム分離れた２つのフィールド信号の加算平均をとることにより時間フィルタをかけ、その出力と１フィールド遅れた同様な出力とで構成されるフレーム信号に図７と同様な空間ＬＰＦをかけて、１ラインおきに走査線を間引くことによりフィールド信号を生成する。
【００７７】
すなわち、図１０に示すように、５フィールドのデータを用いて、１フィールド分離れていながら走査線の垂直方向の空間位置が一致する信号をベクトル検出用輝度信号として出力するフィルタである。用いる空間ＬＰＦは前置フィルタＡと同じタップ数の少ないフィルタであり、時間方向のフィルタリング処理によりノイズの影響を除去することができる。
【００７８】
ＳＮ比の悪い動画像を用いて動きベクトル検出を行う場合、前置フィルタＢの出力を用いると静止領域のベクトル検出用輝度信号間の画像の差分値がほぼ零となるため、ブロック・マッチング法あるいは勾配法のベクトル検出方法と組み合わせると、零ベクトルを検出することができる。また、動領域では、時間方向に４タップの時間フィルタをかけたものと等価になるため、勾配法のベクトル検出方法と組み合わせるとフィールド間の動きを検出することできる。
【００７９】
ハ）前置フィルタＣについて
図１１の前置フィルタＣは、フィールド信号に対して１フレームで巡回する時間方向の巡回フィルタをかけ、その出力と、１フィールド遅れた同様な出力とで構成されるフレーム信号に図７と同様な空間ＬＰＦをかけて、１ラインおきに走査線を間引くことによりフィールド信号を生成する。
【００８０】
すなわち、３フィールドのデータを用いて、１フィールド分離れていながら走査線の垂直方向の空間位置が一致する信号をベクトル検出用輝度信号として出力するフィルタである。これにより、タップ数の少ない空間フィルタや時間フィルタでは除去できないノイズ（例えば、Ｆ／Ｖ変換画像の粒状性ノイズやＳＮの極めて悪い動画像のノイズ）を取り除くことができる。
【００８１】
前置フィルタＣの出力を用いると静止領域のベクトル検出用輝度信号間の画像の差分値がほぼ零となるため、ブロック・マッチング法あるいは勾配法のベクトル検出方法と組み合わせると零ベクトルを検出することができる。また、動領域では、巡回フィルタにより動物体に疑似勾配が付くため、動物体の勾配を用いる動きベクトル検出方法（勾配法）での動きベクトル検出が容易になる。
【００８２】
ニ）前置フィルタＤについて
図１２の前置フィルタＤは、図１３に示すように、３フィールドにわたって連続するフィールド信号に時空間ＬＰＦをかけた信号をベクトル検出用輝度信号とするフィルタを２個並列に含むものである。空間フィルタは、水平フィルタ（例えば、タップ数５で係数が１／４，２／４，２／４，２／４，１／４の一次元のＬＰＦ）と垂直フィルタ（例えば、タップ数５で係数が１／４，２／４，２／４，２／４，１／４の一次元のＬＰＦ）を縦続接続したものである。すなわち、図１２に示すように、４フィールドのデータを用いて、連続する２フィールドのベクトル検出用輝度信号を同時に出力するフィルタである。
【００８３】
ＣＧやシャッター付きのＣＣＤカメラの画像やＦ／Ｖ変換の画像は、動物体が動いたことによるボケがないため、前置フィルタＤに含まれる時間フィルタにより動物体に疑似勾配を付けることで、動物体の勾配を用いる動きベクトル検出方法（勾配法）での動きベクトル検出が容易になる。
【００８４】
ホ）前置フィルタＥについて
図１４の前置フィルタＥは、動領域と静止領域の境界でフィルタリング処理による影響を避けるためにタップ数の少ない（例えば３タップ）空間ＬＰＦをフィールド信号にかけてノイズを除去し、連続する２フィールド分のベクトル検出用輝度信号を出力する。
【００８５】
動物体の勾配を用いない動きベクトル検出方法（ブロック・マッチング法）を用いた場合、時空間方向のフィルタ特性を持たない前置フィルタＥの出力を使用することにより、動物体の画像に疑似勾配が付かない。従って、ブロック・マッチング法のベクトル検出方法を用いた場合、２つのフィールド間の画像の輝度レベルの差分値の絶対値で表される相関平面において、正しい動きベクトルに相当する位置の相関のピークが顕著に現れるため、正しく動きベクトル検出を行うことができる。
【００８６】
次に、ＳＮ比を表すｓｎと前フィールドの検出ベクトルＶ(t-1) と前フィールドのグローバル・ベクトルＶｇと動領域，静止領域の情報を用いて、前置フィルタの特性を切り換える実施例を図１５に示す。本図において、検出領域の静動判定部１５０は、図４に示したブロック構成を有する。また、前置フィルタＡ〜Ｅ（１５１Ａ〜１５１Ｅ）は、それぞれ図６，図９，図１１，図１２，図１４に示した構成を有する。また、グローバルベクトル検出部１５６は、図５に示したブロック構成を有する。
【００８７】
既に述べた通り、本発明の主たる目的は、検出領域の静動状態により、前置フィルタを適応的に切り換えることである。静動判定は、１フレーム分離れた画像間の輝度レベルの差分値の絶対値をあるしきい値で比較して判定する従来の方法や、本発明で用いる図４に示すような方法では、２つの画像間の輝度レベルの相関が不定のところが有るため、１００％の確率で静動判定を行うことはできない。
【００８８】
そこで、本発明の一実施例では、図４に示す静動判定と前フィールドで検出した動きベクトルとを併用して静動判定の正確さを高める。本発明の一実施例において、静止領域と判定する条件は、図４の動き検出で静止領域と判定され、さらに前フィールドの検出ベクトルＶ(t-1) が零の場合である。
【００８９】
まず、静止領域は、主に２つの場合が考えられる。１つは、カメラが固定されて撮影された画像の静止領域と、カメラが動物体を追跡するために移動して撮影された画像の中の動物体の画像は、ほぼ静止領域になる。これらの判定は、グローバル・ベクトルＶｇで判定を行う。静止領域では、飛び越し走査の影響を避けるため、前置フィルタＡを使用する。また、静止領域ではノイズの影響を避けるためにＳＮ比の値により、ＳＮ比が悪い場合は、前置フィルタＣを使用し、カメラが移動して撮影した画像の静止領域では、静動判定の精度を考慮して、時間方向のフィルタ特性が少ない前置フィルタＢを使用する。
【００９０】
次に、動領域およびＶ(t-1) ≠０で動領域と判定された領域では、ブロック・マッチング法を動きベクトル検出方法として用いた場合は、時間方向のフィルタ特性を持たない前置フィルタＥを使用する。勾配法を動きベクトル検出方法として用いた場合は、動物体の画像に動物体が動いたことによるボケ（輝度レベルの勾配）が無くてはならない。ＣＧやシャッター付きカメラの画像やＦ／Ｖ変換画像が入力された場合は、入力画像の動物体の画像にはボケが少ない。
【００９１】
従って、入力画像の種別（ＣＧ等のボケの少ない画像、動画像のボケが大きい撮像管のカメラで撮影された画像）にかかわりなく、カメラが固定で撮影された動物体の画像の動領域に対しては、時間方向のフィルタ特性を持つ前置フィルタＤの出力を用い、カメラが移動して撮影された画像の動領域では、時空間フィルタをかけて疑似勾配を作らなくても動領域にボケが有るため、前置フィルタＥを使用する。
【００９２】
ベクトル検出方法としてブロック・マッチング法を用いる場合の前置フィルタの切り換えの一実施例を図１６に示す。
【００９３】
また、ベクトル検出方法として勾配法を用いる場合の前置フィルタの切り換えの一実施例を図１７に示す。
【００９４】
なお、ベクトル検出方法としてブロック・マッチング法と勾配法を切り換えて用いる場合は、図１５の切り換え情報デコーダ１５２の内容を図１６，図１７に従い切り換えて用いればよい。
【００９５】
【発明の効果】
以上説明した通り、本発明によれば、フィルタリング処理単位であるフィルタ入力信号の画像小領域毎に静動状態やＳＮ比を判定し、特性の異なる複数のフィルタの中からその領域の特性と動きベクトル検出方法との組み合わせに最適な前置フィルタを選択する構成としてあるので、領域毎の最適なフィルタリング処理が可能となり、動きベクトル検出精度を高めた動きベクトル検出が可能となる。
【００９６】
すなわち本発明では、ベクトル検出用輝度信号を出力する特性の異なるフィルタを複数用意し、検出対象領域の画像特性と検出方法に最適なベクトル検出用輝度信号を出力するフィルタを検出対象領域毎に切り換えて使用することにより、誤った動きベクトル検出を行わないようにすることができる。かくして、本発明を実施することにより、誤った動きベクトルを検出することがなくなるため、動きベクトル検出精度を高めることができるという格別な効果を奏することができる。
【図面の簡単な説明】
【図１】（Ａ）は飛び越し走査の垂直方向の走査線の位置関係を示す説明図、（Ｂ）は従来の前置フィルタの出力例を示す説明図、（Ｃ）は飛び越し走査線の時空間位置を示す説明図である。
【図２】（Ａ）は飛び越し走査の垂直方向の走査線の位置関係を示す説明図、（Ｂ）は従来の前置フィルタの出力例を示す説明図である。
【図３】従来の前置フィルタの回路の一実施例を示すブロック図である。
【図４】本発明の静動判定回路の一実施例を示すブロック図である。
【図５】本発明のグローバル・ベクトル検出の一実施例を示すブロック図である。
【図６】本発明の前置フィルタＡの回路の一実施例を示すブロック図である。
【図７】本発明の前置フィルタＡ，Ｂ，Ｃに含まれるフレーム→フィールド・サブサンプリングのＬＰＦの回路の一実施例を示すブロック図である。
【図８】本発明の前置フィルタＡ，Ｂ，Ｃに含まれるフレーム→フィールド・サブサンプリングのＬＰＦの回路の動作例を示す説明図である。
【図９】本発明の前置フィルタＢの回路の一実施例を示すブロック図である。
【図１０】本発明の前置フィルタＢの回路の動作例を示す説明図である。
【図１１】本発明の前置フィルタＣの回路の一実施例を示すブロック図である。
【図１２】本発明の前置フィルタＤの回路の一実施例を示すブロック図である。
【図１３】本発明の前置フィルタＤに含まれる時空間フィルタの回路の一実施例を示すブロック図である。
【図１４】本発明の前置フィルタＥの回路の一実施例を示すブロック図である。
【図１５】本発明の前置フィルタの切り換え回路の一実施例を示すブロック図である。
【図１６】本発明の前置フィルタのブロック・マッチング法に対応したデコーダにおける切り換えの一実施例を示す図である。
【図１７】本発明の前置フィルタの勾配法に対応したデコーダにおける切り換えの一実施例を示す図である。
【符号の説明】
１５０ 検出領域の静動判定部
１５１Ａ〜１５１Ｅ 前置フィルタＡ〜前置フィルタＥ
１５２ 切り換え情報デコーダ
１５３ ベクトル検出用輝度信号の選択部
１５４ 動きベクトル検出部
１５５ 動きベクトル値メモリ
１５６ グローバルベクトル検出部
１５７ ＳＮ比検出部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a motion vector detection method and an adaptive switching prefilter for motion vector detection.
[0002]
More specifically, the present invention provides an optimum detection from an input moving image when detecting a motion vector of a video signal. Movement The present invention relates to a motion vector detection method and an adaptive switching prefilter for motion vector detection suitable for generating a vector detection luminance signal.
[0003]
[Prior art]
One frame signal of the interlaced scanning television signal is composed of two field signals (odd field and even field) spatially shifted by one scanning line from each other in the vertical direction.
[0004]
Therefore, the vector detection between two fields improves the vector detection accuracy and further simplifies the hardware, so that the vertical spatial position of one field signal from the other field signal in the odd field or even field. Match pixel sample point to The The generated sequential scanning signal is generated.
[0005]
1A and 2A show specific examples of the positional relationship between the scanning lines in the vertical direction of the input image and the sample points of the luminance signal. The scanning lines in FIGS. 1A and 2A are cross-sectional views as shown in FIG. 1C, and the scanning lines exist in a direction penetrating the paper surface.
[0006]
Conventionally, when a sampling point is matched with a vertical spatial position of one field signal in the other field signal, an interpolated signal is generated by applying LPF in the field. For example, in order to match the sampling point of the scanning line of the f (t + 1) field with the sampling point of the scanning line of the f (t) field, the pixel values on the adjacent scanning lines above and below the f (t + 1) field are added and divided by two. When such a filter is applied, filter outputs as shown in FIGS. 1B and 2B are obtained. As can be seen from the example of FIG. 1B, the luminance level of the sample point interpolated at the luminance level boundary is inappropriate, so that the region having the frequency component in the vertical direction in the static region (for example, the boundary of the luminance level) In the region where) exists, the signal difference between two pseudo-sequential signals at the same spatial position does not become zero, so a zero vector (still) cannot be detected. Further, as can be seen from the example of FIG. 2B, since noise removal considering the interlace structure is not performed, an erroneous motion vector may be detected near the noise.
[0007]
FIG. 3 shows an example of a pre-filter using a spatio-temporal filter (paper “HDTV system converter” Yuji Nojiri et al., TV Society Journal Vol. 48, No. 1, pp. 84-94, 1994). . In order to perform motion vector detection by the gradient method, that is, the vector detection method using the motion vector detection method for investigating the correlation of the luminance gradient, if a pre-filter as shown in FIG. Since the time filter of the tap and the horizontal and vertical spatial filters are applied, the motion vector can be detected even if the image of the moving object is not blurred.
[0008]
However, since the prefilter shown in FIG. 3 has time-direction filter characteristics, in the motion vector detection method using the block matching method, that is, the motion vector detection method for examining the correlation between the difference values of two images, There is a problem that it is difficult to calculate a correct matching value. Further, depending on the coefficient of the time filter, there may be an influence due to the interlace structure as shown in FIG.
[0009]
[Problems to be solved by the invention]
Conventionally, a motion vector detection device using a plurality of vector detection methods connected in cascade ("TV system conversion device using motion vectors" Shimano et al., TV Society National Convention 1989), or switching between a plurality of vector detection methods In the motion vector detection device, since a single pre-filter is used to generate the vector detection luminance signal, the single pre-filter is used as the image characteristic of the detection region that is the target of motion vector detection and In all combinations of a plurality of vector detection methods, optimal filter characteristics cannot always be obtained.
[0010]
There is also a method of performing motion vector detection by adaptively selecting a filter having optimum filter characteristics for the combination of the image characteristics of the input moving image and the vector detection method for each detection region to be detected by the vector. Not proposed.
[0011]
Therefore, in the conventional motion vector detection device, a case where a luminance signal for vector detection inappropriate for vector detection occurs may occur, and in this case, an accurate motion vector cannot be detected. Specifically, the following inconvenience occurs.
[0012]
A conventional pre-filter that filters only the signal in the field in a static region having a vertical frequency component when motion vector detection is performed using interlaced scanning signals such as television signals. When the output (for example, see FIG. 1B) of the output (for example, vertical LPF in the field, vertical DD (Digital-Digital) conversion filter in the field) is used, the difference value of the image between the fields does not become zero. For this reason, if a detection method (for example, a block matching method) in which the correlation between fields is examined to detect a motion vector is combined, an incorrect motion vector is detected.
[0013]
In addition, when motion vector detection is performed using a moving image with a poor signal-to-noise ratio in a television signal for interlace scanning, a conventional prefilter (for example, a field filter having no temporal filter characteristics and in-frame filter characteristics) is used. If the output (for example, see FIG. 2 (B)) of the vertical LPF in the field and the vertical DD conversion filter in the field) is used, the image difference between the fields in the still region due to the influence of noise and the influence of the frequency component in the vertical direction. Since the value does not become zero, combining a detection method (for example, a block matching method) for detecting a motion vector by checking a correlation between fields, an erroneous motion vector is detected in a still region. In addition, an erroneous motion vector is detected even in the moving region. In particular, when the detection area is reduced, the influence of noise becomes significant.
[0014]
Furthermore, in CG (Computer Graphic) and CCD camera images with shutters and F / V conversion images (images converted from film to television signals), blur due to the movement of the moving object is seen in the moving object image. Absent. A method of performing motion vector detection using the moving object's gradient by using such a moving image as an input image (for example, the gradient method) has a conventional prefix having no spatio-temporal filter characteristics. Combining filters (eg, vertical LPF in a field, vertical DD conversion filter in a field) does not allow vector detection at all.
[0015]
When a motion vector detection method that does not use the gradient of the moving object (eg, block matching method) is combined with a conventional prefilter (eg, temporal filter) having a temporal or spatial filter characteristic, the temporal filter Due to the effect of the above, an image of the moving object is synthesized in the stationary region, so that it is difficult to detect a correct motion vector.
[0016]
Furthermore, a vector detection method (for example, a block matching method, a gradient method) is used for one of the detection conditions that the luminance level of the moving object does not change even if the moving object moves with respect to an image having an extremely poor SN ratio. If used, erroneous vector detection may be performed due to changes in noise, the luminance level of the moving object, or the luminance level of the background.
[0017]
When detecting a motion vector, if a vector detection luminance signal that does not match the vector detection method is used, many erroneous motion vectors are detected. Although it is necessary to confirm that the detection condition of the vector detection method is satisfied, motion vector detection is performed without confirming this, so an incorrect motion vector is detected in a detection region that does not meet the detection condition. It is caused by detecting.
[0018]
As described above, in a motion vector detection device that uses a plurality of vector detection methods in cascade or a motion vector detection device that uses a plurality of vector detection methods by switching, a single prefilter is used for vector detection. When the luminance signal is generated, the vector detection may be erroneous depending on the combination of the input image characteristics and the vector detection method. Therefore, in such a motion vector detection device, an optimum filter cannot be used for all combinations of the image characteristics of the vector detection target region and the vector detection method.
[0019]
Therefore, in view of the above points, an object of the present invention is to determine the image characteristics of an input moving image for each detection region, and to adaptively select an optimum filter for the combination of the image characteristics and the vector detection method. Another object of the present invention is to provide a motion vector detection method and an adaptive switching prefilter for motion vector detection.
[0020]
[Means for Solving the Problems]
In order to achieve the above object, in the motion vector detection method according to the present invention, a plurality of filters having different characteristics for outputting a vector detection luminance signal are prepared, and the image characteristics and vector detection method in the detection target area of the input moving image Motion vector detection is performed by switching and using a filter for outputting a vector detection luminance signal suitable for each detection target region.
[0021]
In the adaptive switching pre-filter for motion vector detection according to the present invention, a plurality of filter means having different filter characteristics, a determination means for determining image characteristics for each detection region of the input moving image, and the determination means And selecting means for switching the filter means for each detection region in accordance with a combination of the determination result obtained by the above and various vector detection methods, and detecting a motion vector based on a vector detection luminance signal output from the filter means It is what.
[0022]
Furthermore, in another embodiment to which the present invention is applied, in the pre-filter connected to the front stage of the motion vector detecting device for an image displayed by the interlace scanning method, the filter has a characteristic of each input image signal being input. Two or more different filters, a filter switching unit that switches the two or more filters and outputs one filter processing result to the motion vector detection device, and motion vector detection of the previous field that is the output of the motion vector detection device Based on the result, if it is determined that there is a global motion in the entire previous field image, the global motion level is output as a global vector detection value, and it is determined that there is no global motion. In some cases, a global vector detection means for outputting a global vector non-detection signal and a detection region which is a unit of filter processing And a static motion determination means for outputting a determination result as to whether the region is a dynamic region or a static region, and the SN ratio of the entire input image is determined for each field to determine whether the region is a low SN region or a high SN region. An S / N ratio determining means for outputting the determination result, and the detection based on the global vector detection result, the motion vector detection result of the immediately preceding field for each detection region, the static motion determination result, and the S / N ratio determination result. And a switching information decoder that outputs switching information for each region to the filter switching unit.
[0023]
Here, the two or more filters having different characteristics have a continuous 3 field 2: 1 interlaced image signal as an input signal, a frame signal formed from the first field signal and the second field signal, and a second field signal. Each of the frame signal formed from the field signal and the third field signal is subjected to a spatial low-pass filter to generate two sequential scanning signals, and then a scanning line is formed for each line for each of the two sequential scanning signals. By subtracting out, the filter A that outputs two field signals that match the odd / even field format and the continuous 5: 1 2: 1 interlaced image signal are input signals, and the first and third field signals are averaged. Field obtained by averaging the field signal obtained in this way and the second and fourth field signals Of the frame signal formed from the sum of the frame signal formed from the signal and the field signal obtained by averaging the second and fourth field signals, and the field signal obtained by averaging the third and fifth field signals. Two field signals in which the odd-even field format is matched by thinning out the scanning line for each line for each of the two sequential scanning signals after generating a spatial scanning low pass filter for each of the two sequential scanning signals. , A cyclic filter that adds a signal obtained by delaying the output signal by one frame to a 2: 1 interlaced image signal that is an input signal, and outputs the continuous signal that is an output of the cyclic filter. The 3 field 2: 1 interlaced image signal is used as the input signal, and the first field signal and the second field are input. The frame signal formed from the second signal and the frame signal formed from the second field signal and the third field signal are each subjected to a spatial low-pass filter to generate two sequential scan signals, and then the two sequential scans A filter C in which a filter that outputs two field signals in which the scan lines are thinned out for each line and the odd-even field format is matched is connected to each of the signals, a spatial low-pass filter output, and the spatial low-pass filter output And a filter E that outputs a two-field 2: 1 interlaced image signal of two consecutive fields, and the switching information decoder includes the detection region in a stationary region, a high SN region, and the previous detection region in the detection region. When the motion vector detection value of the field is “0”, the filter A is A detection value of a global vector or a non-detection signal of a global vector other than “0” and “0” in the motion vector detection value of the previous field in the detection region is a static region, a low SN region, and the detection region is output. If the detection area is a static area, a low SN area, and the motion vector detection value of the previous field in the detection area is "0" and the detection value of the global vector is "0" Filter C is selected when the detection area is a static area and the motion vector detection value of the previous field in the detection area is other than “0”, or when the detection area is a moving area. It is preferable that the switching information to be output is output. Alternatively, the two or more filters having different characteristics may have a frame signal and a second field formed from a first field signal and a second field signal using a continuous 3: 1 2: 1 interlaced image signal as an input signal. Each frame signal formed from the signal and the third field signal is subjected to a spatial low-pass filter to generate two sequential scanning signals, and then the scanning lines are thinned out for each of the two sequential scanning signals. As a result, the filter A that outputs two field signals that match the odd / even field format and the continuous 5: 1 2: 1 interlaced image signal are input signals, and the first and third field signals are added and averaged. Field obtained by averaging the obtained field signal and the second and fourth field signals A frame signal formed from the signal and a field signal obtained by averaging the second and fourth field signals and a field signal obtained by averaging the third and fifth field signals. Two field signals in which the odd-even field format is matched by thinning out the scanning line for each line for each of the two sequential scanning signals after generating a spatial scanning low pass filter for each of the two sequential scanning signals. , A cyclic filter that adds a signal obtained by delaying the output signal by one frame to a 2: 1 interlaced image signal that is an input signal, and outputs the continuous signal that is an output of the cyclic filter. The 3 field 2: 1 interlaced image signal is used as the input signal, and the first field signal and the second field signal are input. A frame signal formed from the frame signal and a frame signal formed from the second field signal and the third field signal by applying a spatial low-pass filter to generate two sequential scan signals, and then the two sequential scans A filter C obtained by cascade-connecting a filter that outputs two field signals in which the scan lines are thinned out for each line and the odd-even field format is matched to each of the signals, and a 2: 1 interlaced image signal of four consecutive fields Is a time low-pass filter that adds a continuous three-field signal and each one-line delayed output signal at an arbitrary coefficient ratio and outputs a sequential scanning signal, and a spatial low-pass that converts the sequential scanning signal into a field signal A pair of filters formed by cascade-connecting filters is provided, and three consecutive filters of the pair of filters are provided. The field input signal is a first, second, or third field signal, or a second, third, or fourth field signal, and the pair of filter outputs is a 2: 1 interlace of two consecutive fields. A switching information decoder comprising: a filter and D which are image signals; a spatial low-pass filter output; and a filter E which outputs a one-field delayed output of the spatial low-pass filter output as a continuous two-field 2: 1 interlaced image signal. When the detection area is a static area and a high SN area and the motion vector detection value of the previous field in the detection area is “0”, the filter A is selected, and the detection area is a static area and a low SN area and the detection is performed. The detection value of the global vector whose motion vector detection value of the previous field in the region is “0” and other than “0”. When a global vector non-detection signal is output, filter B is used, and the detection region is a static region and a low SN region, and the motion vector detection value of the previous field in the detection region is “0” and the global vector When the detection value of the detection vector is “0”, the filter C is set to be a still region, the motion vector detection value of the previous field in the detection region is other than “0”, and the detection value of the global vector is “0” or When a global vector non-detection signal is output, the filter D is connected to a global region where the detection region is a static region and the motion vector detection value of the previous field in the detection region is other than “0” and other than “0”. If a vector detection value is output, or if the detection area is a moving area and a global vector other than “0” Filter E when the detected value is output, and filter D when the detected region is the moving region and the detected value of the global vector is "0" or the non-detected signal of the global vector is output. Can be configured to output the switching information to be selected.
[0024]
Further, the global vector detection means identifies a plurality of vector values having a large detection ratio and their detection ratios by comparing the motion vectors of the immediately preceding field for each detection region within an allowable error range. When the sum of the detection ratios of the vector values included in a certain range out of the plurality of vector values exceeds a certain value, the value of the most frequently detected vector is output as the detected value of the global vector and falls below the certain value. In some cases, it is preferable to output a non-detection signal of a global vector. Further, the static motion determination means includes a first field signal and a third field signal, or a third field signal and a fifth field signal, among 2 consecutive interlaced image signals of 5 fields. Between the pixels occupying the same spatial position between the second field signal and the third field signal, or between the third field signal and the fourth field signal, And the absolute value of the difference in luminance signal level between the pixel in the temporally subsequent field and the pixel in the temporally subsequent field located either above or below the position corresponding to the spatial position of the pixel. Only when the quotient obtained by dividing the absolute value of the gradient and the absolute value of the vertical gradient by a large value exceeds a certain value in all the four cases, it is determined as a moving pixel, and the movement within the detection region is determined. The area only when the number of the unit is above a certain value determined to moving region, when below a certain value it is preferable to determine a still region. In addition, the S / N ratio determining means may detect that the ratio of the detection areas determined as still areas by the static movement determining means is below a certain value and the detection value of the global vector is “0”. When it is determined as a high SN region, and the ratio of the detection region is below a certain value and has a detection value of a global vector other than “0”, or a non-detection signal of a global vector is output When the ratio of the detection areas determined as the low SN area and determined as the static area by the static motion determination means exceeds a certain value, 1% is obtained for all the pixels in all the detection areas determined as the static area. The absolute value of the difference in luminance signal level is obtained for each pixel from all pixels after the frame, and when these total averages are below a certain value, it is determined as a high SN region, and when it exceeds a certain value, the low SN region is obtained. Structure It is also possible to be.
[0025]
As described above, in the prefilter according to an embodiment of the present invention, in order to solve the above-described problem and perform motion vector detection with high detection accuracy, an interlaced scanning image signal is used as an input signal, and a vector detection luminance signal is generated. A plurality of pre-filters having different filter characteristics to be output are prepared, and a configuration in which these are adaptively switched is adopted. Furthermore, in order to switch a plurality of filters, image characteristics are determined for each detection region of the input moving image. Then, the optimum filter for the combination of the image characteristic of the determination result and the vector detection method is selectively switched for each detection region, and the optimum vector detection luminance signal is supplied to a circuit for vector detection.
[0026]
DETAILED DESCRIPTION OF THE INVENTION
As an embodiment of the present invention, a pre-filter used when detecting a motion vector of a moving image between consecutive fields is disclosed.
[0027]
That is, in order to create a vector detection luminance signal, a plurality of filters having different characteristics (for example, spatial LPF, temporal LPF, spatiotemporal LPF, cyclic filter in time direction, and progressive scan conversion filter) are prepared, and the input moving image Image characteristics (for example, static state, past motion vector, SN state, global vector representing global motion in the moving image), and image characteristics and vector detection methods (for example, block matching method, Depending on the combination of the gradient method and the phase correlation method, the optimal (for example, when using the block matching method as the vector detection method) When vector detection is performed by the gradient method using an image without blurring, a filter with filter characteristics in the time direction is used. Select a filter with filter characteristics from a plurality of filters and use the output signal of the selected filter for vector detection. By using a luminance signal, vector detection accuracy is improved.
[0028]
More specifically,
A vector detection luminance signal is generated from an input moving image, the vector detection luminance signal is divided into small areas (hereinafter referred to as detection areas), and a continuous field is generated using the vector detection luminance signal for each detection area. In a pre-filter (see FIG. 15) that generates a vector detection luminance signal of a vector detection device that performs motion vector detection of moving images between
The prefilter is
Prefilter A (151A), Prefilter B (151B), Prefilter C (151C), Prefilter D (151D), Prefilter E (151E) connected in parallel,
Including switching means (153) for selecting and outputting one of the plurality of filters,
Image characteristics of the input moving image region corresponding to the detection region (for example, the static state of the region, the past motion vector of the region, the SN state of the input moving image including the region, the global in the input moving image A global vector representing a particular motion), a static state (150) of the region, a past motion vector (154, 155) of the region, and an SN state (157) of an input moving image including the region Using the past global vector (156) representing the global movement in the input moving image, and determining means according to the image characteristic determination table shown in FIG. 16 and FIG.
Furthermore, as a control signal of the switching means (153), the optimum filter for the vector detection method among the plurality of filters (for example, a block matching method is used as the vector detection method) according to the determination result of the image characteristics of the input moving image area. A decoder (152) is provided that outputs a signal for selecting one filter having a filter characteristic (if a filter having no filter characteristic in the time direction is used).
[0029]
As described above, when a filter matched to the combination of the image characteristic and the vector detection method is used, that is, in the combination of the static region having the frequency component in the vertical direction and the block matching method, the prefilter A (FIG. 6). Or a pre-filter B (see FIG. 9), a pre-filter C (see FIG. 11), or a pre-filter D (see FIG. 12) in the combination of a gradient method and a material that does not blur the image of the moving object. In a combination of an image with a very bad SN ratio and a block matching method or a gradient method, when the prefilter B (see FIG. 9) or the prefilter C (see FIG. 11) is used, There is no inconvenience as in the conventional example described above.
[0030]
【Example】
Prior to describing the prefilter to which the present invention is applied, a method for obtaining control information for selecting the output of the prefilter according to the image characteristics will be described below.
[0031]
Here, the current field image is f (t), the next field image is f (t + 1), the next field image is f (t + 2), the previous field image is f (t-1), and the previous field image is f (t-1). Let f (t-2) be the field image.
[0032]
(I) About the static state of the detection area
As shown in FIG. 4, motion detection between f (t) and f (t + 1), motion detection between f (t) and f (t + 2), and f (t) and f (t− The motion detection between 1) and the motion detection between f (t) and f (t-2) are performed for each pixel, and the motion region and the still region on the image f (t) are detected. Motion detection is performed by, for example, calculating the absolute value of the difference in luminance level of the same pixel between fields with the larger one of the absolute value of the horizontal gradient and the vertical gradient of the luminance level of the pixel of f (t). It is obtained by the method of dividing.
[0033]
A specific motion detection method will be described below by taking as an example one pixel p of sampling points sampled in a grid pattern on f (t).
[0034]
Let q be a pixel on f (t + 1) at the same spatial position as pixel p on f (t). When the image f (t) is an interlaced scanning signal, f (t) and f (t + 1) are separated by one field, so f (t) corresponding to the spatial position of the pixel p on f (t). +1) There is no upper pixel. Therefore, the pixel on f (t + 1) above or below the position corresponding to the spatial position of the pixel p is selected as the pixel q on f (t + 1). The spatial position of the pixel p is (x, y), the luminance level is expressed as F (x, y), and the luminance level of the pixel selected as the pixel q on f (t + 1) is G (x, y). ).
[0035]
The gradient ▽ p of pixel p
[0036]
[Expression 1]

If defined, the movement m1 of the pixel p is
[0037]
[Expression 2]

[0038]
It can be expressed. Here, in order to avoid the influence of noise, mth is defined,
m1 ≧ mth
When
M1 = 1
age,
m1 <mth
In the case of
M1 = 0
And
[0039]
Similarly, between a pixel p on f (t) and each pixel on f (t + 2), f (t-1) and f (t-2) at the same spatial position. Then, motion detection is performed, and the respective motion detection results are denoted as M2, M3, and M4.
[0040]
next,
[0041]
[Equation 3]
Mp = (M1 and M2 and M3 and M4) (3)
Check
When Mp = 0, the pixel p is a static region
age,
When Mp = 1, the pixel p is a moving region.
Is determined.
[0042]
The above processing is performed for all the pixels on f (t), and a moving area and a stationary area for each pixel on f (t) are obtained.
[0043]
Next, in order to perform static motion determination for each detection area on f (t), among the pixels in the detection area,
Mp = 1
Let m be the number of Define nth for static judgment,
m> nth
In this case, the detection area is the moving area,
M = 1
age,
m ≦ nth
In this case, the detection area is a static area,
M = 0
Is determined. Similarly, static motion determination is performed for all detection areas on f (t).
[0044]
(II) SN ratio
Next, for all pixels in all still areas where M = 0 of f (t), between pixels occupying the same spatial position as the still area of f (t) on f (t + 2) Thus, the absolute value of the difference in luminance level between frames is obtained, and the average N of the absolute values of the differences in luminance level per pixel in all still regions is obtained. The S / N ratio is determined on a field basis. The number M of detection areas where M = 0, which represents the static state of the detection area (I), is M = 0 in the screen, and that number is C. Here, in the panning image, since there is little or no static region, a threshold value Cth is introduced,
C ≦ Cth
In this case, the global vector Vg of the previous field separately obtained (see (III) described below) is referred to. When Vg ≠ 0 or Vg = none, the SN ratio is bad.
sn = 1
When Vg = 0, the SN ratio is good.
sn = 0
And
[0045]
When C> Cth, Nth for determining the SN ratio is introduced,
N <Nth
In this case, the signal-to-noise ratio is good.
sn = 0
And
[0046]
N ≧ Nth
In this case, the signal-to-noise ratio is bad,
sn = 1
And
[0047]
(III) About Global Vector
As shown in FIG. 5, the global vector of the previous field is obtained. The global vector is determined on a field basis. Using the motion vector V (t-1) for each detection area of the known previous field detected by the motion vector detection, the first, second, and third vector values from the top among the vector values detected most in the screen Then, the ratio of the number to the total number of detection areas in the screen is examined, and the vector values are compared as follows to determine the value of the global vector.
[0048]
Here, the global vector value is represented by Vg, the probability thereof is represented by Rg, and the ratio of the detected vector value and the number is represented by v1, v2, v3, n1, n2, and n3 from the top. Further, the comparison of vector values is performed while allowing an error α, which is the basis for determining global vector values. For comparison of vector values that allow the error α, for example, if a certain value Vn and Vm are compared, the vector values of Vn = (Vnx, Vny) and Vm = (Vmx, Vmy) are
[0049]
[Expression 4]

If Vn is satisfied, Vn and Vm are considered equal.
[0050]
The global vector detection is performed according to the following procedure.
[0051]
1) The detection vector values are compared with each other while allowing the error α = 0, and the base data v1.org of the most detected vector values is derived.
[0052]
2) Next, the detection vector values are compared with each other while allowing the error α = 3, and the ratio of the detection vector values that are equal to or considered to be equal to v1.org is determined as v1 = v1.org. Let the ratio be n1.
[0053]
3) Further, the detection vector values are compared with each other while allowing the error α = 3, and the second largest detection vector value v2 and its ratio n2 are detected from the detection vector values excluding the vector value corresponding to v1. Do the same. For the value of v2, the first value selected from the second most detected vectors is the representative value of v2.
[0054]
4) According to the same procedure as described above, the detection vector values are compared with each other while allowing the error α = 3, and the third largest detection vector value among the detection vector values excluding the vector values corresponding to v1 and v2. Check v3 and its ratio n3. As the value of v3, the first selected value from the third most detected vectors is the representative value of v3.
[0055]
5) Using the v1, v2, v3, n1, n2, and n3 obtained above, the initial value is set to Rg = 0, and points are sequentially added to Rg, and the global vector reliability Rg Ask for.
[0056]
Here, the vector values v1, v2, and v3 are compared with each other with an error β = 20%, and the two vector values are equal.
[0057]
[Outside 1]

[0058]
If the two vector values differ by more than an error β = 20%, they are represented by the symbol ≠. For comparison of vector values that allow the error β, for example, if a certain value Vn and Vm are compared, the vector values of Vn = (Vnx, Vny) and Vm = (Vmx, Vmy) are
[0059]
[Equation 5]

If Vn is satisfied, Vn and Vm are considered equal.
[0060]
(A) If n1> 80%
If so, add 10 points to Rg.
[0061]
(B) If n1> 50%
If so, add 4 points to Rg.
[0062]
[Outside 2]

[0063]
If the point system is introduced to the reliability Rg as described above and the final Rg value is 6 or more, v1 is a global vector,
Vg = v1
And
[0064]
If Rg is less than 6, then Vg is not a global vector,
Vg = None
And
[0065]
Let V (t-1) be the motion vector for each detection area of the known previous field detected by the motion vector detection method (for example, block matching method, gradient method).
[0066]
Subsequently, a plurality of filters having different characteristics, which are constituent elements of the pre-filter to which the present invention is applied, will be described individually with reference to FIGS. 6, 9, 11, 12, and 14.
[0067]
B) Pre-filter A
The pre-filter A shown in FIG. 6 applies a space LPF with a small number of taps (for example, 3 taps) to the frame signal and thins out scanning lines every other line to generate field signals (60A, 60B). Are included in parallel.
[0068]
A configuration example of the frame → field subsampling LPF used in FIGS. 6, 9, and 11 is shown in FIG. 7, and an operation example thereof is shown in FIG.
[0069]
Since the input field signal f (t) is an interlaced scanning television signal, f (t-1) and f (t) and f (t) and f (t + 1) are scanned one time in the vertical direction. The line segment is off. However, f (t-1) and f (t + 1) separated from each other in the frame do not have a vertical shift. If f (t) is an odd field, f (t-1) and f (t + 1) are even fields. Here, the frame signal composed of f (t-1) and f (t) is F (t), and the frame signal composed of f (t) and f (t + 1) is F (t + 1). And Frame signals F (t) and F (t + 1) Frame signals multiplied by the space LPF are denoted by G (t) and G (t + 1), respectively.
[0070]
Of the field signals constituting the frame signal G (t), the odd field is designated as go (t), the even field is designated as ge (t), and the odd number field among the field signals constituting the frame signal G (t + 1). Is go (t + 1), and the even field is ge (t + 1).
[0071]
In FIG. 7, when f (t) of the input image is an odd field, the vector detection luminance signals f ′ (t) and f ′ (t + 1) output from the frame-to-field subsampling LPF are Go (t) and go (t + 1) are output. When f (t) of the input image is an even field, the vector detection luminance signals f ′ (t) and f ′ (t + 1) output from the frame-to-field sub-sampling LPF are the ge ( t) Output ge (t + 1). That is, if f (t) is an odd field,
f ′ (t) = go (t)
f ′ (t + 1) = go (t + 1)
It becomes. Accordingly, the field signals go (t) and go (t + 1) output at the same time have the same spatial position in the vertical direction of the scanning line, so that the output signal f ′ (t) is sequentially reduced by half the scanning line. It is equivalent to the scanning signal.
[0072]
Here, in the static region, f (t-1) and f (t + 1) of the input signal are
f (t-1) = f (t + 1)
Therefore, frame signals F (t) and F (t + 1) composed of f (t-1) and f (t) and f (t) and f (t + 1) are
F (t) = F (t + 1)
It becomes. Therefore, the frame signals G (t) and G (t + 1) obtained by multiplying F (t) and F (t + 1) by the spatial LPF are
G (t) = G (t + 1)
It becomes. Accordingly, field signals go (t) and ge (t) constituting G (t) and G (t + 1),
And go (t + 1) and ge (t + 1) are
go (t) = go (t + 1)
ge (t) = ge (t + 1)
It becomes. Therefore, in the still region, the zero vector can be detected while avoiding the influence of the interlaced scanning by using the vector detection luminance signal of the output of the LPF of the frame → field subsampling in spite of using the interlaced scanning signal.
[0073]
If such a filter is not used, when the input signal is interlaced scanning, the field signal separated by one frame is used in the still region, so that the determination of the static region is wrong and the moving region is used. However, inconvenience due to erroneous processing has occurred, such as performing static processing or performing dynamic processing in spite of being a static region. By performing motion vector detection using the output of the frame-> field subsampling LPF according to the present embodiment, it is possible to detect stillness correctly in the stationary region as described above, and in the moving region, go (t) and go Since (t + 1) or ge (t) and ge (t + 1) are equivalent to a field signal obtained by applying a 2-tap LPF filter in the time direction, motion can be detected correctly.
[0074]
In other words, the pre-filter A uses three fields of data as shown in FIG. 6 and simultaneously outputs a signal that is separated by one field but whose spatial position in the vertical direction of the scanning line matches as a vector detection luminance signal. It is a filter. Thereby, it is possible to detect noise removal and motion between fields. That is, when motion vector detection between fields is performed using interlaced scanning signals such as television signals, in the still region having vertical frequency components, if the output of the prefilter A is used, vector detection separated by one field is performed. Since the difference value of the image between the luminance signals for use is almost zero, the zero vector can be detected when combined with the vector detection method of the block matching method or the gradient method.
[0075]
Also, since the motion region is equivalent to a time-tapped 2-tap time filter, it is possible to detect motion between fields when combined with the gradient vector detection method.
[0076]
B) Pre-filter B
The pre-filter B of FIG. 9 applies a time filter by taking an average of two field signals separated by one frame, and converts a frame signal composed of the output and a similar output delayed by one field into FIG. A field signal is generated by thinning out the scanning lines every other line by applying the same space LPF.
[0077]
That is, as shown in FIG. 10, this is a filter that uses five fields of data and outputs as a vector detection luminance signal a signal in which the spatial positions in the vertical direction of the scanning lines match while being separated by one field. The spatial LPF to be used is a filter having the same number of taps as that of the pre-filter A, and the influence of noise can be removed by filtering in the time direction.
[0078]
When motion vector detection is performed using a moving image with a poor S / N ratio, if the output of the pre-filter B is used, the difference value of the image between the vector detection luminance signals in the still region becomes almost zero. Alternatively, a zero vector can be detected by combining with a gradient vector detection method. In addition, since the motion region is equivalent to a time-filtered 4-tap time filter, it is possible to detect motion between fields when combined with a gradient vector detection method.
[0079]
C) Pre-filter C
The prefilter C in FIG. 11 applies a cyclic filter in the time direction that circulates in one frame to the field signal, and a frame signal composed of the output and a similar output delayed by one field is the same as in FIG. A field signal is generated by thinning out scanning lines every other line by applying a small space LPF.
[0080]
That is, it is a filter that uses data of three fields and outputs a signal in which the spatial positions in the vertical direction of the scanning lines coincide with each other while being separated by one field as a vector detection luminance signal. As a result, it is possible to remove noise that cannot be removed by a spatial filter or a temporal filter with a small number of taps (for example, granularity noise of an F / V converted image or noise of a moving image having a very poor SN).
[0081]
If the output of the prefilter C is used, the difference value of the image between the luminance signals for detecting the vector in the still region is almost zero, so that the zero vector can be detected when combined with the block matching method or the gradient method vector detection method. Can do. In the moving region, since the moving object is given a pseudo gradient by the cyclic filter, it is easy to detect the motion vector by the motion vector detecting method (gradient method) using the gradient of the moving object.
[0082]
D) Pre-filter D
As shown in FIG. 13, the prefilter D in FIG. 12 includes two filters in parallel that use a signal obtained by multiplying a field signal continuous over three fields by a spatiotemporal LPF as a vector detection luminance signal. The spatial filter includes a horizontal filter (for example, a one-dimensional LPF with 5 taps and coefficients of 1/4, 2/4, 2/4, 2/4, and 1/4) and a vertical filter (for example, with 5 taps). A one-dimensional LPF whose coefficients are 1/4, 2/4, 2/4, 2/4, and 1/4) are cascade-connected. That is, as shown in FIG. 12, it is a filter that simultaneously outputs two fields of vector detection luminance signals using four fields of data.
[0083]
Since there is no blur caused by the movement of the moving object, the image of the CCD camera with a CG or shutter and the image of the F / V conversion are given a pseudo gradient to the moving object by the time filter included in the pre-filter D. Motion vector detection by the motion vector detection method (gradient method) using the gradient of the moving object becomes easy.
[0084]
E) Pre-filter E
The pre-filter E in FIG. 14 removes noise by applying a space LPF with a small number of taps (for example, 3 taps) to a field signal in order to avoid the influence of the filtering process at the boundary between the moving region and the stationary region, and removes two continuous fields The vector detection luminance signal is output.
[0085]
When the motion vector detection method (block matching method) that does not use the gradient of the moving object is used, by using the output of the pre-filter E that does not have the filter characteristics in the spatio-temporal direction, the pseudo gradient is added to the image of the moving object. Is not attached. Therefore, when the vector detection method of the block matching method is used, the correlation peak at the position corresponding to the correct motion vector is present on the correlation plane represented by the absolute value of the difference between the luminance levels of the images between the two fields. Since it appears prominently, motion vector detection can be performed correctly.
[0086]
Next, an embodiment in which the characteristics of the prefilter are switched using the SN representing the SN ratio, the detection vector V (t-1) of the previous field, the global vector Vg of the previous field, and the information of the moving area and the stationary area. As shown in FIG. In the figure, the detection region static motion determination unit 150 has the block configuration shown in FIG. Further, the pre-filters A to E (151A to 151E) have the configurations shown in FIGS. 6, 9, 11, 12, and 14, respectively. The global vector detection unit 156 has the block configuration shown in FIG.
[0087]
As described above, the main object of the present invention is to adaptively switch the prefilter according to the static state of the detection region. In the conventional method for determining the static motion by comparing the absolute value of the difference value of the luminance level between the images separated by one frame with a certain threshold, or the method as shown in FIG. 4 used in the present invention, Since the correlation of the luminance level between the two images is indefinite, it is not possible to make a static determination with a probability of 100%.
[0088]
Therefore, in one embodiment of the present invention, the accuracy of static motion determination is improved by using the static motion determination shown in FIG. 4 and the motion vector detected in the previous field. In one embodiment of the present invention, the condition for determining a still region is that the motion detection in FIG. 4 determines that the region is a still region, and that the detection vector V (t−1) in the previous field is zero.
[0089]
First, there can be mainly two cases of the static region. One is that a still area of an image captured with the camera fixed and an image of the moving object in an image captured by moving the camera to track the moving object are substantially stationary areas. These determinations are made using the global vector Vg. In the static region, the pre-filter A is used to avoid the influence of interlaced scanning. Further, in order to avoid the influence of noise in the still region, if the SN ratio is poor due to the value of the SN ratio, the pre-filter C is used, and the static motion determination is performed in the still region of the image taken by moving the camera. In consideration of accuracy, the pre-filter B having a small filter characteristic in the time direction is used.
[0090]
Next, in the moving region and the region determined to be a moving region with V (t−1) ≠ 0, when the block matching method is used as a motion vector detection method, a prefilter having no time direction filter characteristic is used. Use E. When the gradient method is used as a motion vector detection method, blur (brightness level gradient) due to movement of the moving object must be present in the moving object image. When an image of a CG, a camera with a shutter, or an F / V converted image is input, there is little blur in the moving body image of the input image.
[0091]
Therefore, regardless of the type of input image (an image with little blur such as CG, an image shot with a camera with a camera tube with a large blur of moving images), the moving area of the moving object image shot with the camera fixed is used. On the other hand, using the output of the pre-filter D having the filter characteristic in the time direction, the moving area of the image taken by moving the camera is moved to the moving area without applying a spatio-temporal filter and creating a pseudo gradient. Since there is blur, the pre-filter E is used.
[0092]
FIG. 16 shows an embodiment of switching the prefilter when the block matching method is used as the vector detection method.
[0093]
FIG. 17 shows an embodiment of switching the prefilter when the gradient method is used as the vector detection method.
[0094]
When the block matching method and the gradient method are switched and used as the vector detection method, the contents of the switching information decoder 152 in FIG. 15 may be switched according to FIGS. 16 and 17.
[0095]
【The invention's effect】
As described above, according to the present invention, the static motion state and the S / N ratio are determined for each small image area of the filter input signal, which is a filtering processing unit, and the characteristics and movements of the area are selected from a plurality of filters having different characteristics. Since the pre-filter that is optimal for the combination with the vector detection method is selected, optimal filtering processing for each region is possible, and motion vector detection with improved motion vector detection accuracy is possible.
[0096]
In other words, in the present invention, a plurality of filters having different characteristics for outputting vector detection luminance signals are prepared, and a filter for outputting vector detection luminance signals optimal for the image characteristics and detection method of the detection target area is switched for each detection target area. Therefore, it is possible to prevent erroneous motion vector detection. Thus, by carrying out the present invention, an erroneous motion vector is not detected, so that it is possible to obtain a special effect that the motion vector detection accuracy can be improved.
[Brief description of the drawings]
1A is an explanatory diagram showing the positional relationship of scanning lines in the vertical direction of interlaced scanning, FIG. 1B is an explanatory diagram showing an example of output of a conventional prefilter, and FIG. 1C is an interlaced scanning line. It is explanatory drawing which shows a spatial position.
FIG. 2A is an explanatory diagram showing the positional relationship of scanning lines in the vertical direction of interlaced scanning, and FIG. 2B is an explanatory diagram showing an output example of a conventional pre-filter.
FIG. 3 is a block diagram showing an embodiment of a conventional prefilter circuit.
FIG. 4 is a block diagram showing an embodiment of a static determination circuit of the present invention.
FIG. 5 is a block diagram showing an embodiment of global vector detection according to the present invention.
FIG. 6 is a block diagram showing an embodiment of the circuit of the prefilter A according to the present invention.
FIG. 7 is a block diagram showing an embodiment of an LPF circuit of frame → field subsampling included in the pre-filters A, B, and C of the present invention.
FIG. 8 is an explanatory diagram showing an operation example of the LPF circuit of frame → field subsampling included in the pre-filters A, B, and C of the present invention.
FIG. 9 is a block diagram showing an embodiment of the circuit of the prefilter B according to the present invention.
FIG. 10 is an explanatory diagram showing an operation example of the circuit of the pre-filter B of the present invention.
FIG. 11 is a block diagram showing an embodiment of the circuit of the prefilter C according to the present invention.
FIG. 12 is a block diagram showing an embodiment of the circuit of the prefilter D of the present invention.
FIG. 13 is a block diagram showing an embodiment of a spatio-temporal filter circuit included in the pre-filter D of the present invention.
FIG. 14 is a block diagram showing an embodiment of the circuit of the prefilter E according to the present invention.
FIG. 15 is a block diagram showing an embodiment of a prefilter switching circuit according to the present invention.
FIG. 16 is a diagram showing an embodiment of switching in a decoder corresponding to the pre-filter block matching method of the present invention.
FIG. 17 is a diagram showing an embodiment of switching in a decoder corresponding to the prefilter gradient method of the present invention.
[Explanation of symbols]
150 Detection region static motion determination unit
151A to 151E Prefilter A to Prefilter E
152 switching information decoder
153 Vector signal luminance signal selector
154 Motion vector detection unit
155 Motion vector value memory
156 Global vector detector
157 SN ratio detector

Claims (10)

As a pre-filter connected to the previous stage of the motion vector detection device for images displayed by the interlace scanning method, a plurality of filters having different characteristics for outputting vector detection luminance signals are prepared, and the image in the detection target area of the input moving image When performing motion vector detection by switching and using a filter that outputs a vector detection luminance signal suitable for the characteristics and vector detection method for each detection target region,
As a filter having different characteristics,
A frame signal formed from a first field signal and a second field signal, and a frame formed from a second field signal and a third field signal, using a continuous 3: 1 2: 1 interlaced image signal as an input signal Each of the signals is subjected to a spatial low-pass filter to generate two sequential scanning signals, and then the scanning lines are thinned out for each of the two sequential scanning signals to match the odd-even field format. A filter A for outputting a field signal;
Field obtained by adding and averaging the field signal obtained by averaging the first and third field signals and the second and fourth field signals by using the continuous 5: 1 2: 1 interlaced image signal as an input signal A frame signal formed from the signal and a field signal obtained by averaging the second and fourth field signals and a field signal obtained by averaging the third and fifth field signals. Two field signals in which the odd-even field format is matched by thinning out the scanning line for each line for each of the two sequential scanning signals after generating a spatial scanning low pass filter for each of the two sequential scanning signals. A filter B that outputs
A cyclic filter that adds a signal obtained by delaying an output signal by one frame to a 2: 1 interlaced image signal that is an input signal at an arbitrary coefficient ratio, and 2: 1 of continuous 3 fields that are the output of the cyclic filter. Using an interlaced image signal as an input signal, a spatial low-pass filter is applied to each of the frame signal formed from the first field signal and the second field signal and the frame signal formed from the second field signal and the third field signal. To generate two sequential scanning signals, and then subordinately connect a filter that outputs two field signals in which the scanning lines are thinned out for each line to match the odd / even field format for each of the two sequential scanning signals. Filter C,
A time low-pass filter that uses a continuous 4-field 2: 1 interlaced image signal as an input signal, adds a continuous 3-field signal and each one-line delayed output signal at an arbitrary coefficient ratio, and sequentially outputs a scanning signal; A pair of filters formed by cascade-connecting spatial low-pass filters for converting progressive scanning signals into field signals is provided, and three consecutive field input signals of the pair of filters are first, second, and third field signals. Alternatively, the second, third, and fourth field signals, and the pair of filter outputs includes a filter D that is a 2: 1 interlaced image signal of two consecutive fields, and
There use 1 and a filter E, which is output as an interlaced image signal,: a spatial low-pass filter output, the 1-field delay output of spatial low-pass filter output, two consecutive fields 2
If it is determined that there is a global motion in the entire previous field image based on the motion vector detection result of the previous field, which is the output of the motion vector detection device, the global motion level is determined as a global vector. A global vector detection step that outputs a detection value and outputs a global vector non-detection signal when it is determined that there is no global movement;
A static motion determination step for outputting a determination result of whether the region is a dynamic region or a static region for each detection region which is a filter processing unit;
An SN ratio determining step of determining an SN ratio of the entire input image for each field and outputting a determination result as to whether the input signal is a low SN area or a high SN area;
The global vector detection result, the motion vector detection result of the previous field for each detection region, the movement determination result and on the basis of the SN ratio determination result, switching information output for outputting the toggle information for each of the detection area Steps,
A filter switching step of switching filters having different characteristics based on switching information for each detection region and outputting one filter processing result to the motion vector detection device;
With
The switching information output step
When the detection area is a static area, a high SN area, and the motion vector detection value of the previous field in the detection area is “0”, the filter A is
The detection area is a static area, a low SN area, and a detection value of a global vector other than “0” and “0” in the previous field in the detection area or a non-detection signal of a global vector is output. Filter B if
When the detection region is a static region, a low SN region, and the motion vector detection value of the previous field in the detection region is “0” and the detection value of the global vector is “0”, the filter C is
When the detection area is a static area and the motion vector detection value of the previous field in the detection area is other than “0”, or when the detection area is a moving area, the filter E is
Output the switching information to select,
A motion vector detection method characterized by the above . As a pre-filter connected to the previous stage of the motion vector detection device for images displayed by the interlace scanning method, a plurality of filters having different characteristics for outputting vector detection luminance signals are prepared, and the image in the detection target area of the input moving image When performing motion vector detection by switching and using a filter that outputs a vector detection luminance signal suitable for the characteristics and vector detection method for each detection target region,
As a filter having different characteristics ,
A frame signal formed from a first field signal and a second field signal, and a frame formed from a second field signal and a third field signal, using a continuous 3: 1 2: 1 interlaced image signal as an input signal Each of the signals is subjected to a spatial low-pass filter to generate two sequential scanning signals, and then the scanning lines are thinned out for each of the two sequential scanning signals to match the odd-even field format. A filter A for outputting a field signal;
Field obtained by adding and averaging the field signal obtained by averaging the first and third field signals and the second and fourth field signals by using the continuous 5: 1 2: 1 interlaced image signal as an input signal A frame signal formed from the signal and a field signal obtained by averaging the second and fourth field signals and a field signal obtained by averaging the third and fifth field signals. Two field signals in which the odd-even field format is matched by thinning out the scanning line for each line for each of the two sequential scanning signals after generating a spatial scanning low pass filter for each of the two sequential scanning signals. A filter B that outputs
A cyclic filter that adds a signal obtained by delaying an output signal by one frame to a 2: 1 interlaced image signal that is an input signal at an arbitrary coefficient ratio, and 2: 1 of continuous 3 fields that are the output of the cyclic filter. Using an interlaced image signal as an input signal, a spatial low-pass filter is applied to each of the frame signal formed from the first field signal and the second field signal and the frame signal formed from the second field signal and the third field signal. To generate two sequential scanning signals, and then subordinately connect a filter that outputs two field signals in which the scanning lines are thinned out for each line to match the odd / even field format for each of the two sequential scanning signals. Filter C,
A time low-pass filter that uses a continuous 4-field 2: 1 interlaced image signal as an input signal, adds a continuous 3-field signal and each one-line delayed output signal at an arbitrary coefficient ratio, and sequentially outputs a scanning signal; A pair of filters formed by cascade-connecting spatial low-pass filters for converting progressive scanning signals into field signals is provided, and three consecutive field input signals of the pair of filters are first, second, and third field signals. Alternatively, the second, third, and fourth field signals, and the pair of filter outputs includes a filter D that is a 2: 1 interlaced image signal of two consecutive fields, and
Using a spatial low-pass filter output and a filter E that outputs a 1-field delayed output of the spatial low-pass filter output as a continuous 2-field 2: 1 interlaced image signal ,
If it is determined that there is a global motion in the entire previous field image based on the motion vector detection result of the previous field, which is the output of the motion vector detection device, the global motion level is determined as a global vector. A global vector detection step that outputs a detection value and outputs a global vector non-detection signal when it is determined that there is no global movement;
A static motion determination step for outputting a determination result of whether the region is a dynamic region or a static region for each detection region which is a filter processing unit;
An SN ratio determining step of determining an SN ratio of the entire input image for each field and outputting a determination result as to whether the input signal is a low SN area or a high SN area;
A switching information output step for outputting switching information for each detection region based on the global vector detection result, the motion vector detection result of the immediately preceding field for each detection region, the static motion determination result, and the SN ratio determination result; ,
A filter switching step of switching filters having different characteristics based on switching information for each detection region and outputting one filter processing result to the motion vector detection device;
With
The switching information output step
When the detection region is a static region, a high SN region, and the motion vector detection value of the previous field in the detection region is “0”, the filter A is
The detection area is a static area, a low SN area, and a motion vector detection value of the previous field in the detection area is “0” and a global vector detection value other than “0” or a global vector non-detection signal is output. If so, filter B
When the detection region is a static region, a low SN region, and the motion vector detection value of the previous field in the detection region is “0” and the detection value of the global vector is “0”, the filter C is
When the detection area is a static area, the motion vector detection value of the previous field in the detection area is other than “0”, the detection value of the global vector is “0”, or a global vector non-detection signal is output Filter D
When the detection region is a static region and a detection value of a global vector other than “0” and a motion vector detection value of the previous field in the detection region is output, or the detection region is a moving region If a detected value of a global vector other than “0” is output, filter E is
When the detection area is a dynamic area and the detection value of the global vector is “0” or the non-detection signal of the global vector is output, the filter D is
Output the switching information to select,
A motion vector detection method characterized by the above . In claim 1 or 2,
The global vector detection step includes:
By comparing the motion vectors of the immediately preceding field for each detection area with each other within an allowable error range, a plurality of vector values having a large detection ratio and its detection ratio are specified, and among the plurality of vector values, When the sum of the detection ratios of vector values within a certain range exceeds a certain value, the value of the most detected vector is output as the detected value of the global vector, and when it falls below a certain value, the non-detected signal of the global vector Output,
A motion vector detection method characterized by the above. In claim 1 or 2,
The static motion determination step includes
Among 5 consecutive 2: 1 interlaced image signals,
Between the first field signal and the third field signal, or between the third field signal and the fifth field signal, between the pixels occupying the same spatial position in both,
Between the second field signal and the third field signal, or between the third field signal and the fourth field signal, the pixels in the temporally previous field and the positions corresponding to the spatial positions of the pixels are Between pixels in a temporally later field located anywhere,
The quotient obtained by dividing the absolute value of the difference in luminance signal level by the larger value of the absolute value of the horizontal gradient and the vertical gradient of the pixel in the previous field in time is a constant value in all the above four cases. Only when it exceeds, it is determined as a moving pixel,
Only when the number of moving pixels in the detection area exceeds a certain value, the area is determined as a moving area.
A motion vector detection method characterized by the above. In claim 1 or 2,
The S / N ratio determining step includes:
When the ratio of the detection area determined as the static area by the static motion determination means is below a certain value and the detection value of the global vector is “0”, it is determined as the high SN area, and the detection area If the ratio is below a certain value and has a detected value of a global vector other than “0”, or if a non-detected signal of a global vector is output, it is determined as a low SN region,
When the ratio of the detection areas determined as still areas by the static motion determination means exceeds a certain value, all the pixels in one detection frame determined as the still area are compared with all the pixels after one frame. The absolute value of the difference of the luminance signal level between each pixel is obtained, and when the total average of these is less than a certain value, it is determined as a high SN region, and when it exceeds a certain value, it is determined as a low SN region.
A motion vector detection method characterized by the above. In the pre-filter connected to the front stage of the motion vector detection device of the image displayed by the interlace scanning method,
As a filter with different characteristics to which each input image signal is input,
A frame signal formed from a first field signal and a second field signal, and a frame formed from a second field signal and a third field signal, using a continuous 3: 1 2: 1 interlaced image signal as an input signal Each of the signals is subjected to a spatial low-pass filter to generate two sequential scanning signals, and then the scanning lines are thinned out for each of the two sequential scanning signals to match the odd-even field format. A filter A for outputting a field signal;
Field obtained by adding and averaging the field signal obtained by averaging the first and third field signals and the second and fourth field signals by using the continuous 5: 1 2: 1 interlaced image signal as an input signal A frame signal formed from the signal and a field signal obtained by averaging the second and fourth field signals and a field signal obtained by averaging the third and fifth field signals. Two field signals in which the odd-even field format is matched by thinning out the scanning line for each line for each of the two sequential scanning signals after generating a spatial scanning low pass filter for each of the two sequential scanning signals. A filter B that outputs
A cyclic filter that adds a signal obtained by delaying an output signal by one frame to a 2: 1 interlaced image signal that is an input signal at an arbitrary coefficient ratio, and 2: 1 of continuous 3 fields that are the output of the cyclic filter. Using an interlaced image signal as an input signal, a spatial low-pass filter is applied to each of the frame signal formed from the first field signal and the second field signal and the frame signal formed from the second field signal and the third field signal. To generate two sequential scanning signals, and then subordinately connect a filter that outputs two field signals in which the scanning lines are thinned out for each line to match the odd / even field format for each of the two sequential scanning signals. Filter C,
A time low-pass filter that uses a continuous 4-field 2: 1 interlaced image signal as an input signal, adds a continuous 3-field signal and each one-line delayed output signal at an arbitrary coefficient ratio, and sequentially outputs a scanning signal; A pair of filters formed by cascade-connecting spatial low-pass filters for converting progressive scanning signals into field signals is provided, and three consecutive field input signals of the pair of filters are first, second, and third field signals. Alternatively, the second, third, and fourth field signals, and the pair of filter outputs includes a filter D that is a 2: 1 interlaced image signal of two consecutive fields, and
Two or more filters including a spatial low-pass filter output, and a filter E that outputs a one-field delayed output of the spatial low-pass filter output as a continuous two-field 2: 1 interlaced image signal;
A filter switching unit that switches the two or more filters and outputs one filter processing result to the motion vector detection device;
If it is determined that there is a global motion in the entire previous field image based on the motion vector detection result of the previous field, which is the output of the motion vector detection device, the global motion level is determined as a global vector. A global vector detection means that outputs a detection value and outputs a global vector non-detection signal when it is determined that there is no global movement;
Static motion determination means for outputting a determination result of whether the region is a dynamic region or a static region for each detection region which is a filter processing unit;
SN ratio determination means for determining the SN ratio of the entire input image for each field and outputting a determination result as to whether the input signal is a low SN area or a high SN area;
Based on the global vector detection result, the motion vector detection result of the immediately preceding field for each detection region, the static motion determination result, and the SN ratio determination result, switching information for each detection region is output to the filter switching unit. A switching information decoder ;
Equipped with a,
The switching information decoder
When the detection area is a static area, a high SN area, and the motion vector detection value of the previous field in the detection area is “0”, the filter A is
The detection area is a static area, a low SN area, and a detection value of a global vector other than “0” and “0” in the previous field in the detection area or a non-detection signal of a global vector is output. Filter B if
When the detection region is a static region, a low SN region, and the motion vector detection value of the previous field in the detection region is “0” and the detection value of the global vector is “0”, the filter C is
When the detection area is a static area and the motion vector detection value of the previous field in the detection area is other than “0”, or when the detection area is a moving area, the filter E is
Output the switching information to select,
An adaptive switching pre-filter for motion vector detection , characterized in that In the pre-filter connected to the front stage of the motion vector detection device of the image displayed by the interlace scanning method,
As a filter with different characteristics to which each input image signal is input,
A frame signal formed from a first field signal and a second field signal, and a frame formed from a second field signal and a third field signal, using a continuous 3: 1 2: 1 interlaced image signal as an input signal Each of the signals is subjected to a spatial low-pass filter to generate two sequential scanning signals, and then the scanning lines are thinned out for each of the two sequential scanning signals to match the odd-even field format. A filter A for outputting a field signal;
Of 5 consecutive fields 2: 1 interlaced image signal of the input signal, obtained by a field signal obtained by the first and third field signals are averaged and the second and fourth field signal averaging A frame signal formed from the field signal obtained by averaging the frame signal formed from the field signal and the second and fourth field signals and the field signal obtained by averaging the third and fifth field signals. Two spatial scan filters are applied to each of the two sequential scanning signals to generate two sequential scanning signals, and then the scanning fields are thinned out for each of the two sequential scanning signals to thereby match the odd-even field format. A filter B for outputting a signal;
A cyclic filter that adds a signal obtained by delaying an output signal by one frame to a 2: 1 interlaced image signal that is an input signal at an arbitrary coefficient ratio, and 2: 1 of continuous 3 fields that are the output of the cyclic filter. Using an interlaced image signal as an input signal, a spatial low-pass filter is applied to each of the frame signal formed from the first field signal and the second field signal and the frame signal formed from the second field signal and the third field signal. To generate two sequential scanning signals, and then subordinately connect a filter that outputs two field signals in which the scanning lines are thinned out for each line to match the odd / even field format for each of the two sequential scanning signals. Filter C,
A time low-pass filter that uses a continuous 4-field 2: 1 interlaced image signal as an input signal, adds a continuous 3-field signal and each one-line delayed output signal at an arbitrary coefficient ratio, and sequentially outputs a scanning signal; A pair of filters formed by cascade-connecting spatial low-pass filters for converting progressive scanning signals into field signals is provided, and three consecutive field input signals of the pair of filters are first, second, and third field signals. Alternatively, the second, third, and fourth field signals, and the pair of filter outputs includes a filter D that is a 2: 1 interlaced image signal of two consecutive fields, and
A spatial low-pass filter output, and a filter E that outputs a 1-field delayed output of the spatial low-pass filter output as a continuous 2-field 2: 1 interlaced image signal;
Two or more filters including
A filter switching unit that switches the two or more filters and outputs one filter processing result to the motion vector detection device;
If it is determined that there is a global motion in the entire previous field image based on the motion vector detection result of the previous field, which is the output of the motion vector detection device, the global motion level is determined as a global vector. A global vector detection means that outputs a detection value and outputs a global vector non-detection signal when it is determined that there is no global movement;
Static motion determination means for outputting a determination result of whether the region is a dynamic region or a static region for each detection region which is a filter processing unit;
SN ratio determination means for determining the SN ratio of the entire input image for each field and outputting a determination result as to whether the input signal is a low SN area or a high SN area;
Based on the global vector detection result, the motion vector detection result of the immediately preceding field for each detection region, the static motion determination result, and the SN ratio determination result, switching information for each detection region is output to the filter switching unit. A switching information decoder;
With
The switching information decoder
When the detection region is a static region, a high SN region, and the motion vector detection value of the previous field in the detection region is “0”, the filter A is
The detection area is a static area, a low SN area, and a motion vector detection value of the previous field in the detection area is “0” and a global vector detection value other than “0” or a global vector non-detection signal is output. If so, filter B
When the detection region is a static region, a low SN region, and the motion vector detection value of the previous field in the detection region is “0” and the detection value of the global vector is “0”, the filter C is
When the detection area is a static area, the motion vector detection value of the previous field in the detection area is other than “0”, the detection value of the global vector is “0”, or a global vector non-detection signal is output Filter D
When the detection region is a static region and a detection value of a global vector other than “0” and a motion vector detection value of the previous field in the detection region is output, or the detection region is a moving region If a detected value of a global vector other than “0” is output, filter E is
When the detection area is a dynamic area and the detection value of the global vector is “0” or the non-detection signal of the global vector is output, the filter D is
Output the switching information to select ,
An adaptive switching pre-filter for motion vector detection , characterized in that In claim 6 or 7,
The global vector detection means includes
By comparing the motion vectors of the immediately preceding field for each detection area with each other within an allowable error range, a plurality of vector values having a large detection ratio and its detection ratio are specified, and among the plurality of vector values, When the sum of the detection ratios of vector values within a certain range exceeds a certain value, the value of the most detected vector is output as the detected value of the global vector, and when it falls below a certain value, the non-detected signal of the global vector Output ,
An adaptive switching pre-filter for motion vector detection , characterized in that In claim 6 or 7,
The static motion determination means includes
Among 5 consecutive 2: 1 interlaced image signals,
Between the first field signal and the third field signal, or between the third field signal and the fifth field signal, between the pixels occupying the same spatial position in both,
Between the second field signal and the third field signal, or between the third field signal and the fourth field signal, the pixels in the temporally previous field and the positions corresponding to the spatial positions of the pixels are Between pixels in a temporally later field located anywhere,
The quotient obtained by dividing the absolute value of the difference in luminance signal level by the larger value of the absolute value of the horizontal gradient and the vertical gradient of the pixel in the previous field in time is a constant value in all the above four cases. Only when it exceeds, it is determined as a moving pixel,
Only when the number of moving pixels in the detection area exceeds a certain value, the area is determined as a moving area .
An adaptive switching pre-filter for motion vector detection, characterized in that In claim 6 or 7,
The SN ratio determining means is
When the ratio of the detection area determined as the static area by the static motion determination means is below a certain value and the detection value of the global vector is “0”, it is determined as the high SN area, and the detection area If the ratio is below a certain value and has a detected value of a global vector other than “0”, or if a non-detected signal of a global vector is output, it is determined as a low SN region,
When the ratio of the detection areas determined as still areas by the static motion determination means exceeds a certain value, all the pixels in one detection frame determined as the still area are compared with all the pixels after one frame. The absolute value of the difference of the luminance signal level between each pixel is obtained, and when the total average of these is less than a certain value, it is determined as a high SN region, and when it exceeds a certain value, it is determined as a low SN region .
An adaptive switching pre-filter for motion vector detection, characterized in that

Images