JP2004208087A - Movement detecting method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method capable of conducting a highly precise movement detection by a small amount of computation, even if a subject moves at a high speed and at random on a screen. <P>SOLUTION: A processing block for conducting next movement detection smoothes over motion vectors from processing blocks which have completed movement detection in the past, and obtains an expected vector (S204). An evaluated value is obtained from the expected vector and an weighting factor (S208). By conducting movement detection using the evaluated value, it can be attained to enable the highly accurate movement detection with the small amount of computation, even if an object moves in high speed on a screen. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

但し、ｋは今回動き検出を行う間引き処理ブロックの番号を示し、Ｖｅｐｘｕ［ｋ−１］は前回の間引き処理ブロックにおける単位期待ベクトルであり、Ｖｕ［ｋ−１］は前回の間引き処理ブロックにおける動きベクトルを１フィールド当たりに換算した単位動きベクトルである。また、係数ａ及び係数ｂはａ＋ｂ＝１の関係を持つ定数である。つまり、単位期待ベクトルＶｅｘｐｕは過去に動き検出を行った間引き処理ブロックにおける単位動きベクトルＶｕを平滑化したものである。係数ａと係数ｂは、今回の間引き処理ブロックから遠い過去の単位動きベクトルの影響を受け過ぎず、またランダムに動いている様な特異的な過去の単位動きベクトルの影響を受けないように、統計的に決定される値である。
【００３４】
期待ベクトル算出手段７は間引き処理ブロックに関して、参照フレームから処理フレームへ移動した画面距離を推定する期待ベクトルＶｅｘｐを（式２）の様に算出する。
【００３５】
【数２】

但し、ＤＩＳＴは参照フレームと処理フレームとの時間距離をフィールド数で表したものである。
【００３６】
次に、減算器８は減算器３からの動きベクトルと期待ベクトル算出手段からの期待ベクトルＶｅｘｐとを減算することで、今回の間引き候補ブロックが期待ベクトルからどれだけ離れているかを示すベクトルオフセットを算出し、スイッチ９を介して重み付け手段１７へ供給する（Ｓ２０５）。
【００３７】
一方、処理ブロック生成手段１で生成した間引き処理ブロックは１画素ずつ出力され、スイッチ１０を介して減算器１２へ供給される。また、候補ブロック生成手段２で生成した間引き候補ブロックも処理ブロック生成手段１から出力される間引き処理ブロックの画素に対応して１画素ずつ出力され、スイッチ１１を介して減算器１２へ供給される。減算器１２は間引き候補ブロックの画素から間引き処理ブロックの画素を減算し、ＡＢＳ演算器１３で絶対値演算を行う。ＡＢＳ演算器１３の出力は加算器１４においてメモリ１５の出力と加算され、再びメモリ１５へ格納される。以上をｍ／２×ｎ／２画素全てについて繰り返すことによりメモリ１５には間引き候補ブロックと間引き処理ブロックとの差分絶対値総和ＤＭＣが格納される（Ｓ２０６）。以上の処理は（式３）のように表わすことができる。
【００３８】
【数３】

ここで、（ｘ，ｙ）は間引き候補ブロックまたは間引き処理ブロックの先頭画素からの相対的な画素位置を、Ｙｒ（ｘ，ｙ）は間引き候補ブロックにおける画素位置（ｘ，ｙ）の輝度データを、Ｙｔ（ｘ，ｙ）は間引き処理ブロックにおける画素位置（ｘ，ｙ）の輝度データを表している。
【００３９】
次に、重み付け手段１７はスイッチ９を介して減算器８から入力したベクトルオフセットに対応した重み係数ｋを重みテーブル１６から入力し（Ｓ２０７）、メモリ１５の差分絶対値総和ＤＭＣに乗算して、今回ブロックマッチングを行った間引き処理ブロックと間引き候補ブロックとにおける評価値ＥＳＴを算出する（Ｓ２０８）。以上の処理は（式４）のように表わすことができる。
【００４０】
【数４】

次に、比較器１８ではメモリ１９に格納されている前回までの暫定最小評価値を入力し、今回の評価値ＥＳＴと比較評価する（Ｓ２０９）。今回の評価値ＥＳＴの方が小さいと判断した場合には今回の評価値を暫定最小評価値としてメモリ１９を更新する（Ｓ２１０）と共に、候補ブロック生成手段２とメモリ４へ今回の評価値が最小であったことを伝えるＭＩＮ信号を出力する（Ｓ２１１）。ここで、重みテーブル１６は重み係数ｋを配列したＲＯＭテーブルで構成されている。
【００４１】
このＲＯＭテーブルについて図４を用いて説明する。
図４（ａ）において、横軸はベクトルオフセットのｘ成分、縦軸はベクトルオフセットのｙ成分であり、それぞれ０を中心に±１６の値について示している。ベクトルオフセットのｘ成分とｙ成分が交差する所の数値が重み係数ｋとなる。
【００４２】
また、図４（ｂ）は図４（ａ）の重みテーブルにおいてベクトルオフセットのｙ成分が０の場合の重み係数ｋをグラフ化したものであり、横軸がベクトルオフセットのｘ成分、縦軸が重み係数ｋの値である。図４より、ベクトルオフセットが（０，０）の位置に近い程に重み係数ｋは小さい数値であるのがわかる。ベクトルオフセットが（０，０）の位置に近いということは、今回の動きベクトルの値が期待ベクトルに近い値であることを意味しており、つまり、動きベクトルが期待ベクトルに近くなる程に、差分絶対値総和ＤＭＣに乗じる重み係数ｋは小さい値になり、結果的に評価値ＥＳＴは小さい値をとりやすくなる。
【００４３】
ここで、図５または図６は重みテーブル１６について別の一例を示したものであり、図５においてはベクトルオフセットが一定以上に大きくなった時に重み係数ｋを設定値Ｃに固定した例、図６においてはベクトルオフセットが（０，０）から距離Ｄの範囲内において設定値をＡに固定した例である。
つまり、図５の例は、動きベクトルが期待ベクトルから懸け離れた場合には、動き検出中の処理ブロックが他の処理ブロックとは違った動きをしている可能性が大きく、動きベクトルがこの範囲に入った評価値同士においては係数ｋを一定にすることによって期待ベクトルとの一致度とは無関係に公平に評価しようというものである。
【００４４】
また、図６の例は、動きベクトルが期待ベクトルに非常に近い値の場合には、期待ベクトルの信頼性による悪影響を回避するため、動きベクトルがこの範囲に入った評価値同士においては重み係数ｋを一定にすることによって期待ベクトルとの一致度とは無関係に公平に評価しようというものである。設定値Ｃと設定値Ａは固定値でなくてもよく、傾きＢよりも緩い傾きを持たせた値にすれば、その目的を達成できる。また、重み付け手段１７は重み係数ｋを差分絶対値総和ＤＭＣに乗じることで、期待ベクトルの値に近い動きベクトルを採用し易くする訳だが、実際の画像では画面一様に移動せずに、一部の小さい被写体のみがランダムに移動している場合もある。この場合には正しい動きベクトルは期待ベクトルとは一致しなくなる。しかし、正しい動きベクトルの時の差分絶対値総和ＤＭＣは他と比較して非常に小さい値をとるため、重み係数ｋを乗じた場合でも評価値ＥＳＴは大抵は最小となり、精度の高い結果を得ることができる。
【００４５】
図２において、候補ブロック生成手段２は比較器１８からＭＩＮ信号を受け取ると、現在の間引き候補ブロックを動き検出の有力候補とする。メモリ４はＭＩＮ信号を受け取ると、減算器３から出力されている現在の動きベクトルを一時的に記憶する。
【００４６】
候補ブロック生成手段２は参照範囲内において１画素ずつずらしながら間引き候補ブロックを順次出力し、上記処理（Ｓ２０２〜Ｓ２１１）を繰り返すことにより、メモリ１９には参照範囲内における最小評価値が格納される。
【００４７】
参照範囲内の全ての間引き候補ブロックを処理すると外部よりＥＮＤ信号が入力される（Ｓ２１２）。比較器２０はＥＮＤ信号を入力すると、メモリ１９に格納されている参照範囲内における最小評価値ＥＳＴ［ｋ］を入力し、メモリ２２に格納されている相場評価値ＥＳＴｐｒ［ｋ］と比較する。最小評価値ＥＳＴ［ｋ］が相場評価値ＥＳＴｐｒ［ｋ］以下の場合には、比較器２０は単位期待ベクトル算出手段５へ単位期待ベクトルＶｅｘｐｕを更新するように要求するＣＡＬＣ信号を出力する。また、比較器２０は相場評価値算出手段２１へ最小評価値ＥＳＴ［ｋ］を出力する。相場評価値算出手段２１は、最小評価値ＥＳＴ［ｋ］とメモリ２２からの相場評価値ＥＳＴｐｒ［ｋ］とから、次回の動き検出時に使用する相場評価値ＥＳＴｐｒ［ｋ＋１］を算出する（Ｓ２１３）。この処理は（式５）の様に表わすことができる。
【００４８】
【数５】

ここで、相場評価値ＥＳＴｐｒとは、過去に動き検出を行った間引き処理ブロックにおける最小評価値の相場を示した指標であり、（式５）において、係数ｃ及び係数ｄはｃ＋ｄ＝１の関係を持つ定数である。つまり、相場評価値ＥＳＴｐｒは過去に動き検出を行った間引き処理ブロックにおける最小評価値ＥＳＴを平滑化したものである。係数ｃと係数ｄは、今回の間引き処理ブロックから遠い過去の最小評価値ＥＳＴの影響を受け過ぎず、また最適な間引き候補ブロックがなく特異的に大きくなった過去の最小評価値ＥＳＴの影響を受けないように、統計的に決定される値である。
【００４９】
単位期待ベクトル算出手段５は比較器２０からＣＡＬＣ信号を入力すると（Ｓ２１４）、最小評価値を算出した時の動きベクトルをメモリ４から入力し、単位動きベクトルＶｕ［ｋ］を求め、（式１）に従って次回の動き検出時に使用する単位期待ベクトルＶｅｘｐｕ［ｋ＋１］を算出し、メモリ６に記憶しておく（Ｓ２１５）。つまり、間引き処理ブロックの最小評価値ＥＳＴが相場評価値ＥＳＴｐｒ以下の場合に限って、今回の動きベクトルを加味した単位期待ベクトルＶｅｘｐｕに更新する。間引き処理ブロックの最小評価値ＥＳＴが相場評価値ＥＳＴｐｒ以下の場合とは、今回のブロックマッチング結果が信頼性の高いものであることを意味している。
【００５０】
また、動き補償予測方式には、時間的に異なる第１フィールドと第２フィールドとを合成したデータから処理ブロック及び候補ブロックを生成して動き検出を行うフレーム予測モードと、第１フィールドのみまたは第２フィールドのみのデータから処理ブロックおよび候補ブロックを生成して動き検出を行う第１フィールド予測モードまたは第２フィールド予測モードがある。ちなみに、間引き処理ブロックまたは間引き候補ブロックの生成は予測モードに合わせて、間引き方法が切り換えられる。また、各予測モードにおいても、処理フレームに対して時間的に古いフレームを参照フレームとする前方向参照モードと、時間的に新しいフレームを参照フレームとする後方向参照モードとがある。以上の様に３つの予測モードにおいて各々２つの参照モードがあるため、組み合わせとしては計６種類のモードが存在することになるが、前述したメモリ６の単位期待ベクトルと、メモリ２２の相場評価値は６種類のモード毎に区別して格納される。つまり、単位期待ベクトル算出手段５は現在のモードの単位期待ベクトルをメモリ６から読み出して新たな単位期待ベクトルを算出し、メモリ６に格納している現在のモードの単位期待ベクトルのみを更新する。同様に、期待ベクトル算出手段７も現在のモードの単位期待ベクトルをメモリ６から読出し、期待ベクトルを算出する。比較器２０、相場評価値算出手段２１においても、同様に現在のモードの相場評価値によって処理を行う。これは、処理ブロックと候補ブロックの構成方法や時間方向が各モードによって異なり、単位期待ベクトルや相場評価値の値に差がでるために別々に処理を行うものである。これにより、精度の高い動き検出が可能となる。また、参照モードには処理フレームに対して時間的に古いフレームと時間的に新しいフレームの２つを参照フレームとする両方向参照モードがあるが、これは前方参照モードと後方参照モードの両データを使用することで対応できる。
【００５１】
以上で、広い参照範囲内における間引きブロックマッチングが終了したことになり、引き続き、最小評価値を導き出した候補ブロックを中心とする狭い参照範囲内において全画素を用いたブロックマッチングを開始する。ここで、最小評価値を導き出した候補ブロックの位置は間引き候補ブロックのｘ座標とｙ座標を各々２倍することで容易に換算できる。
【００５２】
全画素を用いたブロックマッチングの開始時点では、スイッチ１０及びスイッチ１１は端子ｗ側のままであり、ＯＵＴ信号は解除されているためスイッチ２３はＯＦＦ動作をしている。また、全画素を用いたブロックマッチングの実行中においてはＷＥＩＧＨＴ信号は常に解除されているため、スイッチ９はＯＦＦ動作のままとなり、単位期待ベクトル算出手段５、メモリ６、期待ベクトル算出手段７、減算器８、比較器２０、相場評価値算出手段２１及びメモリ２２での処理は何ら影響を及ぼさない。
【００５３】
処理ブロック生成手段１は符号化の対象となる処理ブロックをスイッチ１０を介して減算器１２へ１画素ずつ出力すると共に、処理ブロックの画面位置を減算器３へ出力する（Ｓ３０１）。また、候補ブロック生成手段２は間引きブロックマッチングで検出された候補ブロックを中心として、新たに狭い参照範囲を設定し、参照範囲内から候補ブロックを生成する。候補ブロック生成手段２は生成した候補ブロックをスイッチ１１を介して減算器１２へ１画素ずつ出力すると共に、候補ブロックの画面位置を減算器３へ出力する（Ｓ３０２）。減算器３では候補ブロックの画面位置から処理ブロックの画面位置を減算することで動きベクトルを算出する（Ｓ３０３）。一方、減算器１２では候補ブロックの画素から処理ブロックの画素を減算し、ＡＢＳ演算器１３で絶対値演算を行う。ＡＢＳ演算器１３の出力は加算器１４においてメモリ１５の出力と加算され、再びメモリ１５へ格納される。以上をｍ×ｎ画素全てについて繰り返すことによりメモリ１５には候補ブロックと処理ブロックとの差分絶対値総和ＤＭＣが格納される（Ｓ３０４）。以上の処理は（式６）の様に表わすことができる。
【００５４】
【数６】

ここで、（式６）において（ｘ，ｙ）は候補ブロックまたは処理ブロックの先頭画素からの相対的な画素位置を、Ｙｒ（ｘ，ｙ）は候補ブロックにおける画素位置（ｘ，ｙ）の輝度データを、Ｙｔ（ｘ，ｙ）は処理ブロックにおける画素位置（ｘ，ｙ）の輝度データを表している。
【００５５】
重み付け手段１７はスイッチ９のＯＦＦ動作によってベクトルオフセットが入力されないため、重みテーブル１６から常に重み係数＝１を入力し、メモリ１５の差分絶対値総和ＤＭＣに乗算することで、今回ブロックマッチングを行った処理ブロックと候補ブロックとにおける評価値ＥＳＴを算出する（Ｓ３０５）。以上の処理は（式７）の様に表わすことができる。
【００５６】
【数７】

比較器１８ではメモリ１９に格納されている前回までの暫定最小評価値を入力し、今回の評価値ＥＳＴと比較評価する（Ｓ３０６）。今回の評価値ＥＳＴの方が小さいと判断した場合には今回の評価値ＥＳＴを暫定最小評価値としてメモリ１９を更新する（Ｓ３０７）と共に、候補ブロック生成手段２とメモリ４へ今回の評価値ＥＳＴが最小であったことを伝えるＭＩＮ信号を出力する（Ｓ３０８）。候補ブロック生成手段２は比較器１８からＭＩＮ信号を受け取ると、現在の候補ブロックを予測ブロックの有力候補とする。メモリ４はＭＩＮ信号を受け取ると、減算器３から出力されている現在の動きベクトルを一時的に記憶する。
【００５７】
候補ブロック生成手段２は参照範囲内において１画素ずつずらしながら候補ブロックを順次出力し、上記処理（Ｓ３０２〜３０８）を繰り返すことにより、メモリ１９には参照範囲内における最小評価値が格納される。
【００５８】
参照範囲内の全ての候補ブロックを処理するとＯＵＴ信号が入力される（Ｓ３０９）ため、スイッチ１０及びスイッチ１１は端子ｚ側へ切り換わり、スイッチ２３はＯＮ動作を行う。候補ブロック生成手段２はＯＵＴ信号を入力すると、ＭＩＮ信号が最後に入力された候補ブロックを予測ブロックとして最終決定する。そして、候補ブロック生成手段２は予測ブロックの画素データを、処理ブロック生成手段１は処理ブロックの画素データを各々１画素ずつスイッチ１１またはスイッチ１０を介して順次減算器２４へ出力する。減算器２４では予測ブロックの画素データから処理ブロックの画素データを減算することで画素差分を作成し、符号化処理を実行する符号化手段へ出力する（Ｓ３１０）。また、メモリ４はＯＵＴ信号を入力すると、ＭＩＮ信号が最後に入力されたときに格納した動きベクトルを、スイッチ２３を介してベクトル符号化処理を実行するベクトル符号化手段へ出力する（Ｓ３１１）。この動きベクトルは画素差分を算出した候補ブロックと処理ブロックの画面位置から作成したものである。以上で、今回の動き検出の対象である処理ブロックにおける動き検出は終了する。他の未処理の処理ブロックについても同様に処理していく（Ｓ３１２）。
【００５９】
以上のように、実施の形態１における動き検出方法は、まず広い参照範囲内において、粗い動き検出（間引きブロックマッチング）を行う。間引きブロックマッチングにおいては、単位期待ベクトル算出手段５は過去に動き検出を行った間引き処理ブロックにおける単位動きベクトルＶｕを平滑化して単位期待ベクトルＶｅｘｐｕを求め、期待ベクトル算出手段７は現在処理を行っている間引き処理ブロックと間引き候補ブロックとの時間距離ＤＩＳＴを乗算することによって期待ベクトルＶｅｘｐを算出する。重み付け手段１７は、この期待ベクトルＶｅｘｐが示す位置と現在の動きベクトルが示す位置との画面距離に対応した重み係数ｋと、メモリ１５の差分絶対値総和ＤＭＣ（ブロックマッチング結果）とを乗算したものを評価値ＥＳＴとする。候補ブロック生成手段２は、比較器１８での比較評価から評価値ＥＳＴが最小となる候補ブロックを検出する。その後、間引きブロックマッチングで最小評価値を導き出した候補ブロックの画面位置を中心とする狭い参照範囲内において、候補ブロックを順次生成し、詳細の動き検出（全画素を用いたブロックマチング）を行うことにより動きベクトルを決定する。
【００６０】
また、単位期待ベクトル算出手段５は最小評価値が相場評価値ＥＳＴｐｒ以下の場合にのみ単位期待ベクトルを算出する。つまり、最小評価値が信頼性の高い場合にのみ、検出された動きベクトルを単位期待ベクトルＶｅｘｐｕに反映させるように構成しており、精度の高い期待ベクトルを得ることができる。
【００６１】
以上の方法により、広範囲における動き検出を行うことができるため、被写体が高速かつランダムに画面を移動している場合でも少ない演算量で精度の高い動き検出が可能となる。
【００６２】
（実施の形態２）
以下、実施の形態２の動き検出方法について図３，図４，図５，図６，図７，図８を用いて説明する。
【００６３】
図７は本発明の実施の形態２における動き検出方法を用いた動き検出装置の構成図である。また、図８は本発明の実施の形態１における間引きブロックマッキングを示すフローチャートである。
【００６４】
図７は本発明の動き検出方法を用いた動き検出装置の一実施例を示したものであり、図１と同じ機能を持ったものには同じ番号を付加している。
図７において、メモリ７０１はメモリ１５から入力した差分絶対値総和を一時的に記憶するためのメモリ、相場ＤＭＣ算出手段７０２はメモリ７０１に格納している値と、メモリ７０３に格納している値とを演算することによって、相場差分絶対値総和を算出する手段、メモリ７０３は相場ＤＭＣ算出手段７０２から入力した相場差分絶対値総和を一時的に記憶するためのメモリ、テーブル切り換え手段７０４はメモリ７０３の出力とメモリ１５の出力とを比較して重みテーブルを切り換える手段、重みテーブル７０５は重み係数を配列したテーブルを複数記憶しており、テーブル切り換え手段７０４から指示された重みテーブルを設定するテーブルである。
【００６５】
以上のように構成された動き検出装置について、以下にその動作を述べる。図７の動き検出装置の処理の流れを図８及び図３のフローチャートに示す。
図８において、図２と同じ処理のものには同じ番号を付加している。本装置は、実施の形態１と同様に、まず広い参照範囲内において間引きブロックマッチングにより評価し、次に評価値が最小となる候補ブロックを中心として、狭い参照範囲内において全画素を用いたブロックマッチングにより評価し、評価値が最小となる候補ブロックを動き検出の検出結果とする手順で処理される。ここで、説明の都合上、間引き方法は水平方向及び垂直方向において２画素につき１画素ずつを等間隔に間引くものとする。つまり、水平及び垂直方向の画素数は１／２（ｍ／２×ｎ／２画素のブロック）になる。また、説明の都合上、メモリ７０３には前回の間引き処理ブロックの動き検出において求めた相場差分絶対値総和が格納されているものとする。相場差分絶対値総和の詳細については後述する。更に、説明の都合上、重みテーブル７０５はテーブル１及びテーブル２の２種類のテーブルを持っているものとする。
【００６６】
まず、広い参照範囲内における間引きブロックマッチングが行われる（図８のＳ２００）。この時には、スイッチ１０及びスイッチ１１は端子ｗ側に切り換えられ、またＷＥＩＧＨＴ信号が入力されているためスイッチ９はＯＮ動作をし、ＯＵＴ信号は解除されているためスイッチ２３はＯＦＦ動作をしている。
【００６７】
処理ブロック生成手段１は入力した処理フレームから符号化の対象となる処理ブロックに対応した間引き処理ブロックを生成すると共に、間引き処理ブロックの画面位置を出力する（Ｓ２０１）。また、候補ブロック生成手段２は入力した参照フレームから間引き参照フレームを生成し、参照範囲に対応した範囲内から間引き候補ブロックを生成すると共に、間引き候補ブロックの画面位置を出力する（Ｓ２０２）。以下、実施の形態１と同様に、減算器３、メモリ６、期待ベクトル算出手段７、減算器８、スイッチ９によってベクトルオフセットが算出される（Ｓ２０３〜２０５）。その後、実施の形態１と同様に、スイッチ１０、スイッチ１１、減算器１２、ＡＢＳ演算器１３、加算器１４、メモリ１５によって差分絶対値総和ＤＭＣが算出される（Ｓ２０６）。
【００６８】
テーブル切り換え手段７０４はメモリ１５から差分絶対値総和ＤＭＣを、メモリ７０３から相場差分絶対値総和ＤＭＣｐｒ［ｋ］を入力し、比較する（Ｓ８０１）。
【００６９】
ここで、相場差分絶対値総和ＤＭＣｐｒ［ｋ］は、過去に動き検出を行った間引き処理ブロックにおける差分絶対値総和ＤＭＣの相場を示す値であり、（式８）により求めることができる。
【００７０】
【数８】

但し、ｋは今回動き検出を行う間引き処理ブロックの番号を示し、ＤＭＣ［ｋ−１］は前回の間引き処理ブロックにおける差分絶対値総和であり、ＤＭＣｐｒ［ｋ−１］は前回の間引き処理ブロックにおける相場差分絶対値総和である。また、係数ｅ及び係数ｆはｅ＋ｆ＝１の関係を持つ定数である。つまり、相場差分絶対値総和ＤＭＣｐｒは過去に動き検出を行った間引き処理ブロックにおける差分絶対値総和ＤＭＣを平滑化したものである。係数ｅと係数ｆは、今回の間引き処理ブロックから遠い過去の差分絶対値総和ＤＭＣの影響を受け過ぎないように、統計的に決定される値である。また、相場差分絶対値総和ＤＭＣｐｒの算出に使用する差分絶対値総和ＤＭＣは、間引きブロックマッチングにおいて最小評価値を導き出した間引き候補ブロックの差分絶対値総和ＤＭＣのみを反映させている。また、（式８）から解かるように、今回使用する相場差分絶対値総和ＤＭＣｐｒ［ｋ］は前回の間引き処理ブロックの動き検出後に算出した値を使っている。
【００７１】
テーブル切り換え手段７０４は、今回算出した差分絶対値総和ＤＭＣが相場差分絶対値総和ＤＭＣｐｒ［ｋ］よりも小さい場合には重みテーブル７０５へテーブル１を使用するように指示し（Ｓ８０２）、逆に大きい場合には重みテーブル７０５へテーブル２を使用するように指示する（Ｓ８０３）。重みテーブル７０５は記憶している２種類のテーブルから、テーブル切り換え手段７０４が指示したテーブルを設定する。重み付け手段１７は重みテーブル７０５に設定されているテーブルからベクトルオフセットに対応した重み係数ｋを入力し（Ｓ２０７）、メモリ１５に格納されている差分絶対値総和ＤＭＣに乗じることによって評価値ＥＳＴを算出する（Ｓ２０８）。
【００７２】
ここで、重みテーブル７０５が記憶している２種類のテーブルは、図４、図５または図６中の設定値Ａ、Ｃや距離Ｄや傾きＢ（以下、テーブルパラメータと称す）を調整した、お互いに異なる重み係数ｋの配列で構成されている。テーブル１はベクトルオフセットに依存し難い様にテーブルパラメータを調整し、重み係数ｋの変化度を小さくしている。逆に、テーブル２はベクトルオフセットに依存し易い様にテーブルパラメータを調整し、重み係数ｋの変化度を大きくしている。つまり、今回算出した差分絶対値総和ＤＭＣが小さく、間引きブロックマッチングの一致度が高いと判断した場合には重み係数ｋの影響を小さくして評価し、逆に差分絶対値総和ＤＭＣが大きく、一致度が低いと判断した場合には重み係数ｋの影響を大きくして、期待ベクトルに近い位置を選択しやすくしている。説明上においては、重みテーブル７０５は２種類のテーブルを持つものとしたが、数種類持つように構成して、その場合に差分絶対値総和ＤＭＣと比較する閾値は、例えば、相場差分絶対値総和ＤＭＣｐｒ−ｇ、ＤＭＣｐｒ−ｈ、ＤＭＣｐｒ＋ｐ、ＤＭＣｐｒ＋ｑ（ｇ、ｈ、ｐ、ｑは規定値）等とすることにより対応することができる。以上の処理は（式４）のように表わすことができる。
【００７３】
比較器１８ではメモリ１９に格納されている前回までの暫定最小評価値を入力し、今回の評価値ＥＳＴと比較評価する（Ｓ２０９）。今回の評価値ＥＳＴの方が小さいと判断した場合には今回の評価値ＥＳＴを暫定最小評価値としてメモリ１９を更新する（Ｓ２１０）と共に、候補ブロック生成手段２、メモリ４及びメモリ７０１へ今回の評価値ＥＳＴが最小であったことを伝えるＭＩＮ信号を出力する（Ｓ２１１）。
【００７４】
候補ブロック生成手段２は比較器１８からＭＩＮ信号を受け取ると、現在の間引き候補ブロックを動き検出の有力候補とする。メモリ４はＭＩＮ信号を受け取ると、減算器３から出力されている現在の動きベクトルを一時的に記憶する。また、メモリ７０１はＭＩＮ信号を受け取ると、メモリ１５に格納している差分絶対値総和ＤＭＣを入力し、記憶する。この差分絶対値総和ＤＭＣは、比較器１８で比較され新たな暫定最小評価値としてメモリ１９を更新した評価値ＥＳＴを算出した元の差分絶対値総和ＤＭＣである。
【００７５】
候補ブロック生成手段２は参照範囲内において１画素ずつずらしながら間引き候補ブロックを順次出力し、上記処理（Ｓ２０２〜Ｓ２１２）を繰り返すことにより、メモリ１９には参照範囲内における最小評価値が格納され、メモリ７０１には最小評価値を導き出した差分絶対値総和ＤＭＣが格納される。
【００７６】
参照範囲内の全ての間引き候補ブロックを処理するとＥＮＤ信号が入力され、相場ＤＭＣ算出手段７０２はＥＮＤ信号を入力する（Ｓ２１２）と、最小評価値に対応した差分絶対値総和ＤＭＣ［ｋ］をメモリ７０１から入力し、また今回の相場差分絶対値総和ＤＭＣｐｒ［ｋ］をメモリ７０３から入力し、（式８）に従って、次回の動き検出時に使用する相場差分絶対値総和ＤＭＣｐｒ［ｋ＋１］を算出し、メモリ７０３に格納しておく（Ｓ８０４）。同時に、実施の形態１と同様に、比較器２０、相場評価値算出手段２１、メモリ２２においてＣＡＬＣ信号の出力制御を行うと共に、次回の動き検出時に使用する相場評価値ＥＳＴｐｒ［ｋ＋１］を算出する（Ｓ２１３）。同時に、ＣＡＬＣ信号が入力される（Ｓ２１４）と、実施の形態１と同様に、メモリ４、単位期待ベクトル算出手段５、メモリ６において次回の動き検出時に使用する単位期待ベクトルＶｅｘｐｕ［ｋ＋１］を算出する（Ｓ２１５）。
【００７７】
以上で、広い参照範囲内における間引きブロックマッチングが終了したことになり、引き続き、実施の形態１と全く同様に、最小評価値を導き出した候補ブロックを中心とする狭い参照範囲内において全画素を用いたブロックマッチングを行うことで予測ブロックが決定され、符号化手段へ画素差分が、ベクトル符号化手段へ動きベクトルが出力される（図３のＳ３０１〜３１１）。
【００７８】
以上で、今回の動き検出の対象である処理ブロックにおける動き検出は終了したことになるが、他の未処理の処理ブロックについても同様に処理していく（Ｓ２００〜２１６、３００〜３１２）。
【００７９】
以上のように、実施の形態２における動き検出装置は、実施の形態１における動き検出装置に相場差分絶対値総和ＤＭＣｐｒを算出するためのメモリ７０１、相場ＤＭＣ算出手段７０２、メモリ７０３を加え、また重みテーブルを切り換えるためのテーブル切り換え手段７０４と重みテーブル７０５を加えた構成となり、テーブル切り換え手段７０４は現在評価している候補ブロックの差分絶対値総和ＤＭＣと、相場差分絶対値総和ＤＭＣｐｒとを比較し、差分絶対値総和ＤＭＣの方が小さい場合には重み係数ｋの緩いテーブル１を、逆に差分絶対値総和ＤＭＣの方が大きい場合には重み係数ｋの厳しいテーブル２を使用するように重みテーブル７０５へ指示を出すものである。
【００８０】
これにより、ブロックマッチング結果の信頼性が高い場合には重み係数の影響を受け難い評価を行い、逆にブロックマッチング結果の信頼性が低い場合には重み係数の影響を受け安い評価を行うことによって、特徴に応じた最適な評価をすることができるため、被写体が高速かつランダムに画面を移動している場合でも少ない演算量で、より精度の高い動き検出が可能となる。
【００８１】
なお、以上の説明では、広い参照範囲内におけるブロックマッチングは間引き画像で行う場合について説明をしたが、間引きしていない全画素を使った画像で行っても良い。この場合、各処理ブロックの動きベクトルが同じ値に揃いやすくなり、ベクトル符号化手段におけるベクトル差分が小さくなるため、符号化効率がよくなる。また、この場合、変形をしながら移動している様な処理ブロックについても正確な動き検出が可能になるという効果がある。
【００８２】
また、以上の説明では、ブロックマッチング結果は差分絶対値総和ＤＭＣで求めるとしたが、差分二乗総和で求めても構わない。
また、以上の説明では、期待ベクトルＶｅｘｐの算出、単位期待ベクトルＶｅｘｐｕの算出、相場評価値ＥＳＴｐｒの算出は処理ブロックがフレームデータで構成されている場合と、第１のフィールドデータのみで構成されている場合と、第２のフィールドデータのみで構成されている場合とで別々に算出するようにしたが、区別せずに算出するようにしてもよい。
【００８３】
また、以上の説明では、比較器２０はメモリ１９からの最小評価値ＥＳＴが、メモリ２２に格納している相場評価値ＥＳＴｐｒ以下の場合に単位期待ベクトル算出手段５へＣＡＬＣ信号を出力する例で説明したが、最小評価値ＥＳＴが相場評価値ＥＳＴｐｒ未満としてもよく、また相場評価値ＥＳＴｐｒを基に別の閾値を作成して最小評価値ＥＳＴと比較するように構成してもよい。別の閾値の一例としては、相場評価値ＥＳＴｐｒに規定値を加算したり、減算したりすることで作成し、単位期待ベクトルＶｅｘｐｕの算出条件を易しくしたり、厳しくしたりすることが可能になる。
【００８４】
また、以上の説明では、重み係数ｋを配列した重みテーブル１６または７０５は、ベクトルオフセットのｘ成分とｙ成分の二次元テーブルにより取得する構成としたが、ベクトルオフセット長をパラメータに取得するようにしてもよい。
【００８５】
また、以上の説明では、重み付け手段１７において、メモリ１５に格納しているブロックマッチング結果である差分絶対値総和値ＤＭＣに重み係数ｋを乗算するとしたが、加算して重み付けをしても構わない。
【００８６】
また、以上の説明では、図４、図５、図６に示した重みテーブル中の重み係数ｋは期待ベクトルＶｅｘｐが示す位置（０，０）を中心に同心円状に大きくなる例について説明したが、非同心円状のテーブル等にしても構わない。
【００８７】
また、実施の形態２の説明では、重みテーブル７０５が記憶している複数のテーブルは、テーブルパラメータの少なくとも１つ以上が異なるように構成したが、それ以外に図４、図５、図６の様なテーブルを持ち、切り換えるように構成してもよい。
【００８８】
また、以上の説明では、ハードウェアで本発明を実施した場合の一例について説明したが、ソフトウェアで実施しても同じ効果が得られるのは明らかである。この場合には、図２、図３及び図８で示したフローチャートの通りにソフトウェアを構成すれば容易に適用できる。
【００８９】
【発明の効果】
以上のように本発明の動き検出方法によれば、次に動き検出を行う処理ブロックは、過去に動き検出を完了している処理ブロックの動きベクトルから求めた期待ベクトルを基に動き検出を行うことにより、被写体が高速に画面を移動している場合でも少ない演算量で精度の高い動き検出が可能になるという顕著な効果が得られる。
【００９０】
また、処理ブロックと候補ブロックとの時間距離を正確に反映させた期待ベクトルを算出することによって、更に精度の高い動き検出が可能になるという顕著な効果が得られる。
【００９１】
また、期待ベクトルを算出するための単位期待ベクトルは、単なるフレーム全体の平均的な動きを推測するだけではなく、動き検出を行っている処理ブロックの周辺付近の動きを推測していると共に、他の処理ブロックに対して特異的なランダムな動きをしている処理ブロックの影響を受けないように演算合成することにより、被写体が高速かつランダムに画面を移動している場合でも精度の高い動き検出が可能になるという顕著な効果が得られる。
【００９２】
また、着目している処理ブロックの動き検出結果が信頼性の高い結果なのか否かを最小評価値と第１の閾値により判断し、信頼性が高い場合にのみ着目している処理ブロックの動きベクトルを単位期待ベクトルに反映することにより、更に精度の高い動き検出が可能になるという顕著な効果が得られる。
【００９３】
また、第１の閾値は、遠い過去に動き検出を行った処理ブロックの最小評価値の影響を受け過ぎず、動き検出を行っている処理ブロックの周辺付近の最小評価値を重視していると共に、他の処理ブロックの最小評価値と比較して、特異的な最小評価値の影響を受けないように演算することにより、動き検出結果の信頼性の判断が精度よく実現可能になるという顕著な効果が得られる。
【００９４】
また、期待ベクトル、単位期待ベクトルまたは第１の閾値は、フレーム予測の場合と、第１のフィールドデータにおけるフィールド予測の場合と、第２のフィールドデータにおけるフィールド予測の場合とで、別々に算出されることにより、更に精度の高い動き検出が可能になるという顕著な効果が得られる。
【００９５】
また、期待ベクトル、単位期待ベクトルまたは第１の閾値は、処理ブロックに対する候補ブロックの時間方向によって、別々に算出されることにより、更に精度の高い動き検出が可能になるという顕著な効果が得られる。
【００９６】
また、候補ブロックとのブロックマッチング結果に期待ベクトルが示す位置からの画面距離に対応した重み付けを行ったものを評価値とすることにより、期待ベクトルに対応した動きベクトルを持つ候補ブロックを選び易くなり、被写体が高速かつランダムに画面を移動している場合でも少ない演算量で精度の高い動き検出が可能になるという顕著な効果が得られる。
【００９７】
また、評価値は、ブロックマッチング結果に画面距離に対応した設定値を加算または乗算して算出することにより、期待ベクトルに対応した動きベクトルを持つ候補ブロックを選び易くなり、被写体が高速かつランダムに画面を移動している場合でも少ない演算量で精度の高い動き検出が可能になるという顕著な効果が得られる。
【００９８】
また、前記期待ベクトルが示す位置を中心として画面距離が遠くなるに従って重み付けの設定値を小さくなることにより、期待ベクトルに近い値を有する動きベクトルを持つ候補ブロックを選び易くしており、被写体が高速かつランダムに画面を移動している場合でも少ない演算量で精度の高い動き検出が可能になるという顕著な効果が得られる。
【００９９】
また、動きベクトルが前記期待ベクトルに近い値の場合には、重み付けの設定値を所定の固定値にすることにより、比較的平等に評価を行うことができ、期待ベクトルの多少の誤差に影響されない精度の高い動き検出が可能になるという顕著な効果が得られる。
【０１００】
また、動きベクトルと前記期待ベクトルの差が所定の値以上ある場合には、前記重み付けの設定値を所定の固定値にすることにより、期待ベクトルが示す位置から所定の範囲以上に遠い位置の動きベクトルを持つ候補ブロックに関しては、所定の重み付けを施すことによって、他の処理ブロックとは異なるランダムな動きをしている処理ブロックについても、正確な動き検出が可能になるという顕著な効果が得られる。
【０１０１】
また、固定値にされた重み付けの設定値は、今回のブロックマッチング結果と過去に確定した１または２以上のブロックマッチング結果を平滑化した閾値との比較結果によって設定することにより、着目している処理ブロックのブロックマッチング結果の信頼性に対応して重み係数を調整し、各処理ブロックの特徴に応じた最適な評価をすることができるため、被写体が高速かつランダムに画面を移動している場合でも少ない演算量で、より精度の高い動き検出が可能になるという顕著な効果が得られる。
【図面の簡単な説明】
【図１】本発明の実施の形態１における動き検出方法を用いた動き検出装置の構成図
【図２】本発明の実施の形態１における間引きブロックマッキングを示すフローチャート
【図３】本発明の実施の形態１における全画素を用いたブロックマッキングを示すフローチャート
【図４】重みテーブルを示す図
【図５】一定以上のベクトルオフセットの重み係数を固定した重みテーブルを示す図
【図６】一定範囲のベクトルオフセットの重み係数を固定した重みテーブルを示す図
【図７】本発明の実施の形態２における動き検出方法を用いた動き検出装置の構成図
【図８】本発明の実施の形態１における間引きブロックマッキングを示すフローチャート
【図９】従来の間引きデータを用いた動き検出方法を説明する概念図
【符号の説明】
１ 処理ブロック生成手段
２ 候補ブロック生成手段
３ 減算器
４ メモリ
５ 単位期待ベクトル算出手段
６ メモリ
７ 期待ベクトル算出手段
８ 減算器
９ スイッチ
１０ スイッチ
１１ スイッチ
１２ 減算器
１３ ＡＢＳ演算器
１４ 加算器
１５ メモリ
１６ 重みテーブル
１７ 重み付け手段
１８ 比較器
１９ メモリ
２０ 比較器
２１ 相場評価値算出手段
２２ メモリ
２３ スイッチ
２４ 減算器
７０１ メモリ
７０２ 相場ＤＭＣ算出手段
７０３ メモリ
７０４ テーブル切り替え手段
７０５ 重みテーブル[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a motion detection method used for high-efficiency encoding of a moving image and the like.
[0002]
[Prior art]
2. Description of the Related Art In recent years, as a high-efficiency encoding method for a moving image, a motion compensation prediction method adopted in MPEG2 or the like has been used. In the motion compensation prediction method, luminance data of a processing frame to be encoded is divided into blocks of m × n pixels (hereinafter, referred to as processing blocks), and each processing block is temporally different from another frame (hereinafter, referred to as a reference block). A sum of absolute differences between pixels corresponding to a block of m × n pixels (hereinafter, referred to as a candidate block) in luminance data of a frame (hereinafter referred to as a frame) is set as an evaluation value, and a candidate block (hereinafter, referred to as a prediction block) having the minimum evaluation value is used. A block is searched in the reference area, and a motion amount to the position (hereinafter, referred to as a motion vector) and a pixel difference including a difference between corresponding color difference data are output. Such a motion detection method is called a block matching method. Here, the evaluation value may be something other than the sum of absolute difference values, and for example, the sum of squared differences may be used as the evaluation value. The motion vector output by the motion compensation prediction means is subjected to a difference calculation process with a vector of an adjacent macro, a variable length coding process for a vector, and the like (the above is collectively referred to as a vector coding process). Orthogonal transformation processing, quantization processing, variable-length coding processing, and the like (the above are collectively referred to as coding processing) are performed. When the motion compensation prediction method is used, the pixel difference is coded. Therefore, highly efficient encoding can be performed.
[0003]
In motion detection, detection in a wide reference range is required in order to achieve high image quality even when the motion of a subject in a processing frame is fast, such as in high-speed zoom and high-speed panning. However, to perform motion detection in a wide reference range, an enormous amount of calculation is required, and the circuit scale and the memory bandwidth are increased when implemented by hardware, and a high-speed CPU is implemented when implemented by software. Even if used, real-time processing becomes very difficult.
[0004]
As a first method for solving this, a motion vector of a rough frame is detected by calculating a motion vector of the whole processing frame with respect to the entire reference frame, and thereafter, the motion vector of the whole frame is indicated for each individual processing block. There is one that performs motion detection in detail centering on the screen position. In other words, by searching only a relatively narrow reference range around the screen position indicated by the motion vector of the entire frame, the same effect as when a virtually wide reference range is searched can be obtained. A representative example of this method is disclosed in Japanese Patent No. 3171249. In this method, an evaluation value is obtained by performing block matching between a processing block and each candidate block in a wide reference range, and this processing is similarly performed on a relatively small number of processing blocks. Are accumulated, and the motion vector of the entire frame is obtained from the position where the accumulated value is minimum. Thereafter, in each processing block, a narrow reference range is searched centering on the position indicated by the motion vector of the entire frame, thereby enabling motion detection in a wide reference range with a small amount of calculation (for example, see Japanese Patent Application Laid-Open No. H10-163873). 1).
[0005]
As a second method for solving the above problem, there is a method of using data obtained by thinning out luminance data for both the processing block and the candidate block. In this example, a motion vector is detected by block matching between a thinned processing block of i × j pixels (i <m, j <n) obtained by thinning a processing block of m × n pixels and a candidate block similarly thinned. Since the number of pixels constituting one block decreases in accordance with the thinning rate, the amount of calculation decreases drastically. Normally, when a motion vector is searched for by the thinning-out processing block, detailed motion detection is performed in a block of m × n pixels in which a narrow reference range centered on the screen position indicated by the motion vector is not thinned out. In other words, a rough motion vector is detected by performing motion detection within a wide reference range using thinned data, and then a narrow range is searched centered on the position indicated by the rough motion vector, so that a wide reference can be performed with a small amount of calculation. This enables motion detection in a range.
[0006]
[Patent Document 1]
Japanese Patent No. 3171249
[0007]
[Problems to be solved by the invention]
However, in the first conventional motion detection method, the effect is obtained when the pan image or the subject moves uniformly on the screen, but when the zoom image or the subject moves randomly, The motion vector of the entire frame does not match the actual motion of the processing block. For this reason, there is a problem in that an optimal prediction block is lost in a narrow reference range in which motion detection is performed, and the image quality is significantly deteriorated.
[0008]
Further, in the second conventional motion detection method, by performing block matching between the thinned processing block and the thinned candidate block, a feature point is thinned depending on an image, so that a block matching result is deteriorated. There is a problem that a motion vector to be detected cannot be detected. This will be described with reference to the drawings.
[0009]
9A and 9B are conceptual diagrams illustrating a conventional motion detection method using thinned data. FIG. 9A is a processing block, FIG. 9B is a candidate block, and FIG. 9C is a thinning processing block corresponding to the processing block (a). , (D) show thinning-out candidate blocks corresponding to the candidate block (b).
[0010]
In FIG. 9, (c) and (d) show examples in which the horizontal pixels in (a) and (b) are thinned out to １ ／, respectively. When thinning data is created, since thinning is normally performed on a frame or a field, if the relative position with respect to the frame or the field differs between the processing block and the candidate block, as shown in FIG. A match may occur in block matching between a and b), but may not match in block matching using the thinned image (block matching between c and d). For this reason, there is a problem that image quality is significantly deteriorated because other candidate blocks are coded as prediction blocks instead of candidate blocks that should be originally detected by motion detection.
[0011]
An object of the present invention is to enable high-precision motion detection with a small amount of computation even when a subject moves on a screen at high speed and randomly.
[0012]
[Means for Solving the Problems]
In order to achieve the above object, a motion detection method according to claim 1 of the present invention is a motion detection method for obtaining a motion vector by block matching between a processing block to be subjected to motion detection and a candidate block created from within a reference range. The method includes calculating an expected vector from a motion vector of one or more previously processed blocks, and calculating a motion vector of the processed block from the expected vector.
[0013]
The motion detection method according to claim 2, wherein in the motion detection method according to claim 1, the expected vector is a time distance between the processing block and the candidate block based on a unit expected vector indicating a magnitude per unit time. It is characterized in that it is changed to a size corresponding to.
[0014]
A motion estimation method according to a third aspect is the motion estimation method according to the first or second aspect, wherein the unit expectation vector considers the degree of influence of each motion vector of one or more previously processed blocks. And smoothing.
[0015]
A motion estimation method according to a fourth aspect is the motion estimation method according to any one of the first, second, and third aspects, wherein a minimum evaluation value determined by block matching in the processing block is equal to or less than a first threshold. If the expected value is less than or equal to the first threshold value, the expected vector is calculated and used. If the minimum evaluation value is greater than or equal to the first threshold value, the expected vector used in the previous detection is used.
[0016]
According to a fifth aspect of the present invention, in the motion detection method according to the fourth aspect, the first threshold value is determined in consideration of an influence of each of minimum evaluation values of one or more past processing blocks obtained in the past. It is characterized by being smoothed.
[0017]
According to a sixth aspect of the present invention, in the motion detection method according to any one of the first, second, third, fourth, and fifth aspects, the expected vector, the unit expected vector, The first threshold is calculated based on data obtained by combining first and second field data that are temporally different from each other, when the first threshold is calculated using only the first field data, and when the second threshold is calculated. It is characterized in that it may be calculated using only field data.
[0018]
7. The motion detection method according to claim 7, wherein the expectation vector and the unit are set in the motion detection method according to any one of claims 1, 2, 3, 4, 5, or 6. The expectation vector and the first threshold value may be calculated using a temporally old candidate block for the processing block, or may be calculated using a temporally new candidate block for the processing block. Features.
[0019]
The motion detection method according to claim 8 is the motion detection method according to any one of claim 1, claim 2, claim 3, claim 4, claim 5, claim 6, or claim 7. An evaluation value in matching is calculated in consideration of a weight corresponding to a screen distance from a position indicated by an expectation vector.
[0020]
According to a ninth aspect of the present invention, in the motion detection method according to the eighth aspect, the evaluation value is calculated by adding a set value corresponding to a screen distance from a position indicated by the expected vector to the block matching result. It is characterized by multiplication and calculation.
[0021]
According to a tenth aspect of the present invention, in the motion detection method according to the eighth or ninth aspect, the set value of the weighting is set as the screen distance becomes longer around the position indicated by the expected vector. It is characterized in that it is set from a first data table or a calculation formula that is difficult to select.
[0022]
A motion detection method according to claim 11, wherein in the motion detection method according to any one of claims 8, 9, or 10, the motion vector is a value closer to a center of the expected vector than a first predetermined value. In the case of (1), the set value of the weighting is set from the second data table or the calculation formula, which is a gradual or fixed value than the first data table or the calculation formula.
[0023]
According to a twelfth aspect of the present invention, in the motion detection method according to any one of the eighth to ninth aspects, the difference between the motion vector and the expected vector is a second predetermined value. When the value is equal to or more than the value, the set value of the weight is set from a third data table or a calculation formula having a slope that is gentler or a fixed value than the first table or the calculation formula.
[0024]
According to a thirteenth aspect of the present invention, in the motion detection method according to the tenth, eleventh, or twelfth aspect, the first, the second, or the third data table or the calculation formula is set with a reduced weight. The value is adjusted based on a comparison result between the current block matching result and a threshold value obtained by smoothing one or more block matching results determined in the past.
[0025]
As described above, even when the subject is moving on the screen at high speed and at random, highly accurate motion detection can be performed with a small amount of calculation.
[0026]
BEST MODE FOR CARRYING OUT THE INVENTION
Embodiments of the present invention will be described with reference to the drawings.
(Embodiment 1)
Hereinafter, the motion detection method according to the first embodiment will be described with reference to FIGS. 1, 2, 3, 4, 5, and 6.
[0027]
FIG. 1 is a configuration diagram of a motion detection device using the motion detection method according to Embodiment 1 of the present invention. FIG. 2 is a flowchart showing thinned-out block masking according to the first embodiment of the present invention. FIG. 3 is a flowchart showing block masking using all pixels according to the first embodiment of the present invention. FIG. 5 is a diagram illustrating a table, FIG. 5 is a diagram illustrating a weight table in which weight coefficients of vector offsets equal to or greater than a predetermined value are fixed, and FIG. 6 is a diagram illustrating a weight table in which weight coefficients of vector offsets in a predetermined range are fixed.
[0028]
In FIG. 1, a processing block generating means 1 inputs a processing frame, outputs a processing block of m × n pixels for which motion detection is to be performed and corresponding color difference data, and outputs a coordinate position of the processing block in the frame. Means for outputting a reference frame, outputting a candidate block of m × n pixels and corresponding color difference data from within the reference range, and outputting a screen position of the candidate block. , A subtractor 3 for subtracting the screen position output from the processing block generating means 1 from the screen position output from the candidate block generating means 2 to calculate a motion vector, and a memory 4 for calculating a motion vector output from the subtractor 3 The memory for temporarily storing the unit expected vector calculating means 5 calculates the unit expected vector from the motion vector stored in the memory 4. Memory 6 is a memory for temporarily storing the unit expected vector output from the unit expected vector calculating means 5, and an expected vector calculating means 7 is a means for calculating the expected vector from the unit expected vector stored in the memory 6. , A subtractor 8 for subtracting the expected vector output from the expected vector calculation means 7 from the motion vector output from the subtractor 3, a switch 9 for sending out the output of the subtractor 8 when a WEIGHT signal is input, and a switch 10. Outputs the luminance data output by the processing block generating means 1 when the switch is switched to the terminal w side, and outputs the luminance data and the corresponding color difference data output by the processing block generating means 1 when the switch is switched to the terminal z side. (Which is collectively referred to as pixel data), switch 11 is a switch When the switch is switched to the terminal z side, the luminance data output by the candidate block generating means 2 is output. When the switch is switched to the terminal z side, the switch which outputs the pixel data output by the candidate block generating means 2 is used. A subtractor for subtracting the luminance data of the processing block input via the switch from the luminance data of the candidate block input via the switch; an ABS arithmetic unit for calculating the absolute value of the output of the subtractor; 14 is an adder for adding the output of the ABS arithmetic unit 13 and the output of the memory 15 to calculate a block matching result, the memory 15 is a memory for temporarily storing the output of the adder 14, and the weight table 16 is an array of weight coefficients. Weighting means 17 corresponding to the output of the subtractor 8 input via the switch 9 Means for inputting a weighting coefficient from the cable 16 and multiplying the block matching result by the weighting coefficient to generate an evaluation value. The comparator 18 compares the output of the weighting means 17 with the data stored in the memory 19, and determines the smaller one as the provisional minimum. A comparator that outputs the evaluation value to the memory 19, the memory 19 temporarily stores the provisional minimum evaluation value output by the comparator 18, and the comparator 20 sends the output of the memory 19 to the market evaluation value calculation unit 21. And a comparator for comparing the output of the memory 19 with the output of the memory 22 and outputting a CALC signal to the unit expectation vector calculation means 5 when the output of the memory 19 is equal to or less than the output of the memory 22; Reference numeral 21 denotes a unit for calculating a market evaluation value by calculating an output of the comparator 20 and an output of the memory 22, and the memory 22 includes a market price from the market evaluation value calculation unit 21. A memory for temporarily storing the evaluation value, a switch 23 is a switch for outputting the motion vector from the memory 4 to the vector encoding means when the OUT signal is inputted, and a subtractor 24 is a candidate block inputted via the switch 11. Is a subtractor that subtracts the pixel data of the processing block input via the switch 10 from the pixel data of, and calculates a pixel difference. Further, the processing block generating means 1 regularly thins out the luminance data constituting the processing block, generates a thinning processing block composed of the remaining luminance data, outputs the thinning processing block to the switch 10, and also performs the processing frame thinning out according to the same rule. It has a function of outputting the screen position of the thinning-out processing block to the subtractor 3 (hereinafter, referred to as a thinning-out processing frame). Further, the candidate block generation means 2 thins out the luminance data of the reference frame according to the same rule as when the thinning processing block is generated, and in the thinned reference frame composed of the remaining luminance data, the thinning candidate It has a function of generating a block and outputting it to the switch 11 and outputting the screen position of the thinning candidate block with respect to the thinning reference frame to the subtractor 3.
[0029]
The operation of the motion detection device configured as described above will be described below.
In the motion detection method according to the first embodiment of the present invention, first, evaluation is performed by block matching (hereinafter, referred to as “thinning-out block matching”) between a thinning-out processing block and a thinning-out candidate block within a wide reference range. Is evaluated by block matching (hereinafter simply referred to as block matching) between a processing block using all pixels and a candidate block within a narrow reference range, with the candidate block having the smallest evaluation value as the center, and the candidate block having the smallest evaluation value is obtained. Is set as a detection result of the motion detection. Here, for convenience of explanation, it is assumed that the thinning-out method thins out one pixel every two pixels at equal intervals in the horizontal direction and the vertical direction. That is, the number of pixels in the horizontal and vertical directions is （(block of m / 2 × n / 2 pixels). Further, the thinning method does not limit the scope of the present invention, and there is no problem even if the thinning method is changed. Further, for convenience of explanation, it is assumed that the memory 6 stores the unit expected vector calculated up to the previous time, and the memory 22 stores the market evaluation value calculated up to the previous time.
[0030]
The processing flow of the motion detection device of FIG. 1 will be described with reference to the flowcharts of FIGS.
2 and 3, first, thinned-out block matching within a wide reference range is performed (S200). At this time, the switch 10 and the switch 11 are switched to the terminal w side, and the switch 9 is turned on because the WEIGHT signal is input, and the switch 23 is turned off because the OUT signal is released. .
[0031]
The processing block generating means 1 generates a thinning processing block corresponding to the processing block to be encoded from the input processing frame, and outputs the screen position of the thinning processing block (S201). Next, the candidate block generation means 2 generates a thinned reference frame from the input reference frame, generates a thinned candidate block from the range corresponding to the reference range, and outputs the screen position of the thinned candidate block (S202). Here, since the thinning rate is が in the horizontal and vertical directions, the x- and y-coordinates of the screen positions of the thinning-out processing block and the thinning-out candidate block are respectively 1 x coordinate and y-coordinate of the processing block and the candidate block. / 2. The subtractor 3 subtracts the screen position of the thinning processing block output by the processing block generating means 1 from the screen position of the thinning candidate block output by the candidate block generating means 2 to calculate a motion vector (S203). On the other hand, the expected vector calculation means 7 receives the unit expected vector Vexpu from the memory 6 and calculates the expected vector Vexp (S204).
[0032]
Here, the unit expectation vector Vexpu is obtained by estimating the screen distance that the thinning-out processing block moves per field, and is obtained as a vector as in (Equation 1).
[0033]
(Equation 1)

Here, k indicates the number of the thinning processing block for performing the current motion detection, Vepxu [k-1] is the unit expected vector in the previous thinning processing block, and Vu [k-1] is the motion in the previous thinning processing block. This is a unit motion vector obtained by converting a vector per field. The coefficient a and the coefficient b are constants having a relation of a + b = 1. That is, the unit expectation vector Vexpu is obtained by smoothing the unit motion vector Vu in the thinning-out processing block for which the motion has been detected in the past. The coefficient a and the coefficient b are not excessively affected by past unit motion vectors far from the current thinning-out processing block, and are not affected by specific past unit motion vectors that are moving randomly. It is a value that is determined statistically.
[0034]
The expected vector calculation means 7 calculates an expected vector Vexp for estimating the screen distance moved from the reference frame to the processing frame with respect to the thinning-out processing block, as shown in (Expression 2).
[0035]
(Equation 2)

However, DIST represents the time distance between the reference frame and the processing frame by the number of fields.
[0036]
Next, the subtractor 8 subtracts the motion vector from the subtractor 3 and the expectation vector Vexp from the expectation vector calculation means to obtain a vector offset indicating how far the thinning candidate block this time is away from the expectation vector. The calculated value is supplied to the weighting means 17 via the switch 9 (S205).
[0037]
On the other hand, the thinning processing blocks generated by the processing block generating means 1 are output one pixel at a time and supplied to the subtractor 12 via the switch 10. The thinning candidate blocks generated by the candidate block generating means 2 are also output one pixel at a time corresponding to the pixels of the thinning processing block output from the processing block generating means 1 and supplied to the subtractor 12 via the switch 11. . The subtracter 12 subtracts the pixels of the thinning-out processing block from the pixels of the thinning-out candidate block, and performs an absolute value operation in the ABS calculator 13. The output of the ABS calculator 13 is added to the output of the memory 15 in the adder 14 and stored in the memory 15 again. By repeating the above for all m / 2 × n / 2 pixels, the memory 15 stores the sum of absolute differences DMC between the thinning candidate block and the thinning processing block (S206). The above processing can be expressed as (Equation 3).
[0038]
[Equation 3]

Here, (x, y) indicates the relative pixel position from the top pixel of the thinning candidate block or the thinning processing block, and Yr (x, y) indicates the luminance data of the pixel position (x, y) in the thinning candidate block. , Yt (x, y) represent luminance data at the pixel position (x, y) in the thinning-out processing block.
[0039]
Next, the weighting means 17 inputs the weighting coefficient k corresponding to the vector offset input from the subtractor 8 through the switch 9 from the weighting table 16 (S207), and multiplies the difference absolute value sum DMC of the memory 15 by An evaluation value EST is calculated for the thinning processing block and the thinning candidate block that have been subjected to the block matching this time (S208). The above processing can be expressed as (Equation 4).
[0040]
(Equation 4)

Next, the comparator 18 inputs the provisional minimum evaluation value up to the previous time stored in the memory 19 and compares and evaluates it with the current evaluation value EST (S209). When it is determined that the current evaluation value EST is smaller, the memory 19 is updated with the current evaluation value as the provisional minimum evaluation value (S210), and the current evaluation value is stored in the candidate block generation means 2 and the memory 4 at the minimum. Is output (S211). Here, the weight table 16 is composed of a ROM table in which the weight coefficients k are arranged.
[0041]
This ROM table will be described with reference to FIG.
In FIG. 4A, the horizontal axis represents the x component of the vector offset, and the vertical axis represents the y component of the vector offset. The numerical value at the point where the x component and the y component of the vector offset intersect is the weight coefficient k.
[0042]
FIG. 4B is a graph of the weight coefficient k when the y component of the vector offset is 0 in the weight table of FIG. 4A, and the horizontal axis is the x component of the vector offset, and the vertical axis is the vertical axis. This is the value of the weight coefficient k. FIG. 4 shows that the closer the vector offset is to the position (0, 0), the smaller the weight coefficient k is. The fact that the vector offset is close to the position of (0, 0) means that the value of the current motion vector is close to the expected vector, that is, the closer the motion vector is to the expected vector, The weight coefficient k by which the difference absolute value sum DMC is multiplied becomes a small value, and as a result, the evaluation value EST tends to take a small value.
[0043]
Here, FIG. 5 or FIG. 6 shows another example of the weight table 16, and FIG. 5 shows an example in which the weight coefficient k is fixed to the set value C when the vector offset becomes larger than a certain value. 6 is an example in which the set value is fixed to A within the range of the distance D from the vector offset (0, 0).
In other words, in the example of FIG. 5, when the motion vector deviates from the expected vector, it is highly likely that the processing block whose motion is being detected moves differently from other processing blocks. The evaluation values entered are intended to be evaluated fairly irrespective of the degree of coincidence with the expected vector by keeping the coefficient k constant.
[0044]
Further, in the example of FIG. 6, when the motion vector has a value very close to the expected vector, a weighting factor is used between evaluation values whose motion vectors fall within this range in order to avoid an adverse effect due to the reliability of the expected vector. By making k constant, fair evaluation is performed regardless of the degree of coincidence with the expected vector. The set value C and the set value A do not have to be fixed values, and the object can be achieved if the set value C and the set value A have a gentler slope than the slope B. Also, the weighting means 17 multiplies the difference absolute value sum DMC by the weighting coefficient k to make it easier to adopt a motion vector close to the value of the expected vector. In some cases, only the subject whose part is small moves randomly. In this case, the correct motion vector does not match the expected vector. However, the difference absolute value sum DMC at the time of a correct motion vector takes a very small value as compared with the others, so that even when multiplied by the weight coefficient k, the evaluation value EST usually becomes the minimum, and a highly accurate result is obtained. be able to.
[0045]
In FIG. 2, upon receiving the MIN signal from the comparator 18, the candidate block generation means 2 sets the current thinning candidate block as a strong candidate for motion detection. Upon receiving the MIN signal, the memory 4 temporarily stores the current motion vector output from the subtractor 3.
[0046]
The candidate block generating means 2 sequentially outputs the thinned candidate blocks while shifting one pixel at a time in the reference range, and repeats the above processing (S202 to S211), so that the minimum evaluation value in the reference range is stored in the memory 19. .
[0047]
When all the thinning candidate blocks in the reference range are processed, an END signal is input from the outside (S212). Upon input of the END signal, the comparator 20 inputs the minimum evaluation value EST [k] in the reference range stored in the memory 19 and compares the minimum evaluation value EST [k] stored in the memory 22 with the market evaluation value ESTpr [k]. If the minimum evaluation value EST [k] is equal to or smaller than the market evaluation value ESTpr [k], the comparator 20 outputs a CALC signal requesting the unit expectation vector calculation means 5 to update the unit expectation vector Vexpu. Further, the comparator 20 outputs the minimum evaluation value EST [k] to the market price evaluation value calculation means 21. The market evaluation value calculation means 21 calculates the market evaluation value ESTpr [k + 1] to be used at the time of the next motion detection from the minimum evaluation value EST [k] and the market evaluation value ESTpr [k] from the memory 22 (S213). . This processing can be expressed as in (Equation 5).
[0048]
(Equation 5)

Here, the market evaluation value ESTpr is an index indicating the market of the minimum evaluation value in the thinning-out processing block for which the motion has been detected in the past. In (Equation 5), the coefficient c and the coefficient d are represented by the relationship of c + d = 1. Is a constant with That is, the market evaluation value ESTpr is a value obtained by smoothing the minimum evaluation value EST in the thinning-out processing block for which motion detection has been performed in the past. The coefficient c and the coefficient d are not excessively affected by the past minimum evaluation value EST far from the current thinning-out processing block, and are affected by the past minimum evaluation value EST that has become specifically large without an optimal thinning candidate block. It is a value that is determined statistically so as not to receive it.
[0049]
When the unit expected vector calculation means 5 receives the CALC signal from the comparator 20 (S214), it inputs the motion vector at the time of calculating the minimum evaluation value from the memory 4 to obtain the unit motion vector Vu [k] (Equation 1). ), The unit expectation vector Vexpu [k + 1] to be used in the next motion detection is calculated and stored in the memory 6 (S215). That is, only when the minimum evaluation value EST of the thinning-out processing block is equal to or smaller than the market evaluation value ESTpr, the unit expectation vector Vexpu is updated with the current motion vector. The case where the minimum evaluation value EST of the thinning processing block is equal to or less than the market evaluation value ESTpr means that the result of the current block matching is highly reliable.
[0050]
The motion compensation prediction method includes a frame prediction mode in which a processing block and a candidate block are generated from data obtained by combining a temporally different first field and a second field to perform motion detection, and only the first field or the first field. There is a first field prediction mode or a second field prediction mode in which a processing block and a candidate block are generated from data of only two fields to perform motion detection. Incidentally, the generation method of the thinning-out processing block or the thinning-out candidate block switches the thinning-out method according to the prediction mode. Also in each prediction mode, there are a forward reference mode in which a temporally older frame is used as a reference frame and a backward reference mode in which a temporally new frame is used as a reference frame. As described above, since there are two reference modes in each of the three prediction modes, there are a total of six types of combinations, and the unit expected vector of the memory 6 and the market evaluation value of the memory 22 described above are combined. Are stored separately for each of the six modes. That is, the unit expectation vector calculation means 5 reads the unit expectation vector of the current mode from the memory 6 to calculate a new unit expectation vector, and updates only the unit expectation vector of the current mode stored in the memory 6. Similarly, the expected vector calculating means 7 reads the unit expected vector of the current mode from the memory 6 and calculates the expected vector. Similarly, the comparator 20 and the market evaluation value calculation means 21 perform the processing based on the market evaluation value of the current mode. In this method, the processing method and the time direction of the processing block and the candidate block differ depending on each mode, and the processing is performed separately because there is a difference in the value of the unit expected vector and the market evaluation value. This enables highly accurate motion detection. The reference mode includes a bidirectional reference mode in which two frames, a temporally old frame and a temporally new frame, are used as reference frames with respect to the processing frame. We can cope by using.
[0051]
As described above, the thinned-out block matching in the wide reference range is completed, and subsequently, block matching using all pixels is started in the narrow reference range centered on the candidate block from which the minimum evaluation value has been derived. Here, the position of the candidate block from which the minimum evaluation value is derived can be easily converted by doubling the x-coordinate and the y-coordinate of the thinning-out candidate block.
[0052]
At the start of block matching using all the pixels, the switches 10 and 11 remain on the terminal w side, and the switch 23 is in the OFF operation because the OUT signal has been released. Further, during execution of block matching using all pixels, the WEIGHT signal is always released, so that the switch 9 remains OFF, and the unit expected vector calculation means 5, the memory 6, the expected vector calculation means 7, the subtraction The processing in the compensator 8, the comparator 20, the market evaluation value calculating means 21 and the memory 22 has no influence.
[0053]
The processing block generating means 1 outputs the processing block to be encoded to the subtractor 12 one pixel at a time via the switch 10 and outputs the screen position of the processing block to the subtractor 3 (S301). The candidate block generation unit 2 sets a new narrow reference range around the candidate block detected by the thinned block matching, and generates the candidate block from the reference range. The candidate block generation means 2 outputs the generated candidate block to the subtractor 12 via the switch 11 one pixel at a time, and outputs the screen position of the candidate block to the subtractor 3 (S302). The subtractor 3 calculates a motion vector by subtracting the screen position of the processing block from the screen position of the candidate block (S303). On the other hand, the subtractor 12 subtracts the pixel of the processing block from the pixel of the candidate block, and the ABS calculator 13 performs an absolute value operation. The output of the ABS calculator 13 is added to the output of the memory 15 in the adder 14 and stored in the memory 15 again. By repeating the above for all m × n pixels, the memory 15 stores the sum of absolute differences DMC between the candidate block and the processing block (S304). The above processing can be represented as in (Equation 6).
[0054]
(Equation 6)

Here, in (Equation 6), (x, y) represents the relative pixel position from the top pixel of the candidate block or the processing block, and Yr (x, y) represents the luminance of the pixel position (x, y) in the candidate block. In the data, Yt (x, y) represents luminance data at a pixel position (x, y) in the processing block.
[0055]
Since the weighting means 17 does not receive the vector offset due to the OFF operation of the switch 9, the block matching is performed this time by always inputting the weighting coefficient = 1 from the weighting table 16 and multiplying the difference absolute value sum DMC in the memory 15 by this time. An evaluation value EST for the processing block and the candidate block is calculated (S305). The above processing can be represented as in (Equation 7).
[0056]
(Equation 7)

The comparator 18 inputs the provisional minimum evaluation value stored in the memory 19 up to the previous time and compares and evaluates it with the current evaluation value EST (S306). When it is determined that the current evaluation value EST is smaller, the memory 19 is updated with the current evaluation value EST as a provisional minimum evaluation value (S307), and the current evaluation value EST is stored in the candidate block generation means 2 and the memory 4. Is output (S308). Upon receiving the MIN signal from the comparator 18, the candidate block generation means 2 sets the current candidate block as a potential candidate for the prediction block. Upon receiving the MIN signal, the memory 4 temporarily stores the current motion vector output from the subtractor 3.
[0057]
The candidate block generation means 2 sequentially outputs the candidate blocks while shifting one pixel at a time within the reference range, and repeats the above processing (S302 to 308), so that the minimum evaluation value within the reference range is stored in the memory 19.
[0058]
When all candidate blocks within the reference range have been processed, an OUT signal is input (S309), so that the switches 10 and 11 are switched to the terminal z side, and the switch 23 performs an ON operation. When the OUT signal is input, the candidate block generation means 2 finally decides the candidate block to which the MIN signal was input last as a prediction block. Then, the candidate block generation means 2 outputs the pixel data of the prediction block, and the processing block generation means 1 outputs the pixel data of the processing block one pixel at a time to the subtractor 24 via the switch 11 or the switch 10. The subtractor 24 generates a pixel difference by subtracting the pixel data of the processing block from the pixel data of the prediction block, and outputs the difference to an encoding unit that executes an encoding process (S310). When the OUT signal is input, the memory 4 outputs the motion vector stored when the MIN signal was last input to the vector encoding unit that executes the vector encoding process via the switch 23 (S311). This motion vector is created from the screen position of the candidate block for which the pixel difference has been calculated and the processing block. Thus, the motion detection in the processing block to be subjected to the current motion detection ends. The other unprocessed processing blocks are similarly processed (S312).
[0059]
As described above, the motion detection method according to Embodiment 1 first performs coarse motion detection (thinning-out block matching) within a wide reference range. In the decimated block matching, the unit expected vector calculation means 5 obtains a unit expected vector Vexpu by smoothing the unit motion vector Vu in the decimated processing block for which motion detection has been performed in the past, and the expected vector calculation means 7 performs the current processing. The expected vector Vexp is calculated by multiplying the time distance DIST between the thinning processing block and the thinning candidate block. The weighting means 17 multiplies the weight coefficient k corresponding to the screen distance between the position indicated by the expected vector Vexp and the position indicated by the current motion vector by the sum of absolute difference DMC (block matching result) in the memory 15. Is an evaluation value EST. The candidate block generation means 2 detects a candidate block having the minimum evaluation value EST from the comparison evaluation by the comparator 18. Thereafter, candidate blocks are sequentially generated within a narrow reference range centered on the screen position of the candidate block from which the minimum evaluation value has been derived by thinning block matching, and detailed motion detection (block matching using all pixels) is performed. Thus, a motion vector is determined.
[0060]
The unit expected vector calculation means 5 calculates the unit expected vector only when the minimum evaluation value is equal to or less than the market evaluation value ESTpr. In other words, only when the minimum evaluation value is highly reliable, the detected motion vector is reflected on the unit expected vector Vexpu, so that a highly accurate expected vector can be obtained.
[0061]
With the above-described method, motion detection can be performed in a wide range, and therefore, even when the subject is moving on the screen at high speed and randomly, highly accurate motion detection can be performed with a small amount of calculation.
[0062]
(Embodiment 2)
Hereinafter, the motion detection method according to the second embodiment will be described with reference to FIGS. 3, 4, 5, 6, 7, and 8. FIG.
[0063]
FIG. 7 is a configuration diagram of a motion detection device using the motion detection method according to Embodiment 2 of the present invention. FIG. 8 is a flowchart showing the thinning block masking according to the first embodiment of the present invention.
[0064]
FIG. 7 shows an embodiment of a motion detecting apparatus using the motion detecting method of the present invention, and those having the same functions as those in FIG. 1 are given the same numbers.
In FIG. 7, a memory 701 is a memory for temporarily storing the sum of absolute difference values input from the memory 15, and a market DMC calculation unit 702 is a memory for storing a value stored in the memory 701 and a value stored in the memory 703. The memory 703 is a memory for temporarily storing the sum of the market difference absolute values input from the market DMC calculating means 702, and the table switching means 704 is a memory 703. Means for comparing the output of the memory 15 with the output of the memory 15 to switch the weight table. The weight table 705 stores a plurality of tables in which weight coefficients are arranged, and is a table for setting the weight table specified by the table switching means 704. is there.
[0065]
The operation of the motion detection device configured as described above will be described below. The flow of processing of the motion detection device in FIG. 7 is shown in the flowcharts of FIGS.
8, the same processes as those in FIG. 2 are denoted by the same reference numerals. As in the first embodiment, the present apparatus first evaluates by thinning-out block matching within a wide reference range, and then blocks using all pixels within a narrow reference range, centering on the candidate block having the smallest evaluation value. The evaluation is performed by matching, and the candidate block having the smallest evaluation value is processed as a motion detection detection result. Here, for convenience of explanation, it is assumed that the thinning-out method thins out one pixel every two pixels at equal intervals in the horizontal direction and the vertical direction. That is, the number of pixels in the horizontal and vertical directions is （(block of m / 2 × n / 2 pixels). For convenience of explanation, it is assumed that the memory 703 stores the sum of the market difference absolute values obtained in the motion detection of the previous thinning processing block. The details of the sum of the market difference absolute values will be described later. Further, for convenience of explanation, it is assumed that the weight table 705 has two types of tables, Table 1 and Table 2.
[0066]
First, thinning block matching within a wide reference range is performed (S200 in FIG. 8). At this time, the switches 10 and 11 are switched to the terminal w side, and the switch 9 is turned on because the WEIGHT signal is input, and the switch 23 is turned off because the OUT signal is released. .
[0067]
The processing block generating means 1 generates a thinning processing block corresponding to the processing block to be encoded from the input processing frame, and outputs the screen position of the thinning processing block (S201). Further, the candidate block generating means 2 generates a thinned reference frame from the input reference frame, generates a thinned candidate block from a range corresponding to the reference range, and outputs the screen position of the thinned candidate block (S202). Thereafter, as in the first embodiment, the vector offset is calculated by the subtractor 3, the memory 6, the expected vector calculation means 7, the subtractor 8, and the switch 9 (S203 to S205). After that, as in the first embodiment, the switch 10, the switch 11, the subtractor 12, the ABS calculator 13, the adder 14, and the memory 15 calculate the absolute difference sum DMC (S 206).
[0068]
The table switching means 704 receives the sum of absolute difference DMC from the memory 15 and the sum of absolute difference DMCpr [k] from the memory 703 and compares them (S801).
[0069]
Here, the market difference absolute value sum DMCpr [k] is a value indicating the market value of the difference absolute value sum DMC in the thinning-out processing block for which the motion has been detected in the past, and can be obtained by (Equation 8).
[0070]
(Equation 8)

Here, k indicates the number of the thinning processing block that performs the current motion detection, DMC [k−1] is the sum of absolute differences in the previous thinning processing block, and DMCpr [k−1] is the sum of the previous thinning processing block. This is the sum of the market difference absolute values. The coefficient e and the coefficient f are constants having a relation of e + f = 1. That is, the market difference absolute value sum DMCpr is obtained by smoothing the difference absolute value sum DMC in the thinning-out processing block for which the motion has been detected in the past. The coefficient e and the coefficient f are values that are statistically determined so as not to be excessively affected by the sum of absolute difference DMCs in the past far from the current thinning processing block. The sum of absolute differences DMC used for calculating the sum of market difference absolute values DMCpr reflects only the sum of absolute differences DMC of the thinning candidate blocks from which the minimum evaluation value is derived in the thinning block matching. As can be seen from (Equation 8), the market difference absolute value sum DMCpr [k] used this time uses the value calculated after the previous motion detection of the thinning processing block.
[0071]
If the total sum of absolute differences DMC calculated this time is smaller than the total sum of absolute differences DMCpr [k], the table switching unit 704 instructs the weight table 705 to use Table 1 (S802), and conversely increases. In this case, the weight table 705 is instructed to use the table 2 (S803). The weight table 705 sets the table designated by the table switching unit 704 from the two types of stored tables. The weighting means 17 inputs the weight coefficient k corresponding to the vector offset from the table set in the weight table 705 (S207), and calculates the evaluation value EST by multiplying the difference absolute value sum DMC stored in the memory 15 by the input. (S208).
[0072]
Here, the two types of tables stored in the weight table 705 are obtained by adjusting the set values A and C, the distance D, and the inclination B (hereinafter, referred to as table parameters) in FIG. 4, FIG. 5, or FIG. It is composed of an array of different weighting factors k. In Table 1, the table parameters are adjusted so as not to depend on the vector offset, and the degree of change of the weight coefficient k is reduced. Conversely, in Table 2, the table parameters are adjusted so as to easily depend on the vector offset, and the degree of change of the weight coefficient k is increased. That is, when it is determined that the difference absolute value sum DMC calculated this time is small and the matching degree of the thinned block matching is high, the influence of the weight coefficient k is reduced and the evaluation is performed. When it is determined that the degree is low, the influence of the weight coefficient k is increased to make it easier to select a position close to the expected vector. In the description, the weight table 705 has two types of tables. However, the weight table 705 is configured to have several types, and in this case, the threshold value to be compared with the sum of absolute differences DMC is, for example, a sum of market absolute differences DMCpr. -G, DMCpr-h, DMCpr + p, DMCpr + q (g, h, p, and q are specified values) and the like. The above processing can be expressed as (Equation 4).
[0073]
The comparator 18 inputs the provisional minimum evaluation value stored in the memory 19 up to the previous time and compares it with the current evaluation value EST (S209). When it is determined that the current evaluation value EST is smaller, the memory 19 is updated with the current evaluation value EST as a provisional minimum evaluation value (S210), and the current evaluation value EST is stored in the candidate block generation means 2, the memory 4, and the memory 701. A MIN signal is transmitted to notify that the evaluation value EST was the minimum (S211).
[0074]
Upon receiving the MIN signal from the comparator 18, the candidate block generation means 2 sets the current thinned candidate block as a strong candidate for motion detection. Upon receiving the MIN signal, the memory 4 temporarily stores the current motion vector output from the subtractor 3. When the memory 701 receives the MIN signal, the memory 701 inputs and stores the sum of absolute difference DMC stored in the memory 15. This sum of absolute difference DMC is the original sum of absolute difference DMC which is compared by the comparator 18 and calculates the evaluation value EST obtained by updating the memory 19 as a new provisional minimum evaluation value.
[0075]
The candidate block generating means 2 sequentially outputs the thinned candidate blocks while shifting them by one pixel in the reference range, and repeats the above processing (S202 to S212), whereby the minimum evaluation value in the reference range is stored in the memory 19, The memory 701 stores the absolute difference sum DMC from which the minimum evaluation value is derived.
[0076]
When all the thinning-out candidate blocks in the reference range are processed, an END signal is input. When the market DMC calculating unit 702 inputs the END signal (S212), a sum of absolute difference DMC [k] corresponding to the minimum evaluation value is stored in the memory. 701, and the current market difference absolute value sum DMCpr [k] is input from the memory 703, and according to (Equation 8), the market difference absolute value sum DMCpr [k + 1] to be used at the next motion detection is calculated. It is stored in the memory 703 (S804). At the same time, in the same manner as in the first embodiment, the output of the CALC signal is controlled in the comparator 20, the market evaluation value calculation means 21, and the memory 22, and the market evaluation value ESTpr [k + 1] to be used at the next motion detection is calculated. (S213). At the same time, when the CALC signal is input (S214), the unit expected vector Vexpu [k + 1] to be used at the time of the next motion detection is calculated in the memory 4, the unit expected vector calculating means 5, and the memory 6, as in the first embodiment. (S215).
[0077]
With the above, the thinned-out block matching within the wide reference range is completed. Then, similarly to the first embodiment, all pixels are used within the narrow reference range around the candidate block from which the minimum evaluation value is derived. The predicted block is determined by performing the block matching, and the pixel difference is output to the encoding unit, and the motion vector is output to the vector encoding unit (S301 to S311 in FIG. 3).
[0078]
As described above, the motion detection in the processing block to be subjected to the current motion detection has been completed, but the other unprocessed processing blocks are processed in the same manner (S200 to 216, 300 to 312).
[0079]
As described above, the motion detection device according to the second embodiment includes the memory 701 for calculating the sum of the market difference absolute values DMCpr, the market DMC calculation unit 702, and the memory 703 in addition to the motion detection device according to the first embodiment. A table switching unit 704 for switching the weight table and a weight table 705 are added, and the table switching unit 704 compares the sum of absolute difference DMC of the candidate block currently being evaluated with the sum of market absolute difference DMCpr. If the sum of absolute differences DMC is smaller, the weight table is used such that the table 1 with a smaller weight coefficient k is used. 705 is issued.
[0080]
In this way, when the reliability of the block matching result is high, the evaluation is hardly affected by the weight coefficient, and when the reliability of the block matching result is low, the evaluation is cheaply affected by the weight coefficient. In addition, since it is possible to perform the optimum evaluation according to the feature, even when the subject moves on the screen at high speed and at random, it is possible to detect the motion with higher accuracy with a small amount of calculation.
[0081]
In the above description, a case has been described in which block matching within a wide reference range is performed on a thinned image, but it may be performed on an image using all pixels that have not been thinned. In this case, the motion vector of each processing block is likely to be equal to the same value, and the vector difference in the vector encoding unit is reduced, so that the encoding efficiency is improved. Also, in this case, there is an effect that accurate motion detection can be performed even for a processing block that moves while being deformed.
[0082]
In the above description, the block matching result is obtained by the sum of absolute difference values DMC, but may be obtained by the sum of squared differences.
In the above description, the calculation of the expected vector Vexp, the calculation of the unit expected vector Vexpu, and the calculation of the market evaluation value ESTpr are performed when the processing block is configured by frame data and when the processing block is configured by only the first field data. Although the calculation is performed separately for the case where the data is included and the case where the data is composed of only the second field data, the calculation may be performed without distinction.
[0083]
In the above description, the comparator 20 outputs the CALC signal to the unit expectation vector calculation means 5 when the minimum evaluation value EST from the memory 19 is equal to or less than the market evaluation value ESTpr stored in the memory 22. Although described, the minimum evaluation value EST may be smaller than the market evaluation value ESTpr, or another threshold may be created based on the market evaluation value ESTpr and compared with the minimum evaluation value EST. As another example of the threshold value, the threshold value is created by adding or subtracting a specified value to or from the market evaluation value ESTpr, so that the calculation condition of the unit expectation vector Vexpu can be made easier or stricter. .
[0084]
Further, in the above description, the weight table 16 or 705 in which the weight coefficients k are arranged is obtained by the two-dimensional table of the x component and the y component of the vector offset, but the vector offset length is obtained as a parameter. You may.
[0085]
In the above description, the weighting means 17 multiplies the difference absolute value sum value DMC, which is the block matching result stored in the memory 15, by the weighting coefficient k. However, the weighting may be performed by adding. .
[0086]
In the above description, an example has been described in which the weight coefficient k in the weight tables shown in FIGS. 4, 5, and 6 increases concentrically around the position (0, 0) indicated by the expected vector Vexp. Alternatively, a non-concentric table or the like may be used.
[0087]
In the description of the second embodiment, the plurality of tables stored in the weight table 705 are configured such that at least one of the table parameters is different. Such a table may be provided and configured to be switched.
[0088]
In the above description, an example in which the present invention is implemented by hardware has been described. However, it is apparent that the same effect can be obtained by implementing the present invention by software. In this case, the software can be easily applied by configuring the software according to the flowcharts shown in FIG. 2, FIG. 3, and FIG.
[0089]
【The invention's effect】
As described above, according to the motion detection method of the present invention, the next processing block that performs motion detection performs motion detection based on the expected vector obtained from the motion vector of the processing block whose motion detection has been completed in the past. As a result, a remarkable effect that a highly accurate motion can be detected with a small amount of calculation even when the subject is moving on the screen at high speed can be obtained.
[0090]
In addition, by calculating an expected vector that accurately reflects the time distance between the processing block and the candidate block, a remarkable effect that more accurate motion detection becomes possible can be obtained.
[0091]
In addition, the unit expected vector for calculating the expected vector not only estimates the average motion of the entire frame but also estimates the motion near the processing block around which the motion is being detected. High-precision motion detection even when the subject is moving quickly and randomly on the screen by performing arithmetic synthesis so that the processing block that is performing a specific random motion is not affected by the processing block Is obtained.
[0092]
Also, whether the motion detection result of the processing block of interest is a highly reliable result is determined based on the minimum evaluation value and the first threshold, and the motion of the processing block of interest only when the reliability is high. By reflecting the vector in the unit expectation vector, a remarkable effect that higher-precision motion detection becomes possible is obtained.
[0093]
Also, the first threshold value is not excessively affected by the minimum evaluation value of the processing block on which motion detection has been performed in the distant past, and the first threshold value places importance on the minimum evaluation value near the processing block on which motion detection is performed. In comparison with the minimum evaluation values of other processing blocks, the calculation is performed so as not to be affected by the specific minimum evaluation value, so that the reliability of the motion detection result can be accurately determined. The effect is obtained.
[0094]
Further, the expected vector, the unit expected vector, or the first threshold is separately calculated in the case of frame prediction, the case of field prediction in the first field data, and the case of field prediction in the second field data. By doing so, a remarkable effect is obtained in that more accurate motion detection is possible.
[0095]
Further, since the expectation vector, the unit expectation vector, or the first threshold value is separately calculated according to the time direction of the candidate block with respect to the processing block, a remarkable effect that more accurate motion detection becomes possible is obtained. .
[0096]
In addition, a candidate block having a motion vector corresponding to an expected vector can be easily selected by using a result obtained by performing weighting corresponding to a screen distance from a position indicated by an expected vector on a result of block matching with the candidate block as an evaluation value. Also, a remarkable effect is obtained that even when the subject is moving at random on the screen at high speed, highly accurate motion detection can be performed with a small amount of calculation.
[0097]
Also, the evaluation value is calculated by adding or multiplying the set value corresponding to the screen distance to the block matching result, so that it is easy to select a candidate block having a motion vector corresponding to the expected vector, and the subject can be quickly and randomly selected. A remarkable effect that highly accurate motion detection can be performed with a small amount of calculation even when the screen is moving is obtained.
[0098]
Also, by setting the weighting value smaller as the screen distance becomes longer around the position indicated by the expected vector, it becomes easy to select a candidate block having a motion vector having a value close to the expected vector. In addition, a remarkable effect that highly accurate motion detection can be performed with a small amount of calculation even when the screen is randomly moved is obtained.
[0099]
Further, when the motion vector is a value close to the expected vector, by setting the weighting set value to a predetermined fixed value, the evaluation can be performed relatively evenly, and is not affected by some error of the expected vector. A remarkable effect that highly accurate motion detection becomes possible is obtained.
[0100]
Further, when the difference between the motion vector and the expected vector is equal to or more than a predetermined value, by setting the set value of the weight to a predetermined fixed value, the motion at a position far from the position indicated by the expected vector by a predetermined range or more By applying a predetermined weight to a candidate block having a vector, a remarkable effect is obtained in that accurate motion detection becomes possible even for a processing block having a random motion different from other processing blocks. .
[0101]
Also, the set value of the fixed weighting is focused on by setting the result of comparison between the current block matching result and a threshold value obtained by smoothing one or two or more block matching results determined in the past. When the subject is moving quickly and randomly on the screen because the weighting factor can be adjusted according to the reliability of the block matching result of the processing block and the optimal evaluation according to the characteristics of each processing block can be performed. However, a remarkable effect that more accurate motion detection becomes possible with a small amount of calculation is obtained.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a motion detection device using a motion detection method according to a first embodiment of the present invention.
FIG. 2 is a flowchart showing thinning-out block masking according to the first embodiment of the present invention;
FIG. 3 is a flowchart showing block masking using all pixels according to the first embodiment of the present invention;
FIG. 4 is a diagram showing a weight table;
FIG. 5 is a diagram showing a weight table in which weight coefficients of vector offsets equal to or more than a certain value are fixed.
FIG. 6 is a diagram showing a weight table in which weight coefficients of vector offsets in a certain range are fixed.
FIG. 7 is a configuration diagram of a motion detection device using a motion detection method according to Embodiment 2 of the present invention.
FIG. 8 is a flowchart showing thinning block masking according to the first embodiment of the present invention.
FIG. 9 is a conceptual diagram illustrating a conventional motion detection method using thinned data.
[Explanation of symbols]
1 Processing block generation means
2 Candidate block generation means
3 Subtractor
4 memory
5 Unit expected vector calculation means
6 memory
7 Expected vector calculation means
8 Subtractor
9 Switch
10 switches
11 switch
12 Subtractor
13 ABS computing unit
14 Adder
15 Memory
16 Weight table
17 Weighting means
18 Comparators
19 Memory
20 Comparators
21 Market evaluation value calculation means
22 memory
23 switch
24 Subtractor
701 memory
702 Market DMC calculation means
703 memory
704 Table switching means
705 weight table

Claims (13)

In a motion detection method for obtaining a motion vector by block matching between a processing block to be subjected to motion detection and a candidate block created from within a reference range,
A motion detection method, comprising: calculating an expected vector from motion vectors of one or more past processing blocks obtained in the past; and calculating a motion vector of the processing block from the expected vector. The said expectation vector changed into the magnitude | size corresponding to the time distance of the said process block and a candidate block based on the unit expectation vector which shows the magnitude | size per unit time, The said expectation vector is characterized by the above-mentioned. Motion detection method. 3. The motion according to claim 1, wherein the unit expectation vector is smoothed in consideration of an influence degree of each of motion vectors of one or more past processing blocks obtained in the past. Detection method. If the minimum evaluation value determined by block matching in the processing block is equal to or less than a first threshold, the expected vector is calculated and used. If the minimum evaluation value is greater than or equal to the first threshold, 4. The motion detection method according to claim 1, wherein the expected vector used in the previous detection is used. 5. The motion detection method according to claim 4, wherein the first threshold is smoothed in consideration of the degree of influence of each of minimum evaluation values of one or more processing blocks obtained in the past. The expectation vector, the unit expectation vector, and the first threshold are calculated from data obtained by combining first field data and second field data that are temporally different from each other, and using only the first field data. 6. The calculation according to claim 1, wherein the calculation is performed using only the second field data, and the calculation is performed using only the second field data. Motion detection method. The expected vector, the unit expected vector, and the first threshold are calculated using a temporally old candidate block for the processing block, and calculated using a temporally new candidate block for the processing block. 7. The motion detection method according to claim 1, wherein the motion detection is performed. 5. An evaluation value in the block matching is calculated in consideration of a weight corresponding to a screen distance from a position indicated by an expectation vector. The motion detection method according to claim 5 or claim 6. 9. The motion detection method according to claim 8, wherein the calculation of the evaluation value is performed by adding or multiplying the block matching result by a set value corresponding to a screen distance from a position indicated by the expected vector. The setting value of the weighting is set from a first data table or a calculation formula that is harder to be selected as the screen distance becomes longer with the position indicated by the expected vector as a center. The motion detection method according to any one of the above. When the motion vector is a value closer to the first predetermined value around the expected vector, the set value of the weight is set to a second value having a gentler slope or a fixed value according to the first data table or formula. 11. The motion detection method according to claim 8, wherein the motion is set from a data table or a calculation formula. When the difference between the motion vector and the expected vector is equal to or more than a second predetermined value, the weighted set value is set to a third data table having a slope that is gentler or a fixed value than the first table or the calculation formula. 12. The motion detection method according to claim 8, wherein the motion detection method is set by a calculation formula. The first, second, or third data table or the set value of the weighted formula is calculated by comparing the current block matching result with a threshold value obtained by smoothing one or more block matching results determined in the past. 13. The motion detection method according to claim 10, wherein the adjustment is performed based on a comparison result.

Images