JP4092773B2 - Method and apparatus for converting the number of frames of an image signal - Google Patents

Description

【００３６】
（数１）で、Ｓ２（ｘ，ｙ）は算出領域内の現フレームの画素の信号、Ｓ３（ＢＶ）は動きベクトルＢＶで位置を移動させた前フレームの画素の信号、ａｂｓ｛ ｝は絶対値、Σは算出領域内の画素の総和、ＢＶｘは動きベクトルＢＶのｘ成分、ＢＶｙはｙ成分である。
【００３７】
画素動きベクトル設定部２１は、図１０の第２ステップの閾値未満および第３ステップの信号処理を行う。すなわち、制御信号ＰＣ２が閾値未満を示すブロックは、ブロック内の画素に対して現ブロックの動きベクトルＢＶを画素単位動きベクトルＰＶに出力する。一方、閾値以上のブロックは、各ミニブロック内の画素に対して動き補正誤差ＥＲ０、ＥＲ１、…、ＥＲＮの内で最少値をとる動きベクトルを画素単位動きベクトルＰＶに出力する。
また、制御信号ＰＣ２で動き検出信号ＭＤ２が０の画素に対しては、画素単位動きベクトルＰＶに０を出力する。
この結果、画素単位の動きベクトルを高精度、かつ、全探索に較べて必要な演算量を１桁〜２桁程度低減できる。
【００３８】
図１２は画素単位動きベクトル生成部６の第２構成例を示す図、図１３は第２構成例における画素単位動きベクトル生成処理のフローチャートである。この第２の構成例は、ブロックミニブロック予測誤差適応探索処理を行うに好適なものである。
【００３９】
図１２に示す第２構成例においては、補正誤差算出部２２及び制御部１８における動作が第１の構成例と異なる。
同図において、補正誤差算出部２２は、図１３に示す信号処理フローチャートの第１ステップの処理を行う。すなわち、現フレームの信号Ｓ２と前フレームの信号Ｓ３の輝度信号成分に対し、ブロック単位動きベクトルＢＶによるブロック動き補正誤差ＰＭ（画素換算相当）とミニブロック動き補正誤差ＭＢＥ（画素換算相当）を出力する。
制御部１８は、信号ＰＭ、ＭＢＥと動き検出信号ＭＤ２をもとに、図１３の第２、第３ステップの信号処理に必要な制御信号ＰＣ１、ＰＣ２を生成する。まず、ブロック動き補正誤差ＰＭの値に応じて閾値ＴＨの値をブロック毎に設定する。例えば、ＰＭ＜＝８ではＴＨ＝１６、８＜ＰＭ＜＝１２ではＴＨ＝１２、１２＜ＰＭ＜＝１６ではＴＨ＝８、ＰＭ＞１６ではＴＨ＝４に設定する。そして、ミニブロック動き補正誤差ＭＢＥと閾値ＴＨを比較し、閾値を越えるミニブロックに対しては、第２ステップの閾値以上のミニブロック分割探索の信号処理、閾値未満のミニブロックに対しては第２ステップの閾値未満の信号処理を行うように制御信号を生成する。さらに、動き検出信号ＭＤ２が０か１かに応じて第３ステップの信号処理に必要な制御信号を生成する。
なお、参照動きベクトル生成部１９、補正誤差算出部２０、画素動きベクトル設定部２１は、第１の構成例と同様の動作を行うもので、説明は省略する。
【００４０】
図１４は画素単位動きベクトル生成部６の第３構成例を示す図、図１５は第３構成例における画素単位動きベクトル生成処理のフローチャートである。この第３の構成例は、Ｖエッジ適応探索処理を行うに好適なものである。
【００４１】
図１４はＶエッジ適応探索処理を第１構成例（図９）に適用した場合のブロック構成で、Ｖエッジ検出部２３と制御部２６の動作が第１構成例と異なる。
同図においてＶエッジ検出部２３は、水平周波数の低域成分を通過域とするローパス特性のフィルタＨＬＰＦ２４と垂直周波数の高域成分を通過域とするハイパス特性のフィルタＶＨＰＦ２５との組み合わせで構成し、現フレームの信号Ｓ２の輝度信号成分より水平低域・垂直高域のＶエッジ信号ＶＥＧを抽出する。
制御部２６は、ブロック単位動きベクトルＢＶによる動き補正誤差の値が閾値ＴＨ未満か以上かを示す信号ＰＭＢとＶエッジ信号ＶＥＧと動き検出信号ＭＤ２をもとに、図１５の第２、第３ステップの信号処理に必要な制御信号ＰＣ１、ＰＣ２を生成する。
【００４２】
そして、図１５に示すように、第２ステップの閾値以上の時のミニブロック分割探索の信号処理を異にする。すなわち、Ｖエッジ領域を含むミニブロックでは、ミニブロック（水平ＭＢＸ、垂直ＭＢＹ）に対して、これを内包する水平ＭＢＸ＋４、垂直ＭＢＹでの算出領域で動き補正誤差を算出し、この値が最小な参照ベクトルで画素単位の動きベクトルＰＶを生成する。
また、同一のＶエッジ領域に含まれる画素に対しては、動きベクトルの修正処理を行う。すなわち、これら画素の動きベクトルを平滑化処理（例えば平均化）で修正し、この修正した動きベクトルをＶエッジ領域の画素に割り当てる。
【００４３】
一方、Ｖエッジ領域以外のノ−マル領域では、第１の構成例と同一の動作を行う。
なお、ここに述べたＶエッジ適応探索処理は、第２の構成例（図１２）に適用することもできる。
以上に述べた如く、動きベクトル探索部は、上述のブロック単位動きベクトル探索部と、画素単位動きベクトル生成部との組み合わせで種々の構成が可能である。
【００４４】
さて、後述する動き補正信号処理部でフレーム単位の動き補正処理を行う場合に好適な動きベクトル探索部の構成例について、次に説明する。図１６は動きベクトル探索部の第２構成例を示す図、図１７はこの第２構成例における動きベクトル探索のフローチャートである。
【００４５】
図１６において、ブロックマッチング第１処理部１３、ブロックマッチング第２処理部１４、ベクトル分布計測部１６は前述のブロック単位動きベクトル探索部の第２の構成例（図６）と同一の機能、動作を行う。すなわち、図１７に示すように、動画ブロックに対しては探索モード信号ＭＯＤ１〜ＭＯＤ５による探索処理（図８参照）でブロック単位の動きベクトルＢＶの検出を行う。また、静止ブロックに対しては動きベクトルＢＶに０を設定する。
なお、ベクトル分布計測部１６は、図８に示す領域Ｅの発生頻度が設定値（例えば１０％）を越える時はミニブロック分割探索モード信号ＭＢＦＧに０、設定値未満の時は１を出力する。
【００４６】
また、補正誤差算出部１７、ミニブロック分割探索部２７は、前述の画素単位動きベクトル生成部の第１〜第３の構成例と同様に構成する。そして、モード信号ＭＢＦＧが１の時は、前述した第１〜第３の構成例と同様なミニブロック分割探索の信号処理を行い、画素単位の動きベクトルＰＶを生成する。一方、モード信号ＭＢＦＧが０の時は、後段の動き補正信号処理部では動き補正フレーム内挿処理を行わないため、画素単位の動きベクトルは不要であり、ミニブロック分割探索の信号処理を中止する。
なお、モーションジャダー妨害が目立ちやすい画像（動き補正フレーム内挿処理が必須な画像）は番組全体の１割程度であるため、ミニブロック分割探索処理に要する演算量は、前述の第１〜第３の構成例に較べ、更に１桁程度低減することができる。
また、制御部２８１は、上述の信号処理の動作に必要な制御信号類を生成するものである。
【００４７】
以上で、動きベクトル探索部の説明を終え、次に、動き補正信号処理部７の動き補正内挿フレーム生成部８の構成について説明する。図１８は動き補正内挿フレーム生成部８の第１構成例を示す図、図１９はＭＣベクトル生成部２８及び動き補正信号生成部３０、３１の動作を説明するための図である。この第１構成例は、予測誤差適応動き補正処理を行うに好適なものである。
【００４８】
ＭＣベクトル生成部２８は、画素単位動きベクトルＰＶより動き補正処理に必要な動き補正ベクトルＶｃｔとＶｐｒを生成する。フレーム周波数が５０Ｈｚの信号をフレーム周波数が６０Ｈｚの信号にフレーム数変換する場合を例に、この動作概略を図１９（ａ）〜（ｃ）に示す。フレーム周波数５０Ｈｚの順次走査のフレーム順の１から５の信号に対し、同図（ａ）に示すように、動き補正の信号処理で内挿フレームの信号を生成し、フレーム順１から６のフレーム周波数が６０Ｈｚの順次走査の信号に変換する。この際、動き補正ベクトルは、内挿するフレーム位置と合致させる必要がある。そこで、同図（ｂ）に示す様に、画素単位の動きベクトルＰＶに係数加重する係数値ｋａ，ｋｂをフレーム順に逐次変化させ、（数２）に示す演算で動き補正ベクトルＶｐｒとＶｃｔを生成する。
【００４９】
【数２】
Ｖｐｒ＝ＰＶ＊ｋａ／（ｋａ＋ｋｂ）
Ｖｃｔ＝−ＰＶ＊ｋｂ／（ｋａ＋ｋｂ）・・・・・・・・（数２）
【００５０】
従って、フレーム順２の内挿フレームは、Ｖｐｒ＝ＰＶ＊５／６，Ｖｃｔ＝−ＰＶ＊１／６（ｋａ＝５，ｋｂ＝１）、３のものは、Ｖｐｒ＝ＰＶ＊４／６，Ｖｃｔ＝−ＰＶ＊２／６（ｋａ＝４，ｋｂ＝２）…の如く、時間方向での位置ずれがない動き補正ベクトルを生成する。
【００５１】
動き補正信号生成部３０では、現フレームの信号Ｓ２と動き補正ベクトルＶｃｔで動き補正信号Ｓｃｔを生成する。また、動き補正信号生成部３１は、前フレームの信号Ｓ３と動き補正ベクトルＶｐｒで動き補正信号Ｓｐｒを生成する。この動作概略を図１９（ｃ）に示す。内挿フレームの点Ａ（ｘ，ｙ）の信号は、前フレームの信号Ｓ３では、点Ａ（ｘ，ｙ）を動き補正ベクトルＶｐｒ（水平方向成分Ｖｐｒｘ，垂直方向成分Ｖｐｒｙ）で移動させた点Ａ’（ｘ１，ｙ１）＝（ｘ＋Ｖｐｒｘ，ｙ＋Ｖｐｒｙ）の位置の信号、現フレームの信号Ｓ２では点Ａ（ｘ，ｙ）を動き補正ベクトルＶｃｔ（水平方向成分Ｖｃｔｘ，垂直方向成分Ｖｃｔｙ）で移動させた点Ａ”（ｘ２，ｙ２）＝（ｘ−Ｖｃｔｘ，ｙ−Ｖｃｔｙ）の位置の信号に対応する。従って、動き補正信号ＳｐｒとＳｃｔは、以下に示す（数３）で生成する。
【００５２】
【数３】
Ｓｐｒ＝Ｓ３（ｘ＋Ｖｐｒｘ，ｙ＋Ｖｐｒｙ）
Ｓｃｔ＝Ｓ２（ｘ−Ｖｃｔｘ，ｙ−Ｖｃｔｙ）・・・・・・（数３）
【００５３】
この信号処理は、動き補正信号生成部に内蔵のメモリ回路の読み出し動作を制御することで実現する。すなわち、メモリ読み出しのアドレスを補正補間動きベクトルＶｐｒ，Ｖｃｔに相当する位置だけずらしたアドレスを生成し、このアドレスで点Ａ’，Ａ”に対応した画素の信号を読み出す。
【００５４】
加算部３２は、両者の動き補正信号ＳｐｒとＳｃｔとの加算平均を行い、その出力に図１９（ｂ）に示したフレーム順２，３，…，６の動き補正内挿フレームの信号Ｓｍを得る。
減算部３３は、動き補正信号ＳｃｔとＳｐｒとの減算を行い、差分信号成分ＭＣＥを検出する。動き補正ベクトルが正確な場合は、動き補正信号ＳｃｔとＳｐｒは同じ値をとるので信号ＭＣＥは０になる。しかしながら、動き補正ベクトルが不正確な場合は、動き補正信号ＳｃｔとＳｐｒとで異なる値になり、信号ＭＣＥは零以外の値を持つ。
補正誤差算出部２９は、ミニブロック単位の動き補正誤差（画素換算相当）を算出し、この値を信号ＭＢＥに出力する。
【００５５】
判定部３４は、信号ＭＢＥの値の大小に応じて閾値ＴＨの値を適応的に設定する。例えば、ＭＢＥ＜＝８ではＴＨ＝４０、８＜ＭＢＥ＜＝１６ではＴＨ＝３２、１６＜ＭＢＥ＜＝２４ではＴＨ＝２４、２４＜ＭＢＥではＴＨ＝１６に設定する。そして、信号ＭＣＥが閾値未満の時は０、閾値以上の時は１を制御信号ＣＴに出力する。
【００５６】
選択部３５は、制御信号ＣＴが０の時は動き補正内挿フレームの信号Ｓｍ、制御信号ＣＴが１の時は現フレームの信号Ｓ２と前フレームの信号Ｓ３のうちで動き補正内挿フレームと時間的な距離の近い信号を選択し、内挿フレームの信号ＳＭＣを出力する。この操作で、動きベクトルの不正確さに起因する孤立点的な劣化や動画エッジ周縁部の劣化などの動き補正に固有な画質劣化を大幅に抑圧することができ、高画質化が達成できる。
【００５７】
図２０は動き補正内挿フレーム生成部８の第２構成例を示す図で、動き速度適応動き補正処理を行うに好適なものである。この第２構成例は、第１の構成例（図１８）にベクトル分布計測部３６を追加した構成で実現する。
ベクトル分布計測部３６では、画素単位の動きベクトルＰＶのうち図８（ａ）に示した領域Ｅに相当する動きベクトルの１フレーム期間での発生頻度を計測し、これが設定値（例えば１０％以上）を越える場合は動き補正モード信号ＭＣＦＧに０、設定値未満の場合は１を出力する。
判定部３４は、動き補正モード信号ＭＣＦＧが０の時は、選択部３５で該当する１フレーム期間は現フレームの信号Ｓ２を選択制御する制御信号ＣＴを出力する。一方、動き補正モード信号ＭＣＦＧが１の時は、第１の構成例と同様の動作を行い、信号ＭＣＥが信号ＭＢＥで定まる閾値以上では１、未満では０の制御信号ＣＴを出力する。
【００５８】
なお、本構成では、ベクトル分布計測部３６を省略し、代わりに、前述した図１６の動きベクトル探索部におけるミニブロック分割探索モード信号ＭＢＦＧを動き補正モード信号として使用する構成で実現することも可能である。
以上に述べた如く、本発明の第１の実施例によれば、動きベクトル探索、生成の信号処理に要する演算量が少なく、かつ、動き補正処理に起因する画質劣化の少ない画像信号の動き補正フレーム数変換装置が実現できる。そして、高画質化、低コスト化に顕著な効果を得ることができる。
【００５９】
次に、本発明の第２の実施例について、図２１〜図２２を用いて説明する。
図２１は本実施例のブロック構成図で、先の第１の実施例（図１）の動き補正信号処理部のメモリ部９を省略し、代わりにフレームレートアップ部３７をＩＰ変換部１と１フレーム遅延部２との間に配置して構成する。
ＩＰ変換部１で飛び越し〜順次の走査変換した順次走査の信号系列Ｓ２は、フレームレートアップ部３７でコマ繰り返し処理でフレーム数変換を行い、例えば、フレーム周波数５０Ｈｚの信号系列Ｓ２をフレーム周波数６０Ｈｚの信号系列Ｓ５に変換する。
【００６０】
この動作概略を図２２に示す。Ｓ２（５０Ｈｚ）フレーム順１，２，３，４，５，１，２，…の信号系列に対し、このフレーム順１のフレームを２度繰り返して、Ｓ５（６０Ｈｚ）のフレーム順１，１，２，３，４，５，１，１，２，…の信号系列に変換する。そして、このＳ５を現フレームの信号系列、これを１フレーム遅延部２で１フレーム期間遅延させた信号Ｓ６を前フレームの信号系列として、動きベクトル探索部３と動き補正信号処理部７で第１の実施例と同様の信号処理を行う。従って、現フレームと前フレームの信号系列が同一フレーム順１の場合は静止画像となり、フレームの全ての動きベクトルが０の処理が行われ、出力の信号系列Ｓ４（６０Ｈｚ）はフレーム順１の信号を得る。一方、現フレームと前フレームの信号系列が異なるフレーム順の時は、出力の信号系列Ｓ４は動き補正処理で生成した信号系列（図中にＭＣで表示）を得る。
【００６１】
本発明の第２の実施例によれば、第１の実施例と同様、動きベクトル探索、生成の信号処理に要する演算量が少なく、かつ、動き補正処理に起因する画質劣化の少ない画像信号の動き補正フレーム数変換装置が実現できる。そして、高画質化、低コスト化に顕著な効果を得ることができる。
【００６２】
次に、本発明の第３の実施例について、図２３〜２５を用いて説明する。なお、本実施例は、動きベクトル探索において変換ベクトル探索処理を行うに好適なものである。
図２３は本実施例のブロック構成図であり、ブロック単位動きベクトル探索部３８の構成、動作のみが先の第１の実施例（図１）と異なる。
図２４は、この構成を示す図である。図２４において、Ｐベクトル変換部３９、Ｂベクトル変換部４０は、画像符号化で使用する動きベクトル情報ＭＶのベクトル変換処理を行い、それぞれ１フレーム当たりの変換ベクトルＭＶｐとＭＶｂを生成する。
【００６３】
図２５はこの動作概略を示す図である。画像符号化、特に、国際標準のＭＰＥＧビデオ符号化では、同図（ａ）に示すように、画像信号シ−ケンスをＩピクチャ、Ｐピクチャ、Ｂピクチャに分け、Ｉピクチャではフレーム内ＤＣＴ（離散コサイン変換）符号化、Ｐピクチャでは一方向ＭＣ符号化＋ＤＣＴ符号化、Ｂピクチャでは双方向ＭＣ符号化＋ＤＣＴ符号化を行う。
同図（ｂ）は、Ｐピクチャの符号化に使用する動きベクトルＰＶ１，ＰＶ２，…を示す。この動きベクトルは画像信号シ−ケンスのＰピクチャ間のｎフレーム（図ではｎ＝３）での動きベクトルに相当する。従って、Ｐベクトル変換部３９では、この動きベクトルを１／ｎ（図では１／３）に変換したＰＶ１／３，ＰＶ２／３，…で変換ベクトルＭＶｐを生成する。
同図（ｃ）はＢピクチャの符号化に使用する動きベクトルＢＶ１１，ＢＶ１２、ＢＶ２１，ＢＶ２２…を示す。このうち、ＢＶ１１，ＢＶ２１，ＢＶ３１，…は１フレーム間の動きベクトルに相当している。従って、Ｂベクトル変換部４０では、これらの動きベクトルで変換動きベクトルＭＶｂを生成する。
【００６４】
図２４に戻り、判定部４１では、動き検出信号ＭＤ１が１の動画ブロックに対し、変換ベクトルＭＶｐ，ＭＶｂによるブロック単位の動き補正誤差を算出し、値の小さいものを代表動きベクトルＢＶＴに出力する。
ブロックマッチング第２処理部１４は、代表動きベクトルＢＶＴを起点とする再探索処理を行い、ブロック単位の動きベクトルＢＶを検出する。なお、静止ブロックに対しては、ＢＶに０を出力する。
制御部４２は、これらの動作に必要な制御信号類を生成する。
本発明の第３の実施例によれば、第１、第２の実施例に較べて、動きベクトル探索、生成の信号処理に要する演算量が更に少ない画像信号の動き補正フレーム数変換装置が実現できる。そして、高画質化、低コスト化に顕著な効果を得ることができる。
【００６５】
次に、本発明の第４の実施例を図２６のブロック構成図に示す。本実施例は、先の第３の実施例（図２３）の動き補正信号処理部のメモリ部９を省略し、代わりに第２の実施例（図２１）に示したフレームレートアップ部３７をＩＰ変換部１と１フレーム遅延部２との間に配置して構成する。なお、この動作は前述の第１〜第３の実施例から容易に理解できるため説明は省略する。
本発明の第４の実施例によれば、第３の実施例と同様、動きベクトル探索、生成の信号処理に要する演算量が更に少ない画像信号の動き補正フレーム数変換装置が実現できる。そして、高画質化、低コスト化に顕著な効果を得ることができる。
【００６６】
次に、本発明の第５の実施例を図２７〜図２８を用いて説明する。本実施例は、シーンチェンジの領域で動きベクトル探索や動き補正フレーム内挿の信号処理を中止する動作を行うに好適なものである。
図２７はこのブロック構成図であり、第１の実施例（図１）の構成にシーンチェンジ検出部３９１を追加して実現する。
【００６７】
シーンチェンジ検出部３９１は、１フレーム期間でのフレーム間差分信号成分の発生形態をもとにシーンチェンジの発生したフレームを検出する動作を行う。この一構成例を図２８に示す。減算部４０１は、現フレームの信号Ｓ２と前フレームの信号Ｓ３の輝度信号成分の減算演算を行い、１フレーム間の差分成分ＦＤを抽出する。一般に、シーンチェンジの領域では、画像の内容が切り替わるため、差分成分ＦＤの信号レベルは比較的大きな値を持つ。そこで、２値量子化部４１１は比較的高いレベルの閾値±Ｔｈｂで信号ＦＤを画素毎に２値量子化する。そして、閾値±Ｔｈｂ未満の画素は０、閾値を越える画素は１を信号ＱＳに出力する。１フレーム累積部４２１は、信号ＱＳが１のものの画素の数を１フレーム期間で計測し、１フレーム期間の累積値ＡＱを出力する。判定部４３１は、累積値ＡＱの値が全画面の半分以上の画素数で、かつ、その発生が１フレーム期間に限られる場合をシーンチェンジの発生したフレームと判定する。この理由は、画面全体が一様な速度で動く水平パンや上下パンの動きを誤ってシーンチェンジと検出する誤動作を避けるためである。そして、信号ＳＣＦＧにシーンチェンジの発生したフレームでは１、それ以外のフレームでは０を出力する。
【００６８】
図２７に戻り、動きベクトル探索部３と動き補正信号処理部７は、信号ＳＣＦＧが１のシーンチェンジが発生したフレームでは、動きベクトル探索や動き補正フレーム内挿の信号処理を中止する。そして、このフレームでは現フレームの信号Ｓ２を信号系列Ｓ４に出力する。
一方、信号ＳＣＦＧが０のフレームでは、動きベクトル探索部３と動き補正信号処理部７は、第１の実施例と同一の信号処理の動作を行い、信号系列Ｓ４では動き補正フレーム内挿処理でフレーム数を変換した信号系列を得る。
本発明の第５の実施例によれば、シーンチェンジ領域における動きベクトル探索や生成のための膨大な演算量の発生が回避でき、第１の実施例に較べて、動きベクトル探索、生成の信号処理に要する演算量が更に少ない画像信号の動き補正フレーム数変換装置が実現できる。そして、高画質化、低コスト化に顕著な効果を得ることができる。
【００６９】
次に、本発明の第６〜第８の実施例を図２９〜図３１の図面で説明する。これらは、いずれもシーンチェンジの領域では動きベクトル探索や動き補正フレーム内挿の信号処理を中止する動作を行うに好適なものである。
図２９は第６の実施例のブロック構成例図で、第２の実施例（図２１）の構成にシーンチェンジ検出部３９１を追加して実現したものである。
図３０は第７の実施例のブロック構成例図で、これは第３の実施例（図２３）の構成にシーンチェンジ検出部３９１を追加して実現したものである。
また、図３１は第８の実施例のブロック構成例図で、これは第４の実施例（図２６）の構成にシーンチェンジ検出部３９１を追加して実現したものである。
なお、これら実施例における動作はこれまでの説明で容易に理解できるため、説明は省略する。
本発明の第６〜第８の実施例によれば、第５の実施例と同様、動きベクトル探索、生成の信号処理に要する演算量が更に少ない画像信号の動き補正フレーム数変換装置が実現できる。そして、高画質化、低コスト化に顕著な効果を得ることができる。
【００７０】
本発明の第９の実施例について、図３２〜図３７を用いて説明する。
図３２は、本実施例のブロック構成図である。同図に示すように、先の第１の実施例（図１）では、ＩＰ変換部を、動きベクトル探索部の前段に配置して構成するのに対して、本実施例では、その後段に配置して構成する。
１フレーム遅延部２は、飛び越し走査の入力画像信号Ｓ１（輝度信号成分と色差信号成分）から、これを１フレーム期間遅延させた信号Ｓ２０を生成する。この信号Ｓ１及びＳ２０は、動きベクトル探索部３と動き補正信号処理部７に供給される。
【００７１】
動きベクトル探索部３は、動き検出部４、ブロック単位動きベクトル探索部５、画素単位動きベクトル生成部６で構成する。
動き検出部４は、現フレームの信号系列Ｓ１と、前フレームの信号系列Ｓ２０の輝度信号成分との減算処理で１フレーム間の差分信号を抽出し、これを２値量子化して動き検出信号ＭＤ１、ＭＤ２を出力する。この構成及び動作は、先に説明した第１の実施例の動き検出部（図２）と同様である。但し同図及びその説明において、Ｓ２をＳ１、Ｓ３をＳ２０と読み替えるものとする。
【００７２】
ブロック単位動きベクトル探索部５は、ブロック単位（例えば１６画素ｘ１６ラインあるいは８画素ｘ８ライン）で動きベクトルを検出する。すなわち、動き検出信号ＭＤ１が０のブロックは静止ブロックと判定し、ブロック単位動きベクトルＢＶに０を出力する。一方、ＭＤ１が１のブロックは動画ブロックと判定し、信号Ｓ１とＳ２０の輝度信号成分に対して、前述した２重代表点ツリー探索処理あるいは動きベクトル分布適応探索処理を行い、ブロック単位動きベクトルＢＶを検出する。この構成及び動作は、先に説明した第１の実施例のブロック単位動きベクトル探索部（図３〜図８）と同様である。但し同図及びその説明において、Ｓ２をＳ１、Ｓ３をＳ２０とそれぞれ読み替えるものとする。
【００７３】
画素単位動きベクトル生成部６は、ミニブロック分割探索処理で画素単位動きベクトルＰＶを生成する。すなわち、前述したブロック予測誤差適応探索処理、ブロックミニブロック予測誤差適応探索処理あるいはＶエッジ適応探索処理でミニブロック単位（例えば２画素ｘ２ライン）で画素単位動きベクトルＰＶを検出する。また、動き検出信号ＭＤ２が０の画素は静止画素と判定してＰＶに０を割り当てる。この構成及び動作は、先に説明した第１の実施例の画素単位動きベクトル生成部（図９〜図１５）と同様である。但し同図及びその説明において、Ｓ２をＳ１、Ｓ３をＳ２０とそれぞれ読み替えるものとする。
【００７４】
また、後述する動き補正信号処理部でフレーム単位の動き補正処理を行う場合に好適な動きベクトル探索部の構成及び動作についても、先に説明した第１の実施例の動きベクトル探索部（図１６〜図１７）と同様である。但し同図及びその説明において、Ｓ２をＳ１、Ｓ３をＳ２０とそれぞれ読み替えるものとする。
【００７５】
動き補正信号処理部７は、ＩＰ変換部７１とＭＣフレーム数変換部７２で構成する。
ＩＰ変換部７１は、動きベクトルＰＶと１フレーム間差分信号成分の大小に応じて動き適応処理の飛び越し−順次の走査変換を行い、順次走査の信号系列Ｓ３０を生成する。この具体的な構成については後述する。
ＭＣフレーム数変換部７２は、画素単位動きベクトルＰＶをもとに動き補正ベクトルを作り、順次走査の信号系列Ｓ３０の画像の位置をこの動き補正ベクトルで移動させた信号で内挿フレームの信号系列を生成し、この出力に動き補正フレーム数変換処理でフレーム周波数をアップした順次走査の画像信号系列Ｓ４（輝度信号成分と色差信号成分）を得る。この具体的な構成についても後述する。
以上で全体ブロック構成図の説明を終え、以下、本実施例における動き補正信号処理部７の構成について詳述する。
【００７６】
図３３は、ＩＰ変換部７１の一構成例で、動きベクトルを利用した動き適応処理の飛び越し−順次の走査変換を行うに好適なものである。
この構成を同図（ａ）に示す。ＭＡ補間部７８は走査変換に必要な補間走査線の信号を生成する。このうち、動画補間信号生成部７９は、フィ−ルド内処理（例えば、上下の走査線の信号の平均値）を行い、動画に適した補間走査線信号ＳＭを生成する。また、静止補間信号生成部８０は、フィ−ルド間処理（例えば、前フレームの走査線の信号）を行い、静止画に適した補間走査線信号ＳＳを生成する。
係数加重部８１−１は動き係数ｋ、８１−２は１−ｋを加重し、加算部８２で両者の信号を加算して、補間走査線の信号を生成する。
【００７７】
動き係数生成部８３は、同図（ｂ）に示すように、１フレーム間の輝度信号差分成分の絶対値｜Ｓ１−Ｓ２０｜と、動きベクトルＰＶのスカラ−量｜ＰＶ｜の値の大小に応じて、動き係数の係数値ｋを設定する。すなわち、スカラ−量｜ＰＶ｜が小から大となるに従い、傾斜が漸次きつくなる特性の直線で係数値ｋを設定する。従って、｜Ｓ１−Ｓ２０｜が同一の値でも、動きの速度に比例してｋの値も大きく設定でき、従来技術に較べて、画像の動きにより整合した動き適応信号処理が可能になる。
倍速変換部８４は、生成した補間走査線信号とＳ１との時間軸の１／２圧縮と時系列多重の信号処理を行い、順次走査の信号系列Ｓ３０を出力する。
【００７８】
次に、このＭＣフレーム数変換部７２について、図３４〜図３７を用いて説明する。
図３４は、第１の構成例で、予測誤差適応動き補正処理を行うに好適なものである。
図３４の構成例において、１フレームメモリ部８５は、順次走査の現フレームの信号系列Ｓ３０を１フレーム期間遅延させた前フレームの信号系列Ｓ３Ｐを生成する。
ＭＣベクトル生成部８６は、飛び越し走査系列で検出した画素単位動きベクトルＰＶより動き補正処理に必要な動き補正ベクトルＶｃｔとＶｐｒを生成する。そして、動き補正信号生成部８７では、現フレームの信号Ｓ３０と動き補正ベクトルＶｃｔで動き補正信号Ｓｃｔを生成する。また、動き補正信号生成部８８は、前フレームの信号Ｓ３Ｐと動き補正ベクトルＶｐｒで動き補正信号Ｓｐｒを生成する。
【００７９】
これらの動作概略は、先の図１９（ａ）〜（ｃ）で説明したとおりである。但し、同図及びその説明において、ＰＶをＰＶ’と読み替えるものとする。ここでＰＶ’は、順次走査系列での画素単位の動きベクトルで、飛び越し走査系列で検出した動きベクトルＰＶとはＰＶ’＝ＰＶ／２の関係が成立する。従って、フレーム順２の内挿フレームは、Ｖｐｒ＝ＰＶ’＊５／６（ＰＶ＊５／１２），Ｖｃｔ＝−ＰＶ’＊１／６（−ＰＶ＊１／１２）（ｋａ＝５，ｋｂ＝１）、フレーム順３のものは、Ｖｐｒ＝ＰＶ’＊４／６（ＰＶ＊４／１２），Ｖｃｔ＝−ＰＶ’＊２／６（−ＰＶ＊２／１２）（ｋａ＝４，ｋｂ＝２）…の如く、時間方向での位置ずれがない動き補正ベクトルを生成する。
【００８０】
加算部８９は、両者の動き補正信号ＳｐｒとＳｃｔとの加算平均を行い、フレーム順２，３，…，６の動き補正内挿フレームの信号Ｓｍｃを得る。
減算部９０は、動き補正信号ＳｃｔとＳｐｒとの減算を行い、差分信号成分ＭＣＥを検出する。動き補正ベクトルが正確な場合は、動き補正信号ＳｃｔとＳｐｒは同じ値をとるので信号ＭＣＥは０になる。しかしながら、動き補正ベクトルが不正確な場合は、動き補正信号ＳｃｔとＳｐｒとで異なる値になり、信号ＭＣＥは零以外の値を持つ。
補正誤差算出部９１は、ミニブロック単位の動き補正誤差（画素換算相当）を算出し、この値を信号ＭＢＥに出力する。
【００８１】
判定部９２は、信号ＭＢＥの値の大小に応じて閾値ＴＨの値を適応的に設定する。例えば、ＭＢＥ＜＝８ではＴＨ＝４０、８＜ＭＢＥ＜＝１６ではＴＨ＝３２、１６＜ＭＢＥ＜＝２４ではＴＨ＝２４、２４＜ＭＢＥではＴＨ＝１６に設定する。そして、信号ＭＣＥが閾値未満の時は０、閾値以上の時は１を制御信号ＳＬに出力する。
【００８２】
選択部９３は、制御信号ＳＬが０の時は動き補正内挿フレームの信号Ｓｍｃ、制御信号ＳＬが１の時は現フレームの信号Ｓ３０と前フレームの信号Ｓ３Ｐのうちで動き補正内挿フレームと時間的な距離の近い信号を選択し、内挿フレームの信号Ｓ５０を出力する。この操作で、動きベクトルの不正確さに起因する孤立点的な劣化や動画エッジ周縁部の劣化などの動き補正に固有な画質劣化を大幅に抑圧することができ、高画質化が達成できる。
メモリ部９４は、以下の書き込み、読み出し動作を行い、フレーム数変換した信号系列Ｓ４を得る。書き込み動作では、現フレームの信号Ｓ３０のフレーム順１の信号と、信号Ｓ５０のフレーム順２〜６の信号を書き込む。一方、読み出し動作ではフレーム順１から６の信号を順次読み出す。
【００８３】
図３５は、この第２の構成例のブロック構成図で、動き速度適応動き補正処理を行うに好適なものである。これは第１の構成例（図３４）にベクトル分布計測部９５を追加した構成で実現する。
ベクトル分布計測部９５では、画素単位の動きベクトルＰＶのうち、先の図８（ａ）に示した領域Ｅに相当する動きベクトルの１フレーム期間での発生頻度を計測し、これが設定値（例えば１０％以上）を越える場合は動き補正モード信号ＭＣＦＧに０、設定値未満の場合は１を出力する。
【００８４】
判定部９２は、動き補正モード信号ＭＣＦＧが０の時は、選択部９３で該当する１フレーム期間では現フレームの信号Ｓ３０を選択制御する制御信号ＳＬを出力する。一方、動き補正モード信号ＭＣＦＧが１の時は、第１の構成例と同様の動作を行い、信号ＭＣＥが信号ＭＢＥで定まる閾値以上では１、未満では０の制御信号ＳＬを出力する。
なお、本構成では、ベクトル分布計測部９５を省略し、代わりに、前述した図１６の動きベクトル探索部におけるミニブロック分割探索モード信号ＭＢＦＧを動き補正モード信号として使用する構成で実現することも可能である。
【００８５】
図３６は、この第３の構成例で、図３４の第１の構成例の最終段のメモリ部９４を省略し、代わりにフレームレートアップ部９７を先頭に配置した形態で実現する。
フレームレートアップ部９７は、コマ繰り返し処理でフレーム数変換を行い、例えば、フレーム周波数５０Ｈｚの順次走査の信号系列Ｓ３０をフレーム周波数６０Ｈｚの信号系列Ｓ１０に変換する。このＳ１０を現フレームの信号系列、これを１フレーム遅延部３５で１フレーム期間遅延させた信号Ｓ１０Ｐを前フレームの信号系列として、以降、第１の構成例と同様の信号処理を行う。
【００８６】
この動作概略は、先の図２２で説明したとおりである。但し、同図及びその説明において、Ｓ２をＳ３０、Ｓ５をＳ１０、Ｓ６をＳ１０Ｐとそれぞれ読み替えるものとする。従って、現フレームと前フレームの信号系列が同一フレーム順１の場合は静止画像となり、フレームの全ての動きベクトルが０の処理が行われ、出力の信号系列Ｓ４（６０Ｈｚ）はフレーム順１の信号を得る。一方、現フレームと前フレームの信号系列が異なるフレーム順の時は、出力の信号系列Ｓ４は動き補正処理で生成した信号系列（図２２中にＭＣで表示）を得る。
【００８７】
図３７は、この第４の構成例で、図３５の第２の構成例の最終段のメモリ部９４を省略し、代わりにフレームレートアップ部９７を先頭に配置した形態で実現する。この動作に関しては、上述の構成例と同様なため、説明は省略する。
本発明の第９の実施例によれば、動きベクトル探索、生成の信号処理に要する演算量が少なく、かつ、動き補正処理に起因する画質劣化の少ない画像信号のフレーム数変換装置が実現できる。そして、高画質化、低コスト化に顕著な効果を得ることができる。
【００８８】
本発明の第１０の実施例について、図３８を用いて説明する。本実施例は、動きベクトル探索において変換ベクトル探索処理を行うに好適なものである。同図はこのブロック構成例を示す図であり、ブロック単位動きベクトル探索部９８の構成、動作のみが先の第９の実施例（図３２）と異なる。このブロック単位動きベクトル探索部９８の構成と動作は、先の図２４〜図２５で説明したとおりである。但し同図及びその説明において、Ｓ２をＳ１、Ｓ３をＳ２０とそれぞれ読み替えるものとする。
本発明の第１０の実施例によれば、動きベクトル探索、生成の信号処理に要する演算量が更に少ない画像信号のフレーム数変換装置が実現できる。そして、高画質化、低コスト化に顕著な効果を得ることができる。
【００８９】
本発明の第１１の実施例を図３９を用いて説明する。これは、シーンチェンジの領域では動きベクトル探索や動き補正フレーム内挿の信号処理を中止する動作を行うに好適なものである。同図はこのブロック構成例を示す図で、第９の実施例（図３２）の構成にシーンチェンジ検出部９９を追加して実現する。シーンチェンジ検出部９９では、１フレーム期間でのフレーム間差分信号成分の発生形態をもとにシーンチェンジの発生したフレームを検出する動作を行う。この構成と動作は、先の図２８で説明したとおりである。但し同図及びその説明において、Ｓ２をＳ１、Ｓ３をＳ２０とそれぞれ読み替えるものとする。
【００９０】
図３９に戻り、動きベクトル探索部３と動き補正信号処理部７は、信号ＳＣＦＧが１のシーンチェンジが発生したフレームでは、動きベクトル探索や動き補正フレーム内挿の信号処理を中止する。一方、信号ＳＣＦＧが０のフレームでは、動きベクトル探索部３と動き補正信号処理部７は、第１の実施例と同一の信号処理の動作を行い、信号系列Ｓ４に動き補正フレーム内挿処理でフレーム数を変換した信号系列を得る。
本発明の第１１の実施例によれば、シーンチェンジ領域における動きベクトル探索や生成のための膨大な演算量の発生が回避でき、第９の実施例に較べて、動きベクトル探索、生成の信号処理に要する演算量が更に少ない画像信号のフレーム数変換装置が実現できる。そして、高画質化、低コスト化に顕著な効果を得ることができる。
【００９１】
本発明の第１２の実施例を図４０を用いて説明する。これは、第１０の実施例（図３８）の構成にシーンチェンジ検出部９９を追加して実現したものである。この動作は、上述の実施例の説明で容易に理解できるので説明は省略する。そして、第１１の実施例と同様、動きベクトル探索、生成の信号処理に要する演算量が更に少ない画像信号のフレーム数変換装置が実現できる。そして、高画質化、低コスト化に顕著な効果を得ることができる。
【００９２】
次に、本発明のフレーム数変換装置をテレビジョン受像機に適用した実施例について、図４１〜図４４の図面を用いて説明する。
【００９３】
図４１は、この第１の実施例のブロック構成例で、画像を順次走査の形態で表示するに好適なものである。図において、チューナー部４４は、地上放送信号ＴＶをベースバンド帯域のテレビジョン信号に復調する。 ＢＳチューナー部４５は、衛星放送信号ＢＳをベースバンド帯域のテレビジョン信号に復調する。デコーダ部４６は、アナログ方式のテレビジョン信号の所定の復調処理を行い、輝度信号と色差信号を復調する。ＩＲＤ部４７は、デジタル方式のテレビジョン信号の所定の復号化処理を行い、輝度信号と色差信号を復調する。外部入力信号Ｅｘには、ＶＣＲ等のパッケージメディアやＰＣ画像などの輝度信号と色差信号を入力する。スイッチ部４８は、制御部５４の制御信号で指定する信号系列の選択を行う。
【００９４】
ＩＰ変換部４９は、飛び越し走査の信号系列に対しては、例えば動き適応の走査変換の信号処理を行い、順次走査の信号系列に変換する。ＭＣフレーム数変換部５０は、本発明のフレーム数変換装置に相当し、動き補正フレーム内挿処理でフレーム周波数をアップした信号系列を生成する。例えば、ＰＡＬ方式のフレーム周波数５０Ｈｚの信号の５０Ｈｚ→６０Ｈｚ変換では、前述の実施例と同様にフレーム順１〜５の入力信号系列からフレーム順１〜６（２〜６は動き補正フレーム内挿処理で生成）の信号系列を生成する。また、５０Ｈｚ→７５Ｈｚ変換では、フレーム順１〜２の入力信号系列からフレーム順１〜３（２〜３は動き補正フレーム内挿処理で生成）の信号系列を生成する。あるいは、５０Ｈｚ→１００Ｈｚ変換では、フレーム順１の入力信号系列からフレーム順１〜２（２は動き補正フレーム内挿処理で生成）の信号系列を生成する。
【００９５】
スケーリング処理部５１は、画像サイズの拡大／縮小、アスペクト比の変換、走査線数の変換などの信号処理を行う。画質改善部５２は、鮮鋭度改善や輝度階調補正などの画質改善信号処理と色空間変換による３原色信号への変換を行う。順次表示部５３は、フレーム周波数が６０Ｈｚ、又は７５Ｈｚか１００Ｈｚの順次走査の形態で画像表示を行う。
【００９６】
リモコン受信部５５は、ユーザがリモコン端子で操作する各種ユーザ情報（チャネル選択、表示モードなど）を受信する。制御部５４は、この受信した各種ユーザ情報に応じて、各部の動作に必要な制御信号類（図面には明示せず）を生成し、被制御部５８に出力する。
【００９７】
図４２は、この第２の実施例のブロック構成例で、画像を飛び越し走査の形態で表示するに好適なものである。これは、上述した図４１の構成にＰＩ変換部５６を追加し、インターレース表示部５７で飛び越し走査の形態の画像を表示する。ＰＩ変換部５６は、順次走査の信号系列の走査線の２：１の間引き処理を行い、飛び越し走査の信号系列に変換する信号処理を行う。また、インターレース表示部５７は、フィールド周波数が６０Ｈｚ、又は７５Ｈｚか１００Ｈｚの飛び越し走査の形態で画像表示を行う。なお、この他のブロックは図４１のものと同一の動作を行うので、説明は省略する。
【００９８】
図４３は、この第３の実施例のブロック構成例で、画像を順次走査の形態で表示するに好適なものである。これは、上述した図４１のＩＰ変換部４１及びＭＣフレーム数変換部５０に代えて、動き補正フレーム数変換部６３を設けたものである。この動き補正フレーム数変換部６３は、本発明のフレーム数変換装置に相当し、動き補正フレーム内挿処理でフレーム周波数をアップした順次走査の信号系列を生成する。例えば、ＰＡＬ方式のフレーム周波数５０Ｈｚの信号の５０Ｈｚ→６０Ｈｚ変換では、前述の実施例と同様にフレーム順１〜５の入力信号系列からフレーム順１〜６（２〜６は動き補正フレーム内挿処理で生成）の信号系列を生成する。また、５０Ｈｚ→７５Ｈｚ変換では、フレーム順１〜２の入力信号系列からフレーム順１〜３（２〜３は動き補正フレーム内挿処理で生成）の信号系列を生成する。あるいは、５０Ｈｚ→１００Ｈｚ変換では、フレーム順１の入力信号系列からフレーム順１〜２（２は動き補正フレーム内挿処理で生成）の信号系列を生成する。なお、この他のブロックは図４１のものと同一の動作を行うので、説明は省略する。
【００９９】
図４４は、この第４の実施例のブロック構成例で、画像を飛び越し走査の形態で表示するに好適なものである。これは、上述した図４３の構成にＰＩ変換部５６を追加し、インターレース表示部５７で飛び越し走査の形態の画像を表示する。このＰＩ変換部５６及びインターレース表示部５７は、図４２の第２の実施例で説明したものと同じものである。この他のブロックは図４３のものと同一の動作を行うので、説明は省略する。
これらのテレビジョン受像機の実施例によれば、マルチソース対応の高画質な装置を低コストで実現することができる。
【０１００】
以上で実施の形態の説明を終了するが、本発明は、ＰＡＬ−ＮＴＳＣテレビジョン信号の方式変換、ＰＡＬ５０Ｈｚ−６０Ｈｚ／７５Ｈｚ／１００Ｈｚ変換、フィルム画像（フレーム周波数２４Ｈｚ／３０Ｈｚ）−６０Ｈｚ変換、テレビジョン信号−パソコン画像信号変換など、各種のフレーム数変換装置に適用し、高画質化・低コスト化に顕著な効果を有する。
【０１０１】
このように本発明では、動き補正に必要な動きベクトル探索に要する信号処理の演算量や回路規模が小さく、また、動き補正に固有な画質劣化を抑圧した動き補正フレーム変換装置が実現できる。このため、各種のフレーム数変換装置の高画質化・低コスト化に顕著な効果が得られる。
また、本発明をテレビジョン受像機に適用して、マルチソース対応の高画質で低コストの受像機を実現することができる。また、受像機の多機能化にも顕著な効果がある。
【０１０２】
【発明の効果】
本発明によれば、高画質でしかも回路規模の小さい画像信号のフレーム数変換方法および装置を得ることができる。
【図面の簡単な説明】
【図１】本発明の第１の実施例のブロック構成図である。
【図２】動き検出部の一構成例を示す図である。
【図３】ブロック単位動きベクトル探索部の第１構成例を示す図である。
【図４】第１構成例のブロック単位動きベクトル探索のフローチャートである。
【図５】ブロックマッチング第１処理における代表点の配列図である。
【図６】ブロック単位動きベクトル探索部の第２構成例を示す図である。
【図７】第２構成例のブロック単位動きベクトル探索のフローチャートである。
【図８】（ａ）は動きベクトル発生頻度の計測の一例を、（ｂ）は探索モード設定の一例をそれぞれ示す図である。
【図９】画素単位動きベクトル生成部の第１構成例を示す図である。
【図１０】第１構成例の画素単位動きベクトル生成処理のフローチャートである。
【図１１】画素単位動きベクトル生成の動作概略を示す図である。
【図１２】画素単位動きベクトル生成部の第２構成例を示す図である。
【図１３】第２構成例の画素単位動きベクトル生成処理のフローチャートである。
【図１４】画素単位動きベクトル生成部の第３構成例を示す図である。
【図１５】第３構成例の画素単位動きベクトル生成処理のフローチャートである。
【図１６】動きベクトル探索部の他の構成例を示す図である。
【図１７】動きベクトル探索の信号処理フローチャートである。
【図１８】動き補正内挿フレーム生成部の第１構成例を示す図である。
【図１９】（ａ）、（ｂ）はそれぞれＭＣベクトル生成部の動作概略を、（ｃ）は動き補正信号生成部の動作概略を示す図である。
【図２０】動き補正内挿フレーム生成部の第２構成例を示す図である。
【図２１】本発明の第２の実施例のブロック構成図である。
【図２２】（ａ）〜（ｄ）はそれぞれフレームレートアップ部の動作概略を説明するための図である。
【図２３】本発明の第３の実施例のブロック構成図である。
【図２４】ブロック単位動きベクトル探索部の一構成例を示す図である。
【図２５】（ａ）〜（ｃ）はそれぞれベクトル変換部の動作概略を示す図である。
【図２６】本発明の第４の実施例のブロック構成図である。
【図２７】本発明の第５の実施例のブロック構成図である。
【図２８】シーンチェンジ検出部の一構成例を示す図である。
【図２９】本発明の第６の実施例のブロック構成図である。
【図３０】本発明の第７の実施例のブロック構成図である。
【図３１】本発明の第８の実施例のブロック構成図である。
【図３２】本発明の第９の実施例のブロック構成図である。
【図３３】（ａ）、（ｂ）はそれぞれＩＰ変換部の一構成例を示す図である。
【図３４】ＭＣフレーム数変換部の第１構成例を示す図である。
【図３５】ＭＣフレーム数変換部の第２構成例を示す図である。
【図３６】ＭＣフレーム数変換部の第３構成例を示す図である。
【図３７】ＭＣフレーム数変換部の第４構成例を示す図である。
【図３８】本発明の第１０の実施例のブロック構成図である。
【図３９】本発明の第１１の実施例のブロック構成図である。
【図４０】本発明の第１２の実施例のブロック構成図である。
【図４１】本発明に係るテレビジョン受像機の第１の実施例を示す図である。
【図４２】本発明に係るテレビジョン受像機の第２の実施例を示す図である。
【図４３】本発明に係るテレビジョン受像機の第３の実施例を示す図である。
【図４４】本発明に係るテレビジョン受像機の第４の実施例を示す図である。
【符号の説明】
１ ＩＰ変換部
２ １フレーム遅延部
３ 動きベクトル探索部
４ 動き検出部
５ ブロック単位動きベクトル探索部
６ 画素単位動きベクトル生成部
７ 動き補正信号処理部
８ 動き補正内挿フレーム生成部
９ メモリ部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image signal frame number conversion method and apparatus, and more particularly to a frame number conversion method and apparatus suitable for converting the frame number by motion correction signal processing.
[0002]
[Prior art]
In recent years, with the development of multimedia, television receivers are also required to have a function of displaying signals of different image formats such as various types of television signals and PC image signals. In addition, an image display function by sequential scanning is required as a function corresponding to high image quality and a flat display. In order to realize these functions, it is necessary to convert the number of input signals into the format of the image display unit and to perform signal processing such as frame number conversion and sequential scanning. When this signal processing is performed by a simple method such as frame repetition or frame dropping, image quality degradation such as motion judder in which smoothness is lost in a moving image occurs.
[0003]
The motion correction frame number conversion process is a technique for avoiding this kind of image quality degradation, and generates an interpolated frame signal by moving the image position of the previous and next frames using a motion vector. With regard to this motion correction frame number conversion process, for example, Japanese Patent Laid-Open No. 7-170496 discloses a technique for efficiently searching for motion vectors. Japanese Patent Application Laid-Open No. 7-336650 discloses a technique for avoiding deterioration such as a reduction in resolution at a moving image edge peripheral edge inherent to motion correction.
[0004]
[Problems to be solved by the invention]
However, in these conventional techniques, the former has a problem in the accuracy of motion detection, and the latter has a problem in the complexity of signal processing. That is, in the prior art, the signal processing circuit scale increases when trying to improve the image quality, and conversely, if the signal processing circuit scale is reduced, image quality deterioration cannot be sufficiently eliminated.
[0005]
Accordingly, it is an object of the present invention to provide a method and apparatus for converting the number of frames of an image signal with high image quality and a small circuit scale.
[0006]
[Means for Solving the Problems]
The purpose of the above is to detect a motion vector in a block unit by block matching processing for a moving image block detected from an image signal, and the motion of the current block and an adjacent block according to the magnitude of the motion correction error of the detected block unit motion vector. Generates a motion vector in units of pixels from the vector, generates an interpolation frame of the image signal according to the magnitude of the motion correction error of the motion vector in pixel unit, and converts the number of frames of the image signal using this interpolation frame This is achieved by a method for converting the number of frames of an image signal.
[0007]
Also, in the frame number conversion device according to the present invention, a motion detection unit that detects a still block or a moving image block based on an image signal, and a block unit motion vector that detects a block-based motion vector by block matching processing on the moving image block A search unit, a pixel unit motion vector generation unit that generates a pixel unit motion vector from the motion vectors of the current block and adjacent blocks according to the magnitude of the motion correction error of the detected block unit motion vector, and a pixel unit motion vector A motion correction interpolation frame generation unit that generates an interpolation frame of the image signal according to the magnitude of the motion correction error. Here, as the image signal, an interlaced scanning image signal or a progressive scanning image signal may be used, but the apparatus configurations of both are basically the same. However, there are differences in the specific configuration, and these will be described later.
[0008]
Specifically, the present invention uses the following technical means. That is, in order to greatly reduce the amount of calculation required for the motion vector search process, the following two-stage signal processing of the block unit motion vector search process and the pixel unit motion vector generation process is employed.
[0009]
First, in block unit motion vector search processing, a motion vector is searched by block matching processing, but a block in which motion is not detected by a frame difference signal (hereinafter abbreviated as still block) and a block in which motion is detected (hereinafter abbreviated as moving image block). ) And the search process is performed with one of the following three types only for the moving image block.
(1) Double representative point tree search processing: The representative motion vectors that are preset in the vicinity of the origin are dense and the peripheral portion is roughly set as a reference motion vector having the smallest motion correction error. The motion vector in the block unit is detected by re-searching the one having the smallest motion correction error.
(2) Motion vector distribution adaptive search processing: Select one of a plurality of types of search modes having different search regions and representative motion vector arrangements according to the form of motion vector occurrence frequency of the previous frame, and select the selected mode. The motion vector of the block unit is detected according to
(3) Conversion vector search processing: a motion vector of one frame generated by vector conversion processing based on motion vector information used in image encoding is used as a reference motion vector, and the motion correction error of the neighboring motion vectors is the smallest Are searched again to detect a motion vector in units of blocks.
[0010]
Next, in the pixel unit motion vector generation process, one of the following three types of mini-block division search processes is performed on a mini-block obtained by subdividing a block horizontally and vertically.
(1) Block prediction error adaptive search processing: a calculation area that includes a mini-block among the motion vectors of the current block and adjacent blocks for each mini-block with respect to a block whose motion correction error due to a motion vector in units of blocks is greater than or equal to a threshold value The one with the smallest motion correction error is detected as the motion vector of the pixel in the mini-block.
(2) Block mini-block prediction error adaptive search processing: a threshold is changed according to the magnitude of motion correction error due to a block-by-block motion vector, and mini-block motion correction error due to a block-by-block motion vector is greater than or equal to the threshold Among the motion vectors of the current block and adjacent blocks, the one having the smallest motion correction error in the calculation region including the mini-block is detected as the motion vector of the pixel in the mini-block.
(3) V-edge adaptive search processing: In a mini-block including a V-edge region of an image signal, a horizontal calculation region including a mini-block among motion vectors of a current block and an adjacent block has a minimum motion correction error. It detects as a motion vector of the pixel in a mini block.
According to the motion vector search process of the present invention described above, highly accurate motion vector detection can be performed with a calculation amount reduced by about two to three digits compared to the full search.
[0011]
In the motion compensation signal processing, a motion compensation interpolation frame signal is generated by one of the following two types of processing.
(1) Prediction error adaptive motion correction processing: The threshold value is changed according to the magnitude of the block motion correction error and the mini block motion correction error based on the motion vector detected in the motion vector search processing, and the position of the image of the previous frame is determined as the motion correction vector. When the difference signal component between the pre-motion-corrected frame signal generated by moving the image and the motion-corrected current frame signal generated by moving the position of the image of the current frame by the motion correction vector is equal to or greater than the threshold, the previous and current frames Is replaced with a signal of a frame that is close to the interpolation frame.
(2) Motion speed adaptive motion correction processing: A frame including an image having a speed at which motion judder interference is conspicuous in the form of the motion vector occurrence frequency detected by the motion vector search processing is detected, and motion is limited to the detected frame. A corrected interpolation frame signal is generated.
[0012]
According to the motion correction signal processing of the present invention described above, image quality degradation inherent to motion compensation processing, for example, isolated point degradation in which a part of an image is replaced with an inappropriate image, or an edge portion of a moving image flickers. It is possible to drastically suppress deterioration that occurs and the movement looks unnatural.
In the present invention, in the scene change region, the motion vector search process and the motion correction interpolation frame signal generation process are stopped. As a result, it is possible to suppress the generation of a huge amount of calculation in the motion vector search process that occurs in the scene change area.
According to the technical means of the present invention described above, it is possible to realize an image signal frame number conversion method and apparatus with high image quality and low cost.
[0013]
The television receiver according to the present invention further includes an input unit for inputting an image signal, a motion vector in a block unit based on the image signal, and a current block according to the magnitude of a motion correction error in the block unit motion vector. An image in which the number of frames of an image signal is converted by generating a motion vector in pixel units from the motion vector of an adjacent block and generating an interpolation frame of the image signal according to the magnitude of the motion correction error of the motion vector in pixel units A signal frame number conversion unit and a display unit for displaying the output of the image signal frame number conversion unit are provided.
According to the present invention, a multi-source compatible high-quality television receiver can be manufactured with a simple circuit and at a low cost.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
A first embodiment of the frame number conversion apparatus according to the present invention will be described with reference to FIGS.
[0015]
FIG. 1 is a block diagram of the present embodiment. In the figure, an IP converter 1 receives an interlaced scanning input image signal S1 (luminance signal component and color difference signal component) and performs interlaced-sequential scanning conversion. For example, a luminance signal component of the input image signal S1 is generated by a motion adaptive interpolation process, and a color difference signal is generated by an interpolation process between lines. Signal component). When the input image signal is a telecine image signal (a signal obtained by converting a film image such as a movie into a television signal format by 2-3 pull-down processing), a film mode interpolation process (interlaced scanning signal belonging to the same film frame) (Interpolated scanning line signal is generated in step S1) and converted into a sequential scanning signal sequence.
[0016]
The 1-frame delay unit 2 generates a signal sequence S3 of the previous frame obtained by delaying the signal sequence S2 of the current frame that is sequentially scanned by one frame period. The signal sequences S2 and S3 are supplied to the motion vector search unit 3 and the motion correction signal processing unit 7, respectively.
The motion vector search unit 3 includes a motion detection unit 4, a block unit motion vector search unit 5, and a pixel unit motion vector generation unit 6.
The motion detection unit 4 extracts a difference signal between one frame by subtraction processing of both luminance signal components of the signal sequence S2 of the current frame and the signal sequence S3 of the previous frame, and performs binary quantization on the motion detection signal MD1, MD2 is output. Details of this configuration will be described later.
[0017]
The block unit motion vector search unit 5 detects a motion vector in block units (for example, 16 pixels × 16 lines or 8 pixels × 8 lines). That is, a block with a motion detection signal MD1 of 0 is determined as a still block, and 0 is output to the block unit motion vector BV. On the other hand, the block whose motion detection signal MD1 is 1 is determined to be a moving image block, and the above-described double representative point tree search process or motion vector distribution adaptive search process is performed on the luminance signal components of the signal sequences S2 and S3. A unit motion vector BV is detected. Details of this configuration will be described later.
[0018]
The pixel unit motion vector generation unit 6 generates a pixel unit motion vector PV by mini-block division search processing. That is, the pixel-unit motion vector PV is detected in units of mini-blocks (for example, 2 pixels × 2 lines) by the above-described block prediction error adaptive search processing, block mini-block prediction error adaptive search processing, or V edge adaptive search processing. Further, a pixel whose motion detection signal MD2 is 0 is determined as a still pixel and 0 is assigned to PV. This specific configuration will also be described later.
[0019]
The motion correction signal processing unit 7 includes a motion correction interpolation frame generation unit 8 and a memory unit 9.
The motion compensation interpolation frame generation unit 8 creates a motion compensation vector based on the pixel unit motion vector PV, and a signal obtained by moving the image positions of the current frame signal S2 and the previous frame signal S3 by this motion compensation vector. To generate a signal sequence SMC of the interpolation frame. This specific configuration will also be described later.
The memory unit 9 performs a write operation and a read operation of the signal sequence SMC of the specific frame order of the signal sequence S2 of the current frame and the signal sequence SMC of the interpolated frame, and the frame frequency is increased to the output by the motion correction frame number conversion process. A sequential scanning image signal sequence S4 (luminance signal component and color difference signal component) is obtained.
[0020]
Hereinafter, the main block part in a present Example is demonstrated.
FIG. 2 is a diagram illustrating a configuration example of the motion detection unit 4. In the figure, a subtracting unit 10 subtracts the luminance signal components of the current frame signal S2 and the previous frame signal S3 to extract a difference signal component FD between one frame.
When the signal level of the differential signal component FD is less than the set value ± Tha, the binary quantization unit 11-1 determines that the signal is stationary, and determines that the signal is 1 if it is greater than ± Tha. Output. The static block determination unit 12 detects the presence / absence of 1 of the binary signal in units of blocks. When all are 0, it is determined as a static block, and the motion detection signal MD1 is 0, otherwise it is determined as a moving image block. 1 is output.
The binary quantizing unit 11-2 outputs a motion detection signal MD2 of 0 when the signal level of the differential signal component FD is 0 and determined to be still, otherwise it is determined to be a moving image.
[0021]
Next, a configuration example of the block unit motion vector search unit will be described. FIG. 3 is a diagram showing a first configuration example of the block unit motion vector search unit 5, FIG. 4 is a flowchart of block unit motion vector search in the first configuration example, and FIG. 5 shows an example of representative point arrangement in block matching processing. FIG. This first configuration example is suitable for performing a double representative point tree search process.
[0022]
In FIG. 3, the control unit 15 controls the operations of the block matching first processing unit 13 and the block matching second processing unit 14 in accordance with the motion detection signal MD1 as step 1 shown in FIG. That is, in the still block whose motion detection signal MD1 is 0, as one form of step 2, the motion vector search operation is not performed and 0 is output to the block unit motion vector BV. On the other hand, in the moving image block whose motion detection signal MD1 is 1, the motion vector search operation described below is performed.
[0023]
The block matching first processing unit 13 performs the second step search process shown in FIG. That is, using the luminance signal components of the signal S2 of the current frame and the signal S3 of the previous frame, a plurality of preset representative motion vectors (the region near the origin is dense as shown in FIG. 5 (for example, the interval between the representative points is horizontal). (2 pixels in the direction, 2 lines in the vertical direction), and motion correction is performed by block matching processing for the rough peripheral area (for example, a double representative point array in which the representative point interval is 4 pixels in the horizontal direction and 4 lines in the vertical direction). The error is calculated, and the one with the smallest error is output to the representative motion vector BVT. In this search process, the motion vector of the block immediately before the search can be used as the representative motion vector.
[0024]
As for the search region, the horizontal direction ± shown in FIG. 5 is covered so that the motion judder disturbance is detected, that is, the speed of movement covers the range up to about 1 second / screen width and 1 second / screen height. You may limit to the area | region of MX pixel and the vertical direction +/- MY line. Further, since the television image is a horizontally long image having an aspect ratio of 4: 3 or 16: 9, it is suitable to set a horizontal search area wider than a vertical search area.
[0025]
The block matching second processing unit 14 performs the search process of the third step shown in FIG. That is, using the luminance signal components of the signal S2 of the current frame and the signal S3 of the previous frame, the motion vector determined by the representative motion vector BVT as a starting point and the x component in the range of ± DX and the y component in the range of ± DY is blocked. The motion correction error is calculated by the matching process, and the one that minimizes the error is output to the block unit motion vector BV.
[0026]
As a result, the search for the block unit motion vector can be limited to the moving image block, and the partial search by the representative motion vector is possible, so that the computation amount of signal processing required for the search is about 1 to 2 digits compared to the full search. Can be reduced.
[0027]
6 is a diagram showing a second configuration example of the block unit motion vector search unit 5, FIG. 7 is a flowchart of block unit motion vector search in the second configuration example, and FIG. 8 is a diagram illustrating measurement of motion vector occurrence frequency and search mode setting. It is a figure which shows an example. This second configuration example is suitable for performing motion vector distribution adaptive search processing.
[0028]
As shown in FIG. 6, the second configuration example is realized by a configuration in which the vector distribution measuring unit 16 is added to the first configuration example described above. The vector distribution measuring unit 16 measures the occurrence frequency of the motion vector in one frame period and sets the search mode signal MOD according to the distribution form of the occurrence frequency. This will be described with reference to the flowchart of FIG. That is, in the still block whose motion detection signal MD1 is 0, the motion vector search operation is not performed, and 0 is output to the block unit motion vector BV. On the other hand, in the moving image block whose motion detection signal MD1 is 1, the motion vector search operation is performed according to the search process determined by the search mode signal MOD described below.
[0029]
A setting example of the search mode signal MOD is shown in FIG. FIG. 5A shows a region where the frequency of motion vector occurrence is measured, where the horizontal axis is the absolute value abs (vx) of the x-direction component of the motion vector, and the vertical axis is the absolute value abs (vy) of the y-direction component. is there. In the region A, 0 <abs (vx) <= 4 and 0 <abs (vy) <= 2, and in the region B, 0 <abs (vx) <= 8 and 0 <abs (vy) <= 4 In the region excluding the region A, in the region C, 0 <abs (vx) <= 12 and in the range 0 <abs (vy) <= 6, in the region excluding the regions A and B, in the region D, 0 <abs (vx) < = 16 and 0 <abs (vy) <= 8 in the range excluding areas A, B, and C, and in area E, the number of motion vectors generated in the range of abs (vx)> 16 or abs (vy)> 8. Each is measured over one frame period, and the frequency of occurrence is calculated based on this measurement.
[0030]
FIG. 4B shows an example of search mode setting conditions and search processing characteristics.
The search mode of MOD1 is a case where the occurrence frequency of the region A is 95% or more, and the block matching second processing unit 14 performs a full search for a range of horizontal ± 4 pixels and vertical ± 2 lines to obtain a motion vector BV in block units. To detect.
The search mode of MOD2 is when the total occurrence frequency of the areas A and B is 95% or more, and the block matching second processing unit 14 performs a full search for the range of horizontal ± 8 pixels and vertical ± 4 lines, and performs a block unit. A motion vector BV is detected.
The search mode of MOD3 is when the total occurrence frequency of regions A, B, and C is 95% or more, and the block matching first processing unit 13 uses representative point motion vectors composed of dense regions as horizontal ± 12 pixels and vertical ± 6. A reference motion vector BVT is detected by performing block matching processing on the search area of the line. The block matching second processing unit 14 performs a re-search process starting from BVT, and detects a motion vector BV in units of blocks.
[0031]
The search mode of MOD4 is when the total occurrence frequency of the areas A, B, C, and D is 95% or more, and the block matching first processing unit 13 uses a representative point motion vector consisting of dense areas as horizontal ± 16 pixels, vertical A reference motion vector BVT is detected by performing block matching processing on a search area of ± 8 lines. The block matching second processing unit 14 performs a re-search process starting from BVT, and detects a motion vector BV in units of blocks.
The search mode of MOD5 is the case where the occurrence frequency of the region E is 5% or more, and the double representative point tree search is performed in the search region of horizontal ± 24 pixels and vertical ± 12 lines as in the first configuration example of FIG. Processing is performed to detect a motion vector BV in units of blocks.
In the block unit motion vector search unit described above, the motion correction error is calculated using the luminance signal component of the image signal, but it is also possible to perform the calculation using both the luminance signal component and the color signal component.
[0032]
Next, a configuration example of the pixel unit motion vector generation unit 6 will be described. FIG. 9 is a diagram illustrating a first configuration example of the pixel unit motion vector generation unit 6, FIG. 10 is a flowchart of pixel unit motion vector generation processing in the first configuration example, and FIG. 11 is a schematic operation diagram of pixel unit motion vector generation. FIG. This first configuration example is suitable for performing block prediction error adaptive search processing.
[0033]
In FIG. 9, the correction error calculation unit 17 performs the process of the first step of the signal processing flowchart shown in FIG. That is, the motion correction error by the block unit motion vector BV is calculated for the luminance signal components of the current frame signal S2 and the previous frame signal S3. A block whose error value is less than the threshold value TH outputs 0 as the signal PMB, and a block whose threshold value is equal to or greater than the threshold value TH outputs 1.
Based on the signal PMB and the motion detection signal MD2, the control unit 18 generates control signals PC1 and PC2 necessary for signal processing in the second and third steps of FIG.
[0034]
The reference motion vector generation unit 19 and the correction error calculation units 20-1,..., 20-N have a mini-block for each mini-block subdivided in the horizontal and vertical directions when the block is equal to or greater than the second step threshold value in FIG. Signal processing for generating a motion vector of a pixel in the mini-block is performed by the division search process. That is, in the block in which the control signal PC1 is greater than or equal to the threshold value, the mini-block is represented by the motion vector BV of the current block generated by the reference motion vector generator 19 and the motion vectors BVul,. For each (for example, 2 pixels × 2 lines of horizontal MBX = 2 and vertical MBY = 2), motion correction errors ER0, ER1,... In the calculation area of horizontal 4 (MBX + 2) pixels and vertical 4 (MBY + 2) lines are included. , ERN is calculated. In calculating the motion correction error, the following calculation is performed on either the luminance signal component of the current frame signal S2 and the previous frame signal S3, or the luminance signal component and the color difference signal component. I do.
[0035]
[Expression 1]

[0036]
In (Expression 1), S2 (x, y) is a pixel signal of the current frame in the calculation region, S3 (BV) is a pixel signal of the previous frame whose position is moved by the motion vector BV, and abs {} is absolute The value, Σ is the sum of the pixels in the calculation area, BVx is the x component of the motion vector BV, and BVy is the y component.
[0037]
The pixel motion vector setting unit 21 performs signal processing of the second step and the third step of FIG. That is, the block for which the control signal PC2 is less than the threshold value outputs the motion vector BV of the current block to the pixel unit motion vector PV for the pixels in the block. On the other hand, a block equal to or greater than the threshold outputs a motion vector that takes the minimum value among the motion correction errors ER0, ER1,..., ERN to the pixels in each mini-block as a pixel unit motion vector PV.
Further, 0 is output to the pixel unit motion vector PV for a pixel whose motion detection signal MD2 is 0 in the control signal PC2.
As a result, the motion vector in units of pixels can be highly accurate and the amount of calculation required can be reduced by about one to two digits compared to the full search.
[0038]
FIG. 12 is a diagram illustrating a second configuration example of the pixel unit motion vector generation unit 6, and FIG. 13 is a flowchart of pixel unit motion vector generation processing in the second configuration example. This second configuration example is suitable for performing block mini-block prediction error adaptive search processing.
[0039]
In the second configuration example shown in FIG. 12, the operations in the correction error calculation unit 22 and the control unit 18 are different from those in the first configuration example.
In the figure, the correction error calculator 22 performs the first step of the signal processing flowchart shown in FIG. That is, a block motion correction error PM (equivalent to pixel conversion) and a mini-block motion correction error MBE (equivalent to pixel conversion) based on the block unit motion vector BV are output for the luminance signal components of the current frame signal S2 and the previous frame signal S3. To do.
Based on the signals PM and MBE and the motion detection signal MD2, the control unit 18 generates control signals PC1 and PC2 necessary for signal processing in the second and third steps in FIG. First, the value of the threshold TH is set for each block according to the value of the block motion correction error PM. For example, TH = 16 when PM <= 8, TH = 12 when 8 <PM <= 12, TH = 8 when 12 <PM <= 16, and TH = 4 when PM> 16. Then, the mini-block motion correction error MBE is compared with the threshold value TH. For mini-blocks exceeding the threshold value, signal processing for mini-block division search greater than or equal to the second step threshold value, and for mini-blocks less than the threshold value, the first value is processed. A control signal is generated so as to perform signal processing less than a two-step threshold. Furthermore, a control signal necessary for the signal processing in the third step is generated according to whether the motion detection signal MD2 is 0 or 1.
Note that the reference motion vector generation unit 19, the correction error calculation unit 20, and the pixel motion vector setting unit 21 perform the same operations as those in the first configuration example, and a description thereof will be omitted.
[0040]
FIG. 14 is a diagram illustrating a third configuration example of the pixel unit motion vector generation unit 6, and FIG. 15 is a flowchart of pixel unit motion vector generation processing in the third configuration example. This third configuration example is suitable for performing V-edge adaptive search processing.
[0041]
FIG. 14 shows a block configuration when the V edge adaptive search process is applied to the first configuration example (FIG. 9), and the operations of the V edge detection unit 23 and the control unit 26 are different from those of the first configuration example.
In the figure, the V edge detection unit 23 is composed of a combination of a low pass characteristic filter HLPF 24 having a low frequency component of the horizontal frequency as a pass band and a high pass characteristic filter VHPF 25 having a high frequency component of the vertical frequency as a pass band, From the luminance signal component of the signal S2 of the current frame, the V edge signal VEG of the horizontal low band and the vertical high band is extracted.
Based on the signal PMB, the V edge signal VEG, and the motion detection signal MD2 indicating whether the value of the motion correction error due to the block unit motion vector BV is less than or greater than the threshold value TH, the control unit 26 performs the second and third processing shown in FIG. Control signals PC1 and PC2 necessary for the signal processing of the step are generated.
[0042]
Then, as shown in FIG. 15, the signal processing of the mini-block division search when the second step is equal to or greater than the threshold is made different. That is, in the mini-block including the V edge region, the motion correction error is calculated in the calculation region in the horizontal MBX + 4 and vertical MBY including the mini-block (horizontal MBX, vertical MBY), and this value is minimized. A pixel-based motion vector PV is generated using the reference vector.
Also, motion vector correction processing is performed for pixels included in the same V edge region. That is, the motion vectors of these pixels are corrected by a smoothing process (for example, averaging), and the corrected motion vectors are assigned to the pixels in the V edge region.
[0043]
On the other hand, in the normal region other than the V edge region, the same operation as in the first configuration example is performed.
Note that the V-edge adaptive search process described here can also be applied to the second configuration example (FIG. 12).
As described above, the motion vector search unit can have various configurations by combining the block unit motion vector search unit and the pixel unit motion vector generation unit.
[0044]
Now, a configuration example of a motion vector search unit suitable for performing motion correction processing in units of frames in a motion correction signal processing unit described later will be described next. FIG. 16 is a diagram illustrating a second configuration example of the motion vector search unit, and FIG. 17 is a flowchart of motion vector search in the second configuration example.
[0045]
In FIG. 16, the block matching first processing unit 13, the block matching second processing unit 14, and the vector distribution measuring unit 16 have the same functions and operations as the second configuration example (FIG. 6) of the block unit motion vector search unit described above. I do. That is, as shown in FIG. 17, the motion vector BV in units of blocks is detected for the moving picture block by the search process (see FIG. 8) using the search mode signals MOD1 to MOD5. In addition, the motion vector BV is set to 0 for a still block.
The vector distribution measuring unit 16 outputs 0 to the mini-block division search mode signal MBFG when the occurrence frequency of the region E shown in FIG. 8 exceeds a set value (for example, 10%), and 1 when it is less than the set value. .
[0046]
Further, the correction error calculation unit 17 and the mini-block division search unit 27 are configured in the same manner as the first to third configuration examples of the pixel unit motion vector generation unit described above. When the mode signal MBFG is 1, signal processing for mini-block division search similar to that in the first to third configuration examples described above is performed to generate a pixel-based motion vector PV. On the other hand, when the mode signal MBFG is 0, the motion compensation frame interpolation process is not performed in the subsequent motion compensation signal processing unit, so that a motion vector for each pixel is unnecessary, and the signal processing of the mini-block division search is stopped. .
Note that the number of operations required for the mini-block division search processing is the above-described first to third since the images where motion judder interference is conspicuous (images for which motion correction frame interpolation processing is essential) are about 10% of the entire program. Compared to the above configuration example, it can be further reduced by about one digit.
The control unit 281 generates control signals necessary for the above-described signal processing operation.
[0047]
This is the end of the description of the motion vector search unit. Next, the configuration of the motion correction interpolation frame generation unit 8 of the motion correction signal processing unit 7 will be described. FIG. 18 is a diagram illustrating a first configuration example of the motion compensation interpolation frame generation unit 8, and FIG. 19 is a diagram for explaining operations of the MC vector generation unit 28 and the motion correction signal generation units 30 and 31. This first configuration example is suitable for performing prediction error adaptive motion correction processing.
[0048]
The MC vector generation unit 28 generates motion correction vectors Vct and Vpr necessary for motion correction processing from the pixel unit motion vector PV. An outline of this operation is shown in FIGS. 19A to 19C, taking as an example a case where the number of frames is converted from a signal having a frame frequency of 50 Hz to a signal having a frame frequency of 60 Hz. For signals 1 to 5 in the frame order of progressive scanning at a frame frequency of 50 Hz, as shown in FIG. 5A, an interpolated frame signal is generated by signal processing for motion correction, and the frames in the frame order 1 to 6 are generated. The signal is converted into a progressive scan signal having a frequency of 60 Hz. At this time, the motion correction vector needs to match the frame position to be interpolated. Therefore, as shown in FIG. 5B, coefficient values ka and kb for coefficient weighting on the motion vector PV in units of pixels are sequentially changed in order of frames, and motion correction vectors Vpr and Vct are generated by the calculation shown in (Equation 2). To do.
[0049]
[Expression 2]
Vpr = PV * ka / (ka + kb)
Vct = −PV * kb / (ka + kb) (Equation 2)
[0050]
Therefore, the interpolated frames in the frame order 2 are Vpr = PV * 5/6, Vct = −PV * 1/6 (ka = 5, kb = 1), and those of 3 are Vpr = PV * 4/6, Vct = −PV * 2/6 (ka = 4, kb = 2)... To generate a motion correction vector having no positional deviation in the time direction.
[0051]
The motion correction signal generation unit 30 generates a motion correction signal Sct using the current frame signal S2 and the motion correction vector Vct. In addition, the motion correction signal generation unit 31 generates a motion correction signal Spr using the signal S3 of the previous frame and the motion correction vector Vpr. An outline of this operation is shown in FIG. The signal at the point A (x, y) in the interpolation frame is the point obtained by moving the point A (x, y) by the motion correction vector Vpr (horizontal component Vprx, vertical component Vtry) in the signal S3 of the previous frame. A ′ (x1, y1) = (x + Vprx, y + Vtry) position signal, in the current frame signal S2, the point A (x, y) is moved by the motion correction vector Vct (horizontal direction component Vctx, vertical direction component Vcty). The motion correction signals Spr and Sct are generated by the following (Equation 3), corresponding to the signal at the position A ″ (x2, y2) = (x−Vctx, y−Vcty).
[0052]
[Equation 3]
Spr = S3 (x + Vprx, y + Vtry)
Sct = S2 (x−Vctx, y−Vcty) (Equation 3)
[0053]
This signal processing is realized by controlling the read operation of the memory circuit built in the motion correction signal generation unit. That is, an address is generated by shifting the memory read address by a position corresponding to the corrected interpolated motion vectors Vpr and Vct, and the pixel signals corresponding to the points A ′ and A ″ are read at this address.
[0054]
The adder 32 performs an averaging of the motion correction signals Spr and Sct of the both, and outputs the motion correction interpolation frame signal Sm in the frame order 2, 3,..., 6 shown in FIG. obtain.
The subtracting unit 33 performs subtraction between the motion correction signals Sct and Spr to detect a difference signal component MCE. When the motion correction vector is accurate, the motion correction signals Sct and Spr have the same value, so the signal MCE is zero. However, when the motion correction vector is inaccurate, the motion correction signals Sct and Spr have different values, and the signal MCE has a value other than zero.
The correction error calculation unit 29 calculates a motion correction error (corresponding to pixel conversion) in units of mini-blocks and outputs this value as a signal MBE.
[0055]
The determination unit 34 adaptively sets the threshold value TH according to the magnitude of the value of the signal MBE. For example, TH = 40 when MBE <= 8, TH = 32 when 8 <MBE <= 16, TH = 24 when 16 <MBE <= 24, and TH = 16 when 24 <MBE. When the signal MCE is less than the threshold, 0 is output to the control signal CT, and when the signal MCE is greater than or equal to the threshold, 1 is output to the control signal CT.
[0056]
When the control signal CT is 0, the selection unit 35 selects the motion correction interpolation frame signal Sm of the current frame signal S2 and the previous frame signal S3 when the control signal CT is 1. A signal having a short temporal distance is selected, and an interpolated frame signal SMC is output. By this operation, it is possible to greatly suppress image quality degradation inherent to motion correction such as isolated point degradation due to motion vector inaccuracy and degradation of the peripheral edge of a moving image edge, and high image quality can be achieved.
[0057]
FIG. 20 is a diagram illustrating a second configuration example of the motion compensation interpolation frame generation unit 8 and is suitable for performing motion speed adaptive motion compensation processing. This second configuration example is realized by a configuration in which the vector distribution measurement unit 36 is added to the first configuration example (FIG. 18).
The vector distribution measuring unit 36 measures the frequency of occurrence of a motion vector corresponding to the region E shown in FIG. 8A in the pixel-unit motion vector PV in one frame period, and this is a set value (for example, 10% or more). ) Is output to the motion correction mode signal MCFG, and 1 is output if it is less than the set value.
When the motion correction mode signal MCFG is 0, the determination unit 34 outputs a control signal CT for selecting and controlling the signal S2 of the current frame in the corresponding one frame period by the selection unit 35. On the other hand, when the motion correction mode signal MCFG is 1, the same operation as in the first configuration example is performed, and a control signal CT of 1 is output when the signal MCE is equal to or greater than a threshold determined by the signal MBE, and 0 when less.
[0058]
In this configuration, the vector distribution measurement unit 36 may be omitted, and instead, the mini block division search mode signal MBFG in the motion vector search unit in FIG. 16 described above may be used as a motion correction mode signal. It is.
As described above, according to the first embodiment of the present invention, the motion correction of an image signal that requires a small amount of calculation for signal processing for motion vector search and generation and that has little image quality degradation caused by motion correction processing. A frame number conversion device can be realized. In addition, remarkable effects can be obtained in high image quality and low cost.
[0059]
Next, a second embodiment of the present invention will be described with reference to FIGS.
FIG. 21 is a block diagram of the present embodiment, omitting the memory unit 9 of the motion correction signal processing unit of the first embodiment (FIG. 1), and replacing the frame rate increasing unit 37 with the IP converting unit 1 instead. It is arranged between the 1-frame delay unit 2 and configured.
The signal sequence S2 of the progressive scan obtained by performing the interlaced to sequential scan conversion in the IP conversion unit 1 is subjected to frame number conversion by frame repetition processing in the frame rate increasing unit 37. For example, the signal sequence S2 having a frame frequency of 50 Hz Conversion to signal sequence S5.
[0060]
An outline of this operation is shown in FIG. This frame order 1 frame is repeated twice for the signal sequence of S2 (50 Hz) frame order 1, 2, 3, 4, 5, 1, 2,. The signal sequence is converted to 2, 3, 4, 5, 1, 1, 2,. Then, the motion vector search unit 3 and the motion compensation signal processing unit 7 perform the S5 as a signal sequence of the current frame, the signal S6 obtained by delaying the S5 by the 1 frame delay unit 2 as a signal sequence of the previous frame, The same signal processing as in the embodiment is performed. Therefore, when the signal sequence of the current frame and the previous frame are in the same frame order 1, a still image is obtained, all the motion vectors of the frame are processed with 0, and the output signal series S4 (60 Hz) is a signal in the frame order 1. Get. On the other hand, when the signal sequence of the current frame is different from that of the previous frame, the output signal sequence S4 obtains a signal sequence (indicated by MC in the figure) generated by the motion correction process.
[0061]
According to the second embodiment of the present invention, as in the first embodiment, the amount of calculation required for the signal processing for motion vector search and generation is small, and an image signal with little image quality degradation caused by the motion correction processing is obtained. A motion correction frame number conversion device can be realized. In addition, remarkable effects can be obtained in high image quality and low cost.
[0062]
Next, a third embodiment of the present invention will be described with reference to FIGS. The present embodiment is suitable for performing a conversion vector search process in a motion vector search.
FIG. 23 is a block diagram of the present embodiment, and only the configuration and operation of the block unit motion vector search unit 38 are different from those of the first embodiment (FIG. 1).
FIG. 24 is a diagram showing this configuration. In FIG. 24, a P vector conversion unit 39 and a B vector conversion unit 40 perform vector conversion processing of motion vector information MV used in image encoding, and generate conversion vectors MVp and MVb per frame, respectively.
[0063]
FIG. 25 is a diagram showing an outline of this operation. In image coding, particularly, international standard MPEG video coding, as shown in FIG. 5A, an image signal sequence is divided into I picture, P picture, and B picture. (Cosine transform) encoding, unidirectional MC encoding + DCT encoding for P pictures, and bidirectional MC encoding + DCT encoding for B pictures.
FIG. 5B shows motion vectors PV1, PV2,... Used for encoding a P picture. This motion vector corresponds to a motion vector in n frames (n = 3 in the figure) between P pictures of the image signal sequence. Therefore, the P vector conversion unit 39 generates a conversion vector MVp with PV1 / 3, PV2 / 3,... Obtained by converting this motion vector into 1 / n (1/3 in the figure).
FIG. 4C shows motion vectors BV11, BV12, BV21, BV22... Used for encoding a B picture. Among these, BV11, BV21, BV31,... Correspond to motion vectors between one frame. Therefore, the B vector conversion unit 40 generates a converted motion vector MVb using these motion vectors.
[0064]
Referring back to FIG. 24, the determination unit 41 calculates a motion correction error in units of blocks based on the conversion vectors MVp and MVb for a moving image block whose motion detection signal MD1 is 1, and outputs a smaller value to the representative motion vector BVT. .
The block matching second processing unit 14 performs a re-search process starting from the representative motion vector BVT, and detects a motion vector BV in block units. For still blocks, 0 is output to BV.
The control unit 42 generates control signals necessary for these operations.
According to the third embodiment of the present invention, an apparatus for converting the number of motion-corrected frames of an image signal that requires a smaller amount of computation for motion vector search and generation signal processing than the first and second embodiments is realized. it can. In addition, remarkable effects can be obtained in high image quality and low cost.
[0065]
Next, a fourth embodiment of the present invention is shown in a block diagram of FIG. In the present embodiment, the memory unit 9 of the motion correction signal processing unit of the third embodiment (FIG. 23) is omitted, and the frame rate increasing unit 37 shown in the second embodiment (FIG. 21) is used instead. It is arranged between the IP conversion unit 1 and the 1-frame delay unit 2. Since this operation can be easily understood from the first to third embodiments, the description thereof is omitted.
According to the fourth embodiment of the present invention, as in the third embodiment, it is possible to realize a motion-correction frame number conversion apparatus for an image signal that requires a smaller amount of calculation for signal processing for motion vector search and generation. In addition, remarkable effects can be obtained in high image quality and low cost.
[0066]
Next, a fifth embodiment of the present invention will be described with reference to FIGS. This embodiment is suitable for performing an operation of stopping signal processing for motion vector search and motion correction frame interpolation in a scene change area.
FIG. 27 is a block diagram of this configuration, which is realized by adding a scene change detection unit 391 to the configuration of the first embodiment (FIG. 1).
[0067]
The scene change detection unit 391 performs an operation of detecting a frame in which a scene change has occurred based on the generation form of the inter-frame difference signal component in one frame period. An example of this configuration is shown in FIG. The subtraction unit 401 performs a subtraction operation on the luminance signal components of the signal S2 of the current frame and the signal S3 of the previous frame, and extracts a difference component FD between one frame. In general, in the scene change region, the content of the image is switched, so that the signal level of the difference component FD has a relatively large value. Therefore, the binary quantization unit 411 performs binary quantization on the signal FD for each pixel with a relatively high threshold value ± Thb. Then, 0 is output to the signal QS when the pixel is less than the threshold value ± Thb, and 1 is output when the pixel exceeds the threshold value. The 1-frame accumulating unit 421 measures the number of pixels having the signal QS of 1 in one frame period, and outputs the accumulated value AQ for one frame period. The determination unit 431 determines that the scene change has occurred when the cumulative value AQ is the number of pixels that is half or more of the entire screen and the occurrence is limited to one frame period. The reason for this is to avoid a malfunction that erroneously detects the movement of the horizontal pan or the vertical pan that moves at a uniform speed as a scene change. Then, 1 is output to the signal SCFG in a frame where a scene change has occurred, and 0 is output in other frames.
[0068]
Returning to FIG. 27, the motion vector search unit 3 and the motion correction signal processing unit 7 stop the signal processing for motion vector search and motion correction frame interpolation in a frame where a scene change has occurred with the signal SCFG of 1. In this frame, the signal S2 of the current frame is output to the signal sequence S4.
On the other hand, in the frame in which the signal SCFG is 0, the motion vector search unit 3 and the motion correction signal processing unit 7 perform the same signal processing operation as in the first embodiment, and in the signal sequence S4, the motion correction frame interpolation processing is performed. A signal sequence obtained by converting the number of frames is obtained.
According to the fifth embodiment of the present invention, it is possible to avoid the generation of a large amount of calculation for motion vector search and generation in the scene change area, and the motion vector search and generation signal compared to the first embodiment. It is possible to realize a motion-correction frame number conversion apparatus for image signals that requires a smaller amount of computation. In addition, remarkable effects can be obtained in high image quality and low cost.
[0069]
Next, sixth to eighth embodiments of the present invention will be described with reference to FIGS. These are all suitable for performing an operation of stopping the signal processing of motion vector search and motion correction frame interpolation in the scene change region.
FIG. 29 is a block diagram of the sixth embodiment, which is realized by adding a scene change detector 391 to the configuration of the second embodiment (FIG. 21).
FIG. 30 is a block diagram of the seventh embodiment, which is realized by adding a scene change detector 391 to the configuration of the third embodiment (FIG. 23).
FIG. 31 is a block diagram showing an example of the eighth embodiment, which is realized by adding a scene change detection unit 391 to the configuration of the fourth embodiment (FIG. 26).
In addition, since the operation | movement in these Examples can be easily understood by the above description, description is abbreviate | omitted.
According to the sixth to eighth embodiments of the present invention, as in the fifth embodiment, it is possible to realize a motion-correction frame number conversion device for an image signal that requires a smaller amount of calculation for motion vector search and generation signal processing. . In addition, remarkable effects can be obtained in high image quality and low cost.
[0070]
A ninth embodiment of the present invention will be described with reference to FIGS.
FIG. 32 is a block diagram of this embodiment. As shown in the figure, in the previous first embodiment (FIG. 1), the IP conversion unit is arranged in the preceding stage of the motion vector search unit, whereas in the present embodiment, in the subsequent stage. Arrange and configure.
The 1-frame delay unit 2 generates a signal S20 obtained by delaying the input image signal S1 (brightness signal component and color difference signal component) of interlaced scanning by one frame period. The signals S1 and S20 are supplied to the motion vector search unit 3 and the motion correction signal processing unit 7.
[0071]
The motion vector search unit 3 includes a motion detection unit 4, a block unit motion vector search unit 5, and a pixel unit motion vector generation unit 6.
The motion detection unit 4 extracts a difference signal between one frame by a subtraction process between the signal sequence S1 of the current frame and the luminance signal component of the signal sequence S20 of the previous frame, and binary-quantizes this to detect the motion detection signal MD1. , MD2 is output. This configuration and operation are the same as those of the motion detection unit (FIG. 2) of the first embodiment described above. However, in the figure and the description thereof, S2 is read as S1, and S3 is read as S20.
[0072]
The block unit motion vector search unit 5 detects a motion vector in block units (for example, 16 pixels × 16 lines or 8 pixels × 8 lines). That is, a block with a motion detection signal MD1 of 0 is determined as a still block, and 0 is output to the block unit motion vector BV. On the other hand, the block with MD1 of 1 is determined to be a moving image block, and the above-described double representative point tree search process or motion vector distribution adaptive search process is performed on the luminance signal components of the signals S1 and S20, and the block unit motion vector BV Is detected. This configuration and operation are the same as those of the block unit motion vector search unit (FIGS. 3 to 8) of the first embodiment described above. However, in the figure and the description thereof, S2 is read as S1 and S3 as S20.
[0073]
The pixel unit motion vector generation unit 6 generates a pixel unit motion vector PV by mini-block division search processing. That is, the pixel-unit motion vector PV is detected in units of mini-blocks (for example, 2 pixels × 2 lines) by the above-described block prediction error adaptive search processing, block mini-block prediction error adaptive search processing, or V edge adaptive search processing. Further, a pixel whose motion detection signal MD2 is 0 is determined as a still pixel and 0 is assigned to PV. This configuration and operation are the same as those of the pixel unit motion vector generation unit (FIGS. 9 to 15) of the first embodiment described above. However, in the figure and the description thereof, S2 is read as S1 and S3 as S20.
[0074]
The motion vector search unit of the first embodiment described above (FIG. 16) is also suitable for the configuration and operation of the motion vector search unit suitable for performing motion correction processing in units of frames in the motion correction signal processing unit described later. To FIG. 17). However, in the figure and the description thereof, S2 is read as S1 and S3 as S20.
[0075]
The motion correction signal processing unit 7 includes an IP conversion unit 71 and an MC frame number conversion unit 72.
The IP conversion unit 71 performs the interlaced-sequential scan conversion of the motion adaptation process according to the magnitude of the motion vector PV and the inter-frame difference signal component, and generates a sequential scan signal series S30. This specific configuration will be described later.
The MC frame number conversion unit 72 creates a motion correction vector based on the pixel unit motion vector PV, and uses the signal obtained by moving the position of the image of the signal sequence S30 of the progressive scan by this motion correction vector. And sequentially output image signal sequence S4 (luminance signal component and color difference signal component) whose frame frequency is increased by the motion correction frame number conversion process. This specific configuration will also be described later.
The description of the overall block configuration diagram has been completed, and the configuration of the motion correction signal processing unit 7 in the present embodiment will be described in detail below.
[0076]
FIG. 33 shows an example of the configuration of the IP conversion unit 71, which is suitable for performing interlaced-sequential scan conversion of motion adaptation processing using a motion vector.
This configuration is shown in FIG. The MA interpolation unit 78 generates an interpolated scanning line signal necessary for scan conversion. Among these, the moving image interpolation signal generator 79 performs in-field processing (for example, the average value of the signals of the upper and lower scanning lines), and generates an interpolated scanning line signal SM suitable for moving images. Further, the still interpolation signal generation unit 80 performs inter-field processing (for example, the scanning line signal of the previous frame), and generates an interpolation scanning line signal SS suitable for a still image.
The coefficient weighting unit 81-1 weights motion coefficient k, 81-2 weights 1-k, and the addition unit 82 adds both signals to generate an interpolated scanning line signal.
[0077]
As shown in FIG. 4B, the motion coefficient generation unit 83 increases or decreases the absolute value | S1-S20 | of the luminance signal difference component between one frame and the scalar quantity | PV | of the motion vector PV. Accordingly, the coefficient value k of the motion coefficient is set. That is, the coefficient value k is set by a straight line having a characteristic in which the slope gradually increases as the scalar quantity | PV | increases from small to large. Therefore, even if | S1-S20 | is the same value, the value of k can be set larger in proportion to the speed of motion, and the motion adaptive signal processing matched with the motion of the image can be performed as compared with the prior art.
The double speed conversion unit 84 performs 1/2 time compression and time-series multiplexing signal processing of the generated interpolated scanning line signal and S1, and outputs a signal series S30 for sequential scanning.
[0078]
Next, the MC frame number conversion unit 72 will be described with reference to FIGS.
FIG. 34 is a first configuration example and is suitable for performing prediction error adaptive motion correction processing.
In the configuration example of FIG. 34, the 1-frame memory unit 85 generates the signal sequence S3P of the previous frame obtained by delaying the signal sequence S30 of the current frame of progressive scanning by one frame period.
The MC vector generation unit 86 generates motion correction vectors Vct and Vpr necessary for the motion correction process from the pixel unit motion vector PV detected in the interlaced scanning sequence. Then, the motion correction signal generation unit 87 generates a motion correction signal Sct using the current frame signal S30 and the motion correction vector Vct. In addition, the motion correction signal generation unit 88 generates a motion correction signal Spr from the signal S3P of the previous frame and the motion correction vector Vpr.
[0079]
The outline of these operations is as described above with reference to FIGS. However, PV and PV 'shall be read as PV' in the figure and its description. Here, PV ′ is a pixel-by-pixel motion vector in the sequential scanning sequence, and the relationship PV ′ = PV / 2 is established with the motion vector PV detected in the interlaced scanning sequence. Therefore, the interpolation frame in the frame order 2 is Vpr = PV ′ * 5/6 (PV * 5/12), Vct = −PV ′ * 1/6 (−PV * 1/12) (ka = 5, kb) = 1), the frame order 3 is Vpr = PV ′ * 4/6 (PV * 4/12), Vct = −PV ′ * 2/6 (−PV * 2/12) (ka = 4, kb) = 2)..., A motion correction vector having no positional deviation in the time direction is generated.
[0080]
The adding unit 89 performs addition averaging of both motion correction signals Spr and Sct to obtain a motion correction interpolation frame signal Smc in frame order 2, 3,...
The subtracting unit 90 performs subtraction between the motion correction signals Sct and Spr to detect a difference signal component MCE. When the motion correction vector is accurate, the motion correction signals Sct and Spr have the same value, so that the signal MCE is 0. However, when the motion correction vector is inaccurate, the motion correction signals Sct and Spr have different values, and the signal MCE has a value other than zero.
The correction error calculation unit 91 calculates a motion correction error (corresponding to pixel conversion) in units of mini-blocks and outputs this value as a signal MBE.
[0081]
The determination unit 92 adaptively sets the threshold value TH according to the magnitude of the value of the signal MBE. For example, TH = 40 when MBE <= 8, TH = 32 when 8 <MBE <= 16, TH = 24 when 16 <MBE <= 24, and TH = 16 when 24 <MBE. When the signal MCE is less than the threshold, 0 is output to the control signal SL.
[0082]
When the control signal SL is 0, the selection unit 93 selects the motion correction interpolation frame signal Smc of the current frame signal S30 and the previous frame signal S3P when the control signal SL is 1. A signal having a close temporal distance is selected, and an interpolation frame signal S50 is output. By this operation, it is possible to greatly suppress image quality degradation inherent to motion correction such as isolated point degradation due to motion vector inaccuracy and degradation of the peripheral edge of a moving image edge, thereby achieving high image quality.
The memory unit 94 performs the following writing and reading operations to obtain a signal series S4 obtained by converting the number of frames. In the write operation, the signal in the frame order 1 of the signal S30 of the current frame and the signals in the frame order 2 to 6 of the signal S50 are written. On the other hand, in the read operation, signals in the frame order 1 to 6 are sequentially read.
[0083]
FIG. 35 is a block diagram of the second configuration example, which is suitable for performing motion speed adaptive motion correction processing. This is realized by a configuration in which a vector distribution measuring unit 95 is added to the first configuration example (FIG. 34).
The vector distribution measuring unit 95 measures the frequency of occurrence of a motion vector corresponding to the region E shown in FIG. 8A among the motion vectors PV in pixel units in one frame period, and this is a set value (for example, If it exceeds 10%, 0 is output to the motion correction mode signal MCFG, and 1 is output if it is less than the set value.
[0084]
When the motion correction mode signal MCFG is 0, the determination unit 92 outputs a control signal SL for selecting and controlling the signal S30 of the current frame in the corresponding one frame period by the selection unit 93. On the other hand, when the motion correction mode signal MCFG is 1, the same operation as in the first configuration example is performed, and a control signal SL of 1 is output when the signal MCE is equal to or greater than a threshold determined by the signal MBE, and 0 when it is less.
In the present configuration, the vector distribution measuring unit 95 may be omitted, and instead, the mini block division search mode signal MBFG in the motion vector search unit of FIG. 16 described above may be used as a motion correction mode signal. It is.
[0085]
FIG. 36 shows the third configuration example, which is realized by omitting the last stage memory unit 94 of the first configuration example of FIG. 34 and replacing it with the frame rate increase unit 97 instead.
The frame rate increase unit 97 performs frame number conversion by frame repetition processing, and converts, for example, a progressive scanning signal sequence S30 having a frame frequency of 50 Hz into a signal sequence S10 having a frame frequency of 60 Hz. Signal processing similar to that in the first configuration example is performed thereafter, with S10 as a signal sequence of the current frame and a signal S10P obtained by delaying this S10 by one frame delay unit 35 as a signal sequence of the previous frame.
[0086]
The outline of this operation is as described above with reference to FIG. However, in the figure and the description thereof, S2 is read as S30, S5 as S10, and S6 as S10P. Therefore, when the signal sequence of the current frame and the previous frame are in the same frame order 1, a still image is obtained, all the motion vectors of the frame are processed with 0, and the output signal series S4 (60 Hz) is a signal in the frame order 1. Get. On the other hand, when the signal sequence of the current frame is different from that of the previous frame, the output signal sequence S4 obtains a signal sequence (indicated by MC in FIG. 22) generated by the motion correction process.
[0087]
FIG. 37 shows the fourth configuration example in which the final stage memory unit 94 in the second configuration example in FIG. 35 is omitted and a frame rate increase unit 97 is arranged at the head instead. Since this operation is the same as that of the above-described configuration example, description thereof is omitted.
According to the ninth embodiment of the present invention, it is possible to realize an apparatus for converting the number of frames of an image signal that requires a small amount of calculation for signal processing for motion vector search and generation and that has little image quality deterioration caused by motion correction processing. In addition, remarkable effects can be obtained in high image quality and low cost.
[0088]
A tenth embodiment of the present invention will be described with reference to FIG. This embodiment is suitable for performing conversion vector search processing in motion vector search. This figure shows an example of this block configuration, and only the configuration and operation of the block unit motion vector search unit 98 is different from the previous ninth embodiment (FIG. 32). The configuration and operation of this block unit motion vector search unit 98 are as described above with reference to FIGS. However, in the figure and the description thereof, S2 is read as S1 and S3 as S20.
According to the tenth embodiment of the present invention, it is possible to realize an image signal frame number conversion apparatus that requires a smaller amount of calculation for motion vector search and generation signal processing. In addition, remarkable effects can be obtained in high image quality and low cost.
[0089]
An eleventh embodiment of the present invention will be described with reference to FIG. This is suitable for performing an operation to stop the signal processing for motion vector search and motion correction frame interpolation in the scene change area. This figure shows a block configuration example, which is realized by adding a scene change detection unit 99 to the configuration of the ninth embodiment (FIG. 32). The scene change detection unit 99 performs an operation of detecting a frame in which a scene change has occurred based on the generation form of the inter-frame difference signal component in one frame period. This configuration and operation are as described above with reference to FIG. However, in the figure and the description thereof, S2 is read as S1 and S3 as S20.
[0090]
Returning to FIG. 39, the motion vector search unit 3 and the motion correction signal processing unit 7 stop the signal processing for motion vector search and motion correction frame interpolation in a frame in which a scene change has occurred with the signal SCFG of 1. On the other hand, in the frame in which the signal SCFG is 0, the motion vector search unit 3 and the motion correction signal processing unit 7 perform the same signal processing operation as in the first embodiment, and the signal sequence S4 is subjected to motion correction frame interpolation processing. A signal sequence obtained by converting the number of frames is obtained.
According to the eleventh embodiment of the present invention, it is possible to avoid the generation of a huge amount of calculation for motion vector search and generation in the scene change region, and the motion vector search and generation signal compared to the ninth embodiment. An apparatus for converting the number of frames of an image signal that requires a smaller amount of computation for processing can be realized. In addition, remarkable effects can be obtained in high image quality and low cost.
[0091]
A twelfth embodiment of the present invention will be described with reference to FIG. This is realized by adding a scene change detection unit 99 to the configuration of the tenth embodiment (FIG. 38). Since this operation can be easily understood in the description of the above embodiment, the description thereof is omitted. As in the eleventh embodiment, it is possible to realize a frame number conversion apparatus for image signals that requires a smaller amount of calculation for motion vector search and generation signal processing. In addition, remarkable effects can be obtained in high image quality and low cost.
[0092]
Next, an embodiment in which the frame number conversion apparatus of the present invention is applied to a television receiver will be described with reference to FIGS.
[0093]
FIG. 41 is a block configuration example of the first embodiment, which is suitable for displaying an image in the form of sequential scanning. In the figure, a tuner unit 44 demodulates the terrestrial broadcast signal TV into a baseband band television signal. The BS tuner unit 45 demodulates the satellite broadcast signal BS into a baseband band television signal. The decoder unit 46 performs predetermined demodulation processing of the analog television signal, and demodulates the luminance signal and the color difference signal. The IRD unit 47 performs a predetermined decoding process on the digital television signal and demodulates the luminance signal and the color difference signal. As the external input signal Ex, a luminance signal and a color difference signal of a package medium such as a VCR or a PC image are input. The switch unit 48 selects a signal series specified by a control signal from the control unit 54.
[0094]
The IP conversion unit 49 performs, for example, motion adaptive scan conversion signal processing on the interlaced scanning signal sequence, and converts the signal sequence into a progressive scanning signal sequence. The MC frame number conversion unit 50 corresponds to the frame number conversion apparatus of the present invention, and generates a signal sequence in which the frame frequency is increased by motion compensation frame interpolation processing. For example, in the 50 Hz to 60 Hz conversion of a signal with a frame frequency of 50 Hz in the PAL system, the frame order 1 to 6 (2 to 6 are motion correction frame interpolation processes) from the input signal sequence of the frame order 1 to 5 as in the above-described embodiment. Generate a signal sequence. Further, in the 50 Hz → 75 Hz conversion, a signal sequence in the frame order 1 to 3 (2 to 3 is generated by the motion correction frame interpolation process) is generated from the input signal sequence in the frame order 1 or 2. Alternatively, in the 50 Hz → 100 Hz conversion, a signal sequence of frame order 1 to 2 (2 is generated by motion correction frame interpolation processing) is generated from an input signal sequence of frame order 1.
[0095]
The scaling processing unit 51 performs signal processing such as image size enlargement / reduction, aspect ratio conversion, and scanning line number conversion. The image quality improvement unit 52 performs image quality improvement signal processing such as sharpness improvement and luminance gradation correction and conversion into three primary color signals by color space conversion. The sequential display unit 53 displays an image in the form of sequential scanning with a frame frequency of 60 Hz, or 75 Hz or 100 Hz.
[0096]
The remote control receiving unit 55 receives various user information (channel selection, display mode, etc.) that the user operates with the remote control terminal. The control unit 54 generates control signals (not shown in the drawing) necessary for the operation of each unit according to the received various user information, and outputs the control signals to the controlled unit 58.
[0097]
FIG. 42 is a block configuration example of the second embodiment, which is suitable for displaying an image in the form of interlaced scanning. In this configuration, the PI conversion unit 56 is added to the configuration of FIG. 41 described above, and an interlaced display unit 57 displays an image in the form of interlaced scanning. The PI conversion unit 56 performs 2: 1 decimation processing of the scanning lines of the sequential scanning signal series, and performs signal processing for conversion to the interlaced scanning signal series. The interlace display unit 57 displays an image in the form of interlaced scanning with a field frequency of 60 Hz, or 75 Hz or 100 Hz. Since the other blocks perform the same operation as that of FIG. 41, description thereof is omitted.
[0098]
FIG. 43 is a block configuration example of the third embodiment, which is suitable for displaying images in the form of progressive scanning. This is provided with a motion correction frame number conversion unit 63 in place of the IP conversion unit 41 and the MC frame number conversion unit 50 of FIG. 41 described above. The motion correction frame number conversion unit 63 corresponds to the frame number conversion device of the present invention, and generates a sequential scanning signal sequence in which the frame frequency is increased by motion correction frame interpolation processing. For example, in the 50 Hz to 60 Hz conversion of a signal with a frame frequency of 50 Hz in the PAL system, the frame order 1 to 6 (2 to 6 are motion correction frame interpolation processes) from the input signal sequence of the frame order 1 to 5 as in the above-described embodiment. Generate a signal sequence. Further, in the 50 Hz → 75 Hz conversion, a signal sequence in the frame order 1 to 3 (2 to 3 is generated by the motion correction frame interpolation process) is generated from the input signal sequence in the frame order 1 or 2. Alternatively, in the 50 Hz → 100 Hz conversion, a signal sequence of frame order 1 to 2 (2 is generated by motion correction frame interpolation processing) is generated from an input signal sequence of frame order 1. Since the other blocks perform the same operation as that of FIG. 41, description thereof is omitted.
[0099]
FIG. 44 shows an example of the block configuration of the fourth embodiment, which is suitable for displaying an image in the form of interlaced scanning. This is because the PI conversion unit 56 is added to the configuration of FIG. 43 described above, and an interlaced display unit 57 displays an image in the form of interlaced scanning. The PI conversion unit 56 and the interlace display unit 57 are the same as those described in the second embodiment of FIG. Since the other blocks perform the same operations as those in FIG. 43, description thereof is omitted.
According to these embodiments of the television receiver, a multi-source compatible high-quality device can be realized at low cost.
[0100]
This is the end of the description of the embodiment, but the present invention relates to PAL-NTSC television signal system conversion, PAL 50 Hz-60 Hz / 75 Hz / 100 Hz conversion, film image (frame frequency 24 Hz / 30 Hz) -60 Hz conversion, television. It is applied to various frame number conversion devices such as signal-computer image signal conversion, and has a remarkable effect on image quality and cost reduction.
[0101]
As described above, according to the present invention, it is possible to realize a motion correction frame conversion apparatus that has a small amount of signal processing and circuit scale required for motion vector search necessary for motion correction, and that suppresses image quality degradation inherent to motion correction. For this reason, the remarkable effect is acquired in the high image quality and cost reduction of various frame number conversion apparatuses.
In addition, by applying the present invention to a television receiver, a multi-source compatible high-quality and low-cost receiver can be realized. In addition, it has a remarkable effect on the multi-function of the receiver.
[0102]
【The invention's effect】
According to the present invention, it is possible to obtain a method and an apparatus for converting the number of frames of an image signal with high image quality and a small circuit scale.
[Brief description of the drawings]
FIG. 1 is a block diagram of a first embodiment of the present invention.
FIG. 2 is a diagram illustrating a configuration example of a motion detection unit.
FIG. 3 is a diagram illustrating a first configuration example of a block unit motion vector search unit;
FIG. 4 is a flowchart of block unit motion vector search according to the first configuration example;
FIG. 5 is an arrangement diagram of representative points in the first block matching process.
FIG. 6 is a diagram illustrating a second configuration example of a block unit motion vector search unit;
FIG. 7 is a flowchart of a block unit motion vector search of a second configuration example.
8A is a diagram illustrating an example of measurement of motion vector occurrence frequency, and FIG. 8B is a diagram illustrating an example of search mode setting.
FIG. 9 is a diagram illustrating a first configuration example of a pixel unit motion vector generation unit;
FIG. 10 is a flowchart of pixel unit motion vector generation processing according to the first configuration example;
FIG. 11 is a diagram showing an outline of an operation for generating a pixel unit motion vector.
FIG. 12 is a diagram illustrating a second configuration example of a pixel unit motion vector generation unit.
FIG. 13 is a flowchart of pixel unit motion vector generation processing of the second configuration example;
FIG. 14 is a diagram illustrating a third configuration example of a pixel unit motion vector generation unit;
FIG. 15 is a flowchart of pixel unit motion vector generation processing of the third configuration example;
FIG. 16 is a diagram illustrating another configuration example of the motion vector search unit.
FIG. 17 is a signal processing flowchart of motion vector search.
FIG. 18 is a diagram illustrating a first configuration example of a motion compensation interpolation frame generation unit.
FIGS. 19A and 19B are diagrams illustrating an operation outline of the MC vector generation unit, and FIG. 19C is a diagram illustrating an operation outline of the motion correction signal generation unit;
FIG. 20 is a diagram illustrating a second configuration example of the motion compensation interpolated frame generation unit.
FIG. 21 is a block diagram of a second embodiment of the present invention.
FIGS. 22A to 22D are diagrams for explaining an outline of the operation of the frame rate increase unit. FIG.
FIG. 23 is a block diagram of a third embodiment of the present invention.
FIG. 24 is a diagram illustrating a configuration example of a block unit motion vector search unit.
FIGS. 25A to 25C are diagrams each illustrating an outline of the operation of the vector conversion unit.
FIG. 26 is a block diagram of a fourth embodiment of the present invention.
FIG. 27 is a block diagram of a fifth embodiment of the present invention.
FIG. 28 is a diagram illustrating a configuration example of a scene change detection unit.
FIG. 29 is a block diagram of a sixth embodiment of the present invention.
FIG. 30 is a block diagram of a seventh embodiment of the present invention.
FIG. 31 is a block diagram of an eighth embodiment of the present invention.
FIG. 32 is a block diagram of a ninth embodiment of the present invention.
FIGS. 33A and 33B are diagrams illustrating a configuration example of an IP conversion unit, respectively.
FIG. 34 is a diagram illustrating a first configuration example of an MC frame number conversion unit.
FIG. 35 is a diagram illustrating a second configuration example of the MC frame number conversion unit.
FIG. 36 is a diagram illustrating a third configuration example of the MC frame number conversion unit.
[Fig. 37] Fig. 37 is a diagram illustrating a fourth configuration example of the MC frame number conversion unit.
FIG. 38 is a block diagram of a tenth embodiment of the present invention.
FIG. 39 is a block diagram of an eleventh embodiment of the present invention.
FIG. 40 is a block diagram of a twelfth embodiment of the present invention.
FIG. 41 is a diagram showing a first embodiment of a television receiver according to the present invention.
FIG. 42 is a diagram showing a second embodiment of the television receiver according to the present invention.
FIG. 43 is a diagram showing a third embodiment of the television receiver according to the present invention.
FIG. 44 is a diagram showing a fourth embodiment of the television receiver according to the present invention.
[Explanation of symbols]
1 IP converter
2 1 frame delay
3 Motion vector search unit
4 Motion detector
5 Block unit motion vector search unit
6 Pixel unit motion vector generator
7 Motion compensation signal processor
8 Motion compensation interpolation frame generator
9 Memory part

Claims (2)

A motion detector that detects a still block or a moving image block based on the input image signal;
A block unit motion vector search unit that detects a block unit motion vector by block matching processing on the video block;
When the motion correction error of the block unit motion vector between the previous frame and the current frame of the image signal is less than the threshold value, a block unit motion vector is assigned to all pixels in the block, and the motion correction error is greater than or equal to the threshold value. In some cases, the previous frame of the image signal in the area included in the current block by including the mini block among the motion vectors of the current block and the adjacent block with respect to the mini block obtained by subdividing the block in the horizontal direction and the vertical direction. A pixel unit motion vector generation unit that allocates a motion correction error between the current frame and the current frame to a pixel motion vector in the mini-block;
A motion correction vector generation unit that generates a motion correction vector by multiplying a coefficient for matching with a frame position to be interpolated into the motion vector in pixel units, and an image position of the current frame by an output signal of the motion correction vector generation unit A motion correction signal generation unit that generates a first motion correction signal that has moved the position and a second motion correction signal that has moved the position of the image of the previous frame, and includes the first motion correction signal and the second motion correction signal. When the difference component of the motion correction signal is less than the threshold value, an interpolation frame is generated using the first and second motion correction signals. When the difference component is equal to or greater than the threshold value, the interpolation frame is determined based on the signal of the current frame or the previous frame. A motion compensation interpolation frame generation unit that generates a pixel unit motion vector generation unit that includes a vertical edge detection unit that detects a vertical edge of the image signal; In the mini-block including the detected vertical edge, the frame number conversion of the image signal is characterized in that the motion correction error is calculated in a horizontally long region having a longer horizontal component than in the case of including the mini-block and not including the vertical edge. apparatus. In the image signal frame number conversion device according to claim 1,
The vertical edge detection unit includes a low pass filter having a horizontal frequency reduction component as a pass band, and a high pass filter having a pass band as a high frequency component of the vertical frequency,
A frame number conversion device for detecting a vertical edge by extracting a signal that has passed through the low-pass filter and the high-pass filter from a luminance signal component of a current frame of the image signal.

Images