JP3777593B2 - Image processing apparatus and method, recording medium, and program - Google Patents

Description

Eiは、下位階層特徴量を示する。wiは、予測係数を示す。
【０１２７】
例えば、９つのデータに、上述の１ビットＡＤＲＣを適用し、５１２クラスに分類したとき、各クラス毎に生成された予測係数と下位階層特徴量との積和演算により動き評価値E'が予測される。
【０１２８】
例えば、直交する２つの動き評価値E'を算出し、それぞれ動きベクトルのｘ成分およびｙ成分とし、動きベクトルを算出することができる。また、例えば、ｎ個の動き評価値E'を算出し、それぞれが方向が異なるｎ個の単位ベクトルの大きさを示すものとし、ｎ個の動き評価値E'のそれぞれを、ｎ個の単位ベクトルのそれぞれに乗算して、乗算された結果を加算することにより、動きベクトルを算出することができる。
【０１２９】
また、適応処理により、動きベクトルに限らず、例えば、コンポーネントビデオ信号を予測するとき、式（６）に基づく演算により、Ｙ信号に対応する評価値E'、Ｕ信号に対応する評価値E'、Ｖ信号に対応する評価値E'を個々に算出することができる。
【０１３０】
上述の予測係数は、予め学習により生成しておくため、ここで、その学習について説明する。
【０１３１】
式（６）のモデルに基づく予測係数を最小自乗法により生成する例を示す。
【０１３２】
最小自乗法は、次のように適用される。一般化した例として、Xを入力データ、Wを予測係数、Yを予測値として式（７）を考える。
観測方程式：XW=Y （７）
【０１３３】
【数２】

【０１３４】
上述の観測方程式により収集されたデータに最小自乗法を適用する。式（６）の例においては、nは、25であり、mは、学習データの数である。
【０１３５】
式（７）の観測方程式を基に、式（９）の残差方程式を考える。
【０１３６】
【数３】

【０１３７】
式（９）の残差方程式から、各wiの最確値は、式（１０）に示す値を最小にする条件が成り立つ場合に対応する。
【０１３８】
【数４】

【０１３９】
すなわち、式（１１）の条件を考慮すればよい。
【０１４０】
【数５】

【０１４１】
式（１１）のiに基づくn個の条件を満たすw1乃至wnを算出すればよい。そこで、残差方程式（９）から式（１２）が得られる。
【０１４２】
【数６】

【０１４３】
式（１１）および式（１２）により、式（１３）が得られる。
【０１４４】
【数７】

【０１４５】
式（１０）および式（１３）から、正規方程式（１４）が得られる。
【０１４６】
【数８】

【０１４７】
式（１４）の正規方程式において、未知数の数nと同じ数の方程式を立てることが可能であるので、確wiの最確値を求めることができる。例えば、掃き出し法（Gauss-Jordanの消去法）を用いて、式（１４）が解かれる。
【０１４８】
以上のようにして、クラスごとに最適な予測係数wを求め、さらに、その予測係数wを用い、式（６）により、教師信号の動きベクトルに近いベクトルを示す予測値E'を求めるのが適応処理である。
【０１４９】
なお、コンポーネントビデオ信号を予測するとき、コンポーネントビデオ信号を予測するための、クラスごとに最適な予測係数wを求める必要がある。
【０１５０】
また、適応処理は、式（６）だけを見る限りは、いわゆる補間フィルタを用いての補間処理と同一であるが、その補間フィルタのタップ係数に相当する予測係数wが、教師信号を用いての、いわば学習により求められるため、元の動きベクトルを再現することができる。このことから、適応処理は、いわば創造作用がある処理ということができる。
【０１５１】
図１８は、クラス分類適応処理により下位階層ベクトルを検出する、下位階層動きベクトル検出部２６の他の構成を示すブロック図である。
【０１５２】
クラス分類部５１は、サブキャリア位相情報、および下位階層特徴量を基に、クラス分類し、分類されたクラスを示すクラスコードを予測タップ抽出部５２に供給する。例えば、クラス分類部５１は、注目データの位相に対応して、クラス分類する。また、例えば、クラス分類部５１は、注目データに対応する、所定の数の下位階層特徴量にＡＤＲＣ処理を適用し、クラス分類する。
【０１５３】
予測タップ抽出部５２は、クラスコードで示されるクラス、メモリ５５から供給される、前のフィールドに対応する下位階層ベクトル、および上位階層ベクトルを基に、所定の位置の、所定の数の下位階層特徴量を抽出して、抽出した下位階層特徴量を予測タップとして演算部５３に供給する。
【０１５４】
演算部５３は、予測係数メモリ５４から供給される、所定の数の予測係数からなる係数セットを基に、例えば、式（６）に示す演算を実行して、下位階層ベクトルを予測する。演算部５３は、予測された下位階層ベクトルをメモリ５５に供給するとともに、外部に出力する。
【０１５５】
メモリ５５は、演算部５３から供給された下位階層ベクトルを記憶し、記憶している下位階層ベクトルを予測タップ抽出部５２に供給する。
【０１５６】
このように、動きベクトル検出部２は、精度の高い動きベクトルを検出して、検出した動きベクトルをクラスタップ抽出部３および演算部４に供給することができる。
【０１５７】
クラスタップ抽出部３は、NTSCデコーダ１から供給されたサブキャリア位相情報、および動きベクトル検出部２から供給された動きベクトルを基に、クラス分類のためのデータのセットである、画像信号のデータ、上位階層特徴量のデータ、および下位階層特徴量のデータからなるクラスタップを抽出する。
【０１５８】
図１９は、演算部４の構成を示すブロック図である。
【０１５９】
クラス分類部７１−１は、動きベクトル検出部２から供給された動きベクトル、上位階層特徴量、および下位階層特徴量、並びにクラスタップ抽出部３から供給されたクラスタップ、NTSCデコーダ１から供給されたサブキャリア位相情報を基に、クラス分類し、クラス分類の結果を示すクラスコードを予測タップ抽出部７２−１および係数メモリ７３−１に供給する。
【０１６０】
例えば、クラス分類部７１−１は、NTSCデコーダ１から供給されたサブキャリア位相情報を基に、クラス分類する。また、クラス分類部７１−１は、NTSCデコーダ１から供給されたサブキャリア位相情報により示される、注目データを基準に、動きベクトルにより指定されるデータの位相を基に、クラス分類する。従って、分類されるクラスの数は、４のべき乗となる。
【０１６１】
また、クラス分類部７１−１は、動きベクトル検出部２から供給された上位階層特徴量、および下位階層特徴量を基に、クラス分類する。すなわち、例えば、クラス分類部７１−１は、動きベクトル検出部２から供給された上位階層特徴量に、ＡＤＲＣ処理を適用し、上位階層特徴量を所定のビット数のクラスコードに量子化する。クラス分類部７１−１は、動きベクトル検出部２から供給された下位階層特徴量に、ＡＤＲＣ処理を適用し、下位階層特徴量を所定のビット数のクラスコードに量子化する。閾値判定により量子化するとき、閾値判定による量子化は、ベクトルの信頼性をクラスコード化することと等価である。
【０１６２】
図２０は、上位階層特徴量に対応するクラスタップおよび予測タップの例を説明する図である。
【０１６３】
図２０Ａに示すように、注目データが白丸で示される位相であるとき、例えば、上位階層特徴量は、図２０Ｂに示すように、白丸で示される、注目データと同相のデータのみから構成される。このとき、例えば、上位階層特徴量に対応するクラスタップは、注目データｐ５、および注目データｐ５の２つ上側のデータｐ１、注目データｐ５の１つ上側であって、２つ左側のデータｐ２、注目データｐ５の１つ上側であって、２つ右側のデータｐ３、注目データｐ５の４つ左側のデータｐ４、注目データｐ５の４つ右側のデータｐ６、注目データｐ５の１つ下側であって、２つ左側のデータｐ７、注目データｐ５の１つ下側であって、２つ右側のデータｐ８、注目データｐ５の２つ下側のデータｐ９から構成される。
【０１６４】
例えば、上位階層特徴量は、図２０Ｃに示すように、三角で示される、上下に２つ、および左右に２つの合計４つのデータの平均値から構成される。平均値から構成される上位階層特徴量の上下方向の位置は、元の画像信号のデータの位置に対して、ラインの幅の半分だけ、ずれると共に、平均値から構成される上位階層特徴量の左右方向の位置は、元の画像信号のデータの位置に対して、データの間隔の半分だけ、ずれる。
【０１６５】
注目データをｐ１１としたとき、上位階層特徴量に対応するクラスタップは、例えば、注目データｐ１１に対して、ラインが２つと半分上側であって、元のデータの間隔の半分だけ左側のデータＡ１、注目データｐ１１に対して、ラインが２つと半分上側であって、元のデータの間隔の１つと半分だけ右側のデータＡ２、ラインが１つと半分上側であって、元のデータの間隔の１つと半分だけ左側のデータＡ３、ラインが１つと半分上側であって、元のデータの間隔の半分だけ右側のデータＡ４、ラインが１つと半分上側であって、元のデータの間隔の２つと半分だけ右側のデータＡ５、ラインが半分上側であって、元のデータの間隔の半分だけ左側のデータＡ６、ラインが半分上側であって、元のデータの間隔の１つと半分だけ右側のデータＡ７、ラインが半分下側であって、元のデータの間隔の１つと半分だけ左側のデータＡ８、ラインが半分下側であって、元のデータの間隔の半分だけ右側のデータＡ９、ラインが半分下側であって、元のデータの間隔の２つと半分だけ右側のデータＡ１０、ラインが１つと半分下側であって、元のデータの間隔の半分だけ左側のデータＡ１１、ラインが１つと半分下側であって、元のデータの間隔の１つと半分だけ右側のデータＡ１２、ラインが２つと半分下側であって、元のデータの間隔の１つと半分だけ左側のデータＡ１３、ラインが２つと半分下側であって、元のデータの間隔の半分だけ右側のデータＡ１４、ラインが２つと半分下側であって、および元のデータの間隔の２つと半分だけ左側のデータＡ１５から構成される。
【０１６６】
図２１は、下位階層特徴量に対応するクラスタップおよび予測タップの例を説明する図である。
【０１６７】
現在のフィールド＃−１に対応するクラスタップは、注目データに対応する下位階層特徴量のデータを中心とする、３×３のデータ、すなわち、注目データに対応する下位階層特徴量のデータ、注目データに対応する下位階層特徴量のデータの上側のデータ、注目データに対応する下位階層特徴量のデータの上側であって、左側のデータ、注目データに対応する下位階層特徴量のデータの上側であって、右側のデータ、注目データに対応する下位階層特徴量のデータの左側のデータ、注目データに対応する下位階層特徴量のデータの右側のデータ、注目データに対応する下位階層特徴量のデータの下側であって、左側のデータ、注目データに対応する下位階層特徴量のデータの下側であって、右側のデータで構成される。
【０１６８】
対応するフィールド＃０のクラスタップは、注目データから動きベクトルで指定される下位階層特徴量のデータを中心とする、３×３のデータ、すなわち、注目データから動きベクトルで指定される下位階層特徴量のデータ、注目データから動きベクトルで指定される下位階層特徴量のデータの上側のデータ、注目データから動きベクトルで指定される下位階層特徴量のデータの上側であって、左側のデータ、注目データから動きベクトルで指定される下位階層特徴量のデータの上側であって、右側のデータ、注目データから動きベクトルで指定される下位階層特徴量のデータの左側のデータ、注目データから動きベクトルで指定される下位階層特徴量のデータの右側のデータ、注目データから動きベクトルで指定される下位階層特徴量のデータの下側であって、左側のデータ、注目データから動きベクトルで指定される下位階層特徴量のデータの下側であって、右側のデータで構成される。
【０１６９】
クラス分類部７１−１は、画像信号の注目データ、および注目データの周辺の周辺データを基に、空間クラスにクラス分類する。すなわち、例えば、クラス分類部７１−１は、画像信号の注目データ、および周辺データに、ＡＤＲＣ処理を適用し、注目データ、および周辺データを、所定のビット数のクラスコードに量子化する。
【０１７０】
クラス分類部７１−１は、注目フィールドの注目データから動きベクトルで示される、対応フィールドの対象データ、および対象データの周辺の周辺データを基に、クラス分類する。すなわち、例えば、クラス分類部７１−１は、注目フィールドの注目データから動きベクトルで示される、対応フィールドの対象データ、および対象データの周辺の周辺データに、ＡＤＲＣ処理を適用し、対象データ、および周辺データを、所定のビット数のクラスコードに量子化する。
【０１７１】
クラス分類部７１−１は、動き量を基に、クラス分類するようにしてもよい。
【０１７２】
クラス分類部７１−１は、所定のブロックに属する、画像信号の注目データおよび周辺データのデータ値の最大値と最小値の差であるダイナミックレンジを予測タップ抽出部７２−１に供給する。
【０１７３】
なお、クラス分類部７１−１は、図２１に例を示す、注目フィールドの下位階層特徴量のクラスタップを基に、ダイナミックレンジを算出するようにしてもよい。
【０１７４】
また、クラス分類部７１−１は、クラスタップに含まれる、上位階層特徴量を基に、クラス分類するようにしてもよい。
【０１７５】
予測タップ抽出部７２−１は、動きベクトル検出部２から供給された動きベクトル、および残差などの特徴量、並びにクラス分類部７１−１から供給されたクラスコードおよびダイナミックレンジを基に、画像信号のデータ、上位階層特徴量、および下位階層特徴量から所定のデータを抽出して、抽出したデータを予測タップとして予測演算部７４−１に供給する。
【０１７６】
予測タップ抽出部７２−１は、クラス分類部７１−１から供給されたダイナミックレンジが大きいとき、近傍に画像のエッジが含まれているので予測タップを近傍のデータで構成するため、空間的に広がりの小さい予測タップを抽出する。一方、予測タップ抽出部７２−１は、クラス分類部７１−１から供給されたダイナミックレンジが小さいとき、画像が平坦部なので予測タップを広範囲のデータで構成するため、空間的に広がりの大きい予測タップを抽出する。
【０１７７】
予測タップ抽出部７２−１は、動きベクトル検出部２から供給された特徴量である残差が大きいとき、動きベクトルが誤っている可能性が大きいため、空間的に広がりの大きい予測タップを抽出する。一方、予測タップ抽出部７２−１は、動きベクトル検出部２から供給された特徴量である残差が小さいとき、動きベクトルが誤っている可能性が小さいため、空間的に広がりの小さい予測タップを抽出する。
【０１７８】
なお、予測タップ抽出部７２−１は、クラスコードのみを基に、予測タップを抽出するようにしてもよい。
【０１７９】
図２０Ａに示すように、注目データが白丸で示される位相であるとき、例えば、上位階層特徴量は、図２０Ｂに示すように、白丸で示される、注目データと同相のデータのみから構成される。このとき、例えば、上位階層特徴量に対応する予測タップは、注目データｐ５、および注目データｐ５の２つ上側のデータｐ１、注目データｐ５の１つ上側であって、２つ左側のデータｐ２、注目データｐ５の１つ上側であって、２つ右側のデータｐ３、注目データｐ５の４つ左側のデータｐ４、注目データｐ５の４つ右側のデータｐ６、注目データｐ５の１つ下側であって、２つ左側のデータｐ７、注目データｐ５の１つ下側であって、２つ右側のデータｐ８、注目データｐ５の２つ下側のデータｐ９から構成される。
【０１８０】
例えば、上位階層特徴量は、図２０Ｃに示すように、三角で示される、上下に２つ、および左右に２つの合計４つのデータの平均値から構成される。平均値から構成される上位階層特徴量の上下の位置は、元の画像信号のデータの位置に対して、ラインの幅の半分だけ、ずれる。また、平均値から構成される上位階層特徴量の左右の位置は、元の画像信号のデータの位置に対して、データの間隔の半分だけ、ずれる。
【０１８１】
このとき、注目データをｐ１１とした場合、上位階層特徴量に対応する予測タップは、例えば、注目データｐ１１に対して、ラインが２つと半分上側であって、元のデータの間隔の半分だけ左側のデータＡ１、注目データｐ１１に対して、ラインが２つと半分上側であって、元のデータの間隔の１つと半分だけ右側のデータＡ２、ラインが１つと半分上側であって、元のデータの間隔の１つと半分だけ左側のデータＡ３、ラインが１つと半分上側であって、元のデータの間隔の半分だけ右側のデータＡ４、ラインが１つと半分上側であって、元のデータの間隔の２つと半分だけ右側のデータＡ５、ラインが半分上側であって、元のデータの間隔の半分だけ左側のデータＡ６、ラインが半分上側であって、元のデータの間隔の１つと半分だけ右側のデータＡ７、ラインが半分下側であって、元のデータの間隔の１つと半分だけ左側のデータＡ８、ラインが半分下側であって、元のデータの間隔の半分だけ右側のデータＡ９、ラインが半分下側であって、元のデータの間隔の２つと半分だけ右側のデータＡ１０、ラインが１つと半分下側であって、元のデータの間隔の半分だけ左側のデータＡ１１、ラインが１つと半分下側であって、元のデータの間隔の１つと半分だけ右側のデータＡ１２、ラインが２つと半分下側であって、元のデータの間隔の１つと半分だけ左側のデータＡ１３、ラインが２つと半分下側であって、元のデータの間隔の半分だけ右側のデータＡ１４、ラインが２つと半分下側であって、および元のデータの間隔の２つと半分だけ左側のデータＡ１５から構成される。
【０１８２】
図２１に示すように、下位階層特徴量における、現在のフィールド＃−１に対応する予測タップは、注目データに対応する下位階層特徴量のデータを中心とする、５×５のデータで構成される。
【０１８３】
下位階層特徴量における、対応するフィールド＃０の予測タップは、注目データから動きベクトルで指定される下位階層特徴量のデータを中心とする、５×５のデータで構成される。
【０１８４】
係数メモリ７３−１は、クラス分類部７１−１から供給されたクラスコードを基に、予め記憶している係数セットのうち、クラスコードで指定されるクラスに対応する係数セットを選択し、選択したクラスコードを予測演算部７４−１に供給する。
【０１８５】
予測演算部７４−１は、予測タップ抽出部７２−１から供給された予測タップ、および係数メモリ７３−１から供給された係数セットを基に、例えば、式（６）に対応する演算をより、Ｙ信号を予測し、予測されたＹ信号を出力する。
【０１８６】
クラス分類部７１−２は、動きベクトル検出部２から供給された動きベクトル、上位階層特徴量、および下位階層特徴量、並びにクラスタップ抽出部３から供給されたクラスタップ、NTSCデコーダ１から供給されたサブキャリア位相情報を基に、クラス分類し、クラス分類の結果を示すクラスコードを予測タップ抽出部７２−２および係数メモリ７３−２に供給する。
【０１８７】
例えば、クラス分類部７１−２は、NTSCデコーダ１から供給されたサブキャリア位相情報を基に、クラス分類する。また、クラス分類部７１−２は、NTSCデコーダ１から供給されたサブキャリア位相情報により示される、注目データを基準に、動きベクトルにより指定されるデータの位相を基に、クラス分類する。
【０１８８】
クラス分類部７１−２は、動きベクトル検出部２から供給された上位階層特徴量、および下位階層特徴量を基に、クラス分類する。すなわち、例えば、クラス分類部７１−２は、動きベクトル検出部２から供給された上位階層特徴量に、ＡＤＲＣ処理を適用し、上位階層特徴量を所定のビット数のクラスコードに量子化する。クラス分類部７１−２は、動きベクトル検出部２から供給された下位階層特徴量に、ＡＤＲＣ処理を適用し、下位階層特徴量を所定のビット数のクラスコードに量子化する。
【０１８９】
クラス分類部７１−２は、画像信号の注目データ、および注目データの周辺の周辺データを基に、空間クラスにクラス分類する。すなわち、例えば、クラス分類部７１−２は、画像信号の注目データ、および周辺データに、ＡＤＲＣ処理を適用し、注目データ、および周辺データを、所定のビット数のクラスコードに量子化する。
【０１９０】
クラス分類部７１−２は、注目フィールドの注目データから動きベクトルで示される、対応フィールドの対象データ、および対象データの周辺の周辺データを基に、クラス分類する。すなわち、例えば、クラス分類部７１−２は、注目フィールドの注目データから動きベクトルで示される、対応フィールドの対象データ、および対象データの周辺の周辺データに、ＡＤＲＣ処理を適用し、対象データ、および周辺データを、所定のビット数のクラスコードに量子化する。
【０１９１】
クラス分類部７１−２は、動き量を基に、クラス分類するようにしてもよい。
【０１９２】
クラス分類部７１−２は、所定のブロックの、画像信号の注目データおよび周辺データのデータ値の最大値と最小値の差であるダイナミックレンジを予測タップ抽出部７２−２に供給する。
【０１９３】
なお、クラス分類部７１−２は、図２１に例を示す、注目フィールド下位階層特徴量のクラスタップを基に、ダイナミックレンジを算出するようにしてもよい。
【０１９４】
また、クラス分類部７１−２は、クラスタップに含まれる、上位階層特徴量を基に、クラス分類するようにしてもよい。
【０１９５】
予測タップ抽出部７２−２は、動きベクトル検出部２から供給された動きベクトル、および残差などの特徴量、並びにクラス分類部７１−２から供給されたクラスコードおよびダイナミックレンジを基に、画像信号のデータ、上位階層特徴量、および下位階層特徴量から所定のデータを抽出して、抽出したデータを予測タップとして予測演算部７４−２に供給する。
【０１９６】
なお、予測タップ抽出部７２−２は、クラスコードのみを基に、予測タップを抽出するようにしてもよい。
【０１９７】
係数メモリ７３−２は、クラス分類部７１−２から供給されたクラスコードを基に、予め記憶している係数セットのうち、クラスコードで指定されるクラスに対応する係数セットを選択し、選択しクラスコードを予測演算部７４−２に供給する。
【０１９８】
予測演算部７４−２は、予測タップ抽出部７２−２から供給された予測タップ、および係数メモリ７３−２から供給された係数セットを基に、例えば、式（６）に対応する演算をより、Ｕ信号を予測し、予測されたＵ信号を出力する。
【０１９９】
クラス分類部７１−３は、動きベクトル検出部２から供給された動きベクトル、上位階層特徴量、および下位階層特徴量、並びにクラスタップ抽出部３から供給されたクラスタップ、NTSCデコーダ１から供給されたサブキャリア位相情報を基に、クラス分類し、クラス分類の結果を示すクラスコードを予測タップ抽出部７２−３および係数メモリ７３−３に供給する。
【０２００】
例えば、クラス分類部７１−３は、NTSCデコーダ１から供給されたサブキャリア位相情報を基に、クラス分類する。また、クラス分類部７１−３は、NTSCデコーダ１から供給されたサブキャリア位相情報により示される、注目データを基準に、動きベクトルにより指定されるデータの位相を基に、クラス分類する。
【０２０１】
クラス分類部７１−３は、動きベクトル検出部２から供給された上位階層特徴量、および下位階層特徴量を基に、クラス分類する。すなわち、例えば、クラス分類部７１−３は、動きベクトル検出部２から供給された上位階層特徴量に、ＡＤＲＣ処理を適用し、上位階層特徴量を所定のビット数のクラスコードに量子化する。クラス分類部７１−３は、動きベクトル検出部２から供給された下位階層特徴量に、ＡＤＲＣ処理を適用し、下位階層特徴量を所定のビット数のクラスコードに量子化する。
【０２０２】
クラス分類部７１−３は、画像信号の注目データ、および注目データの周辺の周辺データを基に、空間クラスにクラス分類する。すなわち、例えば、クラス分類部７１−３は、画像信号の注目データ、および周辺データに、ＡＤＲＣ処理を適用し、注目データ、および周辺データを、所定のビット数のクラスコードに量子化する。
【０２０３】
クラス分類部７１−３は、注目フィールドの注目データから動きベクトルで示される、対応フィールドの対象データ、および対象データの周辺の周辺データを基に、クラス分類する。すなわち、例えば、クラス分類部７１−３は、注目フィールドの注目データから動きベクトルで示される、対応フィールドの対象データ、および対象データの周辺の周辺データに、ＡＤＲＣ処理を適用し、対象データ、および周辺データを、所定のビット数のクラスコードに量子化する。
【０２０４】
クラス分類部７１−３は、動き量を基に、クラス分類するようにしてもよい。
【０２０５】
クラス分類部７１−３は、所定のブロックの、画像信号の注目データおよび周辺データのデータ値の最大値と最小値の差であるダイナミックレンジを予測タップ抽出部７２−３に供給する。
【０２０６】
なお、クラス分類部７１−３は、図２１に例を示す、注目フィールド下位階層特徴量のクラスタップを基に、ダイナミックレンジを算出するようにしてもよい。
【０２０７】
また、クラス分類部７１−３は、クラスタップに含まれる、上位階層特徴量を基に、クラス分類するようにしてもよい。
【０２０８】
予測タップ抽出部７２−３は、動きベクトル検出部２から供給された動きベクトル、および残差などの特徴量、並びにクラス分類部７１−３から供給されたクラスコードおよびダイナミックレンジを基に、画像信号のデータ、上位階層特徴量、および下位階層特徴量から所定のデータを抽出して、抽出したデータを予測タップとして予測演算部７４−３に供給する。
【０２０９】
なお、予測タップ抽出部７２−３は、クラスコードのみを基に、予測タップを抽出するようにしてもよい。
【０２１０】
係数メモリ７３−３は、クラス分類部７１−３から供給されたクラスコードを基に、予め記憶している係数セットのうち、クラスコードで指定されるクラスに対応する係数セットを選択し、選択しクラスコードを予測演算部７４−３に供給する。
【０２１１】
予測演算部７４−３は、予測タップ抽出部７２−３から供給された予測タップ、および係数メモリ７３−３から供給された係数セットを基に、例えば、式（６）に対応する演算をより、Ｖ信号を予測し、予測されたＶ信号を出力する。
【０２１２】
このように、演算部４は、クラス分類適応処理により、従来に比較して、より精度の高いコンポーネントビデオ信号である、Ｙ信号、Ｕ信号、およびＶ信号を予測し、予測したＹ信号、Ｕ信号、およびＶ信号を出力することができる。
【０２１３】
以下、クラス分類部７１−１乃至７１−３を個々に区別する必要がないとき、単に、クラス分類部７１と称する。
【０２１４】
以下、予測タップ抽出部７２−１乃至７２−３を個々に区別する必要がないとき、単に、予測タップ抽出部７２と称する。
【０２１５】
以下、係数メモリ７３−１乃至７３−３を個々に区別する必要がないとき、単に、係数メモリ７３と称する。
【０２１６】
以下、予測演算部７４−１乃至７４−３を個々に区別する必要がないとき、単に、予測演算部７４と称する。
【０２１７】
図２２は、演算部４の他の構成を示すブロック図である。図１９に示す場合と同様の部分には同一の番号を付してあり、その説明は省略する。
【０２１８】
セレクタ８１は、クラス分類部７１−１乃至７１−３から個々に供給された３つのクラスコードおよびダイナミックレンジの内、１つのクラスコードおよびダイナミックレンジを選択し、選択したクラスコードおよびダイナミックレンジを予測タップ抽出部８２に供給する。
【０２１９】
セレクタ８１は、係数メモリ７３−１乃至７３−３から個々に供給された３つの係数セットの内、１つの係数セットを選択し、選択した係数セットを予測演算部８３に供給する。
【０２２０】
予測タップ抽出部８２は、動きベクトル検出部２から供給された動きベクトル、上位階層特徴量、および下位階層特徴量、並びにセレクタ８１から供給されたクラスコードおよびダイナミックレンジを基に、予測タップを抽出し、抽出した予測タップを予測演算部８３に供給する。
【０２２１】
予測タップ抽出部８２は、セレクタ８１を介して、クラス分類部７１−１からに出力されたクラスコードおよびダイナミックレンジが供給されたとき、Ｙ信号に対応する予測タップを抽出し、抽出した予測タップを予測演算部８３に供給する。予測タップ抽出部８２は、セレクタ８１を介して、クラス分類部７１−２からに出力されたクラスコードおよびダイナミックレンジが供給されたとき、Ｕ信号に対応する予測タップを抽出し、抽出した予測タップを予測演算部８３に供給する。予測タップ抽出部８２は、セレクタ８１を介して、クラス分類部７１−３からに出力されたクラスコードおよびダイナミックレンジが供給されたとき、Ｖ信号に対応する予測タップを抽出し、抽出した予測タップを予測演算部８３に供給する。
【０２２２】
予測演算部８３は、予測タップ抽出部８２からＹ信号に対応する予測タップが供給されたとき、セレクタ８１を介して、係数メモリ７３−１から供給された係数セットを基に、Ｙ信号を予測して、予測したＹ信号を出力する。予測演算部８３は、予測タップ抽出部８２からＵ信号に対応する予測タップが供給されたとき、セレクタ８１を介して、係数メモリ７３−２から供給された係数セットを基に、Ｕ信号を予測して、予測したＵ信号を出力する。予測演算部８３は、予測タップ抽出部８２からＶ信号に対応する予測タップが供給されたとき、セレクタ８１を介して、係数メモリ７３−３から供給された係数セットを基に、Ｖ信号を予測して、予測したＶ信号を出力する。
【０２２３】
図２３は、演算部４の他の構成を示すブロック図である。図１９に示す場合と同様の部分には同一の番号を付してあり、その説明は省略する。
【０２２４】
クラス分類部９１は、動きベクトル検出部２から供給された動きベクトル、上位階層特徴量、および下位階層特徴量、並びにクラスタップ抽出部３から供給されたクラスタップ、NTSCデコーダ１から供給されたサブキャリア位相情報を基に、クラス分類し、クラス分類の結果を示すクラスコードを予測タップ抽出部７２−１乃至７２−３および係数メモリ７３−１乃至７３−３に供給する。
【０２２５】
予測タップ抽出部７２−１乃至７２−３は、例えば、それぞれ異なる閾値を基に、異なる予測タップを抽出する。
【０２２６】
図２４は、演算部４の他の構成を示すブロック図である。図１９に示す場合と同様の部分には同一の番号を付してあり、その説明は省略する。図２２に示す場合と同様の部分には同一の番号を付してあり、その説明は省略する。
【０２２７】
クラス分類部９１は、動きベクトル検出部２から供給された動きベクトル、上位階層特徴量、および下位階層特徴量、並びにクラスタップ抽出部３から供給されたクラスタップ、NTSCデコーダ１から供給されたサブキャリア位相情報を基に、クラス分類し、クラス分類の結果を示すクラスコードを予測タップ抽出部８２および係数メモリ７３−１乃至７３−３に供給する。
【０２２８】
セレクタ１０１は、係数メモリ７３−１乃至７３−３から個々に供給された３つの係数セットの内、１つの係数セットを選択し、選択した係数セットを予測演算部８３に供給する。
【０２２９】
図２５は、本発明に係る画像処理装置の実施の形態の構成の一例の詳細を示すブロック図である。画像処理装置に入力されたNTSCコンポジット信号は、サブキャリア位相情報検出回路１２１、フィールドメモリ１２２−１、動きベクトル検出回路１２３、および遅延回路１２４−１に供給される。
【０２３０】
サブキャリア位相情報検出回路１２１は、入力されたNTSCコンポジット信号から位相を検出し、検出された位相の情報を動きベクトル検出回路１２３、サブキャリア位相情報検出回路１２５、およびクラス分類部１２７に供給する。
【０２３１】
フィールドメモリ１２２−１は、入力されたNTSCコンポジット信号を記憶し、１つのフィールドに対応する期間遅延させ、記憶しているNTSCコンポジット信号をフィールドメモリ１２２−２、動きベクトル検出回路１２３、および遅延回路１２４−２に供給する。
【０２３２】
フィールドメモリ１２２−２は、フィールドメモリ１２２−１から供給されたNTSCコンポジット信号を記憶し、１つのフィールドに対応する期間遅延させ、記憶しているNTSCコンポジット信号を動きベクトル検出回路１２３、および遅延回路１２４−３に供給する。
【０２３３】
従って、動きベクトル検出回路１２３には、基準となるフィールドの信号、基準となるフィールドの次（未来）のフィールドの信号、および基準となるフィールドの前（過去）のフィールドの信号が同時に供給される。動きベクトル検出回路１２３は、基準となるフィールドの信号、基準となるフィールドの次のフィールドの信号、および基準となるフィールドの前のフィールドの信号を基に、基準となるフィールドと前のフィールドに対応するベクトル（ｔ−１）、および基準となるフィールドと次のフィールドに対応するベクトル（ｔ＋１）を生成する。
【０２３４】
動きベクトル検出回路１２３は、基準となるフィールドと前のフィールドに対応するベクトル（ｔ−１）、および基準となるフィールドと次のフィールドに対応するベクトル（ｔ＋１）をサブキャリア位相情報検出回路１２５に供給する。
【０２３５】
動きベクトル検出回路１２３は、基準となるフィールドと前のフィールドに対応するベクトル（ｔ−１）を領域切出回路１２６−３および領域切出回路１２８−３に供給すると共に、および基準となるフィールドと次のフィールドに対応するベクトル（ｔ＋１）を領域切出回路１２６−１および領域切出回路１２８−１に供給する。
【０２３６】
動きベクトル検出回路１２３は、ベクトル（ｔ−１）およびベクトル（ｔ＋１）の検出の処理で算出された特徴量をクラス分類部１２７に供給する。
【０２３７】
遅延回路１２４−１は、動きベクトル検出回路１２３の処理時間に対応させて、基準となるフィールドの次のフィールドの信号を遅延させ、遅延した基準となるフィールドの次のフィールドの信号を領域切出回路１２６−１に供給する。
【０２３８】
遅延回路１２４−２は、動きベクトル検出回路１２３の処理時間に対応させて、基準となるフィールドの信号を遅延させ、遅延した基準となるフィールドの信号を領域切出回路１２６−１に供給する。
【０２３９】
遅延回路１２４−３は、動きベクトル検出回路１２３の処理時間に対応させて、基準となるフィールドの前のフィールドの信号を遅延させ、遅延した基準となるフィールドの前のフィールドの信号を領域切出回路１２６−１に供給する。
【０２４０】
サブキャリア位相情報検出回路１２５は、サブキャリア位相情報検出回路１２１から供給された位相の情報、および動きベクトル検出回路１２３から供給された基準となるフィールドと前のフィールドに対応するベクトル（ｔ−１）、および基準となるフィールドと次のフィールドに対応するベクトル（ｔ＋１）を基に、前のフィールドに対応する位相情報および次のフィールドに対応する位相情報を生成する。サブキャリア位相情報検出回路１２５は、前のフィールドに対応する位相情報および次のフィールドに対応する位相情報をクラス分類部１２７に供給する。
【０２４１】
領域切出回路１２６−１は、動きベクトル検出回路１２３から供給された、基準となるフィールドと次のフィールドに対応するベクトル（ｔ＋１）を基に、遅延回路１２４−１から供給された、基準となるフィールドの次のフィールドの信号に含まれる所定のデータを切り出して、切り出したデータをクラス分類部１２７に供給する。
【０２４２】
領域切出回路１２６−２は、動きベクトル検出回路１２３から供給された、基準となるフィールドと前のフィールドに対応するベクトル（ｔ−１）を基に、遅延回路１２４−２から供給された、基準となるフィールドの信号に含まれる所定のデータを切り出して、切り出したデータをクラス分類部１２７に供給する。
【０２４３】
領域切出回路１２６−３は、遅延回路１２４−３から供給された、基準となるフィールドの前のフィールドの信号に含まれる所定のデータを切り出して、切り出したデータをクラス分類部１２７に供給する。
【０２４４】
クラス分類部１２７は、サブキャリア位相情報検出回路１２１から供給された位相の情報、動きベクトル検出回路１２３から供給された特徴量、サブキャリア位相情報検出回路１２５から供給された前のフィールドに対応する位相情報および次のフィールドに対応する位相情報、領域切出回路１２６−１から供給された、基準となるフィールドの次のフィールドの信号から切り出されたデータ、領域切出回路１２６−２から供給された、基準となるフィールドの信号から切り出されたデータ、および領域切出回路１２６−３から供給された、基準となるフィールドの前のフィールドの信号から切り出されたデータを基に、クラス分類し、クラス分類の結果を示すクラスコードを領域切出回路１２８−１、領域切出回路１２８−２、領域切出回路１２８−３、係数メモリ１２９、および予測部１３０に供給する。
【０２４５】
クラス分類部１２７は、例えば、位相情報検出回路１２５から供給された前のフィールドに対応する位相情報を基に、４クラスに分類する。クラス分類部１２７は、例えば、位相情報検出回路１２１から供給された基準となるフィールドに対応する位相情報を基に、４クラスに分類する。クラス分類部１２７は、例えば、位相情報検出回路１２５から供給された次のフィールドに対応する位相情報を基に、４クラスに分類する。
【０２４６】
クラス分類部１２７は、例えば、基準となるフィールドの次のフィールドの信号から切り出されたデータ、基準となるフィールドの信号から切り出されたデータ、および基準となるフィールドの前のフィールドの信号から切り出されたデータである５つのデータに、ＡＤＲＣ処理を適用して、３２クラスに分類する。
【０２４７】
クラス分類部１２７は、例えば、動きベクトル検出回路１２３から供給された特徴量を基に、所定の閾値と比較判定し、基準となるフィールドの次のフィールドに対して、２クラスに分類し、基準となるフィールドの前のフィールドに対して、２クラスに分類する。
【０２４８】
クラス分類部１２７は、例えば、以上のクラス分類を掛け合わせて、最終的に、８１９２クラスに分類する。
【０２４９】
クラス分類部１２７は、サブキャリア位相情報検出回路１２１から供給された位相の情報、動きベクトル検出回路１２３から供給された特徴量、サブキャリア位相情報検出回路１２５前のフィールドに対応する位相情報および次のフィールドに対応する位相情報、領域切出回路１２６−１から供給された、基準となるフィールドの次のフィールドの信号から切り出されたデータ、領域切出回路１２６−２から供給された、基準となるフィールドの信号から切り出されたデータ、および領域切出回路１２６−３から供給された、基準となるフィールドの前のフィールドの信号から切り出されたデータを基に、予測タップの構造を指定する構造可変情報を生成し、生成した構造可変情報を領域切出回路１２８−１、領域切出回路１２８−２、および領域切出回路１２８−３に供給する。
【０２５０】
領域切出回路１２８−１は、動きベクトル検出回路１２３から供給された、基準となるフィールドと次のフィールドに対応するベクトル（ｔ＋１）、並びにクラス分類部１２７から供給されたクラスコードおよび構造可変情報を基に、遅延回路１２４−１から供給された、基準となるフィールドの次のフィールドの信号に含まれる所定のデータを切り出して、切り出したデータを予測部１３０に供給する。
【０２５１】
領域切出回路１２８−２は、クラス分類部１２７から供給されたクラスコードおよび構造可変情報を基に、遅延回路１２４−２から供給された、基準となるフィールドの信号に含まれる所定のデータを切り出して、切り出したデータを予測部１３０に供給する。
【０２５２】
領域切出回路１２８−３は、動きベクトル検出回路１２３から供給された、基準となるフィールドと前のフィールドに対応するベクトル（ｔ−１）、並びにクラス分類部１２７から供給されたクラスコードおよび構造可変情報を基に、遅延回路１２４−３から供給された、基準となるフィールドの前のフィールドの信号に含まれる所定のデータを切り出して、切り出したデータを予測部１３０に供給する。
【０２５３】
係数メモリ１２９は、外部から供給された予測モード設定信号、およびクラスコードを基に、分類されたクラスに対応する所定の係数セットを予測部１３０に供給する。
【０２５４】
予測部１３０は、クラス分類部１２７から供給されたクラスコード、係数メモリ１２９から供給された係数セット、領域切出回路１２８−１から供給された、基準となるフィールドの次のフィールドの信号から切り出されたデータ、領域切出回路１２８−２から供給された、基準となるフィールドの信号から切り出されたデータ、領域切出回路１２８−３から供給された、基準となるフィールドの前のフィールドの信号から切り出されたデータを基に、例えば、Ｙ信号、Ｕ信号、およびＶ信号からなるコンポーネントビデオ信号を予測する。
【０２５５】
このように、本発明に係る画像処理装置は、入力されたコンポジット信号に対応する、より正確なコンポーネントビデオ信号を生成することができる。
【０２５６】
図２６は、本発明に係る画像処理装置による、コンポーネントビデオ信号の生成の処理を説明するフローチャートである。
【０２５７】
ステップＳ１において、動きベクトル検出部２は、NTSCデコーダ１から供給された画像信号およびサブキャリア位相情報を基に、動きベクトルを検出する。動きベクトルの検出の処理の詳細は、後述する。
【０２５８】
ステップＳ２において、クラスタップ抽出部３は、NTSCデコーダ１から供給されたサブキャリア位相情報、および動きベクトル検出部２から供給された動きベクトルを基に、画像信号のデータ、上位階層特徴量のデータ、および下位階層特徴量のデータからなるクラスタップを抽出する。
【０２５９】
ステップＳ３において、演算部４は、NTSCデコーダ１から供給された画像信号およびサブキャリア位相情報、動きベクトル検出部２から供給された動きベクトル、上位階層特徴量のデータ、および下位階層特徴量のデータ、並びにクラスタップ抽出部３から供給されたクラスタップを基に、クラス分類する。例えば、図１９に示す、演算部４のクラス分類部７１−１乃至７１−３は、NTSCデコーダ１から供給された画像信号およびサブキャリア位相情報、動きベクトル検出部２から供給された動きベクトル、上位階層特徴量のデータ、および下位階層特徴量のデータ、並びにクラスタップ抽出部３から供給されたクラスタップを基に、クラス分類する。
【０２６０】
ステップＳ４において、演算部４は、分類されたクラス、および動きベクトル検出部２から供給された動きベクトルを基に、画像信号のデータ、上位階層特徴量のデータ、および下位階層特徴量のデータからなる予測タップを抽出する。例えば、図１９に示す予測タップ抽出部７２−１乃至７２−３は、分類されたクラスを示すクラスコード、ダイナミックレンジ、および動きベクトル検出部２から供給された動きベクトルを基に、画像信号のデータ、上位階層特徴量のデータ、および下位階層特徴量のデータからなる予測タップを抽出する。
【０２６１】
ステップＳ５において、演算部４は、抽出された予測タップ、および予め記憶されている係数セットを基に、コンポーネントビデオ信号を予測し、処理は終了する。例えば、図１９に示す演算部４の予測演算部７４−１乃至７４−３は、予測タップおよび係数メモリ７３−１乃至７３−３から供給された係数セットを基に、それぞれ、Ｙ信号、Ｕ信号、およびＶ信号を予測する。
【０２６２】
図２７は、ステップＳ１に対応する、動きベクトル検出部２による、動きベクトルの検出の処理を説明するフローチャートである。ステップＳ１１において、動きベクトル検出部２は、NTSCデコーダ１を介して、NTSCデコーダ１から供給されたサブキャリア位相情報を基に、注目データのサブキャリアの位相を取得する。
【０２６３】
ステップＳ１２において、特徴量変換部２３は、画像信号および取得したサブキャリアの位相を基に、注目フィールドおよび対応するフィールドについて、上位階層の特徴量を算出し、算出された特徴量を上位階層特徴量として出力する。
【０２６４】
ステップＳ１３において、上位階層動きベクトル検出部２５は、特徴量変換部２３から供給された上位階層特徴量を基に、動きベクトルに近似する近似ベクトルである、上位階層ベクトルを検出する。上位階層ベクトルの検出の処理の詳細は、後述する。
【０２６５】
ステップＳ１４において、特徴量変換部２４は、画像信号および取得したサブキャリアの位相を基に、注目フィールドおよび対応するフィールドについて、下位階層の特徴量を算出し、算出された特徴量を下位階層特徴量として出力する。
【０２６６】
ステップＳ１５において、下位階層動きベクトル検出部２６は、特徴量変換部２４から供給された下位階層特徴量、および上位階層動きベクトル検出部２５から供給された上位階層ベクトルを基に、下位階層ベクトルを検出する。下位階層ベクトルの検出の処理の詳細は、後述する。
【０２６７】
ステップＳ１６において、下位階層動きベクトル検出部２６は、動きベクトルを出力して、処理は終了する。すなわち、下位階層動きベクトル検出部２６は、検出した下位階層動きベクトルを動きベクトルとして出力する。
【０２６８】
図２８は、ステップＳ１３に対応する、上位階層動きベクトル検出部２５による、上位階層ベクトルの検出の処理の詳細を説明するフローチャートである。ステップＳ２１において、上位階層動きベクトル検出部２５のベクトル生成部３１は、パラメータＡを基に、所定のベクトルを生成し、相関値算出部３２および判定部３３に供給する。ステップＳ２２において、相関値算出部３２は、注目フィールドの注目データに対応する上位階層特徴量を中心とし、所定の数の上位階層特徴量からなるブロックと、対応するフィールドの、ベクトル生成部３１から供給されたベクトルで指定される位置を中心とし、所定の数の上位階層特徴量からなるブロックとの相関値を算出する。相関値算出部３２は、例えば、相関値として、注目フィールドのブロックに属する上位階層特徴量と、対応するフィールドのブロックに属する上位階層特徴量との差分の絶対値の和を算出する。
【０２６９】
ステップＳ２３において、判定部３３は、ベクトル生成部３１から供給されたベクトルに対応させて、相関値算出部３２から供給された相関値を記憶する。
【０２７０】
ステップＳ２４において、上位階層動きベクトル検出部２５は、パラメータＡにより指定された、所定の範囲の相関値を算出したか否かを判定し、所定の範囲の相関値を算出していないと判定された場合、ステップＳ２１に戻り、次のベクトルに対応して、相関値の演算の処理を繰り返す。
【０２７１】
ステップＳ２４において、パラメータＡに含まれるデータで指定される、所定の範囲の相関値を算出したと判定された場合、ステップＳ２５に進み、判定部３３は、記憶している相関値のうち、最も強い相関に対応するベクトルを選択する。例えば、相関値算出部３２が、相関値として、注目フィールドのブロックに属する上位階層特徴量と、対応するフィールドのベクトルに指定される位置のブロックに属する上位階層特徴量との差分の絶対値の和を算出するとき、判定部３３は、最小の相関値を選択する。
【０２７２】
ステップＳ２６において、判定部３３は、上位階層ベクトルとして、ステップＳ２５の処理で選択したベクトルを出力して、処理は終了する。
【０２７３】
図２９は、ステップＳ１５に対応する下位階層ベクトルの検出の処理の詳細を説明するフローチャートである。ステップＳ３１において、下位階層動きベクトル検出部２６のベクトル生成部４１は、上位階層ベクトルおよびパラメータＢを基に、所定のベクトルを生成し、相関値算出部４２および判定部４３に供給する。ステップＳ３２において、相関値算出部４２は、注目フィールドの注目データに対応する下位階層特徴量を中心とし、所定の数の下位階層特徴量からなるブロックと、対応するフィールドの、上位階層ベクトル、およびベクトル生成部４１から供給されたベクトルで指定される位置を中心とし、所定の数の下位階層特徴量からなるブロックとの相関値を算出する。相関値算出部４２は、例えば、相関値として、注目フィールドのブロックに属する下位階層特徴量と、対応するフィールドのベクトルで指定される位置のブロックに属する下位階層特徴量との相互相関を算出する。
【０２７４】
ステップＳ３３において、判定部４３は、ベクトル生成部４１から供給されたベクトルに対応させて、相関値算出部４２から供給された相関値を記憶する。
【０２７５】
ステップＳ３４において、下位階層動きベクトル検出部２６は、パラメータＢに含まれるデータにより指定された、所定の範囲の相関値を算出したか否かを判定し、所定の範囲の相関値を算出していないと判定された場合、ステップＳ３１に戻り、相関値の演算の処理を繰り返す。
【０２７６】
ステップＳ３４において、所定の範囲の相関値を算出したと判定された場合、ステップＳ３５に進み、判定部４３は、記憶している相関値のうち、最も強い相関に対応するベクトルを選択する。例えば、相関値算出部４２が、相関値として、注目フィールドのブロックに属する下位階層特徴量と、対応するフィールドのベクトルに指定される位置のブロックに属する下位階層特徴量との相互相関を算出するとき、判定部３３は、最大の相関値を選択する。
【０２７７】
ステップＳ３６において、判定部４３は、下位階層ベクトルとして、ステップＳ３５の処理で選択したベクトルを出力して、処理は終了する。
【０２７８】
このように、動きベクトル検出部２は、より精度の高い動きベクトルを検出することができる。
【０２７９】
動きベクトル検出部２は、２つの階層において動きベクトルを検出する処理を実行するようにしたので、回路規模を大きくすることなく、また、複雑な演算を必要とすることなく、より精度の高い動きベクトルを検出できる。
【０２８０】
図３０は、ステップＳ１５の処理に対応する、図１８に構成を示す下位階層動きベクトル検出部２６による、下位階層ベクトルの検出の他の処理の詳細を説明するフローチャートである。
【０２８１】
ステップＳ４１において、クラス分類部５１は、サブキャリア位相情報および下位階層特徴量を基に、注目データに対応してクラス分類する。
【０２８２】
ステップＳ４２において、予測タップ抽出部５２は、分類されたクラス、メモリ５５から供給された、前のフィールドに対応する下位階層ベクトル、および上位階層ベクトルに対応する、所定の数の、所定の下位階層特徴量である予測タップを抽出する。
【０２８３】
ステップＳ４３において、演算部５３は、予測係数メモリ５４に記憶されている係数セット、および予測タップを基に、下位階層ベクトルを演算する。
【０２８４】
ステップＳ４４において、演算部５３は、下位階層ベクトルをメモリ５５に記憶させると共に、下位階層ベクトルを出力して、処理は終了する。
【０２８５】
このように、図１８に構成を示す下位階層動きベクトル検出部２６は、下位階層特徴量にクラス分類適応処理を適用して、より精度の高い動きベクトルを生成することができる。
【０２８６】
以上のように、本発明に係る画像処理装置は、コンポジットビデオ信号から、動きベクトルを考慮して、従来に比較して、より精度の良い、コンポーネントビデオ信号を生成することができる。
【０２８７】
図３１は、コンポーネントビデオ信号の予測の処理に使用される予測係数を生成する、本発明に係る画像処理装置の実施の形態の構成を示すブロック図である。
【０２８８】
図１に示す画像処理装置と同様の部分には、同一の番号を付してあり、その説明は省略する。
【０２８９】
画像処理装置に入力されたコンポーネントビデオ信号は、NTSCエンコーダ２０１および係数算出部２０２に供給される。
【０２９０】
NTSCエンコーダ２０１は、入力されたコンポーネントビデオ信号を基に、NTSCコンポジット信号を生成し、生成したNTSCコンポジット信号をNTSCデコーダ１に供給する。
【０２９１】
係数算出部２０２は、NTSCデコーダ１から供給されたサブキャリア位相情報、動きベクトル検出部２から供給された動きベクトル、およびクラスタップ抽出部３から供給されたクラスタップを基に、入力されたコンポーネントビデオ信号、NTSCデコーダ１から供給された画像信号、並びに動きベクトル検出部２から供給された上位階層特徴量および下位階層特徴量に、式（１４）に例を示す演算を適用して、コンポーネントビデオ信号を予測するための予測係数を生成する。演算部４は、生成した予測係数を係数メモリ２０３に供給する。
【０２９２】
係数メモリ２０３は、係数算出部２０２から供給された予測係数を記憶する。
【０２９３】
図３２は、係数算出部２０２の構成を示すブロック図である。図１９に示す場合と同様の部分には同一の番号を付してあり、その説明は省略する。
【０２９４】
予測係数算出部２２１−１は、予測タップ抽出部７２−１から供給された予測タップ、および入力されたコンポーネントビデオ信号に含まれるＹ信号に、例えば、式（１４）に示す演算を適用して、Ｙ信号を予測するための予測係数を算出し、算出した予測係数を出力する。
【０２９５】
予測係数算出部２２１−２は、予測タップ抽出部７２−２から供給された予測タップ、および入力されたコンポーネントビデオ信号に含まれるＵ信号に、例えば、式（１４）に示す演算を適用して、Ｕ信号を予測するための予測係数を算出し、算出した予測係数を出力する。
【０２９６】
予測係数算出部２２１−３は、予測タップ抽出部７２−３から供給された予測タップ、および入力されたコンポーネントビデオ信号に含まれるＶ信号に、例えば、式（１４）に示す演算を適用して、Ｖ信号を予測するための予測係数を算出し、算出した予測係数を出力する。
【０２９７】
図３３は、係数算出部２０２の他の構成を示すブロック図である。図２２に示す場合と同様の部分には同一の番号を付してあり、その説明は省略する。
【０２９８】
セレクタ２３１は、クラス分類部７１−１乃至７１−３から個々に供給された３つのクラスコードおよびダイナミックレンジの内、１つのクラスコードおよびダイナミックレンジを選択し、選択したクラスコードおよびダイナミックレンジを予測タップ抽出部８２に供給する。
【０２９９】
予測係数算出部２３２は、予測タップ抽出部８２からＹ信号に対応する予測タップが供給されたとき、予測タップ抽出部８２から供給された予測タップ、およびコンポーネントビデオ信号に含まれるＹ信号を基に、Ｙ信号を予測するための予測係数を算出して、算出した予測係数を出力する。予測係数算出部２３２は、予測タップ抽出部８２からＵ信号に対応する予測タップが供給されたとき、予測タップ抽出部８２から供給された予測タップ、およびコンポーネントビデオ信号に含まれるＵ信号を基に、Ｕ信号を予測するための予測係数を算出して、算出した予測係数を出力する。予測係数算出部２３２は、予測タップ抽出部８２からＶ信号に対応する予測タップが供給されたとき、予測タップ抽出部８２から供給された予測タップ、およびコンポーネントビデオ信号に含まれるＶ信号を基に、Ｖ信号を予測するための予測係数を算出して、算出した予測係数を出力する。
【０３００】
図３４は、係数算出部２０２のさらに他の構成を示すブロック図である。図２３に示す場合と同様の部分には同一の番号を付してあり、その説明は省略する。
【０３０１】
予測係数算出部２２１−１は、予測タップ抽出部７２−１から供給された予測タップ、および入力されたコンポーネントビデオ信号に含まれるＹ信号に、例えば、式（１４）に示す演算を適用して、Ｙ信号を予測するための予測係数を算出し、算出した予測係数を出力する。
【０３０２】
予測係数算出部２２１−２は、予測タップ抽出部７２−２から供給された予測タップ、および入力されたコンポーネントビデオ信号に含まれるＵ信号に、例えば、式（１４）に示す演算を適用して、Ｕ信号を予測するための予測係数を算出し、算出した予測係数を出力する。
【０３０３】
予測係数算出部２２１−３は、予測タップ抽出部７２−３から供給された予測タップ、および入力されたコンポーネントビデオ信号に含まれるＶ信号に、例えば、式（１４）に示す演算を適用して、Ｖ信号を予測するための予測係数を算出し、算出した予測係数を出力する。
【０３０４】
図３５は、係数算出部２０２のさらに他の構成を示すブロック図である。図２４に示す場合と同様の部分には同一の番号を付してあり、その説明は省略する。
【０３０５】
予測係数算出部２３２は、予測タップ抽出部８２からＹ信号に対応する予測タップが供給されたとき、予測タップ抽出部８２から供給された予測タップ、およびコンポーネントビデオ信号に含まれるＹ信号を基に、Ｙ信号を予測するための予測係数を算出して、算出した予測係数を出力する。予測係数算出部２３２は、予測タップ抽出部８２からＵ信号に対応する予測タップが供給されたとき、予測タップ抽出部８２から供給された予測タップ、およびコンポーネントビデオ信号に含まれるＵ信号を基に、Ｕ信号を予測するための予測係数を算出して、算出した予測係数を出力する。予測係数算出部２３２は、予測タップ抽出部８２からＶ信号に対応する予測タップが供給されたとき、予測タップ抽出部８２から供給された予測タップ、およびコンポーネントビデオ信号に含まれるＶ信号を基に、Ｖ信号を予測するための予測係数を算出して、算出した予測係数を出力する。
【０３０６】
図３６は、予測係数を算出する画像処理装置の実施の形態の構成の一例の詳細を示すブロック図である。図２５に示す場合と同様の部分には同一の番号を付してあり、その説明は省略する。
【０３０７】
入力信号である、HD(High-Definition)またはプログレッシブのコンポーネントビデオ信号は、間引きフィルタ２６１および選択回路２６３に供給される。
【０３０８】
間引きフィルタ２６１は、入力信号である、HDまたはプログレッシブのコンポーネントビデオ信号から、SD(Standard-Definition)インタレースのコンポーネントビデオ信号を生成する。間引きフィルタ２６１は、生成したコンポーネントビデオ信号をNTSCエンコーダ２６２および選択回路２６３に供給する。
【０３０９】
NTSCエンコーダ２６２は、入力されたコンポーネントビデオ信号を基に、生徒画像としての、NTSCコンポジットビデオ信号を生成する。NTSCエンコーダ２６２は、生成したNTSCコンポジットビデオ信号を、サブキャリア位相情報検出回路１２１、フィールドメモリ１２２−１、動きベクトル検出回路１２３、および遅延回路１２４−１に供給する。
【０３１０】
選択回路２６３は、外部から供給された学習モード設定信号を基に、画像処理装置に入力された、HDまたはプログレッシブのコンポーネントビデオ信号、および間引きフィルタ２６１から供給されたSDインタレースのコンポーネントビデオ信号のいずれか一方を選択し、選択した信号を教師画像として、正規方程式演算回路２６４に供給する。
【０３１１】
正規方程式演算回路２６４は、クラス分類部１２７から供給されたクラスコード、領域切出回路１２８−１から供給された、基準となるフィールドの次のフィールドの信号から切り出されたデータ、領域切出回路１２８−２から供給された、基準となるフィールドの信号から切り出されたデータ、領域切出回路１２８−３から供給された、基準となるフィールドの前のフィールドの信号から切り出されたデータ、および選択回路２６３から供給された教師画像を基に、例えば、Ｙ信号、Ｕ信号、およびＶ信号を予測するための予測係数を演算する。正規方程式演算回路２６４は、演算した予測係数を係数メモリ２６５に供給する。
【０３１２】
係数メモリ２６５は、正規方程式演算回路２６４から供給された予測係数を記憶する。
【０３１３】
このように、本発明に係る画像処理装置は、コンポジットビデオ信号を基に、コンポーネントビデオ信号を予測するための予測係数を算出することができる。
【０３１４】
図３７は、本発明に係る画像処理装置による、コンポジットビデオ信号を基に、コンポーネントビデオ信号を予測するための予測係数を算出する学習の処理を説明するフローチャートである。
【０３１５】
ステップＳ２０１において、NTSCエンコーダ２０１は、入力されたコンポーネントビデオ信号を基に、コンポジットビデオ信号を生成する。
【０３１６】
ステップＳ２０２において、動きベクトル検出部２は、コンポジットビデオ信号である画像信号およびサブキャリア位相情報を基に、動きベクトルを検出する。ステップＳ２０２の処理の詳細は、ステップＳ１の処理と同様なので、その説明は省略する。
【０３１７】
ステップＳ２０３において、クラスタップ抽出部３は、NTSCデコーダ１から供給されたサブキャリア位相情報、および動きベクトル検出部２から供給された動きベクトルを基に、画像信号のデータ、上位階層特徴量のデータ、および下位階層特徴量のデータからなるクラスタップを抽出する。
【０３１８】
ステップＳ２０４において、係数算出部２０２は、NTSCデコーダ１から供給された画像信号およびサブキャリア位相情報、動きベクトル検出部２から供給された動きベクトル、上位階層特徴量のデータ、および下位階層特徴量のデータ、並びにクラスタップ抽出部３から供給されたクラスタップを基に、クラス分類する。例えば、図３２に示す、係数算出部２０２のクラス分類部７１−１乃至７１−３は、NTSCデコーダ１から供給された画像信号およびサブキャリア位相情報、動きベクトル検出部２から供給された動きベクトル、上位階層特徴量のデータ、および下位階層特徴量のデータ、並びにクラスタップ抽出部３から供給されたクラスタップを基に、クラス分類する。
【０３１９】
ステップＳ２０５において、係数算出部２０２は、分類されたクラス、および動きベクトル検出部２から供給された動きベクトルを基に、画像信号のデータ、上位階層特徴量のデータ、および下位階層特徴量のデータからなる予測タップを抽出する。例えば、図３２に示す、係数算出部２０２の予測タップ抽出部７２−１乃至７２−３は、分類されたクラスを示すクラスコード、ダイナミックレンジ、および動きベクトル検出部２から供給された動きベクトルを基に、画像信号のデータ、上位階層特徴量のデータ、および下位階層特徴量のデータからなる予測タップを抽出する。
【０３２０】
ステップＳ２０６において、係数算出部２０２は、抽出された予測タップ、および入力されたコンポーネントビデオ信号を基に、コンポーネントビデオ信号を予測するための予測係数を算出し、処理は終了する。例えば、図３２に示す係数算出部２０２の予測係数演算部部２２１−１乃至２２１−３は、予測タップおよびコンポーネントビデオ信号を基に、それぞれ、Ｙ信号、Ｕ信号、またはＶ信号を予測するための予測係数を算出する。
【０３２１】
このように、本発明に係る画像処理装置は、コンポジットビデオ信号を基に、コンポーネントビデオ信号を予測するための予測係数を算出することができる。
【０３２２】
図３８は、図１８に構成を示す下位階層動きベクトル検出部２６において、下位階層ベクトルの演算の処理に使用される予測係数を生成する画像処理装置の構成を示すブロック図である。
【０３２３】
NTSCデコーダ３０１は、入力された、NTSC方式のコンポジットビデオ信号である画像信号を基に、画像信号に対応する、デジタルデータである画像信号を生成すると共に、デジタルデータである画像信号に対応するサブキャリア位相情報を生成する。NTSCデコーダ３０１は、生成したデジタルデータである画像信号およびサブキャリア位相情報を画像メモリ３０２に供給する。
【０３２４】
画像メモリ３０２は、NTSCデコーダ３０１から供給された画像信号およびサブキャリア位相情報を記憶し、記憶している画像信号およびサブキャリア位相情報を係数生成部３０３に供給する。
【０３２５】
係数生成部３０３は、画像処理装置の外部から供給されるパラメータＡ、および画像メモリ３０２から供給されたサブキャリア位相情報を基に、画像メモリ３０２から供給された画像信号と動きベクトルに対応する係数セットを生成し、生成した係数セットを出力する。係数セットは、各クラスに対応する、所定の数の予測係数からなる。
【０３２６】
パラメータＡは、係数生成部３０３における処理の内容を指定するパラメータであり、例えば、相関の算出方式、マッチングをとるブロックの大きさ、または探索領域の大きさなどを指定する。
【０３２７】
図３９は、係数生成部３０３の構成を示すブロック図である。図３に示す動きベクトル検出部２と同様の部分には同一の番号を付してあり、その説明は省略する。
【０３２８】
学習部３２１は、動きベクトル、特徴量変換部２４から供給された下位階層特徴量およびサブキャリア位相情報、並びに上位階層動きベクトル検出部２５から供給された上位階層ベクトルを基に、動きベクトルを算出するための係数セットを算出し、算出した係数セットを出力する。
【０３２９】
図４０は、学習部３２１の構成を示すブロック図である。
【０３３０】
クラス分類部３５１は、サブキャリア位相情報、および下位階層特徴量を基に、クラス分類し、分類されたクラスを示すクラスコードを予測タップ抽出部３５２に供給する。
【０３３１】
予測タップ抽出部３５２は、クラスコードで示されるクラス、メモリ３５４から供給される、前のフィールドに対応する予測動きベクトル、および上位階層ベクトルを基に、所定の下位階層特徴量を抽出して、抽出した下位階層特徴量を予測タップとして係数算出部３５３に供給する。
【０３３２】
係数算出部３５３は、予測タップ抽出部３５２から供給された、下位階層特徴量の所定のデータからなる予測タップを基に、予測係数および予測動きベクトルを算出する。係数算出部３５３は、算出した係数を予測係数メモリ３５５に供給すると共に、算出した予測動きベクトルをメモリ３５４に供給する。
【０３３３】
メモリ３５４は、係数算出部３５３から供給された予測動きベクトルを記憶し、記憶している予測動きベクトルを予測タップ抽出部３５２に供給する。すなわち、予測タップ抽出部３５２には、１つ前のフィールドに対応する予測動きベクトルが供給される。
【０３３４】
予測係数メモリ３５５は、係数算出部３５３から供給された予測係数を記憶し、記憶している予測係数を係数セットとして出力する。
【０３３５】
次に、図４１のフローチャートを参照して、下位階層ベクトルの演算の処理に使用される予測係数を生成する、画像処理装置の学習の処理を説明する。
【０３３６】
ステップＳ３０１において、係数生成部３０３は、NTSCデコーダ１を介して、NTSCデコーダ１から供給されたサブキャリア位相情報を基に、注目データのサブキャリアの位相を取得する。
【０３３７】
ステップＳ３０２において、係数生成部３０３の特徴量変換部２３は、上位階層の特徴量を算出し、算出された特徴量を上位階層特徴量として出力する。
【０３３８】
ステップＳ３０３において、係数生成部３０３の上位階層動きベクトル検出部２５は、特徴量変換部２３から供給された上位階層特徴量を基に、上位階層ベクトルを検出する。ステップＳ３０３の処理の詳細は、図２８のフローチャートを参照して説明した処理と同様なのでその説明は省略する。
【０３３９】
ステップＳ３０４において、係数生成部３０３の特徴量変換部２４は、下位階層の特徴量を算出し、算出された特徴量を下位階層特徴量として出力する。
【０３４０】
ステップＳ３０５において、学習部３２１は、動きベクトル、特徴量変換部２４から供給された下位階層特徴量、および上位階層動きベクトル検出部２５から供給された上位階層ベクトルを基に、係数セットを算出して、処理は終了する。
【０３４１】
図４２は、ステップＳ３０５の処理に対応する、学習部３２１による、係数セットの算出の処理の詳細を説明するフローチャートである。
【０３４２】
ステップＳ３２１において、クラス分類部３５１は、サブキャリア位相情報および下位階層特徴量を基に、注目データに対応し、クラス分類する。
【０３４３】
ステップＳ３２２において、予測タップ抽出部３５２は、分類されたクラス、メモリ３５４から供給された、前のフィールドに対応する予測動きベクトル、および上位階層ベクトルに対応する、所定の数の、所定の下位階層特徴量である予測タップを抽出する。
【０３４４】
ステップＳ３２３において、係数算出部３５３は、予測タップ抽出部３５２から供給された予測タップ、および外部から供給された動きベクトルを基に、係数を算出する。係数算出部３５３は、算出した係数を予測係数メモリ３５５に供給する。
【０３４５】
ステップＳ３２４において、予測係数メモリ３５５は、係数を記憶して、処理は終了する。
【０３４６】
このように、図３８に構成を示す画像処理装置は、下位階層ベクトルの演算の処理に使用される予測係数を生成することができる。
【０３４７】
なお、以上の説明においてデジタルデータである画像信号は、Ｙ信号、Ｉ信号、またはＱ信号に対応する信号であると説明したが、Ｙ信号、Ｉ信号、またはＱ信号からなる信号に限定されず、Ｙ信号、Ｕ信号、およびＶ信号、Ｙ信号、Ｐｂ信号、およびＰｒ信号、またはＹ信号、Ｃｂ信号、およびＣｒ信号などの他の方式の画像信号であってもよい。
【０３４８】
また、生成されるコンポーネントビデオ信号は、Ｙ信号、Ｕ信号、およびＶ信号であると説明したが、Ｙ信号、Ｕ信号、およびＶ信号に限らず、Ｙ信号、Ｐｂ信号、およびＰｒ信号、またはＹ信号、Ｃｂ信号、およびＣｒ信号などの他の方式の画像信号であってもよい。
【０３４９】
なお、本発明に係る画像処理装置は、NTSC方式のコンポジットビデオ信号である画像信号を基に、コンポーネントビデオ信号を生成すると説明したが、NTSC方式に限らず、PAL(Phase Alternation by Line)方式などの他の方式のコンポジットビデオ信号を基に、コンポーネントビデオ信号を生成するようにしてもよい。
【０３５０】
また、本発明に係る画像処理装置は、注目している注目フィールドと、注目フィールドの次のフィールドとの動きベクトルを検出すると説明したが、注目している注目フィールドと、注目フィールドの前のフィールドとの動きベクトルを検出するようにしてもよい。
【０３５１】
なお、本発明に係る画像処理装置は、フィールドを単位とし動きベクトルを検出すると説明したが、フレームを単位として動きベクトルを検出するようにしてもよい。
【０３５２】
上述した一連の処理は、ハードウェアにより実行させることもできるが、ソフトウェアにより実行させることもできる。一連の処理をソフトウェアにより実行させる場合には、そのソフトウェアを構成するプログラムが、専用のハードウェアに組み込まれているコンピュータ、または、各種のプログラムをインストールすることで、各種の機能を実行することが可能な、例えば汎用のパーソナルコンピュータなどに、記録媒体からインストールされる。
【０３５３】
図４３は、記録媒体およびコンピュータの例を説明する図である。CPU（Central Processing Unit）５０１は、各種アプリケーションプログラムや、OS（Operating System)を実際に実行する。ROM（Read-only Memory）５０２は、一般的には、CPU５０１が使用するプログラムや演算用のパラメータのうちの基本的に固定のデータを格納する。RAM（Random-Access Memory）５０３は、CPU５０１の実行において使用するプログラムや、その実行において適宜変化するパラメータを格納する。これらはCPUバスなどから構成されるホストバス５０４により相互に接続されている。
【０３５４】
ホストバス５０４は、ブリッジ５０５を介して、PCI(Peripheral Component Interconnect/Interface)バスなどの外部バス５０６に接続されている。
【０３５５】
キーボード５０８は、CPU５０１に各種の指令を入力するとき、使用者により操作される。マウス５０９は、ディスプレイ５１０の画面上のポイントの指示や選択を行うとき、使用者により操作される。ディスプレイ５１０は、液晶表示装置またはCRT（Cathode Ray Tube）などから成り、各種情報をテキストやイメージで表示する。HDD（Hard Disk Drive）５１１は、ハードディスクを駆動し、それらにCPU５０１によって実行するプログラムや情報を記録または再生させる。
【０３５６】
ドライブ５１２は、装着されている磁気ディスク５５１、光ディスク５５２、光磁気ディスク５５３、または半導体メモリ５５４に記録されているデータまたはプログラムを読み出して、そのデータまたはプログラムを、インターフェース５０７、外部バス５０６、ブリッジ５０５、およびホストバス５０４を介して接続されているRAM５０３に供給する。
【０３５７】
これらのキーボード５０８乃至ドライブ５１２は、インターフェース５０７に接続されており、インターフェース５０７は、外部バス５０６、ブリッジ５０５、およびホストバス５０４を介してCPU５０１に接続されている。
【０３５８】
ビデオインターフェース５１３は、供給された画像信号を取得し、外部バス５０６、ブリッジ５０５、およびホストバス５０４を介して、取得した画像信号をRAM５０３またはCPU５０１に供給する。ビデオインターフェース５１３は、予測したコンポーネントビデオ信号を出力する。
【０３５９】
記録媒体は、図４３に示すように、コンピュータとは別に、ユーザにブロック図に対応する処理を実行するプログラムを提供するために配布される、プログラムが記録されている磁気ディスク５５１（フロッピディスク（登録商標）を含む）、光ディスク５５２（CD-ROM(Compact Disc-Read Only Memory)、ＤＶＤ(Digital Versatile Disc)を含む）、光磁気ディスク５５３（ＭＤ(Mini-Disc)（商標）を含む）、若しくは半導体メモリ５５４などよりなるリムーバブルメディアにより構成されるだけでなく、コンピュータに予め組み込まれた状態でユーザに提供される、プログラムが記録されているROM５０２や、HDD５１１などで構成される。
【０３６０】
なお、ユーザにブロック図に対応する処理を実行するプログラムは、有線または無線の通信媒体を介して、コンピュータに供給されるようにしてもよい。
【０３６１】
また、本明細書において、記録媒体に格納されるプログラムを記述するステップは、記載された順序に沿って時系列的に行われる処理はもちろん、必ずしも時系列的に処理されなくとも、並列的あるいは個別に実行される処理をも含むものである。
【０３６２】
【発明の効果】
本発明の第１の画像処理装置および方法、記録媒体、並びにプログラムによれば、コンポジットビデオ信号の注目している点である注目点の存在する注目しているフィールドである注目フィールドと他のフィールドとについて、コンポジットビデオ信号の輝度信号と色信号とを加算または減算して得られた画像信号から、注目点における色信号の位相と同じ位相の点の値であるか、それぞれのフィールドの１つのライン上の連続する４つの点であって、それぞれ異なる位相の４つの点の平均値であるか、またはフィールドの２つのライン上の上下および左右に相互に隣接する４つの点であって、それぞれ異なる位相の４つの点の平均値である第１の特徴量を生成し、注目点の周辺の注目フィールドにおける点の第１の特徴量および注目点の第１の特徴量に、配置および値の最も近い第１の特徴量であって、他のフィールドにおける第１の特徴量を検出することにより、動きベクトルに近似する第１のベクトルを検出し、注目フィールドと他のフィールドとについて、動きベクトルの検出の演算の対象となる対象点の値に第１の重みを乗じて得た結果と、対象点の存在するフィールドにおける対象点の周辺の点であって、対象点に隣接するか、対象点から１つまたは２つの点だけ離れた点の値に第２の重みを乗じて得た結果とを加算した値である、対象点に対応する第２の特徴量を生成し、第１のベクトルで指定される位置を中心とした、他のフィールドのサーチする領域から、注目フィールドにおける注目点の周辺の点の第２の特徴量および注目点の第２の特徴量に、配置および値の最も近い第２の特徴量であって、他のフィールドにおける第２の特徴量の位置を示す、第２のベクトルとして動きベクトルが検出され、第１の特徴量または第２の特徴量および動きベクトルによって、予め定めた複数のクラスのうちの１つのクラスに注目点が分類され、分類されたクラス毎に予め定められている予測タップであって、第１の特徴量または第２の特徴量を含む予測タップが抽出され、予め求めておいた予測係数と抽出された予測タップとに積和演算を適用することにより、注目点のコンポジットビデオ信号に対応するコンポーネントビデオ信号が生成されるようにしたので、動きベクトルを考慮して、コンポジットビデオ信号から、より精度良く、コンポーネントビデオ信号を生成できるようになる。
【０３６３】
本発明の第２の画像処理装置および方法、記録媒体、並びにプログラムによれば、コンポジットビデオ信号の注目している点である注目点の存在する注目しているフィールドである注目フィールドと他のフィールドとについて、コンポジットビデオ信号の輝度信号と色信号とを加算または減算して得られた画像信号から、注目点における色信号の位相と同じ位相の点の値であるか、それぞれのフィールドの１つのライン上の連続する４つの点であって、それぞれ異なる位相の４つの点の平均値であるか、またはフィールドの２つのライン上の上下および左右に相互に隣接する４つの点であって、それぞれ異なる位相の４つの点の平均値である第１の特徴量を生成し、注目点の周辺の注目フィールドにおける点の第１の特徴量および注目点の第１の特徴量に、配置および値の最も近い第１の特徴量であって、他のフィールドにおける第１の特徴量を検出することにより、動きベクトルに近似する第１のベクトルを検出し、注目フィールドと他のフィールドとについて、動きベクトルの検出の演算の対象となる対象点の値に第１の重みを乗じて得た結果と、対象点の存在するフィールドにおける対象点の周辺の点であって、対象点に隣接するか、対象点から１つまたは２つの点だけ離れた点の値に第２の重みを乗じて得た結果とを加算した値である、対象点に対応する第２の特徴量を生成し、第１のベクトルで指定される位置を中心とした、他のフィールドのサーチする領域から、注目フィールドにおける注目点の周辺の点の第２の特徴量および注目点の第２の特徴量に、配置および値の最も近い第２の特徴量であって、他のフィールドにおける第２の特徴量の位置を示す、第２のベクトルとして動きベクトルが検出され、第１の特徴量または第２の特徴量および動きベクトルによって、予め定めた複数のクラスのうちの１つのクラスに注目点が分類され、分類されたクラス毎に予め定められている予測タップであって、第１の特徴量または第２の特徴量を含む予測タップが抽出され、コンポーネントビデオ信号と抽出された予測タップとから求められる正規方程式であって、残差を最小とする正規方程式を解くことにより、分類されたクラスの係数が演算されるようにしたので、係数を利用することにより、動きベクトルを考慮して、コンポジットビデオ信号から、より精度良く、コンポーネントビデオ信号を生成できるようになる。
【図面の簡単な説明】
【図１】本発明に係る画像処理装置の実施の形態の構成を示すブロック図である。
【図２】画像信号およびサブキャリア位相情報を説明する図である。
【図３】動きベクトル検出部２の構成を示すブロック図である。
【図４】上位階層動きベクトル検出部２５の構成を示すブロック図である。
【図５】下位階層動きベクトル検出部２６の構成を示すブロック図である。
【図６】上位階層特徴量、下位階層特徴量、およびサーチ領域の例を示す図である。
【図７】上位階層特徴量のブロックを説明する図である。
【図８】下位階層特徴量の算出を説明する図である。
【図９】下位階層動きベクトル検出部２６がサーチする領域およびブロックの例を示す図である。
【図１０】上位階層特徴量の他のブロックの例を示す図である。
【図１１】上位階層特徴量の他の例を示す図である。
【図１２】上位階層特徴量の他の例を示す図である。
【図１３】下位階層特徴量の他の例を示す図である。
【図１４】下位階層特徴量の他の例を示す図である。
【図１５】下位階層特徴量の他の例を示す図である。
【図１６】下位階層特徴量の他の例を示す図である。
【図１７】下位階層特徴量の他の例を示す図である。
【図１８】下位階層動きベクトル検出部２６の他の構成を示すブロック図である。
【図１９】演算部４の構成を示すブロック図である。
【図２０】上位階層特徴量に対応するクラスタップおよび予測タップの例を説明する図である。
【図２１】下位階層特徴量に対応するクラスタップおよび予測タップの例を説明する図である。
【図２２】演算部４の他の構成を示すブロック図である。
【図２３】演算部４の他の構成を示すブロック図である。
【図２４】演算部４の他の構成を示すブロック図である。
【図２５】本発明に係る画像処理装置の実施の形態の構成の一例の詳細を示すブロック図である。
【図２６】コンポーネントビデオ信号の生成の処理を説明するフローチャートである。
【図２７】動きベクトルの検出の処理を説明するフローチャートである。
【図２８】上位階層ベクトルの検出の処理の詳細を説明するフローチャートである。
【図２９】下位階層ベクトルの検出の処理の詳細を説明するフローチャートである。
【図３０】下位階層ベクトルの検出の他の処理の詳細を説明するフローチャートである。
【図３１】予測係数を生成する、本発明に係る画像処理装置の実施の形態の構成を示すブロック図である。
【図３２】係数算出部２０２の構成を示すブロック図である。
【図３３】係数算出部２０２の他の構成を示すブロック図である。
【図３４】係数算出部２０２の他の構成を示すブロック図である。
【図３５】係数算出部２０２の他の構成を示すブロック図である。
【図３６】予測係数を算出する画像処理装置の実施の形態の構成の一例の詳細を示すブロック図である。
【図３７】予測係数を算出する学習の処理を説明するフローチャートである。
【図３８】下位階層ベクトルの演算の処理に使用される予測係数を生成する画像処理装置の構成を示すブロック図である。
【図３９】係数生成部３０３の構成を示すブロック図である。
【図４０】学習部３２１の構成を示すブロック図である。
【図４１】画像処理装置の学習の処理を説明するフローチャートである。
【図４２】係数セットの算出の処理の詳細を説明するフローチャートである。
【図４３】記録媒体およびコンピュータの例を説明する図である。
【符号の説明】
１ NTSCデコーダ， ２ 動きベクトル検出部， ３ クラスタップ抽出部，４ 演算部， ２３ 特徴量変換部， ２４ 特徴量変換部， ２５ 上位階層動きベクトル検出部， ２６ 下位階層動きベクトル検出部， ３１ ベクトル生成部， ３２ 相関値算出部， ３３ 判定部， ４１ ベクトル生成部，４２ 相関値算出部， ４３ 判定部， ５１ クラス分類部， ５２ 予測タップ抽出部， ５３ 演算部， ５４ 予測係数メモリ， ５５ メモリ， ７１−１乃至７１−３ クラス分類部， ７２−１乃至７２−３ 予測タップ抽出部， ７３−１乃至７３−３ 係数メモリ， ７４−１乃至７４−３ 予測演算部， ８１ セレクタ， ８２ 予測タップ抽出部， ８３ 予測演算部， ９１ クラス分類部， １０１ セレクタ， ２０１ NTSCエンコーダ， ２０２ 係数算出部， ２０３ 係数メモリ， ２１１−１乃至２１１−３ 予測係数算出部， ２３１ セレクタ， ２３２ 予測係数算出部， ３０３ 係数生成部， ３２１ 学習部， ３５１ クラス分類部， ３５２ 予測タップ抽出部， ３５３ 係数算出部， ３５４ メモリ， ３５５ 予測係数メモリ， ５０１ ＣＰＵ， ５０２ ＲＯＭ， ５０３ ＲＡＭ， ５１１ HDD， ５５１ 磁気ディスク， ５５２ 光ディスク， ５５３ 光磁気ディスク， ５５４ 半導体メモリ[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus and method, a recording medium, and a program, and in particular, an image for generating a component video signal based on a composite video signal or generating a coefficient used for processing for generating a component video signal. The present invention relates to a processing apparatus and method, a recording medium, and a program.
[0002]
[Prior art]
In so-called three-dimensional YC separation, the amount of motion is detected, and based on the detected amount of motion, either one of two-dimensional filter processing or three-dimensional filter processing is selected and executed. Committee) The composite video signal is separated into component video signals including a luminance signal and a color signal.
[0003]
[Problems to be solved by the invention]
However, conventionally, it has not been considered to separate a composite video signal into component video signals in consideration of motion vectors.
[0004]
The present invention has been made in view of such a situation, and an object thereof is to enable generation of a component video signal with higher accuracy from a composite video signal in consideration of a motion vector.
[0005]
[Means for Solving the Problems]
  The first image processing apparatus of the present invention includes:It is obtained by adding or subtracting the luminance signal and the color signal of the composite video signal for the target field and the other field of interest, where the target point is the target point of the composite video signal. From the obtained image signal, it is the value of a point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, and the average of four points having different phases. A first feature amount that is a value or an average value of four points that are adjacent to each other vertically and horizontally on two lines of a field and that are different in phase, A first feature amount of the point and a first feature amount of the point of interest in the attention field around the point, the first feature amount closest to the arrangement and value, and in the other field By detecting the first feature amount, the first vector that approximates the motion vector is detected, and the value of the target point that is the target of the motion vector detection calculation for the field of interest and other fields is calculated. The value obtained by multiplying the weight of 1 and the value of a point around the target point in the field where the target point exists, adjacent to the target point, or separated by one or two points from the target point A second feature value corresponding to the target point, which is a value obtained by multiplying the result obtained by multiplying the second weight by the other weight, and another field centered on the position specified by the first vector The second feature quantity closest to the second feature quantity and the second feature quantity of the point of interest in the field of interest from the search area of The position of the second feature value in It is,Detecting means for detecting a motion vector as a second vector;The attention point is classified into one of a plurality of predetermined classes based on the first feature amount or the second feature amount and the motion vector.Classification means,A prediction tap that is predetermined for each classified class and that includes the first feature amount or the second feature amount is extracted.Extraction means;
  By applying a sum-of-products operation to the previously calculated prediction coefficient and the extracted prediction tap,Generating means for generating a component video signal corresponding to the composite video signal of the point of interest.
[0006]
  The extracting means further corresponds to the dynamic range of the composite video signal or the composite video signal.1st feature value or 2nd feature valueDynamic rangeIt is determined according toA prediction tap can be extracted.
[0007]
  Classifying means isFor the luminance signal constituting the component video signal, the composite video signal at the point of interest is classified into one of a plurality of classes, and for the color signal constituting the component video signal, the composite video signal at the point of interest is plural. Class into one of the classesCan be.
[0008]
  The extraction means isA prediction tap is extracted for the luminance signal constituting the component video signal, and further, a prediction tap is extracted for the color signal constituting the component video signal.be able to.
[0009]
  The detection means isFor the field of interest and other fields, from the image signal, the value of the point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, A first value that is an average value of four points of different phases, or four points that are adjacent to each other vertically and horizontally on two lines of a field, each of which is an average value of four points of different phases FeaturesFirst feature quantity generating means for generatingThe first feature quantity closest to the first feature quantity of the point and the first feature quantity of the point of interest in the field of interest around the point of interest, and the first feature quantity in the other field closest to the arrangement and value To detect the first vector that approximates the motion vectorFirst vector detecting means forFor the field of interest and other fields, the result obtained by multiplying the value of the target point that is the target of motion vector detection calculation by the first weight, and the points around the target point in the field where the target point exists The value corresponding to the target point is a value obtained by adding the value obtained by multiplying the value of a point adjacent to the target point or separated from the target point by one or two points by the second weight. Feature amount of 2Second feature quantity generating means for generatingArranged from the area to be searched for in the other field, centered on the position specified by the first vector, to the second feature quantity around the target point in the target field and the second feature quantity of the target point And a motion vector as a second vector that is the second feature value closest in value and that indicates the position of the second feature value in another field.Second vector detecting means can be provided.
[0010]
  The first vector detecting means includesThe first feature quantity of the field of interest that is the screen of the moving image to which the point of interest belongs and the first feature quantity having the strongest correlation with the first feature amount of the other field that is the screen of the moving image adjacent to the screen of interest. The first vector approximate to the motion vector can be detected from the position of the feature amount.
[0011]
  The second vector detection means is:A moving image that is the second feature amount in the first range including the attention point and the second feature amount having the strongest correlation in the attention field that is the screen of the moving image to which the attention point belongs, and is adjacent to the attention field. The motion vector can be detected as the second vector from the position of the second feature value in the area to be searched centering on the position specified by the first vector in the other field of the screen.
[0012]
  The second vector detection means is:Second feature valueBy applying the class classification adaptive processing to the motion vector, the motion vector can be detected as the second vector.
[0013]
  The second vector detecting means is a phase of the color signal at the point of interest.UsingCan be classified.
[0014]
  The second vector detection means is:The adaptive process can be applied to the second feature quantity at the position indicated by the second vector detected in the previous field.
[0015]
  The first image processing method of the present invention includes:It is obtained by adding or subtracting the luminance signal and the color signal of the composite video signal for the target field and the other field of interest, where the target point is the target point of the composite video signal. From the obtained image signal, it is the value of a point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, and the average of four points having different phases. A first feature amount that is a value or an average value of four points that are adjacent to each other vertically and horizontally on two lines of a field and that are different in phase, A first feature amount of the point and a first feature amount of the point of interest in the attention field around the point, the first feature amount closest to the arrangement and value, and in the other field By detecting the first feature amount, the first vector that approximates the motion vector is detected, and the value of the target point that is the target of the motion vector detection calculation for the field of interest and other fields is calculated. The value obtained by multiplying the weight of 1 and the value of a point around the target point in the field where the target point exists, adjacent to the target point, or separated by one or two points from the target point A second feature value corresponding to the target point, which is a value obtained by multiplying the result obtained by multiplying the second weight by the other weight, and another field centered on the position specified by the first vector The second feature quantity closest to the second feature quantity and the second feature quantity of the point of interest in the field of interest from the search area of The position of the second feature value in You, as a second vectorA detection step of detecting a motion vector;The attention point is classified into one of a plurality of predetermined classes based on the first feature amount or the second feature amount and the motion vector.A classification step;A prediction tap that is predetermined for each classified class and that includes the first feature amount or the second feature amount is extracted.An extraction step;By applying a sum-of-products operation to the previously calculated prediction coefficient and the extracted prediction tap,Generating a component video signal corresponding to the composite video signal of the point of interest.
[0016]
  The program of the first recording medium of the present invention isIt is obtained by adding or subtracting the luminance signal and the color signal of the composite video signal for the target field and the other field of interest, where the target point is the target point of the composite video signal. From the obtained image signal, it is the value of a point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, and the average of four points having different phases. A first feature amount that is a value or an average value of four points that are adjacent to each other vertically and horizontally on two lines of a field and that are different in phase, A first feature amount of the point and a first feature amount of the point of interest in the attention field around the point, the first feature amount closest to the arrangement and value, and in the other field By detecting the first feature amount, the first vector that approximates the motion vector is detected, and the value of the target point that is the target of the motion vector detection calculation for the field of interest and other fields is calculated. The value obtained by multiplying the weight of 1 and the value of a point around the target point in the field where the target point exists, adjacent to the target point, or separated by one or two points from the target point A second feature value corresponding to the target point, which is a value obtained by multiplying the result obtained by multiplying the second weight by the other weight, and another field centered on the position specified by the first vector The second feature quantity closest to the second feature quantity and the second feature quantity of the point of interest in the field of interest from the search area of The position of the second feature value in You, as a second vectorA detection step of detecting a motion vector;The attention point is classified into one of a plurality of predetermined classes based on the first feature amount or the second feature amount and the motion vector.A classification step;A prediction tap that is predetermined for each classified class and that includes the first feature amount or the second feature amount is extracted.An extraction step;By applying a sum-of-products operation to the previously calculated prediction coefficient and the extracted prediction tap,Generating a component video signal corresponding to the composite video signal of the point of interest.
[0017]
  The first program of the present invention is:It is obtained by adding or subtracting the luminance signal and the color signal of the composite video signal for the target field and the other field of interest, where the target point is the target point of the composite video signal. From the obtained image signal, it is the value of a point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, and the average of four points having different phases. A first feature amount that is a value or an average value of four points that are adjacent to each other vertically and horizontally on two lines of a field and that are different in phase, A first feature amount of the point and a first feature amount of the point of interest in the attention field around the point, the first feature amount closest to the arrangement and value, and in the other field By detecting the first feature amount, the first vector that approximates the motion vector is detected, and the value of the target point that is the target of the motion vector detection calculation for the field of interest and other fields is calculated. The value obtained by multiplying the weight of 1 and the value of a point around the target point in the field where the target point exists, adjacent to the target point, or separated by one or two points from the target point A second feature value corresponding to the target point, which is a value obtained by multiplying the result obtained by multiplying the second weight by the other weight, and another field centered on the position specified by the first vector The second feature quantity closest to the second feature quantity and the second feature quantity of the point of interest in the field of interest from the search area of The position of the second feature value in You, as a second vectorA detection step of detecting a motion vector;The attention point is classified into one of a plurality of predetermined classes based on the first feature amount or the second feature amount and the motion vector.A classification step;A prediction tap that is predetermined for each classified class and that includes the first feature amount or the second feature amount is extracted.An extraction step;By applying a sum-of-products operation to the previously calculated prediction coefficient and the extracted prediction tap,And generating a component video signal corresponding to the composite video signal at the point of interest.
[0018]
  The second image processing apparatus of the present invention isIt is obtained by adding or subtracting the luminance signal and the color signal of the composite video signal for the target field and the other field of interest, where the target point is the target point of the composite video signal. From the obtained image signal, it is the value of a point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, and the average of four points having different phases. A first feature amount that is a value or an average value of four points that are adjacent to each other vertically and horizontally on two lines of a field and that are different in phase, A first feature amount of the point and a first feature amount of the point of interest in the attention field around the point, the first feature amount closest to the arrangement and value, and in the other field By detecting the first feature amount, the first vector that approximates the motion vector is detected, and the value of the target point that is the target of the motion vector detection calculation for the field of interest and other fields is calculated. The value obtained by multiplying the weight of 1 and the value of a point around the target point in the field where the target point exists, adjacent to the target point, or separated by one or two points from the target point A second feature value corresponding to the target point, which is a value obtained by multiplying the result obtained by multiplying the second weight by the other weight, and another field centered on the position specified by the first vector The second feature quantity closest to the second feature quantity and the second feature quantity of the point of interest in the field of interest from the search area of The position of the second feature value in You, as a second vectorDetecting means for detecting a motion vector;The attention point is classified into one of a plurality of predetermined classes based on the first feature amount or the second feature amount and the motion vector.Classification means,A prediction tap that is predetermined for each classified class and that includes the first feature amount or the second feature amount is extracted.Extraction means;A normal equation obtained from the component video signal and the extracted prediction taps, and by solving the normal equation that minimizes the residual,And calculating means for calculating a coefficient.
[0019]
  The extracting means further corresponds to the dynamic range of the composite video signal or the composite video signal.1st feature value or 2nd feature valueDynamic rangeIt is determined according toPrediction taps can be extracted.
[0020]
  Classifying means isFor the luminance signal constituting the component video signal, the composite video signal at the point of interest is classified into one of a plurality of classes, and for the color signal constituting the component video signal, the composite video signal at the point of interest is plural. Can be classified into one of the classes.
[0021]
  The extraction means isPrediction taps can be extracted for the luminance signals constituting the component video signal, and further, prediction taps can be extracted for the color signals constituting the component video signal..
[0022]
  The detection means isFor the field of interest and other fields, from the image signal, the value of the point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, A first value that is an average value of four points of different phases, or four points that are adjacent to each other vertically and horizontally on two lines of a field, each of which is an average value of four points of different phases FeaturesFirst feature quantity generating means for generatingThe first feature quantity closest to the first feature quantity of the point and the first feature quantity of the point of interest in the field of interest around the point of interest, and the first feature quantity in the other field closest to the arrangement and value To detect the first vector that approximates the motion vectorFirst vector detecting means forFor the field of interest and other fields, the result obtained by multiplying the value of the target point that is the target of motion vector detection calculation by the first weight, and the points around the target point in the field where the target point exists The value corresponding to the target point is a value obtained by adding the value obtained by multiplying the value of a point adjacent to the target point or separated from the target point by one or two points by the second weight. Feature amount of 2Second feature quantity generating means for generatingArranged from the area to be searched for in the other field, centered on the position specified by the first vector, to the second feature quantity around the target point in the target field and the second feature quantity of the target point And a motion vector as a second vector that is the second feature value closest in value and that indicates the position of the second feature value in another field.Second vector detecting means can be provided.
[0023]
  The first vector detecting means includesThe first feature quantity of the field of interest that is the screen of the moving image to which the point of interest belongs and the first feature quantity having the strongest correlation with the first feature amount of the other field that is the screen of the moving image adjacent to the screen of interest. The first vector approximate to the motion vector can be detected from the position of the feature amount.
[0024]
  The second vector detection means is:A moving image that is the second feature amount in the first range including the attention point and the second feature amount having the strongest correlation in the attention field that is the screen of the moving image to which the attention point belongs, and is adjacent to the attention field. The motion vector can be detected as the second vector from the position of the second feature value in the area to be searched centering on the position specified by the first vector in the other field on the screen.
[0025]
  The second vector detection means is:Second feature valueBy applying the class classification adaptive processing to the motion vector, the motion vector can be detected as the second vector.
[0026]
  The second vector detecting means is a phase of the color signal at the point of interest.UsingCan be classified.
[0027]
  The second vector detection means is:The adaptive process can be applied to the second feature quantity at the position indicated by the second vector detected in the previous field.
[0028]
  The second image processing method of the present invention comprises:It is obtained by adding or subtracting the luminance signal and the color signal of the composite video signal for the target field and the other field of interest, where the target point is the target point of the composite video signal. From the obtained image signal, it is the value of a point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, and the average of four points having different phases. A first feature amount that is a value or an average value of four points that are adjacent to each other vertically and horizontally on two lines of a field and that are different in phase, A first feature amount of the point and a first feature amount of the point of interest in the attention field around the point, the first feature amount closest to the arrangement and value, and in the other field By detecting the first feature amount, the first vector that approximates the motion vector is detected, and the value of the target point that is the target of the motion vector detection calculation for the field of interest and other fields is calculated. The value obtained by multiplying the weight of 1 and the value of a point around the target point in the field where the target point exists, adjacent to the target point, or separated by one or two points from the target point A second feature value corresponding to the target point, which is a value obtained by multiplying the result obtained by multiplying the second weight by the other weight, and another field centered on the position specified by the first vector The second feature quantity closest to the second feature quantity and the second feature quantity of the point of interest in the field of interest from the search area of The position of the second feature value in You, as a second vectorA detection step of detecting a motion vector;The attention point is classified into one of a plurality of predetermined classes based on the first feature amount or the second feature amount and the motion vector.A classification step;A prediction tap that is predetermined for each classified class and that includes the first feature amount or the second feature amount is extracted.An extraction step;A normal equation obtained from the component video signal and the extracted prediction taps, and by solving the normal equation that minimizes the residual,And a calculation step for calculating a coefficient.
[0029]
  The program of the second recording medium of the present invention isIt is obtained by adding or subtracting the luminance signal and the color signal of the composite video signal for the target field and the other field of interest, where the target point is the target point of the composite video signal. From the obtained image signal, it is the value of a point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, and the average of four points having different phases. A first feature amount that is a value or an average value of four points that are adjacent to each other vertically and horizontally on two lines of a field and that are different in phase, A first feature amount of the point and a first feature amount of the point of interest in the attention field around the point, the first feature amount closest to the arrangement and value, and in the other field By detecting the first feature amount, the first vector that approximates the motion vector is detected, and the value of the target point that is the target of the motion vector detection calculation for the field of interest and other fields is calculated. The value obtained by multiplying the weight of 1 and the value of a point around the target point in the field where the target point exists, adjacent to the target point, or separated by one or two points from the target point A second feature value corresponding to the target point, which is a value obtained by multiplying the result obtained by multiplying the second weight by the other weight, and another field centered on the position specified by the first vector The second feature quantity closest to the second feature quantity and the second feature quantity of the point of interest in the field of interest from the search area of The position of the second feature value in You, as a second vectorA detection step of detecting a motion vector;The attention point is classified into one of a plurality of predetermined classes based on the first feature amount or the second feature amount and the motion vector.A classification step;A prediction tap that is predetermined for each classified class and that includes the first feature amount or the second feature amount is extracted.An extraction step;A normal equation obtained from the component video signal and the extracted prediction taps, and by solving the normal equation that minimizes the residual,And a calculation step for calculating a coefficient.
[0030]
  The second program of the present invention is:It is obtained by adding or subtracting the luminance signal and the color signal of the composite video signal for the target field and the other field of interest, where the target point is the target point of the composite video signal. From the obtained image signal, it is the value of a point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, and the average of four points having different phases. A first feature amount that is a value or an average value of four points that are adjacent to each other vertically and horizontally on two lines of a field and that are different in phase, A first feature amount of the point and a first feature amount of the point of interest in the attention field around the point, the first feature amount closest to the arrangement and value, and in the other field By detecting the first feature amount, the first vector that approximates the motion vector is detected, and the value of the target point that is the target of the motion vector detection calculation for the field of interest and other fields is calculated. The value obtained by multiplying the weight of 1 and the value of a point around the target point in the field where the target point exists, adjacent to the target point, or separated by one or two points from the target point A second feature value corresponding to the target point, which is a value obtained by multiplying the result obtained by multiplying the second weight by the other weight, and another field centered on the position specified by the first vector The second feature quantity closest to the second feature quantity and the second feature quantity of the point of interest in the field of interest from the search area of The position of the second feature value in You, as a second vectorA detection step of detecting a motion vector;The attention point is classified into one of a plurality of predetermined classes based on the first feature amount or the second feature amount and the motion vector.A classification step;A prediction tap that is predetermined for each classified class and that includes the first feature amount or the second feature amount is extracted.An extraction step;A normal equation obtained from the component video signal and the extracted prediction taps, and by solving the normal equation that minimizes the residual,The computer is caused to execute a calculation step for calculating a coefficient.
[0031]
  In the first image processing apparatus and method, recording medium, and program of the present invention,It is obtained by adding or subtracting the luminance signal and the color signal of the composite video signal for the target field and the other field of interest, where the target point is the target point of the composite video signal. From the obtained image signal, it is the value of a point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, and the average of four points having different phases. A first feature amount that is a value or an average value of four points that are adjacent to each other vertically and horizontally on two lines of a field and that are different in phase, A first feature amount of the point and a first feature amount of the point of interest in the attention field around the point, the first feature amount closest to the arrangement and value, and in the other field By detecting the first feature amount, the first vector that approximates the motion vector is detected, and the value of the target point that is the target of the motion vector detection calculation for the field of interest and other fields is calculated. The value obtained by multiplying the weight of 1 and the value of a point around the target point in the field where the target point exists, adjacent to the target point, or separated by one or two points from the target point A second feature value corresponding to the target point, which is a value obtained by multiplying the result obtained by multiplying the second weight by the other weight, and another field centered on the position specified by the first vector The second feature quantity closest to the second feature quantity and the second feature quantity of the point of interest in the field of interest from the search area of The position of the second feature value in You, as a second vectorMotion vectors are detected,The attention point is classified into one of a plurality of predetermined classes based on the first feature amount or the second feature amount and the motion vector.,Prediction taps that are predetermined for each classified class and that include the first feature amount or the second feature amount are extracted.,By applying a sum-of-products operation to the previously calculated prediction coefficient and the extracted prediction tap,A component video signal corresponding to the composite video signal at the point of interest is generated.
[0032]
In the second image processing apparatus and method, recording medium, and program of the present invention,It is obtained by adding or subtracting the luminance signal and the color signal of the composite video signal for the target field and the other field of interest, where the target point is the target point of the composite video signal. From the obtained image signal, it is the value of a point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, and the average of four points having different phases. A first feature amount that is a value or an average value of four points that are adjacent to each other vertically and horizontally on two lines of a field and that are different in phase, A first feature amount of the point and a first feature amount of the point of interest in the attention field around the point, the first feature amount closest to the arrangement and value, and in the other field By detecting the first feature amount, the first vector that approximates the motion vector is detected, and the value of the target point that is the target of the motion vector detection calculation for the field of interest and other fields is calculated. The value obtained by multiplying the weight of 1 and the value of a point around the target point in the field where the target point exists, adjacent to the target point, or separated by one or two points from the target point A second feature value corresponding to the target point, which is a value obtained by multiplying the result obtained by multiplying the second weight by the other weight, and another field centered on the position specified by the first vector The second feature quantity closest to the second feature quantity and the second feature quantity of the point of interest in the field of interest from the search area of The position of the second feature value in You, as a second vectorMotion vectors are detected,The attention point is classified into one of a plurality of predetermined classes based on the first feature amount or the second feature amount and the motion vector.,Prediction taps that are predetermined for each classified class and that include the first feature amount or the second feature amount are extracted.,A normal equation obtained from the component video signal and the extracted prediction taps, and by solving the normal equation that minimizes the residual,The coefficient is calculated.
[0033]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is a block diagram showing a configuration of an embodiment of an image processing apparatus according to the present invention.
[0034]
The NTSC decoder 1 generates an image signal that is digital data corresponding to the image signal based on the input image signal that is an NTSC composite video signal, and also a sub signal corresponding to the image signal that is digital data. Carrier phase information is generated. The NTSC decoder 1 supplies the generated image data and subcarrier phase information, which are digital data, to the motion vector detection unit 2, the class tap extraction unit 3, and the calculation unit 4.
[0035]
The image signal which is digital data generated by the NTSC decoder 1 is another signal indicating the color from the value of the Y signal as a result of subtracting the value of the I signal indicating the color from the value of the Y signal indicating the luminance, for example. As a result of subtracting the value of the Q signal, it consists of one of a result of adding the value of the I signal to the value of the Y signal and a result of adding the value of the Q signal to the value of the Y signal.
[0036]
The subcarrier phase information generated by the NTSC decoder 1 is the result of subtracting the value of the I signal from the value of the Y signal if the value included in the image signal that is digital data generated by the NTSC decoder 1 is Y Whether it is the result of subtracting the value of the Q signal from the value of the signal, the result of adding the value of the I signal to the value of the Y signal, or the result of adding the value of the Q signal to the value of the Y signal It is information which shows.
[0037]
FIG. 2 is a diagram for explaining an image signal and subcarrier phase information which are digital data generated by the NTSC decoder 1.
[0038]
Hereinafter, in this specification, field # -1 indicates a field before field # 0, and field # -2 indicates a field before field # -1. Field # + 1 indicates the next (after) field # 0, and field # + 2 indicates the next field after field # + 1.
[0039]
The field where the absolute value of the field # 0 and the number is an even number is a so-called top field. A field whose absolute number is an odd number is a so-called bottom field.
[0040]
For example, field # -1 is a bottom field and includes image signals corresponding to line 2, line 4, line 6, and the like. Field # 0 is a so-called top field, and includes image signals corresponding to line 1, line 3, line 5, and the like.
[0041]
For example, the frame # -1 includes a field # -2 and a field # -3 (not shown). Frame # 0 is composed of field # 0 and field # -1. Frame # + 1 is composed of field # + 2 and field # + 1.
[0042]
Thus, for example, in frame # 0, line 1 of field # 0, line 2 of field # -1, line 3 of field # 0, line 4 of field # -1, line 5 of field # 0, and field # -1 lines 6 are arranged in order from the top of the screen, and the line of field # 0 and the line of field # -1 are alternately arranged.
[0043]
The digital data generated by the NTSC decoder 1 corresponds to a period in which the phase of a portion of the NTSC composite video signal that is balanced and modulated by the luminance signal and the chrominance signal changes, and is discrete in the spatial direction in the field. It is a lot of data. Individual values corresponding to so-called pixels constituting the digital data generated by the NTSC decoder 1 are generated by quantizing an NTSC composite video signal and correspond to one luminance and color.
[0044]
In FIG. 2, a white square, a white circle, a black square, and a black circle constitute an image signal that is digital data, and each represents one value corresponding to one pixel.
[0045]
A white square indicates a value corresponding to the result of adding the value of the I signal indicating the color to the value of the Y signal indicating the luminance. A white circle indicates a value corresponding to the result of adding the value of the Q signal, which is another signal indicating the color, to the value of the Y signal.
[0046]
A black square indicates a value corresponding to a result obtained by subtracting the value of the I signal from the value of the Y signal. A black circle indicates a value corresponding to a result obtained by subtracting the value of the Q signal from the value of the Y signal.
[0047]
The addition or subtraction of the value of the Y signal and the value of the I signal or the addition or subtraction of the value of the Y signal and the value of the Q signal in the value of the image signal which is digital data generated by the NTSC decoder 1 is performed by the NTSC decoder 1 This corresponds to the inversion of the phases of the I signal and the Q signal corresponding to the color signal component in which the subcarrier is balanced and modulated in the NTSC composite video signal input to.
[0048]
The subcarrier phase information is obtained by subtracting the value of the I signal from the value of the Y signal and the value of the Q signal from the value of the Y signal for each of the image signal values that are digital data generated by the NTSC decoder 1 Information indicating whether the value corresponds to a value obtained by adding the value of the I signal to the value of the Y signal, or a value obtained by adding the value of the Q signal to the value of the Y signal.
[0049]
The motion vector detection unit 2 is based on the parameters A and B supplied from the outside of the image processing apparatus, and the image signal and subcarrier phase information supplied from the NTSC decoder 1, and the upper layer feature amount and the lower layer feature amount. Is generated. The motion vector detection unit 2 performs motion between the field of interest and the next field corresponding to each value of the image signal supplied from the NTSC decoder 1 based on the upper layer feature quantity and the lower layer feature quantity. Generate a vector. Details of the upper layer feature amount and the lower layer feature amount will be described later.
[0050]
The motion vector detection unit 2 supplies the generated upper layer feature quantity, lower layer feature quantity, and motion vector to the class tap extraction unit 3 and the calculation unit 4. The motion vector detection unit 2 uses the class tap extraction unit 3 and the arithmetic unit to calculate other feature amounts such as a residual calculated in the block matching process of the motion vector together with the upper layer feature amount, the lower layer feature amount, and the motion vector. Supply to part 4.
[0051]
The parameter A and the parameter B are parameters for designating the contents of processing in the motion vector detection unit 2. For example, the correlation calculation method in the motion vector detection processing of the motion vector detection unit 2, the size of the block to be matched Specify the size of the search area.
[0052]
Based on the subcarrier phase information supplied from the NTSC decoder 1 and the motion vector supplied from the motion vector detection unit 2, the class tap extraction unit 3 performs image signal data, upper layer feature value data, and lower layer Class taps consisting of feature data are extracted. Although details will be described later, the class tap is data for class classification corresponding to the data of interest.
[0053]
The class tap extraction unit 3 supplies the extracted class tap to the calculation unit 4.
[0054]
The calculation unit 4 performs class classification based on the class tap supplied from the class tap extraction unit 3, based on the subcarrier phase information supplied from the NTSC decoder 1 and the motion vector supplied from the motion vector detection unit 2. In addition, adaptive processing is applied to the image signal supplied from the NTSC decoder 1 and the upper layer feature quantity and lower layer feature quantity supplied from the motion vector detection unit 2 to correspond to the image signal that is a composite video signal. For example, a component video signal composed of a Y signal, a U signal, and a V signal is generated. The computing unit 4 outputs the generated component video signal. Details of the classification processing and the adaptation processing will be described later.
[0055]
As described above, the image processing apparatus according to the present invention generates a component video signal corresponding to an input image signal that is a composite video signal, and outputs the generated component video signal.
[0056]
FIG. 3 is a block diagram illustrating a configuration of the motion vector detection unit 2. The image signal and subcarrier phase information supplied from the NTSC decoder 1 are input to the buffer 21, the feature amount conversion unit 23, and the feature amount conversion unit 24.
[0057]
The buffer 21 stores the image signal and subcarrier phase information supplied from the NTSC decoder 1, and supplies the stored image signal and subcarrier phase information to the feature amount conversion unit 23 and the buffer 22. That is, the buffer 21 delays the input image signal and subcarrier phase information for a period corresponding to one field, and supplies the delayed image signal and subcarrier phase information to the feature amount conversion unit 23 and the buffer 22.
[0058]
The buffer 22 stores the image signal and subcarrier phase information supplied from the buffer 21, and supplies the stored image signal and subcarrier phase information to the feature amount conversion unit 24. That is, the buffer 22 delays the image signal and subcarrier phase information supplied from the buffer 21 for a period corresponding to one field, and supplies the delayed image signal and subcarrier phase information to the feature amount conversion unit 24.
[0059]
Based on the image signal and subcarrier phase information input from the NTSC decoder 1 and the image signal and subcarrier phase information supplied from the buffer 21, the feature amount conversion unit 23 generates and generates an upper layer feature amount. The upper layer feature quantity is output to the outside of the motion vector detection unit 2 and the upper layer feature quantity is supplied to the upper layer motion vector detection unit 25.
[0060]
The feature amount conversion unit 24 generates lower layer feature amounts based on the image signal and subcarrier phase information input from the NTSC decoder 1 and the image signal and subcarrier phase information supplied from the buffer 22, and generates a subcarrier feature amount. The generated upper layer feature quantity is supplied to the lower layer motion vector detection unit 26 together with the phase information. The feature amount conversion unit 24 outputs the generated lower layer feature amount to the outside of the motion vector detection unit 2.
[0061]
Note that the feature quantity conversion unit 23 and the feature quantity conversion unit 24 generate upper layer feature amounts or lower layer feature amounts in real time based on the input image signal and subcarrier phase signal, and sequentially output them. Alternatively, the generated upper layer feature amount or lower layer feature amount may be temporarily stored for each field, and the upper layer feature amount or the lower layer feature amount may be collectively output in units of fields.
[0062]
The upper layer motion vector detection unit 25 approximates the motion vector that is finally detected based on the upper layer feature amount supplied from the feature amount conversion unit 23 by the calculation method specified by the parameter A, and the like. An upper layer vector indicating a rough motion in between is generated, and the generated upper layer vector is supplied to the lower layer motion vector detecting unit 26.
[0063]
The lower layer motion vector detection unit 26 is supplied from the lower layer feature quantity and subcarrier phase information supplied from the feature amount conversion unit 24 and the upper layer motion vector detection unit 25 according to the calculation method specified by the parameter B or the like. Based on the higher layer vector, a lower layer vector with higher accuracy is generated as compared with the upper layer vector, and the generated lower layer vector is output as a motion vector.
[0064]
FIG. 4 is a block diagram showing a configuration of the upper layer motion vector detection unit 25. The vector generation unit 31 sequentially generates vectors having a predetermined range size and direction based on the data included in the parameter A indicating the size and direction range of the generated vector, and correlates the generated vectors. It supplies to the value calculation part 32 and the determination part 33.
[0065]
The correlation value calculation unit 32 is based on the data included in the parameter A that specifies the calculation method of the correlation value, and the upper layer feature quantity corresponding to one field specified by the vector supplied from the vector generation unit 31. Then, a correlation value with the upper layer feature quantity corresponding to another field is calculated, and the calculated correlation value is supplied to the determination unit 33. For example, the correlation value is the sum of the absolute values of the differences between the upper layer feature values. The correlation value can be the sum of the squares of the absolute values of the differences between the upper layer feature values. Alternatively, the correlation value is a correlation in which an upper layer feature amount corresponding to one field and an upper layer feature amount corresponding to another field are a sequence of two random variables, that is, a cross correlation between upper layer feature amounts. It can be.
[0066]
The correlation value calculation unit 32 executes a correlation value calculation process using a sparse block using a higher layer feature quantity that is sparse compared to the data of the original image signal. Accordingly, it is possible to reduce the amount of calculation in the upper layer motion vector detection unit 25.
[0067]
The determination unit 33 stores the correlation value supplied from the correlation value calculation unit 32 in association with the vector supplied from the vector generation unit 31. When the correlation value corresponding to the vector having the size and direction of the predetermined range is calculated, the determination unit 33 stores the correlation stored based on the data included in the parameter A that specifies the calculation method of the correlation value Among the values, a correlation value having the strongest correlation is selected, and a vector corresponding to the selected correlation value is output as an upper layer vector. For example, when the correlation value is the sum of the squares of the absolute values of the differences between the upper layer feature values, the determination unit 33 outputs a vector corresponding to the minimum correlation value as an upper layer vector.
[0068]
FIG. 5 is a block diagram illustrating a configuration of the lower layer motion vector detection unit 26. The vector generation unit 41 has a predetermined range size based on the upper layer vector supplied from the upper layer motion vector detection unit 25 and the data included in the parameter B indicating the size and direction range of the generated vector. Vectors having length and direction are sequentially generated, and the generated vectors are supplied to the correlation value calculation unit 42 and the determination unit 43.
[0069]
The correlation value calculation unit 42, based on the data included in the parameter B that specifies the calculation method of the correlation value, the lower layer feature quantity corresponding to one field specified by the vector supplied from the vector generation unit 41, A correlation value with a lower layer feature amount corresponding to another field is calculated, and the calculated correlation value is supplied to the determination unit 43. For example, the correlation value is the sum of absolute values of differences in lower layer feature quantities. The correlation value can be the sum of the squares of the absolute values of the differences between the lower layer feature values. Alternatively, the correlation value is a correlation in which a lower layer feature amount corresponding to one field and a lower layer feature amount corresponding to another field are a sequence of two random variables, that is, a cross correlation between lower layer feature amounts. It can be.
[0070]
The correlation value calculation unit 42 executes a correlation value calculation process using a dense block using lower layer feature quantities having a density equivalent to that of the original image signal data.
[0071]
The determination unit 43 stores the correlation value supplied from the correlation value calculation unit 42 in association with the vector supplied from the vector generation unit 41. When the correlation value corresponding to the vector having the size and direction of the predetermined range is calculated, the determination unit 43 stores the correlation stored based on the data included in the parameter B that specifies the correlation value calculation method. Among the values, a correlation value having the strongest correlation is selected, and a vector corresponding to the selected correlation value is output as a lower layer vector. For example, when the correlation value is the sum of the absolute values of the differences between the lower layer feature amounts, the determination unit 43 outputs a vector corresponding to the minimum correlation value as a lower layer vector.
[0072]
A detailed example of the process executed by the motion vector detection unit 2 will be described with reference to FIGS.
[0073]
FIG. 6 shows the upper layer feature amount used for the upper layer vector detection process by the upper layer motion vector detection unit 25, and the lower layer feature used for the lower layer vector detection process by the lower layer motion vector detection unit 26. It is a figure which shows the example of the search area | region of quantity and the lower hierarchy motion vector detection part 26. FIG.
[0074]
The numbers in FIG. 6 represent weights. For example, when all of the weights are 1, the feature amount conversion unit 23 outputs data of a predetermined phase included in the image signal as an upper layer feature amount as it is.
[0075]
For example, the upper layer motion vector detection unit 25 includes attention data corresponding to a target point for which a motion vector is to be detected among upper layer feature amounts composed of data having the same phase as the target data, and surroundings of the target data. Six data is made into one block, and matching with the data of interest and the phase in the other field is taken. That is, the upper layer motion vector detecting unit 25 belongs to one block of the field of interest, and the arrangement of a predetermined number of feature values and the value of the feature value of the other field whose value is most approximate. Detect position. The upper layer vector indicates the position of the feature quantity whose arrangement and value of the other field are closest to the position of the attention data of the field of interest.
[0076]
Here, the data phase being the same means that the signal from which the data is calculated is the same, and that the data calculation method is the same as addition or subtraction.
[0077]
For example, when one data has a value obtained by subtracting the value of the I signal from the value of the Y signal and another data has a value obtained by subtracting the value of the I signal from the value of the Y signal, the phase of the two data Are the same. When one data has a value obtained by subtracting the value of the Q signal from the value of the Y signal and the other data has a value obtained by subtracting the value of the Q signal from the value of the Y signal, the phases of the two data are the same. It is.
[0078]
Similarly, when one data has a value obtained by adding the value of the I signal to the value of the Y signal, and another data has a value obtained by adding the value of the I signal to the value of the Y signal, the two data The phase is the same. When one data has a value obtained by adding the value of the Q signal to the value of the Y signal, and the other data has a value obtained by adding a value obtained by adding the value of the Q signal to the value of the Y signal, these two data Are in the same phase.
[0079]
On the other hand, data having a value obtained by subtracting the value of the I signal from the value of the Y signal, data having a value obtained by adding the value of the I signal to the value of the Y signal, and a value obtained by subtracting the value of the Q signal from the value of the Y signal Data having data and data having a value obtained by adding the value of the Q signal to the value of the Y signal have different phases.
[0080]
The feature amount conversion unit 23 can know whether or not the phases of the two data are the same based on the subcarrier phase information.
[0081]
As shown in FIG. 7, for example, the upper layer motion vector detection unit 25 has the same phase as the attention data and the attention data around the attention data indicated by white circles in the field # -1 that is the field of interest. The matching with the field # 0, which is the corresponding field, is performed with the block of the upper layer feature value composed of six data corresponding to the white circles located at the vertices of the hexagon. More specifically, the upper layer motion vector detection unit 25 performs processing on the target data corresponding to the target point to be searched in the block of the upper layer feature amount of the field of interest and the field # 0 corresponding to the field of interest. Correlation between the target data and the block of the upper layer feature quantity composed of the six data corresponding to the white circles located at the vertices of the hexagons having the same phase as the target data around the target data indicated by the white circles To determine matching.
[0082]
Next, returning to FIG. 6, an example of lower layer feature amounts will be described.
[0083]
For example, the feature amount conversion unit 24 calculates a lower layer feature amount corresponding to the attention data based on data around the attention data in the field of interest. For example, the feature amount conversion unit 24 multiplies each of four data adjacent to the data of interest in the field of interest by weight 1 and multiplies the data of interest by weight 4 and adds the multiplication results. The lower layer feature amount.
[0084]
Similarly, the feature quantity conversion unit 24 uses the lower data corresponding to the target data for the target data corresponding to the target point to be searched for the field corresponding to the field of interest based on the data around the target data. A hierarchical feature amount is calculated.
[0085]
Further, for example, as shown in FIG. 8, the feature amount conversion unit 24, in the field of interest # -1, is two data above the attention data, two data below the attention data, and from the attention data. Each of the data on the two right sides and the data on the two left sides from the attention data is multiplied by the weight 1, the weighting data is multiplied by the weight 4, and the multiplication result is added to obtain a lower layer feature amount.
[0086]
In the corresponding field # 0, the feature amount conversion unit 24 has two upper data from the target data, two lower data from the target data, two right data from the target data, and two left data from the target data. Each of the data is multiplied by a weight of 1, the target data is multiplied by a weight of 4, and the multiplication result is added to obtain a lower layer feature quantity.
[0087]
The lower layer motion vector detecting unit 26 searches for a predetermined range of the feature amount of the corresponding field centered on the position of the data specified by the upper layer vector supplied from the upper layer motion vector detecting unit 25. The position of the block composed of a predetermined number of lower layer feature values in the field of interest and the block composed of the predetermined number of lower layer feature values having the strongest correlation are detected. The lower layer motion vector detection unit 26 outputs, as a motion vector, a lower layer vector indicating the center position of the detected block in the corresponding field from the position of the target data in the field of interest.
[0088]
FIG. 9 is a diagram illustrating an example of regions and blocks searched by the lower layer motion vector detection unit 26. In FIG. 9, triangles indicate lower layer feature amounts.
[0089]
In the example shown in FIG. 9, the searched area is a rectangular area in which seven lower layer feature amounts are arranged horizontally and seven lower layer feature amounts are arranged vertically. The block to be searched is a rectangular area in which five lower layer feature values are arranged horizontally and five lower layer feature values are arranged vertically.
[0090]
FIG. 10 is a diagram illustrating an example of another block of upper layer feature values for determining matching. In FIG. 10, triangles indicate data having the same phase as the data of interest.
[0091]
As illustrated in FIG. 10, for example, the upper layer motion vector detection unit 25, in field # -1, which is a field of interest, includes attention data indicated by black triangles and attention data around attention data indicated by white triangles. The block with the strongest correlation in the field # 0, which is the corresponding field, with the block of the upper layer feature quantity as a unit consisting of six data corresponding to the white triangle located at the vertex of the hexagon having the same phase Is detected.
[0092]
When the feature amount conversion unit 23 outputs an upper layer feature amount composed of image signal data in phase with the data of interest, the number of upper layer feature amount data is one-fourth of the original image signal data. Become.
[0093]
FIG. 11 and FIG. 12 are diagrams showing other examples of upper layer feature amounts.
[0094]
As illustrated in FIG. 11, the feature amount conversion unit 23 calculates, for example, an average value of four data of different phases on one line of a predetermined field, and uses the calculated average value as an upper layer feature amount. Output as.
[0095]
As shown in FIG. 12, the feature amount conversion unit 23 calculates, for example, an average value of four data having different phases adjacent to each other vertically and horizontally on two lines vertically adjacent to a predetermined field. The calculated average value is output as an upper layer feature quantity.
[0096]
When the feature quantity conversion unit 23 calculates an average value of four data of different phases and outputs an upper layer feature quantity, the number of upper layer feature quantity data is four times that of the original image signal data. 1
[0097]
13 to 17 are diagrams illustrating other examples of lower layer feature amounts.
[0098]
As illustrated in FIG. 13, the feature amount conversion unit 24 uses the upper data and the lower data of the target data, and the second data and the target data on the left side from the target data in the field # -1 of interest. Based on the second data on the right side, lower layer feature values are calculated. For example, if the attention data is p2, the upper data is p0, the second data is p1 on the left, the second data is p3 on the right, and the lower data is p4, it corresponds to the attention data p2. The lower layer feature amount y to be calculated is calculated by the equation (1).
y = p0 + p1 + 4 * p2 + p3 + p4 (1)
[0099]
Based on the upper and lower data of the target data, the second data on the left side from the target data, and the second data on the right side from the target data, the feature amount conversion unit 24 The lower layer feature amount is calculated. For example, if the target data is p2, the upper data is p0, the second data is p1 on the left, the second data is p3 on the right, and the lower data is p4, it corresponds to the target data p2. The lower layer feature amount y to be calculated is calculated by the equation (1).
[0100]
Since the lower layer feature amount described with reference to FIG. 13 is calculated based on four pieces of data around the target data or target data and having different phases, it is possible to reduce the influence of the fluctuation component of the subcarrier. Therefore, a motion vector with higher accuracy can be detected.
[0101]
As illustrated in FIG. 14, when the feature amount conversion unit 24 calculates the lower layer feature amount corresponding to the attention data of the field # -1 that is the attention field, the feature data and the field # that is the field before the attention field. -2 lines on the upper side and lower side of the line where the target data is located, the upper data and the lower data of the target data, and the second data and the target data on the left side from the target data Based on the second data on the right side, the lower layer feature quantity is calculated. For example, the attention data is p2, the upper data of the attention data in line 3 of the previous field # -2 above the line 4 where the attention data is located is p0, and the second data on the left side of the line 4 is the second data. p1, the second data on the right side of the line 4 is p3, the lower data of the attention data in the line 5 of the previous field # -2 below the line 4 where the attention data is located is p4, The lower layer feature quantity y corresponding to the attention data p2 is calculated by the equation (2).
y = p0 + p1 + 4 * p2 + p3 + p4 (2)
[0102]
When the feature quantity conversion unit 24 calculates the lower layer feature quantity corresponding to the target data of the field # 0 that is the corresponding field, the feature quantity conversion unit 24 is the line of the field # + 1 that is the next field of the target data and the field # 0. , The upper and lower data of the target data on the upper and lower lines of the target data, and the second data on the left side of the target data and the second data on the right side of the target data Based on the above, the lower layer feature quantity is calculated. For example, the target data is p2, the data above the target data in line 2 of the next field # + 1 above the line 3 where the target data is located is p0, and the second data on the left side of the line 3 is p1. P3 is the second data on the right side of line 3, and p4 is the data below the target data in line 4 of the next field # + 1 below the line 3 where the target data is located. The lower layer feature quantity y corresponding to p2 is calculated by the equation (2).
[0103]
As illustrated in FIG. 15, the feature amount conversion unit 24 includes the attention data, the upper data and the lower data of the attention data, the data adjacent to the left side of the attention data, and the attention of the field # -1 Based on the data adjacent to the right side of the data, the lower layer feature quantity corresponding to the data of interest is calculated. For example, the attention data is p2, the upper data of the attention data is p0, the data adjacent to the left side of the attention data is p1, the data adjacent to the right side of the attention data is p3, and the lower data of the attention data is When p4 is set, the lower layer feature quantity y corresponding to the attention data p2 is calculated by the equation (3).
y = p0 + p1 + 4 * p2 + p3 + p4 (3)
[0104]
The feature amount conversion unit 24 stores the target data, the upper data and the lower data of the target data, the data adjacent to the left side of the target data, and the data adjacent to the right side of the target data in the corresponding field # 0. Based on this, a lower layer feature amount corresponding to the target data is calculated. For example, the target data is p2, the upper data of the target data is p0, the data adjacent to the left side of the target data is p1, the data adjacent to the right side of the target data is p3, and the data below the target data is When p4 is set, the lower layer feature quantity y corresponding to the target data p2 is calculated by the equation (3).
[0105]
As illustrated in FIG. 16, the feature amount conversion unit 24 includes the attention data, the data adjacent to the left side of the attention data, the data adjacent to the right side of the attention data, and the left side of the attention data. Based on the second data and the second data on the right side, the lower layer feature quantity corresponding to the attention data is calculated. For example, the attention data is p2, the second data to the left of the attention data is p0, the data adjacent to the left of the attention data is p1, the data adjacent to the right of the attention data is p3, and the data to the right of the attention data is When the second data is p4, the lower layer feature quantity y corresponding to the attention data p2 is calculated by the equation (4).
y = p0 + p1 + 2 * p2 + 2 * p3 + p4 (4)
[0106]
The feature amount conversion unit 24 includes the target data of the target field # 0, the data adjacent to the left side of the target data and the data adjacent to the right side of the target data, and the second data and the target data on the left side from the target data. Based on the second data on the right side, the lower layer feature quantity corresponding to the target data is calculated. For example, the target data is p2, the second data on the left side of the target data is p0, the data adjacent to the left side of the target data is p1, the data adjacent to the right side of the target data is p3, and the right side of the target data Assuming that the second data is p4, the lower layer feature quantity y corresponding to the target data p2 is calculated by Expression (4).
[0107]
As shown in FIG. 17, the feature amount conversion unit 24 calculates the lower layer feature amount of the field # -1 that is the attention field, the attention data, the second data on the left side from the attention data, and the right side from the attention data. The second data, and the line of field # -2, which is the field before the attention field, and the attention data on the upper and lower lines where the attention data is located. Based on the first data and the upper data and the lower data of the second data on the right side from the data of interest, the lower layer feature quantity is calculated.
[0108]
For example, the attention data is p4, the second data is p3 on the left side, the second data is p5 on the right side, and the line 3 in the previous field # -2 above the line 4 where the attention data is located. The upper data of the attention data is p1, the upper data of p3 is p0, the upper data of p5 is p2, and in the line 5 of the previous field # -2 below the line 4 where the attention data is located. The lower data of the target data is p7, the lower data of p3 is p6, the lower data of p5 is p8, and the lower-layer feature quantity y corresponding to the target data p4 is given by equation (5) Calculated.
y = p0 + (-2) * p1 + p2 + (-2) * p3 + 4 * p4 + (-2) * p5 + p6 + (-2) * p7 + p8 (5)
[0109]
When the feature quantity conversion unit 24 calculates the lower layer feature quantity of the corresponding field, field # 0, the target data, the second data on the left side from the target data, the second data on the right side from the target data, and From the target data, the second data to the left of the target data, and the target data on the upper and lower lines of the line where the target data is located, and the line of field # + 1 that is the next field of the target field Based on the upper data and the lower data of the second data on the right side, lower layer feature values are calculated.
[0110]
For example, the target data is p4, the second data is p3 on the left, the second data is p5 on the right, and the target in line 2 of the next field # + 1 above the line 3 where the target data is located. The upper data of the data is p1, the upper data of p3 is p0, the upper data of p5 is p2, and the target in the line 4 of the next field # + 1 below the line 3 where the target data is located The lower data of the data is p7, the lower data of p3 is p6, the lower data of p5 is p8, and the lower layer feature quantity y corresponding to the target data p4 is calculated by equation (5). The
[0111]
The upper hierarchy feature quantity and the lower hierarchy feature quantity exemplified above are not normalized because computation is performed between the feature quantities. Of course, the upper layer feature quantity and the lower layer feature quantity may be normalized.
[0112]
Next, detection of lower layer vectors by class classification adaptation processing will be described.
[0113]
Here, the class classification process will be briefly described.
[0114]
Now, for example, in a lower layer feature amount, a class tap composed of 2 × 2 data is constituted by certain data of interest and three data adjacent thereto, and each data is represented by 1 bit (0 or 1). In this case, a 2 × 2 block of 4 data including the data of interest has 16 (= (2¹)^Four) Can be classified into patterns. Therefore, in this case, the attention data can be classified into 16 patterns, and such pattern division is a class classification process.
[0115]
Here, for example, about 8 bits are assigned to each data. In addition, when class taps are composed of 9 data of 3 × 3, if class classification processing is performed for such class taps, (2⁸)⁹It will be classified into a huge number of classes.
[0116]
Therefore, in the present embodiment, ADRC (Adaptive Dynamic Range Coding) processing is performed on the class tap, and thereby the number of classes is reduced by reducing the number of bits of data constituting the class tap. .
[0117]
In order to simplify the explanation, when considering a class tap composed of four data of a lower layer feature amount arranged on one line, the maximum value MAX and the minimum value MIN of the data value are detected in ADRC processing. Is done. Then, DR = MAX-MIN is set as the local dynamic range of the block constituted by the class taps, and the data value of the data constituting the class tap block is requantized to K bits based on the dynamic range DR. Is done.
[0118]
That is, the minimum value MIN is subtracted from each data value in the block, and the subtracted value is DR / 2.^KDivide by. Then, it is converted into a code (ADRC code) corresponding to the division value obtained as a result. Specifically, for example, when K = 2, the division value has a dynamic range DR of 4 (= 2²) It is determined which range is obtained by equally dividing, and the division value is the range of the lowest level, the range of the second level from the bottom, the range of the third level from the bottom, or the top In the case of belonging to the level range, for example, each bit is encoded into 2 bits such as 00B, 01B, 10B, or 11B (B represents a binary number). On the decoding side, the ADRC code 00B, 01B, 10B, or 11B is the center value L of the lowest level range obtained by dividing the dynamic range DR into four equal parts.₀₀, Center value L of the second level range from the bottom₀₁, Center value L of the third level range from the bottom_TenOr the center value L of the range of the highest level₁₁Decoding is performed by adding the minimum value MIN to the value.
[0119]
Here, such ADRC processing is called non-edge matching.
[0120]
The details of the ADRC processing are disclosed in, for example, Japanese Patent Application Laid-Open No. 3-53778 filed by the applicant of the present application.
[0121]
As described above, the number of classes can be reduced by performing ADRC processing for performing requantization with a smaller number of bits than the number of bits allocated to data constituting the class tap.
[0122]
In the present embodiment, the class classification process is performed based on the ADRC code. However, the class classification process includes, for example, DPCM (predictive coding), BTC (Block Truncation Coding), and VQ (vector quantum). ), DCT (Discrete Cosine Transform), Hadamard Transform, and the like.
[0123]
In the present embodiment, adaptive processing is executed for each class classified in this way. As adaptive processing, there are a method of performing a prediction calculation using a prediction coefficient learned in advance, and a method of learning a predicted value by a centroid method. Moreover, as a condition for performing learning, it is necessary to prepare a target teacher signal composed of an image signal and a motion vector.
[0124]
Next, an adaptive process for performing a prediction calculation using a prediction coefficient for each class generated in advance by learning using the teacher signal will be described. For example, it is assumed that a prediction tap is constituted by 25 taps of 5 × 5 lower layer feature amounts E0 to E24 and the motion evaluation value E ′ is predicted.
[0125]
The predicted value E ′ of the motion evaluation value is calculated by Expression (6).
[0126]
[Expression 1]

Ei indicates a lower layer feature quantity. wi represents a prediction coefficient.
[0127]
For example, when the above 1-bit ADRC is applied to nine data and classified into 512 classes, the motion evaluation value E ′ is predicted by the product-sum operation of the prediction coefficient generated for each class and the lower layer feature quantity. Is done.
[0128]
For example, it is possible to calculate two motion evaluation values E ′ that are orthogonal to each other and use the x and y components of the motion vector as the motion vector. Also, for example, n motion evaluation values E ′ are calculated, each indicating the size of n unit vectors having different directions, and each of the n motion evaluation values E ′ is represented by n units. A motion vector can be calculated by multiplying each of the vectors and adding the multiplied results.
[0129]
Further, not only the motion vector but also, for example, when a component video signal is predicted by the adaptive processing, an evaluation value E ′ corresponding to the Y signal and an evaluation value E ′ corresponding to the U signal are calculated based on Expression (6). , The evaluation value E ′ corresponding to the V signal can be calculated individually.
[0130]
Since the above-described prediction coefficient is generated by learning in advance, the learning will be described here.
[0131]
The example which produces | generates the prediction coefficient based on the model of Formula (6) by the least square method is shown.
[0132]
The least squares method is applied as follows. As a generalized example, Equation (7) is considered with X as input data, W as a prediction coefficient, and Y as a predicted value.
Observation equation: XW = Y (7)
[0133]
[Expression 2]

[0134]
Apply the least squares method to the data collected by the above observation equation. In the example of Expression (6), n is 25, and m is the number of learning data.
[0135]
Consider the residual equation (9) based on the observation equation (7).
[0136]
[Equation 3]

[0137]
From the residual equation of equation (9), the most probable value of each wi corresponds to the case where the condition for minimizing the value shown in equation (10) is satisfied.
[0138]
[Expression 4]

[0139]
That is, the condition of equation (11) may be considered.
[0140]
[Equation 5]

[0141]
What is necessary is just to calculate w1 thru | or wn which satisfy | fills n conditions based on i of Formula (11). Therefore, Equation (12) is obtained from the residual equation (9).
[0142]
[Formula 6]

[0143]
Equation (13) is obtained from Equation (11) and Equation (12).
[0144]
[Expression 7]

[0145]
From equation (10) and equation (13), the normal equation (14) is obtained.
[0146]
[Equation 8]

[0147]
In the normal equation of the equation (14), the same number of equations as the unknown number n can be established, so that the most probable value of the accuracy wi can be obtained. For example, equation (14) is solved using a sweeping method (Gauss-Jordan elimination method).
[0148]
As described above, the optimum prediction coefficient w is obtained for each class, and further, using the prediction coefficient w, a predicted value E ′ indicating a vector close to the motion vector of the teacher signal is obtained by Equation (6). It is an adaptive process.
[0149]
When a component video signal is predicted, it is necessary to obtain an optimal prediction coefficient w for each class for predicting the component video signal.
[0150]
The adaptive process is the same as the interpolation process using a so-called interpolation filter as long as only Expression (6) is seen, but the prediction coefficient w corresponding to the tap coefficient of the interpolation filter is determined using the teacher signal. Since it is obtained by learning, the original motion vector can be reproduced. From this, it can be said that the adaptive process is a process having a creative action.
[0151]
FIG. 18 is a block diagram showing another configuration of the lower layer motion vector detection unit 26 that detects a lower layer vector by the class classification adaptation process.
[0152]
The class classification unit 51 performs class classification based on the subcarrier phase information and the lower layer feature quantity, and supplies a class code indicating the classified class to the prediction tap extraction unit 52. For example, the class classification unit 51 performs class classification corresponding to the phase of the data of interest. Further, for example, the class classification unit 51 applies ADRC processing to a predetermined number of lower layer feature amounts corresponding to the data of interest to classify the class.
[0153]
The prediction tap extraction unit 52 is a predetermined number of lower layers at a predetermined position based on the class indicated by the class code, the lower layer vector corresponding to the previous field, and the upper layer vector supplied from the memory 55. The feature amount is extracted, and the extracted lower layer feature amount is supplied to the calculation unit 53 as a prediction tap.
[0154]
The computing unit 53 predicts the lower layer vector by executing, for example, the computation shown in Expression (6) based on the coefficient set including a predetermined number of prediction coefficients supplied from the prediction coefficient memory 54. The computing unit 53 supplies the predicted lower layer vector to the memory 55 and outputs it to the outside.
[0155]
The memory 55 stores the lower layer vector supplied from the calculation unit 53 and supplies the stored lower layer vector to the prediction tap extraction unit 52.
[0156]
As described above, the motion vector detection unit 2 can detect a highly accurate motion vector and supply the detected motion vector to the class tap extraction unit 3 and the calculation unit 4.
[0157]
Based on the subcarrier phase information supplied from the NTSC decoder 1 and the motion vector supplied from the motion vector detection unit 2, the class tap extraction unit 3 is a set of data for class classification, which is data of image signals. Then, a class tap composed of upper layer feature data and lower layer feature data is extracted.
[0158]
FIG. 19 is a block diagram illustrating a configuration of the calculation unit 4.
[0159]
The class classification unit 71-1 is supplied from the motion vector supplied from the motion vector detection unit 2, the upper layer feature quantity and the lower layer feature quantity, the class tap supplied from the class tap extraction unit 3, and the NTSC decoder 1. Based on the subcarrier phase information, class classification is performed, and a class code indicating the result of class classification is supplied to the prediction tap extraction unit 72-1 and the coefficient memory 73-1.
[0160]
For example, the class classification unit 71-1 performs class classification based on the subcarrier phase information supplied from the NTSC decoder 1. In addition, the class classification unit 71-1 performs class classification based on the phase of data specified by the motion vector with reference to the data of interest indicated by the subcarrier phase information supplied from the NTSC decoder 1. Therefore, the number of classes to be classified is a power of 4.
[0161]
Further, the class classification unit 71-1 performs class classification based on the upper layer feature quantity and the lower layer feature quantity supplied from the motion vector detection unit 2. That is, for example, the class classification unit 71-1 applies ADRC processing to the upper layer feature quantity supplied from the motion vector detection unit 2, and quantizes the upper layer feature quantity into a class code having a predetermined number of bits. The class classification unit 71-1 applies ADRC processing to the lower layer feature quantity supplied from the motion vector detection unit 2, and quantizes the lower layer feature quantity into a class code having a predetermined number of bits. When quantizing by threshold determination, quantization by threshold determination is equivalent to class-coding the reliability of a vector.
[0162]
FIG. 20 is a diagram illustrating an example of class taps and prediction taps corresponding to upper layer feature values.
[0163]
As shown in FIG. 20A, when the data of interest has a phase indicated by a white circle, for example, the upper layer feature quantity is composed only of data in phase with the data of interest indicated by a white circle as shown in FIG. 20B. . At this time, for example, the class tap corresponding to the upper layer feature amount is the data p5, the data p1 that is two data above the data of interest p5, the data p2 that is one data above the data of interest p5, and the data p2 that is two data left. One data point above the attention data p5, two right data p3, four data left data p4 of the attention data p5, four data right data p6 of the attention data p5, and one data data below the attention data p5. The data p7 on the left side and the data p8 that is one lower side of the attention data p5, the data p8 on the right side of the two, and the data p9 that is two lower than the attention data p5
[0164]
For example, as shown in FIG. 20C, the upper layer feature amount is composed of an average value of a total of four data indicated by triangles, two vertically and two horizontally. The position of the upper layer feature amount composed of the average value is shifted by half the line width from the original image signal data position, and the upper layer feature amount composed of the average value. The position in the left-right direction is deviated from the original image signal data position by a half of the data interval.
[0165]
When the target data is p11, the class tap corresponding to the upper layer feature is, for example, two lines half higher than the target data p11 and the data A1 on the left side by half of the original data interval. With respect to the data of interest p11, two lines are on the upper half and the data A2 is on the right side by one and half of the original data interval, and the lines are one and half on the upper side, and the original data interval is 1 Data A3 on the left side by one and a half, the line is one and a half above, and the data A4 on the right side by half the original data interval, and the line is one and a half above, two and a half of the original data interval Data A5 on the right side, the line is half the upper side, and the data A6 on the left side by half of the original data interval, and the line is half the upper side, the data on the right side by one and half of the original data interval. A7, the line is half the lower side and the original data interval is one half and the left side data A8, the line is half the lower side and the half point of the original data interval is the right side data A9, the line is Data A10 that is half the lower side and half the right side of the original data, and one line that is one half lower side of the original data interval, and one data A11 that is the left side by half the interval of the original data One half of the original data interval and half the right side data A12, two lines below the half of the original data interval, and half of the original data interval one half and the left side data A13. Data A14 that is two and a half lower and half the original data interval on the right side, data A14 that is two and a half lower on the original data interval, and data A15 that is two and half the original data interval on the left side Is done.
[0166]
FIG. 21 is a diagram illustrating an example of class taps and prediction taps corresponding to lower layer feature quantities.
[0167]
The class tap corresponding to the current field # -1 is 3 × 3 data centered on the data of the lower layer feature amount corresponding to the attention data, that is, the data of the lower layer feature amount corresponding to the attention data, the attention Data above the lower layer feature value data corresponding to the data, upper data of the lower layer feature value data corresponding to the target data, and above the left side data and data of the lower layer feature amount corresponding to the target data The data on the right side, the data on the left side of the data of the lower layer feature amount corresponding to the attention data, the data on the right side of the data of the lower layer feature amount corresponding to the attention data, the data of the lower layer feature amount corresponding to the attention data Is composed of data on the left side and data on the lower side of the lower layer feature quantity corresponding to the data of interest and on the right side.
[0168]
The class tap of the corresponding field # 0 is 3 × 3 data centered on the data of the lower layer feature amount specified by the motion vector from the attention data, that is, the lower layer feature specified by the motion vector from the attention data. Data, the upper data of the lower layer feature data specified by the motion vector from the attention data, the upper data of the lower hierarchy feature data specified by the motion vector from the attention data, the left data, From the data of the lower layer feature amount specified by the motion vector from the data, the data on the right side, the data of the left side of the data of the lower layer feature amount specified by the motion vector from the attention data, the motion vector from the attention data The data on the right side of the specified lower layer feature data, the lower layer feature data specified by the motion vector from the data of interest A lower data, the left data, a lower Hierarchical feature amount of data specified by the motion vector from the target data, and a right data.
[0169]
The class classification unit 71-1 classifies into a spatial class based on the attention data of the image signal and the peripheral data around the attention data. That is, for example, the class classification unit 71-1 applies ADRC processing to attention data and peripheral data of an image signal, and quantizes the attention data and peripheral data into a class code having a predetermined number of bits.
[0170]
The class classification unit 71-1 performs class classification based on the target data of the corresponding field indicated by the motion vector from the target data of the target field and the peripheral data around the target data. That is, for example, the class classification unit 71-1 applies the ADRC process to target data of the corresponding field indicated by the motion vector from the target data of the target field, and peripheral data around the target data, and the target data, and The peripheral data is quantized into a class code having a predetermined number of bits.
[0171]
The class classification unit 71-1 may perform class classification based on the amount of motion.
[0172]
The class classification unit 71-1 supplies the prediction tap extraction unit 72-1 with a dynamic range that is the difference between the maximum value and the minimum value of the data values of the image data of interest and the peripheral data belonging to the predetermined block.
[0173]
Note that the class classification unit 71-1 may calculate the dynamic range based on the class tap of the lower layer feature amount of the field of interest, which is illustrated in FIG.
[0174]
In addition, the class classification unit 71-1 may perform class classification based on the upper layer feature amount included in the class tap.
[0175]
The prediction tap extraction unit 72-1 is based on the motion vector supplied from the motion vector detection unit 2, feature quantities such as residuals, and the class code and dynamic range supplied from the class classification unit 71-1. Predetermined data is extracted from the signal data, the upper layer feature quantity, and the lower layer feature quantity, and the extracted data is supplied as a prediction tap to the prediction calculation unit 74-1.
[0176]
When the dynamic range supplied from the class classification unit 71-1 is large, the prediction tap extraction unit 72-1 includes the edge of the image in the vicinity, so that the prediction tap is configured with the nearby data, so spatially Extract prediction taps with small spread. On the other hand, when the dynamic range supplied from the class classification unit 71-1 is small, the prediction tap extraction unit 72-1 is composed of a wide range of data because the image is a flat portion. Extract taps.
[0177]
The prediction tap extraction unit 72-1 extracts a prediction tap having a large spatial spread because there is a high possibility that the motion vector is erroneous when the residual that is the feature amount supplied from the motion vector detection unit 2 is large. To do. On the other hand, when the residual, which is the feature amount supplied from the motion vector detection unit 2, is small, the prediction tap extraction unit 72-1 has a small possibility that the motion vector is erroneous. To extract.
[0178]
In addition, you may make it the prediction tap extraction part 72-1 extract a prediction tap based only on a class code.
[0179]
As shown in FIG. 20A, when the data of interest has a phase indicated by a white circle, for example, the upper layer feature quantity is composed only of data in phase with the data of interest indicated by a white circle as shown in FIG. 20B. . At this time, for example, the prediction tap corresponding to the upper layer feature amount is the attention data p5, the data p1 that is two data above the attention data p5, the data p2 that is one data above the attention data p5, and the data p2 that is two data left. One data point above the attention data p5, two right data p3, four data left data p4 of the attention data p5, four data right data p6 of the attention data p5, and one data data below the attention data p5. The data p7 on the left side and the data p8 that is one lower side of the data of interest p5, the data p8 on the right side of the two, and the data p9 that is two lower than the data of interest p5.
[0180]
For example, as shown in FIG. 20C, the upper layer feature amount is composed of an average value of a total of four data indicated by triangles, two vertically and two horizontally. The upper and lower positions of the upper layer feature quantity constituted by the average value are shifted by half the line width from the original image signal data position. In addition, the left and right positions of the upper layer feature quantity composed of the average value are shifted by half the data interval from the original image signal data position.
[0181]
At this time, if the data of interest is p11, the prediction tap corresponding to the upper layer feature amount is, for example, two lines above the data of interest p11 and half left of the original data interval. The data A1 and the target data p11 are two lines and half above the original data, and the data A2 is one half and one half above the original data interval, and the lines are one and half above the original data. Data A3 on the left side by one and half of the interval, one line and one half above the line, and data A4 on the right side by one half of the original data, and one line and one half above the line, Data A5 on the right side by two and half, line is half-upper side, data A6 on the left side by half of the original data interval, line half-upper side, right by one and half of the original data interval Data A7, the line is half lower, and the data A8 is one half and one half left of the original data, and the line is half lower, and the right data A9 is half the original data interval, The line is half lower and the data A10 is two and half the right side of the original data, and the line is one and half half lower, and the data A11 is the left side of the half of the original data interval. Data A12 that is one and a half below the original data interval and half the right side data A12, two lines half and a half below the original data interval, and data A13 that is one half and half the original data interval to the left side, Data A14 that is two and a half below the line and half the original data interval to the right, data A14 that is two and a half below the line, and the data A15 that is two and a half below the original data interval Composed of
[0182]
As shown in FIG. 21, the prediction tap corresponding to the current field # -1 in the lower layer feature quantity is composed of 5 × 5 data centered on the lower layer feature quantity data corresponding to the data of interest. The
[0183]
The prediction tap of the corresponding field # 0 in the lower layer feature quantity is composed of 5 × 5 data centering on the data of the lower layer feature quantity designated by the motion vector from the data of interest.
[0184]
The coefficient memory 73-1 selects and selects a coefficient set corresponding to the class specified by the class code from the coefficient sets stored in advance based on the class code supplied from the class classification unit 71-1. The class code is supplied to the prediction calculation unit 74-1.
[0185]
Based on the prediction tap supplied from the prediction tap extraction unit 72-1 and the coefficient set supplied from the coefficient memory 73-1, the prediction calculation unit 74-1 performs, for example, a calculation corresponding to Expression (6). , Y signal is predicted, and the predicted Y signal is output.
[0186]
The class classification unit 71-2 is supplied from the NTSC decoder 1, the motion vector supplied from the motion vector detection unit 2, the upper layer feature quantity, the lower layer feature quantity, the class tap supplied from the class tap extraction unit 3, and the NTSC decoder 1. Based on the subcarrier phase information, class classification is performed, and a class code indicating the result of class classification is supplied to the prediction tap extraction unit 72-2 and the coefficient memory 73-2.
[0187]
For example, the class classification unit 71-2 performs class classification based on the subcarrier phase information supplied from the NTSC decoder 1. The class classification unit 71-2 performs class classification based on the phase of data specified by the motion vector with reference to the data of interest indicated by the subcarrier phase information supplied from the NTSC decoder 1.
[0188]
The class classification unit 71-2 performs class classification based on the upper layer feature amount and the lower layer feature amount supplied from the motion vector detection unit 2. That is, for example, the class classification unit 71-2 applies ADRC processing to the upper layer feature amount supplied from the motion vector detection unit 2, and quantizes the upper layer feature amount into a class code having a predetermined number of bits. The class classification unit 71-2 applies ADRC processing to the lower layer feature quantity supplied from the motion vector detection unit 2, and quantizes the lower layer feature quantity into a class code of a predetermined number of bits.
[0189]
The class classification unit 71-2 performs class classification into a space class based on the attention data of the image signal and the peripheral data around the attention data. That is, for example, the class classification unit 71-2 applies ADRC processing to attention data and peripheral data of an image signal, and quantizes the attention data and peripheral data into a class code having a predetermined number of bits.
[0190]
The class classification unit 71-2 performs class classification based on the target data of the corresponding field indicated by the motion vector from the target data of the target field and the peripheral data around the target data. That is, for example, the class classification unit 71-2 applies the ADRC process to the target data of the corresponding field indicated by the motion vector from the target data of the target field and the peripheral data around the target data, and the target data, and The peripheral data is quantized into a class code having a predetermined number of bits.
[0191]
The class classification unit 71-2 may perform class classification based on the amount of movement.
[0192]
The class classification unit 71-2 supplies the dynamic range that is the difference between the maximum value and the minimum value of the data values of the image data of interest and the peripheral data of the predetermined block to the prediction tap extraction unit 72-2.
[0193]
Note that the class classification unit 71-2 may calculate the dynamic range based on the class tap of the attention field lower layer feature quantity as shown in FIG.
[0194]
Further, the class classification unit 71-2 may perform class classification based on the upper layer feature amount included in the class tap.
[0195]
Based on the motion vector supplied from the motion vector detection unit 2 and the feature quantity such as a residual, and the class code and dynamic range supplied from the class classification unit 71-2, the prediction tap extraction unit 72-2 Predetermined data is extracted from the signal data, the upper layer feature quantity, and the lower layer feature quantity, and the extracted data is supplied to the prediction calculation unit 74-2 as a prediction tap.
[0196]
In addition, you may make it the prediction tap extraction part 72-2 extract a prediction tap based only on a class code.
[0197]
The coefficient memory 73-2 selects and selects a coefficient set corresponding to the class specified by the class code from the coefficient sets stored in advance based on the class code supplied from the class classification unit 71-2. The class code is supplied to the prediction calculation unit 74-2.
[0198]
Based on the prediction tap supplied from the prediction tap extraction unit 72-2 and the coefficient set supplied from the coefficient memory 73-2, the prediction calculation unit 74-2 performs, for example, a calculation corresponding to Expression (6). , Predict the U signal and output the predicted U signal.
[0199]
The class classification unit 71-3 is supplied from the motion vector supplied from the motion vector detection unit 2, the upper layer feature quantity and the lower layer feature quantity, the class tap supplied from the class tap extraction unit 3, and the NTSC decoder 1. Based on the subcarrier phase information, class classification is performed, and a class code indicating the result of the class classification is supplied to the prediction tap extraction unit 72-3 and the coefficient memory 73-3.
[0200]
For example, the class classification unit 71-3 performs class classification based on the subcarrier phase information supplied from the NTSC decoder 1. In addition, the class classification unit 71-3 performs class classification based on the phase of data specified by the motion vector with reference to the data of interest indicated by the subcarrier phase information supplied from the NTSC decoder 1.
[0201]
The class classification unit 71-3 performs class classification based on the upper layer feature amount and the lower layer feature amount supplied from the motion vector detection unit 2. That is, for example, the class classification unit 71-3 applies ADRC processing to the upper layer feature quantity supplied from the motion vector detection unit 2, and quantizes the upper layer feature quantity into a class code of a predetermined number of bits. The class classification unit 71-3 applies ADRC processing to the lower layer feature quantity supplied from the motion vector detection unit 2, and quantizes the lower layer feature quantity into a class code of a predetermined number of bits.
[0202]
The class classification unit 71-3 classifies the data into a spatial class based on the attention data of the image signal and the peripheral data around the attention data. That is, for example, the class classification unit 71-3 applies ADRC processing to attention data and peripheral data of an image signal, and quantizes the attention data and peripheral data into a class code having a predetermined number of bits.
[0203]
The class classification unit 71-3 performs class classification based on the target data of the corresponding field indicated by the motion vector from the target data of the target field and the peripheral data around the target data. That is, for example, the class classification unit 71-3 applies ADRC processing to the target data of the corresponding field indicated by the motion vector from the target data of the target field, and the peripheral data around the target data, and the target data, and The peripheral data is quantized into a class code having a predetermined number of bits.
[0204]
The class classification unit 71-3 may perform class classification based on the motion amount.
[0205]
The class classification unit 71-3 supplies the dynamic range that is the difference between the maximum value and the minimum value of the data values of the image data of interest and the peripheral data of the predetermined block to the prediction tap extraction unit 72-3.
[0206]
Note that the class classification unit 71-3 may calculate the dynamic range based on the class tap of the attention field lower layer feature quantity as shown in FIG.
[0207]
In addition, the class classification unit 71-3 may perform class classification based on the upper layer feature amount included in the class tap.
[0208]
Based on the motion vector supplied from the motion vector detection unit 2 and the feature quantity such as a residual, and the class code and dynamic range supplied from the class classification unit 71-3, the prediction tap extraction unit 72-3 Predetermined data is extracted from the signal data, the upper layer feature amount, and the lower layer feature amount, and the extracted data is supplied to the prediction calculation unit 74-3 as a prediction tap.
[0209]
Note that the prediction tap extraction unit 72-3 may extract prediction taps based only on the class code.
[0210]
The coefficient memory 73-3 selects and selects a coefficient set corresponding to the class specified by the class code from the coefficient sets stored in advance based on the class code supplied from the class classification unit 71-3. The class code is supplied to the prediction calculation unit 74-3.
[0211]
Based on the prediction tap supplied from the prediction tap extraction unit 72-3 and the coefficient set supplied from the coefficient memory 73-3, the prediction calculation unit 74-3 performs, for example, a calculation corresponding to Expression (6). , V signal is predicted, and the predicted V signal is output.
[0212]
As described above, the calculation unit 4 predicts the Y signal, the U signal, and the V signal, which are component video signals with higher accuracy than before, by the class classification adaptive processing, and predicts the predicted Y signal, U A signal and a V signal can be output.
[0213]
Hereinafter, when it is not necessary to individually distinguish the class classification units 71-1 to 71-3, they are simply referred to as a class classification unit 71.
[0214]
Hereinafter, when there is no need to individually distinguish the prediction tap extraction units 72-1 to 72-3, they are simply referred to as the prediction tap extraction unit 72.
[0215]
Hereinafter, the coefficient memories 73-1 to 73-3 are simply referred to as the coefficient memory 73 when it is not necessary to distinguish them individually.
[0216]
Hereinafter, when it is not necessary to individually distinguish the prediction calculation units 74-1 to 74-3, they are simply referred to as the prediction calculation unit 74.
[0217]
FIG. 22 is a block diagram showing another configuration of the calculation unit 4. The same parts as those shown in FIG. 19 are denoted by the same reference numerals, and the description thereof is omitted.
[0218]
The selector 81 selects one class code and dynamic range from among the three class codes and dynamic range individually supplied from the class classification units 71-1 to 71-3, and predicts the selected class code and dynamic range. This is supplied to the tap extraction unit 82.
[0219]
The selector 81 selects one coefficient set from the three coefficient sets individually supplied from the coefficient memories 73-1 to 73-3, and supplies the selected coefficient set to the prediction calculation unit 83.
[0220]
The prediction tap extraction unit 82 extracts prediction taps based on the motion vector supplied from the motion vector detection unit 2, the upper layer feature quantity, and the lower layer feature quantity, and the class code and dynamic range supplied from the selector 81. Then, the extracted prediction tap is supplied to the prediction calculation unit 83.
[0221]
When the class code and the dynamic range output from the class classification unit 71-1 are supplied via the selector 81, the prediction tap extraction unit 82 extracts the prediction tap corresponding to the Y signal and extracts the extracted prediction tap. Is supplied to the prediction calculation unit 83. When the class code and the dynamic range output from the class classification unit 71-2 are supplied via the selector 81, the prediction tap extraction unit 82 extracts a prediction tap corresponding to the U signal, and extracts the extracted prediction tap. Is supplied to the prediction calculation unit 83. When the class code and the dynamic range output from the class classification unit 71-3 are supplied via the selector 81, the prediction tap extraction unit 82 extracts a prediction tap corresponding to the V signal, and extracts the extracted prediction tap. Is supplied to the prediction calculation unit 83.
[0222]
When the prediction tap corresponding to the Y signal is supplied from the prediction tap extraction unit 82, the prediction calculation unit 83 predicts the Y signal based on the coefficient set supplied from the coefficient memory 73-1 via the selector 81. Then, the predicted Y signal is output. When the prediction tap corresponding to the U signal is supplied from the prediction tap extraction unit 82, the prediction calculation unit 83 predicts the U signal based on the coefficient set supplied from the coefficient memory 73-2 via the selector 81. Then, the predicted U signal is output. When the prediction tap corresponding to the V signal is supplied from the prediction tap extraction unit 82, the prediction calculation unit 83 predicts the V signal based on the coefficient set supplied from the coefficient memory 73-3 via the selector 81. Then, the predicted V signal is output.
[0223]
FIG. 23 is a block diagram illustrating another configuration of the calculation unit 4. The same parts as those shown in FIG. 19 are denoted by the same reference numerals, and the description thereof is omitted.
[0224]
The class classification unit 91 includes the motion vector supplied from the motion vector detection unit 2, the upper layer feature quantity, the lower layer feature quantity, the class tap supplied from the class tap extraction unit 3, and the sub-class supplied from the NTSC decoder 1. Based on the carrier phase information, class classification is performed, and class codes indicating the result of class classification are supplied to the prediction tap extraction units 72-1 to 72-3 and the coefficient memories 73-1 to 73-3.
[0225]
For example, the prediction tap extraction units 72-1 to 72-3 extract different prediction taps based on different threshold values.
[0226]
FIG. 24 is a block diagram illustrating another configuration of the calculation unit 4. The same parts as those shown in FIG. 19 are denoted by the same reference numerals, and the description thereof is omitted. The same parts as those shown in FIG. 22 are denoted by the same reference numerals, and the description thereof is omitted.
[0227]
The class classification unit 91 includes the motion vector supplied from the motion vector detection unit 2, the upper layer feature quantity, the lower layer feature quantity, the class tap supplied from the class tap extraction unit 3, and the sub-class supplied from the NTSC decoder 1. Based on the carrier phase information, class classification is performed, and a class code indicating the result of class classification is supplied to the prediction tap extraction unit 82 and the coefficient memories 73-1 to 73-3.
[0228]
The selector 101 selects one coefficient set from the three coefficient sets individually supplied from the coefficient memories 73-1 to 73-3, and supplies the selected coefficient set to the prediction calculation unit 83.
[0229]
FIG. 25 is a block diagram showing details of an example of the configuration of the embodiment of the image processing apparatus according to the present invention. The NTSC composite signal input to the image processing apparatus is supplied to the subcarrier phase information detection circuit 121, the field memory 122-1, the motion vector detection circuit 123, and the delay circuit 124-1.
[0230]
The subcarrier phase information detection circuit 121 detects the phase from the input NTSC composite signal, and supplies the detected phase information to the motion vector detection circuit 123, the subcarrier phase information detection circuit 125, and the class classification unit 127. .
[0231]
The field memory 122-1 stores the input NTSC composite signal, delays it for a period corresponding to one field, and stores the stored NTSC composite signal in the field memory 122-2, motion vector detection circuit 123, and delay circuit. 124-2.
[0232]
The field memory 122-2 stores the NTSC composite signal supplied from the field memory 122-1, delays it for a period corresponding to one field, and moves the stored NTSC composite signal to the motion vector detection circuit 123 and the delay circuit. 124-3.
[0233]
Therefore, the motion vector detection circuit 123 is simultaneously supplied with the signal of the reference field, the signal of the next (future) field of the reference field, and the signal of the previous (previous) field of the reference field. . The motion vector detection circuit 123 corresponds to the reference field and the previous field on the basis of the signal of the reference field, the signal of the next field of the reference field, and the signal of the field before the reference field. And a vector (t + 1) corresponding to the reference field and the next field are generated.
[0234]
The motion vector detection circuit 123 supplies a vector (t−1) corresponding to the reference field and the previous field, and a vector (t + 1) corresponding to the reference field and the next field to the subcarrier phase information detection circuit 125. Supply.
[0235]
The motion vector detection circuit 123 supplies the reference field and the vector (t−1) corresponding to the previous field to the region extraction circuit 126-3 and the region extraction circuit 128-3, and the reference field. And a vector (t + 1) corresponding to the next field is supplied to the region extraction circuit 126-1 and the region extraction circuit 128-1.
[0236]
The motion vector detection circuit 123 supplies the feature amount calculated by the detection processing of the vector (t−1) and the vector (t + 1) to the class classification unit 127.
[0237]
The delay circuit 124-1 delays the signal of the next field of the reference field in accordance with the processing time of the motion vector detection circuit 123, and extracts the region of the signal of the next field of the delayed reference field Supply to the circuit 126-1.
[0238]
The delay circuit 124-2 delays the reference field signal in accordance with the processing time of the motion vector detection circuit 123, and supplies the delayed reference field signal to the region extraction circuit 126-1.
[0239]
The delay circuit 124-3 delays the signal in the field preceding the reference field in accordance with the processing time of the motion vector detection circuit 123, and extracts the region signal in the field preceding the delayed reference field. Supply to the circuit 126-1.
[0240]
The subcarrier phase information detection circuit 125 receives the phase information supplied from the subcarrier phase information detection circuit 121 and the reference field supplied from the motion vector detection circuit 123 and a vector (t−1) corresponding to the previous field. ) And the vector (t + 1) corresponding to the reference field and the next field, phase information corresponding to the previous field and phase information corresponding to the next field are generated. The subcarrier phase information detection circuit 125 supplies the phase information corresponding to the previous field and the phase information corresponding to the next field to the class classification unit 127.
[0241]
Based on the reference field and the vector (t + 1) corresponding to the next field supplied from the motion vector detection circuit 123, the region extraction circuit 126-1 includes the reference and the reference supplied from the delay circuit 124-1. Predetermined data included in the signal of the field next to that field is cut out, and the cut out data is supplied to the class classification unit 127.
[0242]
The region extraction circuit 126-2 is supplied from the delay circuit 124-2 based on the vector (t-1) corresponding to the reference field and the previous field supplied from the motion vector detection circuit 123. The predetermined data included in the signal of the reference field is cut out, and the cut out data is supplied to the class classification unit 127.
[0243]
The area cutout circuit 126-3 cuts out predetermined data included in the signal of the field before the reference field supplied from the delay circuit 124-3, and supplies the cut out data to the class classification unit 127. .
[0244]
The class classification unit 127 corresponds to the phase information supplied from the subcarrier phase information detection circuit 121, the feature amount supplied from the motion vector detection circuit 123, and the previous field supplied from the subcarrier phase information detection circuit 125. Phase information and phase information corresponding to the next field, data supplied from the signal of the next field of the reference field supplied from the area extraction circuit 126-1, and supplied from the area extraction circuit 126-2 Based on the data extracted from the signal of the reference field and the data extracted from the signal of the field preceding the reference field supplied from the region extraction circuit 126-3, the classification is performed. The class code indicating the result of class classification is divided into an area extraction circuit 128-1, an area extraction circuit 128-2, and an area extraction time. 128-3, coefficient memory 129, and supplies the prediction unit 130.
[0245]
For example, the class classification unit 127 classifies into four classes based on the phase information corresponding to the previous field supplied from the phase information detection circuit 125. For example, the class classification unit 127 performs classification into four classes based on the phase information corresponding to the reference field supplied from the phase information detection circuit 121. For example, the class classification unit 127 classifies into four classes based on the phase information corresponding to the next field supplied from the phase information detection circuit 125.
[0246]
For example, the class classification unit 127 extracts data extracted from the signal of the field next to the reference field, data extracted from the signal of the reference field, and the signal of the field before the reference field. ADRC processing is applied to the five data that are the data, and the data is classified into 32 classes.
[0247]
For example, the class classification unit 127 performs comparison and determination with a predetermined threshold based on the feature amount supplied from the motion vector detection circuit 123, classifies the next field of the reference field into two classes, The field before the field is classified into two classes.
[0248]
For example, the class classification unit 127 multiplies the above class classifications, and finally classifies them into 8192 classes.
[0249]
The class classification unit 127 includes the phase information supplied from the subcarrier phase information detection circuit 121, the feature amount supplied from the motion vector detection circuit 123, the phase information corresponding to the field before the subcarrier phase information detection circuit 125, and the next The phase information corresponding to the field of the field, the data supplied from the signal of the field next to the reference field supplied from the area extraction circuit 126-1, and the reference supplied from the area extraction circuit 126-2 A structure for designating the structure of the prediction tap based on the data cut out from the signal of the field to be used and the data cut out from the signal of the field preceding the reference field supplied from the area cut-out circuit 126-3 Variable information is generated, and the generated structure variable information is converted into an area extraction circuit 128-1, an area extraction circuit 128-2, and And supplies to the area cut-out circuit 128-3.
[0250]
The region extraction circuit 128-1 is supplied from the motion vector detection circuit 123, the vector (t + 1) corresponding to the reference field and the next field, and the class code and structure variable information supplied from the class classification unit 127. Based on the above, the predetermined data included in the signal of the field next to the reference field supplied from the delay circuit 124-1 is cut out, and the cut out data is supplied to the prediction unit 130.
[0251]
Based on the class code and the structure variable information supplied from the class classification unit 127, the region extraction circuit 128-2 outputs predetermined data included in the signal of the reference field supplied from the delay circuit 124-2. Cut out and supply the cut out data to the prediction unit 130.
[0252]
The region extraction circuit 128-3 includes the vector (t-1) corresponding to the reference field and the previous field supplied from the motion vector detection circuit 123, and the class code and structure supplied from the class classification unit 127. Based on the variable information, the predetermined data included in the signal of the field before the reference field supplied from the delay circuit 124-3 is cut out, and the cut-out data is supplied to the prediction unit 130.
[0253]
The coefficient memory 129 supplies a predetermined coefficient set corresponding to the classified class to the prediction unit 130 based on the prediction mode setting signal and the class code supplied from the outside.
[0254]
The prediction unit 130 is cut out from the class code supplied from the class classification unit 127, the coefficient set supplied from the coefficient memory 129, and the signal of the field next to the reference field supplied from the region extraction circuit 128-1. Data, the data extracted from the signal of the reference field supplied from the region extraction circuit 128-2, and the signal of the field before the reference field supplied from the region extraction circuit 128-3 For example, a component video signal composed of a Y signal, a U signal, and a V signal is predicted on the basis of the data extracted from.
[0255]
As described above, the image processing apparatus according to the present invention can generate a more accurate component video signal corresponding to the input composite signal.
[0256]
FIG. 26 is a flowchart for explaining component video signal generation processing by the image processing apparatus according to the present invention.
[0257]
In step S <b> 1, the motion vector detection unit 2 detects a motion vector based on the image signal and subcarrier phase information supplied from the NTSC decoder 1. Details of the motion vector detection process will be described later.
[0258]
In step S2, the class tap extraction unit 3 uses the subcarrier phase information supplied from the NTSC decoder 1 and the motion vector supplied from the motion vector detection unit 2 to generate image signal data and higher layer feature value data. , And class taps composed of lower layer feature data.
[0259]
In step S <b> 3, the calculation unit 4 includes the image signal and subcarrier phase information supplied from the NTSC decoder 1, the motion vector supplied from the motion vector detection unit 2, upper layer feature data, and lower layer feature data. And class classification based on the class tap supplied from the class tap extraction unit 3. For example, the class classification units 71-1 to 71-3 of the calculation unit 4 shown in FIG. 19 include the image signal and subcarrier phase information supplied from the NTSC decoder 1, the motion vector supplied from the motion vector detection unit 2, Class classification is performed based on the data of the upper layer feature value, the data of the lower layer feature value, and the class tap supplied from the class tap extraction unit 3.
[0260]
In step S <b> 4, based on the classified class and the motion vector supplied from the motion vector detection unit 2, the calculation unit 4 calculates the image signal data, the upper layer feature amount data, and the lower layer feature amount data. Extract a prediction tap. For example, the prediction tap extraction units 72-1 to 72-3 shown in FIG. 19 are based on the class code indicating the classified class, the dynamic range, and the motion vector supplied from the motion vector detection unit 2, and A prediction tap consisting of data, upper layer feature data, and lower layer feature data is extracted.
[0261]
In step S5, the calculation unit 4 predicts the component video signal based on the extracted prediction tap and the coefficient set stored in advance, and the process ends. For example, the prediction calculation units 74-1 to 74-3 of the calculation unit 4 illustrated in FIG. 19 are based on the prediction taps and the coefficient sets supplied from the coefficient memories 73-1 to 73-3, respectively. Predict signal and V signal.
[0262]
FIG. 27 is a flowchart for explaining a motion vector detection process by the motion vector detection unit 2 corresponding to step S1. In step S11, the motion vector detection unit 2 acquires the subcarrier phase of the data of interest based on the subcarrier phase information supplied from the NTSC decoder 1 via the NTSC decoder 1.
[0263]
In step S12, the feature amount conversion unit 23 calculates the upper layer feature amount for the field of interest and the corresponding field based on the image signal and the acquired subcarrier phase, and calculates the calculated feature amount as the upper layer feature. Output as a quantity.
[0264]
In step S <b> 13, the upper layer motion vector detection unit 25 detects an upper layer vector, which is an approximate vector that approximates a motion vector, based on the upper layer feature amount supplied from the feature amount conversion unit 23. Details of the process of detecting the upper layer vector will be described later.
[0265]
In step S14, the feature amount conversion unit 24 calculates a lower layer feature amount for the field of interest and the corresponding field based on the phase of the image signal and the acquired subcarrier, and calculates the calculated feature amount as a lower layer feature. Output as a quantity.
[0266]
In step S15, the lower layer motion vector detection unit 26 calculates the lower layer vector based on the lower layer feature amount supplied from the feature amount conversion unit 24 and the upper layer vector supplied from the upper layer motion vector detection unit 25. To detect. Details of the lower layer vector detection process will be described later.
[0267]
In step S16, the lower layer motion vector detection unit 26 outputs a motion vector, and the process ends. That is, the lower layer motion vector detection unit 26 outputs the detected lower layer motion vector as a motion vector.
[0268]
FIG. 28 is a flowchart for explaining the details of the upper layer vector detection process by the upper layer motion vector detection unit 25 corresponding to step S13. In step S <b> 21, the vector generation unit 31 of the upper layer motion vector detection unit 25 generates a predetermined vector based on the parameter A and supplies the predetermined vector to the correlation value calculation unit 32 and the determination unit 33. In step S <b> 22, the correlation value calculation unit 32 centers on the upper layer feature quantity corresponding to the attention data of the field of interest, the block composed of a predetermined number of upper layer feature quantities, and the corresponding field from the vector generation unit 31. A correlation value with a block composed of a predetermined number of upper layer feature quantities is calculated with the position specified by the supplied vector as the center. For example, the correlation value calculation unit 32 calculates, as the correlation value, the sum of absolute values of differences between the upper layer feature quantity belonging to the block of the field of interest and the upper layer feature quantity belonging to the block of the corresponding field.
[0269]
In step S <b> 23, the determination unit 33 stores the correlation value supplied from the correlation value calculation unit 32 in association with the vector supplied from the vector generation unit 31.
[0270]
In step S24, the upper layer motion vector detection unit 25 determines whether or not the correlation value in the predetermined range specified by the parameter A has been calculated, and it is determined that the correlation value in the predetermined range has not been calculated. If it is, the process returns to step S21, and the correlation value calculation process is repeated corresponding to the next vector.
[0271]
In step S24, when it is determined that the correlation value in the predetermined range specified by the data included in the parameter A is calculated, the process proceeds to step S25, and the determination unit 33 selects the most of the stored correlation values. Select the vector corresponding to the strong correlation. For example, the correlation value calculation unit 32 calculates, as the correlation value, the absolute value of the difference between the upper layer feature quantity belonging to the block of the field of interest and the upper layer feature quantity belonging to the block at the position specified by the corresponding field vector. When calculating the sum, the determination unit 33 selects the minimum correlation value.
[0272]
In step S26, the determination unit 33 outputs the vector selected in the process of step S25 as the upper layer vector, and the process ends.
[0273]
FIG. 29 is a flowchart for explaining the details of the lower layer vector detection process corresponding to step S15. In step S 31, the vector generation unit 41 of the lower layer motion vector detection unit 26 generates a predetermined vector based on the upper layer vector and the parameter B, and supplies the vector to the correlation value calculation unit 42 and the determination unit 43. In step S32, the correlation value calculation unit 42 centers on the lower layer feature amount corresponding to the attention data of the field of interest, and includes a block composed of a predetermined number of lower layer feature amounts, the upper layer vector of the corresponding field, and A correlation value with a block made up of a predetermined number of lower layer feature quantities is calculated with the position designated by the vector supplied from the vector generation unit 41 as the center. The correlation value calculation unit 42 calculates, for example, a cross-correlation between a lower layer feature amount belonging to the block of the target field and a lower layer feature amount belonging to the block at the position specified by the corresponding field vector as the correlation value. .
[0274]
In step S <b> 33, the determination unit 43 stores the correlation value supplied from the correlation value calculation unit 42 in association with the vector supplied from the vector generation unit 41.
[0275]
In step S34, the lower layer motion vector detection unit 26 determines whether or not a predetermined range of correlation values specified by the data included in the parameter B has been calculated, and calculates the predetermined range of correlation values. If it is determined that there is not, the process returns to step S31 to repeat the correlation value calculation process.
[0276]
If it is determined in step S34 that a correlation value in a predetermined range has been calculated, the process proceeds to step S35, and the determination unit 43 selects a vector corresponding to the strongest correlation among the stored correlation values. For example, the correlation value calculation unit 42 calculates, as the correlation value, a cross-correlation between a lower layer feature amount belonging to the block of the field of interest and a lower layer feature amount belonging to the block at the position specified by the corresponding field vector. At this time, the determination unit 33 selects the maximum correlation value.
[0277]
In step S36, the determination unit 43 outputs the vector selected in the process of step S35 as the lower hierarchy vector, and the process ends.
[0278]
In this way, the motion vector detection unit 2 can detect a motion vector with higher accuracy.
[0279]
Since the motion vector detection unit 2 executes the process of detecting motion vectors in two layers, the motion vector detection unit 2 can perform motion with higher accuracy without increasing the circuit scale and without requiring complicated calculations. A vector can be detected.
[0280]
FIG. 30 is a flowchart for explaining the details of other processing of lower layer vector detection by the lower layer motion vector detecting unit 26 having the configuration shown in FIG. 18 corresponding to the processing of step S15.
[0281]
In step S41, the class classification unit 51 performs class classification corresponding to the data of interest based on the subcarrier phase information and the lower layer feature amount.
[0282]
In step S42, the prediction tap extraction unit 52 supplies a predetermined number of predetermined lower layers corresponding to the classified class, the lower layer vector corresponding to the previous field, and the upper layer vector supplied from the memory 55. A prediction tap that is a feature amount is extracted.
[0283]
In step S43, the calculation unit 53 calculates a lower layer vector based on the coefficient set stored in the prediction coefficient memory 54 and the prediction tap.
[0284]
In step S44, the calculation unit 53 stores the lower layer vector in the memory 55 and outputs the lower layer vector, and the process ends.
[0285]
In this way, the lower layer motion vector detection unit 26 having the configuration shown in FIG. 18 can generate a motion vector with higher accuracy by applying the class classification adaptation process to the lower layer feature quantity.
[0286]
As described above, the image processing apparatus according to the present invention can generate a component video signal with higher accuracy than a conventional video signal from a composite video signal in consideration of a motion vector.
[0287]
FIG. 31 is a block diagram showing a configuration of an embodiment of an image processing apparatus according to the present invention, which generates a prediction coefficient used for a component video signal prediction process.
[0288]
The same parts as those in the image processing apparatus shown in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted.
[0289]
The component video signal input to the image processing apparatus is supplied to the NTSC encoder 201 and the coefficient calculation unit 202.
[0290]
The NTSC encoder 201 generates an NTSC composite signal based on the input component video signal, and supplies the generated NTSC composite signal to the NTSC decoder 1.
[0291]
The coefficient calculation unit 202 inputs the component based on the subcarrier phase information supplied from the NTSC decoder 1, the motion vector supplied from the motion vector detection unit 2, and the class tap supplied from the class tap extraction unit 3. By applying the operation shown in the equation (14) to the video signal, the image signal supplied from the NTSC decoder 1, and the upper layer feature quantity and lower layer feature quantity supplied from the motion vector detection unit 2, the component video A prediction coefficient for predicting the signal is generated. The calculation unit 4 supplies the generated prediction coefficient to the coefficient memory 203.
[0292]
The coefficient memory 203 stores the prediction coefficient supplied from the coefficient calculation unit 202.
[0293]
FIG. 32 is a block diagram illustrating a configuration of the coefficient calculation unit 202. The same parts as those shown in FIG. 19 are denoted by the same reference numerals, and the description thereof is omitted.
[0294]
The prediction coefficient calculation unit 221-1 applies, for example, the calculation shown in Expression (14) to the prediction tap supplied from the prediction tap extraction unit 72-1, and the Y signal included in the input component video signal. , A prediction coefficient for predicting the Y signal is calculated, and the calculated prediction coefficient is output.
[0295]
The prediction coefficient calculation unit 221-2 applies, for example, the calculation shown in Expression (14) to the prediction tap supplied from the prediction tap extraction unit 72-2 and the U signal included in the input component video signal. , A prediction coefficient for predicting the U signal is calculated, and the calculated prediction coefficient is output.
[0296]
The prediction coefficient calculation unit 221-3 applies, for example, the calculation shown in Expression (14) to the prediction tap supplied from the prediction tap extraction unit 72-3 and the V signal included in the input component video signal. , A prediction coefficient for predicting the V signal is calculated, and the calculated prediction coefficient is output.
[0297]
FIG. 33 is a block diagram illustrating another configuration of the coefficient calculation unit 202. The same parts as those shown in FIG. 22 are denoted by the same reference numerals, and the description thereof is omitted.
[0298]
The selector 231 selects one class code and dynamic range from among the three class codes and dynamic range individually supplied from the class classification units 71-1 to 71-3, and predicts the selected class code and dynamic range. This is supplied to the tap extraction unit 82.
[0299]
When the prediction tap corresponding to the Y signal is supplied from the prediction tap extraction unit 82, the prediction coefficient calculation unit 232 is based on the prediction tap supplied from the prediction tap extraction unit 82 and the Y signal included in the component video signal. , A prediction coefficient for predicting the Y signal is calculated, and the calculated prediction coefficient is output. When the prediction tap corresponding to the U signal is supplied from the prediction tap extraction unit 82, the prediction coefficient calculation unit 232 is based on the prediction tap supplied from the prediction tap extraction unit 82 and the U signal included in the component video signal. , A prediction coefficient for predicting the U signal is calculated, and the calculated prediction coefficient is output. When the prediction tap corresponding to the V signal is supplied from the prediction tap extraction unit 82, the prediction coefficient calculation unit 232 is based on the prediction tap supplied from the prediction tap extraction unit 82 and the V signal included in the component video signal. , A prediction coefficient for predicting the V signal is calculated, and the calculated prediction coefficient is output.
[0300]
FIG. 34 is a block diagram showing still another configuration of the coefficient calculation unit 202. The same parts as those shown in FIG. 23 are denoted by the same reference numerals, and the description thereof is omitted.
[0301]
The prediction coefficient calculation unit 221-1 applies, for example, the calculation shown in Expression (14) to the prediction tap supplied from the prediction tap extraction unit 72-1, and the Y signal included in the input component video signal. , A prediction coefficient for predicting the Y signal is calculated, and the calculated prediction coefficient is output.
[0302]
The prediction coefficient calculation unit 221-2 applies, for example, the calculation shown in Expression (14) to the prediction tap supplied from the prediction tap extraction unit 72-2 and the U signal included in the input component video signal. , A prediction coefficient for predicting the U signal is calculated, and the calculated prediction coefficient is output.
[0303]
The prediction coefficient calculation unit 221-3 applies, for example, the calculation shown in Expression (14) to the prediction tap supplied from the prediction tap extraction unit 72-3 and the V signal included in the input component video signal. , A prediction coefficient for predicting the V signal is calculated, and the calculated prediction coefficient is output.
[0304]
FIG. 35 is a block diagram showing still another configuration of the coefficient calculation unit 202. The same parts as those shown in FIG. 24 are denoted by the same reference numerals, and the description thereof is omitted.
[0305]
When the prediction tap corresponding to the Y signal is supplied from the prediction tap extraction unit 82, the prediction coefficient calculation unit 232 is based on the prediction tap supplied from the prediction tap extraction unit 82 and the Y signal included in the component video signal. , A prediction coefficient for predicting the Y signal is calculated, and the calculated prediction coefficient is output. When the prediction tap corresponding to the U signal is supplied from the prediction tap extraction unit 82, the prediction coefficient calculation unit 232 is based on the prediction tap supplied from the prediction tap extraction unit 82 and the U signal included in the component video signal. , A prediction coefficient for predicting the U signal is calculated, and the calculated prediction coefficient is output. When the prediction tap corresponding to the V signal is supplied from the prediction tap extraction unit 82, the prediction coefficient calculation unit 232 is based on the prediction tap supplied from the prediction tap extraction unit 82 and the V signal included in the component video signal. , A prediction coefficient for predicting the V signal is calculated, and the calculated prediction coefficient is output.
[0306]
FIG. 36 is a block diagram illustrating details of an example of the configuration of the embodiment of the image processing apparatus that calculates the prediction coefficient. The same parts as those shown in FIG. 25 are denoted by the same reference numerals, and the description thereof is omitted.
[0307]
An HD (High-Definition) or progressive component video signal, which is an input signal, is supplied to a thinning filter 261 and a selection circuit 263.
[0308]
The thinning filter 261 generates an SD (Standard-Definition) interlaced component video signal from an HD or progressive component video signal that is an input signal. The thinning filter 261 supplies the generated component video signal to the NTSC encoder 262 and the selection circuit 263.
[0309]
The NTSC encoder 262 generates an NTSC composite video signal as a student image based on the input component video signal. The NTSC encoder 262 supplies the generated NTSC composite video signal to the subcarrier phase information detection circuit 121, the field memory 122-1, the motion vector detection circuit 123, and the delay circuit 124-1.
[0310]
The selection circuit 263 selects an HD or progressive component video signal input from the image processing apparatus and an SD interlace component video signal supplied from the thinning filter 261 based on the learning mode setting signal supplied from the outside. Either one is selected, and the selected signal is supplied to the normal equation calculation circuit 264 as a teacher image.
[0311]
The normal equation calculation circuit 264 includes the class code supplied from the class classification unit 127, the data supplied from the area extraction circuit 128-1, the data extracted from the signal of the field next to the reference field, and the area extraction circuit. Data extracted from the signal of the reference field supplied from 128-2, data extracted from the signal of the field before the reference field supplied from the region extraction circuit 128-3, and selection Based on the teacher image supplied from the circuit 263, for example, prediction coefficients for predicting the Y signal, the U signal, and the V signal are calculated. The normal equation calculation circuit 264 supplies the calculated prediction coefficient to the coefficient memory 265.
[0312]
The coefficient memory 265 stores the prediction coefficient supplied from the normal equation calculation circuit 264.
[0313]
Thus, the image processing apparatus according to the present invention can calculate the prediction coefficient for predicting the component video signal based on the composite video signal.
[0314]
FIG. 37 is a flowchart for describing learning processing for calculating a prediction coefficient for predicting a component video signal based on a composite video signal by the image processing apparatus according to the present invention.
[0315]
In step S201, the NTSC encoder 201 generates a composite video signal based on the input component video signal.
[0316]
In step S202, the motion vector detection unit 2 detects a motion vector based on the image signal that is the composite video signal and the subcarrier phase information. The details of the process of step S202 are the same as the process of step S1, and thus the description thereof is omitted.
[0317]
In step S203, the class tap extraction unit 3 uses the subcarrier phase information supplied from the NTSC decoder 1 and the motion vector supplied from the motion vector detection unit 2 to generate image signal data and upper layer feature value data. , And class taps composed of lower layer feature data.
[0318]
In step S204, the coefficient calculation unit 202 outputs the image signal and subcarrier phase information supplied from the NTSC decoder 1, the motion vector supplied from the motion vector detection unit 2, the upper layer feature data, and the lower layer feature data. Based on the data and the class tap supplied from the class tap extraction unit 3, the classification is performed. For example, the class classification units 71-1 to 71-3 of the coefficient calculation unit 202 illustrated in FIG. 32 include the image signal and subcarrier phase information supplied from the NTSC decoder 1 and the motion vector supplied from the motion vector detection unit 2. Then, class classification is performed based on the data of the upper layer feature amount, the data of the lower layer feature amount, and the class tap supplied from the class tap extraction unit 3.
[0319]
In step S205, the coefficient calculation unit 202, based on the classified class and the motion vector supplied from the motion vector detection unit 2, the image signal data, the upper layer feature value data, and the lower layer feature value data. The prediction tap which consists of is extracted. For example, the prediction tap extraction units 72-1 to 72-3 of the coefficient calculation unit 202 illustrated in FIG. 32 use the class code indicating the classified class, the dynamic range, and the motion vector supplied from the motion vector detection unit 2. Based on this, a prediction tap consisting of image signal data, upper layer feature data, and lower layer feature data is extracted.
[0320]
In step S206, the coefficient calculation unit 202 calculates a prediction coefficient for predicting the component video signal based on the extracted prediction tap and the input component video signal, and the process ends. For example, the prediction coefficient calculation units 221-1 to 221-3 of the coefficient calculation unit 202 illustrated in FIG. 32 predict a Y signal, a U signal, or a V signal based on the prediction tap and the component video signal, respectively. The prediction coefficient is calculated.
[0321]
Thus, the image processing apparatus according to the present invention can calculate the prediction coefficient for predicting the component video signal based on the composite video signal.
[0322]
FIG. 38 is a block diagram illustrating a configuration of an image processing apparatus that generates a prediction coefficient used for processing of a lower layer vector in the lower layer motion vector detection unit 26 having the configuration illustrated in FIG.
[0323]
The NTSC decoder 301 generates an image signal that is digital data corresponding to an image signal based on the input image signal that is an NTSC composite video signal, and a sub signal corresponding to the image signal that is digital data. Carrier phase information is generated. The NTSC decoder 301 supplies an image signal and subcarrier phase information, which are generated digital data, to the image memory 302.
[0324]
The image memory 302 stores the image signal and subcarrier phase information supplied from the NTSC decoder 301, and supplies the stored image signal and subcarrier phase information to the coefficient generation unit 303.
[0325]
The coefficient generation unit 303 is a coefficient corresponding to the image signal and motion vector supplied from the image memory 302 based on the parameter A supplied from the outside of the image processing apparatus and the subcarrier phase information supplied from the image memory 302. Generate a set and output the generated coefficient set. The coefficient set is composed of a predetermined number of prediction coefficients corresponding to each class.
[0326]
The parameter A is a parameter that specifies the content of processing in the coefficient generation unit 303, and specifies, for example, a correlation calculation method, the size of a block to be matched, or the size of a search area.
[0327]
FIG. 39 is a block diagram illustrating a configuration of the coefficient generation unit 303. The same parts as those in the motion vector detecting unit 2 shown in FIG. 3 are denoted by the same reference numerals, and the description thereof is omitted.
[0328]
The learning unit 321 calculates a motion vector based on the motion vector, the lower layer feature amount and subcarrier phase information supplied from the feature amount conversion unit 24, and the upper layer vector supplied from the upper layer motion vector detection unit 25. A coefficient set for performing the calculation is calculated, and the calculated coefficient set is output.
[0329]
FIG. 40 is a block diagram illustrating a configuration of the learning unit 321.
[0330]
The class classification unit 351 performs class classification based on the subcarrier phase information and the lower layer feature amount, and supplies a class code indicating the classified class to the prediction tap extraction unit 352.
[0331]
The prediction tap extraction unit 352 extracts a predetermined lower layer feature quantity based on the class indicated by the class code, the predicted motion vector corresponding to the previous field supplied from the memory 354, and the upper layer vector, The extracted lower layer feature quantity is supplied to the coefficient calculation unit 353 as a prediction tap.
[0332]
The coefficient calculation unit 353 calculates a prediction coefficient and a prediction motion vector based on the prediction tap that is supplied from the prediction tap extraction unit 352 and includes predetermined data of lower layer feature quantities. The coefficient calculation unit 353 supplies the calculated coefficient to the prediction coefficient memory 355 and also supplies the calculated prediction motion vector to the memory 354.
[0333]
The memory 354 stores the prediction motion vector supplied from the coefficient calculation unit 353 and supplies the stored prediction motion vector to the prediction tap extraction unit 352. That is, the prediction tap extraction unit 352 is supplied with a prediction motion vector corresponding to the previous field.
[0334]
The prediction coefficient memory 355 stores the prediction coefficient supplied from the coefficient calculation unit 353, and outputs the stored prediction coefficient as a coefficient set.
[0335]
Next, with reference to the flowchart of FIG. 41, the learning process of the image processing apparatus for generating the prediction coefficient used for the calculation process of the lower layer vector will be described.
[0336]
In step S301, the coefficient generation unit 303 acquires the phase of the subcarrier of the data of interest based on the subcarrier phase information supplied from the NTSC decoder 1 via the NTSC decoder 1.
[0337]
In step S302, the feature amount conversion unit 23 of the coefficient generation unit 303 calculates a higher-layer feature amount, and outputs the calculated feature amount as an upper-layer feature amount.
[0338]
In step S <b> 303, the upper layer motion vector detection unit 25 of the coefficient generation unit 303 detects an upper layer vector based on the upper layer feature quantity supplied from the feature quantity conversion unit 23. Details of the processing in step S303 are the same as the processing described with reference to the flowchart of FIG.
[0339]
In step S304, the feature amount conversion unit 24 of the coefficient generation unit 303 calculates a lower layer feature amount, and outputs the calculated feature amount as a lower layer feature amount.
[0340]
In step S305, the learning unit 321 calculates a coefficient set based on the motion vector, the lower layer feature amount supplied from the feature amount conversion unit 24, and the upper layer vector supplied from the upper layer motion vector detection unit 25. Then, the process ends.
[0341]
FIG. 42 is a flowchart for explaining the details of the coefficient set calculation processing by the learning unit 321 corresponding to the processing in step S305.
[0342]
In step S321, the class classification unit 351 performs class classification corresponding to the data of interest based on the subcarrier phase information and the lower layer feature amount.
[0343]
In step S322, the prediction tap extraction unit 352 outputs a predetermined number of predetermined lower layers corresponding to the classified class, the predicted motion vector corresponding to the previous field, and the upper layer vector supplied from the memory 354. A prediction tap that is a feature amount is extracted.
[0344]
In step S323, the coefficient calculation unit 353 calculates a coefficient based on the prediction tap supplied from the prediction tap extraction unit 352 and the motion vector supplied from the outside. The coefficient calculation unit 353 supplies the calculated coefficient to the prediction coefficient memory 355.
[0345]
In step S324, the prediction coefficient memory 355 stores the coefficient, and the process ends.
[0346]
As described above, the image processing apparatus having the configuration shown in FIG. 38 can generate the prediction coefficient used for the calculation of the lower layer vector.
[0347]
In the above description, the image signal which is digital data has been described as a signal corresponding to the Y signal, the I signal, or the Q signal. , Y signal, U signal, and V signal, Y signal, Pb signal, and Pr signal, or other types of image signals such as Y signal, Cb signal, and Cr signal.
[0348]
In addition, the component video signal to be generated has been described as the Y signal, the U signal, and the V signal, but is not limited to the Y signal, the U signal, and the V signal, Other types of image signals such as Y signal, Cb signal, and Cr signal may be used.
[0349]
The image processing apparatus according to the present invention has been described as generating a component video signal based on an image signal that is an NTSC composite video signal. However, the image processing apparatus is not limited to the NTSC system, and may be a PAL (Phase Alternation by Line) system or the like. A component video signal may be generated based on a composite video signal of another method.
[0350]
The image processing apparatus according to the present invention has been described as detecting a motion vector between a focused field of interest and a field next to the focused field. However, the focused field of interest and a field before the focused field are described. The motion vector may be detected.
[0351]
Although the image processing apparatus according to the present invention has been described as detecting a motion vector in units of fields, the motion vector may be detected in units of frames.
[0352]
The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.
[0353]
FIG. 43 is a diagram illustrating an example of a recording medium and a computer. A CPU (Central Processing Unit) 501 actually executes various application programs and an OS (Operating System). A ROM (Read-only Memory) 502 generally stores basically fixed data among programs used by the CPU 501 and calculation parameters. A RAM (Random-Access Memory) 503 stores programs used in the execution of the CPU 501 and parameters that change as appropriate during the execution. These are connected to each other by a host bus 504 including a CPU bus.
[0354]
The host bus 504 is connected to an external bus 506 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 505.
[0355]
The keyboard 508 is operated by the user when inputting various commands to the CPU 501. The mouse 509 is operated by the user when specifying or selecting a point on the screen of the display 510. The display 510 includes a liquid crystal display device, a CRT (Cathode Ray Tube), or the like, and displays various types of information as text and images. An HDD (Hard Disk Drive) 511 drives a hard disk and records or reproduces a program executed by the CPU 501 and information.
[0356]
The drive 512 reads out data or a program recorded in the mounted magnetic disk 551, optical disk 552, magneto-optical disk 553, or semiconductor memory 554, and the data or program is read out to the interface 507, external bus 506, bridge. 505 and the RAM 503 connected via the host bus 504.
[0357]
These keyboard 508 to drive 512 are connected to an interface 507, and the interface 507 is connected to the CPU 501 via an external bus 506, a bridge 505, and a host bus 504.
[0358]
The video interface 513 acquires the supplied image signal, and supplies the acquired image signal to the RAM 503 or the CPU 501 via the external bus 506, the bridge 505, and the host bus 504. The video interface 513 outputs the predicted component video signal.
[0359]
As shown in FIG. 43, the recording medium is distributed to provide a program for executing processing corresponding to the block diagram to the user separately from the computer, and a magnetic disk 551 (floppy disk ( Optical disc 552 (including CD-ROM (Compact Disc-Read Only Memory) and DVD (Digital Versatile Disc)), magneto-optical disc 553 (including MD (Mini-Disc) (trademark)), Alternatively, it is configured not only by a removable medium such as a semiconductor memory 554 but also by a ROM 502, a HDD 511, and the like that are provided to the user in a state of being preinstalled in a computer and in which a program is recorded.
[0360]
Note that the program for executing the processing corresponding to the block diagram for the user may be supplied to the computer via a wired or wireless communication medium.
[0361]
Further, in the present specification, the step of describing the program stored in the recording medium is not limited to the processing performed in time series according to the described order, but is not necessarily performed in time series. It also includes processes that are executed individually.
[0362]
【The invention's effect】
  According to the first image processing apparatus and method, recording medium, and program of the present invention,It is obtained by adding or subtracting the luminance signal and the color signal of the composite video signal for the target field and the other field of interest, where the target point is the target point of the composite video signal. From the obtained image signal, it is the value of a point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, and the average of four points having different phases. A first feature amount that is a value or an average value of four points that are adjacent to each other vertically and horizontally on two lines of a field and that are different in phase, A first feature amount of the point and a first feature amount of the point of interest in the attention field around the point, the first feature amount closest to the arrangement and value, and in the other field By detecting the first feature amount, the first vector that approximates the motion vector is detected, and the value of the target point that is the target of the motion vector detection calculation for the field of interest and other fields is calculated. The value obtained by multiplying the weight of 1 and the value of a point around the target point in the field where the target point exists, adjacent to the target point, or separated by one or two points from the target point A second feature value corresponding to the target point, which is a value obtained by multiplying the result obtained by multiplying the second weight by the other weight, and another field centered on the position specified by the first vector The second feature quantity closest to the second feature quantity and the second feature quantity of the point of interest in the field of interest from the search area of The position of the second feature value in You, as a second vectorMotion vectors are detected,The attention point is classified into one of a plurality of predetermined classes based on the first feature amount or the second feature amount and the motion vector.,Prediction taps that are predetermined for each classified class and that include the first feature amount or the second feature amount are extracted.,By applying a sum-of-products operation to the previously calculated prediction coefficient and the extracted prediction tap,Since the component video signal corresponding to the composite video signal of the attention point is generated, the component video signal can be generated from the composite video signal with higher accuracy in consideration of the motion vector.
[0363]
  According to the second image processing apparatus and method, recording medium, and program of the present invention,It is obtained by adding or subtracting the luminance signal and the color signal of the composite video signal for the target field and the other field of interest, where the target point is the target point of the composite video signal. From the obtained image signal, it is the value of a point having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, and the average of four points having different phases. A first feature amount that is a value or an average value of four points that are adjacent to each other vertically and horizontally on two lines of a field and that are different in phase, A first feature amount of the point and a first feature amount of the point of interest in the attention field around the point, the first feature amount closest to the arrangement and value, and in the other field By detecting the first feature amount, the first vector that approximates the motion vector is detected, and the value of the target point that is the target of the motion vector detection calculation for the field of interest and other fields is calculated. The value obtained by multiplying the weight of 1 and the value of a point around the target point in the field where the target point exists, adjacent to the target point, or separated by one or two points from the target point A second feature value corresponding to the target point, which is a value obtained by multiplying the result obtained by multiplying the second weight by the other weight, and another field centered on the position specified by the first vector The second feature quantity closest to the second feature quantity and the second feature quantity of the point of interest in the field of interest from the search area of The position of the second feature value in You, as a second vectorMotion vectors are detected,The attention point is classified into one of a plurality of predetermined classes based on the first feature amount or the second feature amount and the motion vector.,Prediction taps that are predetermined for each classified class and that include the first feature amount or the second feature amount are extracted.,A normal equation obtained from the component video signal and the extracted prediction taps, and by solving the normal equation that minimizes the residual,Since the coefficient is calculated, the component video signal can be generated from the composite video signal with higher accuracy by taking the motion vector into consideration by using the coefficient.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an embodiment of an image processing apparatus according to the present invention.
FIG. 2 is a diagram illustrating an image signal and subcarrier phase information.
FIG. 3 is a block diagram showing a configuration of a motion vector detection unit 2;
4 is a block diagram illustrating a configuration of an upper layer motion vector detection unit 25. FIG.
5 is a block diagram showing a configuration of a lower layer motion vector detection unit 26. FIG.
FIG. 6 is a diagram illustrating examples of upper layer feature values, lower layer feature values, and search areas;
FIG. 7 is a diagram illustrating a block of an upper layer feature amount.
FIG. 8 is a diagram illustrating calculation of lower layer feature amounts.
FIG. 9 is a diagram illustrating an example of regions and blocks searched by the lower layer motion vector detection unit 26;
FIG. 10 is a diagram illustrating an example of another block of upper layer feature values.
FIG. 11 is a diagram illustrating another example of upper layer feature amounts.
FIG. 12 is a diagram illustrating another example of upper layer feature amounts.
FIG. 13 is a diagram illustrating another example of lower layer feature amounts.
FIG. 14 is a diagram illustrating another example of lower layer feature amounts.
FIG. 15 is a diagram illustrating another example of lower layer feature amounts.
FIG. 16 is a diagram illustrating another example of lower layer feature amounts.
FIG. 17 is a diagram illustrating another example of lower layer feature amounts.
18 is a block diagram showing another configuration of the lower layer motion vector detection unit 26. FIG.
FIG. 19 is a block diagram showing a configuration of a calculation unit 4;
FIG. 20 is a diagram for describing an example of a class tap and a prediction tap corresponding to an upper layer feature quantity.
FIG. 21 is a diagram illustrating an example of a class tap and a prediction tap corresponding to a lower layer feature amount.
FIG. 22 is a block diagram showing another configuration of the calculation unit 4;
FIG. 23 is a block diagram showing another configuration of the calculation unit 4;
24 is a block diagram showing another configuration of the calculation unit 4. FIG.
FIG. 25 is a block diagram showing details of an example of the configuration of an embodiment of an image processing apparatus according to the present invention;
FIG. 26 is a flowchart illustrating a component video signal generation process.
FIG. 27 is a flowchart for describing motion vector detection processing;
FIG. 28 is a flowchart illustrating details of processing for detecting an upper layer vector.
FIG. 29 is a flowchart illustrating details of processing for detecting a lower layer vector.
FIG. 30 is a flowchart illustrating details of another process of detecting a lower layer vector.
FIG. 31 is a block diagram showing a configuration of an embodiment of an image processing apparatus according to the present invention for generating a prediction coefficient.
32 is a block diagram showing a configuration of a coefficient calculation unit 202. FIG.
33 is a block diagram illustrating another configuration of the coefficient calculation unit 202. FIG.
34 is a block diagram showing another configuration of the coefficient calculation unit 202. FIG.
35 is a block diagram illustrating another configuration of the coefficient calculation unit 202. FIG.
FIG. 36 is a block diagram illustrating details of an example of the configuration of an embodiment of an image processing apparatus that calculates a prediction coefficient.
FIG. 37 is a flowchart illustrating learning processing for calculating a prediction coefficient.
FIG. 38 is a block diagram illustrating a configuration of an image processing apparatus that generates a prediction coefficient used for a calculation process of a lower layer vector.
39 is a block diagram showing a configuration of a coefficient generation unit 303. FIG.
40 is a block diagram showing a configuration of a learning unit 321. FIG.
FIG. 41 is a flowchart illustrating learning processing performed by the image processing apparatus.
FIG. 42 is a flowchart illustrating details of a coefficient set calculation process.
FIG. 43 is a diagram illustrating an example of a recording medium and a computer.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 NTSC decoder, 2 Motion vector detection part, 3 Class tap extraction part, 4 Operation part, 23 Feature-value conversion part, 24 Feature-value conversion part, 25 Upper layer motion vector detection part, 26 Lower layer motion vector detection part, 31 Vector Generation unit, 32 correlation value calculation unit, 33 determination unit, 41 vector generation unit, 42 correlation value calculation unit, 43 determination unit, 51 class classification unit, 52 prediction tap extraction unit, 53 operation unit, 54 prediction coefficient memory, 55 memory , 71-1 to 71-3 class classification unit, 72-1 to 72-3 prediction tap extraction unit, 73-1 to 73-3 coefficient memory, 74-1 to 74-3 prediction calculation unit, 81 selector, 82 prediction Tap extraction unit, 83 prediction calculation unit, 91 class classification unit, 101 selector, 201 NTSC encoder , 202 coefficient calculation unit, 203 coefficient memory, 211-1 to 211-3 prediction coefficient calculation unit, 231 selector, 232 prediction coefficient calculation unit, 303 coefficient generation unit, 321 learning unit, 351 class classification unit, 352 prediction tap extraction unit , 353 coefficient calculation unit, 354 memory, 355 prediction coefficient memory, 501 CPU, 502 ROM, 503 RAM, 511 HDD, 551 magnetic disk, 552 optical disk, 553 magneto-optical disk, 554 semiconductor memory

Claims (26)

In an image processing apparatus for generating a component video signal from a composite video signal,
Addition or subtraction of the luminance signal and the color signal of the composite video signal with respect to the field of interest and the other field of interest where there is a point of interest that is the point of interest of the composite video signal From the obtained image signal, the values of the points having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, each having a different phase Or four points adjacent to each other vertically and horizontally on two lines of the field, each of which is an average value of four points of different phases. Generating a feature amount, and arranging and value the first feature amount of the point in the attention field around the attention point and the first feature amount of the attention point; By detecting the first feature value closest to the first feature value in the other field, a first vector that approximates a motion vector is detected, and the attention field and the other field are detected. A result obtained by multiplying a value of a target point that is a target of the motion vector detection calculation by a first weight, and a point around the target point in a field where the target point exists, A second value corresponding to the target point, which is a value obtained by adding a value obtained by multiplying the value of a point adjacent to the target point or separated from the target point by one or two points by the second weight. And the second feature value of a point around the target point in the target field from the search area of the other field, centered on the position specified by the first vector. And before The second feature of the subject point, a second feature quantity closest arrangement and values, indicating a position of the second feature amount in the other fields, the motion vector as a second vector Detecting means for detecting
Class classification means for classifying the attention point into one of a plurality of predetermined classes based on the first feature quantity or the second feature quantity and the motion vector ;
Extracting means for extracting a prediction tap that is predetermined for each classified class and includes the first feature amount or the second feature amount ;
Generating means for generating the component video signal corresponding to the composite video signal at the point of interest by applying a product-sum operation to the prediction coefficient obtained in advance and the extracted prediction tap. A featured image processing apparatus. The extraction means further includes the prediction tap determined according to a dynamic range of the composite video signal or a dynamic range of the first feature amount or the second feature amount corresponding to the composite video signal. The image processing device according to claim 1, wherein the image processing device is extracted. The class classifying unit classifies the composite video signal at the point of interest into one of a plurality of classes with respect to a luminance signal constituting the component video signal, and further, a color signal constituting the component video signal The image processing apparatus according to claim 1, wherein the composite video signal at the attention point is classified into one of a plurality of classes . The extraction means extracts the prediction tap for a luminance signal constituting the component video signal, and further extracts the prediction tap for a color signal constituting the component video signal. The image processing apparatus according to 1. The detection means includes
For the field of interest and the other field, from the image signal, the values of the points having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field The average value of four points having different phases, or four points adjacent to each other vertically and horizontally on the two lines of the field, each of four points having different phases. First feature value generation means for generating a first feature value that is an average value ;
The first feature quantity closest in arrangement and value to the first feature quantity of the point in the attention field around the attention point and the first feature quantity of the attention point, First vector detecting means for detecting a first vector that approximates the motion vector by detecting the first feature quantity in a field ;
For the field of interest and other fields, the result obtained by multiplying the value of the target point that is the target of the motion vector detection calculation by the first weight, and the target point in the field where the target point exists A value obtained by multiplying a value obtained by multiplying a value of a point adjacent to the target point or a point separated from the target point by one or two points by a second weight. Second feature value generation means for generating a second feature value corresponding to the target point ;
From the area to be searched for in the other field, centered on the position specified by the first vector, the second feature amount of the points around the point of interest in the field of interest and the first of the points of interest The second feature quantity closest to the feature quantity of 2 is the second feature quantity that is closest to the feature quantity, and a motion vector is detected as the second vector that indicates the position of the second feature quantity in the other field . The image processing apparatus according to claim 1, further comprising: two vector detection units. The first vector detecting means is the first feature amount having the strongest correlation with the first feature amount of the attention field that is a moving image screen to which the attention point belongs. The image according to claim 5, wherein the first vector that approximates the motion vector is detected from a position of the first feature amount in the other field that is a screen of the moving image adjacent thereto. Processing equipment. The second vector detection means is the second feature having the strongest correlation with the second feature amount in the first range including the attention point in the attention field which is a screen of a moving image to which the attention point belongs. The second feature amount of the area to be searched centering on the position specified by the first vector in the other field that is the screen of the moving image adjacent to the field of interest. The image processing apparatus according to claim 5, wherein the motion vector is detected as the second vector from the position of the image. The image processing apparatus according to claim 5, wherein the second vector detection unit applies a class classification adaptive process to the second feature amount , and detects the motion vector as the second vector. . The image processing apparatus according to claim 8, wherein the second vector detection unit performs class classification using a phase of the color signal at the point of interest. 9. The image according to claim 8, wherein the second vector detection means applies an adaptive process to the second feature amount at a position indicated by the second vector detected in the previous field. Processing equipment. In an image processing method for generating a component video signal from a composite video signal,
Addition or subtraction of the luminance signal and the color signal of the composite video signal with respect to the field of interest and the other field of interest where there is a point of interest that is the point of interest of the composite video signal From the obtained image signal, the values of the points having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, each having a different phase Or four points adjacent to each other vertically and horizontally on two lines of the field, each of which is an average value of four points of different phases. Generating a feature amount, and arranging and value the first feature amount of the point in the attention field around the attention point and the first feature amount of the attention point; By detecting the first feature value closest to the first feature value in the other field, a first vector that approximates a motion vector is detected, and the attention field and the other field are detected. A result obtained by multiplying a value of a target point that is a target of the motion vector detection calculation by a first weight, and a point around the target point in a field where the target point exists, A second value corresponding to the target point, which is a value obtained by adding a value obtained by multiplying the value of a point adjacent to the target point or separated from the target point by one or two points by the second weight. And the second feature value of a point around the target point in the target field from the search area of the other field, centered on the position specified by the first vector. And before The second feature of the subject point, a second feature quantity closest arrangement and values, indicating a position of the second feature amount in the other fields, the motion vector as a second vector A detecting step for detecting
A class classification step of classifying the attention point into one of a plurality of predetermined classes based on the first feature quantity or the second feature quantity and the motion vector ;
An extraction step for extracting a prediction tap that is predetermined for each classified class and includes the first feature amount or the second feature amount ; and
Generating a component video signal corresponding to the composite video signal at the point of interest by applying a product-sum operation to the prediction coefficient obtained in advance and the extracted prediction tap. A featured image processing method. An image processing program for generating a component video signal from a composite video signal,
Addition or subtraction of the luminance signal and the color signal of the composite video signal with respect to the field of interest and the other field of interest where there is a point of interest that is the point of interest of the composite video signal From the obtained image signal, the values of the points having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, each having a different phase Or four points adjacent to each other vertically and horizontally on two lines of the field, each of which is an average value of four points of different phases. Generating a feature amount, and arranging and value the first feature amount of the point in the attention field around the attention point and the first feature amount of the attention point; By detecting the first feature value closest to the first feature value in the other field, a first vector that approximates a motion vector is detected, and the attention field and the other field are detected. A result obtained by multiplying a value of a target point that is a target of the motion vector detection calculation by a first weight, and a point around the target point in a field where the target point exists, A second value corresponding to the target point, which is a value obtained by adding a value obtained by multiplying the value of a point adjacent to the target point or separated from the target point by one or two points by the second weight. And the second feature value of a point around the target point in the target field from the search area of the other field, centered on the position specified by the first vector. And before The second feature of the subject point, a second feature quantity closest arrangement and values, indicating a position of the second feature amount in the other fields, the motion vector as a second vector A detecting step for detecting
A class classification step of classifying the attention point into one of a plurality of predetermined classes based on the first feature quantity or the second feature quantity and the motion vector ;
An extraction step for extracting a prediction tap that is predetermined for each classified class and includes the first feature amount or the second feature amount ; and
Generating a component video signal corresponding to the composite video signal at the point of interest by applying a product-sum operation to the prediction coefficient obtained in advance and the extracted prediction tap. A recording medium on which a computer-readable program is recorded. A computer that controls an image processing apparatus that generates a component video signal from a composite video signal;
Addition or subtraction of the luminance signal and the color signal of the composite video signal with respect to the field of interest and the other field of interest where there is a point of interest that is the point of interest of the composite video signal From the obtained image signal, the values of the points having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, each having a different phase Or four points adjacent to each other vertically and horizontally on two lines of the field, each of which is an average value of four points of different phases. Generating a feature amount, and arranging and value the first feature amount of the point in the attention field around the attention point and the first feature amount of the attention point; By detecting the first feature value closest to the first feature value in the other field, a first vector that approximates a motion vector is detected, and the attention field and the other field are detected. A result obtained by multiplying a value of a target point that is a target of the motion vector detection calculation by a first weight, and a point around the target point in a field where the target point exists, A second value corresponding to the target point, which is a value obtained by adding a value obtained by multiplying the value of a point adjacent to the target point or separated from the target point by one or two points by the second weight. And the second feature value of a point around the target point in the target field from the search area of the other field, centered on the position specified by the first vector. And before The second feature of the subject point, a second feature quantity closest arrangement and values, indicating a position of the second feature amount in the other fields, the motion vector as a second vector A detecting step for detecting
A class classification step of classifying the attention point into one of a plurality of predetermined classes based on the first feature quantity or the second feature quantity and the motion vector ;
An extraction step for extracting a prediction tap that is predetermined for each classified class and includes the first feature amount or the second feature amount ; and
A program for executing the generation step of generating the component video signal corresponding to the composite video signal at the point of interest by applying a product-sum operation to the prediction coefficient obtained in advance and the extracted prediction tap . In an image processing apparatus for generating coefficients used in a process of predicting a component video signal from a composite video signal,
Addition or subtraction of the luminance signal and the color signal of the composite video signal with respect to the field of interest and the other field of interest where there is a point of interest that is the point of interest of the composite video signal From the obtained image signal, the values of the points having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, each having a different phase Or four points adjacent to each other vertically and horizontally on two lines of the field , each of which is an average value of four points of different phases. Generating a feature quantity, and arranging and value the first feature quantity of the point in the attention field around the attention point and the first feature quantity of the attention point; By detecting the first feature value closest to the first feature value in the other field, the first vector approximate to the motion vector is detected, and the target field and the other field are detected. And a result obtained by multiplying the value of the target point to be subjected to the calculation of the motion vector by a first weight, and a point around the target point in the field where the target point exists, A value corresponding to the target point is a value obtained by adding a value obtained by multiplying a value of a point adjacent to the target point or separated from the target point by one or two points by a second weight. 2 is generated, and the second feature of the point around the target point in the target field is searched from the region searched for in the other field with the position specified by the first vector as the center. Quantity and The second feature quantity of the target point, a second feature quantity closest arrangement and values, indicating a position of the second feature amount in the other fields, the second motion as a vector Detecting means for detecting a vector;
Class classification means for classifying the attention point into one of a plurality of predetermined classes based on the first feature quantity or the second feature quantity and the motion vector ;
Extracting means for extracting a prediction tap that is predetermined for each classified class and includes the first feature amount or the second feature amount ;
A normal equation obtained from the component video signal and the extracted prediction tap, and calculating a coefficient of the classified class by solving a normal equation that minimizes a residual. An image processing apparatus. The extraction unit further includes the prediction tap determined according to a dynamic range of the composite video signal or a dynamic range of the first feature amount or the second feature amount corresponding to the composite video signal. The image processing device according to claim 14, wherein the image processing device is extracted. The class classifying unit classifies the composite video signal at the point of interest into one of a plurality of classes with respect to a luminance signal constituting the component video signal, and further, a color signal constituting the component video signal The image processing apparatus according to claim 14, wherein the composite video signal of the attention point is classified into one class among a plurality of classes . The extraction means extracts the prediction tap for a luminance signal constituting the component video signal, and further extracts the prediction tap for a color signal constituting the component video signal. An image processing apparatus according to 1. The detection means includes
For the field of interest and the other field, from the image signal, the values of the points having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field The average value of four points having different phases, or four points adjacent to each other vertically and horizontally on the two lines of the field, each of four points having different phases. First feature value generation means for generating a first feature value that is an average value ;
The first feature quantity closest in arrangement and value to the first feature quantity of the point in the attention field around the attention point and the first feature quantity of the attention point, First vector detecting means for detecting a first vector that approximates the motion vector by detecting the first feature quantity in a field ;
For the field of interest and other fields, the result obtained by multiplying the value of the target point that is the target of the motion vector detection calculation by the first weight, and the target point in the field where the target point exists A value obtained by multiplying a value obtained by multiplying a value of a point adjacent to the target point or a point separated from the target point by one or two points by a second weight. Second feature value generation means for generating a second feature value corresponding to the target point ;
From the area to be searched for in the other field, centered on the position specified by the first vector, the second feature amount of the points around the point of interest in the field of interest and the first of the points of interest The second feature quantity closest to the feature quantity of 2 is the second feature quantity that is closest to the feature quantity, and a motion vector is detected as the second vector indicating the position of the second feature quantity in the other field . The image processing apparatus according to claim 14, further comprising: two vector detection units. The first vector detecting means is the first feature amount having the strongest correlation with the first feature amount of the attention field that is a moving image screen to which the attention point belongs. 19. The image according to claim 18, wherein the first vector that approximates the motion vector is detected from a position of the first feature amount in the other field that is a screen of the moving image adjacent to the image. Processing equipment. The second vector detection means is the second feature having the strongest correlation with the second feature amount in the first range including the attention point in the attention field which is a screen of a moving image to which the attention point belongs. The second feature amount of the area to be searched centering on the position specified by the first vector in the other field that is the screen of the moving image adjacent to the field of interest. The image processing apparatus according to claim 18, wherein the motion vector is detected as the second vector from the position of the image. The image processing apparatus according to claim 18, wherein the second vector detection unit applies a class classification adaptive process to the second feature quantity , and detects the motion vector as the second vector. . The image processing apparatus according to claim 21, wherein the second vector detection unit performs class classification using a phase of the color signal at the point of interest. The image according to claim 21, wherein the second vector detection means applies an adaptive process to the second feature amount at a position indicated by the second vector detected in the previous field. Processing equipment. In an image processing method for generating coefficients used in a process of predicting a component video signal from a composite video signal,
Addition or subtraction of the luminance signal and the color signal of the composite video signal with respect to the field of interest and the other field of interest where there is a point of interest that is the point of interest of the composite video signal From the obtained image signal, the values of the points having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, each having a different phase Or four points adjacent to each other vertically and horizontally on two lines of the field, each of which is an average value of four points of different phases. Generating a feature amount, and arranging and value the first feature amount of the point in the attention field around the attention point and the first feature amount of the attention point; A nearest the first feature amount, by detecting the first feature amount in the other fields, and detecting a first vector approximating said motion vector, the target field and other fields And a result obtained by multiplying the value of the target point to be subjected to the calculation of the motion vector by a first weight, and a point around the target point in the field where the target point exists, A value corresponding to the target point is a value obtained by adding a value obtained by multiplying a value of a point adjacent to the target point or separated from the target point by one or two points by a second weight. 2 is generated, and the second feature of the point around the target point in the target field is searched from the region searched for in the other field with the position specified by the first vector as the center. Quantity and The second feature quantity of the target point, a second feature quantity closest arrangement and values, indicating a position of the second feature amount in the other fields, the second motion as a vector A detection step for detecting a vector;
A class classification step of classifying the attention point into one of a plurality of predetermined classes based on the first feature quantity or the second feature quantity and the motion vector ;
An extraction step for extracting a prediction tap that is predetermined for each classified class and includes the first feature amount or the second feature amount ; and
A normal equation obtained from the component video signal and the extracted prediction tap, and calculating a coefficient of the classified class by solving a normal equation that minimizes a residual. An image processing method characterized by the above. An image processing program for generating coefficients used in a process of predicting a component video signal from a composite video signal,
Addition or subtraction of the luminance signal and the color signal of the composite video signal with respect to the field of interest and the other field of interest where there is a point of interest that is the point of interest of the composite video signal From the obtained image signal, the values of the points having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, each having a different phase Or four points adjacent to each other vertically and horizontally on two lines of the field, each of which is an average value of four points of different phases. Generating a feature amount, and arranging and value the first feature amount of the point in the attention field around the attention point and the first feature amount of the attention point; By detecting the first feature value closest to the first feature value in the other field, the first vector approximate to the motion vector is detected, and the target field and the other field are detected. And a result obtained by multiplying the value of the target point to be subjected to the calculation of the motion vector by a first weight, and a point around the target point in the field where the target point exists, A value corresponding to the target point is a value obtained by adding a value obtained by multiplying a value of a point adjacent to the target point or separated from the target point by one or two points by a second weight. 2 is generated, and the second feature of the point around the target point in the target field is searched from the region searched for in the other field with the position specified by the first vector as the center. Quantity and The second feature quantity of the target point, a second feature quantity closest arrangement and values, indicating a position of the second feature amount in the other fields, the second motion as a vector A detection step for detecting a vector;
A class classification step of classifying the attention point into one of a plurality of predetermined classes based on the first feature quantity or the second feature quantity and the motion vector ;
An extraction step for extracting a prediction tap that is predetermined for each classified class and includes the first feature amount or the second feature amount ; and
A normal equation obtained from the component video signal and the extracted prediction tap, and calculating a coefficient of the classified class by solving a normal equation that minimizes a residual. A recording medium on which a computer-readable program is recorded. A computer that controls an image processing device that generates coefficients used in the process of predicting a component video signal from a composite video signal;
Addition or subtraction of the luminance signal and the color signal of the composite video signal with respect to the field of interest and the other field of interest where there is a point of interest that is the point of interest of the composite video signal From the obtained image signal, the values of the points having the same phase as the phase of the color signal at the point of interest, or four consecutive points on one line of each field, each having a different phase Or four points adjacent to each other vertically and horizontally on two lines of the field, each of which is an average value of four points of different phases. Generating a feature amount, and arranging and value the first feature amount of the point in the attention field around the attention point and the first feature amount of the attention point; By detecting the first feature value closest to the first feature value in the other field, the first vector approximate to the motion vector is detected, and the target field and the other field are detected. And a result obtained by multiplying the value of the target point to be subjected to the calculation of the motion vector by a first weight, and a point around the target point in the field where the target point exists, A value corresponding to the target point is a value obtained by adding a value obtained by multiplying a value of a point adjacent to the target point or separated from the target point by one or two points by a second weight. 2 is generated, and the second feature of the point around the target point in the target field is searched from the region searched for in the other field with the position specified by the first vector as the center. Quantity and The second feature quantity of the target point, a second feature quantity closest arrangement and values, indicating a position of the second feature amount in the other fields, the second motion as a vector A detection step for detecting a vector;
A class classification step of classifying the attention point into one of a plurality of predetermined classes based on the first feature quantity or the second feature quantity and the motion vector ;
An extraction step for extracting a prediction tap that is predetermined for each classified class and includes the first feature amount or the second feature amount ; and
A normal equation obtained from the component video signal and the extracted prediction tap, and calculating a coefficient of the classified class by solving a normal equation that minimizes a residual . program.

Images