JP4206730B2 - Image signal processing apparatus and processing method, coefficient data generating apparatus and generating method used therefor, and program for executing each method - Google Patents

Description

【０１０２】
この画像信号処理部１１０の動作を説明する。
クラス生成部１２１では、バッファメモリ１０８に格納されている、ピクチャ情報ＰＩ、残差データおよびリファレンスデータが用いられて、画像信号Ｖｂにおける注目位置の画素データが属する、動き補償用ベクトル情報ＭＩの不正確さに基づくクラスを示すクラスコードＣＬ０を生成される。
【０１０３】
また、クラス分類部１２３では、バッファメモリ１０８に記憶されている画像信号Ｖａを構成する複数の画素データおよびクラス生成部１２１で生成されるクラスコードＣＬ０を用いて、画像信号Ｖｂにおける注目位置の画素データが属するクラスを示すクラスコードＣＬが生成される。
【０１０４】
このようにクラス分類部１２３で生成されるクラスコードＣＬは読み出しアドレス情報として係数メモリ１２４に供給される。これにより、係数メモリ１２４からクラスコードＣＬに対応した係数データＷｉが読み出されて、推定予測演算回路１２５に供給される。
【０１０５】
また、バッファメモリ１０８に記憶されている画像信号Ｖａより、予測タップ選択回路１２２で、画像信号Ｖｂにおける注目位置の周辺に位置する予測タップの画素データが選択的に取り出される。
【０１０６】
推定予測演算回路１２５では、予測タップの画素データｘｉと、係数メモリ１２４より読み出される係数データＷｉとを用いて、上述の（１）式に示す推定式に基づいて、作成すべき画像信号Ｖｂにおける注目位置の画素データｙが求められる。
【０１０７】
このように画像信号処理部１１０では、画像信号Ｖａから係数データＷｉを用いて画像信号Ｖｂが得られる。この場合、画像信号Ｖａに基づいて選択された、画像信号Ｖｂにおける注目位置の周辺に位置する複数の画素データ（予測タップの画素データ）、およびこの画像信号Ｖｂにおける注目位置の画素データが属するクラスＣＬに対応した係数データＷｉを用いて、推定式に基づいて画像信号Ｖｂにおける注目位置の画素データｙを生成するものである。
【０１０８】
したがって、係数データＷｉとして、画像信号Ｖａに対応しこの画像信号Ｖａと同様の符号化雑音を含む生徒信号と画像信号Ｖｂに対応した符号化雑音を含まない教師信号とを用いた学習によって得られた係数データＷｉを用いることで、画像信号Ｖｂとして画像信号Ｖａに比べて符号化雑音が大幅に軽減されたものを良好に得ることができる。
【０１０９】
また、クラス生成部１２１では、動き補償用ベクトル情報ＭＩの不正確さに基づくクラスを示すクラスコードＣＬ０が生成される。そして、クラス分類部１２３では、このクラスコードＣＬ０が他のクラスコードと統合されて、クラスコードＣＬが生成される。そのため、画像信号処理部１１０では、動き補償用ベクトル情報ＭＩの不正確さの情報に基づいてクラス分類が行われることとなり、画像信号Ｖｂの品質の向上を図ることができる。
【０１１０】
また、クラス生成部１２１の相関判定部３７は、最も相関の高い候補ブロックの相関レベルが予め設定された閾値より小さいときは、動きベクトル情報に代えてその旨を示す情報ＮＧを出力する。そして、クラス生成回路３８は、クラスコードＣＬ０としてその情報ＮＧに対応したものを出力する。したがって、クラス生成部１２１から不正確な動きベクトル情報（Δｘ，Δｙ）によるクラスコードＣＬ０が出力されることがなく、クラス分類の精度が低下することを防止できる。
【０１１１】
次に、係数メモリ１２４に記憶される係数データＷｉの生成方法について説明する。この係数データＷｉは、予め学習によって生成されたものである。
【０１１２】
まず、この学習方法について説明する。上述の、（１）式において、学習前は係数データＷ₁，Ｗ₂，‥‥，Ｗ_nは未定係数である。学習は、クラス毎に、複数の信号データに対して行う。学習データ数がｍの場合、（１）式に従って、以下に示す（２）式が設定される。ｎは予測タップの数を示している。
ｙ_k＝Ｗ₁×ｘ_k1＋Ｗ₂×ｘ_k2＋‥‥＋Ｗ_n×ｘ_kn ・・・（２）
（ｋ＝１，２，‥‥，ｍ）
【０１１３】
ｍ＞ｎの場合、係数データＷ₁，Ｗ₂，‥‥，Ｗ_nは、一意に決まらないので、誤差ベクトルｅの要素ｅ_kを、以下の式（３）で定義して、（４）式のｅ²を最小にする係数データを求める。いわゆる最小２乗法によって係数データを一意に定める。
ｅ_k＝ｙ_k−｛Ｗ₁×ｘ_k1＋Ｗ₂×ｘ_k2＋‥‥＋Ｗ_n×ｘ_kn｝ ・・・（３）
（ｋ＝１，２，‥‥ｍ）
【０１１４】
【数２】

【０１１５】
（４）式のｅ²を最小とする係数データを求めるための実際的な計算方法としては、まず、（５）式に示すように、ｅ²を係数データＷｉ(ｉ＝１，２，・・・，ｎ）で偏微分し、ｉの各値について偏微分値が０となるように係数データＷｉを求めればよい。
【０１１６】
【数３】

【０１１７】
（５）式から係数データＷｉを求める具体的な手順について説明する。（６）式、（７）式のようにＸji，Ｙiを定義すると、（５）式は、（８）式の行列式の形に書くことができる。
【０１１８】
【数４】

【０１１９】
【数５】

【０１２０】
（８）式は、一般に正規方程式と呼ばれるものである。この正規方程式を掃き出し法（Gauss-Jordanの消去法）等の一般解法で解くことにより、係数データＷｉ（ｉ＝１，２，・・・，ｎ）を求めることができる。
【０１２１】
図９は、図１の画像信号処理部１１０の係数メモリ１２４に格納すべき係数データＷｉを生成する係数データ生成装置１５０の構成を示している。
この係数データ生成装置１５０は、画像信号Ｖｂに対応した教師信号ＳＴが入力される入力端子１５１と、この教師信号ＳＴに対して符号化を行ってＭＰＥＧ２ストリームを得るＭＰＥＧ２符号化器１５２と、このＭＰＥＧ２ストリームに対して復号化を行って画像信号Ｖａに対応した生徒信号ＳＳを得るＭＰＥＧ２復号化器１５３とを有している。ここで、ＭＰＥＧ２復号化器１５３は、図１に示すデジタル放送受信機１００におけるＭＰＥＧ２復号化器１０７およびバッファメモリ１０８に対応したものである。
【０１２２】
また、係数データ生成装置１５０は、クラス生成部１５４を有している。このクラス生成部１５４は、上述した画像信号処理部１１０のクラス生成部１２１と同様に構成され、教師信号ＳＴにおける注目位置の画素データが属する、動き補償用ベクトル情報ＭＩの不正確さに基づくクラスを示すクラスコードＣＬ０を生成する。このクラス生成部１５４では、復号化器１５３より出力されるピクチャ情報ＰＩ、残差データおよびリファレンスデータが用いられて、クラスコードＣＬ０が生成される。
【０１２３】
また、係数データ生成装置１５０は、ＭＰＥＧ２復号化器１５３より出力される生徒信号ＳＳより、教師信号ＳＴにおける注目位置の周辺に位置する複数の画素データを選択的に取り出して出力する予測タップ選択回路１５５を有している。この予測タップ選択回路１５５は、上述した画像信号処理部１１０の予測タップ選択回路１２２と同様に構成される。
【０１２４】
また、係数データ生成装置１５０は、教師信号ＳＴにおける注目位置の画素データが属するクラスを検出するクラス検出手段としてのクラス分類部１５６を有している。このクラス分類部１５６は、上述した画像信号処理部１１０のクラス分類部１２３と同様に構成される。
【０１２５】
このクラス分類部１５６は、ＭＰＥＧ２復号化器１５３より得られる生徒信号ＳＳを構成する複数の画素データおよびクラス生成部１５４で生成されるクラスコードＣＬ０を用いて、教師信号ＳＴにおける注目位置の画素データが属するクラスを示すクラスコードＣＬを生成する。
【０１２６】
また、係数データ生成装置１５０は、入力端子１５１に供給される教師信号ＳＴの時間調整を行うための遅延回路１５７と、この遅延回路１５７で時間調整された教師信号ＳＴより得られる各注目位置の画素データｙと、この各注目位置の画素データｙにそれぞれ対応して予測タップ選択回路１５５で選択的に取り出される予測タップの画素データｘｉと、各注目位置の画素データｙにそれぞれ対応してクラス分類部１５６で生成されるクラスコードＣＬとから、クラス毎に、係数データＷｉ（ｉ＝１〜ｎ）を得るための正規方程式（上述の（８）式参照）を生成する正規方程式生成部１５８を有している。
【０１２７】
この場合、１個の画素データｙとそれに対応するｎ個の予測タップの画素データｘｉとの組み合わせで１個の学習データが生成されるが、教師信号ＳＴと生徒信号ＳＳとの間で、クラス毎に、多くの学習データが生成されていく。これにより、正規方程式生成部１５８では、クラス毎に、係数データＷｉ（ｉ＝１〜ｎ）を得るための正規方程式が生成される。
【０１２８】
また、係数データ生成装置１５０は、正規方程式生成部１５８で生成された正規方程式のデータが供給され、その正規方程式を解いて、各クラスの係数データＷｉを求める係数データ決定部１５９と、この求められた各クラスの係数データＷｉを格納する係数メモリ１６０とを有している。
【０１２９】
次に、図９に示す係数データ生成装置１５０の動作を説明する。
入力端子１５１には画像信号Ｖｂに対応した教師信号ＳＴが供給され、そしてＭＰＥＧ２符号化器１５２で、この教師信号ＳＴに対して符号化が施されて、ＭＰＥＧ２ストリームが生成される。このＭＰＥＧ２ストリームは、ＭＰＥＧ２復号化器１５３に供給される。ＭＰＥＧ２復号化器１５３で、このＭＰＥＧ２ストリームに対して復号化が施されて、画像信号Ｖａに対応した生徒信号ＳＳが生成される。
【０１３０】
クラス生成部１５４では、復号化器１５３より出力されるピクチャ情報ＰＩ、残差データおよびリファレンスデータを用いて、教師信号ＳＴにおける注目位置の画素データが属する、動き補償用ベクトル情報ＭＩの不正確さに基づくクラスを示すクラスコードＣＬ０が生成される。
【０１３１】
クラス分類部１５６では、ＭＰＥＧ２復号化器１５３より得られる生徒信号ＳＳを構成する複数の画素データおよびクラス生成部１５４で生成されたクラスコードＣＬ０が用いて、教師信号ＳＴにおける注目位置の画素データが属するクラスを示すクラスコードＣＬが生成される。
【０１３２】
また、ＭＰＥＧ２復号化器１５３より得られる生徒信号ＳＳより、予測タップ選択回路１５５で、教師信号ＳＴにおける注目位置の周辺に位置する予測タップの画素データが選択的に取り出される。
【０１３３】
そして、遅延回路１５７で時間調整された教師信号ＳＴから得られる各注目位置の画素データｙと、この各注目位置の画素データｙにそれぞれ対応して予測タップ選択回路１５５で選択的に取り出される予測タップの画素データｘｉと、各注目位置の画素データｙにそれぞれ対応してクラス分類部１５６で生成されるクラスコードＣＬとを用いて、正規方程式生成部１５８では、クラス毎に、係数データＷｉ（ｉ＝１〜ｎ）を得るための正規方程式（（８）式参照）が生成される。この正規方程式は係数データ決定部１５９で解かれて各クラスの係数データＷｉが求められ、その係数データＷｉは係数メモリ１６０に格納される。
【０１３４】
このように、図９に示す係数データ生成装置１５０においては、図１の画像信号処理部１１０の係数メモリ１２４に格納される各クラスの係数データＷｉを生成することができる。
【０１３５】
生徒信号ＳＳは、教師信号ＳＴに対して符号化を施してＭＰＥＧ２ストリームを生成し、その後このＭＰＥＧ２ストリームに対して復号化を施して得たものである。したがって、この生徒信号ＳＳは、画像信号Ｖａと同様の符号化雑音を含んだものとなる。そのため、図１に示す画像信号処理部１１０において、画像信号Ｖａからこの係数データＷｉを用いて得られる画像信号Ｖｂは、画像信号Ｖａに比べて符号化雑音が軽減されたものとなる。
【０１３６】
なお、図１の画像信号処理部１１０における処理を、例えば図１０に示すような画像信号処理装置３００によって、ソフトウェアで実現することも可能である。
【０１３７】
まず、図１０に示す画像信号処理装置３００について説明する。この画像信号処理装置３００は、装置全体の動作を制御するＣＰＵ３０１と、このＣＰＵ３０１の制御プログラムや係数データ等が格納されたＲＯＭ（Read Only Memory）３０２と、ＣＰＵ３０１の作業領域を構成するＲＡＭ（Random Access Memory）３０３とを有している。これらＣＰＵ３０１、ＲＯＭ３０２およびＲＡＭ３０３は、それぞれバス３０４に接続されている。
【０１３８】
また、画像信号処理装置３００は、外部記憶装置としてのハードディスクドライブ（ＨＤＤ）３０５と、フロッピー（登録商標）ディスク３０６をドライブするドライブ（ＦＤＤ）３０７とを有している。これらドライブ３０５，３０７は、それぞれバス３０４に接続されている。
【０１３９】
また、画像信号処理装置３００は、インターネット等の通信網４００に有線または無線で接続する通信部３０８を有している。この通信部３０８は、インタフェース３０９を介してバス３０４に接続されている。
【０１４０】
また、画像信号処理装置３００は、ユーザインタフェース部を備えている。このユーザインタフェース部は、リモコン送信機２００からのリモコン信号ＲＭを受信するリモコン信号受信回路３１０と、ＬＣＤ（liquid Crystal Display）等からなるディスプレイ３１１とを有している。受信回路３１０はインタフェース３１２を介してバス３０４に接続され、同様にディスプレイ３１１はインタフェース３１３を介してバス３０４に接続されている。
【０１４１】
また、画像信号処理装置３００は、画像信号Ｖａを入力するための入力端子３１４と、画像信号Ｖｂを出力するための出力端子３１５とを有している。入力端子３１４はインタフェース３１６を介してバス３０４に接続され、同様に出力端子３１５はインタフェース３１７を介してバス３０４に接続される。
【０１４２】
ここで、上述したようにＲＯＭ３０２に制御プログラムや係数データ等を予め格納しておく代わりに、例えばインターネットなどの通信網４００より通信部３０８を介してダウンロードし、ハードディスクやＲＡＭ３０３に蓄積して使用することもできる。また、これら制御プログラムや係数データ等をフロッピー（登録商標）ディスク３０６で提供するようにしてもよい。
【０１４３】
また、処理すべき画像信号Ｖａを入力端子３１４より入力する代わりに、予めハードディスクに記録しておき、あるいはインターネットなどの通信網４００より通信部３０８を介してダウンロードしてもよい。また、処理後の画像信号Ｖｂを出力端子３１５に出力する代わり、あるいはそれと並行してディスプレイ３１１に供給して画像表示をしたり、さらにはハードディスクに格納したり、通信部３０８を介してインターネットなどの通信網４００に送出するようにしてもよい。
【０１４４】
図１１のフローチャートを参照して、図１０に示す画像信号処理装置３００における、画像信号Ｖａより画像信号Ｖｂを得るため処理手順を説明する。
まず、ステップＳＴ２１で、処理を開始し、ステップＳ２２で、例えば入力端子３１４より装置内に１フレーム分または１フィールド分の画像信号Ｖａを入力する。この場合、画像信号Ｖａの各画素データと対となっているピクチャ情報ＰＩも入力する。ピクチャ情報ＰＩは、画素データがＩピクチャ、Ｐピクチャ、Ｂピクチャのいずれのピクチャに係るものであったかを示す情報である。またこの場合、画像信号Ｖａの各画素データと対となっている、その画素データを得る際に使用された残差データおよびリファレンスデータも入力する。
【０１４５】
このように入力端子３１４より入力される画像信号Ｖａ等はＲＡＭ３０３に一時的に格納される。なお、この画像信号Ｖａ等が装置内のハードディスクドライブ３０５に予め記録されている場合には、このドライブ３０５からこの画像信号Ｖａ等を読み出し、この画像信号Ｖａ等をＲＡＭ３０３に一時的に格納する。
【０１４６】
そして、ステップＳＴ２３で、画像信号Ｖａの全フレームまたは全フィールドの処理が終わっているか否かを判定する。処理が終わっているときは、ステップＳＴ２４で、処理を終了する。一方、処理が終わっていないときは、ステップＳＴ２５に進む。
【０１４７】
ステップＳＴ２５では、画像信号Ｖｂにおける注目位置に対応した画像信号Ｖａの画素データと対となっているピクチャ情報ＰＩ、残差データおよびリファレンスデータを用いて、画像信号Ｖｂにおける注目位置の画素データが属する、動き補償用ベクトル情報ＭＩの不正確さに基づくクラスを示すクラスコードＣＬ０を生成し、さらにこのクラスコードＣＬ０および画像信号Ｖａを構成する複数の画素データを用いて、画像信号Ｖｂにおける注目位置の画素データが属するクラスを示すクラスコードＣＬを生成する。
【０１４８】
次に、ステップＳＴ２６で、ステップＳＴ２２で入力された画像信号Ｖａより、画像信号Ｖｂにおける注目位置の周辺に位置する複数の画素データ（予測タップの画素データ）を取得する。そして、ステップＳＴ２７で、ステップＳＴ２５で生成されたクラスコードＣＬに対応した係数データＷｉとステップＳＴ２６で取得された予測タップの画素データｘｉを使用して、（１）式の推定式に基づいて、画像信号Ｖｂにおける注目位置の画素データｙを生成する。
【０１４９】
次に、ステップＳＴ２８で、ステップＳＴ２２で入力された１フレームまたは１フィールド分の画像信号Ｖａの画素データの全領域において画像信号Ｖｂの画素データを得る処理が終了したか否かを判定する。終了しているときは、ステップＳＴ２２に戻り、次の１フレーム分または１フィールド分の画像信号Ｖａの入力処理に移る。一方、処理が終了していないときは、ステップＳＴ２５に戻って、次の注目位置についての処理に移る。
【０１５０】
このように、図１１に示すフローチャートに沿って処理をすることで、入力された画像信号Ｖａの画素データを処理して、画像信号Ｖｂの画素データを得ることができる。上述したように、このように処理して得られた画像信号Ｖｂは出力端子３１５に出力されたり、ディスプレイ３１１に供給されてそれによる画像が表示されたり、さらにはハードディスクドライブ３０５に供給されてハードディスクに記録されたりする。
【０１５１】
また、処理装置の図示は省略するが、図９の係数データ生成装置１５０における処理も、ソフトウェアで実現可能である。
【０１５２】
図１２のフローチャートを参照して、係数データを生成するための処理手順を説明する。
まず、ステップＳＴ３１で、処理を開始し、ステップＳＴ３２で、教師信号ＳＴを１フレーム分または１フィールド分だけ入力する。そして、ステップＳＴ３３で、教師信号ＳＴの全フレームまたは全フィールドの処理が終了したか否かを判定する。終了していないときは、ステップＳＴ３４で、ステップＳＴ３２で入力された教師信号ＳＴから生徒信号ＳＳを生成する。
【０１５３】
この場合、生徒信号ＳＳの各画素データと対となっているピクチャ情報ＰＩ、さらには各画素データと対となっているその画素データを得る際に使用された残差データおよびリファレンスデータも得るようにする。
【０１５４】
そして、ステップＳＴ３５で、教師信号ＳＴの注目位置に対応した生徒信号ＳＳの画素データと対となっているピクチャ情報ＰＩと、残差データおよびリファレンスデータを用いて、教師信号ＳＴにおける注目位置の画素データが属する、動き補償用ベクトル情報ＭＩの不正確さに基づくクラスを示すクラスコードＣＬ０を生成し、さらにこのクラスコードＣＬ０および生徒信号ＳＳを構成する複数の画素データを用いて、教師信号ＳＴにおける注目位置の画素データが属するクラスを示すクラスコードＣＬを生成する。
【０１５５】
次に、ステップＳＴ３６で、ステップＳＴ３４で生成された生徒信号ＳＳより、教師信号ＳＴにおける注目位置の周辺に位置する複数の画素データ（予測タップの画素データ）を取得する。
【０１５６】
そして、ステップＳＴ３７で、ステップＳＴ３５で生成されたクラスコードＣＬ、ステップＳＴ３６で取得された予測タップの画素データｘｉおよび教師信号ＳＴにおける注目位置の画素データｙを用いて、クラス毎に、（８）式に示す正規方程式を得るための加算をする（（６）式、（７）式参照）。
【０１５７】
次に、ステップＳＴ３８で、ステップＳＴ３２で入力された１フレーム分または１フィールド分の教師信号ＳＴの画素データの全領域において学習処理が終了したか否かを判定する。学習処理を終了しているときは、ステップＳＴ３２に戻って、次の１フレーム分または１フィールド分の教師信号ＳＴの入力を行って、上述したと同様の処理を繰り返す。一方、学習処理を終了していないときは、ステップＳＴ３５に戻って、次の注目位置についての処理に移る。
【０１５８】
上述したステップＳＴ３３で、処理が終了したときは、ステップＳＴ３９で、上述のステップＳＴ３７の加算処理によって生成された、各クラスの正規方程式を掃き出し法などで解いて、各クラスの係数データＷｉを算出する。そして、ステップＳＴ４０で、各クラスの係数データＷｉをメモリに保存し、その後にステップＳＴ４１で、処理を終了する。
【０１５９】
このように、図１２に示すフローチャートに沿って処理をすることで、図９に示す係数データ生成装置１５０と同様の手法によって、各クラスの係数データＷｉを得ることができる。
【０１６０】
なお、上述実施の形態においては、リファレンスデータから抽出されたエッジ成分と残差データから抽出されたエッジ成分とを用いて動きベクトル情報（Δｘ，Δｙ）を検出し、これを復号時に使用した動き補償用ベクトル情報ＭＩの不正確さを示す情報としたものである。
【０１６１】
しかし、動き補償用ベクトル情報ＭＩの不正確さを示す情報はこの動きベクトル情報（Δｘ，Δｙ）に限定されるものではない。例えば、画像信号Ｖａからブロックマッチング法、勾配法等の方法によって動きベクトル情報を新たに検出し、これと復号時に使用した動き補償用ベクトル情報ＭＩとを比較することで、動き補償用ベクトル情報ＭＩの不正確さを示す情報を得るようにしてもよい。
【０１６２】
また、上述実施の形態においては、ＤＣＴを伴うＭＰＥＧ２ストリームを取り扱うものを示したが、この発明は、動き補償予測符号化が行われたその他の符号化されたデジタル画像信号を取り扱うものにも同様に適用することができる。また、ＤＣＴの代わりに、ウォーブレット変換、離散サイン変換などのその他の直交変換を伴う符号化であってもよい。
【０１６３】
【発明の効果】
この発明によれば、動き補償予測符号化が行われたデジタル画像信号を復号化することによって生成される、複数の画素データからなる第１の画像信号を、複数の画素データからなる符号化雑音が軽減された第２の画像信号に変換する際、第２の画像信号における注目位置に対応した第１の画像信号の画素データを得る際に用いられたリファレンスデータの動き補償に使用された第１の動きベクトル情報の不正確さを示す情報を取得し、少なくともこの不正確さを示す情報を用いて第２の画像信号における注目位置の画素データが属するクラスを検出し、このクラスに対応して第２の画像信号における注目位置の画素データを生成するものであり、クラス分類の精度を上げることができ、第２の画像信号の品質の向上を図ることができる。
【図面の簡単な説明】
【図１】実施の形態としてのデジタル放送受信機の構成を示すブロック図である。
【図２】ＭＰＥＧ２復号化器の構成を示すブロック図である。
【図３】クラス生成部の構成を示すブロック図である。
【図４】３×３のラプラシアンフィルタの係数比の一例を示す図である。
【図５】相関判定のためのブロックマッチングの一例を説明するための図である。
【図６】相関判定のためのブロックマッチングの他の例を説明するための図である。
【図７】クラス分類部の構成を示すブロック図である。
【図８】タップ選択用ブロックを示す図である。
【図９】係数データ生成装置の構成を示すブロック図である。
【図１０】ソフトウェアで実現するための画像信号処理装置の構成例を示すブロック図である。
【図１１】画像信号処理を示すフローチャートである。
【図１２】係数データ生成処理を示すフローチャートである。
【符号の説明】
３１，３２・・・入力端子、３２，３５・・・エッジ成分抽出回路、３３，３６・・・バッファメモリ、３７・・・相関判定部、３８・・・クラス生成回路、３９・・・出力端子、１００・・・デジタル放送受信機、１０１・・・システムコントローラ、１０２・・・リモコン信号受信回路、１０５・・・受信アンテナ、１０６・・・チューナ部、１０７・・・ＭＰＥＧ２復号化器、１０８・・・バッファメモリ、１１０・・・画像信号処理部、１１１・・・ディスプレイ部、１２１・・・クラス生成部、１２２・・・予測タップ選択回路、１２３・・・クラス分類部、１２４・・・係数メモリ、１２５・・・推定予測演算回路、１５０・・・係数データ生成装置、３００・・・画像信号処理装置[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image signal processing device and processing method, a coefficient data generating device and generating method used therefor, and a program for executing each method.
[0002]
More specifically, the present invention encodes a first image signal composed of a plurality of pixel data, which is generated by decoding a digital image signal subjected to motion compensation predictive coding, into a plurality of pixel data. Used for motion compensation of reference data used when obtaining pixel data of the first image signal corresponding to the position of interest in the second image signal when converting to the second image signal with reduced noise Acquire information indicating the inaccuracy of the first motion vector information, detect a class to which the pixel data of the target position in the second image signal belongs using at least the information indicating the inaccuracy, and correspond to this class Then, by generating pixel data at the position of interest in the second image signal, the image signal processing is designed to improve the accuracy of the classification and improve the quality of the second image signal. It is those related to 置等.
[0003]
[Prior art]
As an image signal compression encoding method, there is an MPEG2 (Moving Picture Experts Group 2) encoding method using DCT (Discrete Cosine Transform). In this encoding method, motion compensation prediction encoding is performed for each block.
[0004]
DCT performs discrete cosine transform on pixels in a block, requantizes coefficient data obtained by the discrete cosine transform, and further variable-length codes the requantized coefficient data. For this variable length coding, entropy coding such as Huffman code is often used. The image signal is orthogonally transformed to be divided into a large number of frequency data from low frequency to high frequency.
[0005]
When re-quantization is performed on this divided frequency data, human visual characteristics are taken into consideration, and low frequency data with high importance is finely quantized, and high frequency data with low importance is roughly quantized. In this way, the image quality can be maintained and high-efficiency compression can be realized.
[0006]
In conventional decoding using DCT, quantized data for each frequency component is converted into a representative value of the code, and reproduction data is obtained by performing inverse DCT (IDCT: Inverce DCT) on these components. . When converting to this representative value, the quantization step width at the time of encoding is used.
[0007]
[Problems to be solved by the invention]
As described above, the MPEG encoding method using DCT has the advantage that high-quality compression can be realized while maintaining high image quality by performing encoding in consideration of human visual characteristics.
[0008]
However, since coding for performing DCT is processing in units of blocks, block noise, so-called block noise (block distortion), may occur as the compression rate increases. In addition, in a portion where there is a rapid luminance change such as an edge, a noise that is a result of coarse quantization of high-frequency components, so-called mosquito noise, is generated.
[0009]
It is conceivable to reduce coding noise such as block noise and mosquito noise by class classification adaptive processing. That is, an image signal including coding noise is set as a first image signal, an image signal with reduced coding noise is set as a second image signal, and a class to which pixel data of a target position in the second image signal belongs is detected. Then, pixel data of the target position in the second image signal is generated corresponding to this class. In this case, in order to improve the quality of the second image signal, it is necessary to increase the accuracy of the classification.
[0010]
Of the pixel data of the first image signal corresponding to the target position in the second image signal, the pixel data related to the P picture and the B picture is subjected to motion compensation on the residual data obtained by performing DCT conversion. It is generated by adding the reference data motion-compensated with the vector information.
[0011]
Therefore, the pixel data related to the P picture and B picture is affected by the inaccuracy of the motion compensation vector information. Therefore, if class classification is performed based on inaccuracy information of the motion compensation vector information, it is considered that the accuracy of class classification is improved.
[0012]
An object of the present invention is to improve the classification accuracy and improve the quality of the second image signal.
[0013]
[Means for Solving the Problems]
An image signal processing apparatus according to the present invention is configured to decode a first image signal composed of a plurality of pixel data, which is generated by decoding a digital image signal subjected to motion compensation prediction encoding, Reduced coding noise An image signal processing device for converting to a second image signal composed of a plurality of pixel data, First extraction means for extracting an edge component from residual data used when obtaining pixel data of the first image signal corresponding to a target position in the second image signal; Reference data used when obtaining pixel data of the first image signal corresponding to the position of interest in the second image signal A frame composed of the edge component extracted from the second extraction means, and a frame composed of the edge component extracted by the first extraction means as a first frame. And second motion vector information corresponding to the position of interest in the second image signal is obtained from the correlation information between the block of the first frame and the block of the second frame. Motion vector acquisition means for acquiring the second motion vector information as information indicating inaccuracy of the first motion vector information used for motion compensation of the reference data; At least above Motion vector acquisition means Obtained in From the second motion vector information The class to which the pixel data of the target position in the second image signal belongs is Generation Class Generation Means, A teacher signal that is obtained in advance for each class generated by the class generating means and includes a student signal including encoding noise corresponding to the first image signal and an encoding noise corresponding to the second image signal. Coefficient data generating means for generating coefficient data for minimizing an error from the signal, and data selection for selecting a plurality of pixel data located around the position of interest in the second image signal from the first image signal And calculation means for calculating the coefficient data generated by the coefficient data generation means and the plurality of pixel data selected by the data selection means to obtain pixel data of the target position in the second image signal And with Ru Is.
[0014]
In addition, the image signal processing method according to the present invention provides a first image signal composed of a plurality of pixel data, which is generated by decoding a digital image signal subjected to motion compensation predictive coding, Reduced coding noise An image signal processing method for converting to a second image signal composed of a plurality of pixel data, A first step of extracting an edge component from residual data used when obtaining pixel data of the first image signal corresponding to a target position in the second image signal; Reference data used when obtaining pixel data of the first image signal corresponding to the position of interest in the second image signal A second step of extracting an edge component from the first step and a frame composed of the edge component extracted in the first step as a first frame, and a frame composed of the edge component extracted in the second step as a second frame. Second motion vector information corresponding to the position of interest in the second image signal is obtained from the correlation information between the block of the first frame and the block of the second frame, and the reference data A third step of acquiring the second motion vector information as information indicating inaccuracy of the first motion vector information used for motion compensation of At least above Third step Obtained in From the second motion vector information The class to which the pixel data of the target position in the second image signal belongs is Generation First 4 And the steps A student signal that is obtained in advance for each class generated in the fourth step and includes coding noise corresponding to the first image signal and does not include coding noise corresponding to the second image signal. A fifth step of generating coefficient data that minimizes an error from the teacher signal; and a plurality of pixel data positioned around the target position in the second image signal from the first image signal. And calculating the coefficient data generated in step 6 and the fifth step and the plurality of pixel data selected in the sixth step to obtain pixel data of the target position in the second image signal. Steps And with Ru Is.
[0015]
A program according to the present invention is for causing a computer to execute the above-described image signal processing method.
[0016]
In the present invention, the first image signal composed of a plurality of pixel data is generated by decoding a digital image signal that has been subjected to motion compensation predictive coding. For example, the digital image signal has been subjected to MPEG encoding.
[0017]
The pixel data of the first image signal corresponding to the target position in the second image signal is obtained by adding the reference data compensated for motion with the first motion vector information to the residual data corresponding to the target position. Generated. Information indicating the inaccuracy of the first motion vector information is acquired.
[0018]
For example, information indicating this inaccuracy is acquired as follows. That is, an edge component is extracted from the residual data used when obtaining the pixel data of the first image signal, and a frame including the edge component is set as the first frame. In addition, an edge component is extracted from the reference data used when obtaining the pixel data of the first image signal, and a frame including the edge component is set as a second frame. Then, using these first and second frames, second motion vector information corresponding to the position of interest in the second image signal is detected, and this is information indicating inaccuracy.
[0019]
In this case, the second motion vector information is obtained as follows, for example. That is, the first or second frame is a reference frame, and the second or first frame is a search frame. A reference block including an edge component corresponding to the target position in the second image signal of the reference frame; and a plurality of candidate blocks within a predetermined search range centered on the target position in the second image signal of the search frame; Correlation information between is detected. Then, based on the correlation information corresponding to the plurality of detected candidate blocks, the position information of the candidate block having the highest correlation with the reference block is output as the second motion vector information.
[0020]
The class to which the pixel data at the target position in the second image signal belongs is detected using at least information indicating the inaccuracy of the first motion vector information. As information indicating this inaccuracy, in the case of outputting the second motion vector information as described above, for example, the correlation level indicated by the correlation information of the candidate block with the highest correlation is set in advance. When the value is smaller than the threshold value, predetermined information indicating that fact is output. In that case, the class to which the pixel data of the target position in the second image signal belongs is detected using this predetermined information instead of the second motion vector information. In this case, since the inaccuracy of the second motion vector information itself is large, class classification using the second motion vector information itself causes a decrease in accuracy of class classification.
[0021]
Corresponding to the class detected in this way, pixel data of the target position in the second image signal is generated. For example, pixel data is generated as follows. That is, coefficient data used in the estimation formula corresponding to the class is generated. Further, based on the first image signal, a plurality of pixel data located around the target position in the second image signal are selected. Then, these coefficient data and a plurality of pixel data are used, and pixel data of the target position in the second image signal is calculated based on the estimation formula.
[0022]
As described above, the inaccuracy of the first motion vector information used for the motion compensation of the reference data used when obtaining the pixel data of the first image signal corresponding to the target position in the second image signal is determined. Information indicating that the pixel data of the target position in the second image signal belongs is detected using at least the information indicating the inaccuracy, and the target position of the second image signal corresponding to this class is detected. Pixel data is generated, the classification accuracy can be improved, and the quality of the second image signal can be improved.
[0023]
A coefficient data generation device according to the present invention is configured to generate a first image signal composed of a plurality of pixel data, which is generated by decoding a digital image signal subjected to motion compensation prediction encoding, Reduced coding noise Used when converting to a second image signal consisting of multiple pixel data Person in charge Decoding a digital image signal obtained by encoding a teacher signal corresponding to the second image signal to obtain a student signal corresponding to the first image signal And First extraction means for extracting an edge component from residual data used when obtaining pixel data of the student signal corresponding to a target position in the teacher signal; Reference data used to obtain pixel data of the student signal corresponding to the position of interest in the teacher signal A frame composed of the edge component extracted from the second extraction means, and a frame composed of the edge component extracted by the first extraction means as a first frame. Is the second frame, second motion vector information corresponding to the position of interest in the teacher signal is obtained from the correlation information between the block of the first frame and the block of the second frame, and the reference data Motion vector acquisition means for acquiring the second motion vector information as information indicating inaccuracy of the first motion vector information used for motion compensation of At least above Motion vector acquisition means Obtained in From the second motion vector information , The class to which the pixel data of the target position in the teacher signal belongs Generation Class Generation Means and student signal above From , A data selection means for selecting a plurality of pixel data located around a target position in the teacher signal, and the class Generation By means Generation Class data, a plurality of pixel data selected by the data selection means, and pixel data of a target position in the teacher signal To above For each class, Minimizing an error between a plurality of pixel data related to the student signal and pixel data at a target position in the teacher signal And calculation means for obtaining coefficient data Ru Is.
[0024]
Further, the coefficient data generation method according to the present invention provides a first image signal made up of a plurality of pixel data, which is generated by decoding a digital image signal that has been subjected to motion compensation prediction encoding, Reduced coding noise Used when converting to a second image signal consisting of multiple pixel data Person in charge A method of generating numerical data, wherein a digital image signal obtained by encoding a teacher signal corresponding to the second image signal is decoded to obtain a student signal corresponding to the first image signal. 1 step, A second step of extracting an edge component from the residual data used when obtaining the pixel data of the student signal corresponding to the target position in the teacher signal; Reference data used to obtain pixel data of the student signal corresponding to the position of interest in the teacher signal A frame composed of the edge component extracted in the third step and the edge component extracted in the second step is defined as a first frame, and a frame composed of the edge component extracted in the third step is defined as a second frame. Second motion vector information corresponding to the position of interest in the teacher signal is obtained from correlation information between the block of the first frame and the block of the second frame, and motion compensation of the reference data is performed. A fourth step of acquiring the second motion vector information as information indicating inaccuracy of the first motion vector information used for At least above Fourth step Obtained in From the second motion vector information , The class to which the pixel data of the target position in the teacher signal belongs Generation First 5 Steps and student signal above From Selecting a plurality of pixel data located around the target position in the teacher signal; 6 And the above steps 5 In steps Generation Class, above 6 A plurality of pixel data selected in the above step and pixel data of the target position in the teacher signal To above For each class, Minimizing an error between a plurality of pixel data related to the student signal and pixel data at a target position in the teacher signal Find coefficient data 7 With steps Ru Is.
[0025]
A program according to the present invention is for causing a computer to execute the coefficient data generation method described above.
[0026]
In the present invention, the first image signal composed of a plurality of pixel data is generated by decoding a digital image signal that has been subjected to motion compensation predictive coding. The present invention generates coefficient data of an estimation formula used when converting the first image signal into a second image signal composed of a plurality of pixel data.
[0027]
The pixel data of the student signal corresponding to the attention position in the teacher signal is generated by adding the reference data compensated for motion by the first motion vector information to the residual data corresponding to the attention position. Information indicating the inaccuracy of the first motion vector information is acquired. The class to which the pixel data at the target position in the teacher signal belongs is detected using at least information indicating the inaccuracy.
[0028]
Further, based on the student signal, a plurality of pixel data located around the attention position in the teacher signal are selected. Then, coefficient data is obtained for each class using the class to which the pixel data at the target position in the teacher signal belongs, the selected pixel data, and the pixel data at the target position in the teacher signal.
[0029]
As described above, coefficient data of the estimation formula used when converting the first image signal to the second image signal is generated. When converting from the first image signal to the second image signal, The coefficient data corresponding to the class to which the pixel data of the target position in the second image signal belongs is selectively used to calculate the pixel data of the target position in the second image signal by the estimation formula.
[0030]
Thereby, when converting from the first image signal to the second image signal using the estimation formula, the accuracy of the class classification can be improved, and the quality of the second image signal can be improved. it can.
[0031]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows a configuration of a digital broadcast receiver 100 as an embodiment.
[0032]
The digital broadcast receiver 100 includes a microcomputer, and includes a system controller 101 for controlling the operation of the entire system, and a remote control signal receiving circuit 102 for receiving a remote control signal RM. The remote control signal receiving circuit 102 is connected to the system controller 101, receives a remote control signal RM output from the remote control transmitter 200 according to a user operation, and supplies an operation signal corresponding to the signal RM to the system controller 101. Is configured to do.
[0033]
Also, the digital broadcast receiver 100 is supplied with a receiving antenna 105 and a broadcast signal (RF modulated signal) captured by the receiving antenna 105, performs channel selection processing, demodulation processing, error correction processing, etc. And a tuner unit 106 that obtains an MPEG2 stream as an encoded image signal.
[0034]
The digital broadcast receiver 100 also temporarily decodes the MPEG2 decoder 107 that decodes the MPEG2 stream output from the tuner unit 106 to obtain the image signal Va, and the image signal Va output from the MPEG2 decoder 107. A buffer memory 108 for storing the data.
[0035]
In the present embodiment, the MPEG2 decoder 107 also outputs picture information PI in addition to each pixel data constituting the image signal Va. The buffer memory 108 also stores picture information PI in pairs with each pixel data. The picture information PI is information indicating whether the output pixel data relates to an I picture (Intra-Picture), a P picture (Predictive-Picture), or a B picture (Bidirectionally predictive-Picture).
[0036]
Further, from the MPEG2 decoder 107, residual data and reference data used in obtaining the pixel data corresponding to the pixel data related to the P picture and the B picture among the pixel data constituting the image signal Va. Is also output. In this case, the reference data has been subjected to motion compensation based on the motion compensation vector information MI. In the buffer memory 108, residual data and reference data are also stored in pairs with the pixel data of the P picture and the B picture.
[0037]
Here, in the case of a P picture, prediction encoding is performed in the forward direction, so that reference data exists only in the forward direction corresponding to one pixel data, but in the case of a B picture, prediction from both directions is performed. Since it is encoding, reference data exists in each direction corresponding to one pixel data. Incidentally, in the case of a B picture, the addition average value of the reference data in each of these directions becomes the reference data Vref to be added to the residual data.
[0038]
FIG. 2 shows the configuration of the MPEG2 decoder 107.
The decoder 107 has an input terminal 71 to which an MPEG2 stream is input, and a stream buffer 72 that temporarily stores the MPEG2 stream input to the input terminal 71.
[0039]
The decoder 107 extracts the DCT (Discrete Cosine Transform) coefficient as a frequency coefficient from the MPEG2 stream stored in the stream buffer 72 and the extraction circuit 73 extracts the DCT coefficient. A variable-length decoding circuit 74 that performs variable-length decoding on DCT coefficients that have been subjected to variable-length coding, for example, Huffman coding.
[0040]
The decoder 107 also includes an extraction circuit 75 that extracts quantization characteristic designation information QI from the MPEG2 stream stored in the stream buffer 72, and a variable length decoding circuit based on the quantization characteristic designation information QI. 74 has an inverse quantization circuit 76 that performs inverse quantization on the quantized DCT coefficient output from 74, and an inverse DCT circuit 77 that performs inverse DCT on the DCT coefficient output from the inverse quantization circuit 76. is doing.
[0041]
Further, the decoder 107 stores the pixel data of the I picture and the P picture in a memory (not shown), and the residual data of the P picture or the B picture is obtained from the inverse DCT circuit 77 using these pixel data. A prediction memory circuit 78 that generates and outputs corresponding reference data Vref when output is provided.
[0042]
In addition, when the P-picture or B-picture residual data is output from the inverse DCT circuit 77, the decoder 107 adds the reference data Vref generated by the prediction memory circuit 78 to the residual data. And an output terminal 81 that outputs pixel data of each picture output from the adder circuit 79 as an image signal Va. When the pixel data of the I picture is output from the inverse DCT circuit 77, the reference data Vref is not supplied from the prediction memory circuit 78 to the adder circuit 79. Therefore, the I circuit output from the inverse DCT circuit 77 from the adder circuit 79. The pixel data of the picture is output as it is.
[0043]
Here, in MPEG encoding, encoding is performed in an order different from the actual frame / field order, as is conventionally known. That is, the image signal of the I picture and the P picture is encoded first, and the image signal of the B picture sandwiched between them is encoded thereafter. The image signal Va of each picture is output to the output terminal 81 in the encoding order. In the present embodiment, when the image signal Va is read from the buffer memory 108 described above, the image signal of each picture is rearranged from the encoding order to the actual frame / field order.
[0044]
The decoder 107 extracts an encoding control information, that is, picture information PI and motion compensation vector information MI from the MPEG2 stream stored in the stream buffer 72, and the extraction circuit 82 extracts the decoding control information. And an output terminal 83 for outputting the picture information PI.
[0045]
The motion compensation vector information MI extracted by the extraction circuit 82 is supplied to the prediction memory circuit 78, and the prediction memory circuit 78 performs motion compensation when generating the reference data Vref using the motion compensation vector information MI. Is called. The picture information PI extracted by the extraction circuit 82 is also supplied to the prediction memory circuit 78. The prediction memory circuit 78 identifies a picture based on the picture information PI.
[0046]
The decoder 107 also outputs an output terminal 84 that outputs residual data related to the P picture and B picture output from the inverse DCT circuit 77, and a reference related to the P picture and B picture output from the prediction memory circuit 78. And an output terminal 85 for outputting data.
[0047]
The operation of the MPEG2 decoder 107 shown in FIG. 2 will be described.
The MPEG2 stream stored in the stream buffer 72 is supplied to the extraction circuit 73 to extract DCT coefficients as frequency coefficients. This DCT coefficient is variable length encoded, and this DCT coefficient is supplied to the variable length decoding circuit 74 and decoded. The quantized DCT coefficient of each DCT block output from the variable length decoding circuit 74 is supplied to the inverse quantization circuit 76 and subjected to inverse quantization.
[0048]
The inverse DCT circuit 77 performs inverse DCT on the DCT coefficient of each DCT block output from the inverse quantization circuit 76, thereby obtaining data of each picture. The data of each picture is output to the output terminal 81 via the adding circuit 79. In this case, when P-picture or B-picture residual data is output from the inverse DCT circuit 77, the adder circuit 79 adds the reference data Vref output from the prediction memory circuit 78.
[0049]
The picture information PI is output to the output terminal 83 in pairs with each pixel data constituting the image signal Va output from the output terminal 81. In addition, the pixel data corresponding to the P picture and B picture among the pixel data constituting the image signal Va output from the output terminal 81 is paired with the pixel data obtained at the output terminal 84 and the output terminal 85, respectively. The residual data and reference data used at the time are also output.
[0050]
Returning to FIG. 1, the digital broadcast receiver 100 also converts the image signal Va stored in the buffer memory 108 into an image signal Vb in which coding noise such as block noise (block distortion) and mosquito noise is reduced. An image signal processing unit 110 for conversion and a display unit 111 for displaying an image based on the image signal Vb output from the image signal processing unit 110 are provided. The display unit 111 includes a display such as a CRT (Cathode-Ray Tube) display or an LCD (Liquid Crystal Display).
[0051]
The operation of the digital broadcast receiver 100 shown in FIG. 1 will be described.
The MPEG2 stream output from the tuner unit 106 is supplied to the MPEG2 decoder 107 and decoded. The image signal Va output from the decoder 107 is supplied to the buffer memory 108 and temporarily stored.
[0052]
In this case, the decoder 107 outputs picture information PI in pairs with each pixel data of the image signal Va. The encoder 107 also outputs residual data and reference data in pairs with pixel data related to the P picture and B picture. Such information and data are also temporarily stored in the buffer memory 108.
[0053]
The image signal Va temporarily stored in the buffer memory 108 in this way is supplied to the image signal processing unit 110 and converted into an image signal Vb with reduced coding noise. In the image signal processing unit 110, pixel data constituting the image signal Vb is generated from the pixel data constituting the image signal Va. In the image signal processing unit 110, the picture information PI stored in the buffer memory 108, and the residual data and reference data are used, and conversion processing is performed as will be described later.
[0054]
The image signal Vb output from the image signal processing unit 110 is supplied to the display unit 111, and an image based on the image signal Vb is displayed on the screen of the display unit 111.
[0055]
Next, details of the image signal processing unit 110 will be described.
The image signal processing unit 110 includes a class generation unit 121 that generates a class code CL0 indicating a class based on the inaccuracy of the motion compensation vector information MI to which the pixel data of the target position in the image signal Vb belongs. The class generation unit 121 uses the picture information PI, residual data, and reference data stored in the buffer memory 108 to generate a class code CL0.
[0056]
FIG. 3 shows a specific configuration of the class generation unit 121.
The class generation unit 121 has an input terminal 31 to which reference data stored as a pair with pixel data related to P picture and B picture is input to the buffer memory 108 described above, and input to the input terminal 31. An edge component extraction circuit 32 that extracts edge components from the reference data and a buffer memory 33 that temporarily stores the edge components of the reference data extracted by the extraction circuit 32 are provided.
[0057]
The edge component extraction circuit 32 is configured by a two-dimensional differential filter, for example, a 3 × 3 Laplacian filter. FIG. 4 shows an example of the coefficient ratio of a 3 × 3 Laplacian filter. As described above, in the case of a P picture, the reference data exists only in the forward direction corresponding to one pixel data, whereas in the case of a B picture, the reference data corresponds to one pixel data. Exist in the direction of. Therefore, the edge component extraction circuit 32 detects the edge component from the forward reference data in the case of the P picture, but detects the edge component from each of the forward and backward reference data in the case of the B picture. .
[0058]
In addition, the class generation unit 121 has an input terminal 34 to which the residual data stored in a pair with the pixel data related to the P picture and the B picture in the buffer memory 108 described above is input, and an input to the input terminal 34. The edge component extraction circuit 35 extracts an edge component from the residual data, and the buffer memory 36 temporarily stores the edge component of the reference data extracted by the extraction circuit 35.
[0059]
Similarly to the edge component extraction circuit 32 described above, the edge component extraction circuit 35 is also constituted by a two-dimensional differential filter, for example, a 3 × 3 Laplacian filter. In this case, since the residual data also takes a negative value, the edge component is extracted after being offset so that all the values of the residual data become positive values. For example, when the residual data takes a value of −128 to +128, edge component extraction processing is performed after adding +128 to the residual data.
[0060]
In addition, the class generation unit 121 indicates motion vector information (second motion vector information) corresponding to the target position in the image signal Vb, and indicates inaccuracies in the motion compensation vector information MI (first motion vector information). It has the correlation determination part 37 as a motion vector acquisition means obtained as information. The correlation determination unit 37 acquires motion vector information corresponding to the target position in the image signal Vb using the first frame and the second frame.
[0061]
Here, the first frame is a frame including the edge component of the residual data related to the pixel data of the image signal Va corresponding to the pixel data of the target position in the image signal Vb stored in the buffer memory 36. The second frame is a frame including the edge component of the reference data related to the pixel data of the image signal Va corresponding to the pixel data of the target position in the image signal Vb stored in the buffer memory 33.
[0062]
The correlation determination unit 37 acquires motion vector information by, for example, a block matching method. In this case, motion vector information is acquired for each pixel or block.
[0063]
A case where motion vector information (Δx, Δy) is acquired for each pixel will be described with reference to FIG.
In this case, for example, as shown in the figure, the first frame is a reference frame, and the second frame is a search frame. Note that the second frame may be a reference frame, and the first frame may be a search frame. Then, a reference block centered on the target pixel corresponding to the target position in the image signal Vb is considered in the reference frame. Further, a search range centered on the target position in the image signal Vb is considered in the search frame. The search range is, for example, about ± 2 in the vertical and horizontal directions.
[0064]
Then, correlation information between the reference block and a plurality of candidate blocks within the search range is detected. The correlation information is, for example, a difference absolute value sum obtained by adding the difference absolute values of corresponding edge components between the reference block and the candidate block. The difference absolute value sum decreases as the correlation level between the reference block and the candidate block increases.
[0065]
Then, based on the correlation information corresponding to the plurality of candidate blocks detected as described above, the position information of the candidate block having the highest correlation is output as motion vector information (Δx, Δy). In this case, when the correlation information is the above-described sum of absolute differences, the candidate block with the smallest sum of absolute differences is the candidate block with the highest correlation.
[0066]
A case where motion vector information is acquired for each block will be described with reference to FIG.
In this case, for example, as shown in the figure, the first frame is a reference frame, and the second frame is a search frame. Note that the second frame may be a reference frame, and the first frame may be a search frame. Then, a reference block including a target pixel corresponding to a target position in the image signal Vb in the reference frame is considered. This reference block is a block corresponding to the DCT block used when obtaining the pixel data of the image signal Va corresponding to the target position in the image signal Vb, for example. Further, a search range centered on the target position in the image signal Vb is considered in the search frame. For example, the search range is, for example, about ± 2 in the vertical and horizontal directions.
[0067]
Then, correlation information between the reference block and a plurality of candidate blocks within the search range is detected. The correlation information is, for example, a difference absolute value sum obtained by adding the difference absolute values of corresponding edge components between the reference block and the candidate block. The difference absolute value sum decreases as the correlation level between the reference block and the candidate block increases.
[0068]
Then, based on the correlation information corresponding to the plurality of candidate blocks detected as described above, the position information of the candidate block having the highest correlation is output as motion vector information (Δx, Δy). In this case, when the correlation information is the above-described sum of absolute differences, the candidate block with the smallest sum of absolute differences is the candidate block with the highest correlation.
[0069]
As described above, the correlation determination unit 37 functions as a correlation detection unit that detects correlation information and an information output unit that outputs motion vector information, and outputs motion vector information (Δx, Δy). As described above, the reference data has been subjected to motion compensation in the decoder 107 (see FIG. 2) based on the motion compensation vector information MI. Therefore, originally, the edge component extracted from the reference data in the second frame corresponds to the edge component extracted from the residual data in the first frame, and motion vector information (Δx, Δy) = (0 , 0).
[0070]
However, when the motion compensation vector information MI is inaccurate, the motion vector information (Δx, Δy) = (0, 0) is not satisfied, and as the degree of inaccuracy increases, the values of Δx, Δy (absolute Value) increases. Therefore, it can be said that the motion vector information (Δx, Δy) is information indicating the inaccuracy of the motion compensation vector information MI.
[0071]
As described above, in the case of the P picture, the reference data exists only in the forward direction corresponding to one pixel data, but in the case of the B picture, the reference data corresponds to one pixel data. Exists for each direction. Therefore, the correlation determination unit 37 acquires only the motion vector information (Δx, Δy) related to the forward reference data in the case of the P picture, but in the case of the B picture, the correlation data of the forward and backward reference data. The motion vector information (Δx, Δy) related to each is acquired.
[0072]
Further, when the correlation level of the candidate block with the highest correlation is smaller than a preset threshold value, for example, when the sum of absolute differences related to the candidate block with the highest correlation is larger than the threshold value, the correlation determination unit 37 Instead, information NG indicating that is output. This is to avoid class classification due to inaccurate motion vector information (Δx, Δy) and prevent degradation of class classification.
[0073]
The class generation unit 37 includes a class generation circuit 38 and an output terminal 39 that outputs the class code CL0 generated by the class generation circuit 38. The class generation circuit 38 is supplied with the picture information PI paired with the pixel data of the image signal Va corresponding to the target position in the image signal Vb as the operation control information.
[0074]
When the picture information PI indicates an I picture, the class generation circuit 38 generates a specific code as the residual class code CL0. Further, when the picture information PI indicates a P picture or a B picture, the class generation circuit 38 generates a class code CL0 based on the motion vector information (Δx, Δy) or information NG output from the correlation determination unit 37. To do.
[0075]
The operation for generating the class code CL0 indicating the class based on the inaccuracy of the motion compensation vector information MI to which the pixel data of the target position in the image signal Vb belongs in the residual class generation unit 121 illustrated in FIG. 3 will be described. To do.
[0076]
The input terminal 31 receives reference data stored in the buffer memory 108 as a pair with pixel data related to the P picture and B picture, and the reference data is supplied to the edge component extraction circuit 32. The edge component extraction circuit 32 extracts edge components from the reference data. The edge component extracted by the extraction circuit 32 is supplied to the buffer memory 33 and temporarily stored.
[0077]
Further, residual data stored as a pair with pixel data relating to P picture and B picture is input to the buffer memory 108 to the input terminal 34, and this residual data is supplied to the edge component extraction circuit 35. The The edge component extraction circuit 35 extracts edge components from the residual data. The edge component extracted by the extraction circuit 35 is supplied to the buffer memory 36 and temporarily stored.
[0078]
The correlation determination unit 37 sets, as a first frame, a frame including an edge component of residual data related to the pixel data of the image signal Va corresponding to the pixel data of the target position in the image signal Vb stored in the buffer memory 36, A frame including the edge component of the reference data related to the pixel data of the image signal Va corresponding to the pixel data of the target position in the image signal Vb stored in the buffer memory 33 is defined as a second frame. Then, the correlation determination unit 37 acquires motion vector information (Δx, Δy) corresponding to the target position in the image signal Vb by using the first frame and the second frame, for example, by a block matching method. This motion vector information (Δx, Δy) is information indicating the inaccuracy of the motion compensation vector information MI.
[0079]
In this case, the correlation determination unit 37 acquires only motion vector information (Δx, Δy) related to forward reference data in the case of a P picture, but forward and backward reference data in the case of a B picture. The motion vector information (Δx, Δy) relating to each is acquired. When the correlation level of the candidate block with the highest correlation is smaller than a preset threshold value, information NG indicating that is output instead of the motion vector information.
[0080]
The motion vector information (Δx, Δy) or information NG output from the correlation determination unit 37 is supplied to the class generation circuit 38. When the picture information PI indicates an I picture, the class generation circuit 38 generates a specific code as the residual class code CL0. In addition, when the picture information PI indicates a P picture or a B picture, the class generation circuit 38 generates a class code CL0 based on the motion vector information (Δx, Δy) or the information NG. The class code CL0 generated in this way is output to the output terminal 39.
[0081]
3 generates the class code CL0 indicating the class based on the inaccuracy of the motion compensation vector information MI to which the pixel data of the target position in the image signal Vb belongs.
[0082]
Returning to FIG. 1, the image signal processing unit 110 selectively extracts and outputs a plurality of pixel data located around the target position in the image signal Vb from the image signal Va stored in the buffer memory 108. And a prediction tap selection circuit 122 as a data selection means. The prediction tap selection circuit 122 selectively extracts a plurality of pixel data of prediction taps used for prediction.
[0083]
Further, the image signal processing unit 110 includes a class classification unit 123 as a class detection unit that detects a class to which the pixel data of the target position in the image signal Vb belongs.
[0084]
The class classification unit 123 uses the plurality of pixel data constituting the image signal Va stored in the buffer memory 108 and the class code CL0 generated by the class generation unit 121 to obtain pixel data of the target position in the image signal Vb. A class code CL indicating the class to which it belongs is generated.
[0085]
Figure 7 Shows the configuration of the class classification unit 123.
The class classification unit 123 detects n types of classes to which pixel data of the target position in the image signal Vb belongs based on the input terminal 50A for inputting the image signal Va and the image signal Va input to the input terminal 50A. Tap selection circuit 50B for selectively extracting a plurality of pixel data of class taps used for ₁ ~ 50B _n And this tap selection circuit 50B ₁ ~ 50B _n A class generation circuit 50C for generating class codes CL1 to CLn indicating n types of classes using the pixel data extracted in step S1. ₁ ~ 50C _n And have.
[0086]
In the present embodiment, class codes CL1 to CL6 indicating six types of classes are generated. The six types of classes are a spatial waveform class, a time variation class, an AC variation class, a flat class, a line correlation class, and a block edge class. A brief description of each class.
[0087]
(1) The spatial waveform class will be described. Tap selection circuit 50B ₁ And class generation circuit 50C ₁ Suppose that this spatial waveform class detection system is configured.
Tap selection circuit 50B ₁ Are a plurality of positions located in the space direction (horizontal direction and vertical direction) around the position of interest in the image signal Vb from the T frame (current frame) and T-1 frame (frame one frame before) of the image signal Va. The pixel data is selectively extracted and is similar to the prediction tap selection circuit 122 described above. Class generation circuit 50C ₁ The tap selection circuit 50B ₁ For example, 1-bit ADRC (Adaptive Dynamic Range Coding) or the like is applied to each of the plurality of pixel data selected in step S1 to generate a class code CL1 indicating a spatial waveform class.
[0088]
(2) The time variation class will be described. Tap selection circuit 50B ₂ And class generation circuit 50C ₂ Is configured as a detection system of this time variation class.
Tap selection circuit 50B ₂ Extracts the pixel data of the DCT block (the target block shown in FIG. 8) corresponding to the pixel data at the target position in the image signal Vb from the current frame (T frame) of the image signal Va, and at the same time before the image signal Va. The pixel data of the block (the past block shown in FIG. 8) corresponding to the block of interest is extracted from the past frame (T-1 frame).
[0089]
Class generation circuit 50C ₂ Are subtracted for each corresponding pixel between the 8 × 8 pixel data of the block of interest and the 8 × 8 pixel data of the past block to obtain 8 × 8 difference values, and this 8 × The sum of squares of the eight difference values is obtained, the square sum is determined as a threshold value, and the class code CL2 indicating the time variation class is generated.
[0090]
(3) The AC variation class will be described. Tap selection circuit 50B _Three And class generation circuit 50C _Three Constitutes a detection system of this AC fluctuation class.
Tap selection circuit 50B _Three Extracts the pixel data of the DCT block (the target block shown in FIG. 8) corresponding to the pixel data at the target position in the image signal Vb from the current frame of the image signal Va, and from the previous frame one frame before the image signal Va. Then, pixel data of a block (the past block shown in FIG. 8) corresponding to the target block is extracted.
[0091]
Class generation circuit 50C _Three Performs DCT processing on each of 8 × 8 pixel data of the block of interest and 8 × 8 pixel data of the past block to obtain DCT coefficients (frequency coefficients). Then, the class generation circuit 50C _Three Is the number m of base positions where a coefficient exists in either base position of the AC part. ₁ And the number m of base positions of which the sign is inverted and one of the coefficients is 0 ₂ M ₁ / M ₂ Is determined as a threshold value, and a class code CL3 indicating an AC variation class is generated. In a block with little time variation, it is possible to perform class classification corresponding to mosquito distortion by this AC variation class.
[0092]
(4) The flat class will be described. Tap selection circuit 50B _Four And class generation circuit 50C _Four Suppose that this flat class detection system is configured.
Tap selection circuit 50B _Four Extracts the pixel data of the DCT block (the target block shown in FIG. 8) corresponding to the pixel data at the target position in the image signal Vb from the current frame of the image signal Va. Class generation circuit 50C _Four Detects the maximum value and the minimum value of 8 × 8 pixel data of the block of interest, determines the threshold of the dynamic range that is the difference, and generates the class code CL4 indicating the flat class.
[0093]
(5) The line correlation class will be described. Tap selection circuit 50B _Five And class generation circuit 50C _Five Constitutes a detection system of this line correlation class.
Tap selection circuit 50B _Five Extracts the pixel data of the DCT block (the target block shown in FIG. 8) corresponding to the pixel data at the target position in the image signal Vb from the current frame of the image signal Va.
[0094]
Class generation circuit 50C _Five Corresponds to the pixels of the 1st line, 2nd line, 3rd line, 4th line, 5th line, 6th line, 7th line and 8th line of the 8 × 8 pixel data of the target block Subtraction is performed for each pixel to obtain 8 × 4 difference values, and a square sum of the 8 × 4 difference values is obtained, and the square sum is determined as a threshold value, and a class code CL5 indicating a line correlation class is obtained. Generate. This line correlation class indicates whether the correlation in a frame such as a still image is high, or whether the correlation in the field is higher than in the frame because of fast movement.
[0095]
(6) The block edge class will be described. Tap selection circuit 50B ₆ And class generation circuit 50C ₆ Suppose that this block edge class detection system is configured.
Tap selection circuit 50B ₆ Extracts pixel data of the DCT block (the target block shown in FIG. 8) corresponding to the pixel data of the target position in the image signal Vb from the current frame of the image signal Va, and moves up and down with respect to the target block from the current frame. Pixel data of blocks adjacent to the left and right (adjacent blocks shown in FIG. 8) are extracted.
[0096]
Class generation circuit 50C ₆ Are subtracted for each corresponding pixel between the 8 pixel data of each of the 4 sides of the block of interest and the pixel data of the adjacent block adjacent thereto to obtain 4 × 8 difference values, and each 8 A square sum of the difference values is obtained, and four square sums respectively corresponding to the four sides of the block of interest are threshold-determined to generate a class code CL6 indicating a block edge class.
[0097]
The class classification unit 123 includes an input terminal 50D for inputting the class code CL0, and a class generation circuit 50C. ₁ ~ 50C _n A class integration circuit 50E that integrates the class codes CL1 to CLn generated in step S1 and the class code CL0 input to the input terminal 50D into one class code CL, and an output terminal 50F that outputs the class code CL. Have. In the present embodiment, the class integration circuit 50E includes a class generation circuit 50C. ₁ ~ 50C ₆ The class codes CL1 to CL6 generated in step 1 and the class code CL0 are integrated into one class code CL.
[0098]
Returning to FIG. 1, the image signal processing unit 110 also includes a coefficient memory 124. The coefficient memory 124 stores coefficient data Wi (i = 1 to n, n is the number of prediction taps) used in an estimation formula used in the estimation prediction calculation circuit 125 described later for each class. .
[0099]
The coefficient data Wi is information for converting the image signal Va into the image signal Vb. The coefficient data Wi stored in the coefficient memory 124 is generated by learning between a student signal corresponding to the image signal Va and a teacher signal corresponding to the image signal Vb in advance. The coefficient memory 124 is supplied with the class code CL output from the above-described class classification unit 123 as read address information. The coefficient memory 124 reads the coefficient data Wi of the estimation formula corresponding to the class code CL. , And supplied to the estimated prediction calculation circuit 125. A method for generating the coefficient data Wi will be described later.
[0100]
In addition, the image signal processing unit 110 creates the prediction tap pixel data xi selectively extracted by the prediction tap selection circuit 122 and the coefficient data Wi read from the coefficient memory 124 by using the estimation expression (1). It has an estimated prediction calculation circuit 125 for calculating pixel data y at the target position in the image signal Vb to be processed.
[0101]
[Expression 1]

[0102]
The operation of the image signal processing unit 110 will be described.
In the class generation unit 121, the picture information PI, residual data, and reference data stored in the buffer memory 108 are used, and the motion compensation vector information MI to which the pixel data of the target position in the image signal Vb belongs is not included. A class code CL0 indicating a class based on accuracy is generated.
[0103]
Further, the class classification unit 123 uses the plurality of pixel data constituting the image signal Va stored in the buffer memory 108 and the class code CL0 generated by the class generation unit 121, and the pixel at the target position in the image signal Vb. A class code CL indicating the class to which the data belongs is generated.
[0104]
Thus, the class code CL generated by the class classification unit 123 is supplied to the coefficient memory 124 as read address information. As a result, the coefficient data Wi corresponding to the class code CL is read from the coefficient memory 124 and supplied to the estimated prediction calculation circuit 125.
[0105]
Further, the prediction tap selection circuit 122 selectively extracts pixel data of the prediction tap located around the target position in the image signal Vb from the image signal Va stored in the buffer memory 108.
[0106]
The estimated prediction calculation circuit 125 uses the pixel data xi of the prediction tap and the coefficient data Wi read from the coefficient memory 124, based on the estimation expression shown in the above equation (1), in the image signal Vb to be created. Pixel data y at the target position is obtained.
[0107]
As described above, the image signal processing unit 110 obtains the image signal Vb from the image signal Va using the coefficient data Wi. In this case, a plurality of pixel data (prediction tap pixel data) located around the target position in the image signal Vb, selected based on the image signal Va, and a class to which the pixel data of the target position in the image signal Vb belong. Pixel data y at the target position in the image signal Vb is generated based on the estimation formula using the coefficient data Wi corresponding to CL.
[0108]
Therefore, the coefficient data Wi is obtained by learning using a student signal corresponding to the image signal Va and including coding noise similar to the image signal Va and a teacher signal corresponding to the image signal Vb and not including coding noise. By using the coefficient data Wi, it is possible to satisfactorily obtain an image signal Vb in which coding noise is significantly reduced as compared with the image signal Va.
[0109]
Further, the class generation unit 121 generates a class code CL0 indicating a class based on the inaccuracy of the motion compensation vector information MI. Then, in the class classification unit 123, the class code CL0 is integrated with other class codes, and the class code CL is generated. Therefore, in the image signal processing unit 110, class classification is performed based on inaccuracy information of the motion compensation vector information MI, and the quality of the image signal Vb can be improved.
[0110]
Further, when the correlation level of the candidate block with the highest correlation is smaller than a preset threshold, the correlation determination unit 37 of the class generation unit 121 outputs information NG indicating that instead of the motion vector information. Then, the class generation circuit 38 outputs the class code CL0 corresponding to the information NG. Therefore, the class code CL0 based on inaccurate motion vector information (Δx, Δy) is not output from the class generation unit 121, and the accuracy of class classification can be prevented from being lowered.
[0111]
Next, a method for generating the coefficient data Wi stored in the coefficient memory 124 will be described. The coefficient data Wi is generated in advance by learning.
[0112]
First, this learning method will be described. In the above equation (1), before learning, coefficient data W ₁ , W ₂ , ..., W _n Is an undetermined coefficient. Learning is performed on a plurality of signal data for each class. When the number of learning data is m, the following equation (2) is set according to the equation (1). n indicates the number of prediction taps.
y _k = W ₁ X _k1 + W ₂ X _k2 + ... + W _n X _kn ... (2)
(K = 1, 2,..., M)
[0113]
If m> n, coefficient data W ₁ , W ₂ , ..., W _n Is not uniquely determined, so the element e of the error vector e _k Is defined by the following equation (3), and e in equation (4) ² Find coefficient data that minimizes. Coefficient data is uniquely determined by a so-called least square method.
e _k = Y _k -{W ₁ X _k1 + W ₂ X _k2 + ... + W _n X _kn } (3)
(K = 1, 2, ... m)
[0114]
[Expression 2]

[0115]
E in equation (4) ² As a practical calculation method for obtaining coefficient data that minimizes the value, first, as shown in the equation (5), e ² Is partially differentiated with the coefficient data Wi (i = 1, 2,..., N), and the coefficient data Wi is obtained so that the partial differential value becomes 0 for each value of i.
[0116]
[Equation 3]

[0117]
A specific procedure for obtaining the coefficient data Wi from the equation (5) will be described. If Xji and Yi are defined as in the equations (6) and (7), the equation (5) can be written in the form of the determinant of the equation (8).
[0118]
[Expression 4]

[0119]
[Equation 5]

[0120]
Equation (8) is generally called a normal equation. Coefficient data Wi (i = 1, 2,..., N) can be obtained by solving this normal equation by a general solution method such as a sweep-out method (Gauss-Jordan elimination method).
[0121]
FIG. 9 shows a configuration of a coefficient data generation device 150 that generates coefficient data Wi to be stored in the coefficient memory 124 of the image signal processing unit 110 of FIG.
The coefficient data generation device 150 includes an input terminal 151 to which a teacher signal ST corresponding to the image signal Vb is input, an MPEG2 encoder 152 that encodes the teacher signal ST to obtain an MPEG2 stream, An MPEG2 decoder 153 that decodes the MPEG2 stream to obtain a student signal SS corresponding to the image signal Va. Here, the MPEG2 decoder 153 corresponds to the MPEG2 decoder 107 and the buffer memory 108 in the digital broadcast receiver 100 shown in FIG.
[0122]
Further, the coefficient data generation device 150 includes a class generation unit 154. The class generation unit 154 is configured in the same manner as the class generation unit 121 of the image signal processing unit 110 described above, and is based on the inaccuracy of the motion compensation vector information MI to which the pixel data at the target position in the teacher signal ST belongs. Class code CL0 is generated. In this class generation unit 154, the class code CL0 is generated using the picture information PI, residual data, and reference data output from the decoder 153.
[0123]
Also, the coefficient data generation device 150 selectively extracts a plurality of pixel data located around the target position in the teacher signal ST from the student signal SS output from the MPEG2 decoder 153, and outputs the selected pixel data. 155. The prediction tap selection circuit 155 is configured in the same manner as the prediction tap selection circuit 122 of the image signal processing unit 110 described above.
[0124]
In addition, the coefficient data generation apparatus 150 includes a class classification unit 156 as a class detection unit that detects a class to which pixel data of the target position in the teacher signal ST belongs. The class classification unit 156 is configured in the same manner as the class classification unit 123 of the image signal processing unit 110 described above.
[0125]
This class classification unit 156 uses the plurality of pixel data constituting the student signal SS obtained from the MPEG2 decoder 153 and the class code CL0 generated by the class generation unit 154 to use the pixel data of the target position in the teacher signal ST. A class code CL indicating the class to which the file belongs is generated.
[0126]
The coefficient data generation device 150 also includes a delay circuit 157 for adjusting the time of the teacher signal ST supplied to the input terminal 151, and each position of interest obtained from the teacher signal ST adjusted in time by the delay circuit 157. The pixel data y, the pixel data xi of the prediction tap selectively extracted by the prediction tap selection circuit 155 corresponding to the pixel data y of each target position, and the class corresponding to the pixel data y of each target position From the class code CL generated by the classification unit 156, a normal equation generation unit 158 that generates a normal equation (see the above equation (8)) for obtaining coefficient data Wi (i = 1 to n) for each class. have.
[0127]
In this case, one learning data is generated by combining one pixel data y and pixel data xi of n prediction taps corresponding to the pixel data y. A lot of learning data is generated every time. As a result, the normal equation generation unit 158 generates a normal equation for obtaining coefficient data Wi (i = 1 to n) for each class.
[0128]
The coefficient data generation device 150 is supplied with the data of the normal equation generated by the normal equation generation unit 158, solves the normal equation, and obtains the coefficient data Wi of each class, and the coefficient data determination unit 159. And a coefficient memory 160 for storing the obtained coefficient data Wi of each class.
[0129]
Next, the operation of the coefficient data generation device 150 shown in FIG. 9 will be described.
A teacher signal ST corresponding to the image signal Vb is supplied to the input terminal 151, and the MPEG2 encoder 152 encodes the teacher signal ST to generate an MPEG2 stream. This MPEG2 stream is supplied to the MPEG2 decoder 153. The MPEG2 decoder 153 decodes the MPEG2 stream to generate a student signal SS corresponding to the image signal Va.
[0130]
The class generation unit 154 uses the picture information PI, residual data, and reference data output from the decoder 153, and the inaccuracy of the motion compensation vector information MI to which the pixel data of the target position in the teacher signal ST belongs. A class code CL0 indicating a class based on is generated.
[0131]
The class classification unit 156 uses the plurality of pixel data constituting the student signal SS obtained from the MPEG2 decoder 153 and the class code CL0 generated by the class generation unit 154 to generate pixel data of the target position in the teacher signal ST. A class code CL indicating the class to which it belongs is generated.
[0132]
In addition, the prediction tap selection circuit 155 selectively extracts pixel data of prediction taps located around the target position in the teacher signal ST from the student signal SS obtained from the MPEG2 decoder 153.
[0133]
Then, the pixel data y of each target position obtained from the teacher signal ST time-adjusted by the delay circuit 157, and the prediction selectively extracted by the prediction tap selection circuit 155 corresponding to the pixel data y of each target position. Using the tap pixel data xi and the class code CL generated by the class classification unit 156 corresponding to the pixel data y of each target position, the normal equation generation unit 158 uses the coefficient data Wi ( A normal equation (see equation (8)) for obtaining i = 1 to n) is generated. This normal equation is solved by the coefficient data determination unit 159 to obtain the coefficient data Wi of each class, and the coefficient data Wi is stored in the coefficient memory 160.
[0134]
As described above, the coefficient data generation device 150 illustrated in FIG. 9 can generate the coefficient data Wi of each class stored in the coefficient memory 124 of the image signal processing unit 110 in FIG.
[0135]
The student signal SS is obtained by encoding the teacher signal ST to generate an MPEG2 stream and then decoding the MPEG2 stream. Therefore, the student signal SS includes the same coding noise as the image signal Va. Therefore, in the image signal processing unit 110 shown in FIG. 1, the image signal Vb obtained from the image signal Va using the coefficient data Wi has a coding noise reduced as compared with the image signal Va.
[0136]
Note that the processing in the image signal processing unit 110 in FIG. 1 can be realized by software, for example, by an image signal processing apparatus 300 as shown in FIG.
[0137]
First, the image signal processing apparatus 300 shown in FIG. 10 will be described. The image signal processing apparatus 300 includes a CPU 301 that controls the operation of the entire apparatus, a ROM (Read Only Memory) 302 that stores a control program of the CPU 301, coefficient data, and the like, and a RAM (Random) that constitutes a work area of the CPU 301. Access Memory) 303. These CPU 301, ROM 302, and RAM 303 are each connected to a bus 304.
[0138]
The image signal processing apparatus 300 also includes a hard disk drive (HDD) 305 as an external storage device and a drive (FDD) 307 that drives a floppy (registered trademark) disk 306. These drives 305 and 307 are each connected to a bus 304.
[0139]
In addition, the image signal processing apparatus 300 includes a communication unit 308 that is connected to a communication network 400 such as the Internet by wire or wirelessly. The communication unit 308 is connected to the bus 304 via the interface 309.
[0140]
In addition, the image signal processing device 300 includes a user interface unit. The user interface unit includes a remote control signal receiving circuit 310 that receives a remote control signal RM from the remote control transmitter 200, and a display 311 that includes an LCD (liquid Crystal Display) or the like. The receiving circuit 310 is connected to the bus 304 via the interface 312, and similarly the display 311 is connected to the bus 304 via the interface 313.
[0141]
Further, the image signal processing apparatus 300 has an input terminal 314 for inputting the image signal Va and an output terminal 315 for outputting the image signal Vb. The input terminal 314 is connected to the bus 304 via the interface 316, and similarly, the output terminal 315 is connected to the bus 304 via the interface 317.
[0142]
Here, instead of storing the control program and coefficient data in the ROM 302 in advance as described above, for example, they are downloaded from the communication network 400 such as the Internet via the communication unit 308 and stored in the hard disk or RAM 303 for use. You can also. These control programs, coefficient data, and the like may be provided on a floppy (registered trademark) disk 306.
[0143]
Further, instead of inputting the image signal Va to be processed from the input terminal 314, it may be recorded in advance on a hard disk or downloaded from the communication network 400 such as the Internet via the communication unit 308. Further, instead of outputting the processed image signal Vb to the output terminal 315 or in parallel therewith, it is supplied to the display 311 to display an image, further stored in a hard disk, the Internet via the communication unit 308, etc. It may be sent to the communication network 400.
[0144]
A processing procedure for obtaining the image signal Vb from the image signal Va in the image signal processing apparatus 300 shown in FIG. 10 will be described with reference to the flowchart of FIG.
First, in step ST21, processing is started, and in step S22, for example, an image signal Va for one frame or one field is input into the apparatus from the input terminal 314. In this case, picture information PI paired with each pixel data of the image signal Va is also input. The picture information PI is information indicating whether the pixel data relates to an I picture, a P picture, or a B picture. Further, in this case, residual data and reference data that are paired with each pixel data of the image signal Va and used when obtaining the pixel data are also input.
[0145]
Thus, the image signal Va and the like input from the input terminal 314 are temporarily stored in the RAM 303. If the image signal Va or the like is recorded in advance in the hard disk drive 305 in the apparatus, the image signal Va or the like is read from the drive 305 and the image signal Va or the like is temporarily stored in the RAM 303.
[0146]
In step ST23, it is determined whether or not the processing of all frames or all fields of the image signal Va has been completed. When the process is finished, the process ends in step ST24. On the other hand, when the process is not finished, the process proceeds to step ST25.
[0147]
In step ST25, the pixel data of the target position in the image signal Vb belongs using the picture information PI, residual data, and reference data paired with the pixel data of the image signal Va corresponding to the target position in the image signal Vb. The class code CL0 indicating the class based on the inaccuracy of the motion compensation vector information MI is generated, and the position of the target position in the image signal Vb is generated using the class code CL0 and a plurality of pixel data constituting the image signal Va. A class code CL indicating the class to which the pixel data belongs is generated.
[0148]
Next, in step ST26, a plurality of pixel data (prediction tap pixel data) located around the target position in the image signal Vb is acquired from the image signal Va input in step ST22. Then, in step ST27, using the coefficient data Wi corresponding to the class code CL generated in step ST25 and the pixel data xi of the prediction tap acquired in step ST26, based on the estimation expression of the expression (1), Pixel data y at the target position in the image signal Vb is generated.
[0149]
Next, in step ST28, it is determined whether or not the processing for obtaining the pixel data of the image signal Vb has been completed in the entire region of the pixel data of the image signal Va for one frame or one field input in step ST22. If completed, the process returns to step ST22, and the process proceeds to input processing of the image signal Va for the next one frame or one field. On the other hand, when the process has not been completed, the process returns to step ST25 and proceeds to the process for the next target position.
[0150]
In this way, by performing processing according to the flowchart shown in FIG. 11, the pixel data of the input image signal Va can be processed to obtain the pixel data of the image signal Vb. As described above, the image signal Vb obtained by such processing is output to the output terminal 315, supplied to the display 311 to display an image, and further supplied to the hard disk drive 305 to be supplied to the hard disk. Or is recorded.
[0151]
Although illustration of the processing device is omitted, the processing in the coefficient data generation device 150 in FIG. 9 can also be realized by software.
[0152]
A processing procedure for generating coefficient data will be described with reference to the flowchart of FIG.
First, in step ST31, the process is started, and in step ST32, the teacher signal ST is input for one frame or one field. In step ST33, it is determined whether or not processing of all frames or all fields of the teacher signal ST has been completed. If not finished, in step ST34, the student signal SS is generated from the teacher signal ST input in step ST32.
[0153]
In this case, picture information PI paired with each pixel data of the student signal SS, and residual data and reference data used when obtaining the pixel data paired with each pixel data are also obtained. To.
[0154]
In step ST35, the pixel of the target position in the teacher signal ST is used by using the picture information PI paired with the pixel data of the student signal SS corresponding to the target position of the teacher signal ST, the residual data, and the reference data. A class code CL0 indicating a class based on the inaccuracy of the motion compensation vector information MI to which the data belongs is generated, and further using the class code CL0 and a plurality of pixel data constituting the student signal SS, the teacher signal ST A class code CL indicating the class to which the pixel data at the target position belongs is generated.
[0155]
Next, in step ST36, a plurality of pixel data (prediction tap pixel data) located around the target position in the teacher signal ST is acquired from the student signal SS generated in step ST34.
[0156]
Then, in step ST37, for each class, using the class code CL generated in step ST35, the pixel data xi of the prediction tap acquired in step ST36, and the pixel data y of the target position in the teacher signal ST, (8) Addition is performed to obtain the normal equation shown in the equation (see equations (6) and (7)).
[0157]
Next, in step ST38, it is determined whether or not the learning process has been completed in all regions of the pixel data of the teacher signal ST for one frame or one field input in step ST32. When the learning process is finished, the process returns to step ST32, the teacher signal ST for the next one frame or one field is input, and the same process as described above is repeated. On the other hand, when the learning process is not finished, the process returns to step ST35 and moves to the process for the next attention position.
[0158]
When the process is completed in step ST33 described above, in step ST39, the normal equation of each class generated by the addition process in step ST37 described above is solved by a sweeping method or the like to calculate the coefficient data Wi of each class. To do. In step ST40, the coefficient data Wi of each class is stored in the memory, and then the process ends in step ST41.
[0159]
In this way, by performing processing according to the flowchart shown in FIG. 12, the coefficient data Wi of each class can be obtained by the same method as the coefficient data generation device 150 shown in FIG.
[0160]
In the above embodiment, the motion vector information (Δx, Δy) is detected using the edge component extracted from the reference data and the edge component extracted from the residual data, and this is used for decoding. This is information indicating the inaccuracy of the compensation vector information MI.
[0161]
However, the information indicating the inaccuracy of the motion compensation vector information MI is not limited to the motion vector information (Δx, Δy). For example, motion vector information MI is newly detected from the image signal Va by a method such as a block matching method or a gradient method, and this is compared with the vector information MI for motion compensation used at the time of decoding, thereby obtaining vector information MI for motion compensation. You may make it obtain the information which shows inaccuracy.
[0162]
In the above embodiment, the MPEG2 stream with DCT is handled. However, the present invention is also applicable to other coded digital image signals subjected to motion compensation predictive coding. Can be applied to. Further, instead of DCT, encoding with other orthogonal transforms such as a wavelet transform and a discrete sine transform may be used.
[0163]
【The invention's effect】
According to the present invention, a first image signal made up of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensation predictive coding is encoded noise from a plurality of pixel data. Is converted to the second image signal with reduced light, the second used for motion compensation of the reference data used to obtain the pixel data of the first image signal corresponding to the position of interest in the second image signal Information indicating the inaccuracy of one motion vector information is acquired, and a class to which the pixel data of the target position in the second image signal belongs is detected using at least the information indicating the inaccuracy, and the class Thus, pixel data of the target position in the second image signal is generated, the accuracy of classification can be improved, and the quality of the second image signal can be improved.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a digital broadcast receiver as an embodiment.
FIG. 2 is a block diagram showing a configuration of an MPEG2 decoder.
FIG. 3 is a block diagram illustrating a configuration of a class generation unit.
FIG. 4 is a diagram illustrating an example of a coefficient ratio of a 3 × 3 Laplacian filter.
FIG. 5 is a diagram for explaining an example of block matching for correlation determination;
FIG. 6 is a diagram for explaining another example of block matching for correlation determination.
FIG. 7 is a block diagram illustrating a configuration of a class classification unit.
FIG. 8 is a diagram illustrating a tap selection block.
FIG. 9 is a block diagram illustrating a configuration of a coefficient data generation device.
FIG. 10 is a block diagram illustrating a configuration example of an image signal processing apparatus to be realized by software.
FIG. 11 is a flowchart showing image signal processing;
FIG. 12 is a flowchart showing coefficient data generation processing.
[Explanation of symbols]
31, 32... Input terminal, 32, 35... Edge component extraction circuit, 33, 36... Buffer memory, 37. Terminal: 100 ... Digital broadcast receiver 101 ... System controller 102 ... Remote control signal receiving circuit 105 ... Receiving antenna 106 ... Tuner unit 107 ... MPEG2 decoder 108, buffer memory, 110, image signal processing unit, 111, display unit, 121, class generation unit, 122, prediction tap selection circuit, 123, class classification unit, 124 .. Coefficient memory, 125 ... Estimated prediction calculation circuit, 150 ... Coefficient data generation device, 300 ... Image signal processing device

Claims (9)

A first image signal composed of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensation predictive coding is converted into a second image composed of a plurality of pixel data with reduced coding noise . An image signal processing device for converting to an image signal of
First extraction means for extracting an edge component from residual data used when obtaining pixel data of the first image signal corresponding to a target position in the second image signal;
Second extraction means for extracting an edge component from reference data used in obtaining pixel data of the first image signal corresponding to a target position in the second image signal ;
The frame composed of the edge component extracted by the first extraction means is defined as a first frame, the frame composed of the edge component extracted by the second extraction means is defined as a second frame, and the frame of the first frame From the correlation information between the block and the block of the second frame, second motion vector information corresponding to the position of interest in the second image signal is obtained, and the first motion vector used for motion compensation of the reference data is obtained. Motion vector acquisition means for acquiring the second motion vector information as information indicating inaccuracy of the motion vector information;
Class generation means for generating a class to which the pixel data at the position of interest in the second image signal belongs from at least the second motion vector information acquired by the motion vector acquisition means ;
A teacher signal that is obtained in advance for each class generated by the class generating means and includes a student signal including encoding noise corresponding to the first image signal and an encoding noise corresponding to the second image signal. Coefficient data generating means for generating coefficient data that minimizes an error from the signal;
Data selection means for selecting, from the first image signal, a plurality of pixel data located around the position of interest in the second image signal;
Images that calculates a plurality of pixel data selected by the generated coefficient data and said data selecting means in said coefficient data generation means Ru and an arithmetic means for obtaining pixel data of the target position in the second image signal Signal processing device. The motion vector acquisition means includes
The first frame or the second frame is a reference frame, the second frame or the first frame is a search frame, and an edge component corresponding to the position of interest in the second image signal of the reference frame Correlation detecting means for detecting correlation information between a reference block including a plurality of candidate blocks within a predetermined search range centered on a target position in the second image signal of the search frame;
Information output means for outputting position information of a candidate block having the highest correlation with the reference block as the second motion vector information from correlation information corresponding to the plurality of candidate blocks detected by the correlation detection means. the image signal processing apparatus according to 請 Motomeko 1 that Yusuke. When the correlation level indicated by the correlation information of the candidate block with the highest correlation is smaller than a preset threshold, the information output means outputs predetermined information indicating that,
When the information output means outputs the predetermined information, the class generation means uses the predetermined information instead of the second motion vector information to obtain pixel data of the target position in the second image signal. the image signal processing apparatus according to 請 Motomeko 2 that generates a class to which it belongs. The digital image signal has been subjected to MPEG encoding,
The reference block, an image signal according to 請 Motomeko 2 Ru blocks der corresponding to DCT blocks used in obtaining the pixel data of the first image signal corresponding to the target position in the second image signal Processing equipment. A first image signal composed of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensated predictive coding is converted into a second image composed of a plurality of pixel data with reduced coding noise . An image signal processing method for converting into an image signal of
A first step of extracting an edge component from residual data used when obtaining pixel data of the first image signal corresponding to a target position in the second image signal;
A second step of extracting an edge component from the reference data used in obtaining the pixel data of the first image signal corresponding to the target position in the second image signal ;
The frame composed of the edge component extracted in the first step is defined as a first frame, the frame composed of the edge component extracted in the second step is defined as a second frame, and the block of the first frame is defined as First motion vector used for motion compensation of the reference data by obtaining second motion vector information corresponding to the position of interest in the second image signal from the correlation information between the blocks of the second frame. A third step of obtaining the second motion vector information as information indicating information inaccuracy;
A fourth step of generating a class to which the pixel data of the target position in the second image signal belongs from at least the second motion vector information acquired in the third step;
A student signal that is obtained in advance for each class generated in the fourth step and includes coding noise corresponding to the first image signal and does not include coding noise corresponding to the second image signal. A fifth step of generating coefficient data that minimizes an error from the teacher signal;
A sixth step of selecting, from the first image signal, a plurality of pixel data located around a target position in the second image signal;
A seventh step of obtaining the pixel data of the target position in the second image signal by calculating the coefficient data generated in the fifth step and the plurality of pixel data selected in the sixth step. images signal processing method that. A first image signal composed of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensated predictive coding is converted into a second image composed of a plurality of pixel data with reduced coding noise . To convert the image signal to
A first step of extracting an edge component from residual data used when obtaining pixel data of the first image signal corresponding to a target position in the second image signal;
A second step of extracting an edge component from the reference data used in obtaining the pixel data of the first image signal corresponding to the target position in the second image signal ;
The frame composed of the edge component extracted in the first step is defined as a first frame, the frame composed of the edge component extracted in the second step is defined as a second frame, and the block of the first frame is defined as First motion vector used for motion compensation of the reference data by obtaining second motion vector information corresponding to the position of interest in the second image signal from the correlation information between the blocks of the second frame. A third step of obtaining the second motion vector information as information indicating information inaccuracy;
A fourth step of generating a class to which the pixel data of the target position in the second image signal belongs from at least the second motion vector information acquired in the third step;
A student signal that is obtained in advance for each class generated in the fourth step and includes coding noise corresponding to the first image signal and does not include coding noise corresponding to the second image signal. A fifth step of generating coefficient data that minimizes an error from the teacher signal;
A sixth step of selecting, from the first image signal, a plurality of pixel data located around a target position in the second image signal;
A seventh step of obtaining the pixel data of the target position in the second image signal by calculating the coefficient data generated in the fifth step and the plurality of pixel data selected in the sixth step. A program for causing a computer to execute an image signal processing method. A first image signal composed of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensated predictive coding is converted into a second image composed of a plurality of pixel data with reduced coding noise . an apparatus for generating a locking number data that is used when converting the image signal,
Decoding means for decoding a digital image signal obtained by encoding a teacher signal corresponding to the second image signal to obtain a student signal corresponding to the first image signal;
First extraction means for extracting an edge component from residual data used when obtaining pixel data of the student signal corresponding to a target position in the teacher signal;
Second extraction means for extracting an edge component from reference data used when obtaining pixel data of the student signal corresponding to a target position in the teacher signal ;
The frame composed of the edge component extracted by the first extraction means is defined as a first frame, the frame composed of the edge component extracted by the second extraction means is defined as a second frame, and the frame of the first frame First motion vector information used for motion compensation of the reference data by obtaining second motion vector information corresponding to the position of interest in the teacher signal from the correlation information between the block and the block of the second frame Motion vector acquisition means for acquiring the second motion vector information as information indicating the inaccuracy of
Class generation means for generating a class to which the pixel data of the target position in the teacher signal belongs from at least the second motion vector information acquired by the motion vector acquisition means ;
Data selection means for selecting a plurality of pixel data located around the target position in the teacher signal from the student signal;
Class generated by the class generation means, from the pixel data for each of the classes of the target position in the plurality of pixel data and the teacher signal selected by the data selection means, a plurality of pixel data and the teacher according to the student signal engaging the number of data generating device Ru and an arithmetic means for obtaining the coefficient data that minimizes the error between the pixel data of the target position in the signal. A first image signal composed of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensated predictive coding is converted into a second image composed of a plurality of pixel data with reduced coding noise . a method of producing the engagement number data that are used in converting the image signals,
A first step of decoding a digital image signal obtained by encoding a teacher signal corresponding to the second image signal to obtain a student signal corresponding to the first image signal;
A second step of extracting an edge component from the residual data used when obtaining the pixel data of the student signal corresponding to the target position in the teacher signal;
A third step of extracting an edge component from reference data used when obtaining pixel data of the student signal corresponding to the target position in the teacher signal ;
The frame composed of the edge component extracted in the second step is a first frame, the frame composed of the edge component extracted in the third step is a second frame, and the block of the first frame is From the correlation information between the blocks of the second frame, second motion vector information corresponding to the position of interest in the teacher signal is obtained, and the first motion vector information used for motion compensation of the reference data is determined. A fourth step of acquiring the second motion vector information as information indicating accuracy;
A fifth step of generating a class to which the pixel data of the target position in the teacher signal belongs from at least the second motion vector information acquired in the fourth step;
A sixth step of selecting, from the student signal, a plurality of pixel data located around a target position in the teacher signal;
The fifth class generated in step, from the pixel data for each of the classes of the target position in the sixth plurality of pixel data and the teacher signal selected in step, and a plurality of pixel data in accordance with the student signal seventh step and engaging number data generation process of Ru with a seeking coefficient data that minimizes the error between the pixel data of the target position in the teacher signal. A first image signal composed of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensated predictive coding is converted into a second image composed of a plurality of pixel data with reduced coding noise . to generate the engagement number data that is used when converting the image signal,
A first step of decoding a digital image signal obtained by encoding a teacher signal corresponding to the second image signal to obtain a student signal corresponding to the first image signal;
A second step of extracting an edge component from the residual data used when obtaining the pixel data of the student signal corresponding to the target position in the teacher signal;
A third step of extracting an edge component from reference data used when obtaining pixel data of the student signal corresponding to the target position in the teacher signal ;
The frame composed of the edge component extracted in the second step is a first frame, the frame composed of the edge component extracted in the third step is a second frame, and the block of the first frame is From the correlation information between the blocks of the second frame, second motion vector information corresponding to the position of interest in the teacher signal is obtained, and the first motion vector information used for motion compensation of the reference data is determined. A fourth step of acquiring the second motion vector information as information indicating accuracy;
A fifth step of generating a class to which the pixel data of the target position in the teacher signal belongs from at least the second motion vector information acquired in the fourth step;
A sixth step of selecting, from the student signal, a plurality of pixel data located around a target position in the teacher signal;
The fifth class generated in step, from the pixel data for each of the classes of the target position in the sixth plurality of pixel data and the teacher signal selected in step, and a plurality of pixel data in accordance with the student signal A program for causing a computer to execute a coefficient data generation method comprising: a seventh step for obtaining coefficient data that minimizes an error from pixel data of a target position in the teacher signal .

Images