JP4158474B2 - Image signal processing apparatus and processing method, coefficient data generating apparatus and generating method used therefor, and program for executing each method - Google Patents

Description

【０１２３】
この画像信号処理部１１０の動作を説明する。
残差クラス生成部１２１では、画像信号Ｖｂの注目位置に対応した画像信号Ｖａの画素データと対となってバッファメモリ１０８に記憶されている動き補償用ベクトル情報ＭＩおよびピクチャ情報ＰＩと、内蔵のメモリ３２，３４に格納されている量子化特性指定情報ＱＩおよび量子化ＤＣＴ係数が用いられて、残差クラスコードＣＬ０が生成される。この残差クラスコードＣＬ０は、ピクチャ情報ＰＩがＩピクチャを示す場合には特定のコードとなり、ピクチャ情報ＰＩがＰピクチャまたはＢピクチャを示すときは、ブロック段差歪みの補正成分が除去された注目ＤＣＴデータブロックの各ＡＣ係数に基づいて生成される。
【０１２４】
また、クラス分類部１２３では、バッファメモリ１０８に記憶されている画像信号Ｖａを構成する複数の画素データ、バッファメモリ１０８に画像信号Ｖｂにおける注目位置の画素データに対応した画像信号Ｖａの画素データと対となって格納されている画素位置モード情報ｐｉ、残差クラス生成部１２１で生成される残差クラスコードＣＬ０を用いて、画像信号Ｖｂにおける注目位置の画素データが属するクラスを示すクラスコードＣＬが生成される。
【０１２５】
このようにクラス分類部１２３で生成されるクラスコードＣＬは読み出しアドレス情報として係数メモリ１２４に供給される。これにより、係数メモリ１２４からクラスコードＣＬに対応した係数データＷｉが読み出されて、推定予測演算回路１２５に供給される。
【０１２６】
また、バッファメモリ１０８に記憶されている画像信号Ｖａより、予測タップ選択回路１２２で、画像信号Ｖｂにおける注目位置の周辺に位置する予測タップの画素データが選択的に取り出される。
【０１２７】
推定予測演算回路１２５では、予測タップの画素データｘｉと、係数メモリ１２４より読み出される係数データＷｉとを用いて、上述の（１）式に示す推定式に基づいて、作成すべき画像信号Ｖｂにおける注目位置の画素データｙが求められる。
【０１２８】
このように画像信号処理部１１０では、画像信号Ｖａから係数データＷｉを用いて画像信号Ｖｂが得られる。この場合、画像信号Ｖａに基づいて選択された、画像信号Ｖｂにおける注目位置の周辺に位置する複数の画素データ（予測タップの画素データ）、およびこの画像信号Ｖｂにおける注目位置の画素データが属するクラスＣＬに対応した係数データＷｉを用いて、推定式に基づいて画像信号Ｖｂにおける注目位置の画素データｙを生成するものである。
【０１２９】
したがって、係数データＷｉとして、画像信号Ｖａに対応しこの画像信号Ｖａと同様の符号化雑音を含む生徒信号と画像信号Ｖｂに対応した符号化雑音を含まない教師信号とを用いた学習によって得られた係数データＷｉを用いることで、画像信号Ｖｂとして画像信号Ｖａに比べて符号化雑音が大幅に軽減されたものを良好に得ることができる。
【０１３０】
また、残差クラス生成部１２１では、ピクチャ情報ＰＩがＰピクチャまたはＢピクチャを示すとき、注目ＤＣＴデータブロックの各ＡＣ係数に基づいて残差クラスコードＣＬ０を生成するものであるが、この注目ＤＣＴデータブロックの各ＡＣ係数として、注目リファレンスデータブロックに含まれるブロック段差歪みの補正成分が除去されたものが用いられるものであり、残差クラスのクラス分類の精度を向上させることができ、画像信号Ｖｂの品質の向上を図ることができる。
【０１３１】
次に、係数メモリ１２４に記憶される係数データＷｉの生成方法について説明する。この係数データＷｉは、予め学習によって生成されたものである。
【０１３２】
まず、この学習方法について説明する。上述の、（１）式において、学習前は係数データＷ₁，Ｗ₂，‥‥，Ｗ_nは未定係数である。学習は、クラス毎に、複数の信号データに対して行う。学習データ数がｍの場合、（１）式に従って、以下に示す（２）式が設定される。ｎは予測タップの数を示している。
【０１３３】
ｙ_k＝Ｗ₁×ｘ_k1＋Ｗ₂×ｘ_k2＋‥‥＋Ｗ_n×ｘ_kn ・・・（２）
（ｋ＝１，２，‥‥，ｍ）
【０１３４】
ｍ＞ｎの場合、係数データＷ₁，Ｗ₂，‥‥，Ｗ_nは、一意に決まらないので、誤差ベクトルｅの要素ｅ_kを、以下の式（３）で定義して、（４）式のｅ²を最小にする係数データを求める。いわゆる最小２乗法によって係数データを一意に定める。
【０１３５】
ｅ_k＝ｙ_k−｛Ｗ₁×ｘ_k1＋Ｗ₂×ｘ_k2＋‥‥＋Ｗ_n×ｘ_kn｝ ・・・（３）
（ｋ＝１，２，‥‥ｍ）
【０１３６】
【数２】

【０１３７】
（４）式のｅ²を最小とする係数データを求めるための実際的な計算方法としては、まず、（５）式に示すように、ｅ²を係数データＷｉ(ｉ＝１，２，・・・，ｎ）で偏微分し、ｉの各値について偏微分値が０となるように係数データＷｉを求めればよい。
【０１３８】
【数３】

【０１３９】
（５）式から係数データＷｉを求める具体的な手順について説明する。（６）式、（７）式のようにＸji，Ｙiを定義すると、（５）式は、（８）式の行列式の形に書くことができる。
【０１４０】
【数４】

【０１４１】
【数５】

【０１４２】
（８）式は、一般に正規方程式と呼ばれるものである。この正規方程式を掃き出し法（Gauss-Jordanの消去法）等の一般解法で解くことにより、係数データＷｉ（ｉ＝１，２，・・・，ｎ）を求めることができる。
【０１４３】
図８は、図１の画像信号処理部１１０の係数メモリ１２４に格納すべき係数データＷｉを生成する係数データ生成装置１５０の構成を示している。
この係数データ生成装置１５０は、画像信号Ｖｂに対応した教師信号ＳＴが入力される入力端子１５１と、この教師信号ＳＴに対して符号化を行ってＭＰＥＧ２ストリームを得るＭＰＥＧ２符号化器１５２と、このＭＰＥＧ２ストリームに対して復号化を行って画像信号Ｖａに対応した生徒信号ＳＳを得るＭＰＥＧ２復号化器１５３とを有している。ここで、ＭＰＥＧ２復号化器１５３は、図１に示すデジタル放送受信機１００におけるＭＰＥＧ２復号化器１０７およびバッファメモリ１０８に対応したものである。
【０１４４】
また、係数データ生成装置１５０は、残差クラス生成部１５４を有している。この残差クラス生成部１５４は、上述した画像信号処理部１１０の残差クラス生成部１２１と同様に構成され、教師信号ＳＴにおける注目位置の画素データが属する残差クラスを示す残差クラスコードＣＬ０を生成する。この残差クラス生成部１５４に内蔵されたメモリには、ＭＰＥＧ２復号化器１５３より出力される量子化特性指定情報ＱＩおよび量子化ＤＣＴ係数が一時的に格納される。この残差クラス生成部１５４では、教師信号ＳＴの注目位置に対応した生徒信号ＳＳの画素データと対となっている動き補償用ベクトル情報ＭＩおよびピクチャ情報ＰＩと、内蔵のメモリに格納されている量子化特性指定情報ＱＩおよび量子化ＤＣＴ係数が用いられて、残差クラスコードＣＬ０が生成される。
【０１４５】
また、係数データ生成装置１５０は、ＭＰＥＧ２復号化器１５３より出力される生徒信号ＳＳより、教師信号ＳＴにおける注目位置の周辺に位置する複数の画素データを選択的に取り出して出力する予測タップ選択回路１５５を有している。この予測タップ選択回路１５５は、上述した画像信号処理部１１０の予測タップ選択回路１２２と同様に構成される。
【０１４６】
また、係数データ生成装置１５０は、教師信号ＳＴにおける注目位置の画素データが属するクラスを検出するクラス検出手段としてのクラス分類部１５６を有している。このクラス分類部１５６は、上述した画像信号処理部１１０のクラス分類部１２３と同様に構成される。
【０１４７】
このクラス分類部１５６は、ＭＰＥＧ２復号化器１５３より得られる生徒信号ＳＳを構成する複数の画素データ、ＭＰＥＧ２復号化器１５３より得られる、教師信号ＳＴにおける注目位置の画素データに対応した生徒信号ＳＳの画素データと対となっている画素位置モード情報ｐｉ、および残差クラス生成部１５４で生成される残差クラスコードＣＬ０を用いて、教師信号ＳＴにおける注目位置の画素データが属するクラスを示すクラスコードＣＬを生成する。
【０１４８】
また、係数データ生成装置１５０は、入力端子１５１に供給される教師信号ＳＴの時間調整を行うための遅延回路１５７と、この遅延回路１５７で時間調整された教師信号ＳＴより得られる各注目位置の画素データｙと、この各注目位置の画素データｙにそれぞれ対応して予測タップ選択回路１５５で選択的に取り出される予測タップの画素データｘｉと、各注目位置の画素データｙにそれぞれ対応してクラス分類部１５６で生成されるクラスコードＣＬとから、クラス毎に、係数データＷｉ（ｉ＝１〜ｎ）を得るための正規方程式（上述の（８）式参照）を生成する正規方程式生成部１５８を有している。
【０１４９】
この場合、１個の画素データｙとそれに対応するｎ個の予測タップの画素データｘｉとの組み合わせで１個の学習データが生成されるが、教師信号ＳＴと生徒信号ＳＳとの間で、クラス毎に、多くの学習データが生成されていく。これにより、正規方程式生成部１５８では、クラス毎に、係数データＷｉ（ｉ＝１〜ｎ）を得るための正規方程式が生成される。
【０１５０】
また、係数データ生成装置１５０は、正規方程式生成部１５８で生成された正規方程式のデータが供給され、その正規方程式を解いて、各クラスの係数データＷｉを求める係数データ決定部１５９と、この求められた各クラスの係数データＷｉを格納する係数メモリ１６０とを有している。
【０１５１】
次に、図８に示す係数データ生成装置１５０の動作を説明する。
入力端子１５１には画像信号Ｖｂに対応した教師信号ＳＴが供給され、そしてＭＰＥＧ２符号化器１５２で、この教師信号ＳＴに対して符号化が施されて、ＭＰＥＧ２ストリームが生成される。このＭＰＥＧ２ストリームは、ＭＰＥＧ２復号化器１５３に供給される。ＭＰＥＧ２復号化器１５３で、このＭＰＥＧ２ストリームに対して復号化が施されて、画像信号Ｖａに対応した生徒信号ＳＳが生成される。
【０１５２】
残差クラス生成部１５４では、教師信号ＳＴの注目位置に対応した生徒信号ＳＳの画素データと対となっている動き補償用ベクトル情報ＭＩおよびピクチャ情報ＰＩと、内蔵のメモリに格納されている量子化特性指定情報ＱＩおよび量子化ＤＣＴ係数が用いられて、教師信号ＳＴにおける注目位置の画素データが属する残差クラスを示す残差クラスコードＣＬ０が生成される。
【０１５３】
クラス分類部１５６では、ＭＰＥＧ２復号化器１５３より得られる生徒信号ＳＳを構成する複数の画素データ、ＭＰＥＧ２復号化器１５３より得られる、教師信号ＳＴにおける注目位置の画素データに対応した生徒信号ＳＳの画素データと対となっている画素位置モード情報ｐｉ、残差クラス生成部１５４で生成された残差クラスコードＣＬ０を用いて、教師信号ＳＴにおける注目位置の画素データが属するクラスを示すクラスコードＣＬが生成される。
【０１５４】
また、ＭＰＥＧ２復号化器１５３より得られる生徒信号ＳＳより、予測タップ選択回路１５５で、教師信号ＳＴにおける注目位置の周辺に位置する予測タップの画素データが選択的に取り出される。
【０１５５】
そして、遅延回路１５７で時間調整された教師信号ＳＴから得られる各注目位置の画素データｙと、この各注目位置の画素データｙにそれぞれ対応して予測タップ選択回路１５５で選択的に取り出される予測タップの画素データｘｉと、各注目位置の画素データｙにそれぞれ対応してクラス分類部１５６で生成されるクラスコードＣＬとを用いて、正規方程式生成部１５８では、クラス毎に、係数データＷｉ（ｉ＝１〜ｎ）を得るための正規方程式（（８）式参照）が生成される。この正規方程式は係数データ決定部１５９で解かれて各クラスの係数データＷｉが求められ、その係数データＷｉは係数メモリ１６０に格納される。
【０１５６】
このように、図８に示す係数データ生成装置１５０においては、図１の画像信号処理部１１０の係数メモリ１２４に格納される各クラスの係数データＷｉを生成することができる。
【０１５７】
生徒信号ＳＳは、教師信号ＳＴに対して符号化を施してＭＰＥＧ２ストリームを生成し、その後このＭＰＥＧ２ストリームに対して復号化を施して得たものである。したがって、この生徒信号ＳＳは、画像信号Ｖａと同様の符号化雑音を含んだものとなる。そのため、図１に示す画像信号処理部１１０において、画像信号Ｖａからこの係数データＷｉを用いて得られる画像信号Ｖｂは、画像信号Ｖａに比べて符号化雑音が軽減されたものとなる。
【０１５８】
なお、図１の画像信号処理部１１０における処理を、例えば図９に示すような画像信号処理装置３００によって、ソフトウェアで実現することも可能である。
【０１５９】
まず、図９に示す画像信号処理装置３００について説明する。この画像信号処理装置３００は、装置全体の動作を制御するＣＰＵ３０１と、このＣＰＵ３０１の制御プログラムや係数データ等が格納されたＲＯＭ（Read Only Memory）３０２と、ＣＰＵ３０１の作業領域を構成するＲＡＭ（Random Access Memory）３０３とを有している。これらＣＰＵ３０１、ＲＯＭ３０２およびＲＡＭ３０３は、それぞれバス３０４に接続されている。
【０１６０】
また、画像信号処理装置３００は、外部記憶装置としてのハードディスクドライブ（ＨＤＤ）３０５と、フロッピー（登録商標）ディスク３０６をドライブするドライブ（ＦＤＤ）３０７とを有している。これらドライブ３０５，３０７は、それぞれバス３０４に接続されている。
【０１６１】
また、画像信号処理装置３００は、インターネット等の通信網４００に有線または無線で接続する通信部３０８を有している。この通信部３０８は、インタフェース３０９を介してバス３０４に接続されている。
【０１６２】
また、画像信号処理装置３００は、ユーザインタフェース部を備えている。このユーザインタフェース部は、リモコン送信機２００からのリモコン信号ＲＭを受信するリモコン信号受信回路３１０と、ＬＣＤ（liquid Crystal Display）等からなるディスプレイ３１１とを有している。受信回路３１０はインタフェース３１２を介してバス３０４に接続され、同様にディスプレイ３１１はインタフェース３１３を介してバス３０４に接続されている。
【０１６３】
また、画像信号処理装置３００は、画像信号Ｖａを入力するための入力端子３１４と、画像信号Ｖｂを出力するための出力端子３１５とを有している。入力端子３１４はインタフェース３１６を介してバス３０４に接続され、同様に出力端子３１５はインタフェース３１７を介してバス３０４に接続される。
【０１６４】
ここで、上述したようにＲＯＭ３０２に制御プログラムや係数データ等を予め格納しておく代わりに、例えばインターネットなどの通信網４００より通信部３０８を介してダウンロードし、ハードディスクやＲＡＭ３０３に蓄積して使用することもできる。また、これら制御プログラムや係数データ等をフロッピー（登録商標）ディスク３０６で提供するようにしてもよい。
【０１６５】
また、処理すべき画像信号Ｖａを入力端子３１４より入力する代わりに、予めハードディスクに記録しておき、あるいはインターネットなどの通信網４００より通信部３０８を介してダウンロードしてもよい。また、処理後の画像信号Ｖｂを出力端子３１５に出力する代わり、あるいはそれと並行してディスプレイ３１１に供給して画像表示をしたり、さらにはハードディスクに格納したり、通信部３０８を介してインターネットなどの通信網４００に送出するようにしてもよい。
【０１６６】
図１０のフローチャートを参照して、図９に示す画像信号処理装置３００における、画像信号Ｖａより画像信号Ｖｂを得るため処理手順を説明する。
まず、ステップＳＴ２１で、処理を開始し、ステップＳ２２で、例えば入力端子３１４より装置内に１フレーム分または１フィールド分の画像信号Ｖａを入力する。この場合、画像信号Ｖａの各ＤＣＴブロック部分に対応したＤＣＴ係数および量子化特性指定情報ＱＩ、さらには画像信号Ｖａの画素データと対となっている画素位置モード情報ｐｉ、動き補償用ベクトル情報ＭＩおよびピクチャ情報ＰＩも入力する。
【０１６７】
このように入力端子３１４より入力される画像信号Ｖａ等はＲＡＭ３０３に一時的に格納される。なお、この画像信号Ｖａ等が装置内のハードディスクドライブ３０５に予め記録されている場合には、このドライブ３０５からこの画像信号Ｖａ等を読み出し、この画像信号Ｖａ等をＲＡＭ３０３に一時的に格納する。
【０１６８】
そして、ステップＳＴ２３で、画像信号Ｖａの全フレームまたは全フィールドの処理が終わっているか否かを判定する。処理が終わっているときは、ステップＳＴ２４で、処理を終了する。一方、処理が終わっていないときは、ステップＳＴ２５に進む。
【０１６９】
ステップＳＴ２５では、画像信号Ｖｂの注目位置に対応した画像信号Ｖａの画素データと対となっている動き補償用ベクトル情報ＭＩおよびピクチャ情報ＰＩと、量子化特性指定情報ＱＩおよびＤＣＴ係数を用いて、残差クラスコードＣＬ０を生成し、さらにこの残差クラスコードＣＬ０、画像信号Ｖａを構成する複数の画素データおよび画素位置モード情報ｐｉを用いて、画像信号Ｖｂにおける注目位置の画素データが属するクラスを示すクラスコードＣＬを生成する。
【０１７０】
次に、ステップＳＴ２６で、ステップＳＴ２２で入力された画像信号Ｖａより、画像信号Ｖｂにおける注目位置の周辺に位置する複数の画素データ（予測タップの画素データ）を取得する。そして、ステップＳＴ２７で、ステップＳＴ２５で生成されたクラスコードＣＬに対応した係数データＷｉとステップＳＴ２６で取得された予測タップの画素データｘｉを使用して、（１）式の推定式に基づいて、画像信号Ｖｂにおける注目位置の画素データｙを生成する。
【０１７１】
次に、ステップＳＴ２８で、ステップＳＴ２２で入力された１フレームまたは１フィールド分の画像信号Ｖａの画素データの全領域において画像信号Ｖｂの画素データを得る処理が終了したか否かを判定する。終了しているときは、ステップＳＴ２２に戻り、次の１フレーム分または１フィールド分の画像信号Ｖａの入力処理に移る。一方、処理が終了していないときは、ステップＳＴ２５に戻って、次の注目位置についての処理に移る。
【０１７２】
このように、図１０に示すフローチャートに沿って処理をすることで、入力された画像信号Ｖａの画素データを処理して、画像信号Ｖｂの画素データを得ることができる。上述したように、このように処理して得られた画像信号Ｖｂは出力端子３１５に出力されたり、ディスプレイ３１１に供給されてそれによる画像が表示されたり、さらにはハードディスクドライブ３０５に供給されてハードディスクに記録されたりする。
【０１７３】
また、処理装置の図示は省略するが、図８の係数データ生成装置１５０における処理も、ソフトウェアで実現可能である。
【０１７４】
図１１のフローチャートを参照して、係数データを生成するための処理手順を説明する。
まず、ステップＳＴ３１で、処理を開始し、ステップＳＴ３２で、教師信号ＳＴを１フレーム分または１フィールド分だけ入力する。そして、ステップＳＴ３３で、教師信号ＳＴの全フレームまたは全フィールドの処理が終了したか否かを判定する。終了していないときは、ステップＳＴ３４で、ステップＳＴ３２で入力された教師信号ＳＴから生徒信号ＳＳを生成する。この場合、生徒信号ＳＳの各ＤＣＴブロック部分に対応したＤＣＴ係数および量子化特性指定情報ＱＩ、さらには生徒信号ＳＳの画素データと対となっている画素位置モード情報ｐｉ、動き補償用ベクトル情報ＭＩおよびピクチャ情報ＰＩも得る。
【０１７５】
そして、ステップＳＴ３５で、教師信号ＳＴの注目位置に対応した生徒信号ＳＳの画素データと対となっている動き補償用ベクトル情報ＭＩおよびピクチャ情報ＰＩと、量子化特性指定情報ＱＩおよびＤＣＴ係数を用いて、残差クラスコードＣＬ０を生成し、さらにこの残差クラスコードＣＬ０、生徒信号ＳＳを構成する複数の画素データおよび画素位置モード情報ｐｉを用いて、教師信号ＳＴにおける注目位置の画素データが属するクラスを示すクラスコードＣＬを生成する。
【０１７６】
次に、ステップＳＴ３６で、ステップＳＴ３４で生成された生徒信号ＳＳより、教師信号ＳＴにおける注目位置の周辺に位置する複数の画素データ（予測タップの画素データ）を取得する。
【０１７７】
そして、ステップＳＴ３７で、ステップＳＴ３５で生成されたクラスコードＣＬ、ステップＳＴ３６で取得された予測タップの画素データｘｉおよび教師信号ＳＴにおける注目位置の画素データｙを用いて、クラス毎に、（８）式に示す正規方程式を得るための加算をする（（６）式、（７）式参照）。
【０１７８】
次に、ステップＳＴ３８で、ステップＳＴ３２で入力された１フレーム分または１フィールド分の教師信号ＳＴの画素データの全領域において学習処理が終了したか否かを判定する。学習処理を終了しているときは、ステップＳＴ３２に戻って、次の１フレーム分または１フィールド分の教師信号ＳＴの入力を行って、上述したと同様の処理を繰り返す。一方、学習処理を終了していないときは、ステップＳＴ３５に戻って、次の注目位置についての処理に移る。
【０１７９】
上述したステップＳＴ３３で、処理が終了したときは、ステップＳＴ３９で、上述のステップＳＴ３７の加算処理によって生成された、各クラスの正規方程式を掃き出し法などで解いて、各クラスの係数データＷｉを算出する。そして、ステップＳＴ４０で、各クラスの係数データＷｉをメモリに保存し、その後にステップＳＴ４１で、処理を終了する。
【０１８０】
このように、図１１に示すフローチャートに沿って処理をすることで、図８に示す係数データ生成装置１５０と同様の手法によって、各クラスの係数データＷｉを得ることができる。
【０１８１】
なお、上述実施の形態において、画像信号処理部１１０の残差クラス生成部１２１では、使用ＤＣＴデータブロックを用いるものを示したが、この代わりに使用画素データブロックを用いる構成とすることもできる。
【０１８２】
図１２は、その場合における残差クラス生成部１２１Ａの構成を示している。この図１２において、図３と対応する部分には同一符号を付し、その詳細説明は省略する。
【０１８３】
この残差クラス生成部１２１Ａは、上述したＭＰＥＧ２復号化器１０７（図２参）より出力される画像信号Ｖａが入力される入力端子４５と、この入力端子４５に入力される画像信号Ｖａを一時的に格納するためのメモリ４６とを有している。入力端子３５に入力される情報ＭＩ，ＰＩは、メモリ４６に読み出し制御情報として供給される。ピクチャ情報ＰＩがＰピクチャまたはＢピクチャを示す場合、メモリ４６からは、一個または複数個の使用画素データブロックが読み出される。図１３Ａは、例えば、Ｐピクチャの場合であって、４個の使用画素データブロックＤＢａ〜ＤＢｄが用いられた場合を示している。この図１３Ａにおいて、破線は、注目リファレンスデータブロックのブロック位置を示している。なお、この図１３Ａは、図面の簡単化のため、画素データブロックＤＢａ〜ＤＢｄが４×４の画素データを含むように示している。
【０１８４】
また、残差クラス生成部１２１Ａは、メモリ４６より読み出された画素データブロックにおける画素データの平均値をブロック毎に得る平均回路４７を有している。この場合、使用画素データブロックの個数に対応した個数の平均値が得られる。図１３Ａのように、使用画素データブロックの個数が４個である場合には、４個の平均値が得られる。
【０１８５】
また、残差クラス生成部１２１Ａは、上述した一個または複数個の使用画素データブロックを構成する画素データが平均回路４７で得られた平均値で置き換えられた状態で、この一個または複数個の使用画素データブロックより、リファレンスデータブロックのブロック位置に重なる位置に存在する画素データを選択的に取り出して一個の画素データブロックを得るブロック化回路４８を有している。
【０１８６】
例えば、図１３Ａに示すような４個の使用画素データブロックＤＢａ〜ＤＢｄを構成する画素データを平均値で置き換えた場合には、図１３Ｂに示すようになる。この場合、ｄaはａ₀〜ａ₁₅の平均値、ｄbはｂ₀〜ｂ₁₅の平均値、ｄcはｃ₀〜ｃ₁₅の平均値、ｄdはｄ₀〜ｄ₁₅の平均値である。また、リファレンスデータブロックのブロック位置に重なる位置に存在する画素データを選択的に取り出して得られる一個の画素データブロックＤＢ０は、図１３Ｃに示すようになる。
【０１８７】
図１２に示す残差クラス生成部１２１Ａのその他は、図３に示す残差クラス生成部１２１と同様に構成される。
図１２に示す残差クラス生成部１２１において、画像信号Ｖｂにおける注目位置の画素データが属する残差クラスを示す残差クラスコードＣＬ０を生成するための動作を説明する。
【０１８８】
ピクチャ情報ＰＩがＰピクチャまたはＢピクチャを示す場合、メモリ４６から一個または複数個の使用画素データブロックのＤＣＴ係数が読み出されて平均回路４７に供給される（図１３ＡのＤＢａ〜ＤＢｄ参照）。平均回路４７は、メモリ４６より読み出された画素データブロックにおける画素データの平均値をブロック毎に求める。
【０１８９】
平均回路４７で求められた平均値はブロック化回路４８に供給される。ブロック化回路４８は、上述した一個または複数個の使用画素データブロックを構成する画素データが平均回路４７で得られた平均値で置き換えられた状態で（図１３Ｂ参照）、この一個または複数個の使用画素データブロックより、リファレンスデータブロックのブロック位置（破線図示）に重なる位置に存在する画素データを選択的に取り出して一個の画素データブロックを得る（図１３ＣのＤＢ０参照）。
【０１９０】
このブロック化回路３８で得られた一個の画素データブロックはＤＣＴ回路３９に供給される。ＤＣＴ回路３９は、その一個の画素データブロックに対してＤＣＴを施し、ＤＣＴデータブロックを得る（図４ＤのＤＢ０′参照）。このようにＤＣＴ回路３９で得られたＤＣＴデータブロックは量子化回路４０に供給される。この量子化回路４０には、さらに、メモリ３２から注目ＤＣＴデータブロックに対応した量子化特性指定情報ＱＩａが読み出されて供給される。量子化回路４０は、ＤＣＴ回路３９で得られたＤＣＴデータブロックを、情報ＱＩａに基づいて量子化し、補正データとしてのＤＣＴデータブロックを得る。
【０１９１】
この補正データとしてのＤＣＴデータブロックは、一個または複数個の使用画素データブロックを構成する画素データの平均値に基づいて得られた画素データブロックにＤＣＴを施して得られたものである。そのため、この補正データとしてのＤＣＴデータブロックを構成するＡＣ係数も、図３に示す残差クラス生成部１２１におけると同様に、注目リファレンスデータブロックに含まれるブロック段差歪みに対応したものとなる。
【０１９２】
量子化回路４０で得られた補正データとしてのＤＣＴデータブロックは加算回路４１に供給される。この加算回路４１には、メモリ３４から注目ＤＣＴデータブロックが読み出されて供給される。この注目ＤＣＴデータブロックは、そのＡＣ係数部分に、注目リファレンスデータブロックに含まれるブロック段差歪みを補正する成分を有している。加算回路４１は、注目ＤＣＴデータブロックの各ＡＣ係数に、補正データとしてのＤＣＴデータブロックの対応するＡＣ係数を加算する。これにより、注目ＤＣＴデータブロックが有する、注目リファレンスデータブロックに含まれるブロック段差歪みを補正する成分が除去される。
【０１９３】
加算回路４１でブロック段差歪みの補正成分が除去された注目ＤＣＴデータブロックは逆量子化回路４２に供給される。この逆量子化回路４２には、メモリ３２から注目ＤＣＴデータブロックに対応した量子化特性指定情報ＱＩａが読み出されて供給される。逆量子化回路４２は、ブロック段差歪みの補正成分が除去された注目ＤＣＴデータブロックを、情報ＱＩａに基づいて逆量子化する。
【０１９４】
そして、クラス生成回路４３は、ピクチャ情報ＰＩがＩピクチャを示す場合には、残差クラスコードＣＬ０として、特定のコードを生成する。一方、クラス生成回路４３は、ピクチャ情報ＰＩがＰピクチャまたはＢピクチャを示すときは、逆量子化回路４２で得られるＤＣＴデータブロックの各ＡＣ係数に基づいて残差クラスコードＣＬ０を生成する。このようにクラス生成回路４３で生成されるクラスコードＣＬ０は出力端子４４に出力される。
【０１９５】
このように図１２に示す残差クラス生成部１２１Ａは、ピクチャ情報ＰＩがＰピクチャまたはＢピクチャを示すとき、注目ＤＣＴデータブロックの各ＡＣ係数に基づいて残差クラスコードＣＬ０を生成するものであるが、この注目ＤＣＴデータブロックの各ＡＣ係数として、注目リファレンスデータブロックに含まれるブロック段差歪みの補正成分が除去されたものが用いられる。したがって、残差クラスのクラス分類の精度を向上させることができ、画像信号Ｖｂの品質の向上を図ることができる。
【０１９６】
図１４のフローチャートは、上述した残差クラスコードＣＬＯの生成処理の手順を示している。この図１４において、図５と対応する部分には同一符号を付して示している。
【０１９７】
まず、ステップＳＴ１で処理を開始し、ステップＳＴ１１で、ピクチャ情報ＰＩがＰピクチャまたはＢピクチャを示す場合、メモリ４６から一個または複数個の使用画素データブロックを読み出す。そして、ステップＳＴ１２で、読み出された一個または複数個の使用画素データブロックの画素データの平均値をブロック毎に求める。
【０１９８】
次に、ステップＳＴ１３で、一個または複数個の使用画素データブロックを構成する画素データが平均値で置き換えられた状態で、この一個または複数個の使用画素データブロックより、リファレンスデータブロックのブロック位置に重なる位置に存在する画素データを選択的に取り出して一個の画素データブロックを生成する。
【０１９９】
次に、ステップＳＴ６に進む。このステップＳＴ６以下は、上述した図５のフローチャートと同様であるので、説明は省略する。
【０２００】
また、上述実施の形態においては、ＭＰＥＧ２復号化器１０７から可変長復号化回路７４で得られる量子化ＤＣＴ係数が出力され、この量子化ＤＣＴ係数を残差クラス生成部１２１，１２１Ａのメモリ３４に格納して、残差クラスコードＣＬ０の生成に利用するものである。
【０２０１】
しかし、ＭＰＥＧ２復号化器１０７から逆量子化回路７６で得られる逆量子化されたＤＣＴ係数が出力されるようにし、この逆量子化されたＤＣＴ係数を残差クラス生成部１２１，１２１Ａのメモリ３４に格納して、残差クラスコードＣＬ０の生成に利用するように構成することもできる。
【０２０２】
その場合には、残差クラス生成部１２１，１２１Ａにおいて、逆量子化、量子化等の処理は不要となる。したがってその場合には、図３の残差クラス生成部１２１において、メモリ３２、逆量子化回路３６，４２、量子化回路４０は不要となる。同様に、図１２の残差クラス生成部１２１Ａにおいて、メモリ３２、量子化回路４０、逆量子化回路４２は不要となる。
【０２０３】
また、上述実施の形態においては、ＤＣＴを伴うＭＰＥＧ２ストリームを取り扱うものを示したが、この発明は、ブロック毎に動き補償予測符号化が行われたその他の符号化されたデジタル画像信号を取り扱うものにも同様に適用することができる。また、ＤＣＴの代わりに、ウェーブレット変換、離散サイン変換などのその他の直交変換を伴う符号化であってもよい。
【０２０４】
【発明の効果】
この発明によれば、ブロック毎に動き補償予測符号化が行われたデジタル画像信号を復号化することによって生成される、複数の画素データからなる第１の画像信号を、複数の画素データからなる第２の画像信号に変換する際、少なくとも差分データブロックを用いて第２の画像信号における注目位置の画素データが属するクラスを検出し、このクラスに対応して第２の画像信号における注目位置の画素データを生成するものにあって、リファレンスデータブロックに含まれるブロック段差歪みを補正する成分を差分データブロックから除去し、この補正成分が除去された差分データブロックを上述のクラス検出に用いるものであり、クラス分類の精度が向上することから、第２の画像信号の品質の向上を図ることができる。
【図面の簡単な説明】
【図１】実施の形態としてのデジタル放送受信機の構成を示すブロック図である。
【図２】ＭＰＥＧ２復号化器の構成を示すブロック図である。
【図３】残差クラス生成部の構成を示すブロック図である。
【図４】残差クラス生成部の処理を説明するための図である。
【図５】残差クラス生成処理を示すフローチャートである。
【図６】クラス分類部の構成を示すブロック図である。
【図７】タップ選択用ブロックを示す図である。
【図８】係数データ生成装置の構成を示すブロック図である。
【図９】ソフトウェアで実現するための画像信号処理装置の構成例を示すブロック図である。
【図１０】画像信号処理を示すフローチャートである。
【図１１】係数データ生成処理を示すフローチャートである。
【図１２】残差クラス生成部の他の構成を示すブロック図である。
【図１３】残差クラス生成部の処理を説明するための図である。
【図１４】残差クラス生成処理を示すフローチャートである。
【符号の説明】
１００・・・デジタル放送受信機、１０１・・・システムコントローラ、１０２・・・リモコン信号受信回路、１０５・・・受信アンテナ、１０６・・・チューナ部、１０７・・・ＭＰＥＧ２復号化器、１０８・・・バッファメモリ、１１０・・・画像信号処理部、１１１・・・ディスプレイ部、１２１，１２１Ａ・・・残差クラス生成部、１２２・・・予測タップ選択回路、１２３・・・クラス分類部、１２４・・・係数メモリ、１２５・・・推定予測演算回路、１５０・・・係数データ生成装置、３００・・・画像信号処理装置[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image signal processing device and processing method, a coefficient data generating device and generating method used therefor, and a program for executing each method.
[0002]
Specifically, according to the present invention, a first image signal composed of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensation prediction coding for each block is generated from the plurality of pixel data. When converting to the second image signal, the class to which the pixel data of the target position in the second image signal belongs is detected using at least the difference data block, and the target position in the second image signal corresponding to this class The component that generates the pixel data of the reference data block, the component that corrects the block step distortion included in the reference data block is removed from the difference data block, and the difference data block from which the correction component is removed is used for the above-described class detection. According to the image signal processing apparatus or the like that improves the accuracy of the classification and improves the quality of the second image signal. Than is.
[0003]
[Prior art]
As an image signal compression encoding method, there is an MPEG2 (Moving Picture Experts Group 2) encoding method using DCT (Discrete Cosine Transform). In this encoding method, motion compensation prediction encoding is performed for each block.
[0004]
DCT performs discrete cosine transform on pixels in a block, requantizes coefficient data obtained by the discrete cosine transform, and further variable-length-encodes the requantized coefficient data. For this variable length coding, entropy coding such as Huffman code is often used. The image signal is orthogonally transformed to be divided into a large number of frequency data from low frequency to high frequency.
[0005]
When re-quantization is performed on this divided frequency data, human visual characteristics are taken into consideration, and low frequency data with high importance is finely quantized, and high frequency data with low importance is roughly quantized. In this way, the image quality can be maintained and high-efficiency compression can be realized.
[0006]
In conventional decoding using DCT, quantized data for each frequency component is converted into a representative value of the code, and reproduction data is obtained by performing inverse DCT (IDCT: Inverse DCT) on these components. . When converting to this representative value, the quantization step width at the time of encoding is used.
[0007]
[Problems to be solved by the invention]
As described above, the MPEG encoding method using DCT has the advantage that high-quality compression can be realized while maintaining high image quality by performing encoding in consideration of human visual characteristics.
[0008]
However, since coding for performing DCT is processing in units of blocks, block noise, so-called block noise (block distortion), may occur as the compression rate increases. In addition, in a portion where there is a rapid luminance change such as an edge, a noise that is a result of coarse quantization of high-frequency components, so-called mosquito noise, is generated.
[0009]
It can be considered that coding noise such as block noise and mosquito noise is removed by class classification adaptive processing. That is, an image signal including coding noise is set as a first image signal, an image signal from which coding noise is removed is set as a second image signal, and a class to which pixel data of a target position in the second image signal belongs is detected. Then, pixel data of the target position in the second image signal is generated corresponding to this class.
[0010]
In this case, it is also conceivable to detect the class to which the pixel data of the target position in the second image signal belongs using at least a DCT data block as a difference data block.
[0011]
However, this DCT data block has a component for correcting block step distortion included in the reference data block. Therefore, if classifying is performed using the DCT data block as it is, class classification cannot be performed with accuracy by the correction component, and it becomes difficult to improve the quality of the second image signal.
[0012]
Therefore, an object of the present invention is to improve the classification accuracy and improve the quality of the second image signal.
[0013]
[Means for Solving the Problems]
The image signal processing apparatus according to the present invention is configured to decode a first image signal made up of a plurality of pixel data, which is generated by decoding a digital image signal subjected to motion compensation predictive coding for each block. Reduced distortion caused by encoding From multiple pixel data First 2 image signals When converting to the second image signal, One or a plurality of block positions that overlap all or part of the block position of the reference data block related to the difference data block used when obtaining the pixel data of the first image signal corresponding to the target position in Orthogonal inverse transform means for performing orthogonal inverse transform on one or a plurality of difference data blocks in which each DC coefficient is a DC coefficient and all AC coefficients are 0, and the orthogonal inverse transform From one or a plurality of pixel data blocks made up of pixel data obtained by the conversion means, pixel data existing at a position overlapping the block position of the reference data block is selectively taken out to obtain a single pixel made up of pixel data. Blocking means for obtaining a data block, orthogonal transformation means for performing orthogonal transformation on the pixel data block obtained by the blocking means to obtain a difference data block composed of transformation coefficients as the correction data, and the difference data By adding the AC coefficient corresponding to the correction data to each AC coefficient of the block, Removing correction component difference data block has Correction component removal means; At least from the level distribution of the AC coefficient of the difference data block from which the correction component is removed Class detecting means for detecting a class to which the pixel data of the target position in the second image signal belongs; A teacher signal that is obtained in advance for each class detected by the class detecting means and includes a student signal including encoding distortion corresponding to the first image signal and not including encoding distortion corresponding to the second image signal. Coefficient data generating means for generating coefficient data for minimizing an error from the signal, and data selection for selecting a plurality of pixel data located around the position of interest in the second image signal from the first image signal Means for calculating the coefficient data generated by the coefficient data generating means and the plurality of pixel data selected by the data selecting means to obtain pixel data of the target position in the second image signal; With Ru Is.
[0014]
Further, the image signal processing method according to the present invention provides a first image signal made up of a plurality of pixel data, which is generated by decoding a digital image signal that has been subjected to motion compensation predictive coding for each block. Reduced distortion caused by encoding Convert to a second image signal consisting of a plurality of pixel data In case All or part of the block position overlaps the block position of the reference data block related to the difference data block used when obtaining the pixel data of the first image signal corresponding to the target position in the second image signal. One or more A first step of performing orthogonal inverse transform on one or a plurality of difference data blocks in which the DC coefficients of the difference data blocks made up of transform coefficients are DC coefficients and all AC coefficients are 0; The pixel data existing in the position overlapping the block position of the reference data block is selectively taken out from one or a plurality of pixel data blocks made up of pixel data obtained in the above step, and one pixel made up of pixel data. A second step of obtaining a data block; a third step of performing orthogonal transformation on the pixel data block obtained in the second step to obtain a difference data block composed of transformation coefficients as the correction data; By adding the corresponding AC coefficient of the correction data to each AC coefficient of the difference data block, A first component for removing the correction component of the difference data block. 4 And at least the above-mentioned step 4 Difference data block from which correction components have been removed in step From the AC coefficient level distribution Detecting a class to which pixel data of a target position in the second image signal belongs. 5 And the above steps 5 Detected in the previous step A coefficient which is obtained every time and minimizes an error between a student signal including encoding distortion corresponding to the first image signal and a teacher signal not including encoding distortion corresponding to the second image signal. A sixth step of generating data, a seventh step of selecting a plurality of pixel data located around the position of interest in the second image signal from the first image signal, and the sixth step An eighth step of obtaining the pixel data of the target position in the second image signal by calculating the coefficient data generated in step S5 and the plurality of pixel data selected in the seventh step And with Ru Is.
[0015]
A program according to the present invention is for causing a computer to execute the above-described image signal processing method.
[0016]
In the present invention, the first image signal composed of a plurality of pixel data is generated by decoding a digital image signal that has been subjected to motion compensation prediction coding for each block. For example, the digital image signal has been subjected to MPEG encoding.
[0017]
Pixel data of the first image signal corresponding to the target position in the second image signal is obtained using a predetermined difference data block and a reference data block related to the difference data block.
[0018]
The predetermined difference data block has a component that corrects block step distortion included in the reference data block. Correction data for removing the above-described correction component is obtained by using one or a plurality of data blocks in which all or part of the block positions overlap the block positions of the reference data block.
[0019]
Here, the reference data block is obtained by performing motion compensation on the differential data block, and includes data of one or a plurality of data blocks described above. Therefore, this reference data block includes block step distortion caused by one or a plurality of data blocks.
[0020]
The predetermined difference data block is obtained by subtracting the reference data block from the predetermined data block at the time of encoding. Therefore, the predetermined difference data block has a component for correcting the block step distortion included in the reference data block as described above.
[0021]
The component for correcting the block step distortion included in the difference data block is removed using the correction data obtained as described above. At least the class to which the pixel data at the target position in the second image signal belongs is detected using the difference data block from which the correction component has been removed in this way.
[0022]
Then, corresponding to the detected class, pixel data of the target position in the second image signal is generated. For example, coefficient data used in the estimation formula corresponding to the class is generated. Further, based on the first image signal, a plurality of pixel data located around the target position in the second image signal are selected. Then, these coefficient data and a plurality of pixel data are used, and pixel data of the target position in the second image signal is calculated based on the estimation formula.
[0023]
For example, as described above, when the digital image signal is MPEG-coded and the difference data block is a DCT data block including DCT coefficients, correction data is acquired as follows. The
[0024]
That is, the DC coefficients of one or a plurality of DCT data blocks composed of DCT coefficients, in which all or a part of the block position overlaps the block position of the reference data block related to the differential data block, are set as DC coefficients and all AC coefficients Inverse discrete cosine transform is performed on one or a plurality of DCT data blocks with 0 being zero.
[0025]
From one or more pixel data blocks made up of the pixel data obtained by the inverse discrete cosine transform, pixel data existing at a position overlapping the block position of the reference data block is selectively taken out to obtain one piece of pixel data. A pixel data block consisting of A DCT data block composed of DCT coefficients as correction data is obtained by performing discrete cosine transform on the pixel data block.
[0026]
Alternatively, an average value of pixel data in one or a plurality of pixel data blocks composed of pixel data in which all or a part of the block position overlaps the block position of the reference data block related to the difference data block is obtained for each block.
[0027]
The pixel data constituting one or a plurality of pixel data blocks is replaced with the block position of the reference data block from the one or a plurality of pixel data blocks in a state where the pixel data is replaced with the average value obtained as described above. Pixel data existing at the position is selectively extracted to obtain a single pixel data block made up of pixel data. A DCT data block composed of DCT coefficients as correction data is obtained by performing discrete cosine transform on the pixel data block.
[0028]
When the correction data is obtained as described above, the correction component of the difference data block is removed by adding the AC coefficient corresponding to the correction data to each AC coefficient of the difference data block. In that case, at least the class to which the pixel data of the target position in the second image signal belongs is detected using the AC coefficient of the difference data block from which the correction component has been removed.
[0029]
Thus, the first image signal composed of a plurality of pixel data generated by decoding the digital image signal subjected to motion compensation prediction coding for each block is converted into a second image signal composed of a plurality of pixel data. At the time of conversion to the image signal, the class to which the pixel data of the target position in the second image signal belongs is detected using at least the difference data block, and the pixel data of the target position in the second image signal corresponding to this class The component that corrects the block step distortion included in the reference data block is removed from the difference data block, and the difference data block from which the correction component has been removed is used for the above-described class detection. The accuracy of classification can be improved, and the quality of the second image signal can be improved.
[0030]
The coefficient data generation device according to the present invention is configured to decode a first image signal composed of a plurality of pixel data, which is generated by decoding a digital image signal subjected to motion compensation prediction coding for each block. Reduced distortion caused by encoding From multiple pixel data First Decoding means for decoding a digital image signal obtained by encoding a teacher signal corresponding to the second image signal to obtain a student signal corresponding to the first image signal, and corresponding to a target position in the teacher signal One or a plurality of block positions that overlap all or part of the block position of the reference data block related to the difference data block used when obtaining the pixel data of the student signal. Orthogonal inverse transform means for performing orthogonal inverse transform on one or a plurality of difference data blocks in which each DC coefficient is a DC coefficient and all AC coefficients are 0, and the orthogonal inverse transform From one or a plurality of pixel data blocks made up of pixel data obtained by the conversion means, pixel data existing at a position overlapping the block position of the reference data block is selectively taken out to obtain a single pixel made up of pixel data. Blocking means for obtaining a data block, orthogonal transformation means for performing orthogonal transformation on the pixel data block obtained by the blocking means to obtain a difference data block composed of transformation coefficients as the correction data, and the difference data By adding the corresponding AC coefficient of the correction data to each AC coefficient of the block, Correction component removal means for removing the correction component of the difference data block, and difference data block from which the correction component has been removed at least by the correction component removal means From the AC coefficient level distribution Class detecting means for detecting a class to which pixel data of a target position in the teacher signal belongs; and the student signal From , A data selection means for selecting a plurality of pixel data located around a target position in the teacher signal, a class detected by the class detection means, a plurality of pixel data selected by the data selection means, and the teacher signal Pixel data of the target position in To above For each class, Minimizing an error between a plurality of pixel data related to the student signal and pixel data at a target position in the teacher signal And calculation means for obtaining coefficient data Ru Is.
[0031]
Further, the coefficient data generation method according to the present invention provides a first image signal composed of a plurality of pixel data generated by decoding a digital image signal that has been subjected to motion compensation prediction coding for each block. Reduced distortion caused by encoding From multiple pixel data First A first step of decoding a digital image signal obtained by encoding a teacher signal corresponding to the second image signal to obtain a student signal corresponding to the first image signal; and a target position in the teacher signal. One or more blocks in which all or part of the block position overlaps the block position of the reference data block related to the difference data block used when obtaining the corresponding pixel data of the student signal A second step of performing orthogonal inverse transformation on one or a plurality of difference data blocks in which the DC coefficients of the difference data blocks made up of transform coefficients are DC coefficients and all AC coefficients are 0; The pixel data existing in the position overlapping the block position of the reference data block is selectively taken out from one or a plurality of pixel data blocks made up of pixel data obtained in the above step, and one pixel made up of pixel data. A third step of obtaining a data block; and a fourth step of performing orthogonal transformation on the pixel data block obtained in the third step to obtain a difference data block including transformation coefficients as the correction data; By adding the corresponding AC coefficient of the correction data to each AC coefficient of the difference data block, A first component for removing the correction component of the difference data block. 5 And at least the above-mentioned step 5 Difference data block from which correction components have been removed in step From the AC coefficient level distribution Detecting a class to which the pixel data of the target position in the teacher signal belongs. 6 Steps and student signal above From Selecting a plurality of pixel data located around the target position in the teacher signal; 7 And the above steps 6 The class detected in step 7 A plurality of pixel data selected in the above step and pixel data of the target position in the teacher signal To above For each class, Minimizing an error between a plurality of pixel data related to the student signal and pixel data at a target position in the teacher signal Find coefficient data 8 With steps Ru Is.
[0032]
A program according to the present invention is for causing a computer to execute the coefficient data generation method described above.
[0033]
In the present invention, the first image signal composed of a plurality of pixel data is generated by decoding a digital image signal that has been subjected to motion compensation prediction coding for each block. The present invention generates coefficient data of an estimation formula used when converting the first image signal into a second image signal composed of a plurality of pixel data.
[0034]
The pixel data of the student signal corresponding to the target position in the teacher signal is obtained using a predetermined difference data block and a reference data block related to the difference data block. The predetermined difference data block has a component that corrects block step distortion included in the reference data block. Correction data for removing the above-described correction component is obtained by using one or a plurality of data blocks in which all or part of the block positions overlap the block positions of the reference data block.
[0035]
The component for correcting the block step distortion included in the difference data block is removed using the correction data obtained as described above. At least the class to which the pixel data at the target position in the teacher signal belongs is detected using at least the difference data block from which the correction component has been removed.
[0036]
Further, based on the student signal, a plurality of pixel data located around the attention position in the teacher signal are selected. Then, coefficient data is obtained for each class using the class to which the pixel data at the target position in the teacher signal belongs, the selected pixel data, and the pixel data at the target position in the teacher signal.
[0037]
As described above, coefficient data of the estimation formula used when converting the first image signal to the second image signal is generated. When converting from the first image signal to the second image signal, The coefficient data corresponding to the class to which the pixel data of the target position in the second image signal belongs is selectively used to calculate the pixel data of the target position in the second image signal by the estimation formula.
[0038]
Thereby, when converting from the first image signal to the second image signal using the estimation formula, the accuracy of the class classification can be improved, and the quality of the second image signal can be improved. it can.
[0039]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows a configuration of a digital broadcast receiver 100 as an embodiment.
The digital broadcast receiver 100 includes a microcomputer, and includes a system controller 101 for controlling the operation of the entire system, and a remote control signal receiving circuit 102 for receiving a remote control signal RM. The remote control signal receiving circuit 102 is connected to the system controller 101, receives a remote control signal RM output from the remote control transmitter 200 according to a user operation, and supplies an operation signal corresponding to the signal RM to the system controller 101. Is configured to do.
[0040]
Also, the digital broadcast receiver 100 is supplied with a receiving antenna 105 and a broadcast signal (RF modulated signal) captured by the receiving antenna 105, performs channel selection processing, demodulation processing, error correction processing, etc. And a tuner unit 106 that obtains an MPEG2 stream as an encoded image signal.
[0041]
The digital broadcast receiver 100 also temporarily decodes the MPEG2 decoder 107 that decodes the MPEG2 stream output from the tuner unit 106 to obtain the image signal Va, and the image signal Va output from the MPEG2 decoder 107. A buffer memory 108 for storing the data.
[0042]
In the present embodiment, the MPEG2 decoder 107 outputs pixel position mode information pi, picture information PI, and motion compensation vector information MI in addition to the pixel data constituting the image signal Va. The pixel position mode information pi is information indicating which of the 8 × 8 pixel positions of the DCT block the output pixel data is, for example. The picture information PI is information indicating whether the output pixel data relates to an I picture (Intra-Picture), a P picture (Predictive-Picture), or a B picture (Bidirectionally predictive-Picture). The motion compensation vector information MI is information indicating how the reference data block for decoding the pixel data is motion compensated when the output pixel data relates to a P picture and a B picture. .
[0043]
The buffer memory 108 also stores information pi, PI, MI in pairs with each pixel data. Further, from the MPEG2 decoder 107, the quantization characteristic designation information QI corresponding to each DCT block extracted by the extraction circuit 75 described later and the quantized DCT coefficient of each DCT block obtained by the variable length decoding circuit 74 are also obtained. Is output. These quantization characteristic designation information QI and DCT coefficients are stored in a memory built in a residual class generation unit 121 constituting the image signal processing unit 110 described later.
[0044]
FIG. 2 shows the configuration of the MPEG2 decoder 107.
The decoder 107 has an input terminal 71 to which an MPEG2 stream is input, and a stream buffer 72 that temporarily stores the MPEG2 stream input to the input terminal 71.
[0045]
The decoder 107 extracts the DCT (Discrete Cosine Transform) coefficient as a frequency coefficient from the MPEG2 stream stored in the stream buffer 72 and the extraction circuit 73 extracts the DCT coefficient. A variable-length decoding circuit 74 that performs variable-length decoding on DCT coefficients that have been subjected to variable-length coding, for example, Huffman coding.
[0046]
The decoder 107 also includes an extraction circuit 75 that extracts quantization characteristic designation information QI from the MPEG2 stream stored in the stream buffer 72, and a variable length decoding circuit based on the quantization characteristic designation information QI. 74 has an inverse quantization circuit 76 that performs inverse quantization on the quantized DCT coefficient output from 74, and an inverse DCT circuit 77 that performs inverse DCT on the DCT coefficient output from the inverse quantization circuit 76. is doing.
[0047]
In addition, the decoder 107 stores data blocks of I and P pictures in a memory (not shown), and uses these data blocks to generate a difference (residual) between the P picture and the B picture from the inverse DCT circuit 77. When the data block is output, the prediction memory circuit 78 generates and outputs a corresponding reference data block Vref.
[0048]
In addition, when the differential data block of P picture or B picture is output from the inverse DCT circuit 77, the decoder 107 adds the reference data block Vref generated by the prediction memory circuit 78 to the differential data block. 79. When the data block of the I picture is output from the inverse DCT circuit 77, the reference data block Vref is not supplied from the prediction memory circuit 78 to the adder circuit 79. Therefore, the adder circuit 79 outputs from the inverse DCT circuit 77. The data block of the I picture is output as it is.
[0049]
Further, the decoder 107 supplies the I and P picture data blocks output from the adder circuit 79 to the prediction memory circuit 78 and stores them in the memory, and the data of each picture output from the adder circuit 79. A picture selection circuit 80 that rearranges the blocks in the correct order and outputs them as an image signal Va, and an output terminal 81 that outputs an image signal Va output from the picture selection circuit 80 are provided.
[0050]
The decoder 107 also has an extraction circuit 82 that extracts encoding control information, that is, picture information PI and motion compensation vector information MI, from the MPEG2 stream stored in the stream buffer 72. The motion compensation vector information MI extracted by the extraction circuit 82 is supplied to the prediction memory circuit 78. The prediction memory circuit 78 performs motion compensation when generating the reference data block Vref using the motion compensation vector information MI. Done. The picture information PI extracted by the extraction circuit 82 is supplied to the prediction memory circuit 78 and the picture selection circuit 80, and the prediction memory circuit 78 and the picture selection circuit 80 identify pictures based on the picture information PI.
[0051]
In addition, when the image signal Va is output from the picture selection circuit 80, in addition to the pixel data constituting the image signal Va, the pixel data is paired with each pixel data, and the pixel data is 8 × 8 of the DCT block. Pixel position mode information pi indicating where the pixel position is located, and motion compensation vector information MI and picture information PI corresponding to the pixel data are also output.
[0052]
The decoder 107 also outputs an output terminal 83 for outputting quantization characteristic designation information QI corresponding to each DCT block extracted by the extraction circuit 75, and quantization of each DCT block obtained by the variable length decoding circuit 74. And an output terminal 84 for outputting a DCT coefficient.
[0053]
The operation of the MPEG2 decoder 107 shown in FIG. 2 will be described.
The MPEG2 stream stored in the stream buffer 72 is supplied to the extraction circuit 73 to extract DCT coefficients as frequency coefficients. This DCT coefficient is variable length encoded, and this DCT coefficient is supplied to the variable length decoding circuit 74 and decoded. The quantized DCT coefficient of each DCT block output from the variable length decoding circuit 74 is supplied to the inverse quantization circuit 76 and subjected to inverse quantization.
[0054]
The inverse DCT is performed on the DCT coefficient of each DCT block output from the inverse quantization circuit 76 by the inverse DCT circuit 77 to obtain a data block of each picture. The data block of each picture is supplied to the picture selection circuit 80 via the addition circuit 79. In this case, when a difference (residual) data block of P picture or B picture is output from the inverse DCT circuit 77, the reference data block Vref output from the prediction memory circuit 78 is added by the adding circuit 79. The picture data blocks (pixel data blocks) output from the adder circuit 79 are rearranged in the correct order by the picture circuit 80 and output to the output terminal 81.
[0055]
When the image signal Va is output from the picture selection circuit 80 in this way, in addition to the pixel data constituting the image signal Va, the pixel position mode information pi, the motion is paired with each pixel data. Compensation vector information MI and picture information PI are also output.
[0056]
Further, the quantization characteristic designation information QI corresponding to each DCT block extracted by the extraction circuit 75 is output to the output terminal 83. Further, the quantized DCT coefficient of each DCT block obtained by the variable length decoding circuit 74 is output to the output terminal 84.
[0057]
Returning to FIG. 1, the digital broadcast receiver 100 also converts the image signal Va stored in the buffer memory 108 into an image signal Vb in which coding noise such as block noise (block distortion) and mosquito noise is reduced. An image signal processing unit 110 for conversion and a display unit 111 for displaying an image based on the image signal Vb output from the image signal processing unit 110 are provided. The display unit 111 includes a display such as a CRT (Cathode-Ray Tube) display or an LCD (Liquid Crystal Display).
[0058]
The operation of the digital broadcast receiver 100 shown in FIG. 1 will be described.
The MPEG2 stream output from the tuner unit 106 is supplied to the MPEG2 decoder 107 and decoded. The image signal Va output from the decoder 107 is supplied to the buffer memory 108 and temporarily stored.
[0059]
In this case, the decoder 107 outputs pixel position mode information pi, motion compensation vector information MI, and picture information PI in pairs with each pixel data of the image signal Va. These pixel position mode information pi, motion compensation vector information MI, and picture information PI are also temporarily stored in the buffer memory 108.
[0060]
In this case, the decoder 107 also outputs quantization characteristic designation information QI corresponding to each DCT block and quantized DCT coefficients of each DCT data block. The quantization characteristic designation information QI and the quantized DCT coefficient are stored in a memory built in the residual class generation unit 121 constituting the image signal processing unit 110.
[0061]
In this way, the image signal Va stored in the buffer memory 108 is supplied to the image signal processing unit 110 and converted into the image signal Vb with reduced coding noise. In the image signal processing unit 110, pixel data constituting the image signal Vb is obtained from the pixel data constituting the image signal Va. In this image signal processing unit 110, pixel position mode information pi, motion compensation vector information MI and picture information PI stored in the buffer memory 108, and also a quantum stored in a memory in the residual class generation unit 121. Conversion processing is performed using the quantization characteristic designation information QI and the quantized DCT coefficient.
[0062]
The image signal Vb output from the image signal processing unit 110 is supplied to the display unit 111, and an image based on the image signal Vb is displayed on the screen of the display unit 111.
[0063]
Next, details of the image signal processing unit 110 will be described.
The image signal processing unit 110 includes a residual class generation unit 121 that generates a residual class code CL0 indicating a residual class to which pixel data at the target position in the image signal Vb belongs. In this residual class generation unit 121, motion compensation vector information MI and picture information PI stored in the buffer memory 108 as a pair with the pixel data of the image signal Va corresponding to the target position of the image signal Vb, and built-in The residual class code CL0 is generated using the quantization characteristic designation information QI and the quantized DCT coefficient stored in the memory.
[0064]
FIG. 3 shows a specific configuration of the residual class generation unit 121.
The residual class generation unit 121 receives an input terminal 31 to which the quantization characteristic designation information QI output from the above-described MPEG2 decoder 107 is input, and a quantization characteristic designation information QI input to the input terminal 31. The memory 32 for temporarily storing, the input terminal 33 to which the quantized DCT coefficient output from the MPEG2 decoder 107 is input, and the quantized DCT coefficient input to the input terminal 33 are temporarily stored. And a memory 34 for storing the data.
[0065]
The residual class generator 121 has an input terminal 35 to which the motion compensation vector information MI and the picture information PI are input. The information MI and PI are supplied as read control information to the memories 32 and 34, and the information PI is supplied as operation control information to a class generation circuit 43 described later.
[0066]
Here, the DCT data block composed of the DCT coefficients used when obtaining the pixel data of the image signal Va corresponding to the target position in the image signal Vb is set as the target DCT data block, and the inverse DCT is applied to the target DCT data block. A reference data block to be added to the pixel data block obtained by performing the processing is defined as a reference data block of interest. In addition, all or part of the block position overlaps the block position of the target reference data block, and one or a plurality of pixel data blocks used when configuring the target reference data block are used as pixel data blocks. The DCT data block used in obtaining the used pixel data block is defined as a used DCT data block.
[0067]
When the picture information PI indicates a P picture or a B picture, the DCT data block of interest is read from the memory 34, and the DC coefficient of the used DCT data block is read. Here, since prediction encoding is performed in the forward direction in the case of a P picture, the number of used pixel data blocks is one, two, or four, and in the case of a B picture, prediction encoding is performed in both directions. The number of used pixel data blocks is one, two, or four in each direction.
[0068]
FIG. 4A shows, for example, four used DCT data blocks DB1 to DB4 that are used in obtaining four used pixel data blocks in the case of a P picture. ing. In FIG. 4A, the broken line indicates the block position of the reference data block of interest. In this case, the DC coefficients DC1 to DC4 of the four DCT data blocks DB1 to DB4 are read from the memory 34.
[0069]
Similarly, when the picture information PI indicates a P picture or a B picture, the quantization characteristic designation information QIa corresponding to the DCT data block of interest is read from the memory 32, and the quantization characteristic designation corresponding to the used DCT data block is read out. Information QIb is read.
[0070]
The residual class generation unit 121 inversely quantizes the DC coefficient of the used DCT data block read from the memory 34 based on the quantization characteristic designation information QIb corresponding to the used DCT data block read from the memory 32. Corresponding to each of the inverse quantization circuit 36 and the DCT data block to be used, the DC coefficient obtained by the inverse quantization circuit 36 is set as the DC coefficient, and the inverse of the DCT data block with all the AC coefficients set to 0 And an inverse DCT circuit 37 for performing DCT. In this case, although the number of pixel data blocks corresponding to the number of used DCT data blocks is obtained, all the 8 × 8 pixel data constituting the pixel data blocks have the same value.
[0071]
FIG. 4B shows four pixel data blocks DB1 ′ to DB4 ′ obtained corresponding to the four DCT data blocks DB1 to DB4 shown in FIG. 4A. In FIG. 4B, the broken line indicates the block position of the reference data block of interest. In this case, the pixel data blocks DB1 ′ to DB4 ′ are composed of pixel data having values of dc1 to dc4, respectively. Note that FIG. 4B shows that the pixel data blocks DB1 ′ to DB4 ′ include 4 × 4 pixel data for simplification of the drawing.
[0072]
In addition, the residual class generation unit 121 selectively extracts pixel data existing at a position overlapping the block position of the reference data block from the pixel data block obtained by the inverse DCT circuit 37, and obtains one pixel data block. The resulting blocking circuit 38 is provided.
[0073]
FIG. 4C shows one pixel data block DB0 obtained by the blocking circuit 38 when four pixel data blocks DB1 ′ to DB4 ′ are obtained as shown in FIG. 4B.
[0074]
Although not described above, when the picture information PI indicates a P picture, one pixel data block DB0 is obtained by the blocking circuit 38 as described above. However, when the picture information PI indicates a B picture, the inverse quantization circuit 36, the inverse DCT circuit 37, and the blocking circuit 38 described above process in parallel corresponding to the DCT data blocks used in both directions, One pixel data block DB0 is obtained. Then, the blocking circuit 38 finally adds and averages the corresponding pixel data of the pixel data blocks DB0 in both directions to obtain one pixel data block DB0.
[0075]
Further, the residual class generation unit 121 includes a DCT circuit 39 that performs DCT on the pixel data block DB0 obtained by the blocking circuit 38 to obtain a DCT data block. FIG. 4D shows an example of the DCT data block DB0 ′ obtained by the DCT circuit 39. C0 to C15 indicate DCT coefficients, C0 indicates a DC coefficient, and C1 to C15 indicate AC coefficients.
[0076]
Further, the residual class generation unit 121 quantizes the DCT coefficient of the DCT data block obtained by the DCT circuit 39 based on the quantization characteristic designation information QIa corresponding to the target DCT block read from the memory 32, and corrects the correction data. And a quantization circuit 40 for obtaining a DCT data block. In this case, the quantization is performed based on the quantization characteristic designation information QIa in order to match the quantization characteristic of the DCT data block as the correction data with the quantization characteristic of the target DCT data block.
[0077]
Here, the DCT data block as the correction data is obtained by applying DCT to the pixel data block obtained based on the DC coefficient of one or a plurality of used data blocks. Therefore, the AC coefficient constituting this DCT data block corresponds to the block step distortion included in the reference data block of interest.
[0078]
The reference data block of interest is obtained by performing motion compensation on the DCT data block of interest, and includes data of one or a plurality of use data blocks described above. Therefore, the reference data block of interest includes block step distortion due to one or a plurality of data blocks.
[0079]
The noted DCT data block is obtained by further applying DCT to a difference (residual) data block obtained by subtracting the noted reference data block from a predetermined data block at the time of encoding. Therefore, the DCT data block of interest has a component for correcting block step distortion included in the reference data block of interest as described above in the AC coefficient portion.
[0080]
Further, the residual class generation unit 121 adds the corresponding AC coefficient of the DCT data block as the correction data obtained by the quantization circuit 40 to each AC coefficient of the target DCT data block read from the memory 34. And an adder circuit 41 that removes a component that corrects block step distortion included in the target reference data block included in the target DCT data block.
[0081]
As described above, the AC coefficient constituting the DCT data block as the correction data corresponds to the block step distortion included in the target reference data block, and the AC coefficient of the target DCT data block is included in the target reference data block. It has a component that corrects block step distortion. Therefore, by adding the corresponding AC coefficient of the DCT data block as correction data to each AC coefficient of the target DCT data block, the block step distortion included in the target reference data block of the AC coefficient of the target DCT data block is corrected. The component to be removed is removed.
[0082]
Also, the residual class generation unit 121 uses the DCT coefficient of the target DCT data block obtained by removing the block level distortion correction component obtained by the adder circuit 41 as a quantization characteristic corresponding to the target DCT data block read from the memory 32. An inverse quantization circuit 42 that performs inverse quantization based on the designation information QIa is provided.
[0083]
The residual class generation unit 121 includes a class generation circuit 43 and an output terminal 44 that outputs the residual class code CL0 generated by the class generation circuit 43. When the picture information PI indicates an I picture, the class generation circuit 43 generates a specific code as the residual class code CL0. On the other hand, when the picture information PI indicates a P picture or a B picture, the class generation circuit 43 generates a residual class code CL0 based on each AC coefficient of the DCT data block obtained by the inverse quantization circuit 42.
[0084]
In this case, the class generation circuit 43 performs a process such as 1-bit ADRC (Adaptive Dynamic Range Coding) on each AC coefficient of the DCT data block obtained by the inverse quantization circuit 42 to obtain a residual indicating a level distribution. A class code CL0 is generated.
[0085]
ADRC calculates the maximum value and minimum value of all AC coefficients of a DCT data block, calculates the dynamic range that is the difference between the maximum value and minimum value, and quantizes each AC coefficient in accordance with the dynamic range. . In the case of 1-bit ADRC, each AC coefficient is converted into 1 bit depending on whether it is larger or smaller than the average value. The ADRC process is a process for making the number of classes representing the AC coefficient level distribution relatively small. Therefore, not only ADRC but encoding which compresses the number of data bits such as VQ (vector quantization) may be used.
[0086]
The class generation circuit 43 may generate the residual class code CL0 in a state where the P picture and the B picture are distinguished. As a result, different class classifications are performed depending on whether the picture is a P picture or a B picture, and the quality of the image signal Vb generated by the image signal processing unit 110 can be improved.
[0087]
An operation for generating the residual class code CL0 indicating the residual class to which the pixel data of the target position in the image signal Vb belongs in the residual class generating unit 121 illustrated in FIG. 3 will be described.
[0088]
When the picture information PI indicates a P picture or a B picture, the DC coefficient of one or a plurality of used DCT data blocks is read from the memory 34 and supplied to the inverse quantization circuit 36, and the used DCT data is also read from the memory 32. The quantization characteristic designation information QIb corresponding to the block is read and supplied to the inverse quantization circuit 36 (see DC1 to DC4 in FIG. 4A). The inverse quantization circuit 36 inversely quantizes the DC coefficient of the used DCT data block based on the information QIb.
[0089]
The DC coefficient inversely quantized by the inverse quantization circuit 36 is supplied to the inverse DCT circuit 37. The inverse DCT circuit 37 performs inverse DCT on a DCT data block in which the inversely quantized DC coefficient is a DC coefficient and all AC coefficients are 0 to obtain a pixel data block. In this case, the number of pixel data blocks corresponding to the number of used DCT data blocks is obtained, but all the pixel data constituting each pixel data block have the same value (see DB1 ′ to DB4 ′ in FIG. 4B). .
[0090]
The pixel data block obtained by the inverse DCT circuit 37 is supplied to the blocking circuit 38. The blocking circuit 38 selectively extracts pixel data existing at a position overlapping the block position of the reference data block from the pixel data block obtained by performing the inverse DCT, and obtains one pixel data block (see FIG. (See 4C DB0).
[0091]
One pixel data block obtained by the blocking circuit 38 is supplied to the DCT circuit 39. The DCT circuit 39 performs DCT on the one pixel data block to obtain a DCT data block (see DB0 ′ in FIG. 4D). Thus, the DCT data block obtained by the DCT circuit 39 is supplied to the quantization circuit 40. The quantization circuit 40 is further supplied with the quantization characteristic designation information QIa corresponding to the DCT data block of interest read from the memory 32. The quantization circuit 40 quantizes the DCT data block obtained by the DCT circuit 39 based on the information QIa, and obtains a DCT data block as correction data.
[0092]
The DCT data block as the correction data is obtained by applying DCT to the pixel data block obtained based on the DC coefficient of one or a plurality of used DCT data blocks. Therefore, the AC coefficient constituting the DCT data block as the correction data corresponds to the block step distortion included in the target reference data block.
[0093]
The DCT data block as correction data obtained by the quantization circuit 40 is supplied to the addition circuit 41. The DCT data block of interest is read from the memory 34 and supplied to the adder circuit 41. This attention DCT data block has a component for correcting block step distortion included in the attention reference data block in its AC coefficient portion. The adder circuit 41 adds the corresponding AC coefficient of the DCT data block as the correction data to each AC coefficient of the target DCT data block. Thereby, the component which correct | amends the block level | step distortion contained in the attention reference data block which the attention DCT data block has is removed.
[0094]
The DCT data block of interest from which the correction component of the block step distortion has been removed by the adder circuit 41 is supplied to the inverse quantization circuit 42. The inverse quantization circuit 42 is read from the memory 32 and supplied with quantization characteristic designation information QIa corresponding to the target DCT data block. The inverse quantization circuit 42 inversely quantizes the target DCT data block from which the block level distortion correction component has been removed based on the information QIa.
[0095]
Then, when the picture information PI indicates an I picture, the class generation circuit 43 generates a specific code as the residual class code CL0. On the other hand, when the picture information PI indicates a P picture or a B picture, the class generation circuit 43 generates a residual class code CL0 based on each AC coefficient of the DCT data block obtained by the inverse quantization circuit 42. The class code CL0 generated in this way by the class generation circuit 43 is output to the output terminal 44.
[0096]
As described above, the residual class generation unit 121 shown in FIG. 3 generates the residual class code CL0 based on each AC coefficient of the DCT data block of interest when the picture information PI indicates a P picture or a B picture. However, as the AC coefficients of the target DCT data block, those obtained by removing the correction component of the block step distortion included in the target reference data block are used. Therefore, the accuracy of classification of the residual class can be improved, and the quality of the image signal Vb can be improved.
[0097]
The flowchart of FIG. 5 shows the procedure of the generation process of the residual class code CLO described above.
First, processing is started in step ST1, and when the picture information PI indicates a P picture or a B picture in step ST2, DC coefficients of one or a plurality of used DCT data blocks are read from the memory. In step ST3, the quantization characteristic designation information QIb corresponding to the used DCT data block is read from the memory 32, and the DC coefficient read in step ST2 is inversely quantized based on this information QIb.
[0098]
Next, in step ST4, the inverse DCT is performed on the DCT data block in which the inversely quantized DC coefficient is a DC coefficient and all AC coefficients are 0 to obtain a pixel data block. In step ST5, pixel data existing at a position overlapping the block position of the reference data block is selectively extracted from the pixel data block obtained by performing inverse DCT to generate one pixel data block.
[0099]
Next, in step ST6, the generated one pixel data block is subjected to DCT, and the quantization characteristic designation information QIa corresponding to the target DCT data block is read from the memory 32, and the quantization is performed based on the information QIa. Then, a DCT data block as correction data is obtained. Then, in step ST7, the target DCT data block is read from the memory 34, and the corresponding AC coefficient of the DCT data block as the correction data is added to each AC coefficient to be included in the target reference data block included in the target DCT data block. The component which corrects the block step distortion is removed.
[0100]
Next, in step ST8, the target DCT data block from which the block level distortion correction component is removed is inversely quantized based on the quantization characteristic designation information QIa corresponding to the target DCT data block. In step ST9, if the picture information PI indicates an I picture, a specific code is generated as the residual class code CL0. On the other hand, if the picture information PI indicates a P picture or a B picture, step 8 The residual class code CL0 is generated based on each AC coefficient of the DCT data block obtained in step (1). Thereafter, the process ends at step ST10.
[0101]
Returning to FIG. 1, the image signal processing unit 110 selectively extracts and outputs a plurality of pixel data located around the target position in the image signal Vb from the image signal Va stored in the buffer memory 108. And a prediction tap selection circuit 122 as a data selection means. The prediction tap selection circuit 122 selectively extracts a plurality of pixel data of prediction taps used for prediction.
[0102]
Further, the image signal processing unit 110 includes a class classification unit 123 as a class detection unit that detects a class to which the pixel data of the target position in the image signal Vb belongs.
[0103]
The class classifying unit 123 pairs the pixel data of the image signal Va stored in the buffer memory 108 with the pixel data of the image signal Va corresponding to the pixel data of the target position in the image signal Vb. The class code indicating the class to which the pixel data of the target position in the image signal Vb belongs using the pixel position mode information pi stored in the above and the residual class code CL0 generated by the residual class generation unit 121. CL is generated.
[0104]
FIG. 6 shows the configuration of the class classification unit 123.
The class classification unit 123 detects n types of classes to which pixel data of the target position in the image signal Vb belongs based on the input terminal 50A for inputting the image signal Va and the image signal Va input to the input terminal 50A. Tap selection circuit 50B for selectively extracting a plurality of pixel data of class taps used for ₁ ~ 50B _n And this tap selection circuit 50B ₁ ~ 50B _n A class generation circuit 50C for generating class codes CL1 to CLn indicating n types of classes using the pixel data extracted in step S1. ₁ ~ 50C _n And have.
[0105]
In the present embodiment, class codes CL1 to CL6 indicating six types of classes are generated. The six types of classes are a spatial waveform class, a time variation class, an AC variation class, a flat class, a line correlation class, and a block edge class. A brief description of each class.
[0106]
(1) The spatial waveform class will be described. Tap selection circuit 50B ₁ And class generation circuit 50C ₁ Suppose that this spatial waveform class detection system is configured.
Tap selection circuit 50B ₁ Selectively selects a plurality of pixel data located in the periphery in the spatial direction with respect to the target position in the image signal Vb from the T frame (current frame) and the T-1 frame (frame one frame before) of the image signal Va. This is taken out and is similar to the prediction tap selection circuit 122 described above. Class generation circuit 50C ₁ The tap selection circuit 50B ₁ For example, 1-bit ADRC (Adaptive Dynamic Range Coding) or the like is applied to each of the plurality of pixel data selected in step S1 to generate a class code CL1 indicating a spatial waveform class.
[0107]
(2) The time variation class will be described. Tap selection circuit 50B ₂ And class generation circuit 50C ₂ Is configured as a detection system of this time variation class.
Tap selection circuit 50B ₂ Extracts the pixel data of the DCT block (the target block shown in FIG. 7) corresponding to the pixel data at the target position in the image signal Vb from the current frame (T frame) of the image signal Va, and at the same time before the image signal Va. The pixel data of the block (the past block shown in FIG. 7) corresponding to the block of interest is extracted from the past frame (T-1 frame).
[0108]
Class generation circuit 50C ₂ Are subtracted for each corresponding pixel between the 8 × 8 pixel data of the block of interest and the 8 × 8 pixel data of the past block to obtain 8 × 8 difference values, and this 8 × The sum of squares of the eight difference values is obtained, the square sum is determined as a threshold value, and the class code CL2 indicating the time variation class is generated.
[0109]
(3) The AC variation class will be described. Tap selection circuit 50B _Three And class generation circuit 50C _Three Constitutes a detection system of this AC fluctuation class.
Tap selection circuit 50B _Three Extracts the pixel data of the DCT block (the target block shown in FIG. 7) corresponding to the pixel data at the target position in the image signal Vb from the current frame of the image signal Va, and from the previous frame one frame before the image signal Va. Then, the pixel data of the block (the past block shown in FIG. 7) corresponding to the block of interest is extracted.
[0110]
Class generation circuit 50C _Three Performs DCT processing on each of 8 × 8 pixel data of the block of interest and 8 × 8 pixel data of the past block to obtain DCT coefficients (frequency coefficients). Then, the class generation circuit 50C _Three Is the number m of base positions where a coefficient exists in either base position of the AC part. ₁ And the number m of base positions of which the sign is inverted and one of the coefficients is 0 ₂ M ₁ / M ₂ Is determined as a threshold value, and a class code CL3 indicating an AC variation class is generated. In a block with little time variation, it is possible to perform class classification corresponding to mosquito distortion by this AC variation class.
[0111]
(4) The flat class will be described. Tap selection circuit 50B _Four And class generation circuit 50C _Four Suppose that this flat class detection system is configured.
Tap selection circuit 50B _Four Extracts the pixel data of the DCT block (the target block shown in FIG. 7) corresponding to the pixel data at the target position in the image signal Vb from the current frame of the image signal Va. Class generation circuit 50C _Four Detects the maximum value and the minimum value of 8 × 8 pixel data of the block of interest, determines the threshold of the dynamic range that is the difference, and generates the class code CL4 indicating the flat class.
[0112]
(5) The line correlation class will be described. Tap selection circuit 50B _Five And class generation circuit 50C _Five Constitutes a detection system of this line correlation class.
Tap selection circuit 50B _Five Extracts the pixel data of the DCT block (the target block shown in FIG. 7) corresponding to the pixel data at the target position in the image signal Vb from the current frame of the image signal Va.
[0113]
Class generation circuit 50C _Five Corresponds to the pixels of the 1st line, 2nd line, 3rd line, 4th line, 5th line, 6th line, 7th line and 8th line of the 8 × 8 pixel data of the target block Subtraction is performed for each pixel to obtain 8 × 4 difference values, and a square sum of the 8 × 4 difference values is obtained, and the square sum is determined as a threshold value, and a class code CL5 indicating a line correlation class is obtained. Generate. This line correlation class indicates whether the correlation in a frame such as a still image is high, or whether the correlation in the field is higher than in the frame because of fast movement.
[0114]
(6) The block edge class will be described. Tap selection circuit 50B ₆ And class generation circuit 50C ₆ Suppose that this block edge class detection system is configured.
Tap selection circuit 50B ₆ Extracts the pixel data of the DCT block (the target block shown in FIG. 7) corresponding to the pixel data of the target position in the image signal Vb from the current frame of the image signal Va, and moves up and down the target block from the current frame. Pixel data of blocks adjacent to the left and right (adjacent blocks shown in FIG. 7) are extracted.
[0115]
Class generation circuit 50C ₆ Are subtracted for each corresponding pixel between the 8 pixel data of each of the 4 sides of the block of interest and the pixel data of the adjacent block adjacent thereto to obtain 4 × 8 difference values, and each 8 A square sum of the difference values is obtained, and four square sums respectively corresponding to the four sides of the block of interest are threshold-determined to generate a class code CL6 indicating a block edge class.
[0116]
Further, the class classification unit 123 has an input terminal 50D for inputting pixel position mode information pi and an input terminal 50E for inputting a residual class code CL0. The pixel position mode information pi is the class code CLp indicating the pixel position mode class as it is. For example, when the DCT block is composed of 8 × 8 pixel data, this class code CLp is a code representing one of 64 pixel position modes.
[0117]
In addition, the class classification unit 123 includes a class generation circuit 50C. ₁ ~ 50C _n Class integration circuit 50F that integrates the class codes CL1 to CLn generated in step 1 and the class codes CLp and CL0 input to the input terminals 50D and 50E into one class code CL, and an output for outputting the class code CL Terminal 50G.
[0118]
In the present embodiment, the class integration circuit 50F includes a class generation circuit 50C. ₁ ~ 50C ₆ The class codes CL1 to CL6 generated in step 1 and the class codes CLp and CL0 are integrated into one class code CL.
[0119]
Returning to FIG. 1, the image signal processing unit 110 also includes a coefficient memory 124. The coefficient memory 124 stores coefficient data Wi (i = 1 to n, n is the number of prediction taps) used in an estimation formula used in the estimation prediction calculation circuit 125 described later for each class. .
[0120]
The coefficient data Wi is information for converting the image signal Va into the image signal Vb. The coefficient data Wi stored in the coefficient memory 124 is generated by learning between a student signal corresponding to the image signal Va and a teacher signal corresponding to the image signal Vb in advance. The coefficient memory 124 is supplied with the class code CL output from the class classification unit 123 described above as read address information, and the coefficient memory 124 reads the coefficient data Wi of the estimation formula corresponding to the class code CL. To the estimated prediction calculation circuit 125. A method for generating the coefficient data Wi will be described later.
[0121]
In addition, the image signal processing unit 110 creates the prediction tap pixel data xi selectively extracted by the prediction tap selection circuit 122 and the coefficient data Wi read from the coefficient memory 124 by using the estimation expression (1). It has an estimated prediction calculation circuit 125 for calculating pixel data y at the target position in the image signal Vb to be processed.
[0122]
[Expression 1]

[0123]
The operation of the image signal processing unit 110 will be described.
In the residual class generation unit 121, motion compensation vector information MI and picture information PI stored in the buffer memory 108 as a pair with the pixel data of the image signal Va corresponding to the target position of the image signal Vb, The residual class code CL0 is generated using the quantization characteristic designation information QI and the quantized DCT coefficient stored in the memories 32 and 34. This residual class code CL0 is a specific code when the picture information PI indicates an I picture, and when the picture information PI indicates a P picture or a B picture, the DCT of interest from which the correction component for block step distortion has been removed. It is generated based on each AC coefficient of the data block.
[0124]
The class classification unit 123 also includes a plurality of pixel data constituting the image signal Va stored in the buffer memory 108, and pixel data of the image signal Va corresponding to the pixel data of the target position in the image signal Vb in the buffer memory 108. Using the pixel position mode information pi stored in pairs and the residual class code CL0 generated by the residual class generating unit 121, a class code CL indicating the class to which the pixel data of the target position in the image signal Vb belongs. Is generated.
[0125]
Thus, the class code CL generated by the class classification unit 123 is supplied to the coefficient memory 124 as read address information. As a result, the coefficient data Wi corresponding to the class code CL is read from the coefficient memory 124 and supplied to the estimated prediction calculation circuit 125.
[0126]
Further, the prediction tap selection circuit 122 selectively extracts pixel data of the prediction tap located around the target position in the image signal Vb from the image signal Va stored in the buffer memory 108.
[0127]
The estimated prediction calculation circuit 125 uses the pixel data xi of the prediction tap and the coefficient data Wi read from the coefficient memory 124, based on the estimation expression shown in the above equation (1), in the image signal Vb to be created. Pixel data y at the target position is obtained.
[0128]
As described above, the image signal processing unit 110 obtains the image signal Vb from the image signal Va using the coefficient data Wi. In this case, a plurality of pixel data (prediction tap pixel data) located around the target position in the image signal Vb, selected based on the image signal Va, and a class to which the pixel data of the target position in the image signal Vb belong. Pixel data y at the target position in the image signal Vb is generated based on the estimation formula using the coefficient data Wi corresponding to CL.
[0129]
Therefore, the coefficient data Wi is obtained by learning using a student signal corresponding to the image signal Va and including coding noise similar to the image signal Va and a teacher signal corresponding to the image signal Vb and not including coding noise. By using the coefficient data Wi, it is possible to satisfactorily obtain an image signal Vb in which coding noise is significantly reduced as compared with the image signal Va.
[0130]
The residual class generation unit 121 generates a residual class code CL0 based on each AC coefficient of the DCT data block of interest when the picture information PI indicates a P picture or a B picture. As the AC coefficients of the data block, those obtained by removing the correction component of the block level distortion included in the reference data block of interest are used, and the accuracy of classification of the residual class can be improved. The quality of Vb can be improved.
[0131]
Next, a method for generating the coefficient data Wi stored in the coefficient memory 124 will be described. The coefficient data Wi is generated in advance by learning.
[0132]
First, this learning method will be described. In the above equation (1), before learning, coefficient data W ₁ , W ₂ , ..., W _n Is an undetermined coefficient. Learning is performed on a plurality of signal data for each class. When the number of learning data is m, the following equation (2) is set according to the equation (1). n indicates the number of prediction taps.
[0133]
y _k = W ₁ X _k1 + W ₂ X _k2 + ... + W _n X _kn ... (2)
(K = 1, 2,..., M)
[0134]
If m> n, coefficient data W ₁ , W ₂ , ..., W _n Is not uniquely determined, so the element e of the error vector e _k Is defined by the following equation (3), and e in equation (4) ² Find coefficient data that minimizes. Coefficient data is uniquely determined by a so-called least square method.
[0135]
e _k = Y _k -{W ₁ X _k1 + W ₂ X _k2 + ... + W _n X _kn } (3)
(K = 1, 2, ... m)
[0136]
[Expression 2]

[0137]
E in equation (4) ² As a practical calculation method for obtaining coefficient data that minimizes the value, first, as shown in equation (5), e ² Is partially differentiated with the coefficient data Wi (i = 1, 2,..., N), and the coefficient data Wi may be obtained so that the partial differential value becomes 0 for each value of i.
[0138]
[Equation 3]

[0139]
A specific procedure for obtaining the coefficient data Wi from the equation (5) will be described. If Xji and Yi are defined as in the equations (6) and (7), the equation (5) can be written in the form of the determinant of the equation (8).
[0140]
[Expression 4]

[0141]
[Equation 5]

[0142]
Equation (8) is generally called a normal equation. Coefficient data Wi (i = 1, 2,..., N) can be obtained by solving this normal equation by a general solution method such as a sweep-out method (Gauss-Jordan elimination method).
[0143]
FIG. 8 shows a configuration of a coefficient data generation device 150 that generates coefficient data Wi to be stored in the coefficient memory 124 of the image signal processing unit 110 of FIG.
The coefficient data generation device 150 includes an input terminal 151 to which a teacher signal ST corresponding to the image signal Vb is input, an MPEG2 encoder 152 that encodes the teacher signal ST to obtain an MPEG2 stream, An MPEG2 decoder 153 that decodes the MPEG2 stream to obtain a student signal SS corresponding to the image signal Va. Here, the MPEG2 decoder 153 corresponds to the MPEG2 decoder 107 and the buffer memory 108 in the digital broadcast receiver 100 shown in FIG.
[0144]
Further, the coefficient data generation device 150 has a residual class generation unit 154. The residual class generation unit 154 is configured in the same manner as the residual class generation unit 121 of the image signal processing unit 110 described above, and a residual class code CL0 indicating the residual class to which the pixel data at the target position in the teacher signal ST belongs. Is generated. The memory built in the residual class generation unit 154 temporarily stores the quantization characteristic designation information QI and the quantized DCT coefficient output from the MPEG2 decoder 153. In this residual class generation unit 154, motion compensation vector information MI and picture information PI paired with pixel data of the student signal SS corresponding to the target position of the teacher signal ST, and the built-in memory are stored. The residual class code CL0 is generated using the quantization characteristic designation information QI and the quantized DCT coefficient.
[0145]
Also, the coefficient data generation device 150 selectively extracts a plurality of pixel data located around the target position in the teacher signal ST from the student signal SS output from the MPEG2 decoder 153 and outputs the data. 155. The prediction tap selection circuit 155 is configured in the same manner as the prediction tap selection circuit 122 of the image signal processing unit 110 described above.
[0146]
In addition, the coefficient data generation apparatus 150 includes a class classification unit 156 as a class detection unit that detects a class to which pixel data of the target position in the teacher signal ST belongs. The class classification unit 156 is configured in the same manner as the class classification unit 123 of the image signal processing unit 110 described above.
[0147]
The class classification unit 156 includes a plurality of pixel data constituting the student signal SS obtained from the MPEG2 decoder 153 and a student signal SS corresponding to the pixel data at the target position in the teacher signal ST obtained from the MPEG2 decoder 153. Class indicating the class to which the pixel data of the target position in the teacher signal ST belongs, using the pixel position mode information pi that is paired with the pixel data and the residual class code CL0 generated by the residual class generation unit 154 A code CL is generated.
[0148]
The coefficient data generation device 150 also includes a delay circuit 157 for adjusting the time of the teacher signal ST supplied to the input terminal 151, and each position of interest obtained from the teacher signal ST adjusted in time by the delay circuit 157. The pixel data y, the pixel data xi of the prediction tap selectively extracted by the prediction tap selection circuit 155 corresponding to the pixel data y of each target position, and the class corresponding to the pixel data y of each target position From the class code CL generated by the classification unit 156, a normal equation generation unit 158 that generates a normal equation (see the above equation (8)) for obtaining coefficient data Wi (i = 1 to n) for each class. have.
[0149]
In this case, one piece of learning data is generated by a combination of one piece of pixel data y and pixel data xi of n prediction taps corresponding to the piece of pixel data y. A lot of learning data is generated every time. As a result, the normal equation generation unit 158 generates a normal equation for obtaining coefficient data Wi (i = 1 to n) for each class.
[0150]
The coefficient data generation device 150 is supplied with the data of the normal equation generated by the normal equation generation unit 158, solves the normal equation, and obtains the coefficient data Wi of each class, and the coefficient data determination unit 159. And a coefficient memory 160 for storing the obtained coefficient data Wi of each class.
[0151]
Next, the operation of the coefficient data generation device 150 shown in FIG. 8 will be described.
A teacher signal ST corresponding to the image signal Vb is supplied to the input terminal 151, and the MPEG2 encoder 152 encodes the teacher signal ST to generate an MPEG2 stream. This MPEG2 stream is supplied to the MPEG2 decoder 153. The MPEG2 decoder 153 decodes the MPEG2 stream to generate a student signal SS corresponding to the image signal Va.
[0152]
In the residual class generation unit 154, motion compensation vector information MI and picture information PI paired with the pixel data of the student signal SS corresponding to the target position of the teacher signal ST, and the quantum stored in the built-in memory. The residual class code CL0 indicating the residual class to which the pixel data at the target position in the teacher signal ST belongs is generated using the quantization characteristic designation information QI and the quantized DCT coefficient.
[0153]
In the class classification unit 156, a plurality of pixel data constituting the student signal SS obtained from the MPEG2 decoder 153 and the student signal SS corresponding to the pixel data at the target position in the teacher signal ST obtained from the MPEG2 decoder 153 A class code CL indicating a class to which the pixel data of the target position in the teacher signal ST belongs using the pixel position mode information pi paired with the pixel data and the residual class code CL0 generated by the residual class generating unit 154. Is generated.
[0154]
In addition, the prediction tap selection circuit 155 selectively extracts pixel data of prediction taps located around the target position in the teacher signal ST from the student signal SS obtained from the MPEG2 decoder 153.
[0155]
Then, the pixel data y of each target position obtained from the teacher signal ST time-adjusted by the delay circuit 157, and the prediction selectively extracted by the prediction tap selection circuit 155 corresponding to the pixel data y of each target position. Using the tap pixel data xi and the class code CL generated by the class classification unit 156 corresponding to the pixel data y of each target position, the normal equation generation unit 158 uses the coefficient data Wi ( A normal equation (see equation (8)) for obtaining i = 1 to n) is generated. This normal equation is solved by the coefficient data determination unit 159 to obtain the coefficient data Wi of each class, and the coefficient data Wi is stored in the coefficient memory 160.
[0156]
As described above, the coefficient data generation device 150 shown in FIG. 8 can generate the coefficient data Wi of each class stored in the coefficient memory 124 of the image signal processing unit 110 of FIG.
[0157]
The student signal SS is obtained by encoding the teacher signal ST to generate an MPEG2 stream and then decoding the MPEG2 stream. Therefore, the student signal SS includes the same coding noise as the image signal Va. Therefore, in the image signal processing unit 110 shown in FIG. 1, the image signal Vb obtained from the image signal Va using the coefficient data Wi has a coding noise reduced compared to the image signal Va.
[0158]
Note that the processing in the image signal processing unit 110 in FIG. 1 can be realized by software, for example, by an image signal processing apparatus 300 as shown in FIG.
[0159]
First, the image signal processing apparatus 300 shown in FIG. 9 will be described. The image signal processing apparatus 300 includes a CPU 301 that controls the operation of the entire apparatus, a ROM (Read Only Memory) 302 that stores a control program of the CPU 301, coefficient data, and the like, and a RAM (Random) that constitutes a work area of the CPU 301. Access Memory) 303. These CPU 301, ROM 302, and RAM 303 are each connected to a bus 304.
[0160]
The image signal processing apparatus 300 also includes a hard disk drive (HDD) 305 as an external storage device and a drive (FDD) 307 that drives a floppy (registered trademark) disk 306. These drives 305 and 307 are each connected to a bus 304.
[0161]
In addition, the image signal processing apparatus 300 includes a communication unit 308 that is connected to a communication network 400 such as the Internet by wire or wirelessly. The communication unit 308 is connected to the bus 304 via the interface 309.
[0162]
In addition, the image signal processing device 300 includes a user interface unit. The user interface unit includes a remote control signal receiving circuit 310 that receives a remote control signal RM from the remote control transmitter 200, and a display 311 that includes an LCD (liquid Crystal Display) or the like. The receiving circuit 310 is connected to the bus 304 via the interface 312, and similarly the display 311 is connected to the bus 304 via the interface 313.
[0163]
Further, the image signal processing apparatus 300 has an input terminal 314 for inputting the image signal Va and an output terminal 315 for outputting the image signal Vb. The input terminal 314 is connected to the bus 304 via the interface 316, and similarly, the output terminal 315 is connected to the bus 304 via the interface 317.
[0164]
Here, instead of storing the control program and coefficient data in the ROM 302 in advance as described above, for example, they are downloaded from the communication network 400 such as the Internet via the communication unit 308 and stored in the hard disk or RAM 303 for use. You can also. These control programs, coefficient data, and the like may be provided on a floppy (registered trademark) disk 306.
[0165]
Further, instead of inputting the image signal Va to be processed from the input terminal 314, it may be recorded in advance on a hard disk or downloaded from the communication network 400 such as the Internet via the communication unit 308. Further, instead of outputting the processed image signal Vb to the output terminal 315 or in parallel therewith, it is supplied to the display 311 to display an image, further stored in a hard disk, the Internet via the communication unit 308, etc. It may be sent to the communication network 400.
[0166]
A processing procedure for obtaining the image signal Vb from the image signal Va in the image signal processing apparatus 300 shown in FIG. 9 will be described with reference to the flowchart of FIG.
First, in step ST21, processing is started, and in step S22, for example, an image signal Va for one frame or one field is input into the apparatus from the input terminal 314. In this case, the DCT coefficient and quantization characteristic designation information QI corresponding to each DCT block portion of the image signal Va, pixel position mode information pi paired with the pixel data of the image signal Va, and motion compensation vector information MI And picture information PI are also input.
[0167]
Thus, the image signal Va and the like input from the input terminal 314 are temporarily stored in the RAM 303. When the image signal Va or the like is recorded in advance in the hard disk drive 305 in the apparatus, the image signal Va or the like is read from the drive 305 and the image signal Va or the like is temporarily stored in the RAM 303.
[0168]
In step ST23, it is determined whether or not the processing of all frames or all fields of the image signal Va has been completed. When the process is finished, the process ends in step ST24. On the other hand, when the process is not finished, the process proceeds to step ST25.
[0169]
In step ST25, using motion compensation vector information MI and picture information PI paired with pixel data of the image signal Va corresponding to the target position of the image signal Vb, quantization characteristic designation information QI and DCT coefficients, A residual class code CL0 is generated, and a class to which pixel data of the target position in the image signal Vb belongs is generated using the residual class code CL0, a plurality of pixel data constituting the image signal Va, and pixel position mode information pi. The class code CL shown is generated.
[0170]
Next, in step ST26, a plurality of pixel data (prediction tap pixel data) located around the target position in the image signal Vb is acquired from the image signal Va input in step ST22. Then, in step ST27, using the coefficient data Wi corresponding to the class code CL generated in step ST25 and the pixel data xi of the prediction tap acquired in step ST26, based on the estimation expression of the expression (1), Pixel data y at the target position in the image signal Vb is generated.
[0171]
Next, in step ST28, it is determined whether or not the processing for obtaining the pixel data of the image signal Vb has been completed in the entire region of the pixel data of the image signal Va for one frame or one field input in step ST22. If completed, the process returns to step ST22, and the process proceeds to input processing of the image signal Va for the next one frame or one field. On the other hand, when the process has not been completed, the process returns to step ST25 and proceeds to the process for the next target position.
[0172]
As described above, by performing processing according to the flowchart shown in FIG. 10, the pixel data of the input image signal Va can be processed to obtain the pixel data of the image signal Vb. As described above, the image signal Vb obtained by such processing is output to the output terminal 315, supplied to the display 311 to display an image, and further supplied to the hard disk drive 305 to be supplied to the hard disk. Or is recorded.
[0173]
Although illustration of the processing device is omitted, the processing in the coefficient data generation device 150 in FIG. 8 can also be realized by software.
[0174]
A processing procedure for generating coefficient data will be described with reference to the flowchart of FIG.
First, in step ST31, the process is started, and in step ST32, the teacher signal ST is input for one frame or one field. In step ST33, it is determined whether or not processing of all frames or all fields of the teacher signal ST has been completed. If not finished, in step ST34, the student signal SS is generated from the teacher signal ST input in step ST32. In this case, the DCT coefficient and quantization characteristic designation information QI corresponding to each DCT block portion of the student signal SS, pixel position mode information pi paired with the pixel data of the student signal SS, and motion compensation vector information MI And picture information PI.
[0175]
In step ST35, the motion compensation vector information MI and picture information PI, the quantization characteristic designation information QI, and the DCT coefficient paired with the pixel data of the student signal SS corresponding to the target position of the teacher signal ST are used. Thus, the residual class code CL0 is generated, and the pixel data of the target position in the teacher signal ST belongs using the residual class code CL0, the plurality of pixel data constituting the student signal SS and the pixel position mode information pi. A class code CL indicating the class is generated.
[0176]
Next, in step ST36, a plurality of pixel data (prediction tap pixel data) located around the target position in the teacher signal ST is acquired from the student signal SS generated in step ST34.
[0177]
In step ST37, (8) for each class using the class code CL generated in step ST35, the pixel data xi of the prediction tap acquired in step ST36, and the pixel data y of the target position in the teacher signal ST. Addition is performed to obtain the normal equation shown in the equation (see equations (6) and (7)).
[0178]
Next, in step ST38, it is determined whether or not the learning process has been completed in all regions of the pixel data of the teacher signal ST for one frame or one field input in step ST32. When the learning process is finished, the process returns to step ST32, the teacher signal ST for the next one frame or one field is input, and the same process as described above is repeated. On the other hand, when the learning process is not finished, the process returns to step ST35 and moves to the process for the next attention position.
[0179]
When the process is completed in step ST33 described above, in step ST39, the normal equation of each class generated by the addition process in step ST37 described above is solved by a sweeping method or the like to calculate the coefficient data Wi of each class. To do. In step ST40, the coefficient data Wi of each class is stored in the memory, and then the process ends in step ST41.
[0180]
In this way, by performing processing according to the flowchart shown in FIG. 11, coefficient data Wi of each class can be obtained by the same method as the coefficient data generation device 150 shown in FIG.
[0181]
In the above-described embodiment, the residual class generation unit 121 of the image signal processing unit 110 uses the use DCT data block. However, the use pixel data block may be used instead.
[0182]
FIG. 12 shows the configuration of the residual class generation unit 121A in that case. 12, parts corresponding to those in FIG. 3 are denoted by the same reference numerals, and detailed description thereof is omitted.
[0183]
The residual class generation unit 121A temporarily receives an input terminal 45 to which the image signal Va output from the above-described MPEG2 decoder 107 (see FIG. 2) is input, and an image signal Va input to the input terminal 45. And a memory 46 for storing data. Information MI and PI input to the input terminal 35 is supplied to the memory 46 as read control information. When the picture information PI indicates a P picture or a B picture, one or a plurality of used pixel data blocks are read from the memory 46. FIG. 13A shows, for example, a case of a P picture, in which four used pixel data blocks DBa to DBd are used. In FIG. 13A, the broken line indicates the block position of the reference data block of interest. FIG. 13A shows that the pixel data blocks DBa to DBd include 4 × 4 pixel data for simplification of the drawing.
[0184]
Further, the residual class generation unit 121A includes an average circuit 47 that obtains an average value of pixel data in the pixel data block read from the memory 46 for each block. In this case, an average value corresponding to the number of used pixel data blocks is obtained. As shown in FIG. 13A, when the number of used pixel data blocks is four, four average values are obtained.
[0185]
In addition, the residual class generation unit 121A is configured to use the one or more usages in a state where the pixel data constituting the one or more used pixel data blocks is replaced with the average value obtained by the averaging circuit 47. A block circuit 48 is provided which selectively extracts pixel data existing at a position overlapping the block position of the reference data block from the pixel data block to obtain one pixel data block.
[0186]
For example, when the pixel data constituting the four used pixel data blocks DBa to DBd as shown in FIG. 13A are replaced with average values, the result is as shown in FIG. 13B. In this case, da is a ₀ ~ A ₁₅ Average value of db, b is b ₀ ~ B ₁₅ The average value of dc is c ₀ ~ C ₁₅ The average value of dd is d ₀ ~ D ₁₅ Is the average value. Further, one pixel data block DB0 obtained by selectively extracting pixel data existing at a position overlapping the block position of the reference data block is as shown in FIG. 13C.
[0187]
The rest of the residual class generator 121A shown in FIG. 12 is configured in the same manner as the residual class generator 121 shown in FIG.
An operation for generating the residual class code CL0 indicating the residual class to which the pixel data of the target position in the image signal Vb belongs in the residual class generating unit 121 illustrated in FIG. 12 will be described.
[0188]
When the picture information PI indicates a P picture or a B picture, the DCT coefficients of one or a plurality of used pixel data blocks are read from the memory 46 and supplied to the averaging circuit 47 (see DBa to DBd in FIG. 13A). The average circuit 47 obtains the average value of the pixel data in the pixel data block read from the memory 46 for each block.
[0189]
The average value obtained by the averaging circuit 47 is supplied to the blocking circuit 48. The block forming circuit 48 replaces the pixel data constituting the one or more used pixel data blocks with the average value obtained by the averaging circuit 47 (see FIG. 13B). From the used pixel data block, pixel data existing at a position overlapping the block position (shown by a broken line) of the reference data block is selectively extracted to obtain one pixel data block (see DB0 in FIG. 13C).
[0190]
One pixel data block obtained by the blocking circuit 38 is supplied to the DCT circuit 39. The DCT circuit 39 performs DCT on the one pixel data block to obtain a DCT data block (see DB0 ′ in FIG. 4D). Thus, the DCT data block obtained by the DCT circuit 39 is supplied to the quantization circuit 40. The quantization circuit 40 is further supplied with the quantization characteristic designation information QIa corresponding to the DCT data block of interest read from the memory 32. The quantization circuit 40 quantizes the DCT data block obtained by the DCT circuit 39 based on the information QIa, and obtains a DCT data block as correction data.
[0191]
This DCT data block as correction data is obtained by applying DCT to a pixel data block obtained based on an average value of pixel data constituting one or a plurality of used pixel data blocks. Therefore, the AC coefficient constituting the DCT data block as the correction data also corresponds to the block step distortion included in the reference data block of interest as in the residual class generation unit 121 shown in FIG.
[0192]
The DCT data block as correction data obtained by the quantization circuit 40 is supplied to the addition circuit 41. The DCT data block of interest is read from the memory 34 and supplied to the adder circuit 41. This attention DCT data block has a component for correcting block step distortion included in the attention reference data block in its AC coefficient portion. The adder circuit 41 adds the corresponding AC coefficient of the DCT data block as the correction data to each AC coefficient of the target DCT data block. Thereby, the component which correct | amends the block level | step distortion contained in the attention reference data block which the attention DCT data block has is removed.
[0193]
The DCT data block of interest from which the correction component of the block step distortion has been removed by the adder circuit 41 is supplied to the inverse quantization circuit 42. The inverse quantization circuit 42 is read from the memory 32 and supplied with quantization characteristic designation information QIa corresponding to the target DCT data block. The inverse quantization circuit 42 inversely quantizes the target DCT data block from which the block level distortion correction component has been removed based on the information QIa.
[0194]
Then, when the picture information PI indicates an I picture, the class generation circuit 43 generates a specific code as the residual class code CL0. On the other hand, when the picture information PI indicates a P picture or a B picture, the class generation circuit 43 generates a residual class code CL0 based on each AC coefficient of the DCT data block obtained by the inverse quantization circuit 42. The class code CL0 generated in this way by the class generation circuit 43 is output to the output terminal 44.
[0195]
As described above, the residual class generation unit 121A illustrated in FIG. 12 generates the residual class code CL0 based on each AC coefficient of the DCT data block of interest when the picture information PI indicates a P picture or a B picture. However, as the AC coefficients of the target DCT data block, those obtained by removing the correction component of the block step distortion included in the target reference data block are used. Therefore, the accuracy of classification of the residual class can be improved, and the quality of the image signal Vb can be improved.
[0196]
The flowchart of FIG. 14 shows the procedure of the generation process of the residual class code CLO described above. In FIG. 14, parts corresponding to those in FIG.
[0197]
First, processing is started in step ST1, and in step ST11, when the picture information PI indicates a P picture or a B picture, one or a plurality of used pixel data blocks are read from the memory 46. In step ST12, an average value of pixel data of one or a plurality of used pixel data blocks read out is obtained for each block.
[0198]
Next, in step ST13, the pixel data constituting one or a plurality of used pixel data blocks is replaced with an average value, and the block data of the reference data block is moved from the one or a plurality of used pixel data blocks. The pixel data existing at the overlapping position is selectively extracted to generate one pixel data block.
[0199]
Next, the process proceeds to step ST6. Since step ST6 and subsequent steps are the same as those in the flowchart of FIG. 5 described above, description thereof will be omitted.
[0200]
In the above-described embodiment, the quantized DCT coefficient obtained by the variable length decoding circuit 74 is output from the MPEG2 decoder 107, and the quantized DCT coefficient is output to the memory 34 of the residual class generating units 121 and 121A. This is stored and used for generating the residual class code CL0.
[0201]
However, the inverse quantized DCT coefficient obtained by the inverse quantization circuit 76 is output from the MPEG2 decoder 107, and this inverse quantized DCT coefficient is output to the memory 34 of the residual class generating units 121 and 121A. It can also be configured to be stored in and used to generate the residual class code CL0.
[0202]
In this case, the residual class generation units 121 and 121A do not require processing such as inverse quantization and quantization. Therefore, in that case, the memory 32, the inverse quantization circuits 36 and 42, and the quantization circuit 40 are unnecessary in the residual class generation unit 121 of FIG. Similarly, the memory 32, the quantization circuit 40, and the inverse quantization circuit 42 are unnecessary in the residual class generation unit 121A of FIG.
[0203]
In the above embodiment, the MPEG2 stream with DCT is handled. However, the present invention deals with other encoded digital image signals subjected to motion compensation prediction coding for each block. It can be similarly applied to. Also, instead of DCT, Wavelet It may be an encoding with other orthogonal transformation such as transformation or discrete sine transformation.
[0204]
【The invention's effect】
According to the present invention, the first image signal composed of a plurality of pixel data, which is generated by decoding the digital image signal subjected to motion compensation prediction coding for each block, is composed of a plurality of pixel data. When converting to the second image signal, the class to which the pixel data of the target position in the second image signal belongs is detected using at least the difference data block, and the target position of the second image signal corresponding to this class is detected. A component that generates pixel data and removes a component that corrects block step distortion included in a reference data block from the difference data block and uses the difference data block from which the correction component has been removed for the above-described class detection. In addition, since the accuracy of classification is improved, the quality of the second image signal can be improved.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a digital broadcast receiver as an embodiment.
FIG. 2 is a block diagram showing a configuration of an MPEG2 decoder.
FIG. 3 is a block diagram illustrating a configuration of a residual class generation unit.
FIG. 4 is a diagram for explaining processing of a residual class generation unit;
FIG. 5 is a flowchart showing a residual class generation process.
FIG. 6 is a block diagram illustrating a configuration of a class classification unit.
FIG. 7 is a diagram illustrating a tap selection block.
FIG. 8 is a block diagram illustrating a configuration of a coefficient data generation device.
FIG. 9 is a block diagram illustrating a configuration example of an image signal processing device to be realized by software.
FIG. 10 is a flowchart showing image signal processing.
FIG. 11 is a flowchart showing coefficient data generation processing.
FIG. 12 is a block diagram illustrating another configuration of a residual class generation unit.
FIG. 13 is a diagram for explaining processing of a residual class generation unit;
FIG. 14 is a flowchart showing a residual class generation process.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 100 ... Digital broadcast receiver, 101 ... System controller, 102 ... Remote control signal receiving circuit, 105 ... Receiving antenna, 106 ... Tuner part, 107 ... MPEG2 decoder, 108. ..Buffer memory, 110... Image signal processing unit, 111... Display unit, 121, 121 A... Residual class generation unit, 122. 124 ... Coefficient memory, 125 ... Estimated prediction calculation circuit, 150 ... Coefficient data generation device, 300 ... Image signal processing device

Claims (8)

A plurality of pixels in which a first image signal composed of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensation prediction coding for each block is reduced in distortion due to the coding. when converting the second image signal that Do from the data, the reference according to the difference data block used in obtaining the pixel data of the above corresponding to the target position definitive to the second image signal a first image signal The DC coefficient of the difference data block consisting of one or a plurality of transform coefficients whose block positions overlap all or part of the block position of the data block is a DC coefficient and one or a plurality of DC coefficients where all AC coefficients are 0 Orthogonal inverse transform means for performing orthogonal inverse transform on the difference data block;
From one or a plurality of pixel data blocks composed of pixel data obtained by the orthogonal inverse transform means, pixel data existing at a position overlapping the block position of the reference data block is selectively extracted to obtain a piece of pixel data. Blocking means for obtaining a pixel data block comprising:
Orthogonal transformation means for performing orthogonal transformation on the pixel data block obtained by the blocking means to obtain a difference data block comprising transformation coefficients as the correction data;
Correction component removal means for removing the correction component of the difference data block by adding the AC coefficient corresponding to the correction data to each AC coefficient of the difference data block ;
Class detection means for detecting a class to which pixel data at the position of interest in the second image signal belongs from at least the AC coefficient level distribution of the difference data block from which the correction component has been removed ;
A teacher signal that is obtained in advance for each class detected by the class detecting means and includes a student signal including encoding distortion corresponding to the first image signal and not including encoding distortion corresponding to the second image signal. Coefficient data generating means for generating coefficient data that minimizes an error from the signal;
Data selecting means for selecting, from the first image signal, a plurality of pixel data located around the position of interest in the second image signal;
Images that calculates a plurality of pixel data selected by the generated coefficient data and said data selecting means in said coefficient data generation means Ru and an arithmetic means for obtaining pixel data of the target position in the second image signal Signal processing device. The digital image signal has been subjected to MPEG encoding, and the differential data block is a DCT data block composed of DCT coefficients,
The inverse DCT means as the orthogonal inverse transform means is
The DC coefficients of one or more DCT data blocks composed of DCT coefficients, each of which overlaps the block position of the reference data block related to the difference data block, are all DC coefficients, and all AC coefficients are and facilities inverse discrete cosine transform on 0 and the one or a plurality of DCT data blocks,
The blocking means is
From one or a plurality of pixel data blocks made up of pixel data obtained by the inverse DCT means, pixel data existing at a position overlapping with the block position of the reference data block is selectively taken out from one piece of pixel data. and it acquired the composed pixel data blocks,
The DCT means as the orthogonal transform means is:
By performing discrete cosine transform on the pixel data blocks obtained by the blocking means, and acquired the DCT data block of DCT coefficients as the correction data,
The correction component removing means includes
By adding the corresponding AC coefficients of the correction data to each AC coefficient of the DCT data block, remove the correction component which the DCT data block has,
The class detection means is
At least, the a level distribution of correction AC coefficient components DCT data block that has been removed, the image signal processing apparatus according to 請 Motomeko 1 detect a class to which pixel data belongs of the target position in the second image signal . The digital image signal has been subjected to MPEG encoding, and the differential data block is a DCT data block composed of DCT coefficients,
Mean value acquisition means as the orthogonal inverse transform means,
The whole or part of the block located in the blocking position of the reference data block according to the difference data block is one or more overlapping, and acquired the average value of the pixel data for each block in the pixel data block of pixel data,
The blocking means is
In a state where the pixel data constituting the one or more pixel data blocks is replaced with the average value obtained by the average value acquisition means, the block of the reference data block from the one or more pixel data blocks and the pixel data at the position overlapping the regioselectively extraction was acquired the one of the pixel data block of pixel data,
The DCT means as the orthogonal transform means is:
By performing discrete cosine transform on the pixel data blocks obtained by the blocking means, and acquired the DCT data block of DCT coefficients as the correction data,
The correction component removing means includes
By adding the corresponding AC coefficients of the correction data to each AC coefficient of the DCT data block, remove the correction component which the DCT data block has,
The class detection means is
At least, the a level distribution of correction AC coefficient components DCT data block that has been removed, the image signal processing apparatus according to 請 Motomeko 1 detect a class to which pixel data belongs of the target position in the second image signal . A plurality of pixels in which a first image signal composed of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensation prediction coding for each block is reduced in distortion due to the coding. when converting the second image signal consisting of data, reference data blocks of the differential data blocks used in obtaining the pixel data of the above which corresponds to the target position in the second image signal a first image signal One or a plurality of difference data in which a DC coefficient of a difference data block made up of one or a plurality of transform coefficients whose whole or a part of the block position overlaps with a block position is a DC coefficient and all AC coefficients are 0 A first step of performing an orthogonal inverse transform on the block;
From the one or a plurality of pixel data blocks made up of pixel data obtained in the first step, the pixel data existing at a position overlapping the block position of the reference data block is selectively extracted to obtain one piece of pixel data. A second step of obtaining a pixel data block comprising:
A third step of performing orthogonal transformation on the pixel data block obtained in the second step to obtain a difference data block composed of transformation coefficients as the correction data;
A fourth step of removing the correction component of the difference data block by adding the AC coefficient corresponding to the correction data to each AC coefficient of the difference data block;
A fifth step of detecting a class to which the pixel data of the target position in the second image signal belongs from the level distribution of the AC coefficient of the difference data block from which the correction component has been removed at least in the fourth step;
A student signal that is obtained in advance for each class detected in the fifth step and includes coding distortion corresponding to the first image signal and does not include coding distortion corresponding to the second image signal. A sixth step of generating coefficient data that minimizes an error from the teacher signal;
A seventh step of selecting, from the first image signal, a plurality of pixel data located around a target position in the second image signal;
And an eighth step of obtaining the pixel data of the target position in the second image signal by calculating the coefficient data generated in the sixth step and the plurality of pixel data selected in the seventh step. images signal processing method that. A plurality of pixels in which a first image signal composed of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensation prediction coding for each block is reduced in distortion due to the coding. when converting the second image signal consisting of data, reference data blocks of the differential data blocks used in obtaining the pixel data of the above which corresponds to the target position in the second image signal a first image signal One or a plurality of difference data in which a DC coefficient of a difference data block made up of one or a plurality of transform coefficients whose whole or a part of the block position overlaps with a block position is a DC coefficient and all AC coefficients are 0 A first step of performing an orthogonal inverse transform on the block;
From the one or a plurality of pixel data blocks made up of pixel data obtained in the first step, the pixel data existing at a position overlapping the block position of the reference data block is selectively extracted to obtain one piece of pixel data. A second step of obtaining a pixel data block comprising:
A third step of performing orthogonal transformation on the pixel data block obtained in the second step to obtain a difference data block composed of transformation coefficients as the correction data;
A fourth step of removing the correction component of the difference data block by adding the AC coefficient corresponding to the correction data to each AC coefficient of the difference data block;
A fifth step of detecting a class to which the pixel data of the target position in the second image signal belongs from the level distribution of the AC coefficient of the difference data block from which the correction component has been removed at least in the fourth step;
A student signal that is obtained in advance for each class detected in the fifth step and includes coding distortion corresponding to the first image signal and does not include coding distortion corresponding to the second image signal. A sixth step of generating coefficient data that minimizes an error from the teacher signal;
A seventh step of selecting, from the first image signal, a plurality of pixel data located around a target position in the second image signal;
And an eighth step of obtaining the pixel data of the target position in the second image signal by calculating the coefficient data generated in the sixth step and the plurality of pixel data selected in the seventh step. program for executing a that images signal processing method on a computer. A plurality of pixels in which a first image signal composed of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensation prediction coding for each block is reduced in distortion due to the coding. decoding means for obtaining a student signal that the teacher signal corresponding to decodes the digital image signal obtained by coding the first image signal corresponding to Do that the second image signals from the data,
One or a plurality of transform coefficients in which all or part of the block position overlaps the block position of the reference data block related to the difference data block used when obtaining the pixel data of the student signal corresponding to the target position in the teacher signal Orthogonal inverse transform means for performing orthogonal inverse transform on one or a plurality of difference data blocks in which each of the DC coefficients of the difference data block is a DC coefficient and all AC coefficients are 0;
From one or a plurality of pixel data blocks composed of pixel data obtained by the orthogonal inverse transform means, pixel data existing at a position overlapping the block position of the reference data block is selectively extracted to obtain a piece of pixel data. Blocking means for obtaining a pixel data block comprising:
Orthogonal transformation means for performing orthogonal transformation on the pixel data block obtained by the blocking means to obtain a difference data block comprising transformation coefficients as the correction data;
By adding the corresponding AC coefficients of the correction data to each AC coefficient of the differential data blocks, and the correction component removing means for removing a correction component that the differential data blocks have,
Class detection means for detecting a class to which pixel data of a target position in the teacher signal belongs from at least the AC coefficient level distribution of the difference data block from which the correction component has been removed by the correction component removal means;
Data selection means for selecting a plurality of pixel data located around the target position in the teacher signal from the student signal;
The detected class by the class detection means, the pixel data for each of the classes of the target position in the plurality of pixel data and the teacher signal selected by the data selection means, a plurality of pixel data and the teacher according to the student signal engaging the number of data generating device Ru and an arithmetic means for obtaining the coefficient data to the minimum error between the pixel data of the target position in the signal. A plurality of pixels in which a first image signal composed of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensation prediction coding for each block is reduced in distortion due to the coding. a first step of obtaining a student signal that the teacher signal corresponding to the encoded said first decodes the digital image signal obtained by the image signals corresponding to Do that the second image signals from the data,
One or a plurality of transform coefficients in which all or part of the block position overlaps the block position of the reference data block related to the difference data block used when obtaining the pixel data of the student signal corresponding to the target position in the teacher signal A second step of performing orthogonal inverse transform on one or a plurality of difference data blocks in which each DC coefficient of the difference data block is a DC coefficient and all AC coefficients are 0;
From the one or a plurality of pixel data blocks made up of pixel data obtained in the second step, the pixel data existing at the position overlapping the block position of the reference data block is selectively extracted to obtain one piece of pixel data. A third step of obtaining a pixel data block comprising:
A fourth step of performing orthogonal transformation on the pixel data block obtained in the third step to obtain a difference data block including transformation coefficients as the correction data;
A fifth step of removing the correction component of the difference data block by adding the AC coefficient corresponding to the correction data to each AC coefficient of the difference data block;
A sixth step of detecting a class to which the pixel data of the target position in the teacher signal belongs from the level distribution of the AC coefficient of the difference data block from which the correction component is removed at least in the fifth step;
A seventh step of selecting, from the student signal, a plurality of pixel data located around a target position in the teacher signal;
The sixth class detected in step, from the pixel data for each of the classes of the target position in the seventh plurality of pixel data and the teacher signal selected in step, and a plurality of pixel data in accordance with the student signal eighth step and engaging number data generation process of Ru with a seeking coefficient data that minimizes the error between the pixel data of the target position in the teacher signal. A plurality of pixels in which a first image signal composed of a plurality of pixel data generated by decoding a digital image signal subjected to motion compensation prediction coding for each block is reduced in distortion due to the coding. a first step of obtaining a student signal that the teacher signal corresponding to the encoded said first decodes the digital image signal obtained by the image signals corresponding to Do that the second image signals from the data,
One or a plurality of transform coefficients in which all or part of the block position overlaps the block position of the reference data block related to the difference data block used when obtaining the pixel data of the student signal corresponding to the target position in the teacher signal A second step of performing orthogonal inverse transform on one or a plurality of difference data blocks in which each DC coefficient of the difference data block is a DC coefficient and all AC coefficients are 0;
From the one or a plurality of pixel data blocks made up of pixel data obtained in the second step, the pixel data existing at the position overlapping the block position of the reference data block is selectively extracted to obtain one piece of pixel data. A third step of obtaining a pixel data block comprising:
A fourth step of performing orthogonal transformation on the pixel data block obtained in the third step to obtain a difference data block including transformation coefficients as the correction data;
A fifth step of removing the correction component of the difference data block by adding the AC coefficient corresponding to the correction data to each AC coefficient of the difference data block;
A sixth step of detecting a class to which the pixel data of the target position in the teacher signal belongs from the level distribution of the AC coefficient of the difference data block from which the correction component is removed at least in the fifth step;
A seventh step of selecting, from the student signal, a plurality of pixel data located around a target position in the teacher signal;
The sixth class detected in step, from the pixel data for each of the classes of the target position in the seventh plurality of pixel data and the teacher signal selected in step, and a plurality of pixel data in accordance with the student signal program to execute the eighth step and the engaging speed data generating method Ru comprising the obtaining coefficient data that minimizes the error between the pixel data of the target position in the teacher signal to the computer.

Images