JP3627258B2 - High-efficiency encoding and decoding apparatus for digital image signals - Google Patents

Description

【００１９】
係数ｗとして、実際の値との誤差を最小にするものを最小二乗法により求める。このために、観測方程式の右辺に残差行列Ｅを加えた下記の残差方程式を作成する。すなわち、最小二乗法において、残差方程式における残差行列Ｅの要素の二乗、すなわち二乗誤差が最小になる係数行列Ｗを求める。
【００２０】
【数２】

【００２１】
次に、残差方程式（３）から係数行列Ｗの各要素ｗ_ｉ の最確値を見いだすための条件は、ブロック内の画素に対応するｍ個の残差をそれぞれ二乗してその総和を最小にする条件を満足させればよい。この条件は、下記の式（４）により表される。
【００２２】
【数３】

【００２３】
ｎ個の条件を入れてこれを満足する係数行列Ｗの要素である未定係数ｗ_１ ，ｗ_２ ，・・・，ｗ_ｎ を見出せばよい。従って、残差方程式（３）より、
【００２４】
【数４】

【００２５】
となる。式（４）の条件をｉ＝１，２，・・・，ｎ）について立てれば、それぞれ
【００２６】
【数５】

【００２７】
が得られる。式（３）と式（６）から、下記の正規方程式が得られる。
【００２８】
【数６】

【００２９】
正規方程式（７）は、丁度、未知数の数がｎ個だけある連立方程式である。これにより、最確値たる各未定係数ｗ_ｉ を求めることができる。正確には、式（７）における、ｗ_ｉ にかかるマトリクスが正則であれば、解くことができる。実際には、Ｇａｕｓｓ−Ｊｏｒｄａｎの消去法（別名、掃き出し法）を用いて未定係数ｗ_ｉ を求めている。このようにして、非伝送画素の補間のための係数が１フレームでクラス毎に１組確定し、この係数が伝送される。
【００３０】
上述のように係数を決定する時に、補間の対象である注目画素を含む部分的画像の特徴を反映したクラス分けがなされる。このクラス分けとしては、参照される画素の値を使用することが考えられる。しかしながら、各画素の値が８ビットの時に、８個の参照画素の場合で、クラス数が２^６４となり、クラス数が非常に多くなる。この問題を解決するために、参照画素のビット数を圧縮符号化により減少させる。クラス分けの他の方法は、注目画素が含まれる小領域の相関が強い方向を検出し、その方向に応じたクラスを規定しても良い。
【００３１】
図２は、この発明の一実施例の全体的なブロック図である。入力端子１からのディジタル画像データが画素分離回路２に供給され、図１に示すようなパターンに従って、入力画素が第１および第２の画素へ分離される。第１の画素データがビット数をより少ないビット数へ変換するためのエンコーダ３ａに供給される。エンコーダ３ａにおいて、各画素のビット数が８ビットからＭ（＜８）ビットへ変換される。第２の画素データがエンコーダ３ｂに供給される。エンコーダ３ｂにおいて、各画素のビット数がＮ（＜Ｍ）ビットへ変換される。
【００３２】
エンコーダ３ａ、３ｂの一例は、本願出願人の提案にかかるダイナミックレンジに適応した符号化（ＡＤＲＣ）のエンコーダである。ＡＤＲＣは、ブロック毎に画素の最大値および最小値を検出し、その差であるダイナミックレンジを求め、ダイナミックレンジに適応した量子化ステップで、最小値または最大値を除去した後の画素データを量子化するものである（特開昭６１−１４４９８９号公報参照）。画像の局所的相関からブロック内の画素データを元の量子化ビット数（例えば８ビット）より少ないビット数Ｍ（例えば４ビット）で量子化しても、画像の劣化を抑えることができる。
【００３３】
エンコーダ３ａ、３ｂでなされる符号化のより簡単な例は、各８ビットの最下位ビットまたは最下位ビットからの数ビットを除去するものである。エンコーダ３ａ、３ｂの出力が出力端子５ａ、５ｂに伝送コードとして、それぞれ取り出され、また、ローカルデコーダ４ａ、４ｂに供給される。ローカルデコーダ４ａ、４ｂによって、８ビットに復号されたデータが形成される。
【００３４】
エンコーダ３ａ、３ｂからの符号化データがクラス分類回路６に供給される。クラス分類回路６は、上述のように、注目画素の周辺の参照画素を使用して、注目画素を含む小領域の特徴（レベル分布、相関の強い方向等）に対応するクラスを指示する例えば８ビットのインデックスを発生する。参照画素として、第１および第２の画素を使用しているので、エンコーダ３ａ、３ｂの出力がクラス分類回路６に供給される。
【００３５】
クラス分けのための参照画素は、エンコーダ３ａ、３ｂの出力としているが、これに限らず、ローカルデコーダ４ａ、４ｂの出力データを使用しても良い。クラス分類回路６からのインデックスと、注目画素の実データと、ローカルデコーダ４ａ、４ｂからの復号データが最小二乗法の演算回路７ａに供給される。同様に、最小二乗法の演算回路７ｂに対して、インデックスと、注目画素の実データと、ローカルデコーダ４ａ、４ｂからの復号データが供給される。
【００３６】
最小二乗法の演算回路７ａ、７ｂのそれぞれは、ｘ_ｉ として復号データを用い、また、注目画素データの実際の値を用い、上述の最小二乗法のアルゴリズムによって、例えば１フレームで１組の係数ｗ_ｉ を決定する。演算回路７ａ、７ｂからの係数が出力端子８ａ、８ｂに取り出される。伝送画素の符号化出力（コード）と係数とが図示しないが、フレーム化回路、チャンネル変調回路等を介して伝送路へ送出される。伝送路は、通信回線、磁気記録／再生プロセス等である。
【００３７】
送信されるデータは、１フレーム内のビット数がＭビット、Ｎビットへ少なくされたコードと２組の係数である。この係数の情報量は、１フレームあたりのコードの情報量に比べて無視しうるものである。従って、各画素のビット数を少なくすることによって、伝送データ量を減少できる。
【００３８】
図３は、図２の符号化回路と対応する復号回路を示す。受信データは、図示しないが、チャンネル復調、フレーム分解等の処理を受け、２１ａ、２１ｂで示す入力端子にコードが供給され、２２ａ、２２ｂで示す入力端子に係数が供給される。Ｍビットのコードが圧縮符号化のデコーダ２３ａに供給され、８ビットのデータに復号される。Ｎビットのコードが圧縮符号化のデコーダ２３ｂに供給され、８ビットのデータに復号される。
【００３９】
入力端子２１ａ、２１ｂからのコードは、クラス分類回路２６にも供給される。クラス分類回路２６は、符号化回路中のクラス分類回路６と同一のクラス分けを行い、インデックスを発生する。若し、符号化の際に、復号データを使用してクラス分けがなされている時には、デコーダ２３ａ、２３ｂの出力データを使用してクラス分けがなされる。このインデックスが推定値生成回路２４ａ、２４ｂに供給される。推定値生成回路２４ａおよび２４ｂのそれぞれに対して、デコーダ２３ａおよび２３ｂの出力、入力端子２２ａおよび２２ｂからの係数も供給される。
【００４０】
推定値生成回路２４ａ、２４ｂは、復号データと係数との線形１次結合によって、第１および第２の画素に関して８ビットの値（推定値）を生成する。推定値生成回路２４ａ、２４ｂのそれぞれの出力が画素合成回路２５に供給される。画素合成回路２５は、第１および第２の画素を元の位置関係で合成し、出力端子２７に復号画像データが得られる。デコーダ２３ａ、２３ｂによって、８ビットの復号データが得られているが、係数と周辺の復号データの線形１次結合によって作成された推定値は、復号データに比してより誤差が少ないものである。
【００４１】
なお、図２および図３の構成では、入力画像データを使用してリアルタイムで係数を決定しているが、予め学習によって係数を決定することもできる。その場合には、異なる絵柄の画像を使用して、汎用性のある係数が決定され、これが固定係数としてメモリに格納される。注目画素の補間のための復号回路にこのメモリが設けられ、メモリ中の固定係数が使用される。さらに、学習で決定された固定係数をメモリに格納し、このメモリの係数を実際に伝送する入力画像データから決定された係数で更新する構成も可能である。
【００４２】
クラス分類回路６の一例を図４に示す。入力端子４１ａ、４１ｂから選択回路４２にエンコーダ３ａ、３ｂのそれぞれの出力が供給される。選択回路４２は、クラス分類に使用する第１および第２の画素の符号化データを選択し、選択された符号化データが１ビットＡＤＲＣ回路４３ａおよび４３ｂにそれぞれ供給される。
【００４３】
１ビットＡＤＲＣ回路４３ａ、４３ｂは、適当な大きさのブロックの最大値および最小値の差（ダイナミックレンジＤＲ）を検出し、各データをダイナミックレンジＤＲで割算し、その商を０．５と比較し、これが０．５より小のときには｀０’ 、これが０．５以上のときには、｀１’ の出力を発生する。すなわち、入力データが１ビットの出力に変換される。
【００４４】
１ビットＡＤＲＣ回路４３ａ、４３ｂのそれぞれの出力がシフトレジスタ４４ａ、４４ｂにおいて直列並列変換される。シフトレジスタ４４ａ、４４ｂからの合計８ビットがレジスタ４５に取り込まれ、レジスタ４５の出力に８ビットのインデックスが発生する。８ビットのインデックスにより２^８ の数のクラスが指示される。
【００４５】
なお、クラス分けのために、空間的に注目画素の近傍の画素のみでなく、前フレームおよび後フレームの空間的に同一位置の画素をも参照するようにしても良い。また、クラス分けのための圧縮符号化としては、１ビットＡＤＲＣ以外に、ベクトル量子化、ＤＰＣＭ等を使用することができる。
【００４６】
次に、図５を参照して最小二乗法の演算回路７ａについて説明する。演算回路７ｂは、演算回路７ａと同一の構成である。図５に示すように、エンコーダ３ａの出力信号が供給され、時空間モデルを構成するデータ、すなわち、注目画素の実データｙと線形１次結合に使用するデータｘ_ｉ を同時化するための時系列変換回路３１が設けられている。時系列変換回路３１からのデータが乗算器アレー３２に供給される。乗算器アレー３２に対して加算メモリ３３が接続される。インデックスがデコーダ３５に供給され、デコーダ３５からのクラス情報が加算メモリ３３に供給される。これらの乗算器アレー３２および加算メモリ３３は、正規方程式生成回路を構成する。
【００４７】
乗算器アレー３２は、各画素同士の乗算を行ない、加算メモリ３３は、乗算器アレー３２からの乗算結果が供給される加算器アレーとメモリアレーとで構成される。図６は、乗算器アレー３２の具体的構成である。図６において、その一つを拡大して示すように、四角のセルが乗算器を表す。乗算器アレー３２において各画素同士の乗算が行われ、その結果が加算メモリ３３に供給される。
【００４８】
乗算器アレー３２の乗算結果が供給される加算メモリ３３は、図７に示すように、加算器アレー３３ａとメモリ（またはレジスタ、以下同様）アレー３３ｂとからなる。クラスの個数と等しい個数のメモリアレー３３ｂの並列回路が加算器アレー３３ａに対して接続されている。インデックスデコーダ３５からの出力（クラス）に応答して一つのメモリアレー３３ｂが選択される。また、メモリアレー３３ｂの出力が加算器アレー３３ａに帰還される。これらの乗算器アレー３２、加算器アレー３３ａ、メモリアレー３３ｂによって積和演算がなされる。インデックスによって決定されるクラス毎にメモリアレーが選択されて、積和演算の結果によってメモリアレーの内容が更新される。
【００４９】
前述の正規方程式（７）のｗ_ｉ にかかる積和演算の項を見ると、右上の項を反転すると、左下と同じものとなる。従って、（７）式を斜めに分割し、上側の三角形部分に含まれる項のみを演算すれば良い。この点から乗算器アレー３２、加算器アレー３３ａ、メモリアレー３３ｂは、図６および図７に示すように、上側の三角形部分に含まれる項を演算するのに必要とされる、乗算セルあるいはメモリセルを備えている。
【００５０】
以上のようにして、入力画像が到来するに従って積和演算が行われ、正規方程式が生成される。クラス毎の正規方程式の各項の結果は、クラスとそれぞれ対応するメモリアレー３３ｂに記憶されており、次に、この正規方程式の各項が掃き出し法のＣＰＵ演算回路３４に計算される。ＣＰＵを用いた演算によって正規方程式（連立方程式）が解かれ、最確値である係数が求まる。この係数が出力される。
【００５１】
復号のために設けられる、推定値生成回路２４ａは、第１の画素の推定値を作成するためのものであり、図８は、その一例の構成である。推定値生成回路２４ａと２４ｂとは、同一の構成である。４０で示す係数メモリは、例えば１フレーム毎に各クラスの係数組を記憶し、インデックスデコーダ３６からのクラス情報により選択された係数組を出力する。この係数組ｗ_１ 〜ｗ_ｎ がレジスタをそれぞれ介して乗算器３７_１ 〜３７_ｎ にその一方の入力として供給される。乗算器３７_１ 〜３７_ｎ の他方の入力としては、時系列変換回路３８によりまとめられた復号画素データｘ_１ 〜ｘ_ｎ が供給される。乗算器３７_１ 〜３７_ｎ の出力が加算器３９で加算される。加算器３９からは、注目画素の推定値ｙ（＝ｘ_１ ｗ_１ ＋ｘ_２ ｗ_２ ＋・・・・＋ｘ_ｎ ｗ_ｎ ）が得られる。
【００５２】
入力画像データを第１の画素と第２の画素に分離するためのパターンとしては、図１に示すものに限定されない。図９に示すように、水平方向に１画素毎に第１の画素および第２の画素に分離するパターン、あるいは垂直方向に１ライン毎に第１の画素および第２の画素に分離するパターン等を使用することができる。さらに、連続する２フレームの中の一方のフレームの画素を第１の画素として扱い、その他方のフレームの画素を第２の画素として扱うこともできる。
【００５３】
さらに、上述の例は、求める未定係数を１フレームに１組としたが、画像の局所的な特徴に応じて空間内で細分化し、１フレームに複数組の係数を求め、これを伝送しても良い。よりさらに、階層構造の補間を可能とする係数を伝送するようにしても良い。
【００５４】
次に、図１０および図１１を参照してこの発明の他の実施例について説明する。上述の一実施例は、伝送されるコードと係数とを用いて、８ビットの復号値を予測するものである。他の実施例は、誤差成分のみを予測するものである。
【００５５】
図１０は、他の実施例の符号化装置を示し、これは、一実施例の符号化装置（図２）と全体として類似している。最小二乗法の演算回路７ａに対して、クラス分類回路６からのインデックスと、減算回路９ａの出力と、ローカルデコーダ４ａ、４ｂからの復号データとが供給される。他方の最小二乗法の演算回路７ｂに対して、クラス分類回路６からのインデックスと、減算回路９ｂの出力と、ローカルデコーダ４ａ、４ｂからの復号データとが供給される。
【００５６】
減算回路９ａは、注目画素の実データとローカルデコーダ４ａの復号出力との誤差を計算し、減算回路９ｂは、注目画素の実データとローカルデコーダ４ｂの復号出力との誤差を計算する。図３の画素配列のモデルにおいて、＋の画素値を推定するための係数を決定する場合、周辺の画素の復号値を使用する。減算回路９ａは、実際の値をｙ_ｍ とし、ローカルデコード値をｙ_ｍ ´とすると、次の差分値δｙ_ｍ を発生する。
δｙ_ｍ ＝ｙ_ｍ −ｙ_ｍ ´
【００５７】
最小二乗法の演算回路７ａは、この差分値を周辺の画素の復号値ｘ_ｍｉと係数の線形１次結合で表す。すなわち、
δｙ_ｍ ＝Σｗ_ｉ ×ｘ_ｍｉ
そして、上述の（２）式におけるベクトルＹの要素が〔δｙ_１ δｙ_２ ・・・・δｙ_ｍ 〕とされ、最小二乗法により係数ｗ_ｉ が決定される。
【００５８】
図１１は、図１０の符号化装置に対応する復号装置を示す。受信されたコードがデコーダ２３ａ、２３ｂにより復号される。これらのデコーダ２３ａ、２３ｂの復号出力が推定値生成回路２４ａ、２４ｂに供給される。推定値生成回路２４ａには、クラス分類回路２６からのインデックスと、デコーダ２３ａ、２３ｂの復号出力と、入力端子２２ａからの係数とが供給される。推定値生成回路２４ｂには、クラス分類回路２６からのインデックスと、デコーダ２３ａ、２３ｂの復号出力と、入力端子２２ｂからの係数とが供給される。
【００５９】
推定値生成回路２４ａ、２４ｂは、復号出力と係数との線形１次結合によって、推定値（ここでは、差分値δｙ_ｍ ）を発生する。推定値生成回路２４ａからの推定値とデコーダ２３ａの復号出力とが加算回路２８ａに供給され、加算回路２８ａの出力に第１の画素データの復号出力が得られる。同様に、推定値生成回路２４ｂからの推定値とデコーダ２３ｂの復号出力とが加算回路２８ｂに供給され、加算回路２８ｂの出力に第２の画素データの復号出力が得られる。これらの復号出力が画素合成回路２５において合成され、出力端子２７に復号画像データが得られる。
【００６０】
この発明の他の実施例は、復号出力から誤差分のみを推定するので、復号値を直接予測する一実施例と比較して、推定の精度をより高くすることが可能である。
【００６１】
なお、上述の一実施例と同様に、注目画素が図３中の×（または△）の画素の位置にある場合と、これが○（または◎）の画素の位置にある場合とのそれぞれに関して、別々に係数の計算がなされる。
【００６２】
【発明の効果】
以上の説明から明らかなように、この発明は、伝送画素と非伝送画素とに分離するのと異なり、少ないビット数に変換された画素も伝送するので、復号画像の解像度の劣化を防止することができる。また、この発明は、線形１次結合で補間するための最適な係数を送信側で求めているので、補間フィルタを用いるのと比較して、復元画像の品質を良好とできる。
【図面の簡単な説明】
【図１】この発明を適用できる画素の分離パターンの一例とクラス分類に使用する画素を説明するための略線図である。
【図２】この発明が適用された高能率符号化装置の一例のブロック図である。
【図３】図２に示される高能率符号化の復号装置のブロック図である。
【図４】クラス分類回路の一例のブロック図である。
【図５】最小二乗法の演算回路の一例のブロック図である。
【図６】最小二乗法の演算回路に含まれる乗算器アレーを説明するための略線図である。
【図７】最小二乗法の演算回路に含まれる加算器アレーおよびメモリアレーを説明するための略線図である。
【図８】復号装置に含まれる推定値生成回路の一例のブロック図である。
【図９】この発明を適用できる画素の分離パターンの他の例を説明するための略線図である。
【図１０】この発明が適用された高能率符号化装置の他の例のブロック図である。
【図１１】図１０に示される高能率符号化の復号装置のブロック図である。
【符号の説明】
２ 画素分離回路
３ａ、３ｂ エンコーダ
４ａ、４ｂ ローカルデコーダ
６ クラス分類回路
７ａ、７ｂ 最小二乗法の演算回路
９ａ、９ｂ 復号値の誤差を検出する減算回路
２３ａ、２３ｂ デコーダ
２４ａ、２４ｂ 推定値生成回路[0001]
[Industrial application fields]
The present invention relates to a high-efficiency encoding apparatus and decoding apparatus for reducing the amount of transmission data when transmitting a digital image signal.
[0002]
[Prior art]
As one of high-efficiency encoding of digital image signals, there is one that reduces the amount of transmission data by thinning out pixels by sub-sampling. One example is the multiple sub-Nyquist sampling encoding method in the MUSE method. In this system, it is necessary to interpolate non-transmitted pixels that are thinned out at the receiving side.
[0003]
As described above, when creating the value of the pixel of interest, conventionally, an interpolation filter having a fixed tap and a fixed coefficient is usually used.
[0004]
Even if the process of interpolating non-transmission pixels with an interpolation filter is effective for a certain type of image, the interpolation process is generally effective for a wide variety of types of images such as moving images and still images. It is not always demonstrated in As a result, there is a problem that “blur”, “jerkiness” that is unnatural motion, or the like occurs in the restored image composed of the transmission pixel and the interpolation pixel.
[0005]
Furthermore, Japanese Patent Application Laid-Open No. 63-48088 proposed by the applicant of the present application calculates an average value of peripheral reference pixels when interpolating thinned pixels, and responds to the magnitude relationship between the average value and the value of each pixel. Then, each pixel is expressed by 1 bit, classification is performed according to the pattern of (reference pixel number × 1 bit), and a linear 1 between the coefficient determined by learning in advance for each class and the values of a plurality of surrounding pixels It is disclosed that an interpolation value is generated by a next combination.
[0006]
[Problems to be solved by the invention]
The above-mentioned previously proposed application can considerably solve the problems when using a fixed tap, fixed coefficient interpolation filter. However, since pixels are thinned out by sub-sampling, the resolution of the restored image is degraded to some extent.
[0007]
Accordingly, an object of the present invention is to provide a high-efficiency encoding apparatus and decoding apparatus in which resolution degradation is prevented by performing sub-sampling in the level direction of each pixel.
[0008]
[Means for Solving the Problems]
The invention according to claim 1 is a high-efficiency encoding apparatus for a digital image signal for reducing the transmission data amount of a digital image signal in which each pixel data is a P bit.
Separating means for separating a two-dimensional pixel array of the digital image signal into at least first and second pixel data according to a regular pattern;
First encoding means for converting the number of bits in the level direction in each pixel of the first pixel data into M (≦ P) bits and output data of the first encoding means for converting into P bits First local decoding means;
Second encoding means for converting the number of bits in the level direction in each pixel of the second pixel data into N (<M) bits and output data of the second encoding means for converting into P bits A second local decoding means;
First pixel dataAnd at least output data of the first local decoding means,Both of the first coefficients that are unknownmake use ofCoalitionSet up the equationMinimize the error between the first pixel data corresponding to the pixel of interest to be converted from M bits to P bits and the value obtained by the product-sum operation of the pixels in the vicinity of the output data corresponding to the pixel of interest and the first coefficient To be a coalitionBy solving the equation, the first coefficient having an information amount smaller than the information amount reduced by converting the number of bits is obtained.AskFirst coefficient generating means;
Second pixel dataAnd at least output data of the second local decoding means;Both of the second coefficients that are unknownmake use ofCoalitionSet up the equationMinimize the error between the second pixel data corresponding to the pixel of interest to be converted from N bits to P bits and the value obtained by the product-sum operation of the pixels in the vicinity of the output data corresponding to the pixel of interest and the second coefficient To be a coalitionBy solving the equation, a second coefficient having an information amount smaller than the information amount reduced by converting the number of bits is obtained.AskSecond coefficient generating means;
Transmission means for transmitting the output data of the first and second encoding means and the first coefficient and the second coefficient for reducing the decoding error of the output data of the first and second encoding means;
Is a high-efficiency encoding apparatus for digital image signals.
[0009]
According to a seventh aspect of the present invention, there is provided separation means for separating a two-dimensional pixel array of a digital image signal in which each pixel is a P-bit into at least first and second pixel data according to a regular pattern. The first encoding means for converting the number of bits in the level direction of each pixel of the image data to M (≦ P) bits and the first encoding means for converting the output data of the first encoding means to P bits Local decoding means, second encoding means for converting the number of bits in the level direction in each pixel of the second image data into N (<M) bits, and output data of the second encoding means as P Second local decoding means for converting to bits;First pixel dataAnd at least output data of the first local decoding means,Both of the first coefficients that are unknownmake use ofCoalitionSet up the equationMinimize the error between the first pixel data corresponding to the pixel of interest to be converted from M bits to P bits and the value obtained by the product-sum operation of the pixels in the vicinity of the output data corresponding to the pixel of interest and the first coefficient To be a coalitionBy solving the equation, the first coefficient having an information amount smaller than the information amount reduced by converting the number of bits is obtained.AskFirst coefficient generating means;Second pixel dataAnd at least output data of the second local decoding means;Both of the second coefficients that are unknownmake use ofCoalitionSet up the equationMinimize the error between the second pixel data corresponding to the pixel of interest to be converted from N bits to P bits and the value obtained by the product-sum operation of the pixels in the vicinity of the output data corresponding to the pixel of interest and the second coefficient To be a coalitionBy solving the equation, a second coefficient having an information amount smaller than the information amount reduced by converting the number of bits is obtained.AskThe first coefficient for reducing the decoding error of the output data of the second coefficient generating means, the output data of the first and second encoding means, and the output data of the first and second encoding means, and the second A decoding device corresponding to an encoding device comprising transmission means for transmitting coefficients,
Receiving means for receiving the output data of the first and second encoding means, the first and second coefficients;
First decoding means for converting M-bit output data of the first encoding means into P bits;
Second decoding means for converting the N-bit output data of the second encoding means into P bits;
First product-sum operation means for performing product-sum operation of at least the output data of the first decoding means and the first coefficient and creating an estimated value of the first image data;
A second sum-of-products operation means for performing a sum-of-products operation of at least the output data of the second decoding means and the second coefficient to create an estimated value of the second image data;
The estimated value of the first image data and the first2Pixel synthesizing means for synthesizing the estimated value of the image data and outputting as decoded data;
Is a high-efficiency code decoding device.
[0010]
Claim 10The described invention is a digital image signal high-efficiency encoding device for reducing the transmission data amount of a digital image signal in which each pixel data is P bits.
Separating means for separating a two-dimensional pixel array of the digital image signal into at least first and second pixel data according to a regular pattern;
First encoding means for converting the number of bits in the level direction in each pixel of the first image data into M (≦ P) bits and a first encoding means for converting the output data of the first encoding means into P bits. 1 local decoding means;
Second encoding means for converting the number of bits in the level direction at each pixel of the second image data into N (<M) bits and output data of the second encoding means for converting into P bits A second local decoding means;
The output data of the first and second encoding means or the output data of the first and second local decoding means are received, and a plurality of spatially and / or temporally neighboring plural data are respectively received for the plurality of target pixels. Classifying means for determining a class based on the characteristics of the reference pixels of
For each determined classFirst pixel dataAnd at least output data of the first local decoding means,Both of the first coefficients that are unknownmake use ofCoalitionSet up the equationMinimize the error between the first pixel data corresponding to the pixel of interest to be converted from M bits to P bits and the value obtained by the product-sum operation of the pixels in the vicinity of the output data corresponding to the pixel of interest and the first coefficient To be a coalitionBy solving the equation, the first coefficient having an information amount smaller than the information amount reduced by converting the number of bits is obtained.AskFirst coefficient generating means;
For each determined classSecond pixel dataAnd at least output data of the second local decoding means;Both of the second coefficients that are unknownmake use ofCoalitionSet up the equationMinimize the error between the second pixel data corresponding to the pixel of interest to be converted from N bits to P bits and the value obtained by the product-sum operation of the pixels in the vicinity of the output data corresponding to the pixel of interest and the second coefficient To be a coalitionBy solving the equation, a second coefficient corresponding to a class of information amount smaller than the information amount reduced by converting the number of bits is obtained.AskSecond coefficient generating means;
The output data of the first and second encoding means and the first coefficient and the second coefficient for each class for reducing the decoding error of the output data of the first and second encoding means are transmitted. Transmission means and
Is a high-efficiency encoding apparatus for digital image signals.
[0011]
Claim 15According to the described invention, according to a regular pattern, separation means for separating a two-dimensional pixel array of a digital image signal of which each pixel is P bits into at least first and second pixel data, and first image data First encoding means for converting the number of bits in the level direction in each pixel into M (≦ P) bits and first local decoding means for converting the output data of the first encoding means into P bits And the second encoding means for converting the number of bits in the level direction at each pixel of the second image data into N (<M) bits and the output data of the second encoding means are converted into P bits. In order to receive the output data of the second local decoding means and the first and second encoding means or the output data of the first and second local decoding means, spatially and Or temporally based on the characteristics of a plurality of reference pixels in the vicinity, the first classification means for determining a class for each class determined by the first classification meansFirst pixel dataAnd at least output data of the first local decoding means,Both of the first coefficients that are unknownmake use ofCoalitionSet up the equationMinimize the error between the first pixel data corresponding to the pixel of interest to be converted from M bits to P bits and the value obtained by the product-sum operation of the pixels in the vicinity of the output data corresponding to the pixel of interest and the first coefficient To be a coalitionBy solving the equationThe amount of information is smaller than the amount of information reduced by converting the number of bits.The first coefficientAskFirst coefficient generation means and each determined classSecond pixel dataAnd at least output data of the second local decoding means;Both of the second coefficients that are unknownmake use ofCoalitionSet up the equationMinimize the error between the second pixel data corresponding to the pixel of interest to be converted from N bits to P bits and the value obtained by the product-sum operation of the pixels in the vicinity of the output data corresponding to the pixel of interest and the second coefficient To be a coalitionBy solving the equationThe amount of information is smaller than the amount of information reduced by converting the number of bits.The second coefficientAskA first coefficient for each class for reducing the decoding error of the second coefficient generating means, the output data of the first and second encoding means, and the output data of the first and second encoding means, and A decoding device corresponding to an encoding device comprising transmission means for transmitting the second coefficient,
Receiving means for receiving the output data of the first and second encoding means, the first and second coefficients;
First decoding means for converting M-bit output data of the first encoding means into P bits;
Second decoding means for converting the N-bit output data of the second encoding means into P bits;
Receiving the output data of the first and second encoding means or the output data of the first and second decoding means, and features of a plurality of reference pixels spatially and / or temporally adjacent to the target pixel A second class classification means for determining a class based on:
A product-sum operation is performed on at least the output data of the first decoding unit and the first coefficient corresponding to the class determined by the second class classification unit, and an estimated value of the first image data is created. Product-sum operation means;
A second summation operation of at least the output data of the second decoding means and a second coefficient corresponding to the class determined by the second class classification means to create an estimated value of the second image data Product-sum operation means;
Pixel synthesizing means for synthesizing the estimated value of the first image data and the estimated value of the second image data and outputting them as decoded data;
Is a high-efficiency code decoding device.
[0012]
[Action]
The pixels of the digital image signal are separated into first and second pixels according to a predetermined pattern. The first and second pixels are not thinned out, but the number of bits is reduced. Then, for the first and second pixels, when obtaining the original value of the bit number, a coefficient that minimizes the square of the error is generated, and this coefficient is transmitted. On the receiving side, the values of the first and second pixels are formed by linear linear combination using coefficients and transmission pixel data. As a result, decoding error can be reduced and a good restored image can be created.
[0013]
【Example】
In order to facilitate understanding of the present invention, first, generation of coefficients will be described with reference to FIG. FIG. 1 shows an example of a model for encoding and generating coefficients according to the present invention. T1, T2, and T3 indicate three frames that are temporally continuous. In each frame, all pixels are separated into first and second pixels with a five-eye lattice pattern. Between the frames, the phase of the fifth lattice pattern is complementary. As an example, the number of quantization bits P of each pixel of the input digital image signal is 8.
[0014]
In FIG. 1, the first pixel is represented by ○ and ◎, the second pixel is represented by × and △, and the transmission pixels used for classification are represented by ◎ and Δ. That is, eight of the (25 × 3-1 = 74) pixels are reference pixels that are also used for classification. The first pixel is converted to M (≦ 8) bits and the second pixel is converted to N (<M) bits. In the present invention, classification is not essential, but classification is performed in the following embodiment.
[0015]
A pixel of interest (indicated by +) included in the frame T2 is represented by, for example, an n-tap linear linear combination model. More specifically, as shown in FIG. 1, (5 × 5) regions at the same spatial position are cut out from the frames T1, T2, and T3. One three-dimensional block is constituted by three regions. As will be described in more detail below, the pixel of interest in the center of the frame T2 is represented by a linear linear combination model of n peripheral pixels and coefficients, and the square of the error with respect to the actual data of the data expressed by the linear linear combination The coefficient is determined by the least square method so that is minimized. As an example, a set of coefficients for each class is determined in one frame or field, and encoded pixel data and coefficients are transmitted.
[0016]
In the spatio-temporal model shown in FIG. 1, a total of 75 pixels are included in a block including three regions. The value of n pixels other than the target pixel in this pixel is set to x_i(I = 1, 2,..., N). Here, as described above, the first and second pixels are converted into M bits and N bits, respectively, but when the coefficients are determined, data locally decoded into P bits is used. The coefficient multiplied to each transmission pixel is w₁  ~ W_nIt is prescribed. Therefore, if the value of the pixel of interest in the frame T2 is y, this value is the linear linear combination x of the transmission pixel and the coefficient._iw_iIt expresses with. That is, the value y at the center position of the frame T2 is thus a linear primary combination w of n-tap peripheral pixels.₁  x₁  + W₂  x₂  + ... + w_nx_nRepresented by Coefficient w in this linear linear combination model_iAs for, a value that minimizes the square of the error between the actual value and the value represented by linear linear combination is obtained.
[0017]
This undetermined coefficient w_i1 to determine the transmission pixel value x of the block shown in FIG. 1 when the input image is shifted by two pixels in the spatial direction (horizontal direction and vertical direction)._i(I = 1,..., N) and the actual value y of the target pixel to be interpolated_jFormulas of linear linear combinations are substituted for (j = 1,..., M), respectively. For example, when obtaining a set of coefficients for one frame, a large number of expressions, that is, the number of pixels in one frame (= m ) Are created (referred to as observation equations). In order to determine n coefficients, at least (m = n) simultaneous equations are required. The number m of equations can be selected as appropriate in consideration of the problem of interpolation accuracy and the processing time. The observation equation is
XW = Y (1)
It is. Here, X, W, and Y are the following matrices, respectively.
[0018]
[Expression 1]

[0019]
A coefficient w that minimizes an error from an actual value is obtained by a least square method. For this purpose, the following residual equation is created by adding a residual matrix E to the right side of the observation equation. That is, in the least square method, the coefficient matrix W that minimizes the square of the elements of the residual matrix E in the residual equation, that is, the square error, is obtained.
[0020]
[Expression 2]

[0021]
Next, each element w of the coefficient matrix W from the residual equation (3)_iThe condition for finding the most probable value of is to satisfy the condition of squaring m residuals corresponding to the pixels in the block and minimizing the sum thereof. This condition is expressed by the following formula (4).
[0022]
[Equation 3]

[0023]
An undetermined coefficient w that is an element of a coefficient matrix W that satisfies n conditions.₁  , W₂  , ..., w_nFind out. Therefore, from the residual equation (3),
[0024]
[Expression 4]

[0025]
It becomes. If the condition of equation (4) is established for i = 1, 2,..., N),
[0026]
[Equation 5]

[0027]
Is obtained. From the equations (3) and (6), the following normal equation is obtained.
[0028]
[Formula 6]

[0029]
The normal equation (7) is a simultaneous equation having exactly n unknowns. As a result, each undetermined coefficient w which is the most probable value_iCan be requested. To be precise, w in equation (7)_iIf the matrix is regular, it can be solved. Actually, the undetermined coefficient w using the Gauss-Jordan elimination method (also called the sweep-out method)_iSeeking. In this way, one set of coefficients for interpolation of non-transmission pixels is determined for each class in one frame, and this coefficient is transmitted.
[0030]
As described above, when determining the coefficients, classification is performed that reflects the characteristics of the partial image including the target pixel that is the object of interpolation. As this classification, it is conceivable to use a value of a referenced pixel. However, when the value of each pixel is 8 bits, the number of classes is 2 in the case of 8 reference pixels.⁶⁴And the number of classes becomes very large. In order to solve this problem, the number of bits of the reference pixel is reduced by compression encoding. Another method for classifying may be to detect a direction in which a small area including a pixel of interest has a strong correlation and define a class corresponding to the direction.
[0031]
FIG. 2 is an overall block diagram of one embodiment of the present invention. Digital image data from the input terminal 1 is supplied to the pixel separation circuit 2, and the input pixel is separated into first and second pixels according to a pattern as shown in FIG. The first pixel data is supplied to the encoder 3a for converting the number of bits to a smaller number of bits. In the encoder 3a, the number of bits of each pixel is converted from 8 bits to M (<8) bits. The second pixel data is supplied to the encoder 3b. In the encoder 3b, the number of bits of each pixel is converted to N (<M) bits.
[0032]
An example of the encoders 3a and 3b is an encoder (ADRC) encoder adapted to the dynamic range according to the proposal of the present applicant. ADRC detects the maximum and minimum values of pixels for each block, obtains the dynamic range that is the difference between them, and quantizes the pixel data after removing the minimum or maximum values in a quantization step adapted to the dynamic range. (See JP-A-61-144989). Even if the pixel data in the block is quantized with a bit number M (for example, 4 bits) smaller than the original quantization bit number (for example, 8 bits) from the local correlation of the image, deterioration of the image can be suppressed.
[0033]
A simpler example of the encoding performed by the encoders 3a, 3b is to remove several bits from the least significant bit or the least significant bit of each eight bits. The outputs of the encoders 3a and 3b are taken out as transmission codes to the output terminals 5a and 5b, respectively, and supplied to the local decoders 4a and 4b. Data decoded into 8 bits is formed by the local decoders 4a and 4b.
[0034]
The encoded data from the encoders 3a and 3b is supplied to the class classification circuit 6. As described above, the class classification circuit 6 uses the reference pixels around the target pixel to indicate the class corresponding to the characteristics (level distribution, direction with strong correlation, etc.) of the small region including the target pixel, for example, 8 Generate a bit index. Since the first and second pixels are used as the reference pixels, the outputs of the encoders 3a and 3b are supplied to the class classification circuit 6.
[0035]
The reference pixels for classification are output from the encoders 3a and 3b, but are not limited thereto, and output data from the local decoders 4a and 4b may be used. The index from the class classification circuit 6, the actual data of the pixel of interest, and the decoded data from the local decoders 4a and 4b are supplied to the arithmetic circuit 7a of the least square method. Similarly, the index, the actual data of the pixel of interest, and the decoded data from the local decoders 4a and 4b are supplied to the least squares arithmetic circuit 7b.
[0036]
Each of the least squares arithmetic circuits 7a and 7b has an x_iUsing the decoded data, and using the actual value of the pixel-of-interest data, for example, one set of coefficients w per frame by the above-mentioned least-squares algorithm_iTo decide. The coefficients from the arithmetic circuits 7a and 7b are taken out to the output terminals 8a and 8b. The encoded output (code) and coefficient of the transmission pixel are sent to the transmission line via a framing circuit, a channel modulation circuit, etc., although not shown. The transmission path is a communication line, a magnetic recording / reproducing process, or the like.
[0037]
The data to be transmitted is a code in which the number of bits in one frame is reduced to M bits and N bits and two sets of coefficients. The information amount of this coefficient is negligible compared to the information amount of the code per frame. Therefore, the amount of transmission data can be reduced by reducing the number of bits of each pixel.
[0038]
FIG. 3 shows a decoding circuit corresponding to the encoding circuit of FIG. Although not shown, the received data is subjected to processing such as channel demodulation and frame decomposition, the code is supplied to the input terminals indicated by 21a and 21b, and the coefficient is supplied to the input terminals indicated by 22a and 22b. The M-bit code is supplied to the compression-encoding decoder 23a and decoded into 8-bit data. The N-bit code is supplied to the compression-encoding decoder 23b and decoded into 8-bit data.
[0039]
The codes from the input terminals 21a and 21b are also supplied to the class classification circuit 26. The class classification circuit 26 performs the same classification as the class classification circuit 6 in the encoding circuit and generates an index. If the classification is performed using the decoded data during encoding, the classification is performed using the output data of the decoders 23a and 23b. This index is supplied to the estimated value generation circuits 24a and 24b. The estimated value generation circuits 24a and 24b are also supplied with the outputs of the decoders 23a and 23b and the coefficients from the input terminals 22a and 22b, respectively.
[0040]
The estimated value generation circuits 24a and 24b generate 8-bit values (estimated values) for the first and second pixels by linear primary combination of decoded data and coefficients. The outputs of the estimated value generation circuits 24 a and 24 b are supplied to the pixel synthesis circuit 25. The pixel synthesis circuit 25 synthesizes the first and second pixels in the original positional relationship, and decoded image data is obtained at the output terminal 27. Although 8-bit decoded data is obtained by the decoders 23a and 23b, the estimated value created by linear linear combination of the coefficient and the surrounding decoded data has less error than the decoded data. .
[0041]
2 and 3, the coefficients are determined in real time using the input image data. However, the coefficients can be determined in advance by learning. In that case, a versatile coefficient is determined using images of different patterns, and this is stored in the memory as a fixed coefficient. This memory is provided in a decoding circuit for interpolation of the pixel of interest, and a fixed coefficient in the memory is used. Furthermore, it is possible to store a fixed coefficient determined by learning in a memory and update the coefficient of the memory with a coefficient determined from input image data that is actually transmitted.
[0042]
An example of the class classification circuit 6 is shown in FIG. Outputs of the encoders 3a and 3b are supplied to the selection circuit 42 from the input terminals 41a and 41b. The selection circuit 42 selects the encoded data of the first and second pixels used for class classification, and the selected encoded data is supplied to the 1-bit ADRC circuits 43a and 43b, respectively.
[0043]
The 1-bit ADRC circuits 43a and 43b detect the difference between the maximum value and the minimum value (dynamic range DR) of an appropriately sized block, divide each data by the dynamic range DR, and set the quotient to 0.5. By comparison, an output of ｀ 0 'is generated when it is smaller than 0.5, and ｀ 1' is generated when it is greater than 0.5. That is, input data is converted into a 1-bit output.
[0044]
The outputs of the 1-bit ADRC circuits 43a and 43b are serial-parallel converted in the shift registers 44a and 44b. A total of 8 bits from the shift registers 44 a and 44 b are taken into the register 45, and an 8-bit index is generated at the output of the register 45. 2 with 8-bit index⁸  A number of classes are indicated.
[0045]
For classification, not only pixels in the vicinity of the target pixel in space but also pixels in the same position in the previous frame and the subsequent frame may be referred to. In addition to 1-bit ADRC, vector quantization, DPCM, or the like can be used as compression coding for classification.
[0046]
Next, the least squares arithmetic circuit 7a will be described with reference to FIG. The arithmetic circuit 7b has the same configuration as the arithmetic circuit 7a. As shown in FIG. 5, the output signal of the encoder 3a is supplied, and the data constituting the spatio-temporal model, that is, the actual data y of the pixel of interest and the data x used for linear linear combination._iA time series conversion circuit 31 is provided for synchronizing the two. Data from the time series conversion circuit 31 is supplied to the multiplier array 32. An addition memory 33 is connected to the multiplier array 32. The index is supplied to the decoder 35, and the class information from the decoder 35 is supplied to the addition memory 33. These multiplier array 32 and addition memory 33 constitute a normal equation generation circuit.
[0047]
The multiplier array 32 performs multiplication of each pixel, and the addition memory 33 includes an adder array and a memory array to which the multiplication result from the multiplier array 32 is supplied. FIG. 6 shows a specific configuration of the multiplier array 32. In FIG. 6, a square cell represents a multiplier, as one of them is enlarged. The multiplier array 32 multiplies each pixel and supplies the result to the addition memory 33.
[0048]
As shown in FIG. 7, the addition memory 33 to which the multiplication result of the multiplier array 32 is supplied comprises an adder array 33a and a memory (or register, the same applies hereinafter) array 33b. A parallel circuit of a number of memory arrays 33b equal to the number of classes is connected to the adder array 33a. In response to the output (class) from the index decoder 35, one memory array 33b is selected. The output of the memory array 33b is fed back to the adder array 33a. A product-sum operation is performed by the multiplier array 32, the adder array 33a, and the memory array 33b. A memory array is selected for each class determined by the index, and the contents of the memory array are updated according to the result of the product-sum operation.
[0049]
W of the above normal equation (7)_iLooking at the product-sum operation term for, inverting the upper right term results in the same as the lower left. Therefore, it is only necessary to divide equation (7) diagonally and calculate only the terms included in the upper triangular portion. From this point, the multiplier array 32, the adder array 33a, and the memory array 33b are, as shown in FIGS. 6 and 7, a multiplication cell or memory required for calculating a term included in the upper triangular portion. Has a cell.
[0050]
As described above, the product-sum operation is performed as the input image arrives, and a normal equation is generated. The result of each term of the normal equation for each class is stored in the memory array 33b corresponding to each class, and each term of this normal equation is then calculated by the CPU arithmetic circuit 34 of the sweep-out method. A normal equation (simultaneous equations) is solved by calculation using the CPU, and a coefficient which is the most probable value is obtained. This coefficient is output.
[0051]
The estimated value generation circuit 24a provided for decoding is for creating an estimated value of the first pixel, and FIG. 8 shows an example of the configuration. The estimated value generation circuits 24a and 24b have the same configuration. The coefficient memory indicated by 40 stores, for example, the coefficient group of each class for each frame, and outputs the coefficient group selected by the class information from the index decoder 36. This coefficient set w₁  ~ W_nIs a multiplier 37 through each register.₁  ~ 37_nAs one input. Multiplier 37₁  ~ 37_nAs the other input, the decoded pixel data x collected by the time series conversion circuit 38 is used.₁  ~ X_nIs supplied. Multiplier 37₁  ~ 37_nAre added by an adder 39. From the adder 39, the estimated value y (= x) of the target pixel.₁  w₁  + X₂  w₂  + .... + x_nw_n) Is obtained.
[0052]
The pattern for separating the input image data into the first pixel and the second pixel is not limited to the pattern shown in FIG. As shown in FIG. 9, a pattern that separates the first pixel and the second pixel for each pixel in the horizontal direction, or a pattern that separates the first pixel and the second pixel for each line in the vertical direction, etc. Can be used. Furthermore, it is possible to treat a pixel of one frame out of two consecutive frames as a first pixel and a pixel of the other frame as a second pixel.
[0053]
Furthermore, in the above-mentioned example, the set of undetermined coefficients to be obtained is one set per frame, but it is subdivided in the space according to the local characteristics of the image, and a plurality of sets of coefficients are obtained in one frame and transmitted. Also good. Furthermore, a coefficient that enables interpolation of the hierarchical structure may be transmitted.
[0054]
Next, another embodiment of the present invention will be described with reference to FIGS. In the embodiment described above, an 8-bit decoded value is predicted using a transmitted code and coefficient. Another embodiment predicts only the error component.
[0055]
FIG. 10 shows another embodiment of the encoding device, which is generally similar to the encoding device of one embodiment (FIG. 2). The index from the class classification circuit 6, the output from the subtraction circuit 9a, and the decoded data from the local decoders 4a and 4b are supplied to the least squares arithmetic circuit 7a. The index from the class classification circuit 6, the output of the subtraction circuit 9b, and the decoded data from the local decoders 4a and 4b are supplied to the other least squares arithmetic circuit 7b.
[0056]
The subtraction circuit 9a calculates an error between the actual data of the target pixel and the decoded output of the local decoder 4a, and the subtraction circuit 9b calculates an error between the actual data of the target pixel and the decoded output of the local decoder 4b. In the model of the pixel array in FIG. 3, when determining a coefficient for estimating a + pixel value, the decoded values of surrounding pixels are used. The subtraction circuit 9a converts the actual value to y_mAnd the local decode value is y_mIf ′, the next difference value δy_mIs generated.
δy_m= Y_m-Y_m´
[0057]
The least squares arithmetic circuit 7a uses this difference value as the decoded value x of the surrounding pixels._miAnd linear linear combination of coefficients. That is,
δy_m= Σw_iX_mi
The element of the vector Y in the above equation (2) is [δy₁δy₂・ ・ ・ ・ Δy_m] And the coefficient w by the least square method_iIs determined.
[0058]
FIG. 11 shows a decoding apparatus corresponding to the encoding apparatus of FIG. The received code is decoded by the decoders 23a and 23b. The decoded outputs of these decoders 23a and 23b are supplied to estimated value generation circuits 24a and 24b. The estimated value generation circuit 24a is supplied with the index from the class classification circuit 26, the decoded outputs of the decoders 23a and 23b, and the coefficient from the input terminal 22a. The estimated value generation circuit 24b is supplied with the index from the class classification circuit 26, the decoded outputs of the decoders 23a and 23b, and the coefficient from the input terminal 22b.
[0059]
The estimated value generating circuits 24a and 24b perform estimated values (here, difference values δy) by linear linear combination of decoded outputs and coefficients._m). The estimated value from the estimated value generating circuit 24a and the decoded output of the decoder 23a are supplied to the adding circuit 28a, and the decoded output of the first pixel data is obtained at the output of the adding circuit 28a. Similarly, the estimated value from the estimated value generating circuit 24b and the decoded output of the decoder 23b are supplied to the adding circuit 28b, and a decoded output of the second pixel data is obtained at the output of the adding circuit 28b. These decoded outputs are combined in the pixel combining circuit 25, and decoded image data is obtained at the output terminal 27.
[0060]
In another embodiment of the present invention, since only the error is estimated from the decoded output, it is possible to increase the accuracy of the estimation as compared with the embodiment in which the decoded value is directly predicted.
[0061]
As in the above-described embodiment, each of the case where the target pixel is located at the position of the pixel X (or Δ) in FIG. 3 and the case where it is located at the position of the pixel ◯ (or ◎) The coefficients are calculated separately.
[0062]
【The invention's effect】
As is clear from the above description, the present invention transmits a pixel converted to a small number of bits, unlike the case of separating the transmission pixel and the non-transmission pixel, thereby preventing the resolution of the decoded image from deteriorating. Can do. Further, according to the present invention, since the optimum coefficient for interpolation by linear linear combination is obtained on the transmission side, the quality of the restored image can be improved as compared with the case where the interpolation filter is used.
[Brief description of the drawings]
FIG. 1 is a schematic diagram for explaining an example of a pixel separation pattern to which the present invention can be applied and pixels used for classification.
FIG. 2 is a block diagram of an example of a high-efficiency encoding device to which the present invention is applied.
FIG. 3 is a block diagram of a decoding apparatus for high efficiency encoding shown in FIG. 2;
FIG. 4 is a block diagram of an example of a class classification circuit.
FIG. 5 is a block diagram illustrating an example of a least squares arithmetic circuit.
FIG. 6 is a schematic diagram for explaining a multiplier array included in an arithmetic circuit of the least square method.
FIG. 7 is a schematic diagram for explaining an adder array and a memory array included in an arithmetic circuit of the least square method.
FIG. 8 is a block diagram of an example of an estimated value generation circuit included in the decoding device.
FIG. 9 is a schematic diagram for explaining another example of a pixel separation pattern to which the present invention can be applied;
FIG. 10 is a block diagram of another example of a high-efficiency encoding device to which the present invention is applied.
FIG. 11 is a block diagram of the decoding device for high efficiency encoding shown in FIG.
[Explanation of symbols]
2 Pixel separation circuit
3a, 3b encoder
4a, 4b Local decoder
6 classification circuit
7a, 7b Least square method arithmetic circuit
9a, 9b Subtraction circuit for detecting an error in the decoded value
23a, 23b decoder
24a, 24b Estimated value generation circuit

Claims (17)

In a high-efficiency encoding apparatus for digital image signals for reducing the amount of transmission data of digital image signals of P bits for each pixel data,
Separating means for separating a two-dimensional pixel array of the digital image signal into at least first and second pixel data according to a regular pattern;
First encoding means for converting the number of bits in the level direction in each pixel of the first pixel data into M (≦ P) bits and output data of the first encoding means are converted into P bits. First local decoding means for,
Second encoding means for converting the number of bits in the level direction in each pixel of the second pixel data into N (<M) bits and output data of the second encoding means are converted into P bits. A second local decoding means for
Along with the first pixel data and at least the output data of the first local decoding means, making a simultaneous equation using the first coefficient is unknown, corresponding to the pixel of interest to be converted from M bits to the P bits Solving the simultaneous equations so as to minimize an error between the first pixel data and a value obtained by a product-sum operation of a pixel in the vicinity of the output data corresponding to the target pixel and the first coefficient. by a first coefficient generating means for obtaining the first coefficient of less information than the amount of information it was reduced by converting the number of bits,
Along with the second pixel data and at least the output data of the second local decoding means, making a simultaneous equation using the second coefficient is unknown, corresponding to the pixel of interest to be converted from N bits to the P bits Solving the simultaneous equations so as to minimize an error between the second pixel data and a value obtained by a product-sum operation of a pixel in the vicinity of the output data corresponding to the target pixel and the second coefficient. by a second coefficient generating means for determining a second coefficient of less information than the amount of information it was reduced by converting the number of bits,
Transmitting the output data of the first and second encoding means and the first coefficient and the second coefficient for reducing the decoding error of the output data of the first and second encoding means A high-efficiency encoding apparatus for digital image signals, comprising: The encoding device according to claim 1, wherein
The first and second coefficient generating means are:
A high-efficiency encoding apparatus characterized in that the simultaneous equations are established for each frame or field to determine the first and second coefficients. The encoding device according to claim 1, wherein
The value obtained by the product-sum operation of the pixel near the output data corresponding to the target pixel and the first coefficient corresponding to the target pixel in the first coefficient generating means is the spatial and the output data corresponding to the target pixel. / Or is obtained by linear linear combination of a plurality of pixels that are temporally neighboring and the plurality of first coefficients,
The value obtained by the product-sum operation of the pixel near the output data corresponding to the target pixel and the second coefficient corresponding to the target pixel in the second coefficient generating means is the spatial and the output data corresponding to the target pixel. A high-efficiency encoding device characterized in that it is obtained by linear linear combination of a plurality of pixels that are temporally adjacent and a plurality of the second coefficients . In a high-efficiency encoding apparatus for digital image signals for reducing the amount of transmission data of digital image signals of P bits for each pixel data,
Separating means for separating a two-dimensional pixel array of the digital image signal into at least first and second pixel data according to a regular pattern;
First encoding means for converting the number of bits in the level direction in each pixel of the first pixel data into M (≦ P) bits and output data of the first encoding means are converted into P bits. First local decoding means for,
First error data generating means for generating first error data from the first image data from the separating means and the output data of the first local decoding means;
Converting the output data of the second encoding means and the second encoding means to convert the number of bit level direction at each pixel of said second pixel data into N (<M) bits to the P bits Second local decoding means for
Second error data generating means for generating second error data from the second image data from the separating means and the output data of the second local decoding means;
Together with the first error data from the first error data generating means and at least the output data of the first local decoding means, a simultaneous equation is established using a first coefficient which is an unknown number, and M bits are changed to P bits. The first error data corresponding to the target pixel to be converted and the linear 1 of the plurality of pixels and the plurality of first coefficients spatially and / or temporally adjacent to the output data corresponding to the target pixel. By solving the simultaneous equations so as to minimize the error from the error value obtained by the second combination, the first coefficient for obtaining the first coefficient having an information amount smaller than the information amount reduced by converting the number of bits is obtained. 1 coefficient generating means;
Together with the second error data from the second error data generating means and at least the output data of the second local decoding means, a simultaneous equation is established using a second coefficient which is an unknown number, and from N bits to P bits The second error data corresponding to the pixel of interest to be converted, and the spatially and / or temporally neighboring pixels of the output data corresponding to the pixel of interest and the linear 1 of the plurality of second coefficients. A second coefficient for obtaining a second coefficient having an information amount smaller than the information amount reduced by converting the number of bits by solving the simultaneous equations so as to minimize an error with an error value obtained by the second combination. Coefficient generation means,
Transmitting the output data of the first and second encoding means and the first coefficient and the second coefficient for reducing the decoding error of the output data of the first and second encoding means Transmission means to
A highly efficient encoding apparatus for digital image signals . The encoding device according to claim 1, wherein
Said first coefficient generating means, when obtaining the first coefficient, further make the simultaneous equations using the output data of said second local decoding means,
Said second coefficient generating means, when determining the second coefficient, and further, the first local decoding means high-efficiency encoding apparatus in which the output data using, characterized in that make the simultaneous equations . In the high efficiency encoding device according to claim 1,
The first and second coefficient generating means determine the coefficient in real time. Separating means for separating a two-dimensional pixel array of a digital image signal in which each pixel has a P-bit according to a regular pattern into at least first and second pixel data, and a level at each pixel of the first image data First encoding means for converting the number of bits in the direction into M (≦ P) bits, first local decoding means for converting the output data of the first encoding means into P bits, and Second encoding means for converting the number of bits in the level direction at each pixel of the second image data into N (<M) bits and output data of the second encoding means for converting into P bits a second local decoding means, and the first pixel data and at least the output data of the first local decoding means together, the simultaneous equations using the first coefficient is unknown standing on, M-bi The first pixel data corresponding to the pixel of interest to be converted from a bit to a P bit, the value obtained by the product-sum operation of the pixels in the vicinity of the output data corresponding to the pixel of interest and the first coefficient by solving the simultaneous equations to the error to a minimum, the first coefficient generating means for obtaining the first coefficient of less information than the amount of information was reduced by converting the number of bits, the along with the output data of at least the second local decoding means and the second pixel data, making a simultaneous equation using the second coefficient is unknown, corresponding to the pixel of interest to be converted from N bits to the P bits above The simultaneous equations are set so as to minimize an error between the second pixel data and a value obtained by the product-sum operation of the pixel in the vicinity of the output data corresponding to the target pixel and the second coefficient. By solving, the second coefficient generating means for determining a second coefficient of less information than the amount of information was reduced by converting the number of bits, the output data of the first and second encoding means A decoding apparatus corresponding to an encoding apparatus comprising transmission means for transmitting the first coefficient and the second coefficient for reducing decoding errors of output data of the first and second encoding means Because
Receiving means for receiving the output data of the first and second encoding means and the first and second coefficients;
First decoding means for converting M-bit output data of the first encoding means into P bits;
Second decoding means for converting the N-bit output data of the second encoding means into P bits;
First product-sum operation means for performing product-sum operation of at least the output data of the first decoding means and the first coefficient to create an estimated value of the first image data;
A second sum-of-products operation means for performing a sum-of-products operation of at least the output data of the second decoding means and a second coefficient and creating an estimated value of the second image data;
Pixel synthesizing means for synthesizing the estimated value of the first image data and the estimated value of the second image data and outputting them as decoded data;
A high-efficiency code decoding apparatus comprising: The decoding device according to claim 7, wherein
The value obtained by the product-sum operation of the pixel near the output data corresponding to the target pixel and the first coefficient corresponding to the target pixel in the first coefficient generating means is the spatial and the output data corresponding to the target pixel. / Or is obtained by linear linear combination of a plurality of pixels that are temporally neighboring and the plurality of first coefficients,
The value obtained by the product-sum operation of the pixel near the output data corresponding to the target pixel and the second coefficient corresponding to the target pixel in the second coefficient generating means is the spatial and the output data corresponding to the target pixel. / Or is obtained by a linear first-order combination of a plurality of neighboring pixels in time and the plurality of second coefficients,
The first product-sum operation means is:
Receiving at least the output data of the first decoding means and the first coefficient data, and creating an estimated value of the first image data by linear linear combination;
The second product-sum operation means is
A decoding apparatus characterized by receiving at least output data of the second decoding means and second coefficient data, and generating an estimated value of the second image data by linear linear combination. Separating means for separating the two-dimensional pixel array of the digital image signal into at least first and second pixel data according to a regular pattern, and the number of bits in the level direction at each pixel of the first pixel data is M First encoding means for converting to (≦ P) bits, first local decoding means for converting the output data of the first encoding means to P bits, and first from the separating means First error data generating means for generating first error data from the output data of the first local decoding means and the number of bits in the level direction at each pixel of the second pixel data is N ( <M) Second encoding means for converting to bits, second local decoding means for converting the output data of the second encoding means to P bits, and the separation means Second error data generating means for generating second error data from the second image data and output data of the second local decoding means; first error data from the first error data generating means; The first error data corresponding to the pixel of interest to be converted from M bits to P bits by establishing simultaneous equations using the first coefficient which is an unknown number together with at least the output data of the first local decoding means; To minimize an error between an error value obtained by a linear first combination of a plurality of spatially and / or temporally neighboring pixels of the output data corresponding to the pixel of interest and a plurality of the first coefficients. First coefficient generating means for obtaining the first coefficient having an information amount smaller than the information amount reduced by converting the number of bits by solving the simultaneous equations, and the second Together with the output data of the second error data and at least the second local decoding means from the difference data generating means, making a simultaneous equation using the second coefficient is unknown, to be converted from N bits to the P bits A linear linear combination of the second error data corresponding to the pixel of interest and the output data corresponding to the pixel of interest in spatially and / or temporally adjacent pixels and the plurality of second coefficients. Generation of a second coefficient for obtaining a second coefficient having an information amount smaller than the information amount reduced by converting the number of bits by solving the simultaneous equations so as to minimize an error from the obtained error value. Means, the first coefficient for reducing the decoding error of the output data of the first and second encoding means, and the output data of the first and second encoding means, and the first coefficient A decoding device corresponding to a digital image signal encoding device comprising transmission means for transmitting a coefficient of two,
Receiving means for receiving the output data of the first and second encoding means and the first and second coefficients;
First decoding means for converting M-bit output data of the first encoding means into P bits;
Second decoding means for converting the N-bit output data of the second encoding means into P bits;
First product-sum operation means for performing linear linear combination by product-sum operation of at least the output data of the first decoding means and the first coefficient, and creating error data of the first image data;
Second product-sum operation means for performing linear linear combination by product-sum operation of at least the output data of the second decoding means and the second coefficient, and creating error data of the second image data;
First addition means for adding the output data of the first decoding means and the error data of the first product-sum operation means;
Second addition means for adding the output data of the second decoding means and the error data of the second product-sum operation means;
Pixel combining means for combining the output data from the first adding means and the output data from the second adding means and outputting as decoded data;
A high-efficiency code decoding apparatus comprising: In a high-efficiency encoding apparatus for digital image signals for reducing the amount of transmission data of digital image signals of P bits for each pixel data,
Separating means for separating a two-dimensional pixel array of the digital image signal into at least first and second pixel data according to a regular pattern;
First encoding means for converting the number of bits in the level direction in each pixel of the first image data into M (≦ P) bits and output data of the first encoding means for converting into P bits First local decoding means,
Second encoding means for converting the number of bits in the level direction in each pixel of the second image data into N (<M) bits and output data of the second encoding means are converted into P bits. A second local decoding means for
The output data of the first and second encoding means or the output data of the first and second local decoding means are received, and spatially and / or temporally adjacent to each of the plurality of target pixels. Classifying means for determining a class based on the characteristics of a plurality of reference pixels;
Both the output data of at least said first local decoding means with the first pixel data for each class determined above, making a simultaneous equation using the first coefficient is unknown, the conversion from M bits to the P bits An error between the first pixel data corresponding to the target pixel to be corrected and a value obtained by the product-sum operation of the pixel in the vicinity of the output data corresponding to the target pixel and the first coefficient is minimized. the by solving the simultaneous equations, the first coefficient generating means for determining a first coefficient of less information than the amount of information was reduced by converting the number of bits,
Both the output data of at least a second local decoding means and the second pixel data for each class determined above, making a simultaneous equation using the second coefficient is unknown, convert from N bits to the P bits and the second pixel data corresponding to the pixel of interest to, so as to minimize the error of the value obtained by the product sum operation of the pixel and the second coefficient in the vicinity of the output data corresponding to the pixel of interest the by solving the simultaneous equations, and the second coefficient generating means for determining a second coefficient corresponding to the class of small information amount than the amount of information was reduced by converting the number of bits,
The output data of the first and second encoding means, the first coefficient and the second coefficient for each class for reducing the decoding error of the output data of the first and second encoding means, A high-efficiency coding apparatus for digital image signals, comprising transmission means for transmitting a signal. In the high efficiency coding apparatus according to claim 1 0,
The first and second coefficient generating means determine the coefficient in real time. In the encoding device according to claim 1 0,
The value obtained by the product-sum operation of the pixel near the output data corresponding to the target pixel and the first coefficient corresponding to the target pixel in the first coefficient generating means is the spatial and the output data corresponding to the target pixel. / Or is obtained by linear linear combination of a plurality of pixels that are temporally neighboring and the plurality of first coefficients,
The value obtained by the product-sum operation of a plurality of pixels in the output data corresponding to the pixel of interest in the second coefficient generation means and a plurality of pixels that are spatially and / or temporally adjacent to each other, A high-efficiency encoding device, which is obtained by linear first combination of a plurality of pixels in the vicinity of the output data corresponding to the pixel of interest and a plurality of the second coefficients . In a high-efficiency encoding apparatus for digital image signals for reducing the amount of transmission data of digital image signals of P bits for each pixel data,
Separating means for separating a two-dimensional pixel array of the digital image signal into at least first and second pixel data according to a regular pattern;
First encoding means for converting the number of bits in the level direction in each pixel of the first image data into M (≦ P) bits and output data of the first encoding means for converting into P bits First local decoding means,
First error data generating means for generating first error data from the first image data from the separating means and the output data of the first local decoding means;
Second encoding means for converting the number of bits in the level direction in each pixel of the second image data into N (<M) bits and output data of the second encoding means are converted into P bits. A second local decoding means for
Second error data generating means for generating second error data from the second image data from the separating means and the output data of the second local decoding means;
The output data of the first and second encoding means or the output data of the first and second local decoding means are received, and spatially and / or temporally adjacent to each of the plurality of target pixels. Classifying means for determining a class based on the characteristics of a plurality of reference pixels;
For each determined class, together with the first error data from the first error data generation means and the output data of the first local decoding means, a simultaneous equation is established using a first coefficient which is an unknown number. , The first error data corresponding to the target pixel to be converted from M bits to P bits, and a plurality of pixels and a plurality of the spatially and / or temporally neighboring pixels of the output data corresponding to the target pixel. By solving the simultaneous equations so as to minimize an error from an error value created by linear linear combination of the first coefficients, an information amount smaller than the information amount reduced by converting the number of bits First coefficient generating means for obtaining a first coefficient;
For each determined class, together with the second error data from the second error data generating means and the output data of at least the second local decoding means, a simultaneous equation is established using a second coefficient which is an unknown number, The second error data corresponding to the pixel of interest to be converted from N bits to P bits, and the plurality of pixels and the plurality of pixels near the output data corresponding to the pixel of interest spatially and / or temporally. By solving the simultaneous equations so as to minimize an error with an error value generated by linear linear combination of coefficients of 2, the information amount smaller than the information amount reduced by converting the number of bits Second coefficient generating means for obtaining a second coefficient corresponding to the class;
The output data of the first and second encoding means, the first coefficient and the second coefficient for each class for reducing the decoding error of the output data of the first and second encoding means, Transmission means for transmitting
A highly efficient encoding apparatus for digital image signals . In the encoding device according to claim 1 0,
The first coefficient generating means includes:
When finding the first coefficient, further make the simultaneous equations using the output data of said second local decoding means,
The second coefficient generating means is:
Above when calculating the second coefficient, and further, the high-efficiency encoding apparatus, characterized in that make the simultaneous equations using the output data of the first local decoding means. Separating means for separating a two-dimensional pixel array of a digital image signal in which each pixel has a P-bit according to a regular pattern into at least first and second pixel data, and a level at each pixel of the first image data First encoding means for converting the number of bits in the direction into M (≦ P) bits, first local decoding means for converting the output data of the first encoding means into P bits, and Second encoding means for converting the number of bits in the level direction at each pixel of the second image data into N (<M) bits and output data of the second encoding means for converting into P bits Receiving the output data of the second local decoding means and the first and second encoding means or the output data of the first and second local decoding means, And / or temporally based on the characteristics of a plurality of reference pixels in the vicinity, the first classification means for determining a class, the first for each class determined by the first classification means along with the pixel data and the output data of at least said first local decoding means, making a simultaneous equation using the first coefficient is unknown, the first corresponding to the pixel of interest to be converted from M bits to the P bits and pixel data, by solving the simultaneous equations so as to minimize the error between the value determined by the product-sum operation of the pixel and the first coefficient in the vicinity of the output data corresponding to the target pixel, bit a first coefficient generating means for determining a first coefficient of less information than the amount of information was reduced by converting the number, and the second pixel data for each class determined above Both even and output data of said second local decoding means without making a simultaneous equation using the second coefficient is unknown, the second pixel data corresponding to the pixel of interest to be converted from N bits to the P bits The number of bits is converted by solving the simultaneous equations so as to minimize the error between the pixel calculated in the product-sum operation of the output coefficient corresponding to the pixel of interest and the second coefficient. and a second coefficient generating means for determining a second coefficient of less information than the amount of information was reduced by a, and the output data of the first and second encoding means, said first and second A decoding device corresponding to an encoding device comprising transmission means for transmitting the first coefficient and the second coefficient for each class for reducing decoding error of output data of the encoding means,
Receiving means for receiving the output data of the first and second encoding means and the first and second coefficients;
First decoding means for converting M-bit output data of the first encoding means into P bits;
Second decoding means for converting the N-bit output data of the second encoding means into P bits;
Receiving the output data of the first and second encoding means or the output data of the first and second decoding means, and a plurality of reference pixels spatially and / or temporally adjacent to the target pixel A second class classification means for determining a class based on the characteristics of:
A product-sum operation is performed on at least the output data of the first decoding means and the first coefficient corresponding to the class determined by the second class classification means, and an estimated value of the first image data is created. First product-sum operation means;
A product-sum operation is performed on at least the output data of the second decoding means and the second coefficient corresponding to the class determined by the second class classification means, and an estimated value of the second image data is created. Second product-sum operation means;
Pixel synthesizing means for synthesizing the estimated value of the first image data and the estimated value of the second image data and outputting them as decoded data;
A high-efficiency code decoding apparatus comprising: In the decoding apparatus according to claim 1 5,
The first product sum operation means,
Receive at least output data of the first decoding means and first coefficient data corresponding to the class determined by the second class classification means, and estimate the first image data by linear linear combination Create
The second product-sum operation means is
Receive at least output data of the second decoding means and second coefficient data corresponding to the class determined by the second class classification means, and estimate the second image data by linear linear combination A decoding apparatus characterized by creating Separating means for separating the two-dimensional pixel array of the digital image signal into at least first and second pixel data according to a regular pattern, and the number of bits in the level direction of each pixel of the first image data is M ( ≦ P) first encoding means for converting into bits, first local decoding means for converting the output data of the first encoding means into P bits, and first from the separating means First error data generating means for generating first error data from image data and output data of the first local decoding means, and the number of bits in the level direction at each pixel of the second image data is N (< M) second encoding means for converting into bits, second local decoding means for converting the output data of the second encoding means into P bits, and from the separating means Second error data generating means for generating second error data from the second image data and the output data of the second local decoding means, and the output data of the first and second encoding means or the first And receiving the output data of the second local decoding means, and for each of the plurality of pixels of interest, determining a class based on characteristics of a plurality of reference pixels that are spatially and / or temporally neighboring The first class classification means, the first error data from the first error data generation means, and at least the output data of the first local decoding means for each of the determined classes, the first number that is unknown A simultaneous equation is established using coefficients, and the first error data corresponding to the pixel of interest to be converted from M bits to P bits and the space of the output data corresponding to the pixel of interest And / or solving the simultaneous equations so as to minimize an error between a plurality of pixels that are temporally adjacent and a linear first-order combination of the plurality of first coefficients, thereby reducing the number of bits. First coefficient generating means for obtaining a first coefficient having an information amount smaller than the information amount reduced by the conversion, and a second error from the second error data generating means for each of the determined classes The second error data corresponding to the pixel of interest to be converted from N bits to P bits by establishing simultaneous equations using the second coefficient that is an unknown number together with the data and at least the output data of the second local decoding means The error between the output data corresponding to the pixel of interest and the error value created by linear linear combination of the plurality of pixels in the spatial and / or temporal vicinity and the plurality of second coefficients is minimized. Second coefficient generating means for obtaining a second coefficient corresponding to the class of information amount smaller than the information amount reduced by converting the number of bits by solving the simultaneous equations as described above, The output data of the first and second encoding means, and the first coefficient and the second coefficient for each class for reducing the decoding error of the output data of the first and second encoding means. A digital image signal encoding device and a decoding device corresponding to the transmission means for transmitting,
Receiving means for receiving the output data of the first and second encoding means and the first and second coefficients;
First decoding means for converting M-bit output data of the first encoding means into P bits;
Second decoding means for converting the N-bit output data of the second encoding means into P bits;
Receiving the output data of the first and second encoding means or the output data of the first and second decoding means, and a plurality of reference pixels spatially and / or temporally adjacent to the target pixel A second class classification means for determining a class based on the characteristics of:
A linear first combination is performed by a product-sum operation of at least the output data of the first decoding unit and the first coefficient corresponding to the class determined by the second class classification unit, and the first image data First product-sum operation means for creating an estimated value ;
First addition means for adding the output data of the first decoding means and the output data of the first product-sum operation means;
A linear first combination is performed by a product-sum operation of at least the output data of the second decoding means and the second coefficient corresponding to the class determined by the second class classification means, and the second image data Second product-sum operation means for creating an estimated value ;
Second addition means for adding the output data of the second decoding means and the output data of the second product-sum operation means;
Pixel combining means for combining the output of the first adding means and the output of the second adding means and outputting as decoded data;
A high-efficiency code decoding apparatus comprising:

Images