JP4300603B2 - Image information conversion apparatus and method - Google Patents

Description

【００４７】
式（３）においてｎは動きクラスタップ数であり、例えばｎ＝６と設定することができる（図９参照）。そして、ｐａｒａｍの値と、予め設定されたしきい値とを比較することによって動きの指標である動きクラス値が決定される。例えば、ｐａｒａｍ≦２の場合には動きクラス値０、２＜ｐａｒａｍ≦４の場合には動きクラス値１、４＜ｐａｒａｍ≦８の場合には動きクラス値２、ｐａｒａｍ＞８の場合には動きクラス値３というように動きクラス値が生成される。動きクラス値０が動きが最小（静止）であり、動きクラス値１、２、３となるに従って動きが大きいものと判断される。なお、動きベクトルを検出し、検出した動きベクトルに基づいて動きクラスを検出しても良い。
【００４８】
動きクラス検出に使用される動きクラスタップの配置の一例を図９に示す。かかる一例においてフィールドＦ／ｏ内のラインデータｙ１およびｙ２を予測する際に使用される動きクラスタップは、フィールドＦ／ｏ内の画素ｎ１，ｎ３，ｎ５，およびフィールドＦ／ｅ内の画素ｎ２，ｎ４，ｎ６，並びにフィールドＦ−１／ｅ内の画素ｍ２，ｍ４，ｍ６，さらに、フィールドＦ−１／ｏ内の画素ｍ１，ｍ３，ｍ５である。ここで、画素ｍ１と画素ｎ１，画素ｍ２と画素ｎ２，‥‥，画素ｍ６と画素ｎ６の垂直方向の位置が一致している。但し、動きクラスタップの配置はこれに限定されるものでは無い。
【００４９】
図７および図８に示したような空間クラスタップ配置を使用して、１ビットＡＤＲＣでクラス分類する場合には、ｙ１，ｙ２について動きクラス値を共通とし、動きクラス値の分類数を４とすると、クラス数はそれぞれ以下のようになる。
【００５０】
ｙ１についてのクラス数 ＝ ２⁵ ×４ ＝ １２８
ｙ２についてのクラス数 ＝ ２⁶ ×４ ＝ ２５６
この場合、合計のクラス数は３８４である。但し、例えば画像情報変換処理系に備えられるメモリー資産（図６中では係数メモリ４１）がこのクラス数に対応するために充分な記憶容量を有しない場合、または、より少ないクラス数で効率的に画像情報変換処理を行う場合等においては、クラスの総数を削減する必要がある。そこで、クラス値変換処理を行うことによってｙ１，ｙ２についてのクラス数を削減する。図１０は、この場合のクラス総数の削減について模式的に示すものである。なお、図１０中の仮予測係数については後述する。
【００５１】
次に、学習、すなわち適切な予測係数を設定する処理について以下に説明する。まず、学習の第１段階である、クラス値変換テーブルの作成に係る処理系の構成の一例を図１１に示す。出力画像信号と同じ信号形式を有する既知の信号（例えば５２５ｐ信号）が間引きフィルタ５１、および正規方程式加算回路６１、６２に供給される。間引きフィルタ５１は、水平方向および垂直方向で画素数がそれぞれ１／２とされ、全体として供給される信号の１／４の画素数を有するＳＤ信号（例えば５２５ｉ信号）を生成する。
【００５２】
かかる処理として、例えば、入力する画像信号について垂直方向の周波数が１／２になるように垂直間引きフィルタによって画素を間引き、さらに、水平方向の周波数が１／２になるように水平間引きフィルタによって画素を間引く等の処理が行われる。間引きフィルタ５１の特性を変えることによって学習の特性を変え、それによって、変換して得られる画像の画質を制御することができる。
【００５３】
ここで、間引きフィルタ５１に対する入力ＳＤ信号の一例としての５２５ｐ信号と、間引きフィルタ５１からの出力画像信号の一例としての５２５ｉ信号との間の画素の空間的関係を図１２に示す。５２５ｐ信号の奇数番目のフィールドでは、偶数番目のラインが間引かれ、また、奇数番目のラインにおいて画素が水平方向に交互に間引かれる。この間引き処理の様子を、図１２では、第１フィールドと第３フィールドについて図示した。一方、５２５ｐ信号の奇数番目のフィールドでは、奇数番目のラインが間引かれ、また、偶数番目のラインにおいて画素が水平方向に交互に間引かれる。この間引き処理の様子を、図１２では、第２フィールドについて図示した。
【００５４】
図１１において、間引きフィルタ５１が生成するインターレス画像信号がタップ選択回路１０１、１１０、１１１および１２０に供給される。タップ選択回路１０１は、後述する正規方程式の計算において使用され得る予測タップを選択し、選択した予測タップを正規方程式加算回路６１、６２に供給する。一方、タップ選択回路１１０、１１１は、それぞれ、ラインデータｙ１，ｙ２についての空間クラスタップを選択し、選択した空間クラスタップをそれぞれ、空間クラス検出回路１１２、１１３に供給する。さらに、タップ選択回路１２０は、動きクラスタップを選択し、選択した動きクラスタップを動きクラス検出回路１２１に供給する。
【００５５】
空間クラス検出回路１１２、１１３は、空間クラスタップのデータをＡＤＲＣによって圧縮し、それぞれ、ｙ１，ｙ２についての空間クラス値を独立に検出する。また、動きクラス検出回路１２１は、動きクラス値を検出する。空間クラス検出回路１１２の出力と動きクラス検出回路１２１の出力とがクラス合成回路１３０に供給される。また、空間クラス検出回路１１３の出力と動きクラス検出回路１２１の出力とがクラス合成回路１３１に供給される。クラス合成回路１３０、１３１は、それぞれｙ１，ｙ２についての空間クラス値と動きクラス値とを合成する。クラス合成回路１３０、１３１の出力が正規方程式加算回路６１、６２に供給される。
【００５６】
正規方程式加算回路６１は、入力ＳＤ信号、タップ選択回路１０１の出力、およびクラス合成回路１３０の出力に基づいてｙ１の予測生成に係る予測係数を解とする正規方程式に係るデータを得るための加算を行う。同様に、正規方程式加算回路６２は、入力ＳＤ信号、タップ選択回路１０１の出力、およびクラス合成回路１３１の出力に基づいてｙ２の予測生成に係る予測係数を解とする正規方程式に係るデータを得るための加算を行う。
【００５７】
正規方程式加算回路６１、６２は、それぞれ、算出したデータを仮予測係数決定回路６３に供給する。仮予測係数決定回路６３は、供給されるデータに基づき、正規方程式を解く計算処理を行って、ｙ１，ｙ２の予測生成に係る仮予測係数を算出する。そして、算出した仮予測係数を近接係数クラス統合処理部６４に供給する。ここで、仮予測係数と称するのは、最終的な予測係数は、後述するように、クラス統合の結果として決定されるからである。正規方程式を解くための計算処理は、後述する予測係数決定回路２６３（図１３参照）における計算処理と同様である。
【００５８】
近接係数クラス統合処理部６４は、供給される仮予測係数の間の距離ｄを、例えば以下の式（４）に従って計算する。
【００５９】
【数２】

【００６０】
ここで、ｋ_l 、ｋ_l ’は相異なるクラスの予測係数を示す。また、ｌは、予測タップの番号を示す。また、ｎが予測タップの総数を表す。
【００６１】
さらに、近接係数クラス統合処理部６４は、上述したようにして求めたクラス間の予測係数の距離を所定のしきい値と比較し、互いの距離がしきい値より小さくなるような複数個のクラスを１つのクラスに統合する。クラスを統合することによってクラスの総数を減らすことができる。この際に、あるクラスと他の幾つかのクラスとの間の距離が共にしきい値より小さくなる場合には、距離が最小となるクラス同志を統合するようにすれば良い。上述の所定のしきい値を大きく設定する程、クラス数を削減することができる。従って、しきい値を可変することにより、クラス値変換処理によって得られるクラスの総数を調整することができる。
【００６２】
この際に、画像情報変換処理系に備えられるメモリ資産（図６中では係数メモリ４１）の記憶容量等の条件に応じて、ｙ１、ｙ２について的確なクラス数を割り振るようにすることにより、メモリ資産をより有効に使用できる、または、より少ないクラス数で効率的に画像情報変換処理を行うことができる、等の効果を得ることができる。
【００６３】
そして、近接係数クラス統合処理部６４は、クラスの統合に関する情報をクラス値変換テーブル生成部６５に供給する。クラス値変換テーブル生成部６５は、供給される情報に基づいてクラス値変換テーブルを生成する。このクラス値変換テーブルは、上述したクラス値変換回路３２、３３（図６参照）、および後述するクラス値変換回路８１、８２（図１３参照）に供給され、記憶される。そして、これらの構成要素の動作において参照される。
【００６４】
次に、学習の第２段階としての予測係数の算出処理について詳細に説明する。図１３に、予測係数を算出する予測係数算出処理系の構成の一例を示す。出力画像信号と同じ信号形式を有する既知の信号（例えば５２５ｐ信号）が間引きフィルタ２５１、正規方程式加算回路２６１、２６２に供給される。間引きフィルタ２５１は、水平方向および垂直方向で画素数がそれぞれ１／２とされ、全体として供給される信号の１／４の画素数を有するＳＤ信号（例えば５２５ｉ信号）を生成する。
【００６５】
かかる処理として、例えば、入力する画像信号について垂直方向の周波数が１／２になるように垂直間引きフィルタによって画素を間引き、さらに、水平方向の周波数が１／２になるように水平間引きフィルタによって画素を間引く等の処理が行われる。間引きフィルタ５１の特性を変えることによって学習の特性を変え、それによって、変換して得られる画像の画質を制御することができる。間引きフィルタ２５１が生成するＳＤ信号がタップ選択回路２０１、２１０、２１１、２２０に供給される。間引きフィルタ２５１の特性を変えることによって学習の特性を変え、それによって、変換して得られる画像の画質を制御することができる。
【００６６】
タップ選択回路２０１は、予測タップを選択し、選択した予測タップを正規方程式加算回路２６１、２６２に供給する。一方、タップ選択回路２１０、２１１が選択する，ｙ１，ｙ２についての空間クラスタップがそれぞれ空間クラス検出回路２１２、２１３に供給される。さらに、タップ選択回路２２０が選択する予測タップが動きクラス検出回路２２１に供給される。
【００６７】
空間クラス検出回路２１２、２１３は、供給される空間クラスタップに基づいて、それぞれｙ１，ｙ２についての空間クラス値を検出する。また、動きクラス検出回路２２１は、供給される動きクラスタップに基づいて、動きクラス値を検出する。空間クラス検出回路２１２の出力と動きクラス検出回路２２１の出力とがクラス合成回路２３０に供給される。また、空間クラス検出回路２１２の出力と動きクラス検出回路２２１の出力とがクラス合成回路２３１に供給される。クラス合成回路２３０、２３１は、それぞれｙ１，ｙ２についてクラス値を合成し、各クラス値をクラス値変換処理部８１、８２に供給する。
【００６８】
クラス値変換処理部８１、８２は、上述したようにして生成されるクラス値変換テーブルを記憶しており、かかるクラス値変換テーブルを参照して、クラス合成回路２３０、２３１の出力に対してクラス値変換を施す。そして、クラス値変換の結果を、それぞれ、正規方程式加算回路２６１、２６２に供給する。
【００６９】
正規方程式加算回路２６１、２６２は、予測係数を解とする正規方程式を解くための計算処理に使用されるデータを算出する。すなわち、正規方程式加算回路２６１は、入力ＳＤ信号、タップ選択回路２０１の出力、およびクラス値変換回路８１の出力に基づいて加算処理を行うことにより、ｙ１の予測生成に係る予測係数を解とする正規方程式を解くために必要なデータを算出する。また、正規方程式加算回路２６２は、入力ＳＤ信号、タップ選択回路２０１の出力、およびクラス値変換回路８２の出力に基づいて加算処理を行うことにより、ｙ２の予測生成に係る予測係数を解とする正規方程式を解くために必要なデータを算出する。
【００７０】
正規方程式加算回路２６１、２６２が算出したデータが予測係数決定回路２６３に供給される。予測係数決定回路２６３は、供給されるデータに基づいて正規方程式を解くための計算処理を行い、ラインデータｙ１，ｙ２の予測生成に係る予測係数を算出する。算出される予測係数が係数メモリ８４に供給され、記憶される。
【００７１】
予測係数の決定に係る処理、すなわち正規方程式加算回路２６１、２６２、および予測係数決定回路２６３が行う処理について、以下、より詳細に説明する。まず、正規方程式について説明する。上述したように、ｎ個の予測タップを使用して、ラインデータｙ１，ｙ２を構成する各画素を式（１）によって順次予測生成することができる。
【００７２】
式（１）において、学習前は予測係数ｗ₁ ，‥‥，ｗ_n が未定係数である。学習は、クラス毎に複数の信号データに対して行う。信号データの総数をｍと表記する場合、式（１）に従って、以下の式（５）が設定される。
【００７３】
ｙ_k ＝ｗ₁ ×ｘ_k1＋ｗ₂ ×ｘ_k2＋‥‥＋ｗ_n ×ｘ_kn （５）
（ｋ＝１，２，‥‥，ｍ）
ｍ＞ｎの場合、予測係数ｗ₁ ，‥‥，ｗ_n は一意に決まらないので、誤差ベクトルｅの要素ｅ_k を以下の式（６）で定義して、式（７）によって定義される誤差ベクトルｅを最小とするように予測係数を定めるようにする。すなわち、いわゆる最小２乗法によって予測係数を一意に定める。
【００７４】
ｅ_k ＝ｙ_k −｛ｗ₁ ×ｘ_k1＋ｗ₂ ×ｘ_k2＋‥‥＋ｗ_n ×ｘ_kn｝ （６）
（ｋ＝１，２，‥‥ｍ）
【００７５】
【数３】

【００７６】
式（７）のｅ² を最小とする予測係数を求めるための実際的な計算方法としては、ｅ² を予測係数ｗ_i (i=1,2‥‥）で偏微分し（式（８））、ｉの各値について偏微分値が０となるように各予測係数ｗ_i を定めれば良い。
【００７７】
【数４】

【００７８】
式（８）から各予測係数ｗ_i を定める具体的な手順について説明する。式（９）、（１０）のようにＸ_ji，Ｙ_i を定義すると、式（８）は、式（１１）の行列式の形に書くことができる。
【００７９】
【数５】

【００８０】
【数６】

【００８１】
【数７】

【００８２】
式（１１）が一般に正規方程式と呼ばれるものである。正規方程式加算回路２６１、２６２のそれぞれは、クラス値変換回路８１、８２から供給されたクラス値、予測タップ選択回路２０１から供給される予測タップ、および入力画像信号と同じ信号形式を有する既知の画像信号を用いて、正規方程式データ、すなわち、式（９）、（１０）に従うＸ_ji，Ｙ_i の値を算出する。そして、算出した正規方程式データを予測係数決定部４３に供給する。予測係数決定部４３は、正規方程式データに基づいて、掃き出し法等の一般的な行列解法に従って正規方程式を解くための計算処理を行って予測係数ｗ_i を算出する。
【００８３】
なお、以上のように生成される予測係数は、クラス値変換が施されたデータに基づいて算出されるのに対し、図１１を参照して上述した仮予測係数は、クラス値変換が未だ施されていないデータに基づいて算出される。かかる点以外では、両者は正規方程式の解として全く同様な計算処理によって算出される。従って、仮予測係数の決定に係る処理、すなわち図１１中の正規方程式加算回路６１、６２、および予測係数決定回路６３が行う処理も、全く同様なものである。
【００８４】
すなわち、図１１では、クラス合成回路１３０、１３１から供給されるクラス値、予測タップ選択回路１０１から供給される予測タップ、および出力画像信号と同じ信号形式を有する既知の入力画像信号（例えば５２５ｐ信号）を用いて、正規方程式加算回路６１、６２のそれぞれが正規方程式データ、すなわち、式（９）、（１０）に従うＸ_ji，Ｙ_i の値を算出する。そして、算出した正規方程式データを仮予測係数決定部６３に供給する。仮予測係数決定部６３は、正規方程式データに基づいて、掃き出し法等の一般的な行列解法に従って正規方程式を解くための計算処理を行って仮予測係数を算出する。
【００８５】
このように、仮予測係数を算出し、それに基づいてクラス値変換テーブルを生成し、さらに、クラス値変換テーブルを参照してクラス値変換を行いながら予測係数を算出することによって、学習が行なわれる。その結果として、予測係数メモリ８４には、クラス毎に、プログレッシブ画像の注目画素を推定するための、統計的に最も真値に近いと推定ができる予測係数が格納される。予測係数メモリ８４に格納された予測係数は、図６等を参照して上述した画像情報変換処理系内の予測係数メモリ４１にロードされる。
【００８６】
なお、図９示したような画像信号変換処理系、図１１に示したようなクラス値変換テーブルの作成に係る処理系、および図１３に示した予測係数の決定に係る処理系は、各々別個に設けても良いし、スイッチ等を適宜設けることによって構成要素を共通に使用するように構成しても良い。但し、これらの処理系を別個に設ける場合には、学習の有効性を担保するため、同一の機能を有する構成要素は、同一の動作特性を有するものを使用する必要がある。例えばタップ選択回路１、５１、２０１は、同一の動作特性を有する必要がある。構成要素を共通に使用する場合には、スイッチング等に起因して装置全体の処理が複雑化するが、装置全体に関わる回路規模、コスト等の削減に寄与することができる。
【００８７】
また、タップ選択回路２０１が出力する予測タップの個数は、画像情報変換処理系において使用される予測タップの個数より大きいものとされる。従って、図１３中の予測係数決定部２６３では、クラス毎により多くの係数が求まる。このようにして求まった予測係数の中で、絶対値が大きいものから順に使用する数の予測係数が選択される。
【００８８】
次に、線順序変換回路６が行うライン倍速処理について説明する。上述したようにして推定予測演算回路４、５が生成する５２５ｐ信号の水平周期は、画像情報変換処理がなされる前の５２５ｉ信号の水平周期と同一である。線順序変換回路６は、ラインメモリを有し、水平周期を２倍とするライン倍速処理を行う。図１４は、ライン倍速処理をアナログ波形を用いて示すものである。推定予測演算回路４、５によって、ラインデータｙ１およびｙ２が同時に予測生成される。
【００８９】
ラインデータｙ１には、順にａ１，ａ２，ａ３‥‥のラインが含まれ、ラインデータｙ２には、順にｂ１，ｂ２，ｂ３‥‥のラインが含まれる。線順序変換回路５００内には、ラインダブラ（図示せず）とスイッチング回路（図示せず）とが設けられている。ラインダブラは、各ラインのデータを時間軸方向に１／２に圧縮し、圧縮されたデータをスイッチング回路に供給する。スイッチング回路は、供給されるデータを交互に選択して出力する。以上のようにして、線順次出力（ａ０，ｂ０，ａ１，ｂ１，‥‥）が形成される。
【００９０】
なお、上述した構成では、現存ライン上の出力画素値（ラインデータＬ１）と、作成ライン上の出力画素値（ラインデータＬ２）とを並列構成でもって作成するようにしている。これに対し、メモリを追加し、回路の動作速度が速ければ時分割処理でラインデータｙ１およびｙ２を順に生成すると共に、ライン倍速処理を行うように構成しても良い。
【００９１】
また、上述した画像情報変換装置の一例は、５２５ｉ信号を５２５ｐ信号に変換するものであるが、５２５本のライン数は、一例であって、他のライン数であっても良い。例えば、入力ＳＤ信号における水平方向の画素数に対して、２倍以外の画素数を含むラインからなる出力画像信号を予測生成するようにしても良い。より具体的には、出力画像信号として１０５０ｉ信号、すなわち走査線数が１０５０本でインターレス方式の画像信号を予測生成するようにしても良い。図１５は、１フィールドの画像の一部を拡大することによって、５２５ｉ信号と、１０５０ｉ信号とにおける画素の配置を示すものである。大きなドットが５２５ｉ信号の画素を示し、また、小さなドットが１０５０ｉ信号の画素を示す。なお、図１５の実線で示したものは、あるフレームの奇数フィールドの画素配置である。他のフィールド（偶数フィールド）のラインは、点線で示すように空間的に０．５ラインずれたものであり、ラインデータｙ１’，ｙ２’の画素が形成される。
【００９２】
なお、この発明による予測演算装置は、クラス分類適応処理を使用した画像情報変換装置に限らず、複数の画素データの線型１次結合により予測値または補間値を生成する他の装置に対して適用することができる。
【００９３】
【発明の効果】
この発明は、係数データまたはタップデータの一方をゼロデータとすることによって、そのタップに関する演算動作を休止することができ、タップ数の変化に柔軟に対応することができる。また、バイパス信号線、遅延回路を設けずに、積和器をスルーしたタップデータを取り出すことができる。
【００９４】
この発明による予測演算装置を使用した画像情報変換装置は、入力画像信号の走査線との位置関係が異なる複数種類の注目点（具体的には、入力画像信号の走査線上に位置するｙ１と、入力画像信号の走査線の間に位置するｙ２等）の各々について決定されるクラスの数を、注目点の入力画像信号の走査線に対する位置関係に応じて（例えば、予測生成がより困難なｙ２に対してより多くのクラス数を割当てる等）、削減するようにしたものである。
【００９５】
このため、装置内のメモリ資産の記憶容量に応じてクラス数を設定できるので、メモリ資産の記憶容量が小さい場合にも的確な処理を行うことが可能となる。また、より少ないクラス数の下で効率的な処理を行う場合にも、この発明を適用することが有効となる。
【００９６】
また、注目点と入力画像信号の走査線との位置関係の違い等に起因する（具体的にはｙ１とｙ２の間での）予測生成の困難さに応じて、各注目点に対して適切なクラス数が割り当てられる。このため、メモリ資産の記憶容量が限られている場合にも、きめ細かく、また効率的な画像情報変換処理を行うことが可能となる。
【図面の簡単な説明】
【図１】この発明を適用することができる予測器を概略的に示す略線図である。
【図２】この発明の説明の参考に用いる予測器の一例のブロック図である。
【図３】この発明による予測器の一実施形態のブロック図である。
【図４】この発明による予測器の他の実施形態のブロック図である。
【図５】この発明が適用される画像情報変換装置の一例によってなされる画像情報変換処理における画素の配置の一例を示す略線図である。
【図６】この発明が適用される画像情報変換装置の一例における画像情報変換処理系の構成の一例を示すブロック図である。
【図７】この発明が適用される画像情報変換装置の一例における、ラインデータｙ１についての空間クラスタップ配置の一例を示す略線図である。
【図８】この発明が適用される画像情報変換装置の一例における、ラインデータｙ２についての空間クラスタップ配置の一例を示す略線図である。
【図９】この発明が適用される画像情報変換装置の一例における、動きクラスタップ配置の一例を示す略線図である。
【図１０】この発明が適用される画像情報変換装置の一例におけるクラス統合について説明するための略線図である。
【図１１】この発明が適用される画像情報変換装置の一例におけるクラス値変換テーブル生成処理系の構成の一例を示すブロック図である。
【図１２】クラス値変換テーブル生成処理系および予測係数算出処理系において使用される間引きフィルタによる処理について説明するための略線図である。
【図１３】この発明が適用される画像情報変換装置の一例における、予測係数算出処理系の構成の一例を示すブロック図である。
【図１４】ライン倍速処理について説明するための略線図である。
【図１５】この発明が適用される画像情報変換装置の他の例における画素の配置の一例を示す略線図である。
【符号の説明】
１・・・予測タップ選択回路、４，５・・・推定予測演算回路、 ３２、３３・・・クラス値変換回路、４１・・・係数メモリ、６４・・・近接係数クラス統合、６５・・・クラス値変換テーブル生成部、８１、８２・・・クラス値変換回路、１８、１９・・・クラス値変換回路、３０１・・・積和器、３０２₁ 〜３０２_n ・・・乗算器、３０３・・・加算器、Ｓ１〜Ｓｎ・・・セレクタ、Ｇ１〜Ｇｎ・・・アンド回路[0001]
BACKGROUND OF THE INVENTION
  The present invention performs a linear primary combination prediction calculation of a plurality of pixel data and a plurality of coefficient data.imageThe present invention relates to an information conversion apparatus and method.
[0002]
[Prior art]
As one of image signal conversion devices for converting an input digital image signal into a different scanning line structure, a pixel value of an output image signal is predicted by linear primary combination of coefficient data obtained in advance and pixel data of the input image signal What to do has been proposed. A predictor for performing the prediction calculation includes a multiplier that multiplies each pixel data and each coefficient data, and an adder that adds multiplication outputs. The predictor preferably has a versatile configuration that can be used even when the number of pixel data (number of taps) changes. In addition, in order to determine the quality of the prediction result, image data that does not pass through the predictor may be required as an output.
[0003]
[Problems to be solved by the invention]
In the conventional predictor, an unused tap is prevented from occurring by optimizing the number of taps. Therefore, it has been common that there is no versatility that can accommodate different numbers of taps. In addition, when it is desired to output image data that has passed through the predictor, a signal line that bypasses the predictor is provided, and a delay circuit that compensates for the delay required for the processing of the predictor is provided in the bypass signal path. There is a problem that a delay circuit is required.
[0004]
  Therefore, an object of the present invention is versatile with respect to the number of taps, and does not require a delay circuit to obtain a through output that does not perform an operation.imageAn object of the present invention is to provide an information conversion apparatus and method.
[0005]
[Means for Solving the Problems]
  According to a first aspect of the present invention, there is provided an image information conversion apparatus configured to form a plurality of output image signals having different scanning line structures from an input image signal.
  The same position as the line existing in the input image signalOf output image signalTo generate the first line of pixels,GenerationBe doneFirst data selection means for selecting m first pixels of the input image signal located around the pixels;
  first class determining means for detecting a level distribution of m first pixels and determining a first class from the detected level distribution;
  Position between lines present in the input image signalOf output image signalTo generate the second line of pixels,GenerationBe doneSecond data selecting means for selecting n (n> m) second pixels of the input image signal located around the pixels;
  second class determining means for detecting a level distribution of n second pixels and determining a second class from the detected level distribution;
  First and second coefficient data for presuming the output image signal, which are predetermined for each of the first and second classes, are stored, and the stored first and second coefficient data are stored in the first and second coefficient data. Memory means for outputting first and second coefficient data respectively corresponding to each of the first and second classes;
  Third data selection means for selecting a preset number of third pixels around the pixels to be generated from the input image signal;
  First coefficient dataWhen the first line of pixels is generated by a linear estimation formula that performs a product-sum operation on the first pixel and the third pixel, and there is no third pixel that is product-summed with the first coefficient data, the first coefficient data And the first pixel having at least one of the third pixels as zero dataSignal generating means;
  When the second line of pixels is generated by a linear estimation formula that performs a product-sum operation on the second coefficient data and the third pixel, and there is no third pixel that is subjected to a product-sum operation on the second coefficient data, Second signal generation means for setting at least one of the second coefficient data and the third pixel to zero data;
  First and secondScanning conversion means connected to the signal generation means for converting the converted image into a designated scanning line structure;
  The first and second coefficient data are obtained by learning in advance for each of the first and second classes and stored in the memory means.
  Learning is
  An intermediate image signal having the same number of pixels as the input image signal is formed by thinning out the teacher image signal having the same number of pixels as the output image signal, and a plurality of teacher image signals in the vicinity of the pixels of the intermediate image signal are formed. A student image signal is formed by weighted addition of pixel values at corresponding positions of
  This is a process for obtaining an error between the generated pixel value and the true value of the pixel when the teacher image signal is generated from the student image signal by the product-sum operation.
An image information conversion apparatus.
[0006]
  Claim5The present invention provides an image information conversion method in which a plurality of output image signals having different scanning line structures are formed from an input image signal.
  The same position as the line existing in the input image signalOf output image signalTo generate the first line of pixels,GenerationBe doneA first data selection step of selecting m first pixels of the input image signal located around the pixels;
  a first class determining step of detecting a level distribution of m first pixels and determining a first class from the detected level distribution;
  Position between lines present in the input image signalOf output image signalTo generate the second line of pixels,GenerationBe doneA second data selection step of selecting n (n> m) second pixels of the input image signal located around the pixels;
  a second class determining step of detecting a level distribution of n second pixels and determining a second class from the detected level distribution;
  First and second coefficient data which are determined in advance corresponding to each of the first and second classes and for estimating the output image signal are stored in the memory means, and the stored first and second coefficient data Outputting first and second coefficient data respectively corresponding to the first and second classes, respectively,
  A third data selection step of selecting a preset number of third pixels around the pixels to be generated from the input image signal;
  First coefficient dataWhen the first line of pixels is generated by a linear estimation formula that performs a product-sum operation on the first pixel and the third pixel, and there is no third pixel that is product-summed with the first coefficient data, the first coefficient data And the first pixel having at least one of the third pixels as zero dataA signal generation step;
  When the second line of pixels is generated by a linear estimation formula that performs a product-sum operation on the second coefficient data and the third pixel, and there is no third pixel that is subjected to a product-sum operation on the second coefficient data, A second signal generation step in which at least one of the second coefficient data and the third pixel is zero data;
  First and secondA scan conversion step connected to the signal generation step and for converting the converted image into a specified scan line structure;
  The first and second coefficient data are obtained by learning in advance for each of the first and second classes and stored in the memory means.
  Learning is
  An intermediate image signal having the same number of pixels as the input image signal is formed by thinning out the teacher image signal having the same number of pixels as the output image signal, and a plurality of teacher image signals in the vicinity of the pixels of the intermediate image signal are formed. A student image signal is formed by weighted addition of pixel values at corresponding positions of
  This is a process for obtaining an error between the generated pixel value and the true value of the pixel when the teacher image signal is generated from the student image signal by the product-sum operation.
This is an image information conversion method.
[0007]
When the number of input taps changes and a tap that pauses operation occurs, the predicted tap data or coefficient data relating to that tap is set to zero. Further, the coefficient relating to tap data to be passed is set to 1, and the coefficients relating to other tap data are set to 0. Thereby, desired tap data can be passed through.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 schematically shows the flow of prediction processing to which the present invention can be applied. The central part is a product-sum multiplier 313 that performs an operation represented by a linear primary combination of a prediction tap 311 and a prediction coefficient 312 cut out from the input image signal. Commands are provided to the system to set the system's functions. The encoder indicated by 314 generates an instruction for setting the internal state based on the interpretation of the command and the interpretation result.
[0009]
The n prediction coefficients 312 used in the prediction coefficient group 315 are selected by the prediction coefficient switching instruction 316. Further, n prediction taps 311 used in the filter tap 317 are set by the prediction tap switching instruction 318. Further, the input width of the product-sum 313 is n. The number of taps of prediction taps and the number of prediction coefficients n are converted to m or n (n> m) by the input optimization process 319. The input optimization process 319 is performed according to an instruction based on the interpretation of the command formed in the encoder 314. Prediction calculation is performed by the multiplier / summer 313, and the calculation output is processed by the gain adjustment and limiter 320 to generate a final prediction calculation output.
[0010]
As described above, after the input optimization process 319, if the number of prediction taps 311 and prediction coefficients 312 is equal to the input width n of the product-sum 313, no non-operation taps are generated in the predictor 313. However, in the conversion device that converts the image signal, as can be seen from the embodiments of the image signal conversion device described later, the degree of difficulty in prediction differs depending on the positional relationship between the pixel in the input image signal and the pixel to be predicted. As a result, after the input optimization process 319, the number of prediction taps 311 and prediction coefficients 312 may be m (n> m). On the other hand, in order to reduce the scale of the hardware, it is preferable to share the product-summer 313. Therefore, in the case where the number of taps is m, the operation of the n−m taps of the product-sum 313 stops. Furthermore, there is a case where it is desired to output a pixel value of a predetermined tap that is not multiplied by a prediction coefficient as an output as it is.
[0011]
FIG. 2 shows a configuration example conventionally used in the case of outputting through any tap data without performing a prediction calculation. Reference numeral 301 surrounded by a broken line indicates a sum of products. The product-sum multiplier 301 includes n multipliers 302.₁, 302₂, ..., 302_nAnd the multiplier 302₁~ 302_nAnd an adder 303 that adds the outputs of the two, and has n taps as an input width. Multiplier 302₁~ 302_n, COEF1, COEF2,..., COEF from the registers R11, R12,._nAre supplied respectively. Multiplier 302₁~ 302_n, DT, tap data DT1, DT2,... DT from the registers R21, R22,._nAre supplied respectively.
[0012]
The output of the adder 303 of the multiplier / summer 301 is supplied as one input to the selector 305 via the gain and limiter 304. The output of the selector 306 is supplied to the other input of the selector 305 via the delay circuit 307. The selector 305 selectively outputs one input. The selector 306 has tap data DT1, DT2,... DT from the registers R21, R22,._nIs supplied and one piece of data to be passed through is selected.
[0013]
As shown in FIG. 2, in the configuration in which a signal line that bypasses the sum of products 1 is provided and a delay circuit 307 for a time corresponding to the delay generated in the sum of products 301 is provided, the bypass signal line, the selector 306, and the delay circuit 307 are provided. It is necessary to add, and the circuit scale increases. Further, the input width and the number of taps of the product-sum unit 301 are normally selected to be equal, and no countermeasure is taken when the number of taps is reduced from n to m.
[0014]
One embodiment of the predictive calculator according to the present invention shown in FIG. 3 can solve these problems. Similar to the configuration shown in FIG. 2, the product-sum multiplier 301 includes n multipliers 302.₁~ 302_nAnd the multiplier 302₁~ 302_nAnd an adder 303 that adds the outputs of the two, and has n taps as an input width.
[0015]
Multiplier 302₁~ 302_nThe outputs of selectors S1, S2,..., Sn are supplied as one input. Coefficient data COEF1, COEF2,..., COEF from the registers R11, R12,..., R1n as one input of the selectors S1, S2,._nAre supplied respectively. Zero data is supplied as the other input of the selectors S1 to Sn. The selectors S1 to Sn select coefficient data or zero data in response to the control signals CNT1 to CNTn, respectively. Also, the multiplier 302₁~ 302_n, DT, tap data DT1, DT2,._nAre supplied respectively.
[0016]
An input digital image signal is supplied to a prediction tap selection unit (not shown), and n tap data DT1, DT2,._nAre taken out at the same time. On the other hand, predetermined coefficient data COEF1, COEF2, ..., COEF_nAre read from a memory unit (not shown). Multiplier 302₁~ 302_nAre added by the adder 303, and the product-sum operation output is taken out from the adder 303. The output of the multiplier / summer 301 is output as a prediction calculation output via the gain and limiter 304. The gain and limiter 304 corrects the gain for the calculation result and limits the number of bits of the calculation result, and is controlled by the control signal cnt.
[0017]
The operation of the above-described embodiment of the present invention will be described. First, n tap data DT1, DT2, ..., DT_nAnd n coefficient data COEF1, COEF2, ..., COEF_nLinear combination with (COEF1 xDT1 + COEF2 xDT2 + ... + COEF_n× DT_n), When the prediction calculation output is formed, the selectors S1 to Sn respectively convert the coefficient data into the multiplier 2.₁~ 2_nThe selectors S1 to Sn are controlled by the control signals CNT1 to CNTn.
[0018]
Next, when a prediction calculation output is generated by m (n> m) taps and m coefficient data, among the selectors S1 to Sn, m selectors select coefficient data and the remaining n The selectors S1 to Sn are controlled by the control signals CNT1 to CNTn so that the -m number of selectors select zero data. Zero data is, for example, a ground level. Therefore, the multiplier to which zero data is input does not substantially perform the multiplication operation and can prevent wasteful power consumption. Furthermore, as necessary, the gain and limiter 304 are optimized by the control signal cnt according to the difference in the number of taps.
[0019]
Further, a case will be described in which a through operation is performed in which pixel values of one or more input taps are output as they are instead of the prediction calculation output. For example, tap data DT_nIf you want to output the coefficient data COEF_nIs set to 1, and the selector Sn is controlled by the control signal CNTn so that the selector Sn selects the coefficient data of 1. Thus, multiplier 302_nOutput of data DT_nIt is.
[0020]
The selectors other than the selector Sn are controlled by the control signal so as to select all zero data. Thus, multiplier 302_nAll the outputs of the multipliers other than 0 become zero data, and the adder 303 receives data DT._nIs output. The gain and limiter 304 is controlled by the control signal cnt so that the gain is 1 and the number of input bits is output as it is. In this way, as output, tap data DT_nCan be obtained. Since the data to be passed passes through the product-sum multiplier 301, a delay circuit for time adjustment is not necessary.
[0021]
In the present invention, as shown in FIG. 4, a configuration in which AND circuits G1 to Gn are connected instead of the selectors S1 to Sn is possible. That is, the coefficient data COEF1 to COEFn are supplied as one input of the AND circuits G1 to Gn, and the control signals CNT1 to CNTn are supplied as the other input. When it is desired to prohibit the input of coefficient data to the multiplier, the corresponding one of the control signals CNT1 to CNTn may be set to “0”. Other configurations and operations are the same as those of the predictor shown in FIG.
[0022]
In the configuration shown in FIG. 3, selectors S1 to Sn are used to select coefficient data and zero data, and in the configuration shown in FIG. 4, AND circuits G1 to Gn are used to invalidate coefficient data. is doing. However, no processing is performed on the coefficient data, tap data and zero data may be selected, or the tap data may be set to zero data.
[0023]
The above-described motion determination device according to the present invention can be applied to generation of a motion class in an image signal conversion device. This image signal conversion apparatus receives an SD (Standard Definition) signal and outputs an HD (High Definition) signal. In addition, when generating an HD pixel, an SD pixel in the vicinity of the generated HD pixel is divided into classes, and a prediction coefficient value is obtained by learning for each class, thereby obtaining an HD pixel closer to the true value. Is. FIG. 9 shows an image signal conversion apparatus using such a method.
[0024]
Prior to the description of an example of the image information conversion apparatus, an image information conversion process that is the premise thereof will be described. In this process, a standard resolution digital image signal (hereinafter referred to as an SD signal) is converted into a high resolution image signal (sometimes referred to as an HD signal) and output. As the SD signal at this time, for example, an interlaced image signal (hereinafter referred to as a 525i signal) having 525 lines is used. As the high-resolution image signal, for example, a progressive-type output video signal (hereinafter referred to as a 525p signal) having 525 lines is used. Further, the number of pixels in the horizontal direction in the output image signal is twice the number of pixels in the horizontal direction in the input image signal.
[0025]
In such image information conversion processing, resolution is being improved by class classification adaptation processing according to the proposal of the present applicant. The class classification adaptive processing is different from that in which high resolution signals are formed by conventional interpolation processing. That is, the class classification adaptive processing performs class division according to the three-dimensional (spatio-temporal) distribution of the signal level of the input SD signal, and stores the predicted count value obtained by learning in advance for each class in a predetermined storage unit. This is a process of outputting an optimum estimated value by a calculation based on the prediction formula. With the class classification adaptive processing, it becomes possible to obtain a resolution higher than the resolution of the input SD signal.
[0026]
FIG. 5 shows an example of pixel arrangement in an image information conversion process in which a 525i signal as an input SD signal is converted into a 525p signal as an output image signal by enlarging a part of an image of one field. is there. A large dot indicates a pixel of the 525i signal, and a small dot indicates a pixel of the 525p signal. Here, FIG. 5 shows a pixel arrangement in an odd field of a certain frame. In the other field (even field), the lines of the 525i signal are spatially shifted by 0.5 lines. As can be seen from FIG. 5, the line data y1 (shown as small black dots) at the same position as the line of the 525i signal and the line data y2 (shown as small white dots) between the upper and lower lines of the 525i signal. And 525p signal is predicted and generated.
[0027]
Here, y1 is a point that already exists, whereas y2 is a point generated by a new prediction. Therefore, predicting and generating y2 is more difficult than predicting and generating y1. For this reason, when y2 is predicted and generated, it is necessary to allocate a larger number of classes than when y1 is predicted and generated. In addition, for example, when the memory assets provided in the apparatus do not have a sufficient amount, or when efficient image information conversion processing is performed using a smaller number of classes, the total number of classes should be reduced. Class value conversion needs to be performed. From these viewpoints, the image information conversion apparatus described below is configured to appropriately assign the number of classes so that prediction generation of y1 and y2 can be performed efficiently and finely.
[0028]
Hereinafter, an example of such an image information conversion apparatus will be described with reference to the drawings as appropriate. In this example, image information conversion processing for converting a 525i signal as an input SD signal into a 525p signal as an output image signal is performed. FIG. 6 is a block diagram showing an example of the configuration of the image information conversion processing system. An input SD signal (525i signal) is supplied to the tap selection circuits 1, 10 and 20.
[0029]
The tap selection circuit 1 cuts out an area including a plurality of pixels required for calculation processing (calculation processing according to formula (1) described later) for predicting and estimating the line data y1 and y2, and performs line processing from the cut-out area. SD pixels (hereinafter referred to as prediction taps) necessary for predicting and estimating the data y1 and y2 are selected. The selected prediction tap is supplied to the estimated prediction calculation circuits 4 and 5. Here, as a prediction tap, the thing similar to the space class tap mentioned later can be used. However, in order to improve prediction accuracy, a prediction tap may be selected based on prediction tap position information corresponding to a class.
[0030]
In addition, prediction coefficients necessary for predicting and estimating the line data y1 and y2 are supplied from the coefficient memory 41 (described later) to the estimated prediction calculation circuits 4 and 5. Based on the prediction tap supplied from the tap selection circuit 1 and the prediction coefficient supplied from the coefficient memory 41, the estimated prediction calculation circuits 4 and 5 sequentially predict and generate pixel values y according to the following equation (1).
[0031]
y = w₁X₁+ W₂X₂+ ... + w_nX_n      (1)
Where x₁, ..., x_nIs each prediction tap, w₁, ..., w_nAre each prediction coefficient. That is, Expression (1) is an expression for predicting and generating the pixel value y using n prediction taps. As the column of pixel values y, the estimated prediction calculation circuit 4 predicts and generates line data y1, and the estimated prediction calculation circuit 5 predicts and generates line data y2. Prediction coefficient w₁, ..., w_nAre different for y1 and y2.
[0032]
Here, as will be described later, five prediction taps are used to predict line data y1 (ie, n = 5), and six prediction taps are used to predict line data y2. (Ie n = 6). Thus, since the number of taps is different and the prediction coefficients are also different, FIG. 6 shows two estimated prediction calculation circuits 4 and 5. The prediction arithmetic unit according to the present invention described above can be applied to these estimated prediction arithmetic circuits 4 and 5. Since the prediction calculator according to the present invention can cope with the difference in the number of prediction taps, the hardware configuration can be the same. As described above, since the prediction arithmetic unit according to the present invention has versatility, it is possible to reduce the effort for designing the integrated circuit of the estimated prediction arithmetic circuit.
[0033]
The estimated prediction calculation circuits 4 and 5 supply line data y1 and y2 to the line order conversion circuit 6, respectively. The line order conversion circuit 6 performs line double speed processing on the supplied line data to generate a high resolution signal. This high resolution signal is used as the final output image signal of the image information conversion processing system. Although not shown, an output image signal is supplied to the CRT display. The CRT display has a synchronous system so that an output image signal (525p signal) can be displayed. As the input SD signal, a broadcast signal or a reproduction signal of a reproduction device such as a VTR is supplied. That is, the image information conversion device can be built in a television receiver or the like.
[0034]
On the other hand, the tap selection circuits 10 and 11 select SD pixels (hereinafter referred to as space class taps) necessary to detect the space class for the line data y1 and y2 from the input SD signal, respectively. The tap selection circuit 20 selects an SD pixel (hereinafter referred to as a motion class tap) motion class tap necessary for detecting a motion class from the input SD signal. The outputs of the tap selection circuits 10 and 11 are supplied to the space class detection circuits 12 and 13, respectively, and the output of the tap selection circuit 20 is supplied to the motion class detection circuit 21.
[0035]
The space class detection circuits 12 and 13 detect space class values for y1 and y2, respectively, based on the supplied space class tap. Then, the detected space class value is supplied to the class synthesis circuits 30 and 31, respectively. Further, the motion class detection circuit 21 detects a motion class value based on the supplied motion class tap, and supplies the detected motion class value to the class synthesis circuits 30 and 31.
[0036]
The class synthesis circuits 30 and 31 synthesize the supplied space class value and the motion class value. The outputs of the class synthesis circuits 30 and 31 are supplied to class value conversion circuits 32 and 33, respectively. The class value conversion circuits 32 and 33 perform class value conversion processing as described later on the outputs of the class synthesis circuits 30 and 31, respectively. The total number of classes is reduced by the class value conversion process.
[0037]
The converted class value is supplied to the coefficient memory 41. The coefficient memory 41 stores prediction coefficients determined in advance by learning, which will be described later. Among the stored prediction coefficients, the coefficient specified by the class values supplied from the class value conversion circuits 32 and 33 is estimated prediction calculation circuit 4. 5 is output. In order to enable such an operation, for example, a method of storing a prediction coefficient according to an address specified by a class value obtained by class value conversion may be used.
[0038]
Here, the space class detection will be described in more detail. In general, the spatial class detection circuit detects a spatial pattern of the level distribution of image data based on the level distribution pattern of the spatial class tap, and generates a spatial class value based on the detected spatial pattern. In this case, in order to prevent the number of classes from becoming enormous, processing for compressing 8-bit input pixel data into data having a smaller number of bits is performed for each pixel. As an example of such information compression processing, ADRC (Adaptive Dynamic Range Coding) can be used. Moreover, DPCM (predictive coding), VQ (vector quantization), etc. can also be used as information compression processing.
[0039]
ADRC is an adaptive requantization method originally developed for high-efficiency coding for VTR (Video Tape Recoder), but it can efficiently express local patterns of signal level with a short word length. In this example, ADRC is used to generate codes for spatial classification. ADRC uses the maximum value MAX and the minimum value MIN according to the following (2), where DR is the dynamic range of the space class tap, n is the bit allocation, L is the data level of the pixel of the space class tap, and Q is the requantization code. Are re-quantized by equally dividing the interval between and with a specified bit length.
[0040]
DR = MAX-MIN + 1
Q = {(L−MIN + 0.5) × 2 / DR} (2)
However, {} means a truncation process.
[0041]
As described above, the number of classes for line data y2 is larger than the number of classes for line data y1. This point will be specifically described. First, FIG. 7 shows an example of an arrangement of space class taps used for space class detection for y1. In FIG. 7, when time advances in the horizontal direction (left to right), the pixels in each field are shown side by side in the vertical direction.
[0042]
The illustrated fields are expressed as F-1 / o, F-1 / e, F / o, and F / e in order from the left. F-1 / o is a notation indicating that the field is composed of an odd-numbered (odd) th scanning line of the F-1th frame. Similarly, F-1 / e means "a field made up of even-numbered (even) scan lines of the F-1th frame", and F / o means "odd-numbered scanlines of the Fth frame". "F / e" means "a field consisting of even-numbered scanning lines of the F-th frame".
[0043]
In such an example, a total of five space class taps for the line data y1 in the field F / o, T4, T5 in the field F-1 / e, and T1, T2, T3 in the field F / o. Pixel values are used. However, the arrangement of the space class taps is not limited to this. For example, a plurality of horizontal input pixels may be used as the space class tap.
[0044]
Similarly, FIG. 8 shows an example of the arrangement of space class taps used for space class detection for y2. The notation in FIG. 8 is the same as in FIG. In such an example, T6 in field F-1 / e, and T2, T3, T4, T5 in field F / o, and field F / as the space class tap for line data y2 in field F / o. A total of 6 pixel values for T1 in e are used. However, the arrangement of the space class taps is not limited to this. For example, a plurality of horizontal input pixels may be used as the space class tap. Note that the same prediction taps as the space class taps shown in FIGS. 7 and 8 are used to predict the line data y1 and y2.
[0045]
Next, motion class detection will be described in more detail. The motion class detection circuit 21 calculates the average value param of the inter-frame difference absolute value according to the following equation (3) based on the supplied motion class tap. Then, the motion class value is detected based on the calculated param value.
[0046]
[Expression 1]

[0047]
In Equation (3), n is the number of motion class taps, and can be set to n = 6, for example (see FIG. 9). Then, a motion class value that is a motion index is determined by comparing the value of param with a preset threshold value. For example, motion class value 0 when param ≦ 2, motion class value 1 when 2 <param ≦ 4, motion class value 2 when 4 <param ≦ 8, motion when param> 8 A motion class value is generated, such as class value 3. It is determined that the motion class value 0 is the minimum (still) motion, and the motion class values 1, 2, and 3 increase the motion. Note that a motion vector may be detected, and a motion class may be detected based on the detected motion vector.
[0048]
An example of the arrangement of motion class taps used for motion class detection is shown in FIG. In such an example, the motion class taps used in predicting line data y1 and y2 in field F / o are pixels n1, n3, n5 in field F / o and pixels n2, n in field F / e. n4, n6, and pixels m2, m4, and m6 in the field F-1 / e, and pixels m1, m3, and m5 in the field F-1 / o. Here, the vertical positions of the pixel m1, the pixel n1, the pixel m2, the pixel n2,..., The pixel m6, and the pixel n6 are the same. However, the arrangement of the motion class taps is not limited to this.
[0049]
In the case of class classification by 1-bit ADRC using the space class tap arrangement as shown in FIG. 7 and FIG. 8, the motion class value is common for y1 and y2, and the classification number of the motion class value is 4. Then, the number of classes is as follows.
[0050]
Number of classes for y1 = 2^Five× 4 = 128
Number of classes for y2 = 2⁶× 4 = 256
In this case, the total number of classes is 384. However, for example, when the memory resource (coefficient memory 41 in FIG. 6) provided in the image information conversion processing system does not have a sufficient storage capacity to cope with this number of classes, or efficiently with a smaller number of classes. When performing image information conversion processing, etc., it is necessary to reduce the total number of classes. Therefore, the number of classes for y1 and y2 is reduced by performing class value conversion processing. FIG. 10 schematically shows the reduction of the total number of classes in this case. The temporary prediction coefficient in FIG. 10 will be described later.
[0051]
Next, learning, that is, processing for setting an appropriate prediction coefficient will be described below. First, FIG. 11 shows an example of the configuration of a processing system related to creation of a class value conversion table, which is the first stage of learning. A known signal (for example, a 525p signal) having the same signal format as the output image signal is supplied to the thinning filter 51 and the normal equation adding circuits 61 and 62. The thinning filter 51 generates an SD signal (for example, a 525i signal) having the number of pixels halved in the horizontal direction and the vertical direction and having a ¼ pixel number of the signal supplied as a whole.
[0052]
As such processing, for example, the pixels are thinned out by a vertical thinning filter so that the frequency in the vertical direction of the input image signal is halved, and further, the pixels by the horizontal thinning filter so that the frequency in the horizontal direction is halved. Processing such as thinning out is performed. By changing the characteristic of the thinning filter 51, the characteristic of learning can be changed, and thereby the image quality of the image obtained by conversion can be controlled.
[0053]
Here, FIG. 12 shows a spatial relationship of pixels between a 525p signal as an example of an input SD signal to the decimation filter 51 and a 525i signal as an example of an output image signal from the decimation filter 51. In the odd-numbered field of the 525p signal, the even-numbered lines are thinned out, and the pixels are alternately thinned out in the horizontal direction in the odd-numbered lines. This thinning process is illustrated for the first field and the third field in FIG. On the other hand, in the odd-numbered field of the 525p signal, the odd-numbered lines are thinned out, and the pixels are alternately thinned out in the horizontal direction in the even-numbered lines. This thinning process is illustrated for the second field in FIG.
[0054]
In FIG. 11, the interlaced image signal generated by the thinning filter 51 is supplied to the tap selection circuits 101, 110, 111 and 120. The tap selection circuit 101 selects a prediction tap that can be used in the calculation of a normal equation to be described later, and supplies the selected prediction tap to the normal equation addition circuits 61 and 62. On the other hand, the tap selection circuits 110 and 111 select space class taps for the line data y1 and y2, respectively, and supply the selected space class taps to the space class detection circuits 112 and 113, respectively. Further, the tap selection circuit 120 selects a motion class tap and supplies the selected motion class tap to the motion class detection circuit 121.
[0055]
The space class detection circuits 112 and 113 compress the space class tap data by ADRC, and independently detect the space class values for y1 and y2, respectively. The motion class detection circuit 121 detects a motion class value. The output of the space class detection circuit 112 and the output of the motion class detection circuit 121 are supplied to the class synthesis circuit 130. The output of the space class detection circuit 113 and the output of the motion class detection circuit 121 are supplied to the class synthesis circuit 131. The class synthesis circuits 130 and 131 synthesize the space class value and the motion class value for y1 and y2, respectively. The outputs of the class synthesis circuits 130 and 131 are supplied to the normal equation addition circuits 61 and 62.
[0056]
The normal equation addition circuit 61 is an addition for obtaining data related to a normal equation whose solution is a prediction coefficient related to y1 prediction generation based on the input SD signal, the output of the tap selection circuit 101, and the output of the class synthesis circuit 130. I do. Similarly, the normal equation addition circuit 62 obtains data related to a normal equation having a prediction coefficient related to y2 prediction generation as a solution based on the input SD signal, the output of the tap selection circuit 101, and the output of the class synthesis circuit 131. Addition for
[0057]
The normal equation addition circuits 61 and 62 supply the calculated data to the temporary prediction coefficient determination circuit 63, respectively. The temporary prediction coefficient determination circuit 63 performs a calculation process for solving a normal equation based on the supplied data, and calculates temporary prediction coefficients related to y1 and y2 prediction generation. Then, the calculated temporary prediction coefficient is supplied to the proximity coefficient class integration processing unit 64. Here, the temporary prediction coefficient is referred to because the final prediction coefficient is determined as a result of class integration as will be described later. The calculation process for solving the normal equation is the same as the calculation process in the prediction coefficient determination circuit 263 (see FIG. 13) described later.
[0058]
The proximity coefficient class integration processing unit 64 calculates the distance d between the provisional prediction coefficients supplied, for example, according to the following formula (4).
[0059]
[Expression 2]

[0060]
Where k_l, K_l'Represents the prediction coefficient of different classes. L indicates the number of the prediction tap. N represents the total number of prediction taps.
[0061]
Furthermore, the proximity coefficient class integration processing unit 64 compares the distance of the prediction coefficient between the classes obtained as described above with a predetermined threshold, and a plurality of such that the mutual distance is smaller than the threshold. Merge classes into one class. By integrating classes, the total number of classes can be reduced. At this time, if the distances between a certain class and some other classes are both smaller than the threshold value, the classes having the smallest distance may be integrated. As the predetermined threshold value is set larger, the number of classes can be reduced. Therefore, by changing the threshold value, the total number of classes obtained by the class value conversion process can be adjusted.
[0062]
At this time, by assigning an appropriate number of classes for y1 and y2 according to conditions such as the storage capacity of the memory assets (coefficient memory 41 in FIG. 6) provided in the image information conversion processing system, the memory It is possible to obtain an effect that assets can be used more effectively or image information conversion processing can be performed efficiently with a smaller number of classes.
[0063]
Then, the proximity coefficient class integration processing unit 64 supplies information related to class integration to the class value conversion table generation unit 65. The class value conversion table generation unit 65 generates a class value conversion table based on the supplied information. This class value conversion table is supplied to and stored in the class value conversion circuits 32 and 33 (see FIG. 6) and class value conversion circuits 81 and 82 (see FIG. 13) described later. Reference is then made in the operation of these components.
[0064]
Next, prediction coefficient calculation processing as a second stage of learning will be described in detail. FIG. 13 shows an example of a configuration of a prediction coefficient calculation processing system that calculates a prediction coefficient. A known signal (for example, a 525p signal) having the same signal format as the output image signal is supplied to the thinning filter 251 and the normal equation adding circuits 261 and 262. The thinning filter 251 generates an SD signal (for example, a 525i signal) having the number of pixels halved in the horizontal direction and the vertical direction and having a ¼ pixel number of the signal supplied as a whole.
[0065]
As such processing, for example, the pixels are thinned out by a vertical thinning filter so that the frequency in the vertical direction of the input image signal becomes 1/2, and further, the pixels by the horizontal thinning filter so that the frequency in the horizontal direction becomes 1/2. Processing such as thinning out is performed. By changing the characteristic of the thinning filter 51, the characteristic of learning can be changed, and thereby the image quality of the image obtained by conversion can be controlled. The SD signal generated by the thinning filter 251 is supplied to the tap selection circuits 201, 210, 211, and 220. By changing the characteristic of the thinning filter 251, the characteristic of learning can be changed, and thereby the image quality of the image obtained by conversion can be controlled.
[0066]
The tap selection circuit 201 selects a prediction tap and supplies the selected prediction tap to the normal equation addition circuits 261 and 262. On the other hand, the space class taps for y1 and y2 selected by the tap selection circuits 210 and 211 are supplied to the space class detection circuits 212 and 213, respectively. Further, the prediction tap selected by the tap selection circuit 220 is supplied to the motion class detection circuit 221.
[0067]
The space class detection circuits 212 and 213 detect space class values for y1 and y2, respectively, based on the supplied space class tap. The motion class detection circuit 221 detects a motion class value based on the supplied motion class tap. The output of the space class detection circuit 212 and the output of the motion class detection circuit 221 are supplied to the class synthesis circuit 230. Further, the output of the space class detection circuit 212 and the output of the motion class detection circuit 221 are supplied to the class synthesis circuit 231. The class synthesis circuits 230 and 231 synthesize class values for y1 and y2, respectively, and supply the class values to the class value conversion processing units 81 and 82.
[0068]
The class value conversion processing units 81 and 82 store the class value conversion table generated as described above. With reference to the class value conversion table, the class value conversion processing units 81 and 82 class the output of the class synthesis circuits 230 and 231. Perform value conversion. Then, the result of the class value conversion is supplied to the normal equation adding circuits 261 and 262, respectively.
[0069]
The normal equation addition circuits 261 and 262 calculate data used for calculation processing for solving a normal equation having a prediction coefficient as a solution. That is, the normal equation addition circuit 261 performs an addition process based on the input SD signal, the output of the tap selection circuit 201, and the output of the class value conversion circuit 81, thereby obtaining a prediction coefficient relating to the prediction generation of y1 as a solution. Calculate the data required to solve the normal equation. Further, the normal equation addition circuit 262 performs an addition process based on the input SD signal, the output of the tap selection circuit 201, and the output of the class value conversion circuit 82, thereby obtaining a prediction coefficient related to the prediction generation of y2. Calculate the data required to solve the normal equation.
[0070]
Data calculated by the normal equation addition circuits 261 and 262 is supplied to the prediction coefficient determination circuit 263. The prediction coefficient determination circuit 263 performs a calculation process for solving a normal equation based on the supplied data, and calculates a prediction coefficient related to prediction generation of the line data y1 and y2. The calculated prediction coefficient is supplied to the coefficient memory 84 and stored therein.
[0071]
Processing related to determination of the prediction coefficient, that is, processing performed by the normal equation addition circuits 261 and 262 and the prediction coefficient determination circuit 263 will be described in more detail below. First, the normal equation will be described. As described above, each pixel constituting the line data y1 and y2 can be sequentially predicted and generated by the equation (1) using n prediction taps.
[0072]
In equation (1), the prediction coefficient w before learning₁, ..., w_nIs an undetermined coefficient. Learning is performed on a plurality of signal data for each class. When the total number of signal data is expressed as m, the following equation (5) is set according to equation (1).
[0073]
y_k= W₁X_k1+ W₂X_k2+ ... + w_nX_kn      (5)
(K = 1, 2,..., M)
If m> n, prediction coefficient w₁, ..., w_nIs not uniquely determined, the element e of the error vector e_kIs defined by the following equation (6), and the prediction coefficient is determined so as to minimize the error vector e defined by equation (7). That is, the prediction coefficient is uniquely determined by a so-called least square method.
[0074]
e_k= Y_k-{W₁X_k1+ W₂X_k2+ ... + w_nX_kn} (6)
(K = 1, 2, ... m)
[0075]
[Equation 3]

[0076]
E in equation (7)²As a practical calculation method for obtaining the prediction coefficient that minimizes²Prediction coefficient w_i(i = 1, 2...) is partially differentiated (equation (8)), and each prediction coefficient w is set so that the partial differential value becomes 0 for each value of i._iShould be determined.
[0077]
[Expression 4]

[0078]
From Equation (8), each prediction coefficient w_iA specific procedure for determining the above will be described. X as in equations (9) and (10)_ji, Y_i(8) can be written in the form of the determinant of equation (11).
[0079]
[Equation 5]

[0080]
[Formula 6]

[0081]
[Expression 7]

[0082]
Equation (11) is generally called a normal equation. Each of the normal equation addition circuits 261 and 262 is a known image having the same signal format as the class value supplied from the class value conversion circuits 81 and 82, the prediction tap supplied from the prediction tap selection circuit 201, and the input image signal. Using the signal, normal equation data, ie X according to equations (9), (10)_ji, Y_iIs calculated. Then, the calculated normal equation data is supplied to the prediction coefficient determination unit 43. Based on the normal equation data, the prediction coefficient determination unit 43 performs a calculation process for solving the normal equation in accordance with a general matrix solution method such as a sweep-out method and the prediction coefficient w_iIs calculated.
[0083]
The prediction coefficient generated as described above is calculated based on the data subjected to the class value conversion, whereas the provisional prediction coefficient described above with reference to FIG. 11 has not yet been subjected to the class value conversion. It is calculated based on data that has not been processed. Except for this point, both are calculated by the same calculation process as the solution of the normal equation. Therefore, the processing related to the determination of the temporary prediction coefficient, that is, the processing performed by the normal equation addition circuits 61 and 62 and the prediction coefficient determination circuit 63 in FIG. 11 is exactly the same.
[0084]
That is, in FIG. 11, the class value supplied from the class synthesis circuits 130 and 131, the prediction tap supplied from the prediction tap selection circuit 101, and a known input image signal (for example, a 525p signal) having the same signal format as the output image signal. ), Each of the normal equation adding circuits 61 and 62 is normal equation data, that is, X according to the equations (9) and (10)._ji, Y_iIs calculated. Then, the calculated normal equation data is supplied to the provisional prediction coefficient determination unit 63. The temporary prediction coefficient determination unit 63 calculates a temporary prediction coefficient by performing a calculation process for solving the normal equation according to a general matrix solution method such as a sweep-out method based on the normal equation data.
[0085]
Thus, learning is performed by calculating a temporary prediction coefficient, generating a class value conversion table based on the temporary prediction coefficient, and calculating the prediction coefficient while performing class value conversion with reference to the class value conversion table. . As a result, the prediction coefficient memory 84 stores, for each class, a prediction coefficient that can be estimated to be statistically closest to the true value for estimating the pixel of interest of the progressive image. The prediction coefficient stored in the prediction coefficient memory 84 is loaded into the prediction coefficient memory 41 in the image information conversion processing system described above with reference to FIG.
[0086]
The image signal conversion processing system as shown in FIG. 9, the processing system related to the creation of the class value conversion table as shown in FIG. 11, and the processing system related to the determination of the prediction coefficient shown in FIG. The components may be provided in common, or the components may be used in common by appropriately providing a switch or the like. However, when these processing systems are provided separately, in order to ensure the effectiveness of learning, it is necessary to use constituent elements having the same function as those having the same operation characteristics. For example, the tap selection circuits 1, 51, and 201 need to have the same operation characteristics. When the components are used in common, the processing of the entire apparatus is complicated due to switching or the like, but it can contribute to reduction of the circuit scale, cost, etc. relating to the entire apparatus.
[0087]
The number of prediction taps output from the tap selection circuit 201 is larger than the number of prediction taps used in the image information conversion processing system. Therefore, the prediction coefficient determination unit 263 in FIG. 13 determines more coefficients for each class. Of the prediction coefficients obtained in this way, the number of prediction coefficients to be used is selected in descending order of the absolute value.
[0088]
Next, the line double speed process performed by the line order conversion circuit 6 will be described. As described above, the horizontal period of the 525p signal generated by the estimated prediction calculation circuits 4 and 5 is the same as the horizontal period of the 525i signal before the image information conversion process is performed. The line order conversion circuit 6 has a line memory and performs line double speed processing that doubles the horizontal period. FIG. 14 shows line double speed processing using an analog waveform. Line data y1 and y2 are simultaneously predicted and generated by the estimated prediction calculation circuits 4 and 5.
[0089]
The line data y1 includes lines a1, a2, a3,... In order, and the line data y2 includes lines b1, b2, b3,. In the line order conversion circuit 500, a line doubler (not shown) and a switching circuit (not shown) are provided. The line doubler compresses the data of each line by half in the time axis direction, and supplies the compressed data to the switching circuit. The switching circuit alternately selects and outputs the supplied data. As described above, line sequential outputs (a0, b0, a1, b1,...) Are formed.
[0090]
In the configuration described above, the output pixel value (line data L1) on the existing line and the output pixel value (line data L2) on the creation line are created in a parallel configuration. On the other hand, a memory may be added so that the line data y1 and y2 are generated in order by time division processing and the line double speed processing is performed if the operation speed of the circuit is high.
[0091]
Moreover, although an example of the image information conversion apparatus described above converts a 525i signal into a 525p signal, the number of 525 lines is an example and may be another number of lines. For example, an output image signal including a line including a number of pixels other than twice the number of pixels in the horizontal direction in the input SD signal may be predicted and generated. More specifically, a 1050i signal, that is, an interlaced image signal having 1050 scanning lines may be predicted and generated as the output image signal. FIG. 15 shows the pixel arrangement in the 525i signal and the 1050i signal by enlarging a part of the image of one field. A large dot indicates a pixel of a 525i signal, and a small dot indicates a pixel of a 1050i signal. Note that what is indicated by a solid line in FIG. 15 is a pixel arrangement in an odd field of a certain frame. The other field (even field) lines are spatially shifted by 0.5 lines as shown by dotted lines, and pixels of line data y1 'and y2' are formed.
[0092]
Note that the prediction calculation device according to the present invention is not limited to the image information conversion device using the class classification adaptive process, but is applied to other devices that generate a prediction value or an interpolation value by linear linear combination of a plurality of pixel data. can do.
[0093]
【The invention's effect】
In the present invention, by setting one of the coefficient data or the tap data to zero data, the arithmetic operation relating to the tap can be paused, and the change in the number of taps can be flexibly dealt with. In addition, tap data that has passed through the accumulator can be taken out without providing a bypass signal line and a delay circuit.
[0094]
The image information conversion apparatus using the prediction arithmetic apparatus according to the present invention has a plurality of types of attention points (specifically, y1 positioned on the scanning line of the input image signal, and different positional relationships with the scanning line of the input image signal, The number of classes determined for each of the input image signal scan lines (such as y2 positioned between the scan lines of the input image signal) is determined according to the positional relationship of the point of interest with respect to the scan line of the input image signal (for example, y2 is more difficult to predict and generate For example, a larger number of classes are allocated to the above and the like).
[0095]
For this reason, since the number of classes can be set according to the storage capacity of the memory asset in the apparatus, accurate processing can be performed even when the storage capacity of the memory asset is small. Moreover, it is effective to apply the present invention also when efficient processing is performed under a smaller number of classes.
[0096]
Further, depending on the difficulty of predictive generation (specifically, between y1 and y2) due to the difference in the positional relationship between the target point and the scanning line of the input image signal, it is appropriate for each target point. The appropriate number of classes. For this reason, even when the storage capacity of the memory asset is limited, it is possible to perform fine and efficient image information conversion processing.
[Brief description of the drawings]
FIG. 1 is a schematic diagram schematically showing a predictor to which the present invention can be applied.
FIG. 2 is a block diagram of an example of a predictor used for reference in the description of the present invention.
FIG. 3 is a block diagram of an embodiment of a predictor according to the present invention.
FIG. 4 is a block diagram of another embodiment of a predictor according to the present invention.
FIG. 5 is a schematic diagram illustrating an example of pixel arrangement in an image information conversion process performed by an example of an image information conversion apparatus to which the present invention is applied;
FIG. 6 is a block diagram showing an example of the configuration of an image information conversion processing system in an example of an image information conversion apparatus to which the present invention is applied.
FIG. 7 is a schematic diagram illustrating an example of a space class tap arrangement for line data y1 in an example of an image information conversion apparatus to which the present invention is applied.
FIG. 8 is a schematic diagram illustrating an example of a space class tap arrangement for line data y2 in an example of an image information conversion apparatus to which the present invention is applied;
FIG. 9 is a schematic diagram illustrating an example of motion class tap arrangement in an example of an image information conversion apparatus to which the present invention is applied;
FIG. 10 is a schematic diagram for explaining class integration in an example of an image information conversion apparatus to which the present invention is applied;
FIG. 11 is a block diagram showing an example of a configuration of a class value conversion table generation processing system in an example of an image information conversion apparatus to which the present invention is applied.
FIG. 12 is a schematic diagram for explaining processing by a thinning filter used in a class value conversion table generation processing system and a prediction coefficient calculation processing system;
FIG. 13 is a block diagram illustrating an example of a configuration of a prediction coefficient calculation processing system in an example of an image information conversion apparatus to which the present invention is applied.
FIG. 14 is a schematic diagram for explaining line double speed processing;
FIG. 15 is a schematic diagram illustrating an example of pixel arrangement in another example of the image information conversion apparatus to which the present invention is applied;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Prediction tap selection circuit, 4, 5 ... Prediction prediction calculation circuit, 32, 33 ... Class value conversion circuit, 41 ... Coefficient memory, 64 ... Proximity coefficient class integration, 65 ... Class value conversion table generation unit, 81, 82... Class value conversion circuit, 18, 19... Class value conversion circuit, 301.₁~ 302_n... Multiplier, 303 ... Adder, S1-Sn ... Selector, G1-Gn ... AND circuit

Claims (5)

In an image information conversion apparatus configured to form a plurality of output image signals having different scanning line structures from an input image signal,
To generate the first pixel of the line of the input image signal an output image signal having the same position and a line that is present in the first pixel of the input image signal located around the pixel to be the generated m First data selection means for selecting the number;
First class determining means for detecting a level distribution of the m first pixels and determining a first class from the detected level distribution;
A second pixel of the input image signal located around the generated pixel in order to generate a pixel of the second line of the output image signal that is a position between lines present in the input image signal; Second data selection means for selecting n (n> m),
Second class determining means for detecting a level distribution of the n second pixels and determining a second class from the detected level distribution;
First and second coefficient data for presuming the output image signal, stored in advance corresponding to each of the first and second classes, and the stored first and second coefficient data Memory means for outputting first and second coefficient data respectively corresponding to each of the first and second classes,
Third data selection means for selecting a preset number of third pixels around the pixels to be generated from the input image signal;
The pixels of the first line are generated by a linear estimation expression that performs a product-sum operation on the first coefficient data and the third pixel, and the product-sum operation is performed on the third coefficient. If there is no pixel, the first signal generating means that uses at least one of the first coefficient data and the third pixel as zero data ;
The pixel of the second line is generated by a linear estimation formula that performs a product-sum operation on the second coefficient data and the third pixel, and the product-sum operation is performed on the second coefficient data and the third pixel. If there is no pixel, the second signal generating means that uses at least one of the second coefficient data and the third pixel as zero data,
Scanning conversion means connected to the first and second signal generation means for converting the converted image into a designated scanning line structure;
The first and second coefficient data are obtained by learning in advance for each of the first and second classes and stored in the memory means.
The above learning
By thinning out the teacher image signal having the same number of pixels as the output image signal, an intermediate image signal having the same number of pixels as the input image signal is formed, and the teacher near the pixels of the intermediate image signal is formed. A student image signal is formed by weighted addition of pixel values at corresponding positions of a plurality of pixels of the image signal,
An image information conversion apparatus, which is a process for obtaining an error between a generated pixel value and a true value of a pixel when the teacher image signal is generated from the student image signal by the product-sum operation. A fourth data selection means for selecting a plurality of fourth pixels from a plurality of fields of the input image signal located around the pixel to be generated of the output image signal;
Motion class determining means for detecting a level distribution of the plurality of fourth pixels and determining a motion class from the detected level distribution;
First class integration means for integrating the first class and the motion class to generate a first integrated class;
A second class integration unit that integrates the second class and the motion class and generates a second integrated class;
2. The image information conversion apparatus according to claim 1, wherein first and second coefficient data corresponding to each of the first and second integrated classes are output from the memory means. In claim 1,
An image information conversion apparatus, wherein a progressive output image signal is formed from an interlace input image signal. In claim 1,
Furthermore, the image information conversion apparatus characterized by generating the output image signal having the number of pixels twice as large as the input image signal in the horizontal direction. In an image information conversion method for forming a plurality of output image signals having different scanning line structures from an input image signal,
To generate the first pixel of the line of the input image signal an output image signal having the same position and a line that is present in the first pixel of the input image signal located around the pixel to be the generated m A first data selection step for selecting the number;
A first class determining step of detecting a level distribution of the m first pixels and determining a first class from the detected level distribution;
A second pixel of the input image signal located around the generated pixel in order to generate a pixel of the second line of the output image signal that is a position between lines present in the input image signal; A second data selection step of selecting n (n> m),
A second class determining step of detecting a level distribution of the n second pixels and determining a second class from the detected level distribution;
First and second coefficient data, which are predetermined for each of the first and second classes and for estimating the output image signal, are stored in the memory means, and the stored first and second classes are stored. Outputting first and second coefficient data respectively corresponding to the first and second classes from among the coefficient data of
A third data selection step of selecting a preset number of third pixels around the pixels to be generated from the input image signal;
The pixels of the first line are generated by a linear estimation expression that performs a product-sum operation on the first coefficient data and the third pixel, and the product-sum operation is performed on the third coefficient. When there is no pixel, a first signal generation step in which at least one of the first coefficient data and the third pixel is zero data ;
The pixel of the second line is generated by a linear estimation formula that performs a product-sum operation on the second coefficient data and the third pixel, and the product-sum operation is performed on the second coefficient data and the third pixel. When there is no pixel, a second signal generation step in which at least one of the second coefficient data and the third pixel is zero data;
A scan conversion step connected to the first and second signal generation steps for converting the converted image into a specified scan line structure;
The first and second coefficient data are obtained by learning in advance for each of the first and second classes and stored in the memory means,
The above learning
By thinning out the teacher image signal having the same number of pixels as the output image signal, an intermediate image signal having the same number of pixels as the input image signal is formed, and the teacher near the pixels of the intermediate image signal is formed. A student image signal is formed by weighted addition of pixel values at corresponding positions of a plurality of pixels of the image signal,
An image information conversion method, which is a process for obtaining an error between a generated pixel value and a true value of a pixel when the teacher image signal is generated from the student image signal by the product-sum operation.

Images