JP3630246B2 - Image signal interpolation apparatus and image signal interpolation method - Google Patents

Description

により、図５（Ａ）に示すようにブロツク内の最大値ＭＡＸと最小値ＭＩＮとの間を指定されたビツト長で均等に分割して再量子化を行う。因に、図５は３ビツトで再量子化した場合（すなわちｐ＝３の場合）を表わす。
【００２２】
次に、図５（Ａ）の最上位の階調レベル（２^ｐ −１）に相当するデータレベル内に存在するブロツク内画素の平均値をとり、これを図５（Ｂ）に示すように最大値ＭＡＸ′とする。また図５（Ａ）の最下位の階調レベル０に相当するデータレベル内に存在するブロツク内画素の平均値をとり、これを図５（Ｂ）に示すように最小値ＭＩＮ′とする。
【００２３】
次に新しく求めた最大値ＭＡＸ′及び最小値ＭＩＮ′によりブロツク内ダイナミツクレンジＤＲ′を新たに定義し直して、再量子化コードをｑとして、新しく求めたブロツク内の最大値ＭＡＸ′及び最小値ＭＩＮ′に基づいて、次式
【数２】

により図５（Ｂ）に示すような再量子化を行う。
このようにＡＤＲＣ回路４は二重の再量子化を行うことにより、ノイズによる悪影響を低減して効率の良い情報量圧縮を行いパターン圧縮データＤ５を形成する。
【００２４】
クラスコード発生回路６は、このようにＡＤＲＣ回路４によつてｐビツトにデータ圧縮された結果の再量子化データをｑ_１ 〜ｑ_ｎ として、次式
【数３】

を実行することにより、そのブロツクの属するクラスすなわち補間対象画素のクラスを表わすクラスコードｃ（Ｄ６）を決定する。
【００２５】
（３）予測係数メモリの作成
次に予測係数メモリ６に記憶させる予測係数の求め方を、図６に従つて説明する。先ずステツプＳＰ１において予測係数を学習するために、既に知られている画像に対応した学習データを形成する。具体的には、図２におけるサンプリング画素ａ〜ｌと、補間対象画素（×印）のサブサンプル以前の画素値を一組の学習データとする。
なお、このように学習データを形成する際に、１つの画像のみを用いるのではなく複数の画像を用いることで非常に多数の学習データを形成すれば、より正確な予測係数を得ることができる。
【００２６】
ステツプＳＰ２では必要上十分な学習データが形成されたか否か判定する。そして更に学習データが必要であると判定した場合にはステツプＳＰ３に進み、十分な学習データが得られたと判定した場合にはステツプＳＰ４に移る。
ステツプＳＰ３では学習データをクラス分類する。このとき上述した画像信号補間装置１でしたのと同様のクラス分類を行うようにする。具体的には、先ずハイパスフイルタによつて学習サンプリングデータの高域信号を抽出し、当該抽出した学習サンプリングデータの高域信号をＡＤＲＣ符号化により圧縮した後にクラスコードを形成することにより各学習データをクラス分類する。
【００２７】
次にステツプＳＰ５において、クラス分類された学習データに基づき、各クラス毎に正規化方程式を形成する。ステツプＳＰ５での処理を具体的に説明する。ここでは一般化するために学習データとしてｎ個のサンプリング画素が存在する場合について述べる。先ず各サンプリング画素の画素レベルｘ_１ 、……、ｘ_ｎ と注目補間画素のサブサンプル以前の画素レベルｙの関係を、クラス（ｃ）毎に予測係数ｗ_１ 、……、ｗ_ｎ によるｎタツプの線型一次予測式で表現することにより、次式
【数４】

を形成する。
【００２８】
この（４）式における予測係数ｗ_１ 、……、ｗ_ｎ を求めれば良い。そこで実際の補間対象画素と補間処理結果の誤差が最小になるような予測係数ｗ_１ 、……、ｗ_ｎ を求める。ここで学習はクラス毎に複数の学習データに対して行うので、学習データ数がｍとすると一般的なｍ＞ｎである場合には予測係数ｗ_１ 、……、ｗ_ｎ は一意に決定できない。そこで誤差ベクトルｅの要素を、それぞれの学習データｘ_ｋ１、……、ｘ_ｋｎ、ｙ_ｋ （ｋ＝１、２、……、ｍ）における予測誤差をｅ_ｋ として、次式
【数５】

のように定義して、次式
【数６】

を最小にする予測係数ｗ_１ 、……、ｗ_ｎ を求める。いわゆる最小二乗法による解法である。
【００２９】
ここで（６）式のｗ_ｉ による偏微分係数を求めると、次式
【数７】

となる。（７）式が０になるような各ｗ_ｉ を決めればよい。そこで次式
【数８】

及び
【数９】

のように、Ｘ_ｉｊ、Ｙ_ｉ を定義すると、上述した（７）式は行列を用いて、次式
【数１０】

の正規化方程式に書き換えることができる。
【００３０】
ここで（１０）式の正規化方程式は未知数がｎ個の連立方程式であるから、これにより最確値である各未定係数ｗ_１ 、……、ｗ_ｎ を求めることができる。すなわちこの予測係数算出処理手順では、ステツプＳＰ５において各クラス毎に未定係数ｗ_１ 、……、ｗ_ｎ を求めることができるような正規化方程式を形成できるまでステツプＳＰ１−ＳＰ２−ＳＰ３−ＳＰ５−ＳＰ１のループを繰り返す。
【００３１】
やがてステツプＳＰ５において各クラス毎に（１０）式で表わされる正規化方程式が形成され、ステツプＳＰ２において肯定結果が得られると、ステツプＳＰ４に進んで、ここで（１０）式の正規化方程式を解いて各クラス毎の予測係数ｗ_１ 、……、ｗ_ｎ を決定する。具体的には、一般に（１０）式の左辺の行列は正定値対称なので、コレスキー法により解くことができる。
次にステツプＳＰ６において、各クラス毎に決定された予測係数ｗ_１ 、……、ｗ_ｎ を予測係数メモリ６の対応するクラスのアドレスに格納し、続くステツプＳＰ７において当該予測係数算出処理手順を終了する。
【００３２】
（４）実施例の動作
以上の構成において、画像信号補間装置１は間引かれた画素を補間対象画素として、当該補間対象画素をその周辺のブロツク化画像データＤ３の状態に応じてクラス分類する。
この場合、画像信号補間装置１はクラス分類の前処理として、ハイパスフイルタ回路３で補間対象画素の周辺画素から高域信号Ｄ４を抽出する。そして画像信号補間装置１はこの高域信号Ｄ４に対して適応ダイナミツクレンジ符号化処理を施してパターン圧縮データＤ５を形成し、このパターン圧縮データＤ５を用いて補間対象画素のクラス分類を示すクラスコードＤ６を形成する。
このように画像信号補間装置１では、ブロツク化画像データＤ３から緩やかな変化を取り除き、特徴的な変化だけを取り出してクラス分類するようにしたことにより、不必要なクラス数の増加を抑制し得、効率の良いクラス分類ができるようになる。
【００３３】
次に、画像信号補間装置１はクラスコードＤ６を読み出しアドレスとして、予測係数メモリ６に記憶された予測係数Ｄ７を読み出す。そして予測演算回路７によつてこのクラスｃに対応した予測係数ｗ_１ （ｃ）〜ｗ_１２（ｃ）とブロツク化画像データＤ３に含まれるサンプリング画素ａ〜ｌの画素データｘ_１ 〜ｘ_１２とを、次式
【数１１】

のように線形一次結合することにより、補間対象画素に対応する補間値ｙ′を算出し、これを補間データＤ２として出力する。
【００３４】
このようにして画像信号補間装置１においては、原画に含まれる画素データとほとんど変わらない補間データＤ２を形成することができる。この補間データＤ２は合成回路（図示せず）により入力画像データＤ１と合成された後、例えばテレビジヨン受像装置やビデオテープレコーダ装置等に供給される。
【００３５】
（５）実施例の効果
以上の構成によれば、補間対象画素の周辺画素から高域信号Ｄ４を抽出し、当該高域信号Ｄ４を用いて補間対象画素をクラス分類したことにより、補間対象画素を少ないクラス数で的確にクラス分類できる。これにより真値に近い補間データを形成し得る簡易な構成の画像信号補間装置１を実現できる。
【００３６】
（６）他の実施例
なお上述の実施例においては、補間対象画素をクラス分類する場合、周辺画素の高域信号Ｄ４をＡＤＲＣ回路４及びクラスコード発生回路５を用いてビツト圧縮することによりクラス分類した場合について述べたが、クラス分類手段としてはこれに限らず、例えば離散コサイン変換（ＤＣＴ）、差分量子化（ＤＰＣＭ）、サブバンド符号化やウエーブレツト変換等の種々の圧縮手段を適用し得る。さらにクラス分類の方法としてはビツト圧縮によるものに限らず、例えば補間対象画素の周辺画素において相関性の強い方向を検出し、当該検出結果に基づいてクラス分類するようにしても良い。
【００３７】
また上述の実施例においては、クラス毎の予測係数を最小二乗法による学習によつて求めた場合について述べたが、予測係数の求め方はこれに限らず、種々の学習方法を用いることができる。
【００３８】
さらに上述の実施例においては、予測係数メモリ６及び予測演算回路７を設け、予め学習によつて予測係数メモリ６に記憶された予測係数Ｄ７をクラスコードＤ６に応じて読み出し、読み出した予測係数Ｄ７とブロツク化画像データＤ３とを線形一次結合することにより補間データＤ２を求めるようにした場合について述べたが、本発明はこれに限らず、予測係数メモリ６及び予測演算回路７に代えて、予め学習によつて求めたクラス毎の代表値を格納するメモリを設け、クラスに応じた代表値を読み出して、当該読み出した代表値を補間データとするようにしても良い。
【００３９】
この場合、メモリに格納する代表値を求める第１の方法としては、加重平均による学習がある。詳述すれば、補間対象画素に対応する真の画素値をクラス毎に積算し、この積算結果を積算した画素値の個数によつて割るといつた処理を様々な画像に対して行うことによりクラス毎の代表値を得るものである。
また代表値を求める第２の方法としては、正規化による学習がある。詳述すれば、補間対象画素を含む複数の画素からなるブロツクを形成し、当該ブロツク内のダイナミツクレンジによつて、補間対象画素に対応する真の画素値からブロツクの基準値を減算した値を正規化し、この正規化した値の累積値を累積度数で割るといつた処理を様々な画像に対して行うことによりクラス毎の代表値を得るものである。
【００４０】
【発明の効果】
上述のように本発明によれば、伝送画像データを所定の大きさのブロツクに分割する画像信号ブロツク化手段と、ブロツク内画素の高域信号を抽出するハイパスフイルタ手段と、高域信号に基づいて補間対象画素をクラス分類するクラス分類手段と、分類されたクラスに対応する予測係数を発生する予測係数発生手段と、予測係数と伝送画像データとを用いて予測演算することにより、補間対象画素に対応する補間データを算出する補間データ算出手段とを設けるようにしたことにより、折り返し歪みの発生を未然に回避して真値に近い補間画素値を求めることができる。
【００４１】
また本発明によれば、ハイパスフイルタ手段によつてブロツク内画素の高域信号を抽出し、当該高域信号に基づいて補間対象画素をクラス分類するようにしたことにより、補間対象画素を少ないクラス数で的確に分類できるようになり、その結果、構成を簡易化できる。
【図面の簡単な説明】
【図１】本発明の一実施例による画像信号補間装置の構成を示すブロツク図である。
【図２】クラス分類処理に用いる周辺画素の説明に供する略線図である。
【図３】ハイパスフイルタ回路の係数を示す略線図である。
【図４】ＡＤＲＣ回路によるビツト圧縮の説明に供する略線図である。
【図５】ＡＤＲＣ回路によるビツト圧縮の説明に供する略線図である。
【図６】学習による予測係数算出処理手順を示すフローチヤートである。
【図７】オフセツトサブサンプリングの説明に供する略線図である。
【図８】２次元のオフセツトサブサンプリングにより伝送可能な帯域の空間周波数スペクトラムを示す略線図である。
【図９】補間処理の説明に供する略線図である。
【符号の説明】
１……画像信号補間装置、２……ブロツク化回路、３……ハイパスフイルタ回路、４……ＡＤＲＣ回路、５……クラスコード発生回路、６……予測係数メモリ、７……予測演算回路。[0001]
【table of contents】
The present invention will be described in the following order.
Industrial application field Conventional technology (FIGS. 7 to 9)
Means for Solving the Problems to be Solved by the Invention (FIGS. 1 and 2)
Action (Fig. 1)
Example (1) Overall configuration (FIGS. 1 to 3)
(2) Class classification processing (FIGS. 4 and 5)
(3) Creation of prediction coefficient memory (FIG. 6)
(4) Operation of the embodiment (FIG. 1)
(5) Effects of the embodiments (6) Effects of other embodiments of the invention
[Industrial application fields]
The present invention relates to an image signal interpolation apparatus and an image signal interpolation method, and is suitable for application to an image signal interpolation apparatus and an image signal interpolation method for improving the resolution of an image by interpolating pixels thinned out by subsampling, for example. It is a thing.
[0003]
[Prior art]
Conventionally, a method of thinning out pixels of an original image at predetermined intervals by sub-sampling is widely used as a method for band compression or information reduction when recording and transmitting an image signal. As an example, there is a multiple sub Nyquist sampling encoding method in the MUSE (Multiple Sub-nyquist Sampling Encoding) method.
[0004]
As an example of subsampling, offset subsampling is widely used. In this offset sub-sampling, in the two-dimensional case, as shown in FIG. 7, the sampling interval (Tx, Ty) in the horizontal direction (x direction) and the vertical direction (y direction) is set to the pixel interval (Hx in the original signal). , Hy), sub-sampling (×) is performed every other pixel. In offset subsampling, sampling points (（) adjacent in the vertical direction are offset by half of the sampling interval (Tx / 2). As a result, the transmission band of the image signal after offset sub-sampling can widen the spatial frequency component in the horizontal or vertical direction with respect to the spatial frequency in the oblique direction as shown in FIG. It is possible to perform a thinning process in which deterioration is not noticeable.
[0005]
Here, when the offset subsampled image signal is displayed on a monitor or printed out, it is necessary to interpolate pixels between sampling points using adjacent pixels as shown in FIG. Such an interpolation process functions as a spatial filter that passes the frequency components in the hatched area shown in FIG. 8 and prevents the frequency components in the area including the turning point A from passing. Above it is positioned as a post filter.
[0006]
[Problems to be solved by the invention]
By the way, offset subsampling is a very effective method when the prefilter before subsampling is applied correctly, but the prefilter cannot be applied sufficiently due to hardware restrictions, for example. In some cases, or when the pre-filter is not sufficiently applied in order to increase the transmission band, there is a problem that image degradation based on aliasing distortion occurs.
[0007]
As one method for reducing the occurrence of aliasing distortion, an adaptive interpolation method has been proposed. In this method, when interpolation processing is performed on a subsampled image signal, a direction having a strong correlation is detected around the interpolation pixel, and a plurality of different interpolation means are selectively used according to the detection result. Interpolation processing is performed.
[0008]
By the way, in the adaptive interpolation method, the interpolation accuracy largely depends on the detection accuracy when detecting a direction having a strong correlation and the ability of each interpolation means. Therefore, when the ability of each interpolation means is not sufficient and proper interpolation is not possible, or when a direction with strong correlation is misjudged, not only the original signal component is reduced but also the aliasing distortion is increased. There was a problem.
[0009]
The present invention has been made in consideration of the above points, and an image signal interpolation apparatus and an image signal interpolation method having a simple configuration capable of obtaining an interpolation pixel value close to a true value by avoiding occurrence of aliasing distortion in advance. Is to try to propose.
[0010]
[Means for Solving the Problems]
In order to solve this problem, in the present invention, transmission image data (D1) in which predetermined pixels are thinned out is input, and transmission image data (1) is interpolated in the thinned out pixels. The image signal blocking means (2) for dividing D1) into blocks of a predetermined size centered on the interpolation target pixel (marked with x in FIG. 2), and the high frequencies of the pixels in the block (a to l in FIG. 2) The high-pass filter means (3) for extracting the signal (D4) and the level distribution pattern of the pixels (a to l) in the block are detected based on the high-frequency signal (D4), and the pixel to be interpolated according to the detection result Class classification means (4 and 5) for classifying the data, prediction coefficient generation means (6) for generating a prediction coefficient (D7) corresponding to the class classified by the class classification means (4 and 5), and prediction Coefficient (D7) and transmission image By predicting calculation using the chromatography data (D1), and the like and a interpolation data calculation means for calculating the interpolated data (D2) corresponding to the interpolation target pixel (7).
Further, in the present invention, in the image signal interpolation method for inputting the transmission image data (D1) in which the predetermined pixels are thinned out and interpolating the thinned pixels, the transmission image data (D1) is converted into the interpolation target pixels (FIG. 2 is divided into blocks of a predetermined size centered on each other, and a high-frequency signal (D4) of pixels in the block (a to l in FIG. 2) is extracted, and based on the high-frequency signal (D4) A level distribution pattern of the pixels (a to l) in the block is detected, and the interpolation target pixel is classified according to the detection result. Then, a prediction coefficient (D7) corresponding to the classified class is generated, and prediction calculation is performed using the prediction coefficient (D7) and the transmission image data (D1), so that interpolation data (D2) corresponding to the interpolation target pixel is obtained. ) Was calculated.
[0011]
[Action]
By obtaining the interpolation data (D2) using the prediction coefficient (D7) corresponding to the class, it is possible to avoid the aliasing distortion and obtain the interpolation data (D2) close to the true value. Further, the high pass signal (D4) of the pixels (a to l) in the block is extracted by the high pass filter means (3), and the interpolation target pixel is classified based on the high pass signal (D4). As a result, the interpolation target pixels can be accurately classified with a small number of classes, and as a result, the configuration of the prediction coefficient generating means (6) can be simplified.
[0012]
【Example】
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
[0013]
(1) Overall Configuration In FIG. 1, reference numeral 1 denotes an image signal interpolation apparatus as a whole, and by applying adaptive interpolation processing by class classification to input image data D1 whose information amount has been reduced by offset subsampling. The interpolation data D2 close to the true value is generated while avoiding the aliasing distortion.
[0014]
Specifically, the image signal interpolating apparatus 1 performs interpolation target pixels (that is, thinned out) in accordance with the level distribution pattern of sampling pixels (that is, pixels included in the input image data D1) around the interpolation target pixels. Pixels) are classified into classes, prediction coefficients obtained by learning for each class in advance are read out, and interpolation pixel values are obtained using the prediction coefficients.
As a result, in the image signal interpolation device 1, it is possible to obtain an interpolated pixel value that is extremely close to the true value as compared with the case where the interpolation target pixel is obtained simply by average interpolation using the surrounding sampling pixels.
[0015]
The image signal interpolating apparatus 1 inputs the input image data D1 to the block forming circuit 2, and the block forming circuit 2 divides the input image data D1 into blocks of a predetermined size to form the block image data D3. In this case, as shown in FIG. 2, the blocking circuit 2 forms each block by twelve peripheral pixels a to l around the pixel to be interpolated (x mark). .
[0016]
The high-pass filter circuit 3 receives the block image data D3, extracts the high-frequency signal D4 of the peripheral pixels a to l around the pixel to be interpolated in the block, and performs adaptive dynamic range encoding (ADRC (Adaptive Dynamic Range)). Range Coding)) is sent to the circuit 4. In this case, the high-pass filter circuit 3 extracts the high-frequency signal D4 using a coefficient as shown in FIG. 3, for example.
[0017]
The ADRC circuit 4 receives the high-frequency signal D4 sent from the high-pass filter circuit 3, performs adaptive dynamic range encoding processing on the high-frequency signal D4, and bit-compresses each pixel value, thereby compressing the pattern compressed data D5. Form.
The pattern compression data D5 is given to the class code generation circuit 5. The class code generation circuit 5 generates a class code D6 based on the pattern compression data D5 and supplies it to the prediction coefficient memory 6. The prediction coefficient memory 6 uses the class code D6 as a read address, and sends the prediction coefficient D7, which has been obtained and stored in advance for each class by learning described later, to the prediction calculation circuit 7.
[0018]
The prediction calculation circuit 7 calculates the pixel value of the pixel to be interpolated by performing calculation based on the linear linear combination equation using the pixel values a to l and the prediction coefficient D7 included in the blocked image data D3, Is output as interpolation data D2.
[0019]
Thus, in the image signal interpolation apparatus 1, an interpolation pixel value close to the true value can be obtained by forming the interpolation pixel using the prediction coefficient D7 obtained by learning in advance. In addition to this, in the image signal interpolating apparatus 1, the high-pass filter circuit 3 extracts the high-frequency signal D4 for each block (that is, removes a gradual change in the block and only a characteristic change in the block is detected). And classifying the high-frequency signal D4 based on the pattern compression data D5 obtained by adaptive dynamic range encoding processing, thereby improving the efficiency of class classification and improving the number of classes. Can be reduced, and the configuration of the prediction coefficient memory 6 can be simplified.
[0020]
(2) Class Classification Processing Next, class classification processing by the ADRC circuit 4 and the class code generation circuit 5 will be described. The ADRC circuit 4 defines an intra-block dynamic range as a local feature of the image, and adaptively removes redundancy mainly in the level direction. For example, as shown in FIG. 4, in the dynamic range of 0 to 255 of the pixel data of 8 bits per pixel, the in-block dynamic range A and B required for requantization for each block is as follows. It can be seen that it is significantly smaller. Therefore, if requantization is performed within the small dynamic ranges A and B, the number of necessary bits can be greatly reduced.
[0021]
Specifically, the ADRC circuit 4 first sets the dynamic range in the block as DR, the bit allocation as p, the pixel level in the block as x, and the requantization code as Q.

Thus, as shown in FIG. 5A, requantization is performed by equally dividing the maximum value MAX and the minimum value MIN in the block by the designated bit length. Incidentally, FIG. 5 shows a case where requantization is performed with 3 bits (that is, p = 3).
[0022]
Next, an average value of the pixels in the block existing in the data level corresponding to the highest gradation level (2 ^p −1) in FIG. 5A is taken, and this is obtained as shown in FIG. The maximum value is MAX ′. Further, the average value of the pixels in the block existing in the data level corresponding to the lowest gradation level 0 in FIG. 5A is taken, and this is set as the minimum value MIN ′ as shown in FIG. 5B.
[0023]
Next, the dynamic range DR ′ in the block is newly redefined with the newly determined maximum value MAX ′ and minimum value MIN ′, and the requantized code is defined as q, and the maximum value MAX ′ and minimum value in the newly determined block. Based on the value MIN ′,

Thus, requantization as shown in FIG.
In this way, the ADRC circuit 4 performs double requantization, thereby reducing the adverse effects of noise and performing efficient information compression to form pattern compressed data D5.
[0024]
The class code generation circuit 6 uses q ₁ to q _n as re-quantized data resulting from data compression of p bits by the ADRC circuit 4 in this way.

Is executed to determine the class code c (D6) representing the class to which the block belongs, that is, the class of the interpolation target pixel.
[0025]
(3) Creation of Prediction Coefficient Memory Next, how to obtain the prediction coefficient stored in the prediction coefficient memory 6 will be described with reference to FIG. First, in order to learn the prediction coefficient in step SP1, learning data corresponding to an already known image is formed. Specifically, the sampling pixels a to l in FIG. 2 and the pixel values before the sub-sample of the interpolation target pixel (x mark) are set as a set of learning data.
In addition, when forming learning data in this way, a more accurate prediction coefficient can be obtained if a large number of learning data is formed by using a plurality of images instead of using only one image. .
[0026]
In step SP2, it is determined whether necessary and sufficient learning data has been formed. If it is determined that further learning data is necessary, the process proceeds to step SP3. If it is determined that sufficient learning data is obtained, the process proceeds to step SP4.
In step SP3, the learning data is classified. At this time, the same class classification as in the image signal interpolation apparatus 1 described above is performed. Specifically, first, a high-pass signal of the learning sampling data is extracted by a high-pass filter, and each learning data is formed by compressing the extracted high-frequency signal of the learning sampling data by ADRC encoding and then forming a class code. Classify.
[0027]
Next, in step SP5, a normalization equation is formed for each class based on the learning data classified into classes. The processing at step SP5 will be specifically described. Here, for generalization, a case where n sampling pixels exist as learning data will be described. First pixel level x ₁ of each sampling _pixel, ..., a sub-sample previous relationship pixel level y of the target interpolation pixel and x _n, prediction coefficients w ₁ for each class _(c), ..., n according to w _n taps By expressing with the linear primary prediction formula of

Form.
[0028]
Prediction coefficients _w 1 in the equation (4), ..., may be obtained a _{w n.} Therefore, the prediction coefficients w ₁ ,..., W _n are calculated so that the error between the actual interpolation target pixel and the interpolation processing result is minimized. It is performed for a plurality of learning data learned for each class, where the prediction coefficients w _1, if the number of learning data is a general m> n When _m, ......, w _n can not be uniquely determined . Therefore, assuming that the elements of the error vector e are prediction errors in the respective learning data x _k1 ,..., X _kn , y _k (k = 1, 2,..., M), e _k ,

And the following formula:

Prediction coefficients _w 1 to minimize, ..., seek _{w n.} This is a so-called least square method.
[0029]
Here, when the partial differential coefficient by w _i of the equation (6) is obtained, the following equation is obtained:

It becomes. (7) may be determined each w _i such expression becomes zero. Therefore, the following formula:

And [Equation 9]

When X _ij and Y _i are defined as follows, the above equation (7) uses a matrix and the following equation:

Can be rewritten as
[0030]
Since this case (10) normal equation of formula is unknowns are n simultaneous equations, thereby the unknown coefficients are the most probable value w _1, ......, it can be obtained w _n. That is, in this prediction coefficient calculation processing procedure, steps SP1-SP2-SP3-SP5-SP1 are performed until a normalization equation that can determine undetermined coefficients w ₁ ,..., W _n for each class in step SP5 can be formed. Repeat the loop.
[0031]
Eventually, in step SP5, a normalization equation expressed by equation (10) is formed for each class. If a positive result is obtained in step SP2, the process proceeds to step SP4, where the normalization equation of equation (10) is solved. The prediction coefficients w ₁ ,..., W _n for each class are determined. Specifically, since the matrix on the left side of equation (10) is generally positive definite, it can be solved by the Cholesky method.
Next, at step SP6, the prediction coefficients w _1, which is determined for each _class, ..., and stores the w _n to the corresponding address of a class of the prediction coefficient memory 6, ends the prediction coefficient calculation process procedure in the following step SP7 To do.
[0032]
(4) Operation of Embodiment In the above configuration, the image signal interpolation device 1 uses the thinned pixels as interpolation target pixels, and classifies the interpolation target pixels according to the state of the surrounding block image data D3. .
In this case, the image signal interpolating apparatus 1 extracts the high-frequency signal D4 from the peripheral pixels of the interpolation target pixel by the high-pass filter circuit 3 as pre-classification processing. The image signal interpolating apparatus 1 applies adaptive dynamic range encoding processing to the high frequency signal D4 to form pattern compressed data D5, and uses this pattern compressed data D5 to indicate the class classification of the pixel to be interpolated. The code D6 is formed.
As described above, in the image signal interpolating apparatus 1, an increase in the number of unnecessary classes can be suppressed by removing a gradual change from the blocked image data D3 and classifying only the characteristic change. , You will be able to classify efficiently.
[0033]
Next, the image signal interpolation apparatus 1 reads the prediction coefficient D7 stored in the prediction coefficient memory 6 using the class code D6 as a read address. Then, the prediction calculation circuit 7 uses the prediction coefficients w ₁ (c) to w ₁₂ (c) corresponding to the class c and the pixel data x _{1 to} x _{12 of the} sampling pixels a to l included in the blocked image data D3. With the following formula:

By performing linear linear combination as described above, an interpolation value y ′ corresponding to the interpolation target pixel is calculated and output as interpolation data D2.
[0034]
In this way, the image signal interpolation apparatus 1 can form interpolation data D2 that is almost the same as the pixel data included in the original image. The interpolation data D2 is combined with input image data D1 by a combining circuit (not shown), and then supplied to, for example, a television receiver or a video tape recorder.
[0035]
(5) According to the above configuration, the high-frequency signal D4 is extracted from the peripheral pixels of the interpolation target pixel, and the interpolation target pixel is classified using the high-frequency signal D4. Can be accurately classified with a small number of classes. As a result, the image signal interpolation device 1 having a simple configuration capable of forming interpolation data close to the true value can be realized.
[0036]
(6) Other Embodiments In the above-described embodiment, when classifying the interpolation target pixel, the high-frequency signal D4 of the peripheral pixel is bit-compressed by using the ADRC circuit 4 and the class code generation circuit 5, thereby classifying the class. As described above, the class classification means is not limited to this. For example, various compression means such as discrete cosine transform (DCT), differential quantization (DPCM), subband coding, and wavelet transform are applied. obtain. Further, the class classification method is not limited to the bit compression method. For example, a direction with strong correlation may be detected in the peripheral pixels of the interpolation target pixel, and the class classification may be performed based on the detection result.
[0037]
In the above-described embodiment, the case where the prediction coefficient for each class is obtained by learning using the least square method has been described. However, the method for obtaining the prediction coefficient is not limited to this, and various learning methods can be used. .
[0038]
Further, in the above-described embodiment, the prediction coefficient memory 6 and the prediction calculation circuit 7 are provided, the prediction coefficient D7 stored in the prediction coefficient memory 6 by learning in advance is read according to the class code D6, and the read prediction coefficient D7 However, the present invention is not limited to this, and instead of the prediction coefficient memory 6 and the prediction arithmetic circuit 7, the interpolation data D2 is obtained by linearly linearly combining the block image data and the block image data D3. A memory for storing the representative value for each class obtained by learning may be provided, the representative value corresponding to the class may be read, and the read representative value may be used as interpolation data.
[0039]
In this case, as a first method for obtaining the representative value stored in the memory, there is learning by weighted average. More specifically, the true pixel value corresponding to the pixel to be interpolated is accumulated for each class, and when this division result is divided by the number of accumulated pixel values, processing is performed on various images. A representative value for each class is obtained.
As a second method for obtaining the representative value, there is learning by normalization. More specifically, a block formed of a plurality of pixels including an interpolation target pixel is formed, and a block reference value is subtracted from a true pixel value corresponding to the interpolation target pixel by a dynamic range in the block. When the accumulated value of the normalized value is divided by the accumulated frequency, the representative value for each class is obtained by performing processing on various images.
[0040]
【The invention's effect】
As described above, according to the present invention, the image signal blocking means for dividing the transmission image data into blocks of a predetermined size, the high-pass filter means for extracting the high-frequency signal of the pixels in the block, and the high-frequency signal are used. Classifying means for classifying the pixel to be interpolated, prediction coefficient generating means for generating a prediction coefficient corresponding to the classified class, and prediction calculation using the prediction coefficient and the transmission image data, so that the interpolation target pixel By providing the interpolation data calculation means for calculating the interpolation data corresponding to the above, it is possible to obtain the interpolation pixel value close to the true value while avoiding the occurrence of aliasing distortion.
[0041]
Further, according to the present invention, the high-pass filter means extracts the high-frequency signal of the pixel in the block, and classifies the interpolation target pixel based on the high-frequency signal. It becomes possible to classify accurately by numbers, and as a result, the configuration can be simplified.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an image signal interpolation apparatus according to an embodiment of the present invention.
FIG. 2 is a schematic diagram for explaining peripheral pixels used for class classification processing;
FIG. 3 is a schematic diagram illustrating coefficients of a high-pass filter circuit.
FIG. 4 is a schematic diagram for explaining bit compression by an ADRC circuit.
FIG. 5 is a schematic diagram for explaining bit compression by an ADRC circuit.
FIG. 6 is a flowchart showing a prediction coefficient calculation processing procedure by learning.
FIG. 7 is a schematic diagram for explaining offset subsampling.
FIG. 8 is a schematic diagram showing a spatial frequency spectrum of a band that can be transmitted by two-dimensional offset subsampling.
FIG. 9 is a schematic diagram for explaining an interpolation process;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Image signal interpolation apparatus, 2 ... Blocking circuit, 3 ... High pass filter circuit, 4 ... ADRC circuit, 5 ... Class code generation circuit, 6 ... Prediction coefficient memory, 7 ... Prediction arithmetic circuit.

Claims (18)

In an image signal interpolating apparatus for inputting transmission image data in which predetermined pixels are thinned out and interpolating the thinned out pixels,
Image signal blocking means for dividing the transmission image data into blocks of a predetermined size centered on the interpolation target pixel;
A high-pass filter means for extracting a high-frequency signal of the pixel in the block;
Class classification means for detecting a level distribution pattern of the pixels in the block based on the high-frequency signal and classifying the interpolation target pixel according to the detection result;
Prediction coefficient generation means for generating a prediction coefficient corresponding to the class classified by the class classification means;
An image signal interpolation apparatus comprising: interpolation data calculation means for calculating interpolation data corresponding to the interpolation target pixel by performing a prediction calculation using the prediction coefficient and the transmission image data. The interpolation data calculation means includes
The image signal interpolation apparatus according to claim 1, wherein interpolation data corresponding to an interpolation target pixel is calculated by calculating the prediction coefficient and the transmission image data using a linear linear combination formula. 3. The image signal interpolating apparatus according to claim 1, wherein the class classification means is data compression means for compressing the high frequency signal. 4. The image signal interpolating apparatus according to claim 3, wherein the data compression means is an adaptive dynamic range coding (ADRC) means. The prediction coefficient generating means is:
3. The method according to claim 1, further comprising: a memory for storing a prediction coefficient for each class obtained in advance by learning, and generating a prediction coefficient corresponding to the class classified by the class classification unit. The image signal interpolating device described. In an image signal interpolating apparatus for inputting transmission image data in which predetermined pixels are thinned out and interpolating the thinned out pixels,
Image signal blocking means for dividing the transmission image data into blocks of a predetermined size centered on the interpolation target pixel;
A high-pass filter means for extracting a high-frequency signal of the pixel in the block;
Class classification means for detecting a level distribution pattern of the pixels in the block based on the high-frequency signal and classifying the interpolation target pixel according to the detection result;
Representative value generation means for generating a representative value corresponding to the class classified by the class classification means, and the representative value generated by the representative value generation means is used as interpolation data corresponding to the interpolation target pixel. An image signal interpolation device characterized by the above. 7. The image signal interpolating apparatus according to claim 6, wherein the class classification means is data compression means for compressing the high frequency signal. 8. The image signal interpolating apparatus according to claim 7, wherein the data compression means is an adaptive dynamic range coding (ADRC) means. The representative value generating means is:
7. The image signal according to claim 6, further comprising a memory for storing a representative value for each class obtained in advance by learning, and generating a representative value corresponding to the class classified by the class classification means. Interpolator. Input transmission image data with a predetermined pixel thinned out, and interpolate the thinned pixel. In the image signal interpolation method,
A first step of dividing the transmission image data into blocks of a predetermined size centered on the interpolation target pixel;
A second step of extracting a high-frequency signal of the pixel in the block;
A third step of detecting a level distribution pattern of the pixels in the block based on the high-frequency signal and classifying the interpolation target pixel according to the detection result;
A fourth step for generating a prediction coefficient corresponding to the class classified by the third step;
A fifth step of calculating interpolation data corresponding to the interpolation target pixel by performing a prediction calculation using the prediction coefficient and the transmission image data;
An image signal interpolation method comprising: In the fifth step,
Interpolation data corresponding to the interpolation target pixel is calculated by calculating the prediction coefficient and the transmission image data using a linear linear combination formula.
The image signal interpolation method according to claim 10. In the third step, the high frequency signal is compressed.
12. The image signal interpolation method according to claim 10, wherein the image signal interpolation method is performed. In the third step, the high-frequency signal is compressed by adaptive dynamic range coding (ADRC) processing.
The image signal interpolation method according to claim 12. In the fourth step,
A memory for storing a prediction coefficient for each class obtained in advance by learning is generated, and a prediction coefficient corresponding to the class classified by the third step is generated.
12. The image signal interpolation method according to claim 10, wherein the image signal interpolation method is performed. In an image signal interpolation method for inputting transmission image data in which predetermined pixels are thinned and interpolating the thinned pixels,
A first step of dividing the transmission image data into blocks of a predetermined size centered on the interpolation target pixel;
A second step of extracting a high-frequency signal of the pixel in the block;
A third step of detecting a level distribution pattern of the pixels in the block based on the high-frequency signal and classifying the interpolation target pixel according to the detection result;
A fourth step for generating a representative value corresponding to the class classified by the third step;
A fifth step in which the representative value generated in the fourth step is used as interpolation data corresponding to the interpolation target pixel;
An image signal interpolation method comprising: In the third step, the high frequency signal is compressed.
The image signal interpolation method according to claim 15, wherein: In the third step, the high-frequency signal is compressed by adaptive dynamic range coding (ADRC) processing.
The image signal interpolating method according to claim 16. In the fourth step,
It has a memory for storing representative values for each class obtained in advance by learning, and generates representative values corresponding to the classes classified in the third step.
The image signal interpolation method according to claim 15, wherein:

Images