JP2006279383A - Interhierarchy prediction coding method, apparatus thereof, interhierarchy prediction decoding method, apparatus thereof, interhierarchy prediction coding program, interhierarchy prediction decoding program and program recording medium thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve image quality by adaptively changing a filter coefficient of an interpolation filter to be used for interhierarchy prediction in accordance with the local property of an image. <P>SOLUTION: In generating an interpolation signal as a prediction signal of a upper hierarchy from a lower hierarchy reference signal, a portion of signals already coded and decoded which is similar to a coding (decoding) target signal is extracted as a reference signal to be used for calculation of the filter coefficient. A filter coefficient adaptively changing depending on an input image is calculated by using the extracted reference signal. The calculated filter coefficient is applied for each block to execute coding/decoding processing. The space-time position of the reference signal is incorporated into coding information or decided based on a local decoding signal. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a high-efficiency image signal encoding method that performs inter-layer prediction, and in particular, improves the estimation accuracy of filter coefficients when an interpolation signal is generated from a lower-layer signal as an upper-layer prediction signal using an interpolation filter, The present invention relates to an inter-layer predictive encoding / decoding method capable of improving image quality.

In recent years, scalable coding to deal with diversifying network environments and terminal environments has attracted attention. In scalable coding, an image signal is divided hierarchically and coding is performed for each layer. As a method of hierarchy division,
(I) Band division related to spatial frequency,
(Ii) Band division for time frequency,
and so on. Typical examples of (i) are wavelet division (see Non-Patent Document 1), and (ii) is Motion Compensation Temporal Fitering (MCTF) (see Non-Patent Document 2).

  In this case, if each layer is encoded independently, the improvement in encoding efficiency cannot be expected. For the purpose of improving encoding efficiency, an encoding method using correlation between layers has been studied. Specifically, inter-layer prediction for predicting a high spatial resolution image signal from a low spatial resolution image signal is performed for two layers having different spatial resolutions.

In the following Non-Patent Document 3, a 6-tap interpolation filter is used to interpolate pixel values at half-pixel positions in a low resolution image signal to obtain a prediction signal for the high resolution image signal.
“A theory for multiresolution signal decomposition: the wavelet representation”, SGMallat, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.11, No.7, pp.674-693, July, 1989. “Three-dimensional subband coding with motion compensation”, JROhm, IEEE Trans.Image Processing, Vol.3, No.5, pp.559-571, Sept., 1994. J. Reichel, M. Wien and H. Schwarz, “Scalable Video Model 3.0”, ISO / IEC JTC1 / SC29 / WG11 doc. No. N6716, Palma, October 2004.

  The problem with the conventional method described above is that the coefficients of the interpolation filter are fixed. In general, the statistical properties of images vary from image to image. In addition, the properties are different if viewed locally in the image. For this reason, in order to improve the efficiency of inter-layer prediction, it is necessary to adaptively change the configuration of the interpolation filter according to the local properties of each image. However, since such additional processing increases additional information for expressing the filter, how to reduce the additional information is the key to improving the coding efficiency.

  The present invention has been made in view of such circumstances, and an object thereof is to establish an adaptive design method for an interpolation filter in inter-layer prediction according to the local properties of an image while suppressing an increase in additional information. To do.

  In order to solve the above-described problem, the present invention provides a high-order video encoding method that predicts and encodes a high spatial resolution signal (upper layer signal) from a low spatial resolution signal (lower layer signal) using interpolation processing. When generating an interpolation signal from a lower layer signal as a layer prediction signal, specify the spatio-temporal position of the local decoded image, and use the signal at the specified position to generate a filter coefficient for generating the interpolation image for each block. Is estimated. Thereby, it is possible to improve the estimation accuracy of the filter coefficient and improve the image quality of the decoded image.

  Further, the present invention is characterized in that, in the above moving picture coding, a coding parameter for identifying a spatio-temporal position of a decoded signal to be referred to is part of the coding information. Thereby, an adaptive filter coefficient can be calculated with a small amount of calculation.

  Further, according to the present invention, in the moving image coding, information indicating a spatio-temporal position of a decoded signal to be referred to can be determined based on a local decoded signal. Thereby, an increase in the code amount of the filter coefficient can be suppressed. Furthermore, the presence or absence of filter coefficient estimation processing can be adaptively switched according to the quantization parameter of the lower layer reference signal. Thus, the calculated filter coefficient can be used only when the estimation result is reliable, and the default filter coefficient similar to the conventional one can be used when the estimation result is insufficient.

  Further, the present invention provides a temporal and spatial position in moving picture decoding in which a decoded signal is obtained by adding a prediction signal obtained by interpolating a high spatial resolution signal from a low spatial resolution signal and a prediction residual signal. A filter coefficient for generating an interpolated image is estimated for each block using a decoded signal designated by.

  Further, the present invention is characterized in that, in the above moving picture decoding, a coding parameter for identifying a spatio-temporal position of a decoded signal to be referred to is extracted from coding information.

  Further, according to the present invention, in the moving image decoding, information indicating a spatio-temporal position of a reference decoded signal can be determined based on the decoded signal. Furthermore, the presence or absence of filter coefficient estimation processing can be adaptively switched according to the quantization parameter of the lower layer reference signal.

The outline of processing at the time of encoding and decoding is as follows.
(1) A portion similar to an encoding (decoding) target signal is extracted as a reference signal used for calculation of a filter coefficient among already encoded and decoded signals.
(2) Using the extracted reference signal, filter coefficients that adaptively change according to the input image are calculated.
(3) The calculated filter coefficient is applied in block units to perform encoding / decoding processing.

In the following, the coordinates [x, y] in the frame at time t of the j hierarchy to the pixel value at f _j (x, y, t) and, f _j (x, y, t) of the decoded signal for g _j ( x, y, t). f _{j + 1} (x, y, t) has a spatial resolution twice that of f _j (x, y, t). For example, if f ₀ (x, y, t) is a QCIF size, f ₁ (x, y, t) is a CIF size.

  Interpolation from the j-th layer signal to the j + 1-th layer signal is as follows for each direction. Here, the number of interpolation filter taps is N.

  Horizontal interpolation:

垂直方向の補間：

Vertical interpolation:

斜め方向の補間：

Diagonal interpolation:

または，

Or

ここで，Ｎ／２および（Ｎ−１）／２を囲む記号は，floor functionであり，それぞれ実数Ｎ／２および（Ｎ−１）／２を越えない最大の整数を表す（以下，同様）。

Here, symbols surrounding N / 2 and (N-1) / 2 are floor functions, which represent the maximum integers not exceeding real numbers N / 2 and (N-1) / 2, respectively (the same applies hereinafter). .

FIG. 1 shows an example of horizontal interpolation. In FIG. 1, the number of taps N is 4, and the pixel values of [2x + 1, 2y, t] in the j + 1th layer are represented by [x-1, y, t], [x, y, t], In the example, interpolation is performed using pixel values of [x + 1, y, t] and [x + 2, y, t]. alpha _-1 to? _{2 is} a filter coefficient. The same applies to interpolation in the vertical and oblique directions.

When f _{j + 1} (x, y, t) is an encoding target signal, the lower layer signal f _j (x ′, y ′, t) used for interpolation is referred to as a lower layer reference signal.

  In the present invention, since filter coefficients are adaptively changed in units of rectangular areas (vertical / horizontal width L), if each filter coefficient is encoded as it is, additional information for expressing the filter coefficient increases. Therefore, in order to suppress additional information related to the filter coefficient, a method of estimating the filter coefficient using a local decoded signal that has already been encoded is adopted. The present invention has two modes (Implicit mode and Explicit mode) as a method of expressing the estimated filter coefficients. Details of these two modes will be described later.

[Estimation of filter coefficients]
Estimation of the filter coefficients is performed for each rectangular region (length / width L) of the decoded signal of the t ₀ th frame j + 1 hierarchy. This rectangular area is called a reference area for filter coefficient estimation. A rectangular area (vertical / horizontal width L / 2) in the decoded signal of the j-th layer used for interpolation of the reference area and the reference area are collectively referred to as a reference area pair, and a signal included in the reference area pair is referred to as a reference signal. FIG. 2 shows the relationship between these encoding target signals and reference signals.

  Interpolation from the decoded signal of the jth layer to the decoded signal of the j + 1th layer is as follows for each interpolation direction.

  Horizontal interpolation:

垂直方向の補間：

Vertical interpolation:

斜め方向の補間：

Diagonal interpolation:

または，

Or

第ｔ₀ フレームにおける左上角の座標を［ｘ₀ ，ｙ₀ ］とする参照領域に対して，補間による予測誤差は次のように表せる。

For a reference region in which the coordinates of the upper left corner in the t _0th frame are [x ₀ , y ₀ ], the prediction error due to interpolation can be expressed as follows.

この予測誤差を最小化するフィルタ係数α_n （ｘ₀ ，ｙ₀ ，ｔ₀ ），β_n （ｘ₀ ，ｙ₀ ，ｔ₀ ），γ_n （ｘ₀ ，ｙ₀ ，ｔ₀ ），ζ_n （ｘ₀ ，ｙ₀ ，ｔ₀ ）は各々，以下の連立方程式を解くことで求まる。

Filter coefficients α _n (x ₀ , y ₀ , t ₀ ), β _n (x ₀ , y ₀ , t ₀ ), γ _n (x ₀ , y ₀ , t ₀ ), ζ _n that minimize this prediction error (X ₀ , y ₀ , t ₀ ) can be obtained by solving the following simultaneous equations.

α_k （ｘ₀ ，ｙ₀ ，ｔ₀ ）は，式（１２）より得る以下の連立方程式の解として求まる（ただし，ｋは−［（Ｎ−１）／２を越えない最大の整数］から［Ｎ／２を越えない最大の整数］までの整数，以下同様）。

α _k (x ₀ , y ₀ , t ₀ ) is obtained as a solution of the following simultaneous equations obtained from Equation (12) (where k is the maximum integer not exceeding − (N−1) / 2). Integer up to [maximum integer not exceeding N / 2], and so on).

ここで，Ｒ_h ^(j) （ｋ，ｎ），Ｃ_h ^(j) （ｎ）は，次の通りである。

Here, R _h ^(j) (k, n) and C _h ^(j) (n) are as follows.

β_k （ｘ₀ ，ｙ₀ ，ｔ₀ ）は，式（１３）より得る以下の連立方程式の解として求まる。

β _k (x ₀ , y ₀ , t ₀ ) is obtained as a solution of the following simultaneous equations obtained from equation (13).

ここで，Ｒ_v ^(j) （ｋ，ｎ），Ｃ_v ^(j) （ｎ）は，次の通りである。

Here, R _v ^(j) (k, n) and C _v ^(j) (n) are as follows.

γ_k （ｘ₀ ，ｙ₀ ，ｔ₀ ）は，式（１４）より得る以下の連立方程式の解として求まる。

γ _k (x ₀ , y ₀ , t ₀ ) is obtained as a solution of the following simultaneous equations obtained from equation (14).

ここで，Ｒ_dh ^(j) （ｋ，ｎ），Ｃ_dh ^(j) （ｎ）は，次の通りである。なお，以下の＾ｇ_j+1 （２ｘ，２ｙ＋１，ｔ₀ ）は，式（６）より求めた値を用いる。

Here, R _dh ^(j) (k, n) and C _dh ^(j) (n) are as follows. The following ^ g _{j + 1} (2x, 2y + 1, t ₀ ) uses the value obtained from equation (6).

ζ_k （ｘ₀ ，ｙ₀ ，ｔ₀ ）は，式（１５）より得る以下の連立方程式の解として求まる。

ζ _k (x ₀ , y ₀ , t ₀ ) is obtained as a solution of the following simultaneous equations obtained from equation (15).

ここで，Ｒ_dv ^(j) （ｋ，ｎ），Ｃ_dv ^(j) （ｎ）は，次の通りである。なお，以下の＾ｇ_j+1 （２ｘ＋１，２ｙ，ｔ₀ ）は，式（５）より求めた値を用いる。

Here, R _dv ^(j) (k, n) and C _dv ^(j) (n) are as follows. The following ^ g _{j + 1} (2x _{+ 1,} 2y, t ₀ ) uses the value obtained from equation (5).

［座標値の符号化］
上述の通り，フィルタ係数は，参照領域を指す座標値［ｘ₀ ，ｙ₀ ，ｔ₀ ］の関数となる。この座標値がフィルタ係数を表現する情報となる。本発明は，この座標値を符号化する方法として，以下の２つのモード（Implicit mode ，Explicit mode ）を備える。

[Coding of coordinate values]
As described above, the filter coefficient is a function of coordinate values [x ₀ , y ₀ , t ₀ ] indicating the reference region. This coordinate value becomes information expressing the filter coefficient. The present invention comprises the following two modes (Implicit mode and Explicit mode) as a method of encoding the coordinate values.

(1) Explicit mode
Explicit mode is a mode in which coordinate values [x ₀ , y ₀ , t ₀ ] are transmitted to the decoding side as encoded information. The specific estimation method of coordinate values shall be given from the outside. As an example of the coordinate value estimation method, there is a method for minimizing the following costs.

_{[X o, y 0, t} 0] = min xr, yr, tr {E h (x r, y r, t r)
+ E _v (x _r , y _r , _tr )
+ E _d (x _r , y _r , _tr )}
Here, min _{xr, yr, tr} {E} is −B _x <x _r ≦ B _x −1, −B when B _x , B _y , B _t are separately given as parameters indicating the search range. _{y <y r ≦ B y -1} , in the range of _{-B t <t r ≦ B t} -1, means a x _r, y _r, t _r where E is minimized.

However, when the filter coefficient set by default is used without estimating the filter coefficient using the reference region, information indicating that the default coefficient is used is transmitted. For example, there are a method of adding an identification bit indicating whether to perform estimation, or a method of expressing a default coefficient by a value that cannot be taken as a coordinate value [x ₀ , y ₀ , t ₀ ].

Note that the encoding target is a rectangular area whose vertical / horizontal width is L, where the coordinates of the upper left corner in the t frame are [x, y], and the corresponding reference area is the coordinates of the upper left corner in the t ₀ frame. _0, if the vertical / horizontal width to y _0] has a rectangular region of the L, and is also possible to make the displacement of _{[x-x 0, y-} y 0, t-t 0] and encoded.

(2) Implicit mode
Implicit mode is a mode in which coordinate values [x ₀ , y ₀ , t ₀ ] are not transmitted to the decoding side as encoded information. By using the rules that can be shared by the encoder / decoder, the decoder side reproduces the same coordinate value as the encoder, thereby omitting the transmission of the coordinate value information. Two specific methods are shown below. However, when the quantization parameter for the lower layer reference signal is a value equal to or greater than the threshold, the following method is not performed and the default coefficient is used. This is because the result of similarity search for a decoded image with extremely low image quality is not reliable as an estimated value of the similarity of the encoding target signal. In the following, the lower layer reference signal is a rectangular area having a vertical / horizontal width of L / 2 with the coordinates of the upper left corner in the t _c frame being [x _c , y _c ].

[Method 1]: A region in the decoded signal having the maximum similarity to the encoding target signal is set as a reference region. As the similarity S (x ₀ , y ₀ , t ₀ ), a correlation coefficient with the encoding target signal is used.

ここで，ｘ_r ，ｙ_r ，ｔ_r の探索範囲を示すパラメータＢ_x ，Ｂ_y ，Ｂ_t は，別途与えられる。ｍａｘ_xr,yr,trＳ（ｘ_r ，ｙ_r ，ｔ_r ）は，−Ｂ_x ＜ｘ_r ≦Ｂ_x −１，−Ｂ_y ＜ｙ_r ≦Ｂ_y −１，−Ｂ_t ＜ｔ_r ≦Ｂ_t −１の範囲で，類似度Ｓが最大となるｘ_r ，ｙ_r ，ｔ_r である。

Here, x _r, y _r, t parameters B _x indicating the search range of _r, B _y, B _t are given separately. _{max xr, yr, tr S (} x r, y r, t r) _{is, -B x <x r ≦ B} x -1, -B y <y r ≦ B y -1, -B t <t r ≦ in the range of B _t -1, similarity S is maximum x _r, y _r, is t _r.

However, if the maximum similarity is below the threshold Φ _S :
S (x ₀ , y ₀ , t ₀ ) ≦ Φ _S
Filter coefficients are not estimated using the reference region, but default coefficients are used.

[Method 2]: A region in the decoded signal that minimizes the distance to the encoding target signal is set as a reference region. As the distance function D (x ₀ , y ₀ , t ₀ ), the sum of square errors with the signal to be encoded is used.

ここで，ｘ_r ，ｙ_r ，ｔ_r の探索範囲を示すパラメータＢ_x ，Ｂ_y ，Ｂ_t は，別途与えられる。ｍｉｎ_xr,yr,trＤ（ｘ_r ，ｙ_r ，ｔ_r ）は，−Ｂ_x ＜ｘ_r ≦Ｂ_x −１，−Ｂ_y ＜ｙ_r ≦Ｂ_y −１，−Ｂ_t ＜ｔ_r ≦Ｂ_t −１の範囲で，距離関数Ｄが最大となるｘ_r ，ｙ_r ，ｔ_r である。

Here, x _r, y _r, t parameters B _x indicating the search range of _r, B _y, B _t are given separately. _{min xr, yr, tr D (} x r, y r, t r) _{is, -B x <x r ≦ B} x -1, -B y <y r ≦ B y -1, -B t <t r ≦ in the range of B _t -1, the distance function D is maximum x _r, y _r, is t _r.

If the minimum value of the distance function exceeds the threshold Φ _D :
D (x ₀ , y ₀ , t ₀ ) ≧ Φ _D
Filter coefficients are not estimated using the reference region, but default coefficients are used.

  Switching between Implicit mode and Explicit mode is performed in frame units or sequence units.

  According to the present invention, when performing prediction between frames having different resolutions, it is possible to perform adaptive prediction according to the local properties of an image, and an improvement in encoding efficiency can be expected. Further, the present invention can be selectively used according to the coding rate by providing two modes. Of these, the Implicit mode is particularly effective when it is desired to suppress an increase in overhead at a low rate. Further, the Implicit mode can adaptively switch ON / OFF of filter coefficient estimation based on a decoded signal using a quantization parameter. If it is determined that the image quality of the decoded image is large and the reliability is not sufficient as a reference signal for filter coefficient estimation, the filter coefficient estimation is not performed, so that a decrease in prediction performance can be avoided.

[Encoding process]
An embodiment (Explicit mode) of the present invention will be described with reference to the drawings. FIG. 3 is a flowchart of the encoding process in the Explicit mode according to the embodiment of the present invention.

  In step S10, it is selected whether the filter coefficient is estimated from the decoded image or a default coefficient prepared as a default is used. The method of selection shall be given separately. For example, a method using the following quantization parameter is used.

The t _0th frame is divided into L × L rectangular areas, and the coordinates of the upper left corner are (nL, nL), (nL, (n−1) L), ((n−1), for example, as shown in FIG. ) L, nL), ((n-1) L, (n-1) L), the quantization parameters of the rectangular regions are Q _0,0 , Q _{0, -1} , Q ₋₁ , Q _{−, respectively. 1, -1} . When the coordinate value of the reference area is [x ₀ , y ₀ ], it is calculated from the threshold value Q _Th and the quantization parameters Q _0,0 , Q _{0, −1} , Q _−1,0 , Q _{−1, −1.} The following adaptive processing is performed using ^ Q.

(I) If ^ Q < _QTh , upsampling is performed using the filter coefficient obtained by the filter coefficient calculation process described above.

(Ii) If ^ Q ≧ _QTh , upsampling is performed using filter coefficients prepared in advance as defaults.

  Note that ^ Q is obtained as follows.

フィルタ係数としてデフォルト係数を用いる場合，ステップＳ１１へ進み，デフォルト係数を読み込み，その後，ステップＳ１６へ進む。

When the default coefficient is used as the filter coefficient, the process proceeds to step S11, the default coefficient is read, and then the process proceeds to step S16.

  When the filter coefficient is estimated from the decoded image, the process proceeds to step S12, the encoding target signal and the reference signal are input, the process of estimating the coordinate value of the reference area for calculating the filter coefficient is performed, and the coordinate value is output To do. The coordinate value estimation method is assumed to be given separately.

  Next, in step S13, a process of calculating autocorrelation for each direction is performed by inputting the reference area and the coordinate value indicating the same area, and the autocorrelation for each direction is output. A specific calculation method follows Formula (17), Formula (20), Formula (23), and Formula (26). In step S14, a process for calculating the cross-correlation for each direction is performed by inputting the encoding target signal, the reference area, and the coordinate value indicating the area, and the cross-correlation for each direction is output. A specific calculation method follows equations (18), (21), (24), and (27).

  Thereafter, in step S15, the autocorrelation and the cross-correlation for each direction are input, a process for solving simultaneous equations with a filter coefficient that minimizes the prediction error due to upsampling is performed, and the obtained solution is output. Specifically, the simultaneous equations are expressed by the following equations (16), (19), (22), and (25). In addition, a specific method such as the Gauss-Seidel method is given from the outside as a specific solution of the equation.

  In step S16, the lower layer reference signal and the filter coefficient are input, upsampling processing is performed, and a signal after the processing is output.

  In step S17, the signal after the upsampling process output in step S16 and the encoding target signal are input, a process of generating a difference signal (prediction residual signal) of both is performed, and the difference signal is output.

  In step S18, it is determined whether or not the processing has been completed for all the blocks, and the processing in steps S10 to S17 is repeated until the processing is completed for all the blocks.

  Next, an embodiment (Implicit mode) of the present invention will be described with reference to the drawings. FIG. 5 is a flowchart of the encoding process in the Implicit mode according to the embodiment of the present invention.

  First, in step S20, the quantization parameter used for encoding the lower layer reference signal is input, it is determined whether or not the parameter is equal to or greater than a threshold value, and if it is equal to or greater than the threshold value, a true value is output. Otherwise, a false value is output. When the true value is output, the control proceeds to step S24, and the process proceeds to an upsampling process using a default coefficient prepared as a default. On the other hand, if a false value is output, the process proceeds to the determination process after step S21.

  In step S21, the lower layer reference signal and the reference signal are input, a process of estimating the coordinate value of the reference region used for calculating the filter coefficient is performed, and the coordinate value and the maximum similarity are output. As the coordinate value, a reference area that maximizes the similarity given by Expression (28) is searched.

  In step S22, the coordinate value and the maximum similarity of the reference area output in step S21 are stored in a register. Next, in step S23, the maximum similarity stored in step S22 is read from the register, it is determined whether or not the maximum similarity is equal to or less than a threshold value, and if it is equal to or less than the threshold value, a true value is output; Otherwise, a false value is output. If a true value is output, control is transferred to step S24, and the process proceeds to an upsampling process using a default coefficient prepared as a default. On the other hand, when a false value is output, the process proceeds to an upsampling process using a filter coefficient estimated from the decoded image in step S25 and subsequent steps.

  The processing after step S24 is the same as the processing after step S11 in FIG.

  Finally, in step S30, it is determined whether or not processing has been completed for all blocks, and the processing in steps S20 to S29 is repeated until processing is completed for all blocks.

  FIG. 6 shows a block diagram of the encoding device (Explicit mode). FIG. 6 shows only the part mainly related to the present invention, and the conventional technique can be used as it is for the encoding of the prediction residual signal and the processing part for decoding the encoded signal. Omitted. The same applies to FIG. 7 described later.

  The encoding target signal is stored in the encoding target signal storage unit 101. The filter coefficient calculation method selection unit 102 receives the encoding target signal read from the encoding target signal storage unit 101 and selects whether to estimate the filter coefficient from the decoded image or to use the default coefficient prepared as a default. To do. The method of selection shall be given separately.

  The default coefficient is stored in advance in the default coefficient storage unit 103. When the use of the default coefficient is selected by the filter coefficient calculation method selection unit 102, the filter coefficient read from the default coefficient storage unit 103 is It is written in the coefficient storage unit 114.

  When estimating the filter coefficient from the decoded image, the reference region coordinate estimation unit 104 receives the encoding target signal read from the encoding target signal storage unit 101 and performs processing for estimating the reference region to identify the reference region. The coordinate value is written in the reference area coordinate storage unit 105. Also, the same coordinate value is output as encoded information.

  The reference signal storage unit 106 receives the coordinate value of the reference area read from the reference area coordinate storage unit 105, reads a signal corresponding to the coordinate value from the decoded signal storage unit 107, and stores the signal as a reference signal.

  The filter coefficient calculation unit 108 receives the encoding target signal read from the encoding target signal storage unit 101 and the reference signal read from the reference signal storage unit 106, performs the following three processes, and outputs a filter coefficient.

  The autocorrelation calculation unit 109 receives the reference signal read from the reference signal storage unit 106, performs a process of calculating autocorrelation for each direction, and writes the calculation result to the autocorrelation storage unit 110. A specific calculation method follows Formula (17), Formula (20), Formula (23), and Formula (26).

  Further, the cross-correlation calculation unit 111 receives the encoding target signal and the reference signal read from the reference signal storage unit 106, performs a process of calculating the cross-correlation for each direction, and writes the calculation result to the cross-correlation storage unit 112. . A specific calculation method follows equations (18), (21), (24), and (27).

  The simultaneous equation solving unit 113 performs a process of solving simultaneous equations using the autocorrelation and the cross correlation read from the autocorrelation storage unit 110 and the cross correlation storage unit 112 as inputs and using a filter coefficient that minimizes the prediction error as a solution. The obtained solution is written in the filter coefficient storage unit 114. Specifically, the simultaneous equations are expressed by the following equations (16), (19), (22), and (25). In addition, a specific method such as the Gauss-Seidel method is given from the outside as a specific solution of the equation.

  The up-sampling signal generation unit 115 receives the filter coefficient read from the filter coefficient storage unit 114 and the decoded signal of the lower layer reference signal read from the decoded signal storage unit 107, performs up-sampling processing, and performs up-sampling on the encoding target signal. The sampling signal is written to the upsampling signal storage unit 116. A specific upsampling process follows Formula (1), Formula (2), Formula (3), and Formula (4).

  The prediction residual signal calculation unit 117 receives the encoding target signal and the upsampling signal read from the upsampling signal storage unit 116 and performs processing for generating a difference signal (prediction residual signal) between them. Output a signal.

  FIG. 7 shows a block diagram of an encoding device (Implicit mode).

  The encoding target signal storage unit 201 stores the encoding target signal, similarly to the encoding target signal storage unit 101 of FIG. The filter coefficient calculation method first determination unit 202 receives the quantization parameter for the lower layer reference signal read from the quantization parameter storage unit 203, performs a process of determining whether or not the parameter is larger than a threshold, Is output. When the output is a true value (when the parameter is larger than the threshold value), the process moves to a process of reading a default coefficient. On the other hand, when the output is a false value, the process proceeds to the next determination process by the filter coefficient calculation method second determination unit 204.

  The filter coefficient calculation method second determination unit 204 includes a reference area coordinate estimation unit 205, a maximum similarity storage unit 206, a filter coefficient calculation method determination unit 208, and a threshold storage unit 209.

  The reference area coordinate estimation unit 205 receives the encoding target signal as input, performs a process for estimating the reference area, and writes a coordinate value for identifying the reference area in the reference area coordinate storage unit 207. In addition, an area for maximizing the similarity is obtained at the time of estimation, and the maximum value of the obtained similarity is written in the maximum similarity storage unit 206 at that time.

  The filter coefficient calculation method determination unit 208 receives the maximum similarity read from the maximum similarity storage unit 206 and the threshold read from the threshold storage unit 209, and performs a process of determining whether the maximum similarity is greater than the threshold. , Output a boolean value. If the output is a false value, the process moves to reading the default coefficient. On the other hand, if the output is a true value, the process proceeds to processing for calculating the filter coefficient. At this time, the coordinate value of the reference area read from the reference area coordinate storage unit 207 is input, and a signal corresponding to the coordinate value is read from the decoded signal storage unit 211 and written to the reference signal storage unit 212.

  The processes after the default coefficient storage unit 210 and the filter coefficient calculation unit 213 are the same as the processes after the default coefficient storage unit 103 and the filter coefficient calculation unit 108 in FIG. 6 in the Explicit mode.

[Decryption process]
An embodiment (Explicit mode) of the present invention will be described with reference to the drawings. FIG. 8 is a flowchart of the decrypting process in the Explicit mode according to the embodiment of the present invention.

  In step S40, the encoded information is input, the information included in the encoded information is analyzed, it is determined whether the filter coefficient is estimated from the decoded signal, or the default coefficient prepared as a default is used, and the determination result is output. To do. When the determination result indicates that the filter coefficient is estimated from the decoded signal, the coordinate value indicating the reference area is extracted from the encoded information and output.

  When the default coefficient is used as the filter coefficient, the process proceeds to step S41, the default coefficient is read, and then the process proceeds to step S46.

  When the filter coefficient is estimated from the decoded image, the process proceeds to step S42, where the coordinate value indicating the reference area included in the encoded information is input, the reference signal indicated by the coordinate value is read, and the reference signal is output. The subsequent steps S43 to S46 are the same as the steps S13 to S16 in FIG.

  In step S47, the up-sampling signal output in step S46 and the decoded signal for the difference signal (predictive residual signal) between the up-sampling signal and the signal to be encoded are input, and both are added and encoded. A decoded signal of the target signal is output.

  In step S48, it is determined whether or not processing has been completed for all blocks, and the processing in steps S40 to S47 is repeated until processing is completed for all blocks.

  An embodiment (Implicit mode) of the present invention will be described with reference to the drawings. FIG. 9 is a flowchart of a decoding process in the Implicit mode according to the embodiment of the present invention.

  In step S50, the quantization parameter used for encoding the lower layer reference signal included in the encoding information is input, and it is determined whether or not the parameter is equal to or greater than a threshold value. Is output, otherwise a false value is output. When the true value is output, the process proceeds to step S54 and proceeds to an upsampling process using a default coefficient prepared as a default. On the other hand, if a false value is output, the process proceeds to a second determination process after step S51.

  The subsequent steps S51 to S58 are the same as the steps S21 to S28 in FIG. The process in step S59 is the same as the process in step S47 in FIG. Finally, in step S60, it is determined whether or not processing has been completed for all blocks, and the processing in steps S50 to S59 is repeated until processing is completed for all blocks.

  FIG. 10 shows a block diagram of the decryption device (Explicit mode).

  The encoded information analysis unit 300 receives the encoded information, analyzes header information in the encoded information, and a decoded signal of the prediction residual signal, and stores them in the header information storage unit 301 and the prediction residual signal storage unit 318, respectively. Write out. Here, the header information includes discrimination information indicating whether or not to use a default coefficient, and coordinate value information of the reference area only when the default coefficient is not used. A prediction residual signal is a difference signal between a prediction signal generated by upsampling a lower layer reference signal and a signal to be encoded.

  The filter selection unit 302 receives the discrimination information in the header information read from the header information storage unit 301 as an input, and branches to either a process that uses a default coefficient or a process that estimates a filter coefficient according to the discrimination information. To do.

  The functions of the reference signal storage unit 306 to the upsampling signal storage unit 316 are the same as the functions of the reference signal storage unit 106 to the upsampling signal storage unit 116 of FIG.

  The decoded signal generation unit 317 performs processing of adding both of the upsampling signal read from the upsampling signal storage unit 316 and the decoded signal of the prediction residual signal read from the prediction residual signal storage unit 318 as input. The decoded signal of the signal to be converted is output. The output decoded signal is also written to the decoded signal storage unit 307.

  FIG. 11 shows a block diagram of the decoding device (Implicit mode).

  The encoded information analysis unit 400 receives the encoded information, analyzes header information in the encoded information and a decoded signal of the prediction residual signal, and stores them in the header information storage unit 401 and the prediction residual signal storage unit 423, respectively. Write out. Here, the header information includes the quantization parameter of the lower layer reference signal. A prediction residual signal is a difference signal between a prediction signal generated by upsampling a lower layer reference signal and a signal to be encoded.

  The filter coefficient calculation method first determination unit 402 receives the threshold value read from the quantization parameter threshold value storage unit 403 and the quantization parameter for the lower layer reference signal read from the header information storage unit 401 as input. Processing to determine whether or not, and outputs a truth value. When the output is a true value (when the parameter is larger than the threshold value), the process moves to a process of reading a default coefficient. On the other hand, if the output is a false value, the process proceeds to the determination process of the filter coefficient calculation method second determination unit 404.

  The functions of the filter coefficient calculation method second determination unit 404 to the upsampling signal storage unit 421 are the same as the functions of the filter coefficient calculation method second determination unit 204 to the upsampling signal storage unit 221 of FIG.

  The decoded signal generation unit 422 receives the upsampling signal read from the upsampling signal storage unit 421 and the decoded signal of the prediction residual signal read from the prediction residual signal storage unit 423 as inputs, and adds the both. The decoded signal of the signal to be converted is output. The output decoded signal is also written to the decoded signal storage unit 411.

  The encoding process and the decoding process described above can be realized by a computer and a software program, and the program can be provided by being recorded on a computer-readable recording medium or provided via a network. is there.

It is a figure explaining the interpolation of a horizontal direction. It is a figure which shows the relationship between an encoding target signal and a reference signal. It is a flowchart of the encoding process at the time of Explicit mode. It is a figure for demonstrating the process which selects the filter coefficient calculation method. It is a flowchart of the encoding process at the time of Implicit mode. It is a block diagram of an encoding device (Explicit mode). It is a block diagram of an encoding device (Implicit mode). It is a flowchart of the decoding process at the time of Explicit mode. It is a flowchart of the decoding process at the time of Implicit mode. It is a block diagram of a decoding apparatus (Explicit mode). It is a block diagram of a decoding apparatus (Implicit mode).

Explanation of symbols

101, 201 Encoding target signal storage unit 102 Filter coefficient calculation method selection unit 103, 210, 303, 410 Default coefficient storage unit 104, 205, 405 Reference region coordinate estimation unit 105, 207, 305, 407 Reference region coordinate storage unit 106 , 212, 306, 412 Reference signal storage unit 107, 211, 307, 411 Decoded signal storage unit 108, 213, 308, 413 Filter coefficient calculation unit 109, 214, 309, 414 Autocorrelation calculation unit 110, 215, 310, 415 Autocorrelation storage unit 111, 216, 311, 416 Cross correlation calculation unit 112, 217, 312, 417 Cross correlation storage unit 113, 218, 313, 418 Simultaneous equation solving unit 114, 219, 314, 419 Filter coefficient storage unit 115, 220,315,420 up Sampling signal generation unit 116, 221, 316, 421 upsampling signal storage unit 117, 222 prediction residual signal calculation unit 202, 402 filter coefficient calculation method first determination unit 203 quantization parameter storage unit 204, 404 filter coefficient calculation method Two determination units 206, 406 Maximum similarity storage unit 209, 409 Threshold storage unit 208, 408 Filter coefficient calculation method determination unit 300, 400 Encoding information analysis unit 301, 401 Header information storage unit 302 Filter selection unit 304 Reference area coordinate reading 318, 423 Prediction residual signal storage unit 403 Quantization parameter threshold storage unit

Claims (14)

In a video encoding method for predicting and encoding a higher spatial resolution upper layer signal from a lower spatial resolution lower layer signal using interpolation processing,
Designating a spatio-temporal position of a locally decoded image previously encoded and decoded when generating an interpolation signal from a lower layer reference signal as a higher layer prediction signal;
Calculating a filter coefficient that minimizes a prediction error of an upper layer prediction signal in the local decoded image for generating an interpolated image for each predetermined block using a reference signal at a specified position;
Performing an upsampling process on the basis of the calculated filter coefficient and inputting a decoded signal of a lower layer reference signal as an input, and outputting an upsampling signal for an upper layer encoding target signal;
And a step of generating and outputting a prediction residual signal from the difference between the encoding target signal and the up-sampling signal. The inter-layer predictive encoding method according to claim 1,
An inter-layer predictive encoding method characterized in that an encoding parameter for identifying a spatio-temporal position of a locally decoded image to be referred to when calculating the filter coefficient is output as a part of encoding information. The inter-layer predictive encoding method according to claim 1,
Information indicating a spatio-temporal position of a local decoded image referred to when calculating the filter coefficient is determined based on a local decoded signal. In the inter-layer prediction encoding method according to claim 3,
Depending on the quantization parameter of the lower layer reference signal, an upsampling process is performed using the filter coefficient estimated by the calculation of the filter coefficient, or an upsampling process is performed using a filter coefficient determined as a default in advance. An inter-layer predictive coding method characterized by adaptively switching between. In a video encoding apparatus that predicts and encodes a higher spatial resolution upper layer signal from a lower spatial resolution lower layer signal using interpolation processing,
Means for designating a spatio-temporal position of a locally decoded image encoded and decoded in the past when generating an interpolation signal from a lower layer reference signal as an upper layer prediction signal;
Means for calculating a filter coefficient for minimizing a prediction error of an upper layer prediction signal in the local decoded image for generating an interpolated image for each predetermined block using a reference signal at a specified position;
Based on the calculated filter coefficient, a means for performing an upsampling process using a decoded signal of a lower layer reference signal as an input, and outputting an upsampling signal for an upper layer encoding target signal;
An inter-layer prediction encoding apparatus comprising: means for generating and outputting a prediction residual signal from a difference between the encoding target signal and the upsampling signal. In a moving picture decoding method for obtaining a decoded image signal by adding a prediction signal obtained by interpolating a higher layer signal having a higher spatial resolution from a lower layer signal having a lower spatial resolution using an interpolation process, and a prediction residual signal,
Designating a spatio-temporal position of a locally decoded image decoded in the past when generating an interpolation signal from a lower layer reference signal as an upper layer prediction signal;
Calculating a filter coefficient that minimizes a prediction error of an upper layer prediction signal in the local decoded image for generating an interpolated image for each predetermined block using a reference signal at a specified position;
Performing an upsampling process on the basis of the calculated filter coefficient, using the decoded signal of the lower layer reference signal as an input, and outputting an upsampling signal for the upper layer decoding target signal;
An inter-layer predictive decoding method comprising: adding a prediction residual signal decoded from encoded information and the upsampling signal to generate and output a decoded image signal of an upper layer. The inter-layer predictive decoding method according to claim 6,
An inter-layer predictive decoding method, characterized in that an encoding parameter for identifying a spatio-temporal position of a locally decoded image referred to when calculating the filter coefficient is extracted from encoding information. The inter-layer predictive decoding method according to claim 6,
Information indicating a spatio-temporal position of a local decoded image referred to when calculating the filter coefficient is determined based on a local decoded signal. The inter-layer predictive decoding method according to claim 8,
Depending on the quantization parameter of the lower layer reference signal, an upsampling process is performed using the filter coefficient estimated by the calculation of the filter coefficient, or an upsampling process is performed using a filter coefficient determined as a default in advance. An inter-layer predictive decoding method characterized by adaptively switching between. In a moving picture decoding apparatus that obtains a decoded image signal by adding a prediction signal obtained by interpolating an upper layer signal having a high spatial resolution from a lower layer signal having a low spatial resolution using an interpolation process, and a prediction residual signal,
Means for designating a spatio-temporal position of a locally decoded image decoded in the past when generating an interpolation signal from a lower layer reference signal as an upper layer prediction signal;
Means for calculating a filter coefficient for minimizing a prediction error of an upper layer prediction signal in the local decoded image for generating an interpolated image for each predetermined block using a reference signal at a specified position;
Based on the calculated filter coefficient, a means for performing upsampling processing with the decoded signal of the lower layer reference signal as input, and outputting an upsampling signal for the upper layer decoding target signal;
An inter-layer prediction decoding apparatus comprising: means for adding a prediction residual signal decoded from encoded information and the up-sampling signal to generate and output a decoded image signal of an upper layer.   An inter-layer prediction encoding program for causing a computer to execute the inter-layer prediction encoding method according to any one of claims 1 to 4.   An inter-layer predictive decoding program for causing a computer to execute the inter-layer predictive decoding method according to any one of claims 6 to 9.   A computer-readable recording medium on which an inter-layer prediction encoding program for causing a computer to execute the inter-layer prediction encoding method according to any one of claims 1 to 4 is recorded.   A computer-readable recording medium on which an inter-layer predictive decoding program for causing a computer to execute the inter-layer predictive decoding method according to any one of claims 6 to 9 is recorded.

Images