JPH06217291A - System for sub-band encoding between motion compensation frames - Google Patents

Abstract

PURPOSE:To enable the sub-band encoding between motion compensation frames with high pixel precision by performing inter-frame prediction and in-frame encoding for data in a frequency domain. CONSTITUTION:An input image 1 is compared with the image of a preceding frame stored in a frame memory 9 and a 1st-stage sub-band division filter processing part 10 filters the image of the preceding frame stored in the frame memory 9 in the unit of macroblock corresponding to a shift quantity by using a motion vector obtained by calculation. Then a 2nd-(N)th-stage sub-band division filter processing part 11 performs normal sub-band division. The filter output of a final stage, i.e., a prediction filter output coefficient 3 is compared with the filter output coefficient obtained as a result of dividing in sub bands from the input image 1 by the sub-band division of a 1st-(N)th-stage sub-band division filter processing part 2 and when the power of a difference filter coefficient 4 between motion compensation frames is smaller than the power of input data, the coefficient 3 is handled as an object of encoding in inter-frame difference mode.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a high-efficiency coding system for moving pictures using motion-compensated interframe prediction and subband division.

[0002]

2. Description of the Related Art Motion-compensated interframe prediction is known to ensure high coding efficiency in moving picture coding. Discrete Cosine Transfor
m, hereinafter referred to as DCT) hybrid coding combined with an orthogonal transform such as CCITT is a typical example.
It has been adopted as an international standard encoding method such as H.261 and CCIR Rec.723. The DCT has excellent characteristics for compressing the spatial redundancy, but has a problem of giving block distortion at a low bit rate.

In recent years, wavelet and QMF (Quadatur
Sub-band coding by a multi-stage filter bank as represented by the e mirr filter) has been studied.

As a method of constructing hybrid coding in which motion-compensated inter-frame prediction and a multi-stage filter bank are combined, two types are conceivable: a "type after performing band division after difference" and a "type for obtaining difference after band division". .

The former is a conventional hybrid coding system using motion compensation, in which the DCT is replaced with a multi-stage filter bank. Motion vector detection is performed using the input image and the previously encoded previous frame image. Motion-compensated interframe prediction is performed on the input image, and the difference image is divided into subbands. In this type, motion vector detection / prediction is usually performed in block units using the block matching method. Therefore, when the motion vector changes between adjacent blocks, the prediction signal and the prediction error signal often have a sharp level conversion at the block boundary. This change is DCT
Although it is not a problem for transforms that are closed within blocks, such as sub-band coding that uses a filter that does not close blocks leads to a reduction in coding efficiency.

In consideration of this point, the latter type has been conventionally used (H. Gharavi: “Subband Coding Algo
rithm for Video Applications: Videophone to HDTV C
onferencing ", IEEE Trans., CAS for Video Technolog
y, Vol.1, No.2, pp.174-182, June 1991).

The latter type has a hybrid structure in which an input signal is band-divided and motion-compensated interframe prediction is performed for each subband. The input image is directly subband-divided and converted into frequency domain data. Motion vector detection is performed by comparing the previously encoded frequency domain data with the subband output of the input image. Motion vector detection is performed independently for each sub-band, or motion detection is performed only on the lowest band image, and the motion vector obtained here is also applied to other bands. To be In this process, motion vector detection and motion compensation are performed on the subsampled data after filter output. Therefore, the motion vector detection accuracy and the inter-frame prediction accuracy are lower than the processing in the original image area. For example, when using a three-stage two-dimensional full-division filter bank, motion detection with 1-pixel precision in the divided image corresponds to 8-pixel precision in the original image area. Therefore, only coarse motion compensation accuracy is equivalently obtained, which causes a decrease in inter-frame prediction efficiency.

[0008]

The conventional motion-compensated interframe prediction subband coding has the following problems.

1. In the case of a hybrid configuration in which sub-band division is performed after motion compensation frame difference, conventional motion compensation is processed block by block. Therefore, when the motion vector changes between adjacent blocks, the prediction signal and the prediction error signal often have a sharp level change at the block boundary. Normally, the subband filter order and subsampling ratio do not match the motion compensation block size. Therefore, the subband filtering converts a discontinuous signal generated by motion compensation into frequency domain data, which causes an increase in high frequency components and consequently reduces coding efficiency.

2. In the case of the hybrid configuration in which the difference between motion-compensated frames is obtained after sub-band division, the input signal is band-divided and the motion-compensated inter-frame prediction is performed for each sub-band. Since a prediction loop is formed for frequency domain data, there is no problem of converting motion compensation discontinuities. However, since the motion vector detection and the motion compensation are processed on the data subsampled after the filter output, the motion vector detection accuracy and the inter-frame prediction accuracy are lower than the processing in the original image area. In the case of 2-division subband filtering with n multi-stage configuration, the motion compensation accuracy deteriorates to 1/2 ⁿ at the final stage.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide motion compensation in a hybrid coding system in which motion-compensated interframe prediction and a multistage subband division filter are combined. It is possible to obtain a motion-compensated sub-band filter output that removes the deterioration of sub-band coding efficiency due to discontinuity at a block boundary and maintains the accuracy of a motion vector detected for an image size without sub-sampling. To provide possible technology.

The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.

[0013]

In order to achieve the above object, the present invention suppresses temporal redundancy of an input digital image signal sequence using motion-compensated interframe prediction,
In a high-efficiency coding method for a moving image in which spatial redundancy is suppressed by a multistage filter bank such as a wavelet transform, an input image is divided into K × K pixel blocks, and the coded image stored in a frame memory is input. Between images,
The amount of motion between two screens, which is the target of inter-frame prediction, is detected for each block, or the amount of motion for each block is detected in advance between original images, and each block is detected for the corresponding encoded image in the frame memory. The filter output coefficient based on the motion-compensated position is obtained by performing the band division filtering of the first stage with the position shifted by the position corresponding to the motion amount as the starting point, and the band division of the second and subsequent stages is performed. Performs band division on the obtained first-stage filter output regardless of the amount of motion or block, and finally obtains the filter output coefficient as an inter-frame prediction filter coefficient in the frequency domain. , The input image is subjected to normal multi-stage band division regardless of the block or motion amount to obtain the filter output coefficient, and the inter-frame prediction filter coefficient is subtracted from this coefficient to reduce the frame rate. Seeking beam interframe difference coefficients, we quantize the difference coefficient between the obtained frames to the most important feature to transmit coded.

[0014]

According to the above-mentioned means, of the multi-stage sub-band division, the filter processing required for the sub-band division is executed while performing the motion compensation prediction during the band division of the first stage, and the normal processing is performed after the second stage. By repeating the division by filtering, inter-frame prediction or intra-frame encoding is performed on the data in the frequency domain, so that a motion-compensated inter-frame sub-band encoding method with high pixel accuracy can be realized. In addition, local intra-frame coding becomes possible, and block steps in the predicted image due to blocking in motion compensation are eliminated. As a result, the coding efficiency is improved and the coding noise on the visually impaired block is eliminated.

That is, Q of wavelet or multistage structure
In the case of the MF filter, in order to perform motion compensation and filtering in units of macroblocks, the position shifted by the amount of motion vector for each macroblock may be used as the reference of the filter operation start point. That is, an offset is added by the amount corresponding to the motion vector as compared with the normal case.

The motion compensation output obtained by this method is
This is data in the frequency domain, and when it is returned to the time domain, the image generated from the filter output of a certain macroblock is superimposed on the image generated from the adjacent macroblock data.

For example, in a wavelet in which the low frequency component is repeatedly divided into two, the filter order and the motion vector may be shifted at the filter operation start point, and the output data may be accumulated every two pixels. Here, a big difference from the conventional motion compensation after subband division is that a motion vector is used for each macroblock size defined from the viewpoint of the original image. Therefore, unlike the conventional method, the motion compensation filtering output in which the motion detection accuracy of the original image is reflected can be obtained. Further, since the output data of the first stage has already been motion-compensated, it is not necessary to consider the motion vector in the division of the second and subsequent stages, and the redivision may simply be repeated.

[0018]

First, the principle of the present invention will be described. A typical example of a filter bank based on a tree structure in which band division processing is repeated is an octave division filter having a structure in which two divisions shown in FIG. 1 are repeated on the low frequency side. In FIG.
H ₁ , H ₂ , and H ₃ are low-pass filters, G ₁ , G ₂ , and G ₃ are high-pass filters, and ↓ 2 is 2: 1.

QMF is a typical band division filter, but the orthogonality of the analysis and synthesis filter, the perfection of reconstruction,
In terms of feasibility with a finite order, CQF (Conjugate Quadrat)
ureFilter), SSKF (Symmetric Short Kernel)
There are some variations such as Filter). Wavelet filters are also included in this class.
Not only uniform band division but also non-uniform division is possible by using a tree-structured filter bank with multi-stage decimation and interpolation. A filter bank having a level equivalent to the high-speed conversion processing of the uniform filter bank can be configured in terms of the amount of calculation.

The conventional motion compensation is processed in units of K × K pixel macroblocks on the original image. An operation obtained as a result of introducing processing accuracy equivalent to this to the first stage of multistage subband coding configured by a multistage filter will be described.

The input signal is x (i) (i = 0, 1, ...,
T), the coefficient of the Nth-order subband filter is h (i) (i =
0, 1, ..., N-1). The motion vector belonging to the j-th macroblock is v (j), and the time of the input / output signal is n.
(NεM (j)). However, M (j) represents a time index group belonging to macroblock j.

A motion-compensated output signal y (n) is obtained by shifting the input signal x (n) by the number of samples v (j) corresponding to the motion vector and convolving it with the filter h (n). This is expressed by the following equation (1).

[0023]

[Equation 1]

By Z-transforming equation (1),

[0025]

[Equation 2]

Is obtained. When H (z) is a half band filter, the signal D (z ² ) decimated to 2: 1 is

[0027]

[Equation 3]

[0028]

On the other hand, in the conventional method, decimation is performed first, and motion compensation is performed in the frequency domain. Data F (z ² ) that has been motion-compensated after decimation, if the motion vector is w (j),

[0030]

[Equation 4]

It becomes The condition that equations (3) and (4) match is

[0032]

## EQU5 ## v (j) = 2w (j) (5). When w (j) is a 1-pixel precision motion vector, v (j) is 2-pixel precision. Therefore, in the conventional method, it is understood that the motion compensation accuracy after the first-stage decimation is only half that of the original signal area and the efficiency is low. On the contrary, by using the method of the present invention (Equation (1)), it can be seen that motion compensation with higher accuracy than the conventional method can be realized.

The output of the first stage filter of the band division filter is
Since the data is motion-compensated, normal two-division filtering may be repeated for the low frequency components after the second stage. The structure of the motion compensation filter bank is shown in FIG. In FIG. 2, H ₁ , H ₂ and H ₃ are low pass filters, G ₁ , G ₂ and G ₃ are high pass filters, and ↓ 2 is 2: 1.
It shows that it is scattered.

Here, the case of a one-dimensional signal has been described for simplicity. The actual image signal is a two-dimensional signal,
When band division is performed using a divisible filter, it is necessary to perform motion compensation in the first stage of both the horizontal separation filter and the vertical separation filter. Usually, the movement (shift amount) is represented by a vector v (j) = (v _h (j), v _v (j)) including horizontal and vertical components. However, in each of the first-stage filters, the shift amount is not v _h (j) and v _v (j), but both v _h (j) + Lv _v
(j) (L: number of horizontal pixels).

An embodiment of the present invention will be described in detail below with reference to the drawings.

FIG. 3 is a block diagram showing the structure of a motion compensation interframe subband encoder in one embodiment of the motion compensation interframe subband encoding system of the present invention.

In FIG. 3, 1 is an input image and 2 is 1 to N.
Stage sub-band division filter processing unit, 3 is a prediction filter output coefficient, 4 is a motion compensation inter-frame difference filter coefficient, 5
Is a quantizer, 6 is a variable length encoder, 7 is an inverse quantizer, 8
Is a 1 to N-stage subband synthesis filter processing unit, 9 is a frame memory, 10 is a first stage motion compensation subband division filter processing unit, 11 is a 2-N stage subband division filter processing unit, and 12 is a motion vector detection It is a department.

Next, the operation of the motion-compensated interframe subband encoder of this embodiment will be described with reference to FIG. The input image 1 is compared with the image of the previous frame stored in the frame memory 9, and the motion vector is calculated in the motion vector detection unit 12. By using this motion vector,
In the stage sub-band division filter processing unit 10, the image of the previous frame stored in the frame memory 9 is filtered in macro block units according to the shift amount. Subsequently, normal subband division is executed in the second to Nth subband division filter processing units 11. The filter output of the final stage, that is, the prediction filter output coefficient 3 is compared with the filter output coefficient obtained as a result of subband division of the input image 1 in the 1 to N-stage subband division filter processing unit 2, and the difference between motion compensation frames is calculated. When the power of the filter coefficient 4 is smaller than the power of the input data (filter output), it is treated as an encoding target in the interframe difference mode.

If the opposite is true, the input data is taken over as the object of encoding and is passed on to the next processing. This switching can be performed in macroblock units. The interframe difference or the filter output within the frame is quantized in the quantizer 5. The quantization number is entropy coded by the variable length encoder 6 and transmitted to the receiving side. Further, like the normal interframe coding, it is necessary to store data in the coding loop for comparison with the filter output of the next frame. Therefore, the quantized data is returned to the quantized value in the inverse quantizer 7. The quantized value is subjected to the first stage motion compensation sub-band division filter processing (10) and further the 2 to N-stage sub-band division filter processing (11) on the image of the previous frame stored in the frame memory 9. It is added to the prediction filter output coefficient 3 thus obtained. The added coefficient is converted into an encoded image in the 1 to N-stage subband synthesis filter processing unit 8 and stored in the frame memory 9.

Although the present invention has been specifically described based on the embodiments, it is needless to say that the present invention is not limited to the above embodiments and various modifications can be made without departing from the scope of the invention. Absent.

[0041]

As described above, according to the present invention, a motion-compensated interframe subband coding system with high pixel accuracy is realized in multistage subband coding such as motion-compensated interframe prediction and wavelet transform. As a result, local intra-frame coding is possible, and the block difference in the predicted image due to blocking in motion compensation is eliminated. As a result, the coding efficiency is improved and the coding noise on the visually impaired block is eliminated.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of a three-stage octave bank for explaining the principle of the present invention,

FIG. 2 is a block diagram showing the configuration of a motion compensation filter bank for explaining the principle of the present invention;

FIG. 3 is a block diagram showing a configuration of a motion compensation interframe subband encoder in an embodiment of the motion compensation interframe subband encoding system of the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Input image, 2 ... 1-N stage sub-band division filter processing part, 3 ... Prediction filter output coefficient, 4 ... Motion compensation inter-frame difference filter coefficient, 5 ... Quantizer, 6 ... Variable length coder, 7 Inverse quantizer, 8 ... 1 to N-stage subband synthesis filter processing unit, 9 ... Frame memory, 10 ... First stage motion compensation subband division filter processing unit, 11 ... 2 to N stage subband division filter processing Part, 12 ... Motion vector detecting part.

Claims (1)

[Claims] 1. A method for suppressing temporal redundancy of an input digital image signal sequence using motion compensation interframe prediction,
In a high-efficiency coding method for a moving image in which spatial redundancy is suppressed by a multistage filter bank such as a wavelet transform, an input image is divided into K × K pixel blocks, and the coded image stored in a frame memory is input. Between images,
The amount of motion between two screens, which is the target of inter-frame prediction, is detected for each block, or the amount of motion for each block is detected in advance between original images, and each block is detected for the corresponding encoded image in the frame memory. The filter output coefficient based on the motion-compensated position is obtained by performing the band division filtering of the first stage with the position shifted by the position corresponding to the motion amount as the starting point, and the band division of the second and subsequent stages is performed. Performs band division on the obtained first stage filter output regardless of the amount of motion or block, and finally obtains the filter output coefficient as an inter-frame prediction filter coefficient in the frequency domain. The input image is subjected to normal multi-stage band division irrespective of the block and motion amount to obtain the filter output coefficient, and the inter-frame prediction filter coefficient is subtracted from this coefficient to obtain the frame. Calculated between the difference coefficients, the quantized difference coefficients between the obtained frame motion compensation interframe subband coding scheme, wherein the transmission coding.