JP3169147B2 - High-efficiency coding device for video data - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an improvement of a high-efficiency coding apparatus for moving image data using motion compensation inter-frame prediction and subband division / synthesis filter band.

[0002]

2. Description of the Related Art It is known that motion-compensated inter-frame prediction guarantees high coding efficiency in video coding. Discrete Cosine Transform; D
Hybrid coding in combination with orthogonal transforms such as C.I. 261,
It has been adopted as an international standard encoding system such as CCIR recommendation 723 and ISO / MPEG. FIG. 6 is a basic block diagram showing the configuration of an encoding device of these international standard encoding systems.
Shown in

In FIG. 6, reference numeral 101 denotes digitized input image data, 102 denotes a block determiner, 103 denotes an intra-frame / inter-frame switch, 104 denotes an orthogonal transformer, 105 denotes a quantizer, and 106 denotes entropy coding. , 107 is an inverse quantizer, 108 is an inverse orthogonal transformer, 10
9 is an adder, 110 is a frame memory, 111 is a motion compensation predictor, 112 is a motion detector, 113 is a loop filter, and 114 is a subtractor.

The operation of the encoding apparatus shown in FIG. 6 will be described below. The digital input image data 101 divided into small blocks is guided to a motion detector 112, and is compared with past locally decoded image data read from the frame memory 110, so that the motion amount of each small block is reduced. Detected as a vector.

[0005] The local decoded image data read from the frame memory 110 is compensated for motion by the motion vector output from the motion detector 112, becomes predicted image data, and is input to the loop filter 113. After the noise component is removed here, the subtractor 114 extracts the inter-frame difference image data from the input image data 101.

The block decision unit 102 compares the input image data 101 with the inter-frame difference image data from the subtractor 114, and outputs a command signal for selecting the smaller one of the power sums in the block. 103
And switches the switches SW1 and SW2 therein in an interlocked manner.

The image data selected by the switch SW 1 is orthogonally transformed by an orthogonal transformer 104, and is quantized by a quantizer 105.
Is quantized by The quantization index output from the quantizer 105 is encoded into binary data by the entropy encoder 106, and is output as final encoded data.

On the other hand, the quantization index output from the quantizer 105 is returned to the original transform coefficient value by the inverse quantizer 107, and is inversely transformed by the inverse orthogonal transformer 108. The inversely converted data is added to the image data selected by the switch SW2 by the adder 109, and stored in the frame memory 110 as locally decoded image data. The above is the outline of the operation of the encoding device shown in FIG.

Although the DCT exhibits excellent performance in suppressing the spatial redundancy of an image signal, the transmission bit rate is low because the basis function used in the DCT (discrete cosine transform) is completely closed to the block to be transformed. When it is low (that is, when the data compression ratio is increased), there is a disadvantage that discontinuous distortion occurs at the boundary between blocks.

[0010] In a case where an edge and a flat portion are mixed as a picture in one transform block, the DCT coding converts high-frequency components into low-frequency components and high-frequency components. In order to reduce transmission information,
If it is discontinued, mosquito noise (deterioration of image quality) generated by that will also be a problem.

Recently, half-band filter banks such as a quadrature mirror filter (QMF), LOT (Lapped Orthogonal Tran
Subband coding using a parallel filter bank represented by sform) has been widely studied.

In these filter banks, block distortion which is a problem in DCT coding does not occur because the basis overlaps with an adjacent block. Furthermore, since the frequency separation characteristics are superior to those of the orthogonal transform, the degree of power concentration on a specific band (Energy Compactio
n) is high.

In the Wavelet transform having a structure in which only the low-frequency sub-band is recursively divided (octave-divided) using QMF that satisfies a certain special condition, compared to DCT and other filter banks, the wavelet transform has a higher Since the length is shortened, the spatial diffusion of the quantization noise is suppressed, and the effect of reducing the mosquito noise is also obtained.

Therefore, CCITT Recommendation H. By replacing the orthogonal transform unit of the intra-frame / motion-compensated inter-frame adaptive prediction represented by H.261 or the like and DCT coding with LOT, QMF bank, and Wavelet transform, it is possible to improve coding efficiency and subjective image quality. However, when combining a conventional hybrid coding configuration (post-differential transform) with a filter bank whose bases overlap,

(A) Since motion compensation is performed for each block, when a motion vector changes between adjacent blocks, a steep change in the level occurs at the macroblock boundary in the prediction signal and the prediction error signal. I do.

(B) Since intra-frame / motion-compensated inter-frame adaptive prediction is also performed for each macroblock, the adaptive prediction error signal includes a macro-block boundary due to the difference in the average value between the intra-frame block and the motion-compensated inter-frame block. In the section, a sharp change in level occurs.

Therefore, a discontinuous signal generated by motion compensation in macroblock units and adaptive prediction within a frame / motion compensation frame is divided into bands, resulting in an increase in high frequency components and consequently an increase in coding efficiency. Lower.

In order to solve the problem of the level difference, a document (Nyosawa, Watanabe: "Frequency domain realization of motion compensation prediction in subband coding", 1992 IEICE Spring Convention, Article No. D-302) Has proposed a motion compensation filter bank structure in which motion compensation prediction and intra-frame / inter-frame switching are performed in the frequency domain after band division, and this method has also been applied as a patent (Japanese Patent Application No. 3-323240, 4-1451).

FIG. 7 shows a block diagram of an encoding device according to the already proposed method having a motion compensation filter bank structure. In the configuration of FIG. 7, as compared with the configuration of FIG. 6, the orthogonal transformer 104 is removed, and at the same time, a band splitter 120 having the same operation as the orthogonal transformer 104 is inserted immediately before the subtractor 123. And the motion compensation predictor 116
22 in that an inverse orthogonal transformer 108 is substituted for the band combiner 121, respectively.

That is, in the configuration of FIG. 7, the input image data is directly divided into bands, and then the difference between frames is obtained (subtractor 123). In the configuration of FIG. After taking the difference, split the band ( orthogonal
( The function of the converter 104 ).

By adopting the hybrid coding configuration of FIG. 7, the input to the filter bank (indicating band splitter 120 and motion compensation band splitter 122) is not an adaptive prediction error signal but an input image signal or local decoding. It becomes an image signal, and the problem of the level difference does not occur essentially.

[0022]

As described above, the adoption of the motion compensation filter bank structure eliminates the level difference that hinders the improvement of the coding efficiency, and increases the LOT, the QMF bank, and the compression efficiency of the Wavelet transform. And features such as good subjective image quality can be utilized.

However, in general, the inter-frame correlation of high-frequency components of an image is low, and the correlation between frames is drastically reduced even in the case of a large motion exceeding the search range of motion compensation. As means for solving the former, that is, means for improving and increasing the decrease in the inter-frame correlation of the image due to the low inter-frame correlation of the high-frequency component of the image, a loop filter ( Means for enhancing the correlation by applying (low-pass filter) and rounding (decreasing) high-frequency components (although the picture is blurred) is often used.

By applying a low-pass characteristic loop filter to the predicted image data in this manner, the correlation between frames is increased, the power of the inter-frame difference signal is reduced, and consequently the coding efficiency is reduced. Is improved.

However, as shown in FIG. 7, when the motion compensation filter bank structure is employed, there are two band division sections (120, 122), and the amount of calculation required for the band division section is the same as that of the conventional hybrid encoding apparatus ( It is twice that of Fig. 6). That is, in consideration of the feasibility of hardware, it is not advisable to add a filtering process that requires a large amount of computation in an encoding device employing a motion compensation filter bank structure as shown in FIG.

On the other hand, intra-frame / inter-frame switching is often used as a means for solving a decrease in prediction efficiency in the case of a fast motion exceeding the search range of motion compensation. In the case of a very fast motion that cannot be followed by motion compensation, the correlation between frames is drastically reduced, and therefore, it is more efficient to perform direct encoding within a frame. Therefore, FIG.
In the conventional hybrid coding apparatus, the coding is performed by performing intra-frame / inter-frame switching for each block of input data, whether to perform direct coding within a frame or to perform coding by taking a difference between frames. We are trying to improve the conversion efficiency.

However, in the conventional apparatus, the intra-frame / inter-frame switching is performed for the signal in the image area in units of blocks. Therefore, as described above, it cannot be said that the fact that the inter-frame correlation of an image is low in a high-frequency component and high in a low-frequency component can be sufficiently utilized.

Further, even if the conventional apparatus performs intra-frame / inter-frame switching for each pixel, the effect is expected because the amount of additional information increases drastically and the correlation information of the frequency domain data cannot be used. Can not. That is, FIG.
The intra-frame / inter-frame switching (performed in the image domain) in the conventional hybrid coding apparatus as shown in (1) does not truly consider the inter-frame correlation for each frequency component of the frequency domain data of the image. As a result, a loss of coding efficiency may occur.

Accordingly, the present invention solves the above-mentioned problems, and in a coding apparatus (coding apparatus as shown in FIG. 7) adopting a motion compensation filter bank structure, a function of a loop filter (low-pass filter) is provided by a dedicated hardware. DCT (Discrete Cosine Transform) of input image data without switching by adding hardware
By efficiently taking into account the inter-frame correlation in the frequency domain data obtained as a result of the above, and further performing this operation adaptively taking into account the local properties of each block, and thereby improving the overall coding efficiency. It is an object of the present invention to provide a high-efficiency video coding apparatus capable of further improving the video quality.

[0030]

According to the present invention, there is provided a moving picture encoding apparatus which divides input digital image data into a plurality of blocks in frame units and performs an encoding process for each block. So,

Means for dividing the input block data into bands, a frame memory for storing past encoded image data as locally decoded image data,
Motion detection means for detecting and outputting a motion vector of data (input image), motion compensation band division means for outputting motion-compensated inter-frame prediction subband coefficient data, and obtaining motion-compensated inter-frame difference coefficient data Subtraction means, quantization means, entropy coding means, in the moving picture coding device having

For the data of the entire block, the means for performing the intra-frame / inter-frame switching of directly encoding the data in the frame or encoding by taking the difference between the frames is eliminated, and instead, Prior to inputting the motion-compensated inter-frame prediction sub-band coefficient data obtained by the motion-compensation band dividing means to the subtraction means, for each sub-band coefficient data, either encode it directly in the frame or A means for performing coding by taking the difference between the frames or performing intra-frame / frame-to-frame switching is provided.

[0033]

In the coding apparatus employing the motion compensation filter bank structure (the coding apparatus as shown in FIG. 7), the band division processing itself includes the filter processing, and the high frequency band of the motion compensation inter-frame prediction subband coefficient data is included. Truncation of the components in the filtering is equivalent to suppressing (using a low-pass filter) the high-frequency components of the predicted image data. In addition, this is nothing other than directly encoding the high-frequency components of the encoding target block in the frame.

Therefore, the loop filter processing for the prediction image data and the intra-frame / inter-frame switching for each block are integrated by the intra-frame / inter-frame switching for each subband coefficient data.

Therefore, in the method of the present invention, it is not necessary to newly perform a filter operation for removing noise in the predicted image data, and the amount of operation can be reduced as compared with the case where the loop filter processing of the image signal area is performed. Further, the input image data is obtained by DCT (discrete cosine transform).
Optimal intra-frame / inter-frame switching can be performed for each band with respect to data (sub-band coefficient data) expanded in the frequency domain, and each block can be efficiently coded.

Further, the method of the present invention has an advantage that the band used for prediction, that is, switching within a frame or between frames, can be adaptively determined for each block according to the local property of input image data. This adaptive control can be easily realized by appropriately changing the filter coefficient matrix for each block. Therefore, the CCI that can only select whether or not to use a loop filter having a fixed characteristic and that can only switch between intra-frame / inter-frame in block units can be used.
TT Recommendation H. 261 can improve the overall encoding efficiency.

[0037]

Embodiments of the present invention will be described below in detail. FIG. 1 is a block diagram showing a configuration of an encoding device as one embodiment of the present invention. As shown in FIG. 1, in an encoding device according to an embodiment of the present invention, input image data 1
Is obtained by DCT in the band division unit 2 for each N × N block (N is an integer, that is, N pixels × N lines, and when N is 8, blocks composed of 64 × 8 = 64 pixel data). It is converted into subband coefficient data (N × N subband coefficient data) 3.

On the other hand, the local decoded image data 17 of the previous frame stored in the frame memory 16 is input to the motion detecting section 19 together with the input image data 1 and based on the obtained motion vector 20, the motion compensated band dividing section At 18 (D
CT), and is converted into motion-compensated inter-frame prediction subband coefficient data 21.

Here, the intra-frame sub-band coefficient data 3 and the motion-compensated inter-frame predicted sub-band coefficient data 2
1 is compared with each subband by the coefficient determination unit 22.
Intra-frame / inter-frame switching is performed so that the intra-block power is minimized, and intra-frame / inter-frame switching information 23 is output.

Based on the intra-frame / inter-frame switching information 23 sent from the coefficient judging section 22, the motion compensation inter-frame prediction sub-band coefficient data 21 is subjected to intra-frame / inter-frame switching for each band by a coefficient filter 24. (Specifically, execution of whether to multiply a coefficient 0 or 1 for each of the motion-compensated inter-frame prediction subband coefficient data) is performed.

Although the coefficient filter 24 is shown separately from the motion compensation band dividing unit 18 for convenience, it is actually a function originally included in the motion compensation band dividing unit 18. It is not necessary to provide special hardware.

Next, the intra-frame / inter-frame adaptive prediction sub-band coefficient data 25 is subtracted from the intra-frame sub-band coefficient data 3 by the subtractor 4, thereby obtaining motion-compensated inter-frame difference coefficient data 5. The motion compensation inter-frame difference coefficient data 5 is quantized by a quantizer 6,
The obtained quantization index 7 is subjected to variable-length encoding in the entropy encoding unit 8 and sent out to the transmission path as encoded video data 9.

If the coefficient filter 24 multiplies the motion compensation inter-frame prediction sub-band coefficient data 21 by a coefficient 0, the resulting intra-frame / inter-frame adaptive prediction sub-band coefficient data 25 is 0. In the subtractor 4, the intra-frame sub-band coefficient data 3 passes as it is (that is, the intra-frame data) becomes data 5 as it is.

On the other hand, if the coefficient filter 24 multiplies the motion-compensated inter-frame prediction sub-band coefficient data 21 by a coefficient 1, the resulting intra-frame / inter-frame adaptive prediction sub-band coefficient data 25 becomes The motion compensation inter-frame prediction sub-band coefficient data 21 itself, and the subtractor 4
From the motion compensation inter-frame prediction subband coefficient data 2
1 is subtracted, and the inter-frame difference data becomes data 5.

Next, the quantization index 7 is returned to the quantization representative value 11 by the inverse quantizer 10 in order to obtain the local decoded image data 15 to be used for the prediction of the next frame in the encoding device. The intra-frame / inter-frame adaptive prediction sub-band coefficient data 25 is added and sent to the band synthesizing unit 14.

The band synthesizing unit 14 restores the local decoded image data 15 from the received sub-band coefficient data 13 and writes it in the frame memory 16 to prepare for encoding prediction of the next frame. Also, the motion vector 20,
In-frame / inter-frame switching information 23 is also separately encoded and transmitted together with the encoded video data 9.

FIG. 2 is a block diagram showing a configuration of a decoding device corresponding to the encoding device shown in FIG. As shown in FIG. 2, in the decoding device, the encoded data 9
Entropy decoding section 26 performs quantization index 27
And the quantization index 27 is calculated by the inverse quantizer 2
At 8, it is returned to the quantization representative value 29.

Next, the decoded image data 35 of the previous frame stored in the frame memory 34 is converted into motion-compensated inter-frame prediction sub-band coefficient data 37 by a motion-compensation band dividing unit 36 based on the motion vector 20. You. The prediction sub-band coefficient data 37 is switched by the coefficient filter unit 38 for each band based on the intra-frame / inter-frame switching information 23.

The sub-band coefficient data 31 obtained by adding the quantization representative value 29 obtained by the inverse quantization unit 28 and the intra-frame / inter-frame adaptive prediction sub-band coefficient data 39 is input to the band synthesis unit 32. , Decoded image data 3
It becomes 3. Decoded image data 3 obtained by band combining section 32
3 is written to the frame memory 34 and used as predicted image data in decoding the next frame.

Next, in the coefficient filter 24 of FIG.
An example of a setting method of the filter coefficient (1 or 0) for each subband will be described below.

EXAMPLE 1 As shown in FIG. 3, the filter coefficient for the low frequency i × j portion is defined as 1, and the filter coefficient for the remaining high frequency portion is defined as 0. That is, of the motion-compensated inter-frame prediction sub-band coefficient data, the coefficient value of the low band i × j band is used as it is for inter-frame coding, and the coefficient value of the remaining high band is all 0, and the intra-frame direct coding is performed. It will be.

In this case, (i,
j), where, for all combinations of i, j = 1,..., N, the sum of power or the sum of absolute values of motion-compensated inter-frame difference coefficient data is calculated, and the value becomes the smallest (i,
The combination of j) is selected and encoded. In this case, the additional information indicating the intra-frame / inter-frame switching point is 2
log ₂ N (bit / block) is required.

In FIG. 3, N horizontal data and N vertical data are used.
This block is composed of a number of data. When this block is transformed into frequency domain data by DCT (Discrete Cosine Transform), the frequency domain data that is converted and output is converted to the block ( It is well known that the portion corresponding to the upper left corner of (square) is the lowest low-frequency component data, the frequency components are sequentially increased, and the portion corresponding to the lower right corner is the highest high-frequency component data, which is output. It is a matter of.

Example 2 As shown in FIG. 4, according to the rule of zigzag scan (means for converting two-dimensional sub-band coefficient data into a one-dimensional data sequence), bands 1 to k (k
= 1,..., N ² ) are defined as 1, and the remaining high-frequency filter coefficients are defined as 0. That is, it means that low-frequency coefficients from band 1 to band k (k = 1,..., N ² ) are inter-frame coded, and the remaining high-frequency coefficients are directly coded in a frame.

In this case, k = 1, k = 1
, N ² , the power sum or the sum of the absolute values of the motion-compensated inter-frame difference coefficients is calculated, and k having the smallest value is selected for encoding. In this case,
₂ log ₂ N as additional information indicating the switching point between frames
(bit / block) required.

Example 3 As shown in FIG. 5 (example of N = 8),
The filter coefficient is set independently for each band, and switching within a frame or between frames is performed for each sample. In this case, the in-frame sub-band coefficient and the motion-compensated inter-frame difference coefficient are compared for each band, and the one having a smaller absolute value or square value is appropriately selected. However, the amount of overhead information indicating intra-frame / inter-frame for each band is greatly increased as compared with the above two examples, and a maximum of 1 (bit / block) is required.

In the above three examples, when the filter coefficients of all the bands are set to 0, all the predicted subband coefficients become 0, and as a result, all the coefficients in the block are coded in the in-frame direct mode. It is equivalent to If the filter coefficients of all the bands are set to 1, all the coefficients in the block are equivalent to encoding in the inter-frame mode.

[0058]

According to the present invention, the following effects can be expected in the motion compensation inter-frame prediction and the motion compensation filter bank coding using LOT, QMF bank, and Wavelet transform.

(A) Noise reduction filter processing can be performed on predicted image data without increasing the amount of calculation. This requires a much smaller amount of calculation than when filtering is performed on an image area.

(B) Optimum intra-frame / inter-frame switching can be performed for each band with respect to data (sub-band coefficient data) expanded in the frequency domain, and each block is more efficiently coded. can do.

(C) By appropriately changing the filter coefficient matrix, the band to be used for prediction, that is, switching within a frame or between frames, can be adaptively determined for each block in accordance with the local property of input image data. Therefore, it is only possible to select whether or not to use a loop filter having a fixed characteristic, and it is only possible to switch between intra-frame and inter-frame in block units. 261
, The overall coding efficiency can be improved.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an encoding device as one embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a decoding device corresponding to the encoding device shown in FIG.

FIG. 3 is an explanatory diagram showing an example of a coefficient determination criterion in the coefficient filter of FIG. 1;

FIG. 4 is an explanatory diagram showing another example of a criterion for coefficient determination in the coefficient filter of FIG. 1;

FIG. 5 is an explanatory diagram showing still another example of a criterion for coefficient determination in the coefficient filter of FIG. 1;

FIG. 6 is a block diagram illustrating a basic configuration of an encoding device using an international standard encoding method.

FIG. 7 is a block diagram illustrating a configuration of an encoding device according to a previously proposed method having a motion compensation filter bank structure.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Input image data, 2 ... Band division part, 3 ... Subband coefficient data in a frame, 4 ... Subtractor, 5 ... Difference coefficient data between motion compensation frames, 6 ... Quantizer, 7 ... Quantization index, 8 ... Entropy encoder 9, 9 video encoded data 10, inverse quantizer, 11 representative quantization value, 12
Adder, 13: sub-band coefficient data, 14: band synthesis unit, 15: local decoded image data, 16: frame memory, 17: local decoded image data, 18: motion compensation band division unit, 19: motion detection unit, 20 ... motion vector, 21 ...
Motion compensation inter-frame prediction subband coefficient data, 22 ...
Coefficient determination unit, 23 intra-frame / inter-frame switching information, 24 coefficient filter unit, 25 intra-frame / inter-frame adaptive prediction subband coefficient data, 26 entropy decoder, 27 quantization index, 28 inverse Quantizer, 29 Quantized representative value, 30 Adder, 31 Subband coefficient data, 32 Band synthesis unit, 33 Decoded image data, 34 Frame memory, 35 Decoded image data, 36 Motion compensation Band division unit, 37: subband coefficient data between motion compensation frames, 38: coefficient filter unit, 3
9 ... Intra-frame / inter-frame adaptive prediction subband coefficient data.

Continuation of the front page (56) References Hironao Yosawa and Hiroshi Watanabe, "Frequency-domain realization of motion-compensated interframe prediction in equal-division subband coding," IEICE Technical Report, Electronic Information System Communication Society, November 22, 1991, Vol. 91, No. 336, p. 9-16 (IE91-82) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N ^7/ 24-7/68

Claims (1)

(57) [Claims] An input digital image data is divided into blocks of N pixels × N lines (where N is an integer) for each frame, and moving image data of which coding processing is performed for each block data is obtained. A high-efficiency coding apparatus, comprising:
A band dividing means (2) for dividing a band into N × N sub-bands by a × N-band parallel filter bank and outputting it as a block of sub-band coefficient data in a frame; A frame memory (16) for storing an image obtained by restoring image data in an encoding device, that is, a locally decoded image data; a block data input to the band dividing means; and a local memory stored in the frame memory. A motion detecting means (19) for comparing the decoded image data to determine which part best matches, detecting the corresponding block data with respect to the past image, that is, the motion amount of the input image as a motion vector, and outputting the motion vector; Capturing the local decoded image data read from the frame memory and the motion vector information from the motion detecting means. A motion compensated band dividing unit that performs N × N band division starting from a position shifted by the motion vector with respect to the local decoded image data and outputs motion-compensated inter-frame prediction subband coefficient data ( 18), and intra-frame sub-band coefficient data output from the band dividing means and motion-compensated inter-frame predicted sub-band coefficient data obtained from the motion-compensated band dividing means. Subtraction means (4) for obtaining and outputting motion-compensated inter-frame difference coefficient data by subtraction; quantizing means (6) for quantizing the motion-compensated inter-frame difference coefficient data; Coding means (8) for performing variable-length coding on the quantized index, wherein the high-efficiency coding apparatus for moving image data comprises: Prior to inputting the motion-compensated inter-frame prediction sub-band coefficient data obtained from the compensation band dividing means to the subtraction means, a multiplication factor of 1 or 0 is applied to each inter-frame prediction sub-band coefficient data. And then
Coefficient filter means (24) input to said subtraction means
A coefficient judging means for judging whether the weight coefficient is multiplied by 1 or 0 for each inter-frame prediction subband coefficient data and notifying the coefficient filter means of the coefficient filter means ( 22), wherein the intra-frame sub-band coefficient data from the band dividing means and the motion-compensated inter-frame predicted sub-band coefficient data from the motion-compensated band dividing means are fetched.
Intra-frame subband coefficient data from the band dividing means
Data from the motion compensation band dividing means.
Compensated inter-frame prediction subband coefficient data and
Subband coefficient data in the frame from the band dividing means
And the difference between the absolute value and the power.
If the former is smaller than the latter, the weighting factor is 0,
And a coefficient judging means (22) for judging the value as 1 .