JP3175906B2 - Image encoding / decoding method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a high-efficiency encoding / decoding method using a motion compensated inter-frame prediction orthogonal transform.

[0002]

2. Description of the Related Art As a high-efficiency encoding method for an image, there is a method using orthogonal transform represented by discrete cosine transform.

FIG. 3 is a block diagram showing a general orthogonal transform encoding / decoding method. First, the image 202 input from the input terminal 201 is divided into N ×
It is divided into N small blocks. The orthogonal transform unit 204 performs orthogonal transform on each of the divided small blocks, and the obtained orthogonal transform coefficients are quantized with the quantization characteristics determined by the quantization unit 205. Next, the code assignment unit 20
In 6, a code is assigned to the quantized representative value, and the encoded data 207 is transmitted to the receiving side through the output terminal 208. On the other hand, on the receiving side, the code is decoded by the code decoding unit 211 from the coded data 210 received at the reception terminal 209, and the inverse quantization unit 212 reproduces the quantized representative value for each orthogonal transform coefficient. Orthogonal transform unit 213
Perform an inverse orthogonal transform, and output the decoded image data 214 to the output terminal 215.

Generally, a quantization method is as shown in FIG.
When the input value F is F _i ≦ F <F _{i + 1} , the representative quantization value is F
_i ′ (where F _i ≦ F _i ′ <F _{i + 1} ).

In such an orthogonal transform encoding / decoding method, encoding is performed in units of small blocks. Therefore, when a high compression ratio is set, quantization is coarsened, and block noise is added to a decoded image. This causes a problem that image quality deterioration such as the above is detected. Various methods have been considered to avoid this, but multiple constraints are set to be satisfied by the original image, and the decoded image deteriorated by encoding is repeatedly corrected so as to satisfy all these constraints. Then, there is a method of restoring the original image as close as possible (A. Zakhor, “Iterative procedures f
or reduction of blocking effects in transform image
coding, ”IEEE Trans. on Circuits and Systems for V
ideo Technology, vol. 2, No.1, pp.91-95, Mrch 199
2).

In the method described in this document, the constraint conditions are as follows:
Two conditions are used: a condition that the original image is smooth, and a condition that the range in which the true value before quantization exists is limited based on the quantized representative value. The former is achieved by applying a low-pass filter to the image. In the latter case, when the quantized representative value is F _i ′, the true value F before quantization is F _i ≦ F
Since it should exist in < _{Fi + 1} , it may be modified to satisfy this.

Specifically, first, the decrypted signal obtained on the receiving side is
Apply a low-pass filter to the signal. Next, low pass
The orthogonally transformed signal is filtered. Obtained at this time
Orthogonal transform coefficient F_L Is the signal before low-pass filtering.
Coefficient F obtained by orthogonal transformation _iUnlike '
Does not meet the requirements of the elderly. Therefore, the following condition
By F_L To correct.

[0008]

(Equation 1) That is, for a value exceeding the range of the true values, a process of clipping to the closest one of the maximum value or the minimum value in the range of the true values is performed. Inverse orthogonal transformation is performed on the orthogonal transformation coefficient modified by clipping to obtain modified image data. By repeating the above low-pass filter, orthogonal transformation, clipping, and inverse orthogonal transformation a predetermined number of times, a final corrected image can be obtained.

FIG. 5 is a block diagram for executing the above-described method for correcting a decoded image. On the receiving side, the coded data 302 received at the receiving terminal
After the code is decoded in 3, the inverse quantization unit 304 reproduces the quantized representative value for each orthogonal transform coefficient, and the inverse orthogonal transform unit 305 performs the inverse orthogonal transform to obtain the decoded image data 30.
6 is obtained. After the decoded image data 306 is filtered by the low-pass filter unit 307, the orthogonal transform is performed by the orthogonal transform unit 308, and the obtained orthogonal transform coefficients are input to the clipping unit 309. The clipping unit 309 corrects the orthogonal transform coefficient using the maximum and minimum values before quantization obtained by the range setting unit 310 based on the quantization representative value used in the inverse quantization unit 304. The corrected orthogonal transform coefficient is subjected to an inverse transform in an inverse orthogonal transform unit 311 to obtain a first corrected image 312. The corrected image 312 is subjected to a second low-pass filter in a low-pass filter unit 313, subjected to an orthogonal transform in an orthogonal transform unit 314, and the obtained orthogonal transform coefficients are input to a clipping unit 315. In the clipping unit 315, the orthogonal transform coefficient is corrected in the same manner as in the first time, and the corrected orthogonal transform coefficient is subjected to the inverse transform in the inverse orthogonal transform unit 316, and the second corrected image 317 is obtained. Similarly, an n-th corrected image 318 obtained by repeating the process a predetermined number of times
Is finally output to the output terminal 319.

[0010]

Consider the case where the above-described method is applied to an image which has been encoded and decoded by motion-compensated inter-frame difference orthogonal transform in which encoding is also performed using inter-frame correlation.

FIG. 6 shows a block diagram on the transmitting side for performing orthogonal transform coding on a motion compensation inter-Fleum difference signal. First, the image data 402 of the current frame input from the input terminal 401 is divided into small blocks, and the motion of the divided small blocks is determined based on the image data of the past or future frame stored in the frame memory 403. The most similar block is obtained according to the motion vector 405 obtained in the vector detection unit 404. The motion compensation unit 406 obtains motion compensation prediction data 407 from image data of a past or future frame according to the obtained motion vector 405. Using the data 402 of the small block of the current frame and the motion compensation prediction data 407, a difference unit 408 calculates a motion compensation inter-frame difference value 409. Thereafter, either the inter-frame difference value 409 of the small block or the small block pixel value 402 in the current frame itself is selected, and the orthogonal transformation unit 410 performs orthogonal transformation on the selected data for each small block. The obtained orthogonal transform coefficients are quantized by a quantization step width determined by a quantization unit 411 to obtain a quantization level number, and a code allocating unit 412 allocates a code corresponding to the quantization level number and transmits the code. Also, a code is assigned to the motion vector 405 and the inter-frame / intra-frame selection flag 417, and transmitted as data 413 together with the orthogonal transform coefficient encoded data.

On the other hand, the quantization level number is assigned to the inverse quantization unit 41.
After inverse quantization is performed at step 4 to obtain a quantized representative value, an inverse orthogonal transform unit 415 performs inverse orthogonal transform. If the data subjected to the inverse orthogonal transformation is an inter-frame / intra-frame selection flag 4
If 17 is in the frame, the frame memory 4
03, and if the selection flag 417 is between frames, the motion compensation prediction data 407 and the adder 416
Are written in the frame memory 403. These processes are also performed on the receiving side.

As described above, in the motion-compensated inter-frame difference orthogonal transform coding, in a region where motion compensation is appropriately performed, the motion-compensated inter-frame difference value is orthogonally transformed and quantized / encoded. In a region where the inter-frame difference value is large even after the motion compensation, the pixel values in the frame are directly subjected to the orthogonal transformation, quantized and coded.
For the latter, the conventional method can be applied as it is because it can be regarded as intra-frame encoding.However, in the region where the motion compensation inter-frame difference signal is quantized as in the former, the decoding side can know. This is the range of the true values of the orthogonal transform coefficients of the motion-compensated inter-frame difference signal, but not the range of the true values of the orthogonal transform coefficients of the pixel values themselves in the frame. Therefore, it is not appropriate to apply the clipping to the coefficients obtained by orthogonally transforming the image signal after the low-pal filter, but to obtain an appropriate corrected image only by repeating the low-pass filter, orthogonal transformation, clipping, and inverse orthogonal transformation. Can not. For this reason, there is a problem that it is difficult to prevent the deterioration on the decoded image.

An object of the present invention is to provide an image decoding method capable of efficiently removing a noise component contained in a decoded image.

[0015]

According to an image encoding / decoding method of the present invention, a digital image signal such as a television signal is divided on the transmitting side into small blocks of image data of a current frame and divided. After selecting the most similar block from the image data of the past or future frame stored in the frame memory and calculating the motion-compensated inter-frame difference value with the small block of the current frame, Select either the inter-frame difference value of the block or the small block pixel value itself in the current frame, and determine the orthogonal transformation coefficient obtained by performing orthogonal transformation on the selected data for each small block. To obtain a quantized representative value by quantizing with the quantized step width, assign a code corresponding to the quantized representative value, and calculate an inter-frame difference. The data is transmitted together with a motion vector for specifying a similar block and an inter-frame / intra-frame selection flag. On the receiving side, the transmitted code is decoded to reproduce the quantized representative value. After the value is subjected to inverse orthogonal transform, the value after inverse orthogonal transform itself or the value of a small block specified by the motion vector from among past or future decoded frames is determined by the inter-frame / intra-frame selection flag. In an image encoding / decoding method for obtaining a decoded image by obtaining one of the values obtained by adding the values after the inverse orthogonal transform as the decoded value of the small block, a spatio-temporal low-pass is applied to the decoded image obtained on the receiving side. A process of applying a filter, a process of dividing the image signal after the spatio-temporal low-pass filter into the same small blocks as those used on the transmission side, Of the difference between the small block data of the current frame or the divided small block of the current frame and the small block specified by the motion vector from the decoded image data of the past or future frame stored in the frame memory. A process of selecting one of the data by the inter-frame / intra-frame selection flag, a process of performing an orthogonal transform on the selected data, and whether or not the obtained orthogonal transform coefficient is within a predetermined range , A process of clipping within the range when it does not exist, a process of performing inverse orthogonal transform on the orthogonal transform coefficient subjected to the clipping process, and an inverse orthogonal transform based on the inter-frame / intra-frame selection flag. A subsequent value, or specified by the motion vector from among past or future decoded frames A process of obtaining a corrected image by setting one of the values obtained by the inverse orthogonal transformation to the value of the small block to be the corrected value of the small block, and obtaining the corrected image by Is repeatedly obtained a predetermined number of times to obtain a final corrected image.

[0016]

According to the present invention, the clipping is performed on the orthogonal transformation coefficient in the frame when the pixel itself in the frame is orthogonally transformed and encoded, and when the difference value between the motion compensation frames is orthogonally transformed and encoded, Clipping is performed on the orthogonal transform coefficient of the difference signal between compensated frames.

[0017]

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram of an image encoding / decoding apparatus according to one embodiment of the present invention. For the code sequence 102 received at the receiving terminal 101, first, the code decoding unit 103 decodes the transmitted code to obtain a quantization level number 1
04, the inter-frame / intra-frame selection flag 105 and the motion vector 106 are reproduced. After the reproduced quantization level number 104 is inversely quantized by the inverse quantization unit 107 to obtain a quantized representative value, the inverse orthogonal transform unit 108 performs inverse orthogonal transform, and then the interframe / intraframe selection flag 105 The switch 110 is switched, and the motion compensating unit 1 is selected from the value itself after the inverse orthogonal transform or the past or future decoded frame stored in the frame memory 112.
13 adds the value after inverse orthogonal transformation to the motion compensated prediction data 114 specified by the motion vector 106 by the adder 109, and outputs one of the decoded value 11 of the small block.
Let it be 1.

The resulting decoded image 111 is input to the clipping block 120 _1, the space-time low-pass filter 121 is applied. The image signal after the spatio-temporal low-pass filter 121 is divided into the same small blocks as those used on the transmission side, and stored in the frame memory 129 with the data of the small blocks of the current frame or the divided small blocks of the current frame. The difference between the small block specified by the motion vector 106 and the data 132 extracted by the motion compensation unit 131 from the decoded image data of the past or future frame that has The selection is performed by the switch 123 according to the intra-frame selection flag 105. The orthogonal transformation unit 124 performs orthogonal transformation on the selected data, and the obtained orthogonal transformation coefficient is a clipping range 117 determined by the range setting unit 116 based on the quantization level 115 obtained by the inverse quantization unit 107. To determine if it exists. If the orthogonal transform coefficient does not exist in the set range, the clipping unit 125 performs a process of clipping the range.

The clipped orthogonal transform coefficient is subjected to inverse orthogonal transform in an inverse orthogonal transform unit 126, and the inter-frame / intra-frame selection flag 105 is used to decode the value itself after the inverse orthogonal transform or to decode past or future values. The value after the inverse orthogonal transform is added to the value 132 of the small block designated by the motion vector 106 from the frame in the adder 127
Is selected by the switch 128 and is used as the correction value of the small block to obtain the corrected image 130. The obtained corrected image 130 is subjected to the final image 13 from the spatiotemporal low-pass filter 121 described above.
By repeatedly performing the processing until obtaining 0, the final corrected image 133 is obtained at the output terminal 140 of the clipping processing block 120 _n .

Hereinafter, specific examples of the present invention will be described. In this specific example, a case is considered in which a discrete cosine transform is used for a motion compensation frame difference signal as an information compression method.

On the transmitting side, the input image is converted into 8 × 8 small blocks f (i, j) (i, j = 0, 1, 2,...,
7), and the motion-compensated inter-frame difference signal e (i, j) (i, j, = 0, 1, 2,...)
・, 7) is obtained. A value defined by the following equation is calculated for the original signal f (i, j) and the inter-frame difference signal e (i, j) in the frame.

[0023]

(Equation 2) If VAROR> VAR, the in-frame / inter-frame selection flag P (105) is set to 1, and conversely, VAROR
If ≤ VAR, the intra-frame / inter-frame selection flag P is set to 0. If P = 0, discrete cosine transform is performed on f (i, j) to obtain a transform coefficient C (i, j). When P = 1, a discrete cosine transform is performed on e (i, j) to obtain a transform coefficient C (i, j).

The obtained conversion coefficient is calculated as shown in FIG.
Quantization is performed by a quantizer having a step width of 2d. At this time, if the value C (i, j) before quantization is 2nd−d ≦ C (i, j) <2nd + d, the representative quantization value C ′ (ij) is C ′ (i, j). j) = 2nd. In this case, on the receiving side, conversely, C ′ (i, j) =
When 2nd is transmitted, the true value C (i, j) of the DCT coefficient before quantization is 2nd−d ≦ C (i, j) <2
It can be seen that it is in the range of nd + d.

Now, consider a case where the quantized representative value C '(i, j) obtained on the receiving side is 2nd.

First, when the intra-frame / inter-frame selection flag P is P = 0, inverse discrete cosine transform is performed on C ′ (i, j) to obtain a decoded image f ⁰ (i, j). If the intra-frame / inter-frame selection flag P is P = 1, after performing an inverse discrete cosine transform of C ′ (i, j), the area specified by the motion vector 106 in the decoded image of the previous frame Is added by the adder 109 to the decoded image f ⁰
(I, j).

First, the following spatiotemporal low-pass filter 121 is applied to the obtained decoded image f ⁰ (i, j).
Is applied.

[0028]

(Equation 3) Here, g ⁰ (i, j) and h ⁰ (i, j) are f ⁰ (i, j).
This represents a decoded image one frame before and one frame after j). Also, the low-pass filter is applied across the boundary of the small block.

The image signal to which the low-pass filter has been applied is 8
X8 small blocks, and if the intra-frame / inter-frame selection flag P is P = 0,

[0030]

[Outside 1] Is subjected to discrete cosine transform by the orthogonal transform unit 124,
Conversion factor

[0031]

[Outside 2] Get. When P = 1,

[0032]

[Outside 3] And the difference from the region in the previous frame specified by the motion vector 106

[0033]

[Outside 4] Is subjected to discrete cosine transform by the orthogonal transform unit 124,
Conversion factor

[0034]

[Outside 5] Get.

The obtained conversion coefficient

[0036]

[Outside 6] In response, the clipping unit 125 performs the following clipping processing.

[0037]

(Equation 4) That is, when the value exceeds the range that the true value can take, it means that the true value is clipped closer to the maximum value or the minimum value that it can take.

After the inversely orthogonal / transform unit 126 performs an inverse discrete cosine transform on the clipped transform coefficients, if the intra-frame / inter-frame selection flag P is P = 0,
The first correction image f ¹ (i, j) is used as it is. If the intra-frame / inter-frame selection flag P is P = 1, the transform coefficient subjected to the clipping process is subjected to inverse discrete cosine transform, and then added to the data 132 of the region of the previous frame indicated by the motion vector 106. The first correction image f ¹ (i, j) is added by the adder 127.

The above processing is repeated n times to obtain the n-th corrected image f ⁿ (i, j) as the final output 133.

[0040]

As described above, according to the present invention,
If the pixels in the frame are subjected to orthogonal transform coding, clipping is performed on the intra-frame orthogonal transform coefficients, and if the motion-compensated inter-frame difference value is orthogonal-transformed coded, the motion-compensated inter-frame difference value signal is calculated. By performing clipping on the orthogonal transform coefficients, the corrected image can be approximated to the original image before encoding, and noise components included in the decoded image can be efficiently removed.

[Brief description of the drawings]

FIG. 1 is a block diagram of an image encoding / decoding device according to an embodiment of the present invention.

FIG. 2 is a diagram showing quantization characteristics in the embodiment of FIG.

FIG. 3 is a block diagram of a general orthogonal transform encoding / decoding device.

FIG. 4 is an explanatory diagram showing an orthogonal transform coefficient quantization method used for orthogonal transform coding.

FIG. 5 is a block diagram of a conventional decoding device that corrects a decoded image by repeating a low-pass filter and orthogonal transform coefficient clipping.

FIG. 6 is a block diagram of a motion compensation inter-frame difference value orthogonal transform encoding apparatus.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 101 Receiving terminal 102 Received encoded data 103 Decoding unit 104 Quantization level number 105 Inter-frame / intra-frame selection flag 106 Motion vector 107 Inverse quantization unit 108 Inverse orthogonal transformation unit 109 Adder 110 Switch 111 Decoded image 112 frame Memory 113 Motion compensation unit 114 Motion compensation prediction data 115 Quantization level 116 Range setting unit 117 Set clipping range 120 _{1 to} 120 _n Clipping processing block 121 Spatio-temporal low-pass filter 122 Difference unit 123 Switch 124 Orthogonal transformation unit 125 Clipping unit 126 Inverse orthogonal transform unit 127 Adder 128 Switch 129 Frame memory 130 First corrected image 131 Motion compensation unit 132 Motion compensated prediction data 133 Final corrected image 140 Output Terminal

Claims (1)

(57) [Claims] 1. A transmitting side divides image data of a current frame into small blocks for a digital image signal such as a television signal, and divides the divided small blocks into past blocks stored in a frame memory. Or after selecting the most similar block from the image data of the future frame and calculating the motion compensation inter-frame difference value with the small block of the current frame,
Select either the inter-frame difference value of the small block or the small block pixel value itself in the current frame,
The orthogonal transform coefficient obtained by performing orthogonal transform on the selected data for each small block is quantized with a predetermined quantization step width to obtain a quantized representative value, which corresponds to the quantized representative value. A code is allocated and transmitted together with a motion vector for specifying a similar block for calculating an inter-frame difference and an inter-frame / intra-frame selection flag. On the receiving side, the transmitted code is decoded and the quantum After reproducing the quantized representative value and performing the inverse orthogonal transform on the reproduced quantized representative value, the interframe / intra-frame selection flag determines whether the value itself after the inverse orthogonal transform or the value in the past or future decoded frame is used. From the sum of the value of the small block specified by the motion vector and the value after the inverse orthogonal transform is used as the decoded value of the small block. In an image encoding / decoding method for obtaining an image, a process of applying a spatio-temporal low-pass filter to the decoded image obtained on the receiving side, and the same process as using an image signal after the spatio-temporal low-pass filter on the transmitting side The process of dividing into small blocks, the data of the small blocks of the current frame, or the divided small blocks of the current frame and the decoded image data of the past or future frame stored in the frame memory are designated by the motion vector. A process of selecting one of the data of the difference value from the small block according to the inter-frame / intra-frame selection flag, a process of performing orthogonal transform on the selected data, and a process of determining the obtained orthogonal transform coefficient. A process of determining whether or not the image exists in the range, a process of clipping the image in the range if the image does not exist, and A process of performing an inverse orthogonal transform on the orthogonal transform coefficient subjected to the ping process;
Depending on the intra-frame selection flag, either the value after inverse orthogonal transform itself or the value obtained by adding the value after inverse orthogonal transform to the small block value specified by the motion vector from the past or future decoded frame Is a correction value of the small block, and has a process of obtaining a corrected image. The obtained corrected image is repeatedly subjected to all the processes described above a predetermined number of times to obtain a final corrected image. Image encoding and decoding method.