JPH10164594A - Compression encoding method for moving image and device therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce redundant information which is hardly perceived by the eyes of a human being to a time direction and to improve the compressibility of moving images by forming the difference images of reference images and encoding object images by using motion vector information and performing the batch orthogonal transformation processing of the plural difference images of the different time in a time space. SOLUTION: Inputted moving image signals are stored in a frame memory 11 provided with a storage area worth four frames. In a motion vector detection part 12, bidirectional prediction images are compared with the reference images for each macro-block and a closest position is detected, defined as a motion vector and stored in a memory 13. The difference image is calculated in a difference image calculation part 14 from the image of the frame memory 11 and the motion vector of the motion vector memory 13, and the batch discrete cosine transformation processing of the difference image is performed in a time space DCT part 15. A quantization part 16 divides the DCT-processed data by a quantization coefficient, rounds off a remainder, quantizes them, enlarges the quantization coefficient of a wide-band component, inconspicuous to the eyes of the human being and compresses an information amount while suppressing the degradation of image quality.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for compression-encoding a moving image.

[0002]

2. Description of the Related Art Digitized video signals, especially moving images, have a huge amount of information. Therefore, if the recording is to be performed on a recording medium such as a magnetic disk as it is for a long time, a very large storage capacity is required, and the cost is increased. Alternatively, transmission of data in real time by wire or wireless requires a very wide transmission path because the bit rate of video data is high, and the realization is not easy. Therefore, several encoding methods have been proposed for efficiently encoding video data by signal processing and reducing the amount of data.

[0003] One of such encoding systems is MPE.
G2 ("The Journal of the Institute of Television Engineers of Japan"; Vol. 48, No.
1, pp. 44-49). In MPEG2, a plurality of methods described below are mainly used in combination in order to reduce the amount of video data.

First, a method of reducing the number of bits required for encoding by taking a difference between an image to be encoded and an image before and after the image (hereinafter referred to as a reference image) and converting the difference into information having a small amplitude. (Motion compensation prediction).

There are three types of pictures (units of images to be coded) of I, P, and B depending on the image reference method. This is shown in FIG. The starting point of the arrow in the figure is the reference image,
The end point represents the image to be encoded. The I picture does not refer to an image. This is because all information necessary for decoding is encoded in the picture. As a P picture, an I picture or a P picture existing immediately before in the original picture order is used as a reference picture. The B picture uses the I or P picture existing immediately before and immediately after as a reference picture. In the case of a B picture, a P picture that comes later in the order of the original image is used as a reference image. Therefore, as the order of encoded data, two reference images corresponding to the B picture to be encoded as shown in FIG. Encode by rearranging so that it always comes first. This is output after being rearranged in a correct order at the time of decoding.

One picture is called a macroblock.
Encoding is performed by partitioning into sections of 6 × 16 pixels.
The macroblocks are assigned serial numbers called macroblock addresses in order from the upper left of the image. As shown in FIG. 3, for example, when one picture is composed of frame images of 720 × 480 pixels, 0 to 134
There are up to 9 1350 macroblocks.

[0007] A macroblock is a minimum unit that performs motion compensation and takes a difference from a reference image. Information indicating the position of the part used for reference on the reference image (motion vector information)
Is encoded in the encoded data of each macroblock.

Further, the macro block is composed of a unit image (hereinafter, block) of 8 pixels × 8 pixels. For example, in the case of the format of Y: Cb: Cr = 4: 2: 0, as shown in the left side of FIG. 3, the luminance signal is four blocks of Y0 to Y3, and the color difference signal is one each of Cb and Cr. A macro block is composed of a total of six blocks.

A second method for reducing the data amount is a method of performing DCT (Discrete Cosine Transform) on video data to convert the video data into a spatial frequency domain. As shown in FIG. 4, pixel information existing on a space called an xy axis is
The data is converted into data in the spatial frequency domain called μ-ν axis by T. In the case of a video signal, generally, the rate at which energy is concentrated in the low frequency component of the spatial frequency is high, and the amplitude value of the high frequency component is small. Accordingly, the number of bits to be allocated to the coding can be optimized by assigning more bits to the low-frequency component and decreasing the number of bits to the high-frequency component. As a result, the overall number of bits can be reduced. . MP
In EG2, data is reduced by performing DCT on a block-by-block basis and then performing a quantization process so that the higher the frequency component, the smaller the number of bits allocated.

Third, variable-length coding in which codes having different lengths are assigned according to the appearance probabilities of the respective values. By assigning a shorter code to a value having a higher appearance frequency, it is possible to reduce the total number of bits. MP
Conventional technologies such as EG2 use variable-length coding for coding parameters such as macroblock addresses and motion vectors in addition to the above-described quantized DCT coefficients.

[0011]

The image compression technique includes a method of expressing an image with a minimum amount of information by devising a method of expressing a code, such as motion compensation and variable-length coding, a DCT and a quantization. There is a method of reducing redundancy by reducing information that is hardly perceived by the human eye as in the case of image processing. Of the latter, MPEG2
In the case of, DCT and quantization are performed only in the spatial direction, that is, only in the image at the same time. Therefore, the reduction of the redundancy is performed only in the spatial direction.

In the time direction, a scheme has been devised to reduce the code amount by motion compensation. However, it is a technique of reducing the amplitude by taking the difference and reducing the number of bits required for encoding. Therefore, the motion compensation is only a method of expressing the code, and the reduction of the redundancy in consideration of the human visual characteristics is not considered in the time direction.

Therefore, the conventional technique is insufficient in the compression in the time direction. If the amount of codes is constant, the lower the compression efficiency, the lower the image quality of the image that can be expressed, so that the image quality obtained with the conventional technology cannot be said to be sufficient.

[0014]

In order to solve the above problems, the present invention provides motion compensation for predicting an image to be encoded by using motion vector information indicating a position on a reference image, and frequency conversion of pixel information. Orthogonal transformation processing to convert to data in the area, quantization to reduce information by dividing the orthogonally transformed data by a predetermined value, and truncating the remainder, and different lengths depending on the frequency of code generation In a coding method for compressing and coding a moving image by using variable-length coding and assigning a code, a difference image is obtained by taking a difference between a reference image and an image to be coded using the motion vector information. The orthogonal transformation process is performed collectively in space and time using a plurality of difference images of the encoding target image created at different times.

An image to be coded is compared with a reference image to detect a similar portion, and a motion vector, which is information indicating the position of the similar portion on the reference image, is obtained.

Next, a difference image between the image to be encoded and the image of the corresponding part on the reference image is calculated using the obtained motion vector.

Using a plurality of differential images of the image to be coded at different times, orthogonal transform processing is performed collectively in space and time.

A quantization process is performed on the orthogonally transformed spatio-temporal frequency domain data using a predetermined quantization coefficient.

The quantized data obtained as described above,
The quantization coefficient and the motion vector information are encoded by a variable length code. Furthermore, predetermined header information is added,
Output as an encoded video signal.

[0020]

FIG. 1 shows a moving picture compression encoding apparatus 1 according to an embodiment of the present invention.

In this embodiment, as shown in FIG. 2A, two B pictures are placed between an I or P picture as a reference picture.
It is assumed that coding is performed with a picture sandwiched therebetween. As a reference image, one I picture is used periodically for every several P pictures.

The input moving image signal is first stored in the frame memory 11. The frame memory 11 has an area for storing four frames of images therein. Of these, two frames are used for storing I or P pictures as reference images, and the remaining two frames are used for storing B pictures. One of the reference image areas for two frames (Ref
In 1), the immediately preceding reference image that has been encoded and then decoded is already stored. For example, FIG.
Considering the case of encoding B4, B5, and P6 in (a), P3, which is the immediately preceding reference image, has already been encoded and re-decoded and stored in Ref1. Next, moving image signals for three frames (B4, B5, P6) to be newly encoded are input, and each of them is stored in the B memory area (B1, B2) and another reference image area (B) in the frame memory 11. Ref2).

Next, the motion vector detecting section 12 detects a motion vector. Since B4 and B5 are bidirectionally predicted images, a motion vector is obtained by comparing each of the macroblocks with the reference images P3 and P6 and detecting a position having the closest value. Further, since P6 is a forward prediction image, a comparison is made only with P3 of the reference image to obtain a motion vector for each macroblock.

The motion vector thus obtained is stored in the motion vector (MV) memory 13. The motion vector memory 13 internally has one reference image area and two B picture areas, and stores the motion vectors of P6, B4, and B5, respectively.

Next, a difference image is calculated by the difference image calculator 14 using the image stored in the frame memory 11 and the motion vector stored in the motion vector memory 13.
For example, for B4 and B5, a corresponding motion vector is read from the motion vector memory 13 for each macroblock, and an image at a required position is read from the reference images P3 and P6 of the frame memory 11 using the same. Thereafter, a difference image is obtained by subtracting the image read from the B picture itself. Similarly, in the case of P6,
A motion vector corresponding to each macro block is read from the motion vector memory 13, an image at a required position is read from the reference image P3 of the frame memory 11 using the motion vector, and the image read from P6 itself is subtracted to obtain a difference. Get an image.

Here, the calculation of the difference image is not necessarily B4
It is not necessary to perform the processing separately or in order for each picture of B5 and P6. When necessary in the next spatiotemporal DCT unit 15, the necessary picture and the difference image data of the required macroblock position are calculated and output each time.

The spatio-temporal DCT unit 15 performs discrete cosine transform processing on each of the difference images B4, B5, and P6 output from the difference image calculation unit 14. That is, FIG.
, Three-dimensional data having the time t direction in addition to the spatial xy direction is converted to data in the time-space frequency domain of μ-ν-f.

The quantization unit 16 divides the data subjected to the DCT processing by using a quantization coefficient determined for each frequency component, and cuts off the remainder to perform quantization. Generally, the value of the quantization coefficient is increased in a high-frequency component that is hardly noticeable due to human visual characteristics, and the amount of information is compressed while minimizing image quality deterioration.

The quantized data and the quantized coefficients used at that time are encoded by the variable length encoding unit 17 (VLC) into variable length codes having different lengths according to the frequency of appearance of the code. . Further, the motion vectors stored in the motion vector memory 13 are also stored in the variable length coding unit 17.
In the variable length code.

Finally, in the header adding section 18, various coding parameters are added as a header and output as a coded video signal.

Reference numeral 2 denotes a local decoding unit. In this part, in preparation for encoding the next input image, the currently encoded 3
The reference image (P6) among the images is decoded.

First, in the inverse quantization unit 21, the quantization unit 1
Inverse quantization is performed by multiplying the quantized data output from 6 by a quantized coefficient for each frequency component. Further, an inverse discrete cosine transform process is performed in the spatiotemporal IDCT unit 22 to return the pixel data from the frequency domain to the time / space domain.

The motion compensator 23 reads a motion vector from the motion vector memory 13 for each macroblock,
The image data at the corresponding position is read out from the reference image (P3) in the frame memory 11 using this, and is added to the output data from the spatiotemporal IDCT unit 22, thereby obtaining
Perform decryption. Since the local decoding unit 2 decodes only the reference image P6, the motion compensation unit 23
Also decrypts only P6.

The decoded reference image P6 is written again to the frame memory 11 for each macroblock in preparation for encoding the next input image. At this time, P3, which is the previous reference image, is stored in the reference image area Ref1 in the frame memory 11, and may be used until decoding of the entire P6 is completely completed. The decoded P6 is written into one reference image area Ref2.

When the decoding of P6 is completely completed in this way, next, three frames to be newly encoded (FIG. 2)
The moving image signal of (B7, B8, P9) of (a) is input,
A series of encoding steps described above are repeated.
At the time of this next encoding, P6 which is the previous reference image is stored in the reference image area Ref2 of the frame memory 11, so that P9 is not Ref2 but Ref1.
Is stored in As described above, the roles of the reference image regions Ref1 and Ref2 are alternately used for each coding unit (B, B, P). As for B7 and B8, the area B in the frame memory 11 is the same as before.
1 and B2.

In the above description, when the reference picture is coded as an I-picture instead of a P-picture, no motion compensation processing is necessary. Therefore, motion vector detection in coding and motion compensation in local decoding are not performed. .

In this embodiment, it is assumed that encoding is performed in such a manner that two B pictures are sandwiched between I or P pictures, which are reference pictures, but the number of B pictures is 2 It is not limited to sheets. Frame memory 11 and motion vector memory 13 according to the number of B pictures
By preparing an area necessary for the above, the same effect as that of the present invention can be obtained even when the number of B pictures is arbitrary.

[0038]

According to the present invention, in addition to the conventional spatial direction, it is possible to reduce redundant information that is hardly perceived by the human eye in the time direction, thereby further reducing the moving image compression rate. Can be enhanced. In addition, an increase in compression efficiency makes it possible to realize an improvement in image quality for a fixed code amount.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a moving image compression encoding apparatus.

FIG. 2 is an explanatory diagram of the order of an original image and encoded data.

FIG. 3 is an explanatory diagram of macro blocks and blocks.

FIG. 4 is an explanatory diagram showing DCT in a spatial direction in a conventional technique.

FIG. 5 is an explanatory diagram showing a three-dimensional DCT in the spatiotemporal direction according to the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Video compression coding apparatus, 11 ... Frame memory, 1
2 ... Motion vector detection unit, 13 ... Motion vector memory,
14: difference image calculation unit, 15: spatio-temporal DCT unit, 16 ...
Quantization unit, 17: variable length coding unit, 18: header addition unit, 2: local decoding unit, 21: inverse quantization unit, 22
... a spatiotemporal IDCT unit, 23 ... a motion compensation unit.

Claims (2)

[Claims] 1. A motion compensation for predicting an image to be encoded using motion vector information indicating a position on a reference image,
Orthogonal transformation processing for converting the pixel information into frequency domain data, dividing the orthogonally transformed data by a predetermined value, and quantizing to reduce the information by truncating the remainder,
In a coding method for compressing and coding a moving image by using variable length coding that allocates codes of different lengths according to the frequency of occurrence of the code, the reference image and the image to be coded using the motion vector information A difference image is created by taking a difference from the moving image, and a plurality of the difference images of the encoding target image formed at different times are used, and the orthogonal transformation process is collectively performed in a time and space. Compression encoding method. 2. An encoding apparatus for compressing and encoding a moving image, comprising: a frame memory for storing a plurality of input images;
A motion vector detecting unit that compares the image to be encoded among the images stored in the frame memory with a reference image to detect a similar portion, and outputs motion vector information indicating the position of the similar portion on the reference image. A difference image calculating means for calculating a difference image between the image to be coded and the reference image, an orthogonal transform means for performing an orthogonal transform on an output of the difference image calculating means, and an output of the orthogonal transform means. Quantizing means for quantizing with the quantization coefficient of the following, a motion vector storing means for storing the motion vector information, an output of the quantizing means, a quantized coefficient and / or motion vector information stored in the motion vector storing means. Variable-length coding means for coding the video data with a variable-length code, and predetermined header information is added to the output of the variable-length coding means, and output as a coded video signal. The orthogonal transforming means, wherein the orthogonal transforming means performs orthogonal transform processing in a time and space collectively by using a plurality of difference images composed of different times of the original moving picture. Compression encoding device.