KR100474268B1 - The implementation of reference frame prediction for reducing memory bandwidth and computational complexity in embedded system - Google Patents

Abstract

The present invention relates to a multi-reference image compression method for reducing memory bandwidth and computational complexity in an embedded system. In the conventional multi-reference image compression method, a motion is detected using a plurality of reference images, and prediction and compensation for the motion are performed. Although the compression ratio has an effect of improving the compression ratio, there is a problem that cannot be applied to an embedded system with limited memory and computation amount due to an increase in computation amount and memory usage. In consideration of such a problem, the present invention comprises the steps of: obtaining a motion vector by estimating the motion of an object by comparing a current frame with a previous frame; Predicting a background area newly revealed and a background area covered by the object according to the position of the object in the next frame and the change of the position of the object using the motion vector, and storing information on the exposed background area; The use of the previous frame and information on the exposed background area stored in the display of the next frame using only the previous frame having the most correlation with the current frame as a reference image, the movement of the object that does not appear in the previous frame By saving and updating the information on the background area exposed according to the information, and displaying the next frame using the information on the background area that is stored, it reduces the memory usage and calculation amount while preventing the reduction of compression efficiency. It is effective to use the video compression method.

Description

TECHNICAL FIELD OF REFERENCE FRAME PREDICTION FOR REDUCING MEMORY BANDWIDTH AND COMPUTATIONAL COMPLEXITY IN EMBEDDED SYSTEM}

The present invention relates to a multi-reference video compression method for reducing memory bandwidth and computational complexity in an embedded system. In particular, the present invention is to apply a multi-reference video prediction algorithm, which is widely adopted in H.26L, which is being standardized as a next-generation video communication protocol. The present invention relates to a multi-reference image compression method for reducing memory bandwidth and computational complexity in an embedded system, which is suitable for reducing memory bandwidth and computing amount.

H.26L, currently being enacted as the next generation video compression standard, is an existing International Telecommunications Union (ITU) series standard such as H.261, H.263, H.263 +, and H.263 ++, and the International Standards Organization (ISO / IEC). / International Electrotechnical Commission) has established itself as a standard for video communications with better compression rates than MPEG1, MPEG2 and MPEG4 compression standards. The most prominent standard algorithm is the multiple reference image compression method.

In the multi-reference image compression method, a plurality of reference images are used to estimate the motion of the image, and motion estimation is performed on each of them, and then the most optimal block is used together with the reference frame information. The motion estimation and compensation is superior to other motion estimation methods. This is possible and shows a high compression ratio. However, if the above-described method is used in an embedded system that does not use a hard disk and has a limited bandwidth and a large amount of memory, the calculation amount is increased and the memory is exceeded. Detailed description with reference to the following.

1 is a block diagram of an apparatus in which a conventional multi-reference video compression algorithm is hardware-configured. As shown therein, a subtractor 1 for obtaining a difference between a video signal VIDEO IN and a reference picture; A converter (3) which receives the output of the video signal (VIDEO IN) or the subtractor (1) under the control of the coding controller (2), converts it, and outputs it; A quantization unit (4) for quantizing and outputting the output signal of the conversion unit (3) under the control of the coding control unit (2); An inverse quantization unit 5 for inversely quantizing the output Q of the quantization unit 4; A restoring unit (6) for restoring a previous image by an operation opposite to the converting unit (3) with the output of the inverse quantization unit (5); A picture memory 9 for receiving and storing the video signal VIDEO IN; An adder (7) for adding a video signal (VIDEO IN) stored in the picture memory (9) and a video signal reconstructed by the reconstructor (6); And a plurality of additional picture memories APM1 to APMn for storing a reference image which is an output signal of the adder 7.

Through the operation of such an apparatus, a conventional multi-reference image compression algorithm will be described in more detail.

First, when the video signal VIDEO IN of the first frame is input, it is stored in the picture memory 9 and directly to the converter 3 through a switch SW1 operated by the control of the coding controller 2. Is approved.

Then, the video signal VIDEO IN of the first frame is coded in a specific format in the converting section 3, which is quantized by the quantizing section 4, and is a video multi-coder (Q) with a quantization index Q for the transform coefficient. (Not shown in the drawing).

Then, the inverse quantization unit 5, which receives the output of the quantization unit 4, dequantizes and outputs the same as the output state of the conversion unit 3, and the restored unit 6 receives the previous image. To recover the video signal of the first frame.

Then, it is stored in the additional picture memories APM1 to APMn through the adder 7.

At this time, the switch SW2 for applying the image of the previous state to the adder 7 is open so that the first frame of the video signal VIDEO IN is stored in the additional picture memory APM1 as it is.

Then, when the second frame of the video signal (VIDEO IN) is applied, it is stored in the picture memory 9 and the difference from the frame of the previous image is obtained from the subtractor 1, whereby the first frame and the second frame Image changes, i.e., movement, can be detected.

Next, the converter 3, which has received the output of the subtractor 1 through the switch SW1, converts the image, and quantizes it again through the quantizer 4 to output the result to the video multicoder. In addition, the output is applied to the inverse quantization unit 5 to inverse quantization, and restored through the restoration unit 6 again.

Then, the image reconstructed by the reconstructor 6 and the second frame of the video signal VIDEO IN stored in the picture memory 9 are added, and are stored in the additional picture memory APM1 as a reference image signal.

The reference picture previously stored in the additional picture memory APM1 is stored in the next additional picture memory APM2, which is sequentially stored in each additional picture memory APM1 to APMn while the frame is in progress.

In this way, the image of the previous frame and the current frame can be compared to distinguish the background and the moving object that are not changed in the two frames, and to detect a motion vector (V) indicating the direction and speed of the moving object. The same motion vector (V) can be used to predict the position of a moving object, and the high compression ratio can be represented by compensating and displaying the prediction and the actual motion.

In this case, the background changed by the movement of the object may be divided into an exposed background area where the object is located in the previous frame, a hidden background area where the object is located in the current frame, and an unchanging background area.

The unchanged background region can use information included in the reference image as it is, and the hidden background region is also included in the previous reference image and is not displayed in the current frame, and thus can be compressed using the reference image.

However, since the exposed background area is the part where the object was located in the previous reference image, the previous reference image does not have its own information itself, so a lot of prediction errors occur in motion estimation, and the prediction efficiency is lowered. do.

To compensate for this, by providing the plurality of additional picture memories APM1 to APMn and referring to a plurality of reference images stored in the plurality of additional picture memories APM1 to APMn, it is possible to obtain better motion estimation and improve coding efficiency. Will be improved.

As such, a method of estimating motion with multiple reference images requires additional picture memory, and the amount of computation increases. In the system of 5 additional picture memories, each of 5 times the resolution of 4 times 1/2 resolution and 16 times the size of 1/4 resolution is required, and the amount of memory required is at least 100 times considering the decoder. Will increase.

In addition, since the method of performing all motion estimation for each frame is increased, the amount of computation is increased, and the additional information (OVER HEAD) is increased because the method of carrying information on the optimum frame is also used in the bit rate.

If the multi-reference image compression algorithm is applied to an embedded system having limited memory and limited computation capability, the multi-reference image compression method cannot be used due to the limitation of the embedded system.

As described above, the conventional multi-reference image compression method has an effect of detecting a motion using a plurality of reference images and improving the compression ratio by predicting and compensating for the motion, but the amount of computation is increased and memory usage is increased. Therefore, there is a problem that can not be applied to the embedded system with limited memory and amount of calculation.

It is an object of the present invention to provide a multi-reference image compression method for reducing memory bandwidth and computational complexity in an embedded system that can apply a multi-reference image compression method to an embedded system.

The above object is to obtain a motion vector by estimating the motion of an object by comparing a current frame with a previous frame as a reference image; Predicting a background area newly revealed and a background area covered by the object according to the position of the object in the next frame and the change of the position of the object using the motion vector, and storing information on the exposed background area; This is achieved by using information on the background area displayed on the display of the next frame and using the previous frame, which will be described in detail with reference to the accompanying drawings.

FIG. 2 is a block diagram of an apparatus in which a multi-reference video compression method is configured by hardware in order to reduce memory bandwidth and computational complexity in an embedded system of the present invention. As shown in FIG. Part (1); A converter (3) which receives the output of the video signal (VIDEO IN) or the subtractor (1) under the control of the coding controller (2), converts it, and outputs it; A quantization unit (4) for quantizing and outputting the output signal of the conversion unit (3) under the control of the coding control unit (2); An inverse quantization unit 5 for inversely quantizing the output Q of the quantization unit 4; A restoring unit (6) for restoring a previous image by an operation opposite to the converting unit (3) with the output of the inverse quantization unit (5); A picture memory 9 for receiving and storing the video signal VIDEO IN; An adder (7) for adding a video signal (VIDEO IN) stored in the picture memory (9) and a video signal reconstructed by the reconstructor (6); An additional picture memory (APM) for storing a reference image which is an output signal of the adder (7); It consists of an exposed background area memory 10 that stores the background area exposed in the next frame through previous motion detection.

Hereinafter, a multi-reference video compression method for reducing memory bandwidth and computational complexity in the embedded system of the present invention will be described along with the operation of the apparatus shown in FIG.

First, the converter 3 receiving the first frame of the video signal VIDEO IN through the switch SW1 switched under the control of the coding controller 2 converts the first frame into appropriate data and outputs the converted data. The quantization unit 4 receives the quantized data and outputs the quantized data to a video multiple coder (not shown).

Next, the inverse quantization unit 5 applied with the output of the quantization unit 4 inversely quantizes the quantization index Q for the transformation coefficient applied as the output of the conversion unit 3.

Then, the restoring unit 6, which has received the output of the inverse quantization unit 5, restores the data to restore the first frame of the video signal VIDEO IN.

Then, the adder 7 receiving the output of the restoration unit 6 outputs the first frame as it is during the processing of the first frame, and in the subsequent operation, adds the current frame and the previous frame. do.

The additional picture memory APM then stores the output of the adder 7.

Subsequently, the subtractor 1 receiving the next frame of the video signal VIDEO IN extracts motion information by subtracting a frame of the previously stored picture memory 9 from the current frame and through the switch SW1. It applies to the converter 3.

Then, the conversion section 3 converts and outputs the motion information, which is quantized in the quantization section 4, and then restored by the inverse quantization section 5 and the restoration section 6.

In addition, the adder 7 adds the motion information to the frame and stores the motion information in the additional picture memory APM to use as a reference picture.

That is, in the present invention, a plurality of reference images are not used as the reference image, and only the previous frame closest in time is used as the reference image, thereby reducing the memory usage.

However, when only the previous frame is used as the reference image as described above, since the portion where the image of the moving object is located in the previous frame becomes the background area exposed in the subsequent frame, the moving area is separated from the background. Since no information is available, the compression efficiency is reduced.

In order to compensate for such a problem, the exposed background area memory 10 detects a motion and stores the background area exposed according to the predicted motion in advance.

If the motion is predicted as described above and the background area revealed by the motion is stored in advance, the predicted position of the moving object is set in the fixed background area in the next frame, and the data of the background area revealed according to the motion of the object is set. By generating and updating, it is possible to display the next frame with fewer calculations while using less memory.

In the description of FIG. 2, reference numeral p denotes an internal flag, t denotes a flag indicating whether to transmit, qz denotes a quantization unit indication signal, and v denotes a motion vector.

A method of storing, updating, and using the exposed background area will be described in more detail as follows.

FIG. 3 is a schematic diagram of background information according to the characteristics of the displayed image. As shown in FIG. 3, when comparing the two frames, the largest change is made by comparing the previous frame and the current frame to the area that is changed and the area that is not changed. The information of the area not used can be used as is when displaying the current frame.

In addition, the changed area is hidden by the object in the previous frame and the hidden background area in which the background part displayed in the previous frame is covered by the object in the current frame according to the movement of the moving object and the object. There is a background area exposed by the movement of the object.

When the motion vector of the object is obtained by comparing the previous frame with the current frame, information about the direction and speed of movement of the object can be obtained to estimate the position of the object in the next frame. The position of the background area and the exposed background area can be predicted.

By dividing a frame into a plurality of blocks, data of a block corresponding to a background area exposed by the movement of an object in the next frame is stored in the exposed background area memory 10 and used to display the next frame. By displaying, the compression efficiency of the frame to be displayed is increased.

If the estimated exposed background area is stored in the exposed background area memory 10, the background area exposed in a new next frame is displayed each time a frame is displayed without updating the exposed background area memory 10. Update the data for.

The first decoded background region memory 10 stores all of the first decoded images, and stores only the background portion that is exposed according to the movement of the object from the next frame, and continuously updates them to reveal them without using multiple reference images. Information about the background area can be maintained continuously.

That is, the present invention uses only the previous frame having the highest correlation with the current movement as a reference image, and continuously stores information about the background region exposed by the movement of the object, thereby reducing the memory usage while preventing the degradation of the coding efficiency. Excessive computations due to motion estimation and background estimation can be minimized. By minimizing the amount of memory used and the amount of calculation, the image compression method conforming to the international standard can be used even in the embedded system where the memory and the amount of calculation are limited.

As described above, in the embedded system of the present invention, the multi-reference image compression method for reducing memory bandwidth and computational complexity uses only the previous frame having the highest correlation with the current frame as the reference image, and does not appear in the previous frame. By storing and updating the information on the background area exposed by the movement separately and displaying the next frame using the information on the stored background area, the memory usage and calculation amount can be reduced by reducing the memory usage and calculation amount while preventing the deterioration of the compression efficiency. Even in embedded systems with a limited amount of computation, it is possible to use international image compression methods.

1 is a block diagram of an apparatus implementing a conventional multi-reference video compression method.

Figure 2 is a block diagram of a device implementing the present invention multiple reference image compression method.

3 is a schematic diagram of dividing a displayed video frame by features.

* Description of the symbols for the main parts of the drawings *

1: subtraction unit 2: coding control unit

3: conversion section 4: quantization section

5: Reverse quantization department 6: Restoration department

7: Adder 9: picture memory

10: Exposed background memory

Claims (2)

Estimating the motion of the object by comparing the previous frame with the current frame as a reference image to obtain a motion vector; Predicting a background area newly revealed and a background area covered by the object according to the position of the object in the next frame and the change of the position of the object using the motion vector, and storing information on the exposed background area; A method of compressing a multi-reference image to reduce memory bandwidth and computational complexity in an embedded system, characterized by using the information on the exposed background area stored in a display of a next frame and a previous frame. The method of claim 1, further comprising: an initializing step of allocating an initial image frame to a plurality of blocks and storing all of the decoded image frames; A prediction step of acquiring a motion vector by comparing the current image frame with the stored image frame and then obtaining a background region newly revealed according to the movement of the object; A storage step of selectively storing only information on the predicted exposed background area; An update step of continuously updating information on a new exposed background area when the next frame is applied, and maintaining a previous state if previously stored information is available; After excluding the data of the background area exposed in the next frame is compressed, and using the stored data for the exposed background area to display the next frame characterized in that the embedded system to reduce the memory bandwidth and calculation complexity Reference Image Compression Method.