KR100504416B1 - Image coding method and apparatus - Google Patents

Abstract

First, by using the omnidirectional prediction reference block data read from the reference frame memory and the search block data read from the search frame memory, the omnidirectional prediction motion vector and its residual are detected by the omnidirectional prediction motion detection circuit. The motion vectors for forward prediction and the residual storage memory are stored. Next, by using the backward block reference block data read from the reference frame memory and the retrieved block data read from the search frame memory, the backward predictive motion vector and its residual are detected by the backward predictive motion detection circuit. At the same time, the motion vector stored in the omnidirectional predictive motion vector and the residual storage memory and the residual thereof are read out, and encoding is performed using these bidirectional motion vectors.

Description

Image coding method and apparatus

The present invention relates to a picture coding method and apparatus based on, for example, the Moving Picture Image Coding Experts Group (MPEG).

The MPEG method is a coding method that compresses image data by combining DCT (Discrete Cosine Transform), motion compensation prediction, and variable length coding.

1 shows the configuration of a picture coding apparatus based on the MPEG system. In this figure, image data is supplied to the input terminal T1. This image data is input to the motion vector detection circuit 1 and the subtraction circuit 2. The motion vector detection circuit 1 obtains the motion vectors of the current frame and the reference frame (for example, the previous frame) using the input image data, and provides them to the motion compensation circuit 3.

The image data of the reference frame is also stored in the frame memory 4. This image data is supplied to the motion compensation circuit 3. In the motion compensation circuit 3, motion compensation of the image data sent from the frame memory 4 is performed using the motion vector sent from the motion vector detection circuit 1. The output of the motion compensation circuit 3 is sent to the subtraction circuit 2 and the adding circuit 5.

The subtraction circuit 2 obtains prediction error data by subtracting the image data of the current frame supplied from the input terminal T1 and the image data of the motion compensated reference frame supplied from the motion compensation circuit 3 to obtain a prediction error data. 6) to supply. The DCT circuit 2 DCT-processes this prediction error data and sends it to the quantizer 7. The quantizer 7 quantizes the output of the DCT circuit 6 and sends it to the variable length coding circuit.

The output of the quantizer 7 is also supplied to the inverse quantizer 8. Then, inverse quantization processing is performed, and its output is subjected to inverse DCT processing in the inverse DCT circuit 9, and returns to the original prediction error data, which is provided to the addition circuit 5.

In the addition circuit 5, this prediction error data is added to the output data of the motion compensation circuit 3 to obtain image data of the current frame. The obtained image data is stored in the frame memory 4 as image data of the next reference frame.

As a motion vector detection method in such a picture coding apparatus, a block matching method is known. The block matching method divides a screen into small rectangular areas (blocks) and detects motion for each block. Examples of the size of the block include 8 pixels wide x 8 pixels long (hereinafter referred to as 8x8), 16x16, and the like. Next, the block matching method will be described with reference to FIG.

2, the reference block RB of MxN is set in the reference frame 41. In FIG. In addition, a check block SB having the same size as the reference block RB is set in the search frame 42. The inspection block SB is moved around the predetermined search range 43 of ± m x ± n around the same position as the reference block RB. The degree of coincidence between the reference block RB and the check block SB is calculated, and the motion vector is obtained from the matched block using the test block having the highest match as the matching block.

That is, when the coincidence of the check block SBk at the position (u, v) shifted from the check block SB0 at the same position as the reference block RB is the highest, the motion vector of the reference block RB is Let (u, v) be. At this time, an inspection block having the highest coincidence is used as the inspection block having the smallest sum of absolute differences or sum of squares of differences for each pixel at the same position of the reference block RB and the inspection block SB.

In the MPEG system, one sequence of a moving picture is divided into GOPs (Groups of Pictures) composed of a plurality of frames (pictures) to perform encoding. The GOP is an intra-frame coded picture (I picture), an inter-frame coded picture (P picture) that is predicted in a frame that has been previously encoded temporally, and a frame-coded picture (B picture that is predicted in two frames that have been encoded before and after temporally encoded. It is composed of

For example, in FIG. 3, motion detection is first performed using P6, which is a P picture, as a reference frame, and I3, which is an I picture, as a search frame. Next, bidirectional motion detection is performed using B4 as a B picture as a reference frame and I3 and P6 as search frames. Next, bidirectional motion detection is performed using B5 as a B picture as a reference frame and I3 and P6 as search frames.

It demonstrates in detail, referring the timing chart shown in FIG. For example, at the time t1, the current frame B4 is used as a reference macro block, and the forward-looking search frame (search frame 0) I3 and the backward-looking search frame (search frame 1) (P6). The case where bi-directional prediction to) is performed simultaneously and two motion vectors are obtained. In this case, data transmission of the reference block is necessary from the current frame B4, and data transmission of the search block is required from the two search frames I3 and P6.

Therefore, if one pixel is 8 bits, the size of the reference block is 16 × 16, and the search range is ± 16 in both the horizontal and vertical directions, the data transfer amount for processing one reference block is 8 × 16 ×. Since 16x1 = 2K bits and the search block is 8x48x48x2 = 36K bits, the total is 38K bits.

As described above, in the conventional motion detection method, it is necessary to transmit a large amount of data of a search frame when performing bidirectional prediction. And this has become a big problem in realizing hardware.

In addition, in this picture coding apparatus, it was necessary to prepare six frame memories in total for each frame from MEM0RY-0 to MEM0RY-5 and hold them for the period shown in FIG. In addition, a local decoding memory of LOCAL-1 and LOCAL-2 was also required for local decoding output. That is, the conventional image coding apparatus required eight memories in total.

For example, when the input image B1 is supplied, MEMORY-0 holds B1 until encoding is complete. Each B frame is held in each MEM0RY until it is similarly encoded. Here, the I and P frames used as the search frames are each frame memory until they are needed as the search frames for detecting the motion vectors until the forward and backward motion vectors of each frame are detected. Are stored in.

1 is a block diagram showing the configuration of a picture coding apparatus based on MPEG.

2 is a diagram for explaining a block matching method.

3 is a diagram illustrating an example of motion detection in MPEG.

4 is a timing chart for explaining the operation timing of the conventional motion vector detection circuit and the use of the frame memory.

Fig. 5 is a block diagram showing a first form of a motion vector detection circuit that can be applied to an embodiment of a picture coding method and apparatus according to the present invention.

6 is a block diagram showing a second form of the motion vector detection circuit;

7 is a flowchart for explaining the operation of the motion vector detection circuit.

8 is a timing chart for explaining the operation of the motion vector detection circuit.

9 is a timing chart for explaining frame memory reduction of the motion vector detection circuit.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and an object thereof is to provide an image encoding method and apparatus capable of reducing the data transmission amount of a search frame when performing bidirectional prediction.

Moreover, an object of this invention is to provide the image coding method and apparatus which can reduce the memory for a reference frame and a search frame.

For this reason, the image encoding method includes an image encoding method for performing image encoding by detecting motion from images in the forward and backward directions in time, the input image storing step of storing an input image to be encoded in the storage unit, and An omnidirectional motion detection step of detecting an omnidirectional motion vector of the first image from a search image that is temporally omnidirectional than the first image stored in the storage in the input image storage step; and a storage unit in the input image storage step And a backward motion detecting step of detecting a backward motion vector of the second picture temporally before the searched picture from the searched picture, and the front of the first picture detected by the forward motion detecting step. A motion vector preservation step of storing a directional motion vector in a storage unit; and adding the first image When synchronizing, the forward motion vector of the first image is read out in the motion vector storing step, and the backward motion vector for the first image is detected in the backward motion vector detecting step, and the forward and backward directions are detected. An encoding step of encoding using a motion vector is provided.

The first and second images are processed in units of blocks in which the input image is divided into a plurality of units.

The forward and backward motion detection process detects motion vectors using a block matching method.

Further, the picture coding method according to the present invention calculates the picture data of the reference block and the picture data of the search block within the search range, detects the forward motion vector and the backward motion vector, and uses the bidirectional motion vectors. When encoding is performed, the detection timing of the motion vector in one direction is shifted in time forward than the encoding timing, thereby making at least a part of the image data of the search block common to bidirectional motion vector detection.

Here, the motion vectors detected in time before the encoding timing are stored in the storage unit, and are read in accordance with the encoding timing.

In addition, the image encoding apparatus according to the present invention is an image encoding apparatus which performs image encoding by performing motion detection from images in forward and backward directions in time, comprising: input image storage means for storing an input image to encode, and Omnidirectional motion detection means for detecting an omnidirectional motion vector of the first image from a search image that is temporally omnidirectional than the first image stored by the input image storage means, and stored in the input image storage means; A backward motion detecting means for detecting a backward motion vector of a second image that is temporally before the image from the searched image, and a forward motion vector of the first image detected by the forward motion detecting means; Motion vector storage means and the motion vector storage when the first image is encoded. The means reads the forward motion vector of the first picture, the backward motion vector for the first picture is detected by the backward motion vector detection means, and these forward and backward directions are encoded using the motion vector. And encoding means for performing the operation.

Here, the first and second images are processed in units of blocks in which the input image is divided into a plurality of units.

In addition, the forward and backward motion detection means detects a motion vector using a block matching method.

EMBODIMENT OF THE INVENTION Hereinafter, the Example of the image coding method and apparatus which concern on this invention is described with reference to drawings. This embodiment is an image coding apparatus in which the configuration of the motion vector detecting circuit 1 is a motion vector detecting circuit shown in FIG. 5 in the image coding apparatus based on the MPEG system shown in FIG. In addition, below, the description about each other part of the said image coding apparatus is abbreviate | omitted.

In FIG. 5, the motion vector detection circuit includes a reference frame memory 11, a search frame memory 12, a forward prediction motion detection circuit 13, a backward prediction motion detection circuit 14, It consists of the motion vector for omnidirectional prediction, the residual storage memory 15, and the motion compensation mode determination circuit 16. As shown in FIG.

The reference frame memory 12 stores an image of a search frame. Then, the search block data b is read out and input to the forward prediction motion detection circuit 13 and the backward prediction motion detection circuit 14 in common.

The omnidirectional prediction motion detection circuit 13 obtains the omnidirectional prediction motion vector c1 and its residual d1 and outputs it to the omnidirectional prediction motion vector and the residual storage memory 15.

In the forward prediction motion vector and the residual preservation memory 15, the input forward prediction motion vector c1 and the residual d1 are stored. At the same time, the motion vector c2 and the residual d2 for omnidirectional prediction already stored are read out and input to the motion compensation mode determination circuit 16.

The backward predictive motion detecting circuit 14 obtains the backward predictive motion vector e and the residual f, and provides the motion compensation mode determining circuit 16 to the backward predictive motion vector e. This motion vector e is obtained by the block matching method as in the conventional example.

The motion compensation mode determination circuit 16 uses the input data, determines the motion compensation mode, detects motion vectors and residuals, and outputs the result g. Fig. 6 is a block diagram showing a second form of the motion vector detection circuit. Here, the same numerals and symbols as those in FIG. 5 are attached to the same parts as in FIG. 5. The motion vector detection circuit shown in FIG. 6 includes a reference frame memory 11, a search frame memory 12, a motion vector for forward prediction, and a residual storage memory 15 in FIG. ), And the signal lines are bus type.

The operation of the motion vector detection circuits shown in Figs. 5 and 6 will be described with the flowchart of Fig. 7. 8 shows the operation timing.

As shown in Fig. 7, in the motion vector detection circuit, the reference frame 0 is not an encoded frame, and the motion vector and the residual in all directions from this reference frame 0 are forward in step S1. The prediction motion vector and the residual storage memory 15 are stored. At the time to be encoded, the backward prediction vector from the reference frame 1 is obtained at step S2, the forward motion vector and the residual stored in step S1 are read out, and the reference frame 1 is read. Is encoded. This operation is then repeated.

As shown in Fig. 8, for example, at time t0, motion detection is performed using the current frame B4 as a reference macro block and the frame I3 as a search frame for omnidirectional prediction. The motion vector and the residual are recorded in the motion vector for forward prediction and the bank MV0 of the residual storage memory 15.

At time t1, motion detection is performed using the current frame B4 as a reference macroblock and the frame P6 as a search frame for backward prediction. At the same time, the omnidirectional motion vector and the residual stored in the bank MV0 are read at time t0. As a result, the bidirectional motion vectors required for encoding the frame B4 coincide at time t1.

At time t1, data of frame B7 is also transmitted as reference block data, and an omnidirectional motion vector and its residual from frame B7 to frame P6 are detected so that an omnidirectional prediction motion vector, residual It is written to the bank MV1 of the storage memory 5. This omnidirectional motion vector and residual are read at time t2 and used for encoding of frame B7.

In addition, at the time t2, the backward motion vector and the residual from the frame B7 to the frame P9 are similarly obtained.

In this way, one type of search data (data at frame P6 at time t1) and two types of reference block data (data at frame B7 and frame B4 at time t1) are always transmitted. By reading and recording the forward motion vector and the residual in the memory, it is possible to obtain a bidirectional motion vector required for bidirectional prediction and to encode using bidirectional prediction. That is, the timing of detecting the motion vector in the forward direction and the residual thereof is shifted from the timing of the motion vector in the backward direction and the residual detection (and thus also the encoding timing) so that one type of search data is converted into two types of reference block data. Common to. This commonization reduces the transmission amount of the search data. In addition, the residuals are not necessarily stored when they are not necessary for the processing.

In this embodiment, the amount of data transmission required for bidirectional prediction is calculated. Since the amount of transmission when reading and writing the motion vector and the residual thereof is negligible by several tens of bits, if only the amount of transmission of the reference block data and the search block data is calculated, the amount of transmission of the reference block is 8x16x16x2 = The transmission amount of 4K bits and the search block is 8x48x48x1 = 18K bits, and 22K bits in total, which is about 40% smaller than the conventional 38K bits.

In the above description, the case where the search range is ± 16 in both the horizontal direction and the vertical direction has been described. However, the wider the search range, the more the transmission portion of the search block data increases. Grows In addition, although the above description is a case where there are two B pictures, this invention is not limited to this. In the above description, although the search range of the forward prediction is the same as the search range of the backward prediction, the present invention can be applied even when the search range in one direction (forward direction) is wide. Some of the search data is common.

Further, in the motion vector detection circuit, the number of frame memories used as the reference memory 11 and the search frame memory 12 can be reduced than the number of conventional frame memories described with reference to FIG. This frame memory reduction will be described below with reference to FIG.

When B1 is supplied as an input image, memory-0 holds B1 until encoding is completed. Each B frame is held in each memory until it is similarly encoded. Here, the I and P frames used as the search frame are recovered as soon as the motion vector in the forward direction has already been detected and used for the motion vector detection in the backward direction. Thus, according to the image coding apparatus using the motion vector detection circuits shown in Figs. 5 and 6, the time for storing the I and P frames in the memory can be shortened. For this reason, the number of memories used as the reference frame memory and the search frame memory can be reduced from six to one in the conventional motion vector detection circuit, for example, from six to one in FIG.

As described above in detail, according to the present invention, it is possible to reduce the amount of transmission of search data when performing bidirectional motion detection. As the data transfer amount is reduced, the bus width from the memory can be reduced and the transfer clock can be made slow. For this reason, the realization by hardware becomes easy, and as a result, hard cost reduction and low power consumption are realized. In addition, the memory for storing the original image can be reduced.

Claims (8)

In the image encoding method for performing image encoding by detecting motion from images in forward and backward directions in time, An input image storage step of storing an input image to be encoded into a storage unit; An omnidirectional motion detection step of detecting an omnidirectional motion vector of the first image from a search image that is temporally omnidirectional than the first image stored in the storage in the input image storage step; A backward motion detection step of storing a backward motion vector of a second image stored in a storage unit in the input image storage step and temporally ahead of the detected image from the search image; A motion vector storing step of storing an omnidirectional motion vector of the first image detected by the omnidirectional motion detecting step; When the first image is encoded, the forward motion vector of the first image is read in the motion vector preservation process, and a backward motion vector for the first image is detected from the backward motion vector detection process, and these And an encoding step of performing encoding using forward and backward motion vectors. The method of claim 1, And the first and second images are processed in block units obtained by dividing the input image into a plurality of units. The method of claim 2, And said forward and backward motion detection process detects a motion vector using a block matching method. When the image data of the reference block and the image data of the search block within the search range are calculated to detect the forward motion vector and the backward motion vector, and encoding is performed using the bidirectional motion vectors, the motion vector in one direction is obtained. At least a part of the image data of the search block is common to bidirectional motion vector detection by shifting the detection timing of the signal forward in time forward than the encoding timing. The method of claim 4, wherein And a motion vector detected in time before the encoding timing is stored in a storage unit, and read according to the encoding timing. In the image encoding device which performs image encoding by detecting motion from images in the forward and backward directions in time, Input image storage means for storing an input image to be encoded, Omnidirectional motion detection means for detecting an omnidirectional motion vector of the first image from a searched image that is temporally omnidirectional than the first image stored by the input image storage means; Backward motion detecting means for storing the backward motion vector of the second image stored in the input image storage means and temporally ahead of the search image from the search image; Motion vector storing means for storing an omnidirectional motion vector of the first image detected by the omnidirectional motion detecting means; When the first picture is encoded, the forward motion vector of the first picture is read by the motion vector storing means, and a backward motion vector for the first picture is detected from the backward motion vector detecting means, and these And encoding means for encoding using motion vectors in the forward and backward directions. The method of claim 6, And the first and second images are processed in block units obtained by dividing the input image into a plurality of units. The method of claim 7, wherein And said forward and backward motion detection means detects a motion vector using a block matching method.