JPH11122618A - Image coder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a motion vector over a wide range with a simple configuration. SOLUTION: In the case of retrieving a motion vector with respect to a target block 30, an offset 31 is calculated based on a motion vector between a macro block around the target block 30 and a macro block equivalent to the target block 30 of one preceding frame. A 1st layer retrieval is conducted in a range of ±4 pixels around the offset 31. Then a 2nd layer retrieval is conducted within a range of ±2 pixels around a retrieval reference position 33 decided as a macro block minimizing the error function. Similarly a final stage retrieval is conducted within a range of ±1 pixel. In the 1st to 2nd layer retrieval, low pass filter processing is applied to the image and at the final layer, it is relieved.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image coding apparatus suitable for image coding based on the MPEG standard, and more particularly to an improvement for facilitating detection of a motion vector.

[0002]

2. Description of the Related Art Conventionally, in a moving image coding technique represented by the MPEG standard, a motion compensation using a motion vector expressing the amount of local image motion between frames has been introduced. Encoding is performed. As a result, the compression efficiency of moving images is dramatically improved.

[0003]

However, in the conventional image coding technique, when detecting a motion vector,
There is a problem that complicated calculation is required, and the configuration of the device for that purpose must be complicated. Moreover, there is a problem that it is not easy to search for a motion vector over a wide range in an image.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems in the conventional image coding technology, and realizes detection of a motion vector with a simple configuration, and furthermore, searches for a motion vector over a wide range. It is an object of the present invention to provide an image encoding device that facilitates image encoding.

[0005]

According to a first aspect of the present invention, there is provided an apparatus comprising:
In an image encoding apparatus that realizes inter-frame encoding that incorporates motion compensation using a motion vector, when determining a motion vector by comparing a current image with a reference image, a wide image area is narrowed to a narrow image area. Step by step, the search for the motion vector is performed over a plurality of layers, and in the initial layer including the first layer, the current image and the reference image are subjected to low-pass filter processing, and then the search is performed. At least in the last layer, the search is performed after the low-pass filter processing is canceled.

According to a second aspect of the present invention, there is provided an image encoding apparatus for implementing inter-frame encoding incorporating motion compensation using a motion vector, wherein the motion vector is determined by comparing a current image with a reference image. At this time, the search for the motion vector is performed in a plurality of hierarchies stepwise from a wide image area to a narrow image area. Moreover, prior to the search of the first hierarchy, an offset as an initial search position is calculated, and in the search of the first hierarchy, the search is performed around the offset, and the offset is
The motion vectors of three blocks which are located around a target block which is a block for which the motion vector of the current image is to be calculated and whose motion vectors are already calculated, and the motion vectors corresponding to the target block. The determination is made based on the motion vector in the block in the reference image.

[0007]

BEST MODE FOR CARRYING OUT THE INVENTION

<1. Overall Configuration of Apparatus> FIG. 1 is a block diagram showing the configuration of an image coding apparatus according to an embodiment. As described below,
The present invention has a main feature in a device for calculating a motion vector (in FIG. 1, the motion vector detecting unit 14). Therefore, FIG. 1 illustrates an apparatus that performs encoding of an image signal based on the widely known MPEG standard. However, the present invention employs an image encoding apparatus that performs inter-frame encoding by incorporating motion compensation using a motion vector. The present invention is applicable to encoding devices in general.

First, an outline of the overall configuration of the apparatus shown in FIG. 1 will be described, including a peripheral portion which is a premise of the configuration of the motion vector detecting section 14. As shown in FIG. 1, an image input unit 1 such as a color camera outputs an image signal representing a captured image in the form of a digital signal. In this specification, in order to avoid unnecessary complication of the description, an image signal representing an image and an image itself are equally represented as an "image" within a range where there is no possibility of confusion.

An image output from the image input unit 1 is temporarily stored in a frame memory 2 on a frame basis. Thereafter, an image is read out from the frame memory 2 in a fixed order for each frame by the frame memory reading unit 3, and the difference circuit 4 and the switching unit 5
Is entered.

The difference circuit 4 calculates a difference between pixel values between the current image 22 sent from the frame memory reading unit 3 and the predicted image 12 sent from the prediction memory 11. The switching unit 5 outputs the difference image 2 output from the difference circuit 4.
3 and the current image 22 from the frame memory reading unit 3
Select and output one between The signal output from the switching unit 5 is input to the conversion unit 6.

The transform unit 6 executes a so-called discrete cosine transform (DCT). In this discrete cosine transform,
For example, for each block composed of 8 × 8 pixels, the input image has the same number (for example, 8 × 8 = 64) of spatial frequency components (DCT coefficients) as the number of pixels constituting the block.
Is converted to the set The DCT coefficients obtained by the transform are input to the quantization unit 7.

The quantization unit 7 refers to a set of coefficient values defined in a quantization table provided therein and a quantization step instruction signal 25 sent from the coding parameter control unit 19 to obtain each DCT. Quantization is performed for each coefficient. That is, the DCT coefficients are converted into quantization coefficients. The quantization step instructing signal 25 gives the width of the quantization based on the quantization table, that is, the quantization step, by uniformly assigning a common scaling factor to the set of coefficient values defined in the quantization table. Correction is performed with a magnification common to all DCT coefficients. As the quantization step is smaller, the deterioration of the image quality is suppressed, but the signal amount is larger.

The quantized coefficients obtained by the quantization unit 7 are input to a variable length coding unit (VLC) 15 and simultaneously to an inverse quantization unit 8. In the inverse quantization unit 8, an operation opposite to that of the quantization unit 7 is performed. Therefore, a signal of the same format as the DCT coefficient before quantization is obtained.

However, since the processing in the quantization unit 7 is generally irreversible, the DC reconstructed by the inverse quantization unit 8
The T coefficient is generally not the same as the DCT coefficient output from the transform unit 6. That is, the output of the inverse quantization unit 8 generally includes a quantization error generated by the processing of the quantization unit 7. It goes without saying that the smaller the quantization step, the smaller the quantization error.

The DCT coefficients obtained by the inverse quantization section 8 are input to an inverse transform section 9. In the inverse transform unit 9, an operation opposite to the DCT (inverse DCT) is executed. As a result, an image signal in the same format as the image signal before DCT conversion is performed by the conversion unit 6 is obtained. This image signal, needless to say,
Generally, it is not the same as the image signal input to the conversion unit 6, but includes a quantization error.

The reconstructed image signal is stored as a reference image 13 in the prediction memory 11 on a frame basis.
The reference image 13 stored in the prediction memory 11 is input to the motion vector detection unit 14. The current image 22 read from the frame memory reading unit 3 is also input to the motion vector detection unit 14 at the same time. The motion vector detection unit 14 determines the current image 22
Is searched for the portion of the reference image 13 closest to. The searched portion is called a predicted image.

The motion vector detecting section 14 further uses a vector representing the displacement from the predicted image to the corresponding macroblock on the current image 22 as a motion vector 24 as a prediction memory 11 and a variable length code. To the conversion unit 15. The prediction memory 11 stores the reference image 13 for each macroblock based on the motion vector 24.
, And sends the predicted image 12 to the difference circuit 4.

Here, a macroblock is a unit of an image in which the motion vector 24 is defined. For example, 16 × 16 pixels for a luminance signal and 8 × 16 pixels for a color difference signal.
It is composed of eight pixels. When a macroblock is defined by 16 × 16 pixels with respect to a luminance signal, for example, a block composed of 8 × 8 pixels obtained by dividing one macroblock into four is used for the luminance signal. As a unit, DCT conversion processing in the conversion unit 6 and quantization processing in the quantization unit 7 are performed. The switching unit 5 is connected to a control unit (not shown), compares the current image 22 and the difference image 23 in macroblock units, and selects a smaller pixel value, in other words, a smaller signal amount.

The variable length coding unit 15 following the quantization unit 7
At the same time, the input quantized coefficient is subjected to variable-length encoding processing such as Huffman encoding, for example.
Motion vector 2 sent from motion vector detecting section 14
4 is added, and as a result, an encoded signal is obtained. The obtained coded signal is multiplexed by a multiplexing unit 16 and then temporarily stored in a transmission buffer 17.

The coded signal stored in the transmission buffer 17 is transmitted to the transmission path 18 as appropriate. From the transmission buffer 17, a status signal notifying that state, that is, the buffer is overflowing or empty, is sent to the encoding parameter control unit 19. Since the coded signal includes the motion vector 24, a decoding device that receives the coded signal and reconstructs the current image 22 can perform decoding in consideration of the motion vector 24.

<2. Motion Vector Detector> Next, the motion vector detector 14 which is a feature of the present invention will be described. FIG. 2 is a flowchart showing a flow of the operation of the motion vector detection unit 14. The motion vector detection unit 14
The operation in FIG. 2 can be realized by including a memory in which a program is mounted and a computer that operates according to the program. The motion vector detection unit 14 can also be configured only with a hardware wafer including an integrated circuit without using a program.

As shown in FIG. 2, detection of a motion vector,
That is, when the motion prediction is started, first, in step S1
In, a low-pass filter is applied to both the current image 22 and the reference image 13. That is,
A filtering process having a low-pass filter characteristic is performed. This process is performed by calculating an average value of adjacent 4 × 4 (= 16) pixels as shown in Equation 1, for example.

[0023]

(Equation 1)

Next, in step S2, an offset, that is, an initial search reference position is calculated for each block in the current image 22. As shown in FIG. 3, the offset is a block in the current image 22 for which the offset is to be calculated, that is, the target block TB.
Is calculated based on the motion vectors in a total of four blocks, that is, three surrounding blocks for which the motion vectors have already been calculated and blocks in the reference image 13 corresponding to the target block TB.

More specifically, as shown in FIG. 3, the motion vectors M of three blocks around the target block TB are obtained.
V1 to MV3 and a motion vector MV4 of a block in the reference image 13 corresponding to the target block TB.
Is substituted into Equation 2 to calculate an offset with respect to the target block TB.

[0026]

(Equation 2)

Here, the code median means a median value. That is, the offsets are the motion vectors MV1 to MV
It is calculated as the average value of the median value of V3 and the motion vector MV4.

When the target block TB is located at the end of the current image 22, as an exception process,
The offset is calculated based on Equation 3.

[0029]

(Equation 3)

When there is no motion vector MV1 to MV4 due to the intra mode, for example, MV = (0, 0). Alternatively, even in the intra mode, a method of calculating the motion vector MV and using the calculated motion vector MV may be employed. However, when the block in the reference image 13 corresponding to the target block TB is in the intra mode, the offset is calculated based on Expression 4.

[0031]

(Equation 4)

Returning to FIG. 2, in the following step S3, a three-layer (three-step) search is started. The three-level search is performed by repeatedly executing steps S3 to S7 three times. FIG. 4 is an operation explanatory diagram illustrating a three-level search performed through a loop of steps S3 to S7 over three times.

Each of the small areas arranged in the form of a matrix shown in FIGS. 4A to 4C corresponds to a pixel representing one macroblock. The relationship between macroblocks and the pixels that represent them is shown in detail in FIG. In FIG. 5, each of the rectangular small areas represents one pixel, and each of the macroblocks 31 and 32 is composed of 16 × 16 pixels.

Returning to FIG. 4, in the first loop, as shown in FIG. 4A, in step S4 following step S3, the offset 31 calculated in step S2 is applied to the target block 30. Is determined. In the following step S5,
An error function value is calculated for each of the nine macroblocks separated by four pixels from the search reference position 32. The error function is, as shown in FIG.
The difference between the pixel value (for example, luminance component) in the corresponding macroblock between the reference image 13 and the reference image 13 is calculated every other pixel, and the sum of the absolute values is calculated.

In FIG. 6, a black rectangular area indicates a pixel whose difference in pixel value is to be calculated, and a white rectangular area indicates a pixel which is not to be calculated. When the calculation procedure is expressed by a mathematical expression, it is as shown in Expression 5.

[0036]

(Equation 5)

Here, the function f represents an error function.

Returning to FIG. 2, in the following step S6,
The error functions of the nine macroblocks are compared with each other, and the macroblock with the minimum error function is set as the search reference position of the next search hierarchy. In the example of FIG. 4A, the block 33 is determined as the next search reference position.

Referring back to FIG. 2, when the process reaches step S7, the process returns to step S3. Then, FIG.
As illustrated in (b), a search within a range of ± 2 pixels around the new search reference position 33 is performed along steps S4 to S6. As a result, for example, as shown in FIG. 4B, the block 34 having the minimum error function is determined as a search reference position of the next search hierarchy. ±
In the search for four pixels and the search for ± 2 pixels, the search is performed with a low-pass filter applied to both the current image 22 and the reference image 13.

When the next search reference position 34 is determined,
The search along steps S4 to S6 is performed again. At this time, as illustrated in FIG. 4C, the search is performed within a range of ± 1 pixel from the center of the new search reference position 34. As a result, for example, as shown in FIG. 4C, the block 35 having the minimum error function is determined as a new search reference position. In the search of the third hierarchy, that is, the search for ± 1 pixel, the search is performed on both the current image 22 and the reference image 13 with the low-pass filter removed.

Although it is possible to determine the position of the motion vector 24 by using the final search reference position 35 obtained through the search of the three layers as it is, it is preferable that the steps S8 to S8 following the step S7 in FIG. Through S11,
Processing for increasing the accuracy of the motion vector 24 is performed. That is, in step S8, for each of the current image 22 and the reference image 13, a half-pixel value is calculated for the macroblock corresponding to the search reference position 35 and the macroblocks located within ± 1 pixel around the macroblock. Will be

Subsequently, in step S9, a search is performed within a range of ± 1/2 pixel around the final search reference position 35. Next, in step S10, the motion vector 24 is determined based on the position of the macroblock giving the minimum value among the nine error functions. That is,
The motion vector 24 is determined as a vector starting from the target block 30 and ending at a macroblock that minimizes the error function. Through the above processing, the motion vector 24 is determined with half-pixel accuracy.

<3. Advantages of Apparatus> The image encoding apparatus shown in FIG. 1 is configured and operates as described above, and thus has the following advantages. First, since a search is performed after the current image 22 and the reference image 13 have been subjected to a low-pass filter, it is possible to prevent a local error minimum point from being erroneously determined as a search reference position. Also, 3
In the search of the hierarchy, the search is performed on the image to which the low-pass filter has been applied only in the search of the first and second layers in the initial stage, and the low-pass filter is released in the search of the final third hierarchy. . Therefore, it is possible to increase the search accuracy while suppressing the number of searches.
Further, the load of filtering is reduced.

Focusing on the fact that the spatial correlation in the image of one frame and the temporal correlation of the image between frames are strong, the offset is determined by the three macroblocks of the current image 22 and the reference. It is determined based on the motion vector of one macroblock of the image 13. For this reason,
The offset is determined stably and accurately by excluding the influence of the fluctuation of the image correlation between frames resulting from the fluctuation of the movement speed and the frame rate.

The offset is determined not based on the average value of the motion vectors of the three macroblocks but on the median value. Therefore, the offset is obtained as an appropriate value corresponding to the actual movement.

Further, since an offset is used, ±
It is possible to search the position of the motion vector 24 in the search area of [-16, +15.5] pixels only by searching in three layers of ± 4, ± 2, ± 1 pixels without searching for 8 pixels. It has become. Since searching for ± 8 pixels is not required, the number of searches is reduced and the speed of detection is increased. This also means that the motion vector detection unit 14 is configured more easily. That is, the detection of the motion vector 24 can be performed with a simple configuration and over a wide search area.

In the calculation of the error function, the difference between the pixel values is calculated every other pixel, so that the position of the search reference position 35 which is finally determined is compared with the case where the calculation is performed over all the pixels. The operation speed and the operation cost are reduced to 1/4 with almost no deterioration in the accuracy of the operation.

[0048]

In the apparatus according to the first aspect of the present invention, in the search for the motion vector in the initial hierarchy, the current image and the reference image are subjected to low-pass filtering, and then the search is performed. The search is performed after the low-pass filter processing is canceled. For this reason, it is possible to prevent the local minimum error point from being erroneously determined as the search reference position, and to suppress the number of searches to a small value, thereby improving the search accuracy. Also, the load of filtering is reduced.

In the apparatus according to the second aspect of the present invention, in the search for the motion vector in the first layer, the search is performed centering on the offset, and the offset is determined based on the motion vectors in the current image and the reference image. For this reason, the influence of the fluctuation of the correlation of the images between frames resulting from the fluctuation of the motion speed and the frame rate is excluded, and the motion vector is determined stably with high accuracy.

[Brief description of the drawings]

FIG. 1 is a block diagram of an image encoding device according to an embodiment.

FIG. 2 is a flowchart showing an operation of a motion vector detection unit in FIG. 1;

FIG. 3 is an operation explanatory diagram of step S2 in FIG. 2;

FIG. 4 is an operation explanatory diagram of steps S3 to S7 in FIG. 2;

FIG. 5 is a partially enlarged view of FIG.

FIG. 6 is an operation explanatory diagram of step S5 in FIG. 2;

[Explanation of symbols]

 13 Reference image 14 Motion vector detection unit 22 Current image

Claims (2)

[Claims] 1. An image encoding apparatus for implementing inter-frame encoding incorporating motion compensation using a motion vector, comprising: determining a motion vector by comparing a current image with a reference image; The search for the motion vector is performed stepwise from a plurality of layers to a narrow image area, and the current image and the reference image are subjected to low-pass filter processing in the initial layers including the first layer. , The search is performed and at least in the final hierarchy:
An image coding apparatus, wherein the search is performed after the low-pass filter processing is canceled. 2. An image coding apparatus which realizes inter-frame coding incorporating motion compensation using a motion vector, wherein a large image area is determined when a current image is compared with a reference image to determine the motion vector. The search for the motion vector is performed over a plurality of hierarchies in stages from a narrower image area to a narrower image area.Moreover, prior to the search for the first hierarchy, an offset as an initial search position is calculated, and in the search for the first hierarchy, The search is performed around the offset, and the offset is located around a target block which is a block for which the motion vector of the current image is to be calculated, and the motion vector has already been calculated. Motion vectors of the blocks and a block in the reference image corresponding to the target block. An image coding apparatus characterized in that it is determined based on a motion vector.