JP2000350209A - Method and device for high-speed movement estimation for real-time moving video encoding - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method and a device for high-speed movement estimation for real-time moving video encoding. SOLUTION: This method includes a) a stage wherein a layer 1 is provided by lowering the resolution of a layer 0 and a layer 2 is provided by lowering the resolution of the layer 1, b) a stage for calculating the mean absolute error in a search area of the layer 2, c) a stage for selecting the position where the mean absolute error value calculated in the stage (b) is minimum as the initial search point of the layer 1, (d) a stage for further selecting another initial search point of the layer 1, e) a stage for calculating the mean absolute error of the search area of the layer 1, f) a stage for selecting the position where the mean absolute error value becomes minimum in the stage (e) as the search center point of the layer 0, g) a stage for calculating the mean absolute error of the search area of the layer 0 selected in the stage (f), and h) a stage for determining a final moving vector with the position information calculated in the stage (g) and distance information based upon the origin.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an ultra-high-speed motion estimation method for real-time video coding, and more particularly, to a method for reducing the resolution in motion estimation for video coding such as MPEG-2. Determine a plurality of motion vector candidates,
After determining a motion vector candidate using the motion vector correlation of peripheral blocks, a motion estimation is performed by selecting a search area around a plurality of motion vector candidates and calculating a motion vector, thereby shortening a calculation time of the motion vector. About the method.

[0002]

2. Description of the Related Art In order to obtain a high data compression rate, motion-compensated coding for removing temporal redundancy is widely used for encoding moving pictures, and MPEG-1, 2, 4, and H -
It occupies an important position in international video coding standards such as 263.

[0003] Motion compensation coding predicts an image most similar to an input image from information of a previous frame through motion estimation.
This refers to transform coding of the difference image between the predicted image and the input image.

FIG. 1 shows a general apparatus for encoding such a moving image. Referring to FIG. 1, a general moving image encoding apparatus includes a frame memory 102, motion estimating units 104 and 106,
Motion compensator 108, subtractor 110, discrete cosine transform 112,
Quantizing section 114, inverse discrete cosine transform section 118, adding section 120,
It includes a frame delay unit 122, a forward decomposition and coding rate controller 124, a variable length coding unit 126, and a buffer 128.

[0005] The image input in units of frames is stored in the frame memory 102, and the first motion estimator 104 calculates a motion vector for the image in units of integer pixels. The second motion estimating unit 106 uses the motion vector input from the first motion estimating unit 106, the video signal input from the frame memory 102, and the information of the previous frame input from the frame delay unit 122, and And calculate a motion vector in half-pixel units for the captured image.

[0006] The motion compensator 108 performs motion compensation based on the motion vector input from the second motion estimator 106 and information on the previous frame input from the frame delay unit 122, and outputs a predicted image for the current frame. I do. The prediction image thus motion-compensated is subtracted by the subtraction unit 110 from the current image input from the frame memory 102 to obtain a difference image from the current image, thereby eliminating the temporal redundancy of the moving image. Such a motion estimation and compensation process is performed in units of 16 × 16 blocks, and this block is generally called a macroblock. Next, the difference image obtained after the motion estimation and compensation is subjected to the discrete cosine transform and quantization by the discrete cosine transform unit 112 and the quantization unit 114, so that the spatial overlap of the difference image remains is eliminated. Finally, the motion vector and the quantized difference image are encoded by the variable length encoding unit 126, and then transmitted in the form of a bit string through the buffer 128.

On the other hand, the forward decomposition and coding rate control section 124 controls the coding rate of the variable length coding section 126, and the quantized video signal is again interpolated by the inverse quantization section 116 and the inverse discrete cosine transform section 118. After being restored, the video signal motion compensated by the addition unit 120 is added to the frame delay unit 12
Stored in 2. The video frame stored in the frame delay unit 122 corresponds to the video frame before one frame. The previous video frame information stored in the frame delay unit 122 is input to the second motion estimation unit 106 and the motion compensation unit 108.

As a motion estimation and compensation method for processing a moving image, there are generally a frame-based motion estimation and compensation method and a field-based motion estimation and compensation method, which are known to those skilled in the art. The description is omitted in this specification.

On the other hand, in the conventional motion estimation method using the global search method, a current frame is divided into blocks each having a predetermined size, and each block is divided into a search area of a reference frame according to a given matching criterion. This is a method of estimating a two-dimensional motion vector of each block by searching for the position of an optimal matching block by comparing all blocks in the block. As a matching criterion for searching for an optimal matching block in such a conventional block matching method, a mean absolute error (MAD) that is relatively easy to calculate is used.
Use e). Such an average absolute error value is

[0010]

(Equation 1)

Is calculated by using F _t (k, l)
Is the luminance value of the pixel at position (k, l) in the current frame, and f _t-1 (k + i, l + j) is the value from the position (k, l) in the previous frame.
This is the luminance value of the pixel at the position shifted by the position (i, j).

On the other hand, when coding is performed by such a block matching method, the maximum motion estimation range is determined in consideration of the actual video motion. The global search method for estimating a motion vector by comparing all blocks in the motion estimation range determined in this way with the current block has the highest performance in terms of prediction gain, but an excessively large amount of calculation is required. There are disadvantages required. For example, if the maximum displacement per frame is ± p (pulses / frame) for a block of size M × N, the size of the search area in the reference frame is (M + 2p) × (N + 2
p). That is, since the number of candidate blocks to be compared according to the matching criterion is (2p + 1) ² , there is a problem that the larger the p is, the more difficult it is to implement a moving picture encoder in real time.

Another prior art for solving such a problem is disclosed in the technical document "High-speed hierarchical motion vector estimation algorithm using an average pyramid. NAM, KM, Kim, J. et al.
S., Park, RH, Shim, YS, IEEE Trans.of Circ. & S
yst. for Video Tech, 1995, 5, (4), pp. 344-351 ”
"Improvement of accuracy and cost reduction of three-step search block matching algorithm for video coding, IEEE Trans. Circ.
& Syst. For.Video Tech, 1994, 4, (1), pp.88-90) ”
Is shown in According to the above document, a fast hierarchical search technique using a plurality of candidates to replace the global search technique has been proposed.

However, a strategy using multiple candidates as in the above method can reduce the problem of local minima that occurs due to the hierarchical search, but is still different in order to achieve the performance next to the global search method. There is a problem that requires a large amount of calculation. In addition, since the method is based on a three-step search technique, it has an undesired disadvantage in motion estimation in a wide search area.

[0015]

SUMMARY OF THE INVENTION A technical problem to be solved by the present invention is to provide a motion estimation method capable of estimating a motion vector at a high speed with a reduced amount of calculation for calculating the motion vector.

Another technical problem to be solved by the present invention is to provide a motion estimating apparatus embodied by the above-described motion vector estimating method.

[0017]

According to an aspect of the present invention, there is provided a motion estimation method comprising: (a) when an input video frame is set to layer 0, the resolution of layer 0 is reduced; 1, the step of lowering the resolution of the layer 1 to include the layer 2, the step of (b) calculating the average absolute error for the search area of the layer 2, and the step of (c) calculating in the step (b). At least two positions where the averaged absolute error value
(D) selecting one or more hierarchies using the correlation of neighboring blocks of the current block.
Further selecting the initial search point in (1), and (e) hierarchy
Calculating the average absolute error for the search area centered on the initial search center point in 1; and (f) searching the hierarchy 0 for the position where the average absolute error value calculated in the step (e) is minimum. Selecting the center point, and (g) at layer 0, the (f)
Calculating an average absolute error for the search area centered on the initial search center point selected in the step, and (h) position information in which the average absolute error value calculated in the (g) step is minimized. Determining a final motion vector from information on the distance to the origin.

According to another aspect of the present invention, there is provided a motion estimation method comprising: (a) when an input video frame is set to layer 0, the resolution of layer 0 is reduced to include layer 1; Lowering the resolution of layer 1 and providing layer 2; (b) calculating the average absolute error for the search area of layer 2; and (c) the average absolute calculated in step (b). Selecting two positions where the error value is minimized as the initial search center point in the layer 1, and (d) further selecting one initial search center point using the correlation of the peripheral blocks of the current block, (e) calculating the average absolute error for the search area with the three initial search center points as the search centers in layer 1, and (f) minimizing the average absolute error value calculated in step (e). (G) selecting a search area of the hierarchy 0 as the initial search center point in the hierarchy. And (h) determining a final motion vector based on distance information between the position data and the origin at which the average absolute error value calculated in the step (g) is minimized. It is characterized by.

Preferably, in the step (a), the resolution of the input layer 0 is reduced to 1/2 and 1/4 in the horizontal and vertical directions, respectively, to provide the layers 1 and 2 respectively.

In the step (d), the average value of the motion vector of the block whose absolute value of the motion vector difference between the neighboring blocks is less than a predetermined threshold value is calculated, and the position where the average value of the motion vector is minimized is calculated. Is preferably selected as the initial search center point in hierarchy 1.

The search area in the steps (e) and (g) preferably comprises 5 × 5 pixels.

According to another aspect of the present invention, there is provided a motion estimation method according to the present invention, wherein when an input video frame is set to layer 0, the resolution of layer 0 is reduced and layer 1 is provided.
Resolution adjusting means for lowering the resolution and having hierarchy 2;
The first mean absolute error MAD for the search area of layer 2 and layer 1
The second average absolute error for the search area with the position corresponding to the motion vector candidate as the search center point, and
(2) Mean absolute error calculating means for calculating a third mean absolute error for a search area having a position where the mean absolute error value is the minimum as the search center point of hierarchy 0, and the first mean absolute error value is minimized Initial search point selection means for selecting at least two or more positions as initial search center points in the hierarchy 1 and further selecting one or more initial search center points using the correlation of peripheral blocks of the current block; and (3) A final motion vector determining means for determining a final motion vector using distance information between the position where the average absolute error value is minimized and the origin.

[0023]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of an ultra-high-speed motion estimation method and apparatus for real-time video coding according to the present invention will be described in detail with reference to the accompanying drawings. FIG. 2 is a flowchart illustrating main steps of a motion estimation method according to an embodiment of the present invention. FIG. 3 is a diagram illustrating search positions for each layer when performing a search for each layer according to the motion estimation method. Indicated. FIG. 3 is frequently referenced below. Referring to FIG. 2, the motion estimation method according to the present invention first reduces the resolution of an input video frame (layer 0) to と and / 4 in the horizontal and vertical directions, and Step 2 is provided (step 202). The reason for reducing the resolution in this way is to reduce the amount of calculation for calculating the motion vector by reducing the search area. For example, if the size of the entire search area is R × R, the range of the search area in this embodiment is (R / 4) × (R / 4).

Next, an average absolute error (MAD) is calculated for the search area (SR ⁽²⁾ : FIG. 3) of the hierarchy 2 (step 204). Search area hierarchy ^{2 (SR (2):} 3) time for the N _S and a predetermined positive integer, and 0, respectively because the horizontal and vertical is determined in a region from -N _S to + N _S -1 The area includes 2N _S in each of the horizontal and vertical directions, and the number of search positions is (2N _S × 2N _S ) / 16, that is, (N _S × N _S ) / 4.

Next, the two positions where the average absolute error value calculated in the step 204 is minimum, that is, the prediction error is the smallest, are selected as the initial search center points in the next step (layer 1) (step 1). 206).

One initial search center point is further selected together with the two initial search center points, and the process is shown in FIGS.
This will be described with reference to FIG. Digital video compression is performed in the same block unit as a macro block. An example of the processing order of the blocks is that the blocks are sequentially processed.

The peripheral blocks used in the present invention are shown in FIG.
As shown in FIGS. 4A to 4F, three interrelated blocks are used. The motion vectors are divided into five groups as shown in consideration of the correlation between the motion vectors. For example, when the motion vector MVs of the peripheral block is the same as that of FIG. 4A, the absolute value of the motion vector difference between the peripheral blocks is

[0028]

(Equation 2)

And D is defined as a threshold value for checking the similarity between two motion vectors. next,
The motion vectors are divided into five groups in consideration of the correlation between the motion vectors. That is, referring to FIG. 4 (B), group 1 has ξ ₁
≦ D, ξ ₂ ≦ D, ξ ₃ ≦ D. Referring to FIG. 4 (C), group 2 satisfies ξ ₁ ≦ D, ξ ₂ > D, ξ ₃ > D. It is. 4 (D) and FIG. 4 (E), group 3 represents ξ ₁ > D, ξ ₂ ≦ D, ξ ₃ > D
4 is a case of satisfying _{ξ 1> D, ξ 2>} D, ξ 3 ≦ D.
In cases other than those mentioned above, Group 5 applies. The motion vector is, for example, MV = (MV ₁
+ MV ₂ + MV ₃ ) / 3, MV = (MV ₁ + MV ₂ ) /
2, MV = (MV ₂ + MV ₃ ) / 2 for group ₃ , group 4
In the case of, the calculation is performed as MV = (MV ₁ + MV ₃ ) / 2. The position at which the calculated average value of the motion vectors is minimized is selected as the initial search center point.

As described above, the method of dividing into five groups according to the absolute value of the difference between the motion vector values and determining the initial search point by calculating the average motion vector for each group is merely for explanation. , The motion vector information of the peripheral block is read, and another method of selecting one initial search center point using the motion vector correlation of the peripheral block can be used in a modified manner. Is not limited.

Next, in layer 1, the average absolute error is calculated for each of the search areas SR ₁ ⁽¹⁾ , SR ₂ ⁽¹⁾ , and SR ₃ ⁽¹⁾ centered on the three initial search center points. (Step 210). In the present embodiment, the search area in the first hierarchy is set to include ± 2 pixels including the search center point, that is, 5 × 5 pixels in total.

Next, the position at which the average absolute error value calculated in step 210 is minimized is selected as the initial search center point in layer 0 (step 212), and ± 2 pixels are selected with respect to the selected initial search center point. The average absolute error is calculated using the region as a search region of the hierarchy 0 (step 214).

The final motion vector is determined using the distance information between the position where the calculated average absolute error value is minimized and the origin (step 216). In the case of a two-dimensional image, the determined final motion vector has, for example, x-axis and y-axis coordinate values.

In order to evaluate the performance of the motion estimation method according to the present invention, a simulation experiment using an MPEG-2 experimental image was performed. To explain the experimental conditions, the input images include football (fb), car (car), suzie (sus), car and calendar (mob) and cheerleader in the images known to those skilled in the art as experimental images. (c
heer) of five MPEG-2 video sequences.

All pictures have a frame rate of 30 Hertz, each consisting of 150 frames. The size of each frame is 720 × 480. 6 with search point spacing of 4, 2, 1
4 × 64 (N _S = 32), 10 × 10, and 5 × 5 search areas are each hierarchical level 2,
Selected by 1,0. Block size N _B at layer 0
Is 16, and the critical value D for examining the correlation between the peripheral blocks is set to 8 in order to consider that the similarity decreases when 8 pixels deviate. This value is a value corresponding to twice the search point interval in the hierarchy 2.

In order to prove the effect of the motion estimation method according to the present invention, the performance of the global search technique and the performance of a general hierarchical search technique were compared. In layer 2 of the general hierarchical search technique that adopts the multiple candidate strategy, three motion vectors having only the minimum prediction error are determined as candidates. On the other hand, the proposed technique predicts the third candidate in the motion vector of the surrounding block. table
Figure 1 shows the results of comparing the performance of motion prediction and motion compensation for the first 150 frames of MPEG-2 experimental video.

[0037]

[Table 1]

Referring to Table 1, the motion estimation method according to the present invention achieves only a small performance reduction of about 0.2 dB as compared with the global search technique, but has a complexity of only 1.5% of the global search technique. It is shown that. When the number of operations required to compute the mean absolute error MAD per pixel N, the number of blocks and N _B, the complexity,

[0039]

(Equation 3)

Can be estimated by Here, N _s ^2/64 term hierarchical 2,3 × 25/4 term hierarchy 1,25 term shows the proportional value of the calculated amount according resolution hierarchy 0. Therefore, the entire search technique against 32 and 64 of the _{N s (FSBMA: full scale block} matching an
alysis) of 1.5% and 0.7%.

Table 2 shows the results of applying the motion estimation method according to the present invention to an MPEG-2 encoder requiring a relatively large search area. In this experiment, GOP (Group of Picture) N
Is 12, the distance M between P frames is 2, the target bit rate T is 6Mbps
The experiment proceeded with setting. Table 2 shows the result of comparing the peak signal to noise ratio of the restored image.

[0042]

[Table 2]

Referring to Table 2, the motion estimation method according to the present invention achieves a remarkable reduction in complexity of 0.9% of the complexity of the global search technique, and achieves almost the same peak signal-to-noise ratio PSNR. .

As described above, the motion estimation method according to the present invention can reduce the amount of calculation by efficiently using the correlation between a plurality of motion vector candidate strategies based on the inter-frame minimum prediction error and the motion vector space. High-speed motion estimation can be performed with a small amount.

The ultra-high-speed motion estimation method for real-time video coding according to the present invention can be implemented by a motion estimation device. The motion estimator (not shown) includes a resolution adjuster, an average absolute error calculator, an initial search point selector, and a final motion vector determiner.

The operation of the above-described apparatus will be described. First, the resolution adjusting means converts the input video frame into layer 0.
Then, the resolution of the layer 0 is reduced and the layer 1 is provided. In addition, the resolution of the layer 1 is reduced and the layer 2 is provided. For example,
When the resolution is reduced to 1/2, an average of two adjacent pixels is used, and when the resolution is reduced to 1/4, an image frame can be realized using an average of four adjacent pixels.

The average absolute error calculating means includes a first average absolute error for the search area of the hierarchy 2, a second average absolute error for the search area having a position corresponding to the motion vector candidate in the hierarchy 1 as a search center point, and A third average absolute error is calculated for a search area where the position at which the second average absolute error value becomes the minimum at 0 is the search center point of layer 0.

The initial search point selection means determines at least two or more positions where the first average absolute error value is minimum in the next step.
Select as the initial search center point in (layer 1). Further, the motion vector candidate selecting means further selects one or more other initial search center points as initial search center points in the next stage (layer 1) by using the correlation of the peripheral blocks of the current block.

The final motion vector determining means determines the final motion vector using the distance information between the position where the third average absolute error value is minimum and the origin.

Each step of the motion estimation method can be created by a computer-executable program. Further, the present invention can be implemented by a general-purpose digital computer that operates the program from a medium used in a computer. The medium includes a magnetic recording medium such as a floppy disk or a hard disk, an optical recording medium such as a CD-ROM or a DVD, and a storage medium such as a carrier wave such as a transmission through the Internet. Also, such functional programs, codes and code segments can be easily deduced by a programmer in the technical field to which the present invention belongs.

[0051]

As described above, the motion estimation method according to the present invention performs calculation by efficiently using the correlation between a plurality of motion vector candidate strategies based on the inter-frame minimum prediction error and the motion vector space. High-speed motion estimation is possible with a small amount.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a schematic structure of a video encoding device to which the present invention is applied.

FIG. 2 is a flowchart showing main steps of a motion estimation method according to an embodiment of the present invention.

FIG. 3 is a diagram illustrating a search position for each layer when a search for each layer is performed by the motion estimation method according to the embodiment of the present invention.

FIGS. 4A to 4F are diagrams for explaining a process of determining a motion vector candidate using a correlation of a current block with a neighboring block in a motion estimation method according to an embodiment of the present invention; .

[Explanation of symbols]

SR ⁽²⁾ Search area of layer 2 SR ₁ ⁽¹⁾ , SR ₂ ⁽¹⁾ , SR ₃ ⁽¹⁾ Search area of layer 1 SR ⁽⁰⁾ Search area of layer 0

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Lin Keigaki 728-32, Samseong-dong, Gangnam-gu, Seoul, Republic of Korea No. 32, Shosei Billat No. 304 Cave No. 162 Jibayashi Hikari Apartment Building No. 110 F-term (reference) 5C059 KK15 KK19 MA00 NN02 NN28 PP04 UA02

Claims (9)

[Claims] 1. A motion estimation method using block matching for compressing a moving image, comprising the steps of: (a) when an input video frame is a layer 0,
Lowering the resolution of the hierarchy 1 and providing the hierarchy 1 with a lower resolution of the hierarchy 1; (b) calculating the average absolute error for the search area of the hierarchy 2; and (c) (b) selecting at least two or more positions at which the average absolute error value calculated in the step becomes minimum as the initial search points in the layer 1; and (d) using the correlation between neighboring blocks of the current block. (E) calculating an average absolute error with respect to a search area centered on the initial search center point in layer 1; ) Selecting the position where the average absolute error value calculated in the step becomes the minimum as the search center point in the layer 0; and (g) in the layer 0, centering on the initial search center point selected in the step (f). Calculating the average absolute error for the search area to be calculated, and (h) calculating the average absolute error in the step (g). Motion estimation method characterized by comprising the step of determining the final motion vector from the distance information between the position information and the origin mean absolute error value is minimized with. 2. A motion estimation method using block matching for compressing a moving image, comprising the steps of: (a) when an input video frame is a layer 0,
Lowering the resolution of the hierarchy 1 and providing the hierarchy 1 with a lower resolution of the hierarchy 1; (b) calculating the average absolute error for the search area of the hierarchy 2; and (c) (b) selecting two positions, at which the average absolute error value calculated in the step is minimum, as the initial search center point in the layer 1, and (d) one initial position using the correlation of peripheral blocks of the current block. Further selecting a search center point; (e) calculating an average absolute error with respect to a search area having three initial search center points as search centers in layer 1; (f) in the step (e). Selecting a position at which the calculated average absolute error value is minimized as an initial search center point in the hierarchy; (g) calculating an average absolute error for a search area in the hierarchy 0; g) From the position data that minimizes the average absolute error value calculated in step Determining a final motion vector. 3. In the step (a), the resolution of the input layer 0 is reduced to 1/2 and 1/4 in the horizontal and vertical directions, respectively, and
The motion estimation method according to claim 2, comprising: and a hierarchy 2. 4. The step (d) calculates a motion vector average value of a block whose absolute value of a motion vector difference between neighboring blocks is equal to or less than a predetermined threshold value, and calculates a position where the motion vector average value is minimized. The motion estimation method according to claim 2, wherein is selected as an initial search center point in hierarchy 1. 5. The method according to claim 2, wherein the search area in the steps (e) and (g) comprises 5 × 5 pixels. 6. A motion estimator using block matching to compress a moving image, wherein when an input video frame is set to layer 0, the resolution of layer 0 is reduced and layer 1 is provided. Lower resolution and hierarchy
Resolution adjusting means comprising: a first average absolute error for a search area of layer 2; a second average absolute error for a search area having a position corresponding to a motion vector candidate in layer 1 as a search center point; In the second
Average absolute error calculating means for calculating a third average absolute error for a search area having a position where the average absolute error value is the minimum as the search center point of the hierarchy 0, at least the first average absolute error value is the minimum An initial search point selection unit that selects two or more positions as initial search center points in the hierarchy 1 and further selects one or more initial search center points using the correlation of peripheral blocks of the current block; A motion estimating device comprising: a final motion vector determining means for determining a final motion vector using distance information between a position where an average absolute error value is minimized and an origin. 7. The resolution adjustment means according to claim 1, wherein
7. The motion estimating device according to claim 6, wherein the resolution is reduced to 1/2 and 1/4 to provide layers 1 and 2, respectively. 8. The means for selecting an initial search point calculates a motion vector average value of a block whose absolute value of a motion vector difference between neighboring blocks is equal to or less than a predetermined threshold value, and determines that the motion vector average value is a minimum. 7. The motion estimating device according to claim 6, wherein a position that becomes the initial search center point in hierarchy 1 is selected. 9. The motion estimation device according to claim 2, wherein the search area includes 5 × 5 pixels.