KR100978596B1 - Motion estimation procedure by fast multiple reference frame selection procedure - Google Patents

Abstract

Instead of searching through all the multiple reference frames, the present invention determines a measure for quickly terminating the search process for multiple reference frames. According to an embodiment of the present invention, an optimal reference frame is selected based on a reference frame appropriate value H _(i) defined as a sum of a motion vector cost function value and a reference frame cost function value, and early motion vector search is performed. The image quality and bit rate close to the process of searching and selecting all of the plurality of reference frames can be maintained, while reducing the complexity, thereby substantially reducing the motion estimation time.

Multiple reference frame, bit rate, picture quality, motion vector search, motion vector cost function, reference frame cost function

Description

Motion estimation method by fast multiple reference frame selection procedure

The present invention relates to motion estimation of a motion picture, and more particularly, to a motion motion estimation method using a fast multiple reference frame selection method.

H.264 / AVC, which has high coding efficiency, has recently been widely adopted and used as a video compression standard for new multimedia services. In this H.264 / AVC, more research has been performed, and the compression performance is greatly improved compared to the conventional method, and the complexity has been increased while maintaining stable image quality for various images.

First of all, H.264 / AVC applies precise prediction from signals already encoded around the current block even in the intra picture encoding mode using only the signals in the encoded frame. In addition, an optimal prediction mode is selected from among various prediction directions (horizontal, vertical, diagonal, DC components, etc.) according to the characteristics of the image to be encoded.

Second, a drawback of the video compression encoding method using DCT is distortion seen in a block boundary due to a difference in pixel values between adjacent blocks near the block boundary. To reduce this distortion, a deblocking filter is used. In addition, the processing unit of the DCT also uses a smaller 4 × 4 pixel block unit than the conventional method, so that the distortion is not visible visually.

Third, when performing the entropy encoding (variable length encoding) process on the DCT / quantized signal, the encoding is performed in accordance with the characteristics of the image by changing the variable long list by referring to the surrounding information. In addition, a mode of applying arithmetic encoding to obtain high compression efficiency is also used.

Fourth, as compared with the conventional MPEG method, the image processing unit is set to a smaller block size (4 pixels x 4 pixels = 16 pixels), and precise pixel accuracy (up to 1/4 pixel) enables highly precise motion compensation. It became. In addition, by using multiple reference frames, motion compensation for selecting an optimal prediction signal is utilized. In order to efficiently predict a fade image in which the brightness of an image changes in time, the predictive signal is generated after adaptively calculating and weighting a signal for performing motion compensation.

As described above, H.264 / AVC utilizes multiple reference frame motion estimation. Previous video coding standards such as H.261, H.263, MPEG-1,2 and MPEG-4 have found the best matching block as one of the previous or future reference frames. This was best for a smoothly moving sequence. However, in a high-motion sequence, the best matching block can occur in a reference frame rather than before. Each motion vector block of the multi-reference frame motion estimation performs a motion vector search for each candidate reference frame and determines a reference frame having the least redundant data. This process requires more computation than existing H.263 or MPEG-2, 4 using only one reference frame, and is the main factor that makes up 80% or more of motion compensation in overall complexity.

Blocks having a block size of 8 × 8 or more may have different reference frames for each division. That is, four 8x8 blocks may have different reference frames. When encoded using the multiple reference frame technique, the PSNR is improved by up to 1 dB in the case of an image having a repetitive motion such as a mobile or calendar image.

1 is a conceptual diagram illustrating motion estimation using five reference frames as an example of a plurality of reference frames.

Referring to FIG. 1, based on the current frame N, frame N-1, which is the previous reference frame of the current frame N, frame N-2, which is the previous frame of the frame N-1, to the previous frame N-5 in the same manner as described above. Five reference frames are used.

Multiple reference frame motion estimation instead of searching only one previous reference frame, several reference frames are searched and select the best matching block. Therefore, the multiple reference frames increase the access frequency according to the linear model by increasing the amount of calculation proportional to the number of reference frames, which leads to a 25% increase in complexity per each additional frame. Therefore, there is a need for an efficient frame reference motion estimation method that can reduce the complexity and facilitate the practical application of multiple reference frame motion estimation.

Analyzing the inter-MB ratio predicted by the conventional multi-reference frame motion estimation method described above, at least 80% of the optimal motion vectors determined by the global search method are the closest reference frames (frame N-1). ) Therefore, in many cases a large part of the computation time is consumed by useless reference frames, resulting in increased complexity.

SUMMARY OF THE INVENTION An object of the present invention is to provide a motion estimation method using a fast multiple reference frame selection procedure that improves motion estimation time while maintaining image quality and bit rate compared to a conventional motion estimation method using multiple reference frames.

In the motion estimation method according to an embodiment of the present invention for solving the above problems, in the motion estimation method using a plurality of reference frames, a motion vector cost function value and a reference frame cost function value for each of the plurality of reference frames Selecting an optimal reference frame from among the plurality of candidate reference frames by using a reference frame proper value that is a sum of; and obtaining a motion vector for a current block from the optimal reference frame.

According to an aspect of the embodiment, it is preferable to select the optimal reference frame using the appropriate reference frame value for some or all of the plurality of reference frames.

According to another aspect of the embodiment, the step of selecting the optimal reference frame is for the first reference frame that is the previous frame of the current frame and the second reference frame that is the previous frame of the first frame of the plurality of reference frames Compares the appropriate reference frame values and references a reference frame having a smaller reference frame appropriate value among the first and second reference frames and a third reference frame that is a previous frame of the second reference frame among the plurality of reference frames; It is preferable to select the smaller reference frame as the optimal reference frame if the frame reference value is compared and the smaller reference frame is smaller than the third reference frame.

According to another aspect of the embodiment, if the third reference frame is smaller than the smaller reference frame, the appropriate value of the reference frame of the third reference frame and the fourth reference frame that is the previous frame of the third reference frame. It is preferable to compare appropriate values of reference frames and select the third reference frame as the optimal reference frame when the third reference frame is smaller than the fourth reference frame.

According to another aspect of the embodiment, if the fourth reference frame is smaller than the third reference frame, the appropriate value of the reference frame of the fourth reference frame and the fifth reference frame that is the previous frame of the fourth reference frame. It is preferable to compare an appropriate reference frame value and to select the fourth reference frame as the optimal reference frame when the fourth reference frame is smaller than the fifth reference frame.

According to another aspect of the embodiment, preferably, the plurality of reference frames is five, and when the fifth reference frame is smaller than the fourth reference frame, the fifth reference frame is selected as the optimal reference frame. .

According to another aspect of the present invention, there is provided a motion estimation method using a multiple reference frame selection procedure, wherein a first reference frame that is a previous frame of a current frame and a second reference frame that is a previous frame of the first reference frame are solved. Compares the reference frame proper values with respect to the reference frame and compares the reference frame proper values of the third reference frame that is the previous frame of the second reference frame with the reference frame having the smaller appropriate reference frame value among the first and second reference frames. After the comparison, the optimal reference frame is selected, and the comparison procedure is repeated until the reference frame is determined that the reference frame is determined to be smaller than at least twice before the fourth reference frame and the fourth reference frame that are the previous frames of the third reference frame. Following the fifth frame, which is the previous frame, an optimal reference is made to a reference frame determined to have a small reference frame proper value at least twice. Characterized in that the selected frame.

According to an exemplary embodiment of the present invention, in which a reference frame that does not need to be searched is selected instead of searching all of a plurality of reference frames, the reference frame is effectively selected, a full selection process of selecting all the plurality of reference frames. While the image quality and bit rate close to) can be maintained, the complexity can be reduced to substantially shorten the motion estimation time. Therefore, it can be effectively used for encoding that must improve encoding efficiency or be robust against channel errors.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The following examples are provided to aid the understanding of the present invention and the present invention is not limited to the following examples.

In the motion estimation method using multiple reference frames, a high rate of computation time is consumed by useless reference frames. The present invention provides a motion estimation method by an efficient fast multiple reference frame selection procedure that reduces motion estimation time by reducing the final complexity.

2 is a flowchart illustrating a motion estimation method by a fast multiple reference frame selection procedure according to an embodiment of the present invention.

Referring to FIG. 2, first, motion estimation is performed on a reference frame REF (i, i + 1, i + 2). (S11) Here, REF (i) denotes a previous frame (N−) of the current frame (N). 1), REF (i + 1) represents the previous frame N-2 of REF (i), and REF (i + 2) represents the previous frame N-3 of REF (i + 1). .

The motion estimation uses a conventional full search algorithm (FSA) to calculate a matching error using all coordinates in the search area as candidate points (CPs), The point m, n having the minimum matching error may be set as a motion vector. Accordingly, the motion estimation is performed to obtain motion vectors MV _(i) , MV _{(i + 1)} , and MV _{(i + 2)} for the reference frame REF (i, i + 1, i + 2). In addition, Successive Elimination Algorithm (SEA) using the block sum of the reference block and the candidate block, Three Step Algorithm for selectively searching without searching every point in the search area, New Three Step Algorithm (New Three) Fast vector algorithm that reduces computation by rearranging matching scan order using Step Algorithm, Four-step Search Algorithm, Diamond Search Algorithm, and Image Complexity. A motion vector field adaptive search technique (MVFAST) and a predictive motion vector field adaptive search technique (PMVFAST) may be used. In addition to the above, the motion vectors MV _(i) and MV _{(i + 1)} for the reference frame REF (i, i + 1, i + 2) using a motion estimation method that calculates a matching error using a 1-bit transform _. , MV _{(i + 2)} can be obtained.

2, the reference frame appropriate value defined by the sum of the motion vector cost function value MVCost and reference frame cost function value REFCost of REF (i) with respect to the motion estimation according to the method, H _(i) and A reference frame appropriate value defined by the sum of MVCost and REFCost of REF (i + 1), H _{(i + 1),} and The reference frame proper value H _{(i + 2)} defined as the sum of MVCost and REFCost of REF (i + 2) is calculated. (S12) The reference frame appropriate value H _(i) is the motion vector cost function value and It is defined as the sum of the reference frame cost function values. The smaller the value, the better the performance.

Specifically, the reference frame appropriate value, H _(i) can be defined by the following equation (1). MVCost _{REF _ (ⅰ)} and MVCost _{REF _ (ⅰ)} used in Equation 1 are employed as defined in H.264 / AVC.

Here, the motion vector cost function MVCost _{REF _ (ⅰ)} represents the MVCost value of the best matching block of the current block with respect to the _{reference frame} ⅰ. Similarly, the reference frame cost function REFCost _{REF_ (ⅰ)} represents the REFCost value of the best matching block of the current block for reference frame 프레임.

The motion vector cost function (MVcost) is defined by Equation 2 below.

Where f is a lambda factor. cx and cy represent the motion vector of the current macroblock, s represents the transition amount of the motion vector, and px and py represent the predicted motion vector.

The WeightedCost function is a function that takes a bit amount used for encoding a motion vector prediction error as an argument and returns a cost value by using an f value.

In addition, the reference frame cost function (REFcost) is defined by the following equation (3).

Where f is a lambda factor. ref refers to the selected reference frame. WeightedCost is a function that takes a bit amount required to encode a selected reference frame value using an f value and returns a cost value. In general, the reference frame cost function (REFcost) has a much smaller value than the motion vector cost function (MVcost).

In the present embodiment, the reference frame proper value, H _(i) is defined as described above, but within the scope not departing from the spirit of the present invention, the current block and the reference frame of the current block used in the full search algorithm (FSA) A sum of absolute difference (SAD) or similar values for all pixels constituting the search block may be set to a reference frame appropriate value, H _(i) .

2, the calculated H _(i) and H _{(i + 1)} are compared. (S13) Here, the smaller the value, the better the performance. If H _(i) is less than H _{(i + 1)} , frame N-1 is a more appropriate reference frame than frame N-2. In this case, 'an appropriate reference frame' means that an appropriate reference frame value, H _(i), is relatively small. Refers to a reference frame that can maintain more stable image quality.

If so, compare H _(i) and H _{(i + 2)} once more to see if frame N-1 is an appropriate reference frame. (S14) If H _(i) is greater than H _{(i + 2)} If less, reference frame N-1 is more appropriate than reference frame N-3. Accordingly, the reference frame N-1 is finally selected as the optimal reference frame through the two checking steps S13 and S14. (S17) Here, the 'optimal reference frame' is a relative procedure through a series of reference frame comparison procedures. This means a reference frame determined to be able to maintain an optimally stable image quality.

If H _(i) is greater than H _{(i + 2)} in step S14, reference frame N-3 is more appropriate than reference frame N-1. Therefore, in order to further confirm whether reference frame N-3 is an appropriate reference frame, motion estimation is performed on the next reference frame N-4, and then a reference frame appropriate value, H _{(i + 3)} is calculated. Then, compare H _{(i + 2)} and H _{(i + 3)} once more. (S16) If H _{(i + 2)} is less than H _{(i + 3)} , reference frame N-3 More appropriate than reference frame N-4. Accordingly, the reference frame N-3 is selected as the optimal reference frame through the two checking steps (S17). However, approximately 80% of the optimal motion vectors determined by the global search method will be determined as the reference frame N-1. .

2, if H _(i) is greater than H _{(i + 1)} in step S13, reference frame N-2 is more appropriate than reference frame N-1. Accordingly, to further confirm whether reference frame N-2 is an appropriate reference frame, compare H _{(i + 1)} and H _{(i + 2)} . (S15) If H _{(i + 1)} is H _{(i + 2) If} less, reference frame N-2 is more appropriate than reference frame N-3. Again, reference frame N-2 may be selected as the final optimal reference frame (S17). Otherwise, the search is continued in the same manner as described above for the next reference frame N-4.

In the series of comparison procedures, when the reference frame REF (i), which is determined to have a low reference frame proper value at least twice, is selected as the optimal reference frame, the further motion vector search process is stopped and the process ends.

As described above, in the present embodiment, the reference frame REF (i) that has passed two confirmation steps of the appropriate reference frame is selected as the final optimal reference frame, but it is required to shorten the motion estimation time and to stabilize the image quality. Depending on the trade-off, the number of verification steps can be adjusted from two to three or four times.

If the number of checking steps is three, in step S11, motion estimation may be performed on the reference frame REF (i, i + 1, i + 2, i + 3), and the procedure may be performed as described above. .

 Next, experimental examples of the present invention and the effects thereof will be described. This experiment was performed under the conditions shown in Table 1 below.

Referring to Table 1, in order to confirm the performance of the fast multiple reference frame selection algorithm according to an embodiment of the present invention, the present invention indicates the different levels of motion activity of the CIF (352 × 288) size, Five video sequences were compared to the reference software JM10.2. This includes 100 frames of bus, foreman, hall_monitor, stefan, and waterfall. The search range was set to 16 and performed on a Pentium 4 3.2GHz PC with 2G RAM. PSNR, motion prediction (ME) time, and bit rate were compared with JM (Ref_1) and JM (Ref_5), respectively. Quantization parameters were set to 32 and 36, respectively. Here, Ref_1 means that the reference software uses only one reference frame, and Ref_5 indicates that five reference frames are used. The method according to the embodiment of the present invention is compared with respect to Ref_1 and Ref_5, respectively.

Tables 2 and 3 below show a comparison of the luminance peak signal to noise ratio (PSNR) for each algorithm (QP = 32 and QP = 36).

Referring to Tables 2 and 3 above, since JM (Ref_5) provides more accuracy in the video sequence, an average of 0.182 dB (QP = 32) and 0.142 dB (QP = 36) compared to JM (Ref_1). It can be seen that the coding gain is achieved. However, JM (Ref_5) needs to use motion estimation in all reference frames. Compared to JM (Ref_5), the present invention showed no significant and insignificant PSNR reduction. In all experimental sequences, reductions of 0.04 dB and 0.028 dB were found, on average.

Tables 4 and 5 below show a comparison of the mean motion estimation time (ME time) for each algorithm (QP = 32 and QP = 36).

Referring to Tables 4 and 5 above, on average, the present invention is approximately 1.4 times faster than JM (Ref_5). For high motion sequences such as bus, foreman, stefan, and waterfall, the present invention used one or more reference frames in motion estimation. Therefore, the coding consumption of the present invention for a highly motion sequence was higher than that of a flat motion sequence such as hall_monitor. However, this increase can be ignored.

Tables 6 and 7 below show the results of all coded bits. (QP = 32 and QP = 36)

The average of all bits increased by only 0.425% (QP = 32) and 0.491% (QP = 36). This demonstrates that the present invention can retain overall bits similar to subjective quality similar to JM (Ref_5).

3 is a graph showing a motion estimation time (ME time) of the present invention compared to JM (Ref_1) and JM (Ref_5).

Referring to FIG. 3, it shows a reduction of approximately 40% in motion estimation time compared to JM (Ref_5) using five reference frames.

The invention being thus described, it will be obvious that the same way may be varied in many ways. Such modifications are intended to be within the spirit and scope of the invention as defined by the appended claims.

1 is a conceptual diagram illustrating motion estimation using five reference frames as an example of a plurality of reference frames.

2 is a flowchart illustrating a motion estimation method by a fast multiple reference frame selection procedure according to an embodiment of the present invention.

3 is a graph showing a motion estimation time (ME time) of the present invention compared to JM (Ref_1) and JM (Ref_5).

Claims (10)

In the motion estimation method using a plurality of reference frames, Selecting an optimal reference frame among the plurality of candidate reference frames using a reference frame appropriate value that is a sum of a motion vector cost function value and a reference frame cost function value for each of the plurality of reference frames; and Obtaining a motion vector for the current block from the optimal reference frame. The method of claim 1, wherein the optimal reference frame is selected using an appropriate value of the reference frame for some or all of the plurality of reference frames. The method of claim 1, wherein selecting the optimal reference frame Comparing the appropriate values of the reference frame with respect to the first reference frame, which is the previous frame of the current frame, and the second reference frame, which is the previous frame of the first reference frame, among the plurality of reference frames, Comparing a reference frame having a smaller reference frame appropriate value among the first and second reference frames with a reference frame suitable value of a third reference frame that is a previous frame of the second reference frame among the plurality of reference frames, And when the smaller reference frame is smaller than the third reference frame, selecting the smaller reference frame as the optimal reference frame. The reference frame suitability of the fourth reference frame according to claim 3, wherein when the third reference frame is smaller than the smaller reference frame, the reference frame proper value of the third reference frame and the previous reference frame of the fourth reference frame are appropriate. Compare the values, And when the third reference frame is smaller than the fourth reference frame, selecting the third reference frame as the optimal reference frame. The reference frame suitability of the fifth reference frame according to claim 4, wherein when the fourth reference frame is smaller than the third reference frame, the reference frame proper value of the fourth reference frame and the previous frame of the fourth reference frame are appropriate. Compare the values, And when the fourth reference frame is smaller than the fifth reference frame, selecting the fourth reference frame as the optimal reference frame. The method of claim 5, wherein the plurality of reference frames is five, And when the fifth reference frame is smaller than the fourth reference frame, selecting the fifth reference frame as the optimal reference frame. Compares an appropriate value of a reference frame with respect to a first reference frame that is a previous frame of the current frame and a second reference frame that is a previous frame of the first reference frame, Among the first and second reference frames, an optimal reference frame is selected after comparing a reference frame having a smaller reference frame proper value with a reference frame proper value of a third reference frame that is a previous frame of the second reference frame. The procedure for comparing the reference frame proper values until a reference frame determined to be at least twice the appropriate value of the reference frame is determined. Continually selecting at least twice a reference frame determined to have a suitable reference frame value as an optimal reference frame. The method of claim 7, wherein after comparing the reference frame proper values of the second reference frame and the third reference frame, If the second reference frame is small, select the second reference frame as the optimal reference frame, If the third reference frame is small, the optimal reference frame is selected after comparing an appropriate value of the reference frame of the fourth reference frame which is the previous frame of the third reference frame with the third reference frame. Motion estimation method by reference frame selection procedure. The method of claim 8, wherein after comparing the reference frame proper values of the third reference frame and the fourth reference frame, If the third reference frame is small, select the third reference frame as the optimal reference frame, If the fourth reference frame is small, after comparing the appropriate value of the reference frame of the fourth reference frame and the fifth reference frame that is the previous frame of the fourth reference frame, If the fourth reference frame is small, select the fourth reference frame as the optimal reference frame, And if the fifth reference frame is small, selecting the fifth reference frame as an optimal reference frame. 10. The motion estimation method according to any one of claims 7 to 9, wherein the reference frame appropriate value applies the following equation.

In the above equation, MVCost _{REF_ (ⅰ)} is It represents the motion vector cost function value of the best-matching block of the current block for the reference frame ⅰ and is defined by the following equation.

Where f is a lambda factor. cx and cy represent the motion vector of the current macroblock, s represents the transition amount of the motion vector, and px and py represent the predicted motion vector. The WeightedCost function is a function that takes a bit amount used to encode a motion vector prediction error using an f value and returns a cost value. In addition, REFCost _{REF_ (ⅰ)} represents a reference frame cost function value of the best matching block of the current block with respect to the reference frame _{으로 and} is defined by the following equation.

Here, f is a lambda factor, ref is a reference frame selected, and a WeightedCost function is a function that takes a bit amount required to encode a selected reference frame value using an f value as an argument and returns a cost value.

Images