KR100924642B1 - Motion estimation procedure using fast block matching algorithm - Google Patents

Abstract

A motion estimation procedure using a fast partial distortion cancellation algorithm based on the Hadamard transform and its probability model is described. The motion estimation procedure according to an embodiment of the present invention divides a search block for a predetermined candidate position in a search region for motion estimation into a plurality of local blocks, and applies each of the plurality of local blocks using a Hadamard transform. Calculate a local block complexity, calculate a prediction match error for the predetermined candidate location using the local block complexity, more specifically, the local block complexity cumulative distribution function (CDF), and find all candidates in the search region. The matching block is selected by finding a matching error only for some candidate positions having the smallest prediction matching error among the positions. According to an embodiment of the present invention, the complexity of a local block can be obtained by using the Hadamard transform, and a matching error can be obtained without calculating a matching error for a predetermined candidate position by using the cumulative distribution function of the local block complexity. You can predict accurately.

Description

Motion estimation procedure using fast block matching algorithm

The present invention relates to an image codec, and more particularly, to a motion estimation (ME) procedure using a block matching algorithm (BMA) in an image codec.

Video compression techniques such as MPEG-1, 2, 4 or H.264 / AVC can be used to significantly reduce the Motion Estimation and Compensation (ME / MC) procedure, Discrete Cosine Transform (DCT). It has a basic structure consisting of the transform, quantization, and entropy coding procedures. Among them, motion estimation and compensation is a technique of compressing data using characteristics of a video, that is, high spatial and temporal redundancy among components constituting an image. to be. This is because the redundancy of the temporal data having the most redundancy in the video can be eliminated by motion estimation and motion compensation.

The motion estimation method can be roughly divided into a pixel recursive algorithm (PRA) and a block matching algorithm (BMA). Among these, most video codec technologies use block matching algorithms in consideration of the regularity of data flow, computational complexity, and hardware implementation. The block matching algorithm divides a screen into 16 × 16 macroblocks (MB) or any smaller searching block (Searching Blocks) of smaller size and matches them in a defined search window of a reference frame in units of blocks. Refers to the procedure for finding a block. The value indicating the position of the matching block selected according to the block matching algorithm is a motion vector. In video encoding, only the difference value between the matching block and the current block and the motion vector are encoded to remove data redundancy.

The block matching algorithm generally uses a full search algorithm (FSA). The prescan algorithm is a matching error (eg, the sum of the absolute values of differences for all pixels constituting the search block of the current block and the reference frame) using all coordinates in the search area as candidate positions (CP). After calculating the Sum of Absolute Difference (SAD), the candidate position (CP) having the minimum matching error is selected as the matching block. Most video coding standards use this prescan algorithm, which is a way to find the best matching block. However, the pre-search algorithm has a large amount of calculation and a complicated system configuration for implementing it, which makes it difficult to apply to an application for real-time encoding.

Therefore, a number of new algorithms have been proposed to solve the problems caused by these prescan algorithms for real-time applications. One of the new algorithms is a fast block matching algorithm, and it is determined whether the matching block has a predetermined pattern instead of all the coordinates in the search area and only some coordinates are candidate positions CP. Fast block matching algorithms include, for example, Three Step Search (TSS), New Three Step Search (NTSS), Four Step Search (FSS), Block-based Gradient Search (Block-Based GraDient Search, BBGDS), and Diamond Search (DS).

In the above-described conventional block matching algorithm (pre-search algorithm or fast block matching algorithm), a matching error is obtained for each candidate position CP in the search area, and then a candidate position where the matching error is minimum is selected as the matching block. . However, actually obtaining the matching error, for example, SAD, involves calculating the error for all the pixels in the macroblock or the block of the predetermined size, so that the computational amount is large and the processing time is not too small. In particular, in the pre-search algorithm, the matching error must be calculated for all candidate positions (CP) in the search area, so it is not easy to apply in a real-time application due to the increase of the calculation amount.

Therefore, the problem to be solved by the present invention is to improve the processing speed by minimizing the amount of calculation, and in particular, the motion using the block matching algorithm that can be applied to real-time applications by improving the processing speed when applying the pre-scan algorithm It is to provide an estimation method.

Another problem to be solved by the present invention is that a part of the search area, particularly a part in which partial distotion exists, is not included in the candidate position CP without calculating the matching error. It is to provide a motion estimation method using a block matching algorithm that can be applied to a real-time application to improve the speed.

The motion estimation method according to an embodiment of the present invention for solving the above problems is to divide a macro block for a predetermined candidate position in a search region for motion estimation into a plurality of local blocks, and using the Hadamard transform Calculating a local block complexity for each of a plurality of local blocks; calculating a sum of partial matching errors in order from a local block having a large local block complexity; and using the sum of the partial matching errors, the predetermined candidate position. Determining whether can be a matching block. In this case, when the sum of the partial matching errors is equal to or greater than a predetermined threshold, it may be determined that the predetermined candidate position cannot be a matching block.

According to another aspect of the present invention, there is provided a motion estimation method by dividing a macroblock for a predetermined candidate position in a search region for motion estimation into a plurality of local blocks and using a Hadamard transform. Calculating a local block complexity for each of the plurality of local blocks, obtaining a prediction match error for the predetermined candidate location using the local block complexity, and predicting matching among all candidate locations in the search region Selecting a matching block by finding a matching error for only some candidate positions having a small error.

According to an aspect of the embodiment, the complexity of the local block can be obtained using Equation (1). Here, LBC (k) represents a local block complexity (LBC) of the k-th local block included in the macroblock. SumAC (k) is the sum of the absolute values of Alternate Component (AC) in the Hadamard transform coefficients of the k-th local block, and DC _Local (k) is the Hadamard transform coefficients of the k-th local block. The conversion coefficient of the direct component (DC). DC _MB is a DC value of the macroblock and represents an average value of transform coefficients of a DC component for each of the local blocks included in the macroblock.

(1)

(One)

The calculating of the prediction matching error may include calculating a partial matching error by sequentially adding the matching errors in the order of the local block having a high local block complexity to a low local block, and accumulating a cumulative distribution function of the local block complexity and the partial. And calculating the predictive matching error using the similarity and correlation between the cumulative distribution function (CDF) of the matching error. In addition, the partial matching error can be obtained using Equation (2), and the prediction matching error can be obtained using Equation (3). Where Partial SAD (k) is the sum of matching errors up to the k-th local block in descending order of local block complexity, f _cur and f _ref are the current frame and the reference frame, i is the number of local blocks LB, x , y is the candidate motion vector, j is the candidate position in the i th local block, and Predictive SAD (k) is the predictive matching error predicted from the sum of the matching errors up to the k th local block in descending order of local block complexity. Where CDF _LBC (k) is the cumulative distribution function of the local block complexity, CDF _pmSAD (k) is the cumulative distribution function of the previous minimum SAD, SAD _LB (k) is the SAD of the k-th local block, and ω is the weighting factor. ).

(2)

(2)

(3)

(3)

According to an embodiment of the present invention using a fast partial distortion elimination algorithm using a Hadamard transform and its probability model, the complexity of each local block is obtained using the Hadamard transform, and the CDF of the local block complexity and the previous minimum match error For example, SAD can be used to accurately predict matching errors. According to this embodiment of the present invention, it is possible to reduce the computational complexity and improve the processing speed without degrading the rate-distortion performance.

In the first embodiment of the present invention, like the pre-search algorithm, instead of calculating the matching error for all search positions in the search area and then selecting a matching block, the partial matching error is calculated for each search position and the calculated partial match is performed. If the error is larger than the predetermined threshold, additional matching errors for the remaining local blocks are not calculated, thereby reducing the amount of computation and improving processing speed. To this end, in an embodiment of the present invention, the search block is divided into a plurality of local blocks, and partial matching errors are sequentially calculated from local blocks having a large matching error in each local block. Whether a local block has a large matching error is determined using the local block complexity obtained by applying a Hadamard transform to each local block. Hereinafter, with reference to the accompanying drawings will be described in detail a first embodiment of the present invention.

1 is a flowchart showing a motion estimation procedure according to a first embodiment of the present invention. Referring to FIG. 1, first, a search block for a predetermined candidate position (CP) in a search region for motion estimation is divided into a plurality of local blocks (LBs), and then, for the local blocks. By applying the Hadamard transform, Local Block Complexities (LBCs) for each of the plurality of local blocks are calculated (S11).

According to an embodiment of the present invention, the method of dividing the search block into a plurality of local blocks having the same size is not particularly limited. For example, as shown in FIG. 2, when the search block is a macroblock 16 × 16, the search block may be divided into 16 local blocks, each local block having a size of 4 × 4. Or (see FIG. 2 (a)) or 1 × 16 (see FIG. 2 (b)). However, embodiments of the present invention are not limited thereto.

After dividing the search block into local blocks, a local block complexity for each local block is calculated, and an ordered Hadamard transform can be used to calculate the local block complexity. Since the order Hadamard transformation consists only of shift, addition, and subtraction, it is advantageous in that the amount of computation is small and the computation speed is fast. The matrix of Hadamard transform used to calculate local block complexity may vary depending on the size of the local block. Equation 1 shows a Hadamard transform equation that can be applied in the example of FIG. In Equation 1, X and H represent Hadamard transform coefficients (TCs) of the local block and X in the current macroblock (search block), respectively. In the example of FIG. 2B, the 1D Hadamard transform equation is applied.

The Hadamard transform coefficients of each local block are used to determine a Search Order. Equation 2 calculates a local block complexity (LBC) of the k-th local block. In Equation 2, SumAC (k) is the sum of absolute values of Alternate Component (AC) in the Hadamard transform coefficients of the k-th local block, and DC _Local (k) is the Hadamard of the k-th local block. The conversion coefficient of the direct component (DC) in the conversion coefficients. And, DC _MB represents the DC value of the macroblock, that is, the average value of the conversion coefficients of the DC component for each local block included in the macroblock. Referring to Equation 2, the k th local block complexity LBC (k) is the sum of the absolute value of the alternating current component AC and the difference between the transform coefficient of the direct current component of the k th local block and the DC value of the current macroblock. Can be represented as a sum.

Referring to FIG. 1, the sum of partial matching errors is sequentially calculated from a local block having a large local block complexity (LBC), and a matching block may be formed using the sum of the partial matching errors. It is determined (S12). The partial match error may be, for example, a sum of absolute difference (SAD) of the difference for the local block rather than the search block, but embodiments of the present invention are not limited thereto. In the embodiment of the present invention, the LBC (k) obtained in Equation 2 is obtained from the largest local block from the largest local block, and the k-th partial matching error is sequentially determined to determine whether the corresponding search block can be a matching block.

In general, larger local block complexity means that matching errors are greater. Therefore, according to an embodiment of the present invention, when a matching error is sequentially obtained from a block having a high local block complexity and then summed up to obtain a partial matching error, a search block having a larger matching error compared to the matching block may have a small k. The sum of the partial match errors may exceed a predetermined threshold. Here, the predetermined threshold is a reference value for determining the possibility of becoming a matching block, for example, the minimum matching error obtained before calculation for the current candidate position or the minimum (or average) matching error in the previous macroblock, or any It may be a predetermined matching error value selected. According to the embodiment of the present invention, when the partial matching error is equal to or greater than a predetermined threshold, it is possible to determine that the macroblock is not a matching block without having to find the entire matching error. Therefore, in the embodiment of the present invention, since the calculation of the SAD for the candidate position can be performed quickly (more precisely, it is not necessary to obtain the SAD for the entire macroblock for candidate positions having a large matching error). Rapid motion estimation is possible.

The second embodiment of the present invention applies the Hadamard Probability Model, in that it does not select a matching block after calculating a matching error for all the search positions in the search area as in the pre-search algorithm. Same as the first embodiment described above. In this embodiment, however, the calculation error is reduced by calculating the matching error only for a part of the search positions that are likely to be the matching blocks, thereby improving the processing speed. In order to select a search position that is likely to be a matching block among all search positions in the search region, in an embodiment of the present invention, a search position that is less likely to be a matching block, that is, a portion having a low correlation with the current block in the search region. The conventional distortion is excluded from the candidate position (Partial Distortion Elimination, PDE), and the matching error is calculated only for the remaining search positions that are highly correlated with the current block. In an embodiment of the present invention, it is determined whether each search position in the search region has a high or low correlation with the current block by using a prediction match error for each search position, and the prediction match error corresponds to the search block corresponding to each search position. Is divided into a number of local blocks and then obtained using local block complexity and partial matching error obtained by applying a Hadamard transform to each local block. Hereinafter, with reference to the accompanying drawings, a second embodiment of the present invention will be described in detail.

3 is a flowchart illustrating a motion estimation procedure according to a second embodiment of the present invention. Referring to FIG. 3, like the first embodiment described above, first, a search block for a predetermined candidate position CP in a search region for motion estimation is divided into a plurality of local blocks LB, and then the local block. Local block complexity (LBC) for each of the plurality of local blocks is calculated by applying the Hadamard transform to the plurality of local blocks (S21). Since this step is substantially the same as step S11 of the above-described first embodiment, a detailed description thereof will be described.

The prediction matching error for the corresponding candidate position is calculated using the local block complexity obtained in step S21 (S22). More specifically, the minimum matching with the cumulative distribution function of the local block complexity (CDF _LBC ) in descending order of complexity in the overall local block complexity ( LBC ), that is, from the largest local block to the smallest local block. The prediction match error is calculated using the similarity and correlation between the errors, eg, the cumulative distribution function (CDF _mSAD ) of the actual minimum SAD. This method of calculating the prediction matching error using the Hadamard transform is called the Hadamard Prediction Model (HPM). The purpose of HPM is to predict the total SAD quickly and accurately so that the matching block can be generated without actually calculating the total SAD. It is to remove the candidate position (Impossible CP) that can not be.

In order to find the prediction matching error, Equation 3 first defines the sum of the SADs up to the k th local block, that is, the partial SAD (k) in descending order of the local block complexity. In Equation 3, f _cur and f _ref denote the current frame and the reference frame, i denotes the number of local blocks LB, x and y denote Candidate Motion Vectors (MV), and j denotes the i th local. Indicates a candidate position (CP) in the block.

Ideally, assuming, for example, that the CDF _mSAD is known, the predictive SAD (k) for the k-th local block at the candidate position is given by Equation 4.

However, since CDF _mSAD is not known in reality, it is estimated using another value. For example, in an embodiment of the present invention, CDF _mSAD is estimated using a CDF _LBC and a cumulative distribution function (CDF of previous minimum SAD). In Equation 5, an example of obtaining the CDF _mSAD estimated by the above method is defined. In Equation 5, SAD _LB (k) represents the SAD of the k-th local block, and ω is the weighting factor. And SAD _LB (k) is the SAD of the LB for Safe Margin.

If the current candidate position is the best position to be the matching block, then the predicted SAD should not be greater than the minimum SAD. In practice, however, due to inaccurate prediction and / or _{dissimilarity} between CDF _LBC and CDF _mSAD , the predicted SAD may be greater than the minimum SAD. This causes the bit-rate to increase. On the other hand, if the current candidate position is not the best position, the prediction SAD should not be smaller than the minimum SAD. In practice, however, the predicted SAD may be less than the minimum SAD, which causes performance degradation. Therefore, weighting factors are necessary for accurate prediction.

Referring to FIG. 3, a matching error is obtained by selecting a matching block only for some candidate positions having the smallest prediction error among all candidate positions in the search region (S33). For example, according to an aspect of an embodiment of the present invention, after determining the candidate position with the smallest prediction matching error as the matching block or after calculating the actual SAD for some candidate positions with the smallest prediction matching error, SAD may determine that the smallest candidate position is a matching block.

Experimental Results

In order to evaluate the performance of the motion estimation procedure according to an embodiment of the present invention, MPEG-4 optimization models (ISO / IEC JTC1 / SC29 / WG11 N3675, MPEG-4 MoMuSys-OM-2.0-010211 software Optimization Model ver. 2.0, Oct. 2000) various experiments. The experiment used Cki-sized Akiyo (300 frames), Bus (150 frames), Football (90 frames), and Foreman (300 frames) sequences. The IPPP structure and 32 search ranges were used, and the weighting factor (ω) was set to 0.95. In Table 1 and Table 2 and Fig. 4, P1 is the case according to the first embodiment of the present invention (that is, when only the Hadamard transformation is used), and P2 is the case according to the second embodiment of the present invention (Hadamard transformation). And Hadamard transformation model).

Table 1 shows the average power signal-to-noise ratio (PSNR) and bit-rate for all test images. Referring to Table 1, it can be seen that the loss of average PSNR and bit rate is negligible. 4 shows a result of comparing the average rate-distortion rate. Referring to FIG. 4, an embodiment of the present invention is almost identical to the case of the pre-scanning algorithm (FSA) in terms of average rate-distortion performance. It can be seen that.

Table 2 shows widely known methods for fast partial distortion cancellation algorithms (JN Kim, SC Byun, and BH Ahn, "Fast full search motion estimation algorithm using early detection of impossible candidate vectors," IEEE transaction on Signal Processing, Vol. 50, No. 9, pp. 2355-2365, Sept. 2002, Table 1, Table 2, and 4, the complexity required in the embodiment of the present invention is indicated as contrasted with "Kim's". In Table 2, 'Ratio' of 'clock' represents an operation ratio including addition, subtraction, multiplication, division, comparison, and shift. And the ratio of FSA and PDE is based on the total number of local blocks. Table 2 shows the performance efficiency of the algorithm according to the embodiment of the present invention for each bit rate.

As described in detail above, an embodiment of the present invention provides a fast partial distortion elimination algorithm using a Hadamard transform and its probability model. The Hadamard transform can be used to analyze the complexity of each of the local blocks. The matching error can be accurately predicted using the local block complexity CDF and the previous minimum matching error, such as SAD. According to this embodiment of the present invention, it is possible to reduce the complexity of the calculation by an average of 92.4% compared with the conventional method using the prescan algorithm without degrading the rate-distortion performance.

1 is a flowchart showing a motion estimation procedure according to the first embodiment of the present invention.

2 is a diagram illustrating a method of dividing a macroblock, which is a search block, into local blocks, by way of example.

3 is a flowchart showing a motion estimation procedure according to a second embodiment of the present invention.

Figure 4 is a graph showing the average rate-distortion of the test image when the method according to an embodiment of the present invention and the conventional method.

Claims (7)

Dividing a macroblock for a predetermined candidate position in a search region for motion estimation into a plurality of local blocks, and calculating a local block complexity for each of the plurality of local blocks using a Hadamard transform; And Calculating a sum of partial matching errors in order from a local block having a large local block complexity, and determining whether the predetermined candidate position can be a matching block using the sum of the partial matching errors. Way. The motion estimation method of claim 1, wherein the complexity of the local block is obtained by using Equation (E-1).

(E-1) Here, LBC (k) represents a local block complexity (LBC) of the k-th local block included in the macroblock. SumAC (k) is the sum of the absolute values of Alternate Component (AC) in the Hadamard transform coefficients of the k-th local block, and DC _Local (k) is the Hadamard transform coefficients of the k-th local block. The conversion coefficient of the direct component (DC). DC _MB is a DC value of the macroblock and represents an average value of transform coefficients of a DC component for each of the local blocks included in the macroblock. 2. The method of claim 1, wherein if the sum of the partial matching errors is greater than or equal to a predetermined threshold, the candidate position is determined to be a matching block. Dividing a macroblock for a predetermined candidate position in a search region for motion estimation into a plurality of local blocks, and calculating a local block complexity for each of the plurality of local blocks using a Hadamard transform; Obtaining a prediction match error for the predetermined candidate position using the local block complexity; And And selecting a matching block by obtaining a matching error only for some candidate positions having the smallest prediction matching error among all candidate positions in the search region. The method of claim 4, wherein the complexity of the local block is obtained by using Equation (E-2).

(E-2) Here, LBC (k) represents a local block complexity (LBC) of the k-th local block included in the macroblock. SumAC (k) is the sum of the absolute values of Alternate Component (AC) in the Hadamard transform coefficients of the k-th local block, and DC _Local (k) is the Hadamard transform coefficients of the k-th local block. The conversion coefficient of the direct component (DC). DC _MB is a DC value of the macroblock and represents an average value of transform coefficients of a DC component for each of the local blocks included in the macroblock. 6. The method of claim 5, wherein obtaining the prediction match error Calculating a partial match error by sequentially summing matching errors in the order of the local block having a high local block complexity and a low local block; And And estimating the predictive matching error using a similarity and correlation between the cumulative distribution function of the local block complexity and the cumulative distribution function (CDF) of the partial matching error. The method of claim 6, wherein the partial matching error is obtained by using Equation (E-3), and the prediction matching error is obtained by using Equation (E-4).

(E-3)

(E-4) Where Partial SAD (k) is the sum of matching errors up to the k-th local block in descending order of local block complexity, f _cur and f _ref are the current frame and the reference frame, i is the number of local blocks LB, x , y is a candidate motion vector, j is a candidate position in the i th local block, Predictive SAD (k) is the predictive matching error predicted from the sum of the matching errors up to the kth local block in descending order of local block complexity, CDF _LBC (k) is the cumulative distribution function of local block complexity, and CDF _pmSAD (k) is The cumulative distribution function of the minimum SAD, SAD _LB (k) denotes the SAD of the k-th local block, and ω denotes the weighting factor.

Images