CN1319384C - Optimizing distribution method in moving evaluation for hardware computing power resource - Google Patents

Abstract

The present invention discloses an optimizing allocation method of hardware computing power resources in moving evaluation. H. 264 is a new video coding standard which absorbs the advantages of existing video coding standards, simultaneously provides many new algorithms and greatly enhances coding efficiency and image quality; however, the H. 264 enhances coding efficiency by increasing computation complexity and needs to be supported by different hardware; therefore, for aiming at different hardware and different computing capacity, the present invention carries out optimization aiming at the motion estimation aspect in the H. 264 for reaching good allocation of hardware resources, and further, optimized peak value signal-to-noise ratio and optimized code rate are obtained; simultaneously, the present invention has certain extendibility and can realize the isostatic adjustment between arithmetic complexity and prediction precision.

Description

Optimal Allocation Method of Hardware Computing Power Resources in Motion Estimation

technical field

The invention relates to the field of multimedia technology supported by different hardware conditions, in particular to a method for optimally allocating hardware computing power resources in motion estimation.

Background technique

H.264 is the latest video coding standard formulated by JVT, a joint expert group of ITU-T and MPEG. This coding standard can obtain high coding efficiency, especially in terms of low bit rate, which is significantly improved compared with MPEG-4. , very suitable for low-bandwidth, high-quality network video applications. However, in order to improve the encoding efficiency, H.264 adopts many algorithms with high computational complexity, which makes the encoding and decoding calculations very heavy, so the requirements for software and hardware are very high, and it also increases the difficulty of encoding and decoding. Because under normal circumstances, many methods cannot achieve the optimal encoding quality under the condition of limited hardware support.

In order to make H.264 easier to implement in low-bit-rate high-time application systems, its coding algorithm must be optimized. An analysis of each algorithm module of the H.264 encoder shows that the calculation amount of the motion estimation module accounts for more than 80% of the calculation amount of the entire encoder. Therefore, to optimize the entire encoder, the motion estimation module should be the first choice.

When H.264 performs motion estimation, motion vector prediction is the first choice. After the initial motion vector is obtained through prediction, the initial motion vector is used as the search center for block matching search. A range of , all the points in the rectangular area within the range must calculate a matching result, and select the best matching point as the result of the integer pixel search. The full search algorithm of H.264 has the advantage that it can find the global optimal matching result within a limited range, and the motion estimation accuracy is very high. Its shortcoming is that the complexity of the algorithm is too high, which becomes the most time-consuming part of the entire coding system. Therefore, the key to optimizing H.264 is to improve the search speed of motion estimation with as little quality loss as possible.

At present, due to the different characteristics of various hardware devices, their computing capabilities and working capabilities are also different. For example, with the popularization of portable mobile devices, a large number of multimedia applications based on portable mobile devices have emerged. Due to the weak computing power of portable mobile devices, and the relatively high computational complexity of video decoding in multimedia, the traditional video encoding method used in portable mobile devices cannot meet the needs of users for multimedia, especially video technology-related applications. . Therefore, according to the different computing capabilities of different hardware devices, it is necessary to optimize the motion estimation, so as to achieve the purpose of better hardware resource allocation, and then obtain the optimized PSNR and code rate.

Contents of the invention

The purpose of the present invention is to provide a method for optimizing the allocation of hardware computing power resources in motion estimation. In view of the situation that existing hardware has different computing power, it needs to be optimized when doing motion estimation in H.264, so as to achieve a reasonable allocation of limited hardware resources. Allocation, and finally take the best effect.

The technical scheme that the present invention adopts is as follows:

(1) Prioritize based on the absolute error sum of the macroblocks, divide the frame into 16×16 macroblocks according to the size of the frame, and then perform a difference between a macroblock of the current frame and the corresponding macroblock of the previous frame, The absolute error sum is obtained, and the macroblocks are prioritized according to the calculated absolute error sum, and when the difference between the absolute error sums of two macroblocks is within the range of 100, the priority of the two macroblocks is considered At the same time, it is stipulated that the priority of the absolute error sum and the large macroblock is higher, and vice versa. The calculation formula of the absolute error sum is as follows:

$\mathrm{<span\; class="notranslate">\; SAD</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 15</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 15</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>$

in

SAD is the sum of absolute errors,

f is the pixel function of the current frame,

g is the pixel function of the previous frame;

(2) According to the size of the priority, the left side A, the top B, and the top right C of the macroblock E with the highest priority (if there is a macroblock with the same highest priority, select the macroblock in the middle as the target macroblock) The adjacent macros of the macroblocks are motion estimated, and then the obtained motion vectors MVA, MV _B , MV _C and MV _median of the adjacent macroblocks A, _B , C are used as the candidate prediction points of the macroblock E, and finally the current macroblock E performs motion estimation, calculates the error between the current macroblock and its matching reference macroblocks, selects the macroblock with the smallest error as the best reference macroblock, and finally obtains the motion vector of the current macroblock E;

(3) If there are currently macroblocks with the same highest priority, select according to the number of known MV numbers of their respective adjacent macroblocks (left A, upper B, upper right C), if a certain macroblock E is adjacent The greater the number of known motion vectors of the macroblock, the macroblock E is used as the target macroblock;

(4) Make whether to stop the motion estimation of the macroblock of this frame according to the current situation of the current computing power, if it is judged to stop, then consider the motion vector of the remaining macroblocks without motion estimation to be equal to 0, and prepare for the next Motion estimation continues for one frame.

In step (2), when selecting the highest priority macroblock for the first time, if there is currently a macroblock with the same highest priority, then the macroblock in the middle is used as the target macroblock E;

In step (2), for the current macroblock E, it is necessary to obtain the motion vectors of its left A, upper B, and upper right C adjacent macroblocks. If the upper right C is not in the current frame, then select the upper left macroblock C' to replace the upper right macroblock C;

In step (2), when calculating the MV of the adjacent macroblocks on the left A, upper B, and upper right C of the current macroblock E, if its SAD<preset value, then use the horizontal and vertical search method, otherwise use the diamond search method Law;

When the current macroblock E is predicted in step (2), the candidate prediction points are (0, 0), MV _A , MV _B , MV _C , MV _median ;

When performing motion estimation on the current macroblock E in step (2), calculate the absolute error sum of the current macroblock and its matching reference macroblocks, select the macroblock with the smallest absolute error sum as the best reference macroblock, and finally obtain the current The motion vector of the macroblock E;

In step (3), if there are currently macroblocks with the same highest priority, they are selected according to the number of known motion vectors of their adjacent macroblocks (A on the left, B on the top, and C on the right). If a macroblock The more motion vectors are known for the neighboring macroblocks, the macroblock is taken as the target macroblock.

The beneficial effects of the present invention are: motion estimation can be performed on video signals in various formats, especially in the case of limited hardware computing power, which can better reflect the superiority of the invention, if the hardware computing power is strong , the present invention does not affect the final optimization result obtained after motion estimation, because motion estimation is performed for each macroblock in each frame at this time. On the contrary, if the hardware computing power is limited, the present invention fully demonstrates its inherent superiority, because the method proposed by the present invention makes those absolute errors and larger macroblocks in a frame firstly perform motion estimation, according to priority Motion estimation is performed on each macroblock from large to small. Of course, during the motion estimation, the present invention fully considers the limitation of hardware computing power, so various methods are used for processing, so as to achieve the optimal effect.

Description of drawings

FIG. 1 is a schematic diagram of the positional relationship between a current macroblock and adjacent macroblocks in the existing spatial domain;

Fig. 2 is a schematic diagram of the macroblock priority determination method of the present invention;

Fig. 3 is the schematic diagram of existing rhombus search method;

FIG. 4 is a schematic diagram of an existing horizontal and vertical search method;

FIG. 5 is a schematic diagram of a target macroblock selection method of the present invention;

FIG. 6 is a schematic diagram of a second method for selecting a target macroblock in the present invention;

FIG. 7 is a schematic diagram of a third method for selecting a target macroblock according to the present invention.

Detailed ways

The method proposed by the present invention for rationally allocating hardware computing power resources for motion estimation under the premise of limited hardware computing power is mainly applicable to use under the condition of hardware support with different computing power. Mainly proceed as follows:

1. Division of macroblock priority

The current frame is divided into macroblocks and divided into 16×16 macroblocks, and then the difference between one macroblock of the current frame and the corresponding macroblock of the previous frame is obtained to obtain the sum of absolute errors. Then the macroblocks are prioritized according to the size of the absolute error sum. When the difference between the absolute error sums of two macroblocks is within the range of 100, the priorities of the two macroblocks are considered to be equal. At the same time, it is stipulated that the priority of the absolute error and the large macroblock is higher, and vice versa. The positional relationship between the current macroblock and adjacent macroblocks is shown in FIG. 1 .

2. Select the target macroblock with the highest priority for the first time

When selecting the macroblock E with the highest priority, if there is no macroblock with the same priority, directly select the macroblock with the highest priority as the target macroblock; otherwise, if there is a macroblock with the same highest priority, select the middle one The macroblock serves as the target macroblock E. As shown in Figure 2, there are currently two macroblocks E1 and E2 with the same priority, and their absolute error sums are S1 and S2 respectively. If |S2-S1|<=S (S=100), then select the middle one Macroblock E1 serves as a target macroblock.

3. Use a different search method

For different absolute error sums of macroblocks, different search methods are used, because in this way redundant and insufficient searches can be avoided.

1) The diamond search method is used when the absolute error and the sum are large, so that the optimal point can be found quickly. There are three cases, Fig. 3a is a translational search in the x or y direction, Fig. 3b is an oblique translational search, and Fig. 3c is a search towards the center, as shown in Fig. 3 respectively.

2) The horizontal and vertical search method can reduce the search points and achieve better results. As shown in Figure 4: set a starting point, first find the point with the minimum absolute error sum in the horizontal direction, and then find the point with the minimum absolute error sum in the vertical direction of the horizontal optimal point, and the search result is the final optimal point. Therefore, the search process is as follows: the search center is point 1, compared with point 2 on the left side, the absolute error sum of point 1 is smaller, and then compared with point 3 on the right side of point 1, point 1 is obtained The absolute error sum is the smallest, so point No. 1 is the best point in the horizontal direction. Similarly, continue to search in the vertical direction of point No. 1, and the final result is that point No. 5 is the best point.

4. Cutoff (Half-Stop) criterion

When the sum of absolute errors (SAD) of the motion estimation points <T, the motion estimation process is terminated, where T is a threshold value, and the fixed value is 400 for a 16×16 macroblock in the experiment. For example, when it is detected that the sum of absolute errors (SAD) of a certain point is 300, the search process is stopped and it is confirmed as the best search matching point.

5. Select the target macroblock with the highest priority

If there are currently discrete macroblocks E _i , E _i+1 , ..., E _j with the same highest priority (denoted as set A, and 1≤i, j≤M×N, i≤j, where M×N is the total number of macroblocks in one frame, 6×6 in the figure), then according to the number of known motion vectors of their adjacent macroblocks (left A, upper B, upper right C, where i≤p≤j) How much to choose. There may be many situations in the judgment process, as follows:

1) Among the macroblocks with the same priority, in the first case, it is hoped to find a unique macroblock E _p , whose adjacent macroblocks have the most known motion vectors. At this time, the macroblock E _p can be directly used as Target macroblock; as shown in Figure 5: there are 3 MVs known in the adjacent macroblocks A, B, C of E _p , and the known MVs of the adjacent macroblocks of other macroblocks E except E _p No more than 3. So choose E _p as the target macroblock.

2) Among the macroblocks with the same priority, if the above conditions cannot be satisfied, that is, if there are multiple discretely occurring macroblocks _Em , _Em+1 , ..., E _n , and their adjacent macroblocks are known The number of motion vectors is the same. Then among these macroblocks E _m , _Em+1 ,..., E _n , if the adjacent macroblocks of a certain macroblock among them are known to have the largest number of MV≠0, then this macroblock is selected as the target macroblock, such as As shown in Figure 6: the adjacent macroblocks of E _q have 3 known MVs and none of them are equal to 0, while the known MVs of the adjacent macroblocks of other macroblocks E except E _q are all equal to 0 . So choose E _q as the target macroblock.

3) If neither of the above two methods can select the target macroblock, then select the relatively centered macroblock as the target macroblock. As shown in FIG. 7 : macroblocks E _t and E with the same current priority are compared, and the macroblock E _t in the middle is selected as the target macroblock. The reason for this is that the centered macroblock has a relatively high probability of appearing as the predicted value of other macroblocks.

The present invention has passed the test on the self-developed H.264 encoder, which adopts the following technologies in H.264:

1) Number of reference frames: 1;

2) 2) Frame type: I frame, P frame;

3) 3) Rate Distortion Optimization (RDO);

4) CAVAL entropy coding is adopted;

5) Integer pixel motion estimation range: 16;

6) Motion estimation mode: 16×16;

7) Integer pixel search.

The test result of the inventive method shows: under the support of E-Sitsang-PXA255 hardware, test with video series such as foreman, mobile, news, format is CIF (352 * 288), the result can reach about 35fps. At the same time, a relatively good signal-to-noise ratio and code rate are obtained.

Claims (7)

1. A method for optimal allocation of hardware computing power resources in motion estimation, characterized in that: (1) Prioritize based on the absolute error sum of the macroblocks, divide the frame into 16×16 macroblocks according to the size of the frame, and then perform a difference between a macroblock of the current frame and the corresponding macroblock of the previous frame, The absolute error sum is obtained, and the macroblocks are prioritized according to the calculated absolute error sum, and when the difference between the absolute error sums of two macroblocks is within the range of 100, the priority of the two macroblocks is considered At the same time, it is stipulated that the priority of the absolute error sum and the large macroblock is higher, and vice versa. The calculation formula of the absolute error sum is as follows: $\mathrm{<span\; class="notranslate">\; SAD</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 15</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 15</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>$ in SAD is the sum of absolute errors, f is the pixel function of the current frame, g is the pixel function of the previous frame; (2) Perform motion estimation on the adjacent macroblocks on the left A, upper B, and upper right C of the current highest priority macroblock E according to the size of the priority, and then obtain the adjacent macroblocks A, B, C The motion vectors MV _A , MV _B , MV _C and MV _median are used as the candidate prediction points of the macroblock E, and finally the motion estimation is performed on the current macroblock E, and the errors between the current macroblock and its matching reference macroblocks are calculated, and the selection error The smallest macroblock is the best reference macroblock, and finally the motion vector of the current macroblock E is obtained; (3) If there are currently macroblocks with the same highest priority, select according to the number of known MV numbers of their respective adjacent macroblocks on the left A, above B, and above right C. If a certain macroblock E is adjacent to the macroblock The greater the number of known motion vectors, the macroblock E is used as the target macroblock; (4) Make whether to stop the motion estimation of the macroblock of this frame according to the current situation of the current computing power, if it is judged to stop, then consider the motion vector of the remaining macroblocks without motion estimation to be equal to 0, and prepare for the next Motion estimation continues for one frame. 2. A method for optimally allocating hardware computing power resources in motion estimation according to claim 1, characterized in that: when selecting the highest priority macroblock for the first time in step (2), if there is currently the same highest priority macroblock For macroblocks with priority, the macroblock in the center is taken as the target macroblock E. 3. A method for optimally allocating hardware computing power resources in motion estimation according to claim 1, characterized in that: in step (2), current macroblock E needs to find its left A, top B, and top right The motion vector of the adjacent macroblock of C, if the upper right C is not in the current frame, select the upper left macroblock C' to replace the upper right macroblock C. 4. The optimal allocation method of a kind of hardware computing power resources in motion estimation according to claim 1, characterized in that in the step (2), the neighbors of left A, top B, and top right C of the current macroblock E When calculating the MV of a macroblock, if its SAD<preset value, the horizontal and vertical search method is used; otherwise, the diamond search method is used. 5. The optimal allocation method of a kind of hardware computing power resources in motion estimation according to claim 1, characterized in that: when the current macroblock E is predicted in the step (2), the candidate prediction points have (0,0 ), MV _A , MV _B , MV _C , MV _median . 6. The optimal allocation method of a kind of hardware computing capacity resource in motion estimation according to claim 1, it is characterized in that: when doing motion estimation to current macroblock E in step (2), calculate current macroblock and its matching The absolute error sum of each reference macroblock, select the macroblock with the smallest absolute error sum as the best reference macroblock, and finally obtain the motion vector of the current macroblock E. 7. A method for optimally allocating hardware computing power resources in motion estimation according to claim 1, characterized in that: in step (3), if there are currently macroblocks with the same highest priority, then according to their adjacent Macroblocks, A on the left, B on the top, and C on the right, the number of known motion vectors is selected. If the adjacent macroblocks of a certain macroblock have more known motion vectors, this macroblock will be used as the target macroblock. .