CN112291572B - H.264 fractional multiple motion estimation method and system - Google Patents

H.264 fractional multiple motion estimation method and system Download PDF

Info

Publication number
CN112291572B
CN112291572B CN202011101474.7A CN202011101474A CN112291572B CN 112291572 B CN112291572 B CN 112291572B CN 202011101474 A CN202011101474 A CN 202011101474A CN 112291572 B CN112291572 B CN 112291572B
Authority
CN
China
Prior art keywords
interpolation
block
pixel points
data
rate distortion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011101474.7A
Other languages
Chinese (zh)
Other versions
CN112291572A (en
Inventor
刘镇
陈励
余顺文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Tianze Communication Technology Co ltd
Original Assignee
Zhejiang Tianze Communication Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Tianze Communication Technology Co ltd filed Critical Zhejiang Tianze Communication Technology Co ltd
Priority to CN202011101474.7A priority Critical patent/CN112291572B/en
Publication of CN112291572A publication Critical patent/CN112291572A/en
Application granted granted Critical
Publication of CN112291572B publication Critical patent/CN112291572B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/567—Motion estimation based on rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/523—Motion estimation or motion compensation with sub-pixel accuracy
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The invention relates to an H.264 fractional multiple motion estimation method and system, which are characterized in that a matching data block with a first minimum cost rate distortion value with a current frame macro block is searched from a reference frame macro block through an integer multiple motion estimation algorithm, the matching data block is subjected to fractional multiple interpolation block by block, 1/2 pixel points in the horizontal direction are interpolated, 1/2 pixel points in the vertical direction are interpolated, 1/4 pixel points in the horizontal direction are interpolated, 1/4 pixel points in the vertical direction are interpolated, 1/4 pixel points in the right oblique direction are interpolated, and 1/4 pixel points in the left oblique direction are interpolated, so that an interpolation data block formed by 25 pixel points is obtained. After rough estimation is performed by adopting an integer multiple motion estimation algorithm, the interpolation data block formed by 25 pixel points is obtained through fractional multiple interpolation, and the calculation speed is improved only by using 25/49 calculated amount.

Description

H.264 fractional multiple motion estimation method and system
Technical Field
The invention relates to the technical field of image compression, in particular to an H.264 fractional multiple motion estimation method and system.
Background
The h.264 standard requires support for multiple segmentation types to fully exploit their spatial and temporal dependencies for higher compression efficiency. For example, on the basis of the macroblock size p16×16, a p16×8 partition is proposed, which is to divide one macroblock into two sub-blocks, upper and lower; other partitions include p8×16, p8×8, and the like, as shown in fig. 2.
For the sake of computational uniformity, the vector results of integer multiple motion estimation for different partition types are represented by a matrix array of 2×2 (P8×8 for the smallest partition), each array element representing the motion vector of each 8×8 sub-block, e.g. p16×16 would be the same four vector values for the partition, p16×8 the upper two sets of the same, the lower two sets of the same vector values, and so on.
The different segmentation types need to complete the fractional multiple motion estimation under the motion vectors, and the estimation process is basically the same as the interpolation operation to be processed except for the inconsistent segmentation mode caused by segmentation. We can consider multiplexing the same interpolation module to realize so as to reduce the consumption of FPGA resources.
The fractional motion estimation firstly carries out fractional interpolation on the whole pixel, and then carries out cost calculation and comparison on the fractional pixel points. Fractional doubling of the pixels needs to be done by an interpolation module.
The H.264 standard specifies the calculation mode of fractional pixel interpolation, and as shown in FIG. 3, the luminance 1/2 pixel point is obtained by interpolation of six tap filters with coefficients of (1, -5, 20, 1), rounding and then clipping. For example, the horizontal half-pixel h in fig. 1 is calculated by:
h= (E-5f+20g+20h-5i+j), h=clip 1 ((h+16) > > 5), where Clip1 is Clip to (0-255).
The calculation of the vertical half-pel is the same as described above. In addition, there are diagonal half-pixels, such as the point a in fig. 1, where the interpolation is needed after the horizontal half-pixel and the vertical half-pixel are all interpolated. Furthermore, we note that the diagonal half-pixels require interpolation of intermediate values of the horizontal and vertical half-pixel interpolation to avoid error diffusion.
Diagonal half-pixel a= (k-5j+20i+20b-5l+m), a=Clip1 ((a+512) > > 10), and 1/4 pixel point can be obtained by two-point linear interpolation, such as aa=Clip1 ((c+a+1) > > 1), hh=Clip1 ((i+a+1) > > 1), and rounding is floor operation here.
From the above description, it can be found that, in order to implement motion estimation of fractional multiple pixel points, interpolation needs to be performed on all the fractional pixel points, the interpolated data is very much, and the hardware implementation is particularly complex.
Disclosure of Invention
Based on the above, the invention aims to provide an H.264 fractional multiple motion estimation method and system, which only use 25/49 calculated amount and improve the calculation speed.
In order to achieve the above object, the present invention provides the following solutions:
a method of h.264 fractional multiple motion estimation, the method comprising:
dividing a macro block of a current frame to obtain a plurality of data dividing sub-blocks;
searching a matching data block with the minimum cost rate distortion value of each data segmentation sub-block from a reference frame macro block by adopting an integer multiple motion estimation algorithm;
performing fractional multiple interpolation on the matched data blocks block by block: interpolation is carried out on 1/2 pixel points in the horizontal direction, interpolation is carried out on 1/2 pixel points in the vertical direction, interpolation is carried out on 1/4 pixel points in the horizontal direction, interpolation is carried out on 1/4 pixel points in the vertical direction, interpolation is carried out on 1/4 pixel points in the right oblique direction, interpolation is carried out on 1/4 pixel points in the left oblique direction, and an interpolation data block formed by 25 pixel points is obtained; the 25 pixel points comprise 1 whole pixel point, 4 1/2 pixel points and 20 1/4 pixel points;
calculating a cost rate distortion value between each data segmentation sub-block and the interpolation data block corresponding to each data segmentation sub-block;
determining a minimum cost rate distortion value from the cost rate distortion values, and marking the minimum cost rate distortion value as a fractional minimum cost rate distortion value;
calculating a second motion vector between the data segmentation sub-block and the interpolation data block corresponding to the fraction multiple minimum cost rate distortion value;
and predicting fractional multiple motion estimation of the macro block of the current frame according to the second motion vector.
Optionally, the partitioning the macroblock of the current frame to obtain a plurality of data partitioning sub-blocks specifically includes: the current frame macroblock is partitioned into different partition types including p16×16, p16×8, p8×16, and p8×8.
Optionally, the interpolation is performed on 1/2 pixel points in the horizontal direction, specifically, the filtering interpolation of six taps is performed on the whole pixel points in the horizontal direction; and the interpolation is carried out on 1/2 pixel points in the vertical direction, in particular to the filter interpolation of six taps on the whole pixel points in the vertical direction.
Optionally, the first motion vector between the data-partitioning sub-block and the matching data block is expressed as
Figure BDA0002725502820000031
Form of->
Figure BDA0002725502820000032
Motion vectors representing four 8x8 blocks;
when the partition type is p16×16, (x) 1 ,y 1 )＝(x 2 ,y 2 )＝(x 3 ,y 3 )＝(x 4 ,y 4 )；
When the partition type is p16×8, (x) 1 ,y 1 )＝(x 2 ,y 2 )，(x 3 ,y 3 )＝(x 4 ,y 4 )；
When the partition type is p8×16, (x) 1 ,y 1 )＝(x 3 ,y 3 )，(x 2 ,y 2 )＝(x 4 ,y 4 )。
Optionally, the calculating a cost rate distortion value between each data division sub-block and the interpolation data block corresponding to each data division sub-block specifically includes: and calculating the sum of absolute differences or the sum of absolute transformation differences of pixel values between each data segmentation sub-block and the interpolation data block corresponding to each data segmentation sub-block.
The invention also discloses an H.264 fractional multiple motion estimation system, which comprises:
the segmentation module is used for segmenting the macro block of the current frame to obtain a plurality of data segmentation sub-blocks;
the matching data block determining module is used for searching a matching data block with the minimum cost rate distortion value with each data segmentation sub-block from the reference frame macro block by adopting an integer multiple motion estimation algorithm;
the fractional interpolation module is used for carrying out fractional interpolation on the matched data blocks block by block: interpolation is carried out on 1/2 pixel points in the horizontal direction, interpolation is carried out on 1/2 pixel points in the vertical direction, interpolation is carried out on 1/4 pixel points in the horizontal direction, interpolation is carried out on 1/4 pixel points in the vertical direction, interpolation is carried out on 1/4 pixel points in the right oblique direction, interpolation is carried out on 1/4 pixel points in the left oblique direction, and an interpolation data block formed by 25 pixel points is obtained; the 25 pixel points comprise 1 whole pixel point, 4 1/2 pixel points and 20 1/4 pixel points;
the cost rate distortion value calculation module is used for calculating a cost rate distortion value between each data segmentation sub-block and the interpolation data block corresponding to each data segmentation sub-block;
the fractional multiple minimum cost rate distortion value determining module is used for determining a minimum cost rate distortion value from the cost rate distortion values, and recording the minimum cost rate distortion value as a fractional multiple minimum cost rate distortion value;
a second motion vector calculation module, configured to calculate a second motion vector between the data segmentation sub-block and the interpolation data block, where the second motion vector corresponds to the fractional multiple minimum cost rate distortion value;
and the fractional multiple motion estimation module is used for predicting fractional multiple motion estimation of the macro block of the current frame according to the second motion vector.
Optionally, the segmentation module specifically includes:
a partition unit, configured to perform partition of different partition types on the current frame macroblock, where the partition types include p16×16, p16×8, p8×16, and p8×8.
Optionally, the fractional multiple interpolation module specifically includes:
a horizontal 1/2 pixel interpolation unit for performing six-tap filter interpolation on the whole pixel in the horizontal direction;
and the vertical 1/2 pixel point interpolation unit is used for carrying out six-tap filter interpolation on the integral pixel point in the vertical direction.
Optionally, the first motion vector between the data-partitioning sub-block and the matching data block is expressed as
Figure BDA0002725502820000041
Form of->
Figure BDA0002725502820000042
Motion vectors representing four 8x8 blocks;
when the partition type is p16×16, (x) 1 ,y 1 )＝(x 2 ,y 2 )＝(x 3 ,y 3 )＝(x 4 ,y 4 )；
When the partition type is p16×8, (x) 1 ,y 1 )＝(x 2 ,y 2 )，(x 3 ,y 3 )＝(x 4 ,y 4 )；
When the partition type is p8×16, (x) 1 ,y 1 )＝(x 3 ,y 3 )，(x 2 ,y 2 )＝(x 4 ,y 4 )。
Optionally, the cost rate distortion value calculation module specifically includes:
and the cost rate distortion value calculation unit is used for calculating the sum of absolute differences or the sum of absolute transformation differences of pixel values between each data segmentation sub-block and the interpolation data block corresponding to each data segmentation sub-block.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the invention searches a matching data block with a first minimum cost rate distortion value with a current frame macro block from a reference frame macro block through an integer multiple motion estimation algorithm, carries out fractional multiple interpolation on the matching data block, and comprises the steps of interpolating 1/2 pixel points in the horizontal direction, interpolating 1/2 pixel points in the vertical direction, interpolating 1/4 pixel points in the horizontal direction, interpolating 1/4 pixel points in the vertical direction, interpolating 1/4 pixel points in the right oblique direction, and interpolating 1/4 pixel points in the left oblique direction to obtain an interpolation data block formed by 25 pixel points. After rough estimation is performed by adopting an integer multiple motion estimation algorithm, the interpolation data block formed by 25 pixel points is obtained through fractional multiple interpolation, and the calculation speed is improved only by using 25/49 calculated amount.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of an H.264 fractional multiple motion estimation method according to an embodiment of the invention;
FIG. 2 is a schematic diagram of different partition types according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of conventional fractional-multiple-pixel interpolation according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of an 8×8 sub-block extension of an embodiment of the present invention;
FIG. 5 is a schematic diagram of a split sub-block of P8×8 according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of an interpolation data block formed by 25 pixel points according to an embodiment of the present invention;
FIG. 7 is a schematic diagram illustrating classification of pixel points according to an embodiment of the present invention;
FIG. 8 is a diagram of a cost rate distortion value summation in accordance with an embodiment of the present invention;
FIG. 9 is a schematic diagram illustrating interpolation calculation performed on different sub-blocks of different partition types according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of a pipeline structure according to an embodiment of the present invention;
FIG. 11 is a diagram illustrating the result of integer multiple motion estimation according to an embodiment of the present invention;
fig. 12 is a schematic diagram of an h.264 fractional multiple motion estimation system according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention aims to provide an H.264 fractional multiple motion estimation method and system, which only use 25/49 calculated amount and improve the calculation speed.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
Fig. 1 is a flow chart of an h.264 fractional multiple motion estimation method according to the present invention, as shown in fig. 1, and the method for h.264 fractional multiple motion estimation includes:
step 101: and dividing the macro block of the current frame to obtain a plurality of data division sub-blocks.
The method for dividing the macro block of the current frame to obtain a plurality of data dividing sub-blocks specifically comprises the following steps: the current frame macroblock is partitioned into different partition types including p16×16, p16×8, p8×16, and p8×8.
Step 102: a matching data block having a minimum cost rate distortion value (Rate distort optimal criterion RDO) with each of the data partitioning sub-blocks is searched for from a reference frame macroblock using an integer multiple motion estimation algorithm.
The searching mode includes, but is not limited to, full searching, diamond searching, binary searching and the like. The coordinates corresponding to the first minimum cost rate distortion value (Rate distort optimal criterion RDO) are the motion vector intg_mv_est values under different segmentation types. The data block to be subjected to the fractional multiple interpolation operation, i.e., the matching data block, is selected from the reference frame macro blocks by the motion vector.
Considering that the fractional multiple interpolation process is filtering interpolation with six taps, the matched data block needs to be formed by expanding 3 whole pixel points outwards, as shown in fig. 4, the integer multiple motion estimation module determines the integer multiple motion vector of each 8×8 sub-block, that is, determines the position of each 8×8 sub-block in the reference frame, and in order to perform fractional multiple interpolation, the size of each 8×8 sub-block needs to be expanded, and after expansion, each sub-block is 14×14, and then subsequent interpolation processing is performed, so as to obtain fractional multiple pixel values.
The number of data division sub-blocks is determined by the minimum supported division depth, if the P8 x8 division type is used as the minimum, the data is divided into 4 sub-blocks with the size of 8x8, and 4 sub-blocks with 4 division types are shown in fig. 5.
Various divisions after integer multiple searchUnder the cut type, the first motion vector corresponding to the data segmentation sub-block and the matching data block is expressed as
Figure BDA0002725502820000061
Form of->
Figure BDA0002725502820000062
Representing the motion vector of an 8x8 block.
When the partition type is p16×16, (x) 1 ,y 1 )＝(x 2 ,y 2 )＝(x 3 ,y 3 )＝(x 4 ,y 4 ) Indicating that the four 8x8 sub-blocks have the same motion change displacement relative to the reference frame.
When the partition type is p16×8, (x) 1 ,y 1 )＝(x 2 ,y 2 )，(x 3 ,y 3 )＝(x 4 ,y 4 ) Indicating that the upper two 8x8 sub-blocks and the lower two 8x8 sub-blocks have the same motion vector.
When the partition type is p8×16, (x) 1 ,y 1 )＝(x 3 ,y 3 )，(x 2 ,y 2 )＝(x 4 ,y 4 ) Indicating that the left two 8x8 sub-blocks and the right two 8x8 sub-blocks have the same motion vector.
When the partition type is p8×8, each 8×8 sub-block described above has an independent motion vector.
Step 103: performing fractional multiple interpolation on the matched data blocks block by block: interpolation is carried out on 1/2 pixel points in the horizontal direction, interpolation is carried out on 1/2 pixel points in the vertical direction, interpolation is carried out on 1/4 pixel points in the horizontal direction, interpolation is carried out on 1/4 pixel points in the vertical direction, interpolation is carried out on 1/4 pixel points in the right oblique direction, interpolation is carried out on 1/4 pixel points in the left oblique direction, and an interpolation data block formed by 25 pixel points is obtained; the 25 pixel points comprise 1 whole pixel point, 4 12 pixel points and 20 14 pixel points. An interpolation data block composed of 25 pixels is shown in fig. 6.
The determination of 25 pixels is to reduce the complexity of calculation, classify the pixels to be interpolated near the whole pixel, as shown in fig. 7, and divide the pixels to be interpolated into 8 types:
type 1: and (3) integrating the pixel points, and no interpolation is needed.
Type 2: the horizontal 1/2 pixel point is obtained by interpolation of a horizontal whole pixel point, and is specifically obtained by interpolation of a six-tap filter.
Type 3: the vertical 1/2 pixel point is obtained by interpolation of the vertical whole pixel point, and is specifically obtained by interpolation of a six-tap filter.
Type 4: the diagonal 1/2 pixel points are interpolated from the median of horizontal 2 (numbered 2) or vertical 3 (numbered 3).
Type 5: the horizontal 1/4 pixel point is obtained by linear interpolation of adjacent horizontal pixel points in the horizontal direction of the 1/4 pixel point.
Type 6: the vertical 1/4 pixel point is obtained by linear interpolation of the adjacent vertical pixel points in the vertical direction of the 1/4 pixel point.
Type 7: the right-oblique 1/4 pixel point is obtained by linear interpolation of adjacent pixel points in the right-oblique direction of the 1/4 pixel point.
Type 8: the left-oblique 1/4 pixel point is obtained by linear interpolation of adjacent pixel points in the left-oblique direction of the 1/4 pixel point.
Through the above analysis, for the diagonal half-pel 4 (labeled 4), an intermediate value of interpolation of the horizontal or vertical half-pel is needed, which needs to be temporarily stored and needs to be expanded to ensure accuracy, we can consider that the pel 4 and the pel related to the pel 4 are discarded without interpolation, i.e. only the 25 pels as shown in fig. 6 are interpolated.
It can be found that interpolation of a plurality of complex pixels (diagonal half-pixel 4) is abandoned, interpolation of the reserved pixels can be performed without intermediate values, and the periphery centered on the whole pixel 1 still contains half-pixel (1/2 pixel) and 1/4 pixel, so that the motion search of the fractional multiple pixels has less influence. Fig. 6 shows that we need to calculate a pixel block of 25 pixels in total, which 25 pixels comprise 1 whole pixel (reference numeral 1), 4 1/2 pixels (reference numerals 2 and 3) and 20 1/4 pixels ( reference numerals 5, 6, 7, 8).
Step 104: and calculating a cost rate distortion value between each data segmentation sub-block and the interpolation data block corresponding to each data segmentation sub-block.
Wherein step 104 has steps of: carrying out cost calculation (cost rate distortion value calculation) of a pipeline form on interpolation data blocks (data after fractional multiple interpolation of each block), wherein the cost calculation of 25 blocks is finished by adopting 25 beats of pipeline because the data consists of 25 pixel points after fractional multiple interpolation, namely, the cost is calculated one by one in a pipeline mode, and then the calculated cost value is stored in a register group or RAM; the cost calculation here follows the rate-distortion optimization principle, which is a macroblock coding mode selection technique based on a cost function of rate and distortion. This general popular formula combines the bit rate cost R and the distortion cost D in one cost function: j=d+λr.
The rate distortion optimal mode selection algorithm attempts to find the mode that minimizes cost J. The trade-off between rate R and distortion D is controlled by the lagrangian multiplier λ. A small lambda value will intensify the minimum of distortion D, allowing higher rates; while a large lambda value will intensify the minimum of the rate R, at the cost of a higher distortion. It is clear that the lambda value will affect the rate and distortion and that the selection of the best lambda value is a relatively cumbersome problem. Fortunately, the approximate choice of experience can provide better results in practical applications.
One good way is to take the lambda value as a function of QP calculation: λ=0.852 (QP-12)/3 QP is a quantization parameter that is a value of the image compression efficiency quality defined in the h.264 standard, which is adjusted according to the compression efficiency and image quality requirements.
The calculation of distortion D may be D SAD (Sum ofAbsolute Differences sum of absolute differences) or D SATD (SumofAbluteTransformadDifference absolute transform difference sum) and the like.
Figure BDA0002725502820000081
Figure BDA0002725502820000091
Where b (x, y) denotes the pixel of the macroblock of the current frame, b' (x, y) denotes the pixel of the macroblock after fractional interpolation, T is a shorthand for Transform (Transform), and Hadamard Transform (Hadamard) is used in this embodiment.
Calculating the cost and simultaneously carrying out interpolation operation of sub-blocks of other segmentation types; the cost is calculated and stored in a 25×4 storage unit, which may be a register set or a RAM. In the present embodiment, the absolute difference is added to D SAD Or the sum of absolute transformed differences D SATD As a cost value.
For 4 sub-blocks under each type to be segmented, as shown in fig. 8, cost values of the four sub-blocks are respectively stored in 4 independent storage units, and if the current segmentation type is p16×8, the same pixel point location cost addition of the first block and the second block is selected; if the current partition type is P8X 16, selecting the same pixel point location cost of the first block and the third block for addition; if the current segmentation type is P16×16, selecting the same pixel point location cost addition of the first block, the second block, the third block and the fourth block; if the current partition type is P8X 8, the 1-25 pixel point positions of each sub-block are independent. Each partition type contains 25 cost values.
Step 105: and determining a minimum cost rate distortion value from the cost rate distortion values, and marking the minimum cost rate distortion value as a fractional minimum cost rate distortion value.
Step 105 specifically includes: after all the 4 sub-blocks under each type to be segmented are segmented, uniformly performing cost comparison search, and searching for the minimum cost rate distortion value.
The comparison of the 25 cost values is completed by adopting 25 beats of scanning, and parallel comparison resources are not needed.
Step 106: and calculating a second motion vector between the data segmentation sub-block and the interpolation data block corresponding to the fraction multiple minimum cost rate distortion value.
Step 106 specifically includes: and searching to obtain the coordinate position of the minimum cost rate distortion value, namely the corresponding second motion vector.
Step 107: and predicting fractional multiple motion estimation of the macro block of the current frame according to the second motion vector.
The steps 106 to 107 specifically further include: performing block interpolation of 25 pixel points on the reference frame macro block according to the method, wherein the interpolated data are data blocks with fractional multiple pixels, then selecting predicted data according to the second motion vector predicted values under different segmentation types calculated in the step 106, and indexing out the interpolated data result through the coordinates to finish fractional multiple motion estimation as the second motion vector indicates the coordinates of the starting point of the optimal pixel block; and outputting the score predicted value of the pixel and the motion vector search result after the score multiple motion estimation is completed.
In this embodiment, interpolation of other divided sub-blocks is calculated by using each divided sub-block in the time of calculating the cost of 25 points, and the pipeline timing description is shown in fig. 11, and because of the pipeline structure design, interpolation operation of the remaining divided types is performed while calculating the cost of the last division, and so on.
In this embodiment, an interpolation module is used to complete interpolation calculation of different sub-blocks of different segmentation types, and then complete fractional motion estimation, and the specific flow is shown in fig. 9, where the integer motion estimation module outputs first motion vectors intg_mv_est1, intg_mv_est2, intg_mv_est3, and intg_mv_est4 under different segmentation types.
The first motion vector is intg_mv_est1 when the partition type is p16×16, is intg_mv_est2 when the partition type is p16×8, is intg_mv_est3 when the partition type is p8×16, and is intg_mv_est4 when the partition type is p8×8. The integer multiple motion estimation result is shown in fig. 10.
The fractional multiple motion estimation module extracts the data block to be interpolated according to the first motion vector, the number of the data blocks is determined by the minimum partition type, if the minimum partition type of P8 x8 is adopted, the data block is divided into 4 sub-blocks with the size of 8x8, 4 sub-blocks (1/2/3/4 in fig. 5) with the 4 partition types are sent to the interpolation module one by the control module to be interpolated, the interpolation sequence is sub-block 1 of P16 x 16, sub-block 1 of P16 x8, sub-block 1 of P8 x 16, sub-block 2 of P8 x8, sub-block 2 of P16 x 16, sub-block 2 of P8 x8, and the like.
The interpolation module completes interpolation of a sub-block within 8 beats, after interpolation of the sub-block, cost (RDO) calculation of a pixel block formed by 25 pixel points is carried out, and interpolation operation of other sub-blocks of different segmentation types is carried out while the cost is calculated; and after the cost is calculated, the cost is stored in a 25 multiplied by 4 storage unit, wherein the storage unit can be a register group or RAM, and after all 4 sub-blocks under each type to be divided are completed, the cost comparison and search are uniformly carried out.
The invention also discloses an H.264 fractional multiple motion estimation system, as shown in FIG. 12, comprising:
the segmentation module 201 is configured to segment a macroblock of a current frame to obtain a plurality of data segmentation sub-blocks.
The matching data block determining module 202 is configured to search for a matching data block having a minimum cost rate distortion value with each of the data segmentation sub-blocks from the reference frame macro block by using an integer multiple motion estimation algorithm.
A fractional interpolation module 203, configured to perform fractional interpolation on the matched data blocks block by block: interpolation is carried out on 1/2 pixel points in the horizontal direction, interpolation is carried out on 1/2 pixel points in the vertical direction, interpolation is carried out on 1/4 pixel points in the horizontal direction, interpolation is carried out on 1/4 pixel points in the vertical direction, interpolation is carried out on 1/4 pixel points in the right oblique direction, interpolation is carried out on 1/4 pixel points in the left oblique direction, and an interpolation data block formed by 25 pixel points is obtained; the 25 pixel points comprise 1 whole pixel point, 4 1/2 pixel points and 20 1/4 pixel points.
And the cost rate distortion value calculation module 204 is configured to calculate a cost rate distortion value between each of the data segmentation sub-blocks and the interpolation data block corresponding to each of the data segmentation sub-blocks.
The fractional-multiple minimum cost rate distortion value determining module 205 is configured to determine a minimum cost rate distortion value from the cost rate distortion values, and record the minimum cost rate distortion value as a fractional-multiple minimum cost rate distortion value.
A second motion vector calculation module 206, configured to calculate a second motion vector between the data segmentation sub-block and the interpolation data block corresponding to the fractional multiple minimum cost rate distortion value.
A fractional motion estimation module 207, configured to predict fractional motion estimation of the current frame macroblock according to the second motion vector.
The segmentation module specifically comprises:
a partition unit, configured to perform partition of different partition types on the current frame macroblock, where the partition types include p16×16, p16×8, p8×16, and p8×8.
The fractional multiple interpolation module specifically comprises:
and the horizontal 1/2 pixel point interpolation unit is used for carrying out six-tap filter interpolation on the integral pixel point in the horizontal direction.
And the vertical 1/2 pixel point interpolation unit is used for carrying out six-tap filter interpolation on the integral pixel point in the vertical direction.
The first motion vector between the data-partitioning sub-block and the matching data block is expressed as
Figure BDA0002725502820000111
Form of->
Figure BDA0002725502820000112
Representing the motion vectors of four 8x8 blocks.
When the partition type is p16×16, (x) 1 ,y 1 )＝(x 2 ,y 2 )＝(x 3 ,y 3 )＝(x 4 ,y 4 )。
When the partition type is p16×8, (x) 1 ,y 1 )＝(x 2 ,y 2 )，(x 3 ,y 3 )＝(x 4 ,y 4 )。
When the segmentation type isAt P8X 16, (x) 1 ,y 1 )＝(x 3 ,y 3 )，(x 2 ,y 2 )＝(x 4 ,y 4 )。
The cost rate distortion value calculation module specifically includes:
and the cost rate distortion value calculation unit is used for calculating the absolute difference sum or the absolute conversion difference sum of pixel values between each data segmentation sub-block and the interpolation data block corresponding to each data segmentation sub-block, and taking the absolute difference sum or the absolute conversion difference sum as a cost rate distortion value.
The invention provides a simplified interpolation module, which only interpolates 25 pixel points, and compared with the traditional interpolation algorithm, the module only uses 25/49 calculated amount; meanwhile, the invention also provides a structure adopting a pipeline on the basis of the interpolation, and the interpolation and the motion vector search process of all the segmentation modes are completed by multiplexing the same interpolation module. Only one interpolation module is needed in the whole process, the required beat is limited, resources and beats are effectively saved, and the structure is favorable for completing fractional multiple motion estimation under the condition of limited hardware resources.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.

Claims (6)

1. An h.264 fractional multiple motion estimation method, the method comprising:
dividing a macro block of a current frame to obtain a plurality of data dividing sub-blocks;
searching a matching data block with the minimum cost rate distortion value of each data segmentation sub-block from a reference frame macro block by adopting an integer multiple motion estimation algorithm;
performing fractional multiple interpolation on the matched data blocks block by block: interpolation is carried out on 1/2 pixel points in the horizontal direction, interpolation is carried out on 1/2 pixel points in the vertical direction, interpolation is carried out on 1/4 pixel points in the horizontal direction, interpolation is carried out on 1/4 pixel points in the vertical direction, interpolation is carried out on 1/4 pixel points in the right oblique direction, interpolation is carried out on 1/4 pixel points in the left oblique direction, and an interpolation data block formed by 25 pixel points is obtained; the 25 pixel points comprise 1 whole pixel point, 4 1/2 pixel points and 20 1/4 pixel points;
calculating a cost rate distortion value between each data segmentation sub-block and the interpolation data block corresponding to each data segmentation sub-block;
determining a minimum cost rate distortion value from the cost rate distortion values, and marking the minimum cost rate distortion value as a fractional minimum cost rate distortion value;
calculating a second motion vector between the data segmentation sub-block and the interpolation data block corresponding to the fraction multiple minimum cost rate distortion value;
predicting fractional multiple motion estimation of the current frame macroblock according to the second motion vector;
the method for dividing the macro block of the current frame to obtain a plurality of data dividing sub-blocks specifically comprises the following steps: partitioning the current frame macroblock by different partition types including p16×16, p16×8, p8×16, and p8×8;
the first motion vector between the data-partitioning sub-block and the matching data block is expressed as
Figure FDA0004265505400000011
Form of->
Figure FDA0004265505400000012
A motion vector representing four 8x8 blocks;
when the partition type is p16×16, (x) 1 ,y 1 )＝(x 2 ,y 2 )＝(x 3 ,y 3 )＝(x 4 ,y 4 )；
When the partition type is p16×8, (x) 1 ,y 1 )＝(x 2 ,y 2 )，(x 3 ,y 3 )＝(x 4 ,y 4 )；
When the partition type is p8×16, (x) 1 ,y 1 )＝(x 3 ,y 3 )，(x 2 ,y 2 )＝(x 4 ,y 4 )。
2. The h.264 fractional motion estimation method according to claim 1, wherein the interpolation of 1/2 pixel points in the horizontal direction is specifically: performing six-tap filter interpolation on the whole pixel point in the horizontal direction; the interpolation of 1/2 pixel points in the vertical direction is specifically as follows: and carrying out six-tap filter interpolation on the integral pixel point in the vertical direction.
3. The h.264 fractional motion estimation method according to claim 1, wherein calculating a cost rate distortion value between each of the data-segmentation sub-blocks and the interpolated data block corresponding to each of the data-segmentation sub-blocks comprises: and calculating the sum of absolute differences or the sum of absolute transformation differences of pixel values between each data segmentation sub-block and the interpolation data block corresponding to each data segmentation sub-block.
4. An h.264 fractional multiple motion estimation system, the system comprising:
the segmentation module is used for segmenting the macro block of the current frame to obtain a plurality of data segmentation sub-blocks;
the matching data block determining module is used for searching a matching data block with the minimum cost rate distortion value with each data segmentation sub-block from the reference frame macro block by adopting an integer multiple motion estimation algorithm;
the fractional interpolation module is used for carrying out fractional interpolation on the matched data blocks block by block: interpolation is carried out on 1/2 pixel points in the horizontal direction, interpolation is carried out on 1/2 pixel points in the vertical direction, interpolation is carried out on 1/4 pixel points in the horizontal direction, interpolation is carried out on 1/4 pixel points in the vertical direction, interpolation is carried out on 1/4 pixel points in the right oblique direction, interpolation is carried out on 1/4 pixel points in the left oblique direction, and an interpolation data block formed by 25 pixel points is obtained; the 25 pixel points comprise 1 whole pixel point, 4 1/2 pixel points and 20 1/4 pixel points;
the cost rate distortion value calculation module is used for calculating a cost rate distortion value between each data segmentation sub-block and the interpolation data block corresponding to each data segmentation sub-block;
the fractional multiple minimum cost rate distortion value determining module is used for determining a minimum cost rate distortion value from the cost rate distortion values, and recording the minimum cost rate distortion value as a fractional multiple minimum cost rate distortion value;
a second motion vector calculation module, configured to calculate a second motion vector between the data segmentation sub-block and the interpolation data block, where the second motion vector corresponds to the fractional multiple minimum cost rate distortion value;
a fractional multiple motion estimation module, configured to predict fractional multiple motion estimation of the current frame macroblock according to the second motion vector;
the segmentation module specifically comprises:
a partition unit configured to perform partition of different partition types for the current frame macroblock, where the partition types include p16×16, p16×8, p8×16, and p8×8;
the first motion vector between the data-partitioning sub-block and the matching data block is expressed as
Figure FDA0004265505400000031
Form of->
Figure FDA0004265505400000032
Representing four 8A motion vector of x8 blocks;
when the partition type is p16×16, (x) 1 ,y 1 )＝(x 2 ,y 2 )＝(x 3 ,y 3 )＝(x 4 ,y 4 )；
When the partition type is p16×8, (x) 1 ,y 1 )＝(x 2 ,y 2 )，(x 3 ,y 3 )＝(x 4 ,y 4 )；
When the partition type is p8×16, (x) 1 ,y 1 )＝(x 3 ,y 3 )，(x 2 ,y 2 )＝(x 4 ,y 4 )。
5. The h.264 fractional motion estimation system of claim 4, wherein said fractional interpolation module comprises:
a horizontal 1/2 pixel interpolation unit for performing six-tap filter interpolation on the whole pixel in the horizontal direction;
and the vertical 1/2 pixel point interpolation unit is used for carrying out six-tap filter interpolation on the integral pixel point in the vertical direction.
6. The h.264 fractional motion estimation system of claim 4 wherein said cost rate distortion value calculation module comprises:
and the cost rate distortion value calculation unit is used for calculating the sum of absolute differences or the sum of absolute transformation differences of pixel values between each data segmentation sub-block and the interpolation data block corresponding to each data segmentation sub-block.
CN202011101474.7A 2020-10-15 2020-10-15 H.264 fractional multiple motion estimation method and system Active CN112291572B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011101474.7A CN112291572B (en) 2020-10-15 2020-10-15 H.264 fractional multiple motion estimation method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011101474.7A CN112291572B (en) 2020-10-15 2020-10-15 H.264 fractional multiple motion estimation method and system

Publications (2)

Publication Number Publication Date
CN112291572A CN112291572A (en) 2021-01-29
CN112291572B true CN112291572B (en) 2023-07-11

Family

ID=74496847

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011101474.7A Active CN112291572B (en) 2020-10-15 2020-10-15 H.264 fractional multiple motion estimation method and system

Country Status (1)

Country Link
CN (1) CN112291572B (en)

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101790092B (en) * 2010-03-15 2013-02-06 河海大学常州校区 Intelligent filter designing method based on image block encoding information
CN102025960B (en) * 2010-12-07 2012-10-03 浙江大学 Motion compensation de-interlacing method based on adaptive interpolation
CN110392265B (en) * 2019-08-27 2022-03-25 广州虎牙科技有限公司 Inter-frame motion estimation method and device, electronic equipment and readable storage medium
CN112911313B (en) * 2021-01-14 2023-06-02 北京博雅慧视智能技术研究院有限公司 Sub-pixel motion estimation method and circuit device thereof

Also Published As

Publication number Publication date
CN112291572A (en) 2021-01-29

Similar Documents

Publication Publication Date Title
RU2654525C1 (en) Method for image interpolation using asymmetric interpolation filter and device therefor
JP3889040B2 (en) Method and apparatus for wide area-local block motion estimation
EP1262073B1 (en) Methods and apparatus for motion estimation using neighboring macroblocks
KR100659704B1 (en) Method, apparatus and system for obtaining a motion vector between first and second frames of video image data, and computer-readable medium
JP5065404B2 (en) Video coding with intra coding selection
JP5089610B2 (en) Block-based motion estimation method and apparatus
US10070128B2 (en) Intra prediction processor with reduced cost block partitioning and refined intra mode selection
US10621731B1 (en) Apparatus and method for efficient motion estimation for different block sizes
WO2001049039A1 (en) Methods and apparatus for reduction of prediction modes in motion estimation
JPH07262381A (en) Estimation method of image motion by block matching
CN110312132A (en) A kind of decoding method, device and its equipment
JP2005229600A (en) Motion compensation interpolating method by motion estimation of superposed block base and frame rate conversion device applying the same
US6690728B1 (en) Methods and apparatus for motion estimation in compressed domain
CN112055203B (en) Inter-frame prediction method, video coding method and related devices
CN113489987B (en) HEVC sub-pixel motion estimation method and device
KR100644629B1 (en) Method for estimating motion based on hybrid search block matching algorithm and frame-rate converter using thereof
US8542726B2 (en) Directional and motion-compensated discrete cosine transformation
EP3596698B1 (en) Motion estimation method and apparatus for plurality of frames
CN112291572B (en) H.264 fractional multiple motion estimation method and system
JP2008515298A (en) How to realize motion estimation
CN107135393B (en) Compression method of light field image
EP1420595B1 (en) Motion vector selection in a video motion estimator based on a preferred reference point
CN114598877A (en) Inter-frame prediction method and related equipment
KR101059649B1 (en) Encoding device, method and decoding device using adaptive interpolation filter
US20040120402A1 (en) Motion estimation apparatus for image data compression

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant