CN112291572A - H.264 fractional multiple motion estimation method and system - Google Patents
H.264 fractional multiple motion estimation method and system Download PDFInfo
- Publication number
- CN112291572A CN112291572A CN202011101474.7A CN202011101474A CN112291572A CN 112291572 A CN112291572 A CN 112291572A CN 202011101474 A CN202011101474 A CN 202011101474A CN 112291572 A CN112291572 A CN 112291572A
- Authority
- CN
- China
- Prior art keywords
- pixel points
- interpolation
- fractional
- block
- partition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/567—Motion estimation based on rate distortion criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/523—Motion estimation or motion compensation with sub-pixel accuracy
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The invention relates to an H.264 fractional multiple motion estimation method and a system, which search a matched data block with a first minimum cost rate distortion value with a current frame macro block from a reference frame macro block through an integral multiple motion estimation algorithm, and perform fractional multiple interpolation on the matched data block by block, wherein the fractional multiple interpolation comprises the steps of interpolating 1/2 pixel points in the horizontal direction, interpolating 1/2 pixel points in the vertical direction, interpolating 1/4 pixel points in the horizontal direction, interpolating 1/4 pixel points in the vertical direction, interpolating 1/4 pixel points in the right oblique direction, and interpolating 1/4 pixel points in the left oblique direction to obtain an interpolated data block consisting of 25 pixel points. After the rough estimation is carried out by adopting an integral multiple motion estimation algorithm, an interpolation data block consisting of 25 pixel points is obtained through fractional multiple interpolation, and only 25/49 calculated amount is used, so that the calculation speed is improved.
Description
Technical Field
The invention relates to the technical field of image compression, in particular to an H.264 fractional multiple motion estimation method and system.
Background
The h.264 standard requires support of multiple partition types to fully utilize the spatial and temporal correlation thereof, and achieve higher compression efficiency. For example, on the basis of the macroblock size P16 × 16, a partition of P16 × 8 is proposed, which divides a macroblock into two upper and lower subblocks; other examples include P8 × 16 division, P8 × 8 division, and the like, as shown in FIG. 2.
For the sake of computational uniformity, the vector results of integer-multiple motion estimation under different partition types are represented by a 2 × 2 (P8 × 8 for the minimum partition) matrix array, each array element represents the motion vector of each 8 × 8 sub-block, such as the same four vector values under P16 × 16 partition, P16 × 8 is the same vector values of the upper two groups, the same vector values of the lower two groups, and the like.
Different segmentation types need to complete fractional multiple motion estimation under motion vectors, and the estimation process is basically the same except for the inconsistency of the blocking mode caused by segmentation. We can consider multiplexing the same interpolation module to reduce the consumption of FPGA resources.
Fractional interpolation is firstly carried out on the whole pixel in fractional motion estimation, and then cost calculation and comparison of fractional pixel points are carried out. Fractional multiples of the pixel need to be done by the interpolation module.
The h.264 standard specifies a fractional pixel interpolation calculation method, and as shown in fig. 3, a luminance 1/2 pixel is obtained by interpolating through a six-tap filter with coefficients of (1, -5, 20, 1), rounding, and then performing amplitude limiting. For example, the horizontal half-pixel h in fig. 1 is calculated by:
h is (E-5F +20G +20H-5I + J), H is Clip1((H +16) > >5), wherein Clip1 is Clip limited to (0-255).
The calculation of the vertical half pixel is the same as described above. In addition, there are diagonal half-pixel points, such as the point a in fig. 1, which need to be obtained by performing interpolation after the interpolation is completed for both the horizontal half-pixel and the vertical half-pixel. Furthermore, we note that the diagonal half-pixels require interpolation of the intermediate values of the horizontal and vertical half-pixel interpolation to avoid error diffusion.
Diagonal half-pixel a is (k-5j +20i +20b-5l + m), a is Clip1((a +512) > >10), and 1/4 pixel point can be obtained by two-point linear interpolation, such as aa is Clip1((C + a +1) > >1), hh is Clip1((i + a +1) > >1), and the integration is floor operation.
From the above description, it can be found that, in order to implement motion estimation of fractional pixel points, interpolation needs to be performed on all the fractional pixel points, the interpolated data is very much, and the hardware implementation is particularly complex.
Disclosure of Invention
Based on this, the invention aims to provide a method and a system for estimating H.264 fractional multiple motion, which only use 25/49 calculation amount and improve the calculation speed.
In order to achieve the purpose, the invention provides the following scheme:
a h.264 fractional motion estimation method, the method comprising:
dividing a current frame macro block to obtain a plurality of data division sub-blocks;
searching a matched data block with the minimum cost rate distortion value of each data partition subblock from a reference frame macro block by adopting an integer multiple motion estimation algorithm;
performing fractional multiple interpolation on the matched data blocks block by block: interpolating 1/2 pixel points in the horizontal direction, interpolating 1/2 pixel points in the vertical direction, interpolating 1/4 pixel points in the horizontal direction, interpolating 1/4 pixel points in the vertical direction, interpolating 1/4 pixel points in the right oblique direction, interpolating 1/4 pixel points in the left oblique direction, and obtaining an interpolation data block consisting of 25 pixel points; the 25 pixel points comprise 1 whole pixel point, 4 1/2 pixel points and 20 1/4 pixel points;
calculating a cost rate distortion value between each data partition subblock and the interpolation data block corresponding to each data partition subblock;
determining a minimum cost rate distortion value from the cost rate distortion values, and recording the minimum cost rate distortion value as a fractional-fold minimum cost rate distortion value;
calculating a second motion vector between the data partition subblock corresponding to the fractional-minimum cost rate distortion value and the interpolated data block;
and predicting the fractional multiple motion estimation of the current frame macro block according to the second motion vector.
Optionally, the dividing the current frame macroblock to obtain a plurality of data partition subblocks specifically includes: partitioning the current frame macroblock by different partition types, the partition types including P16 × 16, P16 × 8, P8 × 16, and P8 × 8.
Optionally, the interpolating the 1/2 pixel points in the horizontal direction is specifically performed by performing six-tap filtering interpolation on the whole pixel point in the horizontal direction; the interpolation is performed on 1/2 pixel points in the vertical direction, specifically, six-tap filtering interpolation is performed on the whole pixel point in the vertical direction.
Optionally, the first motion vector between the data partition sub-block and the matching data block is represented as
In the form of a sheet of paper,
motion vectors representing four 8x8 blocks;
when the partition type is P16 × 16, (x)1,y1)＝(x2,y2)＝(x3,y3)＝(x4,y4)；
When the partition type is P16 × 8, (x)1,y1)＝(x2,y2)，(x3,y3)＝(x4,y4)；
When the partition type is P8 × 16, (x)1,y1)＝(x3,y3)，(x2,y2)＝(x4,y4)。
Optionally, the calculating a cost rate distortion value between each data partition subblock and the interpolation data block corresponding to each data partition subblock specifically includes: and calculating absolute difference of pixel values or absolute transformation difference sum between each data partition subblock and the interpolation data block corresponding to each data partition subblock.
The invention also discloses an H.264 fractional multiple motion estimation system, which comprises:
the partitioning module is used for partitioning the current frame macro block to obtain a plurality of data partitioning sub-blocks;
a matching data block determining module, configured to search a matching data block having a minimum cost rate distortion value with each of the data partition subblocks from a reference frame macroblock by using an integer-multiple motion estimation algorithm;
a fractional multiple interpolation module, configured to perform fractional multiple interpolation on the matching data blocks block by block: interpolating 1/2 pixel points in the horizontal direction, interpolating 1/2 pixel points in the vertical direction, interpolating 1/4 pixel points in the horizontal direction, interpolating 1/4 pixel points in the vertical direction, interpolating 1/4 pixel points in the right oblique direction, interpolating 1/4 pixel points in the left oblique direction, and obtaining an interpolation data block consisting of 25 pixel points; the 25 pixel points comprise 1 whole pixel point, 4 1/2 pixel points and 20 1/4 pixel points;
a cost rate distortion value calculation module, configured to calculate a cost rate distortion value between each data partition subblock and the interpolation data block corresponding to each data partition subblock;
a fractional-multiple minimum cost rate distortion value determining module, configured to determine a minimum cost rate distortion value from each of the cost rate distortion values, and record the minimum cost rate distortion value as a fractional-multiple minimum cost rate distortion value;
a second motion vector calculation module, configured to calculate a second motion vector between the data partition sub-block and the interpolated data block corresponding to the fractional-minimum cost rate distortion value;
and the fractional motion estimation module is used for predicting the fractional motion estimation of the current frame macro block according to the second motion vector.
Optionally, the segmentation module specifically includes:
a partition unit for performing partitions of different partition types on the current frame macroblock, the partition types including P16 × 16, P16 × 8, P8 × 16, and P8 × 8.
Optionally, the fractional interpolation module specifically includes:
the horizontal 1/2 pixel interpolation unit is used for carrying out six-tap filtering interpolation on the whole pixel in the horizontal direction;
and the vertical 1/2 pixel point interpolation unit is used for carrying out six-tap filtering interpolation on the whole pixel point in the vertical direction.
Optionally, the first motion vector between the data partition sub-block and the matching data block is represented as
In the form of a sheet of paper,
motion vectors representing four 8x8 blocks;
when the partition type is P16 × 16, (x)1,y1)＝(x2,y2)＝(x3,y3)＝(x4,y4)；
When the partition type is P16 × 8, (x)1,y1)＝(x2,y2)，(x3,y3)＝(x4,y4)；
When the partition type is P8 × 16, (x)1,y1)＝(x3,y3)，(x2,y2)＝(x4,y4)。
Optionally, the cost rate distortion value calculating module specifically includes:
and the cost rate distortion value calculation unit is used for calculating the absolute difference of pixel values between each data partition subblock and the interpolation data block corresponding to each data partition subblock or the sum of absolute transformation differences.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the method searches a matching data block with a first minimum cost rate distortion value with a current frame macro block from a reference frame macro block through an integral multiple motion estimation algorithm, and performs fractional multiple interpolation on the matching data block by block, wherein the fractional multiple interpolation comprises the steps of interpolating 1/2 pixel points in the horizontal direction, interpolating 1/2 pixel points in the vertical direction, interpolating 1/4 pixel points in the horizontal direction, interpolating 1/4 pixel points in the vertical direction, interpolating 1/4 pixel points in the right oblique direction, and interpolating 1/4 pixel points in the left oblique direction to obtain an interpolation data block consisting of 25 pixel points. After the rough estimation is carried out by adopting an integral multiple motion estimation algorithm, an interpolation data block consisting of 25 pixel points is obtained through fractional multiple interpolation, and only 25/49 calculated amount is used, so that the calculation speed is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
Fig. 1 is a schematic flow chart of a h.264 fractional multiple motion estimation method according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating different partition types according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating conventional fractional pixel interpolation according to an embodiment of the present invention;
fig. 4 is a schematic diagram of 8 × 8 sub-block expansion according to an embodiment of the present invention;
FIG. 5 is a diagram of a P8 × 8 partition subblock according to an embodiment of the invention;
fig. 6 is a schematic diagram of an interpolation data block composed of 25 pixels according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating classification of pixels according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of cost rate distortion value summation according to an embodiment of the present invention;
FIG. 9 is a schematic diagram illustrating interpolation calculation performed on different subblocks of different partition types according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of a pipeline structure according to an embodiment of the present invention;
FIG. 11 is a diagram illustrating integer multiple motion estimation results according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of an h.264 fractional multiple motion estimation system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide a method and a system for estimating H.264 fractional multiple motion, which only use 25/49 calculated amount and improve the calculation speed.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a schematic flow chart of an h.264 fractional motion estimation method of the present invention, and as shown in fig. 1, the h.264 fractional motion estimation method includes:
step 101: and partitioning the current frame macro block to obtain a plurality of data partitioning sub-blocks.
The dividing a current frame macro block to obtain a plurality of data division sub-blocks specifically includes: partitioning the current frame macroblock by different partition types, the partition types including P16 × 16, P16 × 8, P8 × 16, and P8 × 8.
Step 102: and searching a matching data block with the minimum cost Rate distortion value (Rate distortion optimal criterion RDO) of each data partition subblock from the reference frame macro block by adopting an integer multiple motion estimation algorithm.
The searching method includes, but is not limited to, full search, diamond search, binary search, and the like. The coordinate corresponding to the first minimum cost Rate distortion value (Rate distortion optimal temporal criterion RDO) is the motion vector intg _ mv _ est value under different segmentation types. And selecting a data block to be subjected to fractional multiple interpolation operation from the reference frame macro block through the motion vector, namely, selecting a matching data block.
Considering that the fractional multiple interpolation process is a six-tap filtering interpolation, the configuration of the matched data block needs to be expanded outward by 3 whole pixel points, as shown in fig. 4, the integer multiple motion estimation module determines the integer multiple motion vector of each 8 × 8 sub-block, that is, determines the position of each 8 × 8 sub-block in the reference frame, in order to perform the fractional multiple interpolation, each 8 × 8 sub-block needs to be expanded, the size of each expanded sub-block is 14 × 14, and then subsequent interpolation processing is performed to obtain the fractional multiple pixel value.
The number of sub-blocks of the data partition is determined by the minimum supported partition depth, for example, if the P8 × 8 partition type is used at minimum, the sub-blocks will be divided into 4 sub-blocks with a size of 8 × 8, and 4 sub-blocks with 4 partition types, as shown in fig. 5.
Under various segmentation types after integral multiple search, the first motion vector corresponding to the data segmentation subblock and the matched data block is expressed as
In the form of a sheet of paper,
representing the motion vector of an 8x8 block.
When the partition type is P16 × 16, (x)1,y1)＝(x2,y2)＝(x3,y3)＝(x4,y4) It means that the four 8x8 sub-blocks have the same motion variation displacement with respect to the reference frame.
When the partition type is P16 × 8, (x)1,y1)＝(x2,y2)，(x3,y3)＝(x4,y4) It means that the upper two 8 × 8 sub-blocks and the lower two 8 × 8 sub-blocks have the same motion vector.
When the partition type is P8 × 16，(x1,y1)＝(x3,y3)，(x2,y2)＝(x4,y4) It means that the left two 8 × 8 sub-blocks and the right two 8 × 8 sub-blocks have the same motion vector.
When the partition type is P8 × 8, each 8 × 8 sub-block described above has an independent motion vector.
Step 103: performing fractional multiple interpolation on the matched data blocks block by block: interpolating 1/2 pixel points in the horizontal direction, interpolating 1/2 pixel points in the vertical direction, interpolating 1/4 pixel points in the horizontal direction, interpolating 1/4 pixel points in the vertical direction, interpolating 1/4 pixel points in the right oblique direction, interpolating 1/4 pixel points in the left oblique direction, and obtaining an interpolation data block consisting of 25 pixel points; the 25 pixels include 1 whole pixel, 4 12 pixels and 20 14 pixels. An interpolation data block composed of 25 pixels is shown in fig. 6.
The determination of 25 pixels is to reduce the complexity of calculation, classify the pixels to be interpolated near the whole pixel, as shown in fig. 7, and classify the pixels to be interpolated into 8 types:
type 1: and (4) the pixel is completed without interpolation.
Type 2: and the horizontal 1/2 pixel point is obtained by interpolation of the horizontal whole pixel point, and is specifically obtained by interpolation of a six-tap filter.
Type 3: and the vertical 1/2 pixel points are obtained by interpolation of vertical whole pixel points, and are specifically obtained by interpolation of a six-tap filter.
Type 4: the diagonal 1/2 pixel is interpolated from the median of horizontal 2 (labeled 2) or vertical 3 (labeled 3).
Type 5: and the horizontal 1/4 pixel point is obtained by linear interpolation of the horizontal pixel points adjacent to the 1/4 pixel point in the horizontal direction.
Type 6: and the vertical 1/4 pixel point is obtained by linear interpolation of the vertical pixel point adjacent to the 1/4 pixel point in the vertical direction.
Type 7: and the right oblique 1/4 pixel point is obtained by linear interpolation of the pixel points adjacent to the 1/4 pixel point in the right oblique direction.
Type 8: and the pixel point of the left oblique 1/4 is obtained by linear interpolation of the pixel points adjacent to the 1/4 pixel point in the left oblique direction.
Through the above analysis, for the diagonal half-pixel point 4 (labeled as 4), an intermediate value of the interpolation of the horizontal or vertical half-pixel point is needed, the intermediate value needs to be temporarily stored and needs to be expanded to ensure the precision, and we can consider that the pixel point 4 and the pixel point related to the pixel point 4 are discarded without interpolation, that is, only 25 pixel points as shown in fig. 6 are interpolated.
It can be found that interpolation of a plurality of complex pixels (diagonal half-pixel 4) is abandoned, interpolation of the retained pixels can be performed without intermediate values, and the periphery with the whole pixel 1 as the center still contains half-pixels (1/2 pixels) and 1/4 pixels, so that the motion search influence on fractional-fold pixels is small. Fig. 6 shows that we need to calculate a pixel block composed of 25 pixels, where the 25 pixels include 1 whole pixel (label 1), 4 1/2 pixels (labels 2 and 3), and 20 1/4 pixels (labels 5, 6, 7, and 8).
Step 104: and calculating a cost rate distortion value between each data partition subblock and the interpolation data block corresponding to each data partition subblock.
Wherein step 104 comprises: performing cost calculation (cost rate distortion value calculation) in a pipeline form on interpolated data blocks (data after fractional multiple interpolation of each block), wherein the data consisting of only 25 pixel points is obtained after the fractional multiple interpolation, so that the cost calculation of 25 blocks is completed by adopting 25 beats of pipeline, namely the cost is calculated one by one in a pipeline mode, and then the calculated cost value is stored in a register group or an RAM; the cost calculation here follows the rate-distortion optimization principle, which is a macroblock coding mode selection technique based on the cost function of rate and distortion. This general popular formula combines the bit rate cost R and the distortion cost D in a cost function: j ═ D + λ R.
The rate-distortion optimal mode selection algorithm attempts to find the mode that minimizes the cost J. The trade-off of rate R and distortion D is governed by the lagrange multiplier λ. A small value of λ will enhance the minimum of distortion D, allowing higher rates; while a large value of lambda will emphasize the minimum of the rate R, the price paid for this is a higher degree of distortion. It is clear that the value of λ will affect the rate and distortion, and choosing the best value of λ is a tedious problem. Fortunately, empirical approximation choices can provide better results in practical applications.
A good way is to take the value of λ as a function of the QP calculation: λ ═ 0.852(QP-12)/3QP is a quantization parameter, which is a value of the image compression efficiency quality defined in the h.264 standard, and is adjusted according to the compression efficiency and image quality requirements.
The distortion D may be calculated as DSAD(Sum of absolute Differences) or DSATD(SumofAbsoluteTransformedDifferences sum of absolute transformed differences), and the like.
Where b (x, y) denotes the pixels of the current frame macroblock, b' (x, y) denotes the pixels of the fractional-times interpolated macroblock, T is an abbreviation of Transform (Hadamard) which is employed in the present embodiment.
Calculating the cost and simultaneously performing sub-block interpolation operation of other segmentation types; the cost is calculated and stored in a 25 × 4 memory unit, which may be a register set or a RAM. In the present embodiment, the sum of absolute differences D is addedSADOr sum of absolute transformed differences DSATDAs a cost value.
For 4 subblocks under each to-be-segmented type, as shown in fig. 8, cost values of four subblocks are stored in 4 independent storage units, and if the current segmentation type is P16 × 8, the cost values of the same pixel point of the first block and the second block are selected and added; if the current segmentation type is P8 multiplied by 16, the position cost of the same pixel point of the first block and the third block is selected to be added; if the current segmentation type is P16 multiplied by 16, selecting the same pixel point position cost addition of the first block, the second block, the third block and the fourth block; if the current partition type is P8 multiplied by 8, the position cost of 1-25 pixel points of each sub-block is independent. Each segmentation type contains 25 cost values.
Step 105: and determining the minimum cost rate distortion value from the cost rate distortion values, and recording the minimum cost rate distortion value as a fractional-fold minimum cost rate distortion value.
Wherein, step 105 specifically comprises: after all the 4 sub-blocks in each to-be-segmented type are segmented, uniformly performing cost comparison search, and searching the minimum cost rate distortion value.
The comparison of the 25 cost values is completed by 25-beat scanning, and parallel comparison resources are not needed.
Step 106: and calculating a second motion vector between the data partition subblock corresponding to the fractional-minimum cost rate distortion value and the interpolation data block.
Wherein, step 106 specifically includes: and searching the coordinate position of the minimum cost rate distortion value, namely the coordinate position is the corresponding second motion vector.
Step 107: and predicting the fractional multiple motion estimation of the current frame macro block according to the second motion vector.
Wherein, the steps 106 to 107 further include: carrying out block interpolation of 25 pixel points on a reference frame macro block again according to the method, wherein the interpolated data is a data block with fractional times of pixels, then selecting predicted data according to second motion vector predicted values under different segmentation types calculated in step 106, and finishing the motion estimation of the fractional times as the second motion vector indicates the coordinates of the starting point of the optimal pixel block and the interpolated data result is led out through the coordinate index; and outputting the fraction predicted value of the pixel and the motion vector search result after the fraction multiple motion estimation is finished.
In this embodiment, interpolation of other segmented sub-blocks is calculated by using each segmented sub-block within the time of calculating 25 point costs, the pipeline timing description is as shown in fig. 11, and because the pipeline structure design is adopted, the interpolation operation of the remaining segmentation types is performed while the cost calculation is performed on the former segmentation, and the rest is analogized.
In this embodiment, one interpolation module is used to complete interpolation calculation of different subblocks of different partition types, and then complete fractional motion estimation, and the specific flow is as shown in fig. 9, where the integer multiple motion estimation module outputs first motion vectors intg _ mv _ est1, intg _ mv _ est2, intg _ mv _ est3, and intg _ mv _ est4 under different partition types.
The first motion vector is intg _ mv _ est1 when the partition type is P16 × 16, intg _ mv _ est2 when the partition type is P16 × 8, intg _ mv _ est3 when the partition type is P8 × 16, and intg _ mv _ est4 when the partition type is P8 × 8. The integer-multiple motion estimation results are shown in fig. 10.
The fractional multiple motion estimation module extracts a data block to be interpolated according to the first motion vector, the number of the data block is determined by the minimum partition type, if the minimum partition type is P8 × 8, the data block is divided into 4 sub-blocks with the size of 8 × 8, the 4 sub-blocks (see 1/2/3/4 in fig. 5) with the size of 4 partitions are sent to the interpolation module one by the control module for interpolation, and the interpolation sequence is that sub-block 1 with the size of P16 × 16, sub-block 1 with the size of P16 × 8, sub-block 1 with the size of P8 × 16, sub-block 1 with the size of P8 × 8, sub-block 2 with the size of P16 × 16, sub-block 2 with the size of P16 × 8, sub-block 2 with the size of P8 × 16, sub-block 2 with the size of P8.
The interpolation module completes interpolation of a subblock in 8 beats, cost (RDO) calculation of a pixel block formed by 25 pixel points is carried out after interpolation of the subblock, and interpolation operation of subblocks of other segmentation types is carried out while the cost is calculated; and after the cost is calculated, storing the calculated cost in a 25 x 4 storage unit, wherein the storage unit can be a register group or an RAM, and after all 4 sub-blocks in each to-be-divided type are completed, uniformly performing cost comparison and search.
The invention also discloses an h.264 fractional multiple motion estimation system, as shown in fig. 12, the system comprises:
the partitioning module 201 is configured to partition a current frame macroblock to obtain a plurality of data partition subblocks.
A matching data block determining module 202, configured to search a reference frame macroblock for a matching data block having a minimum cost rate distortion value with each of the data partition subblocks by using an integer multiple motion estimation algorithm.
A fractional multiple interpolation module 203, configured to perform fractional multiple interpolation on the matching data blocks block by block: interpolating 1/2 pixel points in the horizontal direction, interpolating 1/2 pixel points in the vertical direction, interpolating 1/4 pixel points in the horizontal direction, interpolating 1/4 pixel points in the vertical direction, interpolating 1/4 pixel points in the right oblique direction, interpolating 1/4 pixel points in the left oblique direction, and obtaining an interpolation data block consisting of 25 pixel points; the 25 pixels include 1 whole pixel, 4 1/2 pixels and 20 1/4 pixels.
A cost rate distortion value calculating module 204, configured to calculate a cost rate distortion value between each data partition subblock and the interpolation data block corresponding to each data partition subblock.
A fractional-multiple minimum cost rate distortion value determining module 205, configured to determine a minimum cost rate distortion value from each of the cost rate distortion values, and record the minimum cost rate distortion value as a fractional-multiple minimum cost rate distortion value.
A second motion vector calculating module 206, configured to calculate a second motion vector between the data partition sub-block and the interpolated data block corresponding to the fractional-minimum cost rate distortion value.
A fractional motion estimation module 207, configured to predict a fractional motion estimation of the current frame macro block according to the second motion vector.
The segmentation module specifically comprises:
a partition unit for performing partitions of different partition types on the current frame macroblock, the partition types including P16 × 16, P16 × 8, P8 × 16, and P8 × 8.
The fractional interpolation module specifically includes:
and the horizontal 1/2 pixel point interpolation unit is used for performing six-tap filtering interpolation on the whole pixel point in the horizontal direction.
And the vertical 1/2 pixel point interpolation unit is used for performing six-tap filtering interpolation on the whole pixel point in the vertical direction.
A first motion vector between the sub-block of data partitions and the matching data block is represented as
In the form of a sheet of paper,
representing the motion vectors of four 8x8 blocks.
When the partition type is P16 × 16, (x)1,y1)＝(x2,y2)＝(x3,y3)＝(x4,y4)。
When the partition type is P16 × 8, (x)1,y1)＝(x2,y2)，(x3,y3)＝(x4,y4)。
When the partition type is P8 × 16, (x)1,y1)＝(x3,y3)，(x2,y2)＝(x4,y4)。
The cost rate distortion value calculation module specifically includes:
and the cost rate distortion value calculation unit is used for calculating the absolute difference of pixel values between each data partition subblock and the interpolation data block corresponding to each data partition subblock or the sum of absolute transformation differences, and taking the absolute difference or the sum of absolute transformation differences as the cost rate distortion value.
The invention provides a simplified interpolation module, which only interpolates 25 pixel points, and only uses 25/49 calculated amount compared with the traditional interpolation algorithm; meanwhile, the invention also provides a flow line structure on the basis of the interpolation, and the same interpolation module is multiplexed to complete the interpolation and motion vector search processes of all the segmentation modes. The whole process only needs one interpolation module, the required beat is limited, resources and beats are effectively saved, and the structure is beneficial to completing fractional motion estimation under the condition of limited hardware resources.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (10)
1. A h.264 fractional motion estimation method, the method comprising:
dividing a current frame macro block to obtain a plurality of data division sub-blocks;
searching a matched data block with the minimum cost rate distortion value of each data partition subblock from a reference frame macro block by adopting an integer multiple motion estimation algorithm;
performing fractional multiple interpolation on the matched data blocks block by block: interpolating 1/2 pixel points in the horizontal direction, interpolating 1/2 pixel points in the vertical direction, interpolating 1/4 pixel points in the horizontal direction, interpolating 1/4 pixel points in the vertical direction, interpolating 1/4 pixel points in the right oblique direction, interpolating 1/4 pixel points in the left oblique direction, and obtaining an interpolation data block consisting of 25 pixel points; the 25 pixel points comprise 1 whole pixel point, 4 1/2 pixel points and 20 1/4 pixel points;
calculating a cost rate distortion value between each data partition subblock and the interpolation data block corresponding to each data partition subblock;
determining a minimum cost rate distortion value from the cost rate distortion values, and recording the minimum cost rate distortion value as a fractional-fold minimum cost rate distortion value;
calculating a second motion vector between the data partition subblock corresponding to the fractional-minimum cost rate distortion value and the interpolated data block;
and predicting the fractional multiple motion estimation of the current frame macro block according to the second motion vector.
2. The h.264 fractional motion estimation method according to claim 1, wherein the dividing a current frame macroblock to obtain a plurality of data partition sub-blocks specifically comprises: partitioning the current frame macroblock by different partition types, the partition types including P16 × 16, P16 × 8, P8 × 16, and P8 × 8.
3. The method for estimating h.264 fractional multiple motion according to claim 1, wherein the interpolating 1/2 pixels in the horizontal direction specifically comprises: carrying out six-tap filtering interpolation on the whole pixel point in the horizontal direction; the interpolation is carried out on 1/2 pixel points in the vertical direction, and the interpolation specifically comprises the following steps: and carrying out six-tap filtering interpolation on the whole pixel point in the vertical direction.
4. The h.264 fractional motion estimation method of claim 2 wherein the first motion vector between the data partition sub-block and the matching data block is represented as
In the form of a sheet of paper,
represents motion vectors of four 8x8 blocks;
when the partition type is P16 × 16, (x)1,y1)＝(x2,y2)＝(x3,y3)＝(x4,y4)；
When the partition type is PAt 16X 8, (x)1,y1)＝(x2,y2)，(x3,y3)＝(x4,y4)；
When the partition type is P8 × 16, (x)1,y1)＝(x3,y3)，(x2,y2)＝(x4,y4)。
5. The method of claim 1, wherein the calculating the cost rate distortion value between each data partition sub-block and the interpolated data block corresponding to each data partition sub-block comprises: and calculating absolute difference of pixel values or absolute transformation difference sum between each data partition subblock and the interpolation data block corresponding to each data partition subblock.
6. An h.264 fractional multiple motion estimation system, the system comprising:
the partitioning module is used for partitioning the current frame macro block to obtain a plurality of data partitioning sub-blocks;
a matching data block determining module, configured to search a matching data block having a minimum cost rate distortion value with each of the data partition subblocks from a reference frame macroblock by using an integer-multiple motion estimation algorithm;
a fractional multiple interpolation module, configured to perform fractional multiple interpolation on the matching data blocks block by block: interpolating 1/2 pixel points in the horizontal direction, interpolating 1/2 pixel points in the vertical direction, interpolating 1/4 pixel points in the horizontal direction, interpolating 1/4 pixel points in the vertical direction, interpolating 1/4 pixel points in the right oblique direction, interpolating 1/4 pixel points in the left oblique direction, and obtaining an interpolation data block consisting of 25 pixel points; the 25 pixel points comprise 1 whole pixel point, 4 1/2 pixel points and 20 1/4 pixel points;
a cost rate distortion value calculation module, configured to calculate a cost rate distortion value between each data partition subblock and the interpolation data block corresponding to each data partition subblock;
a fractional-multiple minimum cost rate distortion value determining module, configured to determine a minimum cost rate distortion value from each of the cost rate distortion values, and record the minimum cost rate distortion value as a fractional-multiple minimum cost rate distortion value;
a second motion vector calculation module, configured to calculate a second motion vector between the data partition sub-block and the interpolated data block corresponding to the fractional-minimum cost rate distortion value;
and the fractional motion estimation module is used for predicting the fractional motion estimation of the current frame macro block according to the second motion vector.
7. The h.264 fractional-multiple motion estimation system of claim 6, wherein the segmentation module specifically comprises:
a partition unit for performing partitions of different partition types on the current frame macroblock, the partition types including P16 × 16, P16 × 8, P8 × 16, and P8 × 8.
8. The h.264 fractional multiple motion estimation system according to claim 6, wherein the fractional multiple interpolation module specifically comprises:
the horizontal 1/2 pixel interpolation unit is used for performing six-tap filtering interpolation on the whole pixel in the horizontal direction;
and the vertical 1/2 pixel point interpolation unit is used for performing six-tap filtering interpolation on the whole pixel point in the vertical direction.
9. The h.264 fractional multiple motion estimation system of claim 7 wherein the first motion vector between the data partition sub-block and the matching data block is represented as
In the form of a sheet of paper,
representing four 8x8 blocksThe motion vector of (2);
when the partition type is P16 × 16, (x)1,y1)＝(x2,y2)＝(x3,y3)＝(x4,y4)；
When the partition type is P16 × 8, (x)1,y1)＝(x2,y2)，(x3,y3)＝(x4,y4)；
When the partition type is P8 × 16, (x)1,y1)＝(x3,y3)，(x2,y2)＝(x4,y4)。
10. The method of claim 1, wherein the cost rate distortion value calculation module specifically includes:
and the cost rate distortion value calculation unit is used for calculating the absolute difference of pixel values between each data partition subblock and the interpolation data block corresponding to each data partition subblock or the sum of absolute transformation differences.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011101474.7A CN112291572B (en) | 2020-10-15 | 2020-10-15 | H.264 fractional multiple motion estimation method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011101474.7A CN112291572B (en) | 2020-10-15 | 2020-10-15 | H.264 fractional multiple motion estimation method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112291572A true CN112291572A (en) | 2021-01-29 |
| CN112291572B CN112291572B (en) | 2023-07-11 |
Family
ID=74496847
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011101474.7A Active CN112291572B (en) | 2020-10-15 | 2020-10-15 | H.264 fractional multiple motion estimation method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112291572B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101790092A (en) * | 2010-03-15 | 2010-07-28 | 河海大学常州校区 | Intelligent filter designing method based on image block encoding information |
| CN102025960A (en) * | 2010-12-07 | 2011-04-20 | 浙江大学 | Motion compensation de-interlacing method based on adaptive interpolation |
| CN110392265A (en) * | 2019-08-27 | 2019-10-29 | 广州虎牙科技有限公司 | Inter frame motion estimation method, apparatus, electronic equipment and readable storage medium storing program for executing |
| CN112911313A (en) * | 2021-01-14 | 2021-06-04 | 北京博雅慧视智能技术研究院有限公司 | Sub-pixel motion estimation method and circuit device thereof |
-
2020
- 2020-10-15 CN CN202011101474.7A patent/CN112291572B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101790092A (en) * | 2010-03-15 | 2010-07-28 | 河海大学常州校区 | Intelligent filter designing method based on image block encoding information |
| CN102025960A (en) * | 2010-12-07 | 2011-04-20 | 浙江大学 | Motion compensation de-interlacing method based on adaptive interpolation |
| CN110392265A (en) * | 2019-08-27 | 2019-10-29 | 广州虎牙科技有限公司 | Inter frame motion estimation method, apparatus, electronic equipment and readable storage medium storing program for executing |
| CN112911313A (en) * | 2021-01-14 | 2021-06-04 | 北京博雅慧视智能技术研究院有限公司 | Sub-pixel motion estimation method and circuit device thereof |
Non-Patent Citations (2)
| Title |
|---|
| 刘锴;韩进;: "H.264/AVC分数运动估计改进及VLSI实现", 电脑知识与技术, no. 29 * |
| 程霖;郭宝龙;孙宏军;: "基于H.264的一种快速1/4像素运动估计算法", 计算机应用研究, no. 02 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112291572B (en) | 2023-07-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| RU2654525C1 (en) | Method for image interpolation using asymmetric interpolation filter and device therefor | |
| JP3889040B2 (en) | Method and apparatus for wide area-local block motion estimation | |
| US11902563B2 (en) | Encoding and decoding method and device, encoder side apparatus and decoder side apparatus | |
| JP5089610B2 (en) | Block-based motion estimation method and apparatus | |
| US20110261886A1 (en) | Image prediction encoding device, image prediction encoding method, image prediction encoding program, image prediction decoding device, image prediction decoding method, and image prediction decoding program | |
| JP3655651B2 (en) | Data processing device | |
| TW201143451A (en) | Motion prediction method | |
| CN107810632B (en) | Intra prediction processor with reduced cost block segmentation and refined intra mode selection | |
| KR101772252B1 (en) | Motion prediction method | |
| KR100994773B1 (en) | Method and Apparatus for generating motion vector in hierarchical motion estimation | |
| US10621731B1 (en) | Apparatus and method for efficient motion estimation for different block sizes | |
| WO2008003220A1 (en) | Motion vector estimation method | |
| CN100502511C (en) | Method for organizing interpolation image memory for fractional pixel precision predication | |
| EP2076047B1 (en) | Video motion estimation | |
| CN101288310A (en) | Motion estimation | |
| CN112291572A (en) | H.264 fractional multiple motion estimation method and system | |
| EP2953365B1 (en) | Moving image coding device | |
| CN114007078B (en) | Method, device and equipment for constructing motion information candidate list | |
| EP1420595B1 (en) | Motion vector selection in a video motion estimator based on a preferred reference point | |
| CN114598877A (en) | Inter-frame prediction method and related equipment | |
| KR101059649B1 (en) | Encoding device, method and decoding device using adaptive interpolation filter | |
| Jiang et al. | A fine-grained parallel intra prediction for HEVC based on GPU | |
| Chiang et al. | Fast multi-reference frame motion estimation via downhill simplex search | |
| Pan et al. | Adaptive Multi-reference downhill simplex search based on spatial-temporal motion smoothness criterion | |
| Alain et al. | Locally linear embedding methods for inter image coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |