CN116389743A

CN116389743A - Image encoding method, image encoding device, chip, terminal, storage medium, and program product

Info

Publication number: CN116389743A
Application number: CN202310417431.7A
Authority: CN
Inventors: 胡文强
Original assignee: Zeku Technology Shanghai Corp Ltd
Current assignee: Zeku Technology Shanghai Corp Ltd
Priority date: 2023-04-18
Filing date: 2023-04-18
Publication date: 2023-07-04

Abstract

The embodiment of the application discloses an image coding method, an image coding device, a chip, a terminal, a storage medium and a program product, and belongs to the technical field of image coding. The method comprises the following steps: fractional interpolation operation is carried out on integer pixel points in a reference image frame based on asymmetric tap coefficients to obtain row fractional pixels and column fractional pixels, wherein the asymmetric tap coefficients comprise row tap coefficients and column tap coefficients, and the coefficient number of the row tap coefficients is different from the coefficient number of the column tap coefficients; image encoding is performed based on the row fractional pixels and the column fractional pixels. According to the method, by setting the asymmetric tap coefficients, the interpolation operand and the operation complexity in the process of acquiring the row fraction pixels or the column fraction pixels can be reduced, so that the chip power consumption of the video coding chip is reduced.

Description

Image encoding method, image encoding device, chip, terminal, storage medium, and program product

Technical Field

The embodiment of the application relates to the technical field of image coding, in particular to an image coding method, an image coding device, a chip, a terminal, a storage medium and a program product.

Background

Motion estimation is a very important technique in the HEVC video coding standard, which can effectively improve the performance of the encoder. Motion estimation implements the process of finding the best reference block from the encoded reference frame by integer-pixel search and fractional-pixel search; while the motion of the object is continuous, the pixels themselves are discrete, which presents a problem: the best reference block of the block to be encoded is not necessarily located at the integer pixel point position of the reference frame. In order to predict the current image block to be encoded more accurately, motion estimation is required at the fractional position, but the image fractional pixel position is not sampled, so that interpolation operation is required to be performed on the reference block before fractional pixel searching, thereby generating fractional pixels.

Disclosure of Invention

The embodiment of the application provides an image coding method, an image coding device, a chip, a terminal, a storage medium and a program product. The technical scheme is as follows:

in one aspect, an embodiment of the present application provides an image encoding method, including:

performing fractional interpolation operation on integer pixel points in a reference image frame based on asymmetric tap coefficients to obtain row fractional pixels and column fractional pixels, wherein the asymmetric tap coefficients comprise row tap coefficients and column tap coefficients, and the coefficient number of the row tap coefficients is different from the coefficient number of the column tap coefficients;

image encoding is performed based on the row fractional pixels and the column fractional pixels.

In another aspect, an embodiment of the present application provides an image encoding apparatus, including:

the fractional interpolation module is used for performing fractional interpolation operation on integer pixel points in the reference image frame based on asymmetric tap coefficients to obtain row fractional pixels and column fractional pixels, wherein the asymmetric tap coefficients comprise row tap coefficients and column tap coefficients, and the coefficient number of the row tap coefficients is different from the coefficient number of the column tap coefficients;

and the image coding module is used for carrying out image coding based on the row fraction pixels and the column fraction pixels.

In another aspect, embodiments of the present application provide a chip including programmable logic circuits and/or program instructions for implementing the data compression method as described in the above aspects when the chip is running.

In another aspect, an embodiment of the present application provides a terminal, where the terminal includes a processor and a memory, where the memory stores at least one section of program, and the at least one section of program is loaded and executed by the processor to implement the image encoding method as described in the above aspect.

In another aspect, embodiments of the present application provide a computer readable storage medium having at least one program code stored therein, the program code being loaded and executed by a processor to implement the image encoding method as described in the above aspect.

In another aspect, embodiments of the present application provide a computer program product comprising computer instructions stored in a computer-readable storage medium. The processor of the terminal reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the terminal performs the image encoding method provided in various alternative implementations of the above aspect.

The technical scheme provided by the embodiment of the application can bring the following beneficial effects:

by setting the asymmetric tap coefficients, the asymmetric tap coefficients include a row tap coefficient and a column tap coefficient, and the number of coefficients of the row tap coefficient is different from the number of coefficients of the column tap coefficient. Compared with the prior art that the same tap coefficient is adopted for the row and column to carry out fractional interpolation operation, the interpolation operation amount and operation complexity in the process of obtaining row fractional pixels or column fractional pixels can be reduced by setting the asymmetric tap coefficient, so that the chip power consumption of the video coding chip is reduced.

Drawings

FIG. 1 is a schematic diagram of the relationship between integer pixel points and fractional pixel points;

FIG. 2 illustrates a flow chart of an image encoding method illustrated in an exemplary embodiment of the present application;

FIG. 3 illustrates a flow chart of an image encoding method illustrated in an exemplary embodiment of the present application;

FIG. 4 illustrates an interpolation process of fractional pixels as illustrated in an exemplary embodiment of the present application;

FIG. 5 illustrates a power consumption comparison diagram of two interpolation methods according to an exemplary embodiment of the present application;

FIG. 6 illustrates a flowchart of an image encoding method illustrated in an exemplary embodiment of the present application;

FIG. 7 illustrates a flowchart of an image encoding method illustrated in an exemplary embodiment of the present application;

FIG. 8 illustrates a fractional pixel motion estimation architecture diagram shown in an exemplary embodiment of the present application;

fig. 9 is a block diagram showing the structure of an image encoding apparatus according to an embodiment of the present application;

fig. 10 is a block diagram illustrating a structure of a terminal according to an exemplary embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

References herein to "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.

In the video coding process, motion estimation is a very important technology in the HEVC video coding standard, and can effectively improve the performance of an encoder. Motion estimation is the process of finding the best reference block from the encoded reference frame by integer-pixel and fractional-pixel search. For most video, the content between adjacent frames is quite similar, and a reference block that is highly similar to the current encoded block can be found by motion estimation. Since the movement of the object is continuous, but the pixels themselves are discrete, this presents a problem: the best reference block of the block to be encoded is not necessarily located at the integer pixel point position of the reference frame. Then motion estimation is required at the fractional position for more accurate prediction of the current image block to be encoded, but the image fractional pixel position itself is not sampled, so that interpolation operation is required for the reference block before fractional pixel search, thereby generating the fractional pixels. Schematically, the relationship between integer pixels and fractional pixels is shown in fig. 1. 15 fractional pixel points A0 corresponding to the integer pixel points A0,0, b0, c0, d0, e0, f0,0, g0, h0, i0, j0, k0,0, n0, p0, q0,0 and r0,0.

In the interpolation to obtain fractional pixels, there are (luminance) tap coefficients set as shown in table one. The fractional pixel points a0,0, b0,0, c0,0 in the horizontal direction can be obtained by interpolating the 8 integer pixel points in the row where the integer pixel is located. The fractional pixel points d0, e0, f0 in the vertical direction can be obtained by interpolating 8 integer pixel points of the column in which the integer pixel is located.

List one

index	-3	-2	-1	0	1	2	3	4
									1/4	-1	4	-10	58	17	-5	1	0
1/2	-1	4	-11	40	40	-11	4	-1
									3/4	0	1	-5	17	58	-10	4	-1

As can be seen from table one, in the case where the 8 tap coefficients are used for both rows and columns, the calculation process at each fractional pixel position is as follows:

1/4 pixel = [ a6-a0-a5+a4+32 (a3+1) +8 (a3-a2) +4 (a1-a5) +2 (a3-a2) +16 (a3+a4) ] >6 (1)

2/4 pixel = [32 (a3+a4+1) +4[ (a1+a6) + (a2+a5) ] +8 (a3+a4) -16 (a2+a5) - (a0+a7) + (a2+a5) ] >6 (2)

3/4 pixel = [ a1-a7-a2+a3+32 (a4+1) +8 (a4-a5) +4 (a6-a2+2 (a4-a5) +16 (a3+a4) ] >6 (3)

Where a0 represents an integer pixel point where index is-3, a1 represents an integer pixel point where index is-2, a2 represents an integer pixel point where index is-1, a3 represents an integer pixel point where index is 0, a4 represents an integer pixel point where index is 1, a5 represents an integer pixel point where index is 2, a6 represents an integer pixel point where index is 3, and a7 represents a pixel point where index is 4. As shown above, to interpolate 15 fractional pixels, 85 shifters, 100 adders, and 50 subtractors are required. The computational complexity is high, and for the first d0,0 of interpolation, for example, it takes 8 clock cycles to load 8 rows of integer pixels before each interpolation, which is extremely unfriendly to chip performance and power consumption.

In order to improve the problems caused in the fractional interpolation process of the related art, the embodiment of the application provides a method for asymmetric tap coefficients, so as to reduce the computational complexity in the fractional interpolation process. Referring to fig. 2, a flowchart of an image encoding method according to an exemplary embodiment of the present application is shown, where the embodiment of the present application is described by taking application of the method to a terminal as an example, and the method includes:

in step 201, fractional interpolation is performed on integer pixel points in a reference image frame based on asymmetric tap coefficients, so as to obtain row fractional pixels and column fractional pixels, wherein the asymmetric tap coefficients comprise row tap coefficients and column tap coefficients, and the number of coefficients of the row tap coefficients is different from the number of coefficients of the column tap coefficients.

Unlike the prior art in which the same number of tap coefficients are used for fractional interpolation in each row and column, for example, fractional interpolation is performed in each sub-row and column by using 8 tap coefficients, which results in complex fractional interpolation operations for each row and column, an asymmetric tap coefficient is provided in the embodiment of the present application, where the asymmetric tap coefficient includes a row tap coefficient and a column tap coefficient, and the number of coefficients of the row tap coefficient is different from the number of coefficients of the column tap coefficient. That is, in the fractional interpolation processing of integer pixel points using asymmetric tap coefficients, the fractional interpolation processing is performed using row tap coefficients when row fractional pixels are acquired, the interpolation processing is performed using column tap coefficients when column fractional pixels are acquired, and the numbers of coefficients of tap coefficients used for the row fractional pixels and the column fractional pixels are different.

Illustratively, the combination of asymmetric tap coefficients may be 6+4, i.e., the row tap coefficients are 6 tap coefficients and the column tap coefficients are 4 tap coefficients; alternatively, the combination of the asymmetric tap coefficients may be 8+4, that is, the row tap coefficient is an 8 tap coefficient and the column tap coefficient is a 4 tap coefficient; alternatively, the combination of asymmetric tap coefficients may be 4+6, i.e., the row tap coefficients are 4 tap coefficients and the column tap coefficients are 6 tap coefficients. The number of N tap coefficients, i.e., the number of coefficients of the tap coefficients is N, i.e., the number of coefficients of the tap coefficients corresponding to the row of 6 tap coefficients is 6, and the number of coefficients of the tap coefficients corresponding to the column of 4 tap coefficients is 4. The present embodiment is not limited to the specific number of coefficients of the specific asymmetric tap coefficient.

Optionally, the terminal is preset with an asymmetric tap coefficient that needs to be used in the image encoding or video encoding process, and in the image encoding or video encoding process, the terminal may perform fractional interpolation operation on the integer pixel point in the reference image frame based on the asymmetric tap coefficient after determining the integer pixel point in the reference image frame during motion estimation, so as to obtain a row fractional pixel and a column fractional pixel at the position of the integer pixel point. Because the row fractional pixels and the column fractional pixels are obtained by fractional interpolation by adopting tap coefficients with different coefficient numbers, compared with the situation that the row fractional pixels and the column fractional pixels are obtained by fractional interpolation by adopting tap coefficients with the same coefficient number, the interpolation operation amount and the operation complexity of the row fractional pixels or the column fractional pixels can be reduced, so that the chip power consumption of a video coding chip is reduced.

Step 202, image encoding is performed based on the row fractional pixels and the column fractional pixels.

And after the fractional interpolation is finished on the integer pixel points in the reference image frame, obtaining row fractional pixels and column fractional pixels, wherein the terminal can perform fractional pixel motion estimation based on the row fractional pixels and the column fractional pixels so as to determine the optimal fractional pixel points of the current frame to be coded in the reference image frame, and further execute the subsequent image coding process or video coding process.

In summary, in the embodiment of the present application, by setting the asymmetric tap coefficients, the asymmetric tap coefficients include the row tap coefficients and the column tap coefficients, and the number of coefficients of the row tap coefficients is different from the number of coefficients of the column tap coefficients. Compared with the prior art that the same tap coefficient is adopted for the row and column to carry out fractional interpolation operation, the interpolation operation amount and operation complexity in the process of obtaining row fractional pixels or column fractional pixels can be reduced by setting the asymmetric tap coefficient, so that the chip power consumption of the video coding chip is reduced.

Taking fig. 1 as an example, when the column fractional pixels are obtained by interpolation, a part of the column fractional pixels are obtained by interpolation of the row fractional pixels, for example, when e0,0 is calculated, interpolation is needed according to the fractional pixel points of the column where e0,0 is located, that is, e0,0 depends on the already interpolated row fractional pixel points a0,0, and the interpolation accuracy of the row fractional pixels a0,0 affects the interpolation accuracy of a part of the column fractional pixels. Therefore, in order to reduce interpolation operation and ensure accuracy of fractional pixel motion estimation to the maximum extent, when asymmetric tap coefficients are set, the number of coefficients of row tap coefficients in the asymmetric tap coefficients is set to be larger than the number of coefficients of column tap coefficients.

Referring to fig. 3, a flowchart of an image encoding method according to an exemplary embodiment of the present application is shown, where the embodiment of the present application is described by taking application of the method to a terminal as an example, and the method includes:

step 301, performing row fraction pixel interpolation operation on the integer pixel points based on the row tap coefficients to obtain row fraction pixels.

Wherein the asymmetric tap coefficients comprise row tap coefficients and column tap coefficients, and the number of coefficients of the row tap coefficients is greater than the number of coefficients of the column tap coefficients. Illustratively, the asymmetric tap coefficients may be 6+4, i.e. the row tap coefficients are 6 taps (the number of coefficients for the row tap coefficients is 6) and the column tap coefficients are 4 taps (the number of coefficients for the column tap coefficients is 4). That is, 6 integer pixels are required to participate in the operation when calculating the row fractional pixels, while only 4 integer pixels are required to participate in the operation when calculating the column tap coefficients.

As shown in tables two and three, it shows the row tap coefficient corresponding to the row fractional pixel (table two) and the column tap coefficient corresponding to the column fractional pixel (table three) in the case where the asymmetric tap coefficient is 6+4.

Watch II

index	-3	-2	-1	0	1	2	3	4
									1/4	0	3	-10	58	17	-5	1	0
1/2	0	3	-11	40	40	-11	3	0
									3/4	0	1	-5	17	58	-10	3	0

Watch III

index	-3	-2	-1	0	1	2	3	4
									1/4	0	0	-6	58	16	-4	0	0
1/2	0	0	-8	40	40	-8	0	0
									3/4	0	0	-4	16	58	-6	0	0

As is clear from tables two and three, the number of coefficients of the line tap coefficients (the thickened portion in table two) is 6, and the number of coefficients of the column tap coefficients (the thickened portion in table three) is 4. In addition, the sum of 6 tap coefficients is 2≡6, and the sum of 4 tap coefficients is also 2≡6.

Since the number of coefficients of the row tap coefficients and the number of coefficients of the column tap coefficients included in the asymmetric tap coefficients are different, the corresponding row tap coefficients and column tap coefficients are also different. Therefore, when the terminal performs fractional interpolation operation on integer pixel points in the reference image frame, it is required to acquire row tap coefficients in the asymmetric tap coefficients respectively, perform fractional pixel interpolation operation on the integer pixel points by using the row tap coefficients to obtain row fractional pixels, and acquire column tap coefficients in the asymmetric tap coefficients, and perform fractional pixel interpolation operation on the integer pixel points by using the column tap coefficients to obtain column fractional pixels.

When the fractional interpolation operation is performed on the integer pixel points in the reference image frame, not only the current integer pixel point but also the adjacent integer pixel points of the integer pixel points are needed. In an illustrative example, step 301 may include step 301A and step 301B.

In step 301A, a first integer pixel set is obtained from a line where integer pixel points in a reference image frame are located based on the number of coefficients of the line tap coefficients, where the number of integer pixel points in the first integer pixel set is the same as the number of coefficients.

In step 301B, row fractional pixels are determined based on the row tap coefficients and the first set of integer pixels.

Since the fractional interpolation operation is performed according to the tap coefficients and other adjacent integer pixel points in the line where the integer pixel points are located, the number of the integer pixel points in the line where the selected integer pixel points are located depends on the number of the tap coefficients in the line. Therefore, in one possible implementation manner, the terminal needs to obtain, according to the number of coefficients of the line tap coefficients, a first integer pixel set from the line where the integer pixel points in the reference image frame are located, where the number of integer pixel points in the first integer pixel set is the same as the number of coefficients; and further performing fractional interpolation operation based on the first integer pixel set and the line tap coefficients to obtain line fractional pixels.

Illustratively, if the interpolation A0,0 is used and the line tap coefficient is 6 taps, then 6 integer pixel points of the line where A0,0 is located need to be obtained, and the corresponding first integer pixel set is a-2,0, a-1,0, A0,0, A1,0, A2,0 and A3,0.

Optionally, there is also a difference in the line tap coefficients from different fractional pixel positions for the line tap coefficients. When the fractional pixel interpolation operation of the row fractional pixel is performed, the row tap coefficient at the corresponding fractional pixel position is selected to perform the interpolation operation according to the fractional pixel position of the row fractional pixel point. Step 301B may also include steps 301B 1-301B 4, as corresponds to one illustrative example.

Step 301B1 determines a first sub-tap coefficient for a first fractional pixel position, a second sub-tap coefficient for a second fractional pixel position, and a third sub-tap coefficient for a third fractional pixel position from the row tap coefficients.

The pixel positions of the row fractional pixels at least comprise a first fractional pixel position, a second fractional pixel position and a third fractional pixel position, the horizontal distance between the first fractional pixel position and the integer pixel point is smaller than that between the second fractional pixel position and the integer pixel point, and the horizontal distance between the second fractional pixel position and the integer pixel point is smaller than that between the third fractional pixel position and the integer pixel point. Alternatively, the fractional pixels at the first fractional pixel location may be referred to as 1/4 fractional pixels, the fractional pixels at the second fractional pixel location may be referred to as 2/4 fractional pixels (or 1/2 fractional pixels), and the fractional pixels at the third fractional pixel location may be referred to as 3/4 fractional pixels.

As shown in fig. 1, for row fractional pixels a0, b0, and c0, a0,0 is 1/4 fractional pixel, b0,0 is 1/2 fractional pixel, c0,0 is 3/4 fractional pixel.

The row tap coefficients include a first sub-tap coefficient for a first fractional pixel location, a second sub-tap coefficient for a second fractional pixel location, and a third sub-tap coefficient for a third fractional pixel location corresponding to the different fractional pixel locations. When the corresponding terminal performs the row fractional pixel interpolation operation, the fractional pixel interpolation operation is required to be performed on the first integer pixel set based on a first sub-tap coefficient of a first fractional pixel position in the row tap coefficients, a second sub-tap coefficient of a second fractional pixel position, and a third sub-tap coefficient of a third fractional pixel position.

Step 301B2, determining a row fractional pixel at a first fractional pixel location based on the first sub-tap coefficient and the first set of integer pixels.

For a row fractional pixel at the first fractional pixel location (i.e., for 1/4 of the fractional pixels in the row fractional pixel), the terminal needs to interpolate the row fractional pixel at the first fractional pixel location based on the first sub-tap coefficient and the first set of integer pixels.

Illustratively, taking the example of the line tap coefficient being 6 taps, the interpolation process of 1/4 fractional pixel in the line fractional pixel is shown in formula (4) or formula (5).

1/4 fractional pixel = 3a1-10a2+58a3+17a4-5a5+a6> >6 (4)

Or (b)

1/4 fractional pixel = a3+ [16a4+4 (a1-a5) -8 (a3+a2) -2 (a2-a3) - (a1+a5) +a4+a6+32] > >6 (5)

Wherein a1 represents an integer pixel with an index of-2, a2 represents an integer pixel with an index of-1, a3 represents an integer pixel with an index of 0, a4 represents an integer pixel with an index of 1, a5 represents an integer pixel with an index of 2, a6 represents an integer pixel with an index of 3, and >6 represents a reduction of 2≡6 times.

Step 301B3 determines a row fractional pixel at a second fractional pixel location based on the second sub-tap coefficient and a set of integer pixels.

For a row fractional pixel at the second fractional pixel location (i.e., for 1/2 of the fractional pixels in the row fractional pixel), the terminal needs to interpolate the row fractional pixel at the second fractional pixel location based on the second sub-tap coefficient and the first set of integer pixels.

Illustratively, taking the example of the line tap coefficient being 6 taps, the interpolation process of 1/2 fraction pixel or 2/4 fraction pixel in the line fraction pixel is shown in formula (6) or formula (7).

2/4 fractional pixel = 3a1-11a2+40a3+40a4-11a5+3a6> >6 (6)

Or alternatively, the first and second heat exchangers may be,

2/4 pixel= [32 (a3+a4+1) +4 (a1+a6) - (a1+a6) +8[ (a3+a4) - (a2+a5) ] - [2 (a2+a5) + (a2+a5) ] ] ] > >6 (7)

Wherein, the meanings of a1 to a6 are similar to the formula (4) and the formula (5), and are not repeated here.

Step 301B4 determines a row fractional pixel at a third fractional pixel location based on the third sub-tap coefficient and the first set of integer pixels.

For the row fractional pixels at the third fractional pixel location (i.e., for 3/4 of the row fractional pixels), the terminal needs to interpolate the row fractional pixels at the third fractional pixel location based on the third sub-tap coefficient and the first set of integer pixels.

Illustratively, taking the example of the line tap coefficient being 6 taps, the interpolation process of 3/4 fractional pixels in the line fractional pixels is shown in formula (8) or formula (9).

3/4 pixel = a1-5a2+17a3+58a4-10a5+3a6> >6 (8)

Or (b)

3/4 pixel = a4+ [16a3+4 (a6-a2) -8 (a4+a5) -2 (a5-a4) - (a6+a2) +a3+a1+32] > >6 (9)

Since the line tap coefficient has only 6 taps, integer pixel points with indexes a0 and a7 are not involved in the interpolation operation of the line fractional pixels, compared with the scheme in the related art. The 3 fractional pixels corresponding to the row fractional pixels only need to use 15 shifters, 21 adders and 13 subtractors, so that the operation amount in the interpolation operation process is reduced.

Referring to fig. 4, an interpolation process of fractional pixels is shown in an exemplary embodiment of the present application. If the integer pixel points determined from the reference image frame are A0,0, when the line fractional pixel interpolation is performed on the A0,0 by using the 6 tap coefficients, firstly, 6 integer pixel points of the line where the A0,0 is located are obtained: a-2,0, a-1,0, A0, A1,0, A2,0 and A3,0, respectively calculating to obtain A0, b0,0 and c0,0 according to the line tap coefficient and 6 integer pixel points; optionally, since a0,0 is a 1/4 fractional pixel position, fractional interpolation operation is required to be performed based on the line tap coefficient at the 1/4 fractional pixel position in the line tap coefficients and 6 integer pixel points, so as to obtain fractional pixel points a0,0; b0,0 is a 2/4 fractional pixel position, then fractional interpolation operation is required to be performed based on the line tap coefficient at the 2/4 fractional pixel position in the line tap coefficients and 6 integer pixel points to obtain fractional pixel points b0,0; c0,0 is the 3/4 fractional pixel position, then fractional interpolation operation is required based on the line tap coefficient at the 3/4 fractional pixel position in the line tap coefficient and 6 integer pixel points to obtain the fractional pixel points c0,0.

Step 302, performing column fractional pixel interpolation operation on the integer pixel points based on the column tap coefficients to obtain column fractional pixels, wherein the number of coefficients of the row tap coefficients is greater than that of the column tap coefficients.

Taking fig. 1 as an example, taking a column tap coefficient as 4 taps as an example, when interpolating fractional pixels e0,0, interpolation operation needs to be performed based on pixel points A0, -1, A0,0, A0,1 and A0,2 of the column where e0,0 is located, while A0, -1, A0,0, A0,1 and A0,2 need to be performed based on integer pixel points a-2,0, a-1,0, A0,0, A1,0 and A2,0 of the row where A0,0 is located, taking interpolation A0,0 and A0 as an example, and interpolation operation needs to be performed based on integer pixel points a-2,0, a-1,0, A2,0 of the row where A0,0 is located, that is, the column fractional pixel is obtained by interpolation of the row fractional pixel, the interpolation accuracy of the column fractional pixel depends on the accuracy of the row fractional pixel, and the interpolation operation process of the row fractional pixel has the greatest influence on the motion estimation accuracy of the corresponding fractional pixel. In order to reduce the interpolation operation amount and reduce the accuracy of the fractional pixel motion estimation as much as possible, in this embodiment, the coefficient number of the row tap coefficients is set to be larger than the coefficient number of the column tap coefficients, so that when the terminal performs interpolation operation on the row fractional pixels, more integer pixel points can be used as much as possible to participate in the interpolation operation, so as to improve the accuracy of the calculated row fractional pixels, and further improve the accuracy of the column fractional pixels calculated based on the row fractional pixels; in addition, in the process of reading the pixel points of the reference image frame, the row pixel points are read first, and then the column pixel points are read, so that even if the coefficient number of the row tap coefficients is more in the interpolation operation process, the integer pixel points needing to participate in the operation can be read in the same clock period, and the waiting time is not increased; the number of the coefficients of the column tap coefficients is small, so that the time length for waiting to load N rows of integer pixel points can be reduced when the column fraction pixels are calculated, and the interpolation operation efficiency and the performance of a coding chip are improved; and the interpolation operation of the column fraction pixels is accurately dependent on the interpolation operation accuracy of the row fraction pixels, and the accuracy of the column fraction pixels can be only possibly ensured on the premise that the number of coefficients of the row tap coefficients is large.

Optionally, after testing coding performance and coding efficiency of the asymmetric tap coefficients of various combinations, an experimenter finds that when the asymmetric tap coefficient is 6+4 taps (i.e. the row tap coefficient is 6 taps and the column tap coefficient is 4 taps), the image coding quality can be ensured to the greatest extent under the condition of reducing the complexity of interpolation operation.

When the fractional interpolation operation is performed on the integer pixel points in the reference image frame, not only the current integer pixel point but also the adjacent integer pixel points of the integer pixel points are needed. In one illustrative example, step 302 may include steps 302A and 302B.

In step 302A, a second set of integer pixels is obtained from the columns of the integer pixel points in the reference image frame based on the number of coefficients of the column tap coefficients, where the number of integer pixel points in the second set of integer pixels is the same as the number of coefficients.

Step 302B, a column fractional pixel is determined based on the column tap coefficients and the second set of integer pixels.

The number of the integer pixel points in the column is determined by the number of the tap coefficients in the column, because the integer pixel points are calculated according to the tap coefficients and other adjacent integer pixel points in the column. Therefore, in one possible implementation manner, the terminal needs to obtain, according to the number of coefficients of the column tap coefficients, a second integer pixel set from the column where the integer pixel points in the reference image frame are located, where the number of integer pixel points in the second integer pixel set is the same as the number of coefficients; and further performing fractional interpolation operation based on the second integer pixel set and the column tap coefficients to obtain column fractional pixels.

Illustratively, if the interpolation d0,0 is used and the column tap coefficient is 4 taps, 4 integer pixel points of the column where A0,0 is located need to be obtained, and the corresponding second integer pixel sets are A0, -1, A0,0, A0,1 and A0,2.

Alternatively, there is also a difference in column tap coefficients at different fractional pixel positions for the column tap coefficients. When the fractional pixel interpolation operation of the column fractional pixel is performed, the column tap coefficient at the corresponding fractional pixel position is selected to perform the interpolation operation according to the fractional pixel position of the column fractional pixel point. In an illustrative example, step 302B may also include steps 302B 1-302B 4.

Step 302B1 determines a fourth sub-tap coefficient for a fourth fractional pixel location, a fifth sub-tap coefficient for a fifth fractional pixel location, and a sixth sub-tap coefficient for a sixth fractional pixel location from the column tap coefficients.

The pixel positions of the column fractional pixels at least comprise a fourth fractional pixel position, a fifth fractional pixel position and a sixth fractional pixel position, the vertical distance between the fourth fractional pixel position and the integer pixel point is smaller than the vertical distance between the fifth fractional pixel position and the integer pixel point, and the vertical distance between the fifth fractional pixel position and the integer pixel point is smaller than the vertical distance between the sixth fractional pixel position and the integer pixel point. Alternatively, the fractional pixel at the fourth fractional pixel position may be referred to as a 1/4 fractional pixel, the fractional pixel at the fifth fractional pixel position may be referred to as a 2/4 fractional pixel (or 1/2 fractional pixel), and the fractional pixel at the sixth fractional pixel position may be referred to as a 3/4 fractional pixel.

As shown in fig. 1, for column fractional pixels d0, h0,0 and nc0, d0,0 is 1/4 fractional pixel, h0,0 is 1/2 fractional pixel, n0,0 is 3/4 fractional pixel.

The column tap coefficients include a fourth sub-tap coefficient for a fourth fractional pixel location, a fifth sub-tap coefficient for a fifth fractional pixel location, and a sixth sub-tap coefficient for a sixth fractional pixel location, corresponding to different fractional pixel locations. When the corresponding terminal performs column fractional pixel interpolation operation, fractional pixel interpolation operation is required to be performed on the second integer pixel set based on a fourth sub-tap coefficient of a fourth fractional pixel position, a fifth sub-tap coefficient of a fifth fractional pixel position, and a sixth sub-tap coefficient of a sixth fractional pixel position in the column tap coefficients, respectively.

Step 302B2, determining a column fractional pixel at a fourth fractional pixel location based on the fourth sub-tap coefficient and the second set of integer pixels.

For column fractional pixels at the fourth fractional pixel location (i.e., for 1/4 of the fractional pixels of the column fractional pixels), the terminal needs to interpolate the column fractional pixels at the fourth fractional pixel location based on the fourth sub-tap coefficients and the second set of integer pixels.

Illustratively, taking a column tap coefficient of 4 taps as an example, the interpolation process of 1/4 fractional pixels in column fractional pixels is shown in formula (10) or formula (11).

1/4 fractional pixel = -6a2+58a3+16a4-4a5> >6 (10)

Or alternatively, the first and second heat exchangers may be,

1/4 pixel = a3+ [32+16a4-8 (a2+a3) -4a5+2 (a2+a3) ] >6 (11)

Where a2 represents an integer pixel with an index of-1, a3 represents an integer pixel with an index of 0, a4 represents an integer pixel with an index of 1, and a5 represents an integer pixel with an index of 2.

Step 302B3 determines a column fractional pixel at a fifth fractional pixel location based on the fifth sub-tap coefficient and the second set of integer pixels.

For column fractional pixels at the fifth fractional pixel location (i.e., for 1/2 of the fractional pixels in the column fractional pixels), the terminal needs to interpolate the column fractional pixels at the fifth fractional pixel location based on the fifth sub-tap coefficients and the second set of integer pixels.

Illustratively, taking a column tap coefficient of 4 taps as an example, the interpolation process of 1/2 fractional pixel or 2/4 fractional pixel in the column fractional pixel is shown in formula (12) or formula (13).

2/4 fractional pixel= -8a2+40a3+40a4-8a5> >6 (12)

Or alternatively, the first and second heat exchangers may be,

2/4 pixel = [32 (a3+a4+1) +8[ (a3+a4) - (a2+a5) ] ] > >6 (13)

Wherein, the meanings of a2 to a5 are similar to the formula (10) and the formula (11), and are not repeated here.

Step 302B4, determining a column fractional pixel at a sixth fractional pixel location based on the sixth sub-tap coefficient and the second set of integer pixels.

For column fractional pixels at the sixth fractional pixel location (i.e., for 3/4 of the row fractional pixels), the terminal needs to interpolate the row fractional pixels at the third fractional pixel location based on the third sub-tap coefficient and the first set of integer pixels.

Illustratively, taking a column tap coefficient of 4 taps as an example, the interpolation process of 3/4 fractional pixels in column fractional pixels is shown in formula (14) or formula (15).

3/4 pixel = -4a2+16a3+58a4-6a5> >6 (14)

Or (b)

3/4 pixel = a4+ [32+16a3-8 (a4+a5) -4a2+2 (a4+a5) ] >6 (15)

As can be seen from the interpolation process of the column fractional pixels, only 13 shifters, 11 adders and 5 subtractors are needed for the 3 column fractional pixels. In the process of 15 fractional pixels corresponding to the interpolated integer pixel points, 77 shifters, 65 adders and 30 subtractors are used in total. Compared with 15 fractional pixel points interpolated in the related art, 85 shifters, 100 adders and 50 subtractors are needed, the asymmetric tap coefficient provided by the method can save more interpolation operation.

As shown in fig. 4, a schematic diagram of fractional pixel interpolation is shown in accordance with an exemplary embodiment of the present application. If the integer pixel points determined from the reference image frame are A0,0, when the column fractional pixel interpolation is performed on the A0,0 by using the 4 tap coefficient, firstly, 4 integer pixel points of the column where the A0,0 is located are obtained: a0, -1, A0, A0,1 and A0,2, and further respectively calculating d0, h0,0 and n0,0 according to the column tap coefficient and 4 integer pixel points; optionally, since d0,0 is a 1/4 fractional pixel position, fractional interpolation operation is required to be performed based on the column tap coefficient at the 1/4 fractional pixel position in the column tap coefficients and the 4 integer pixel points to obtain fractional pixel points d0,0; h0,0 is a 2/4 fractional pixel position, then fractional interpolation operation is required to be performed based on the column tap coefficient at the 2/4 fractional pixel position in the column tap coefficients and the 4 integer pixel points to obtain fractional pixel points h0,0; n0,0 is a 3/4 fractional pixel position, then fractional interpolation operation is required based on the column tap coefficient at the 3/4 fractional pixel position and 4 integer pixel points in the column tap coefficient to obtain fractional pixel points n0,0. Alternatively, for the fractional pixel points e0,0, it is necessary to perform fractional interpolation according to the fractional pixel points in the column where e0,0 is located, for example, fractional interpolation is performed according to the fractional pixel points a0, -1, a0,0, a0,1 and a0, 2.

As shown in fig. 5, a power consumption comparison diagram of two interpolation methods according to an exemplary embodiment of the present application is shown. Taking calculation of the fractional pixel points e0 and 0 as an example, if the symmetrical tap coefficient 8+8 is adopted, calculating the fractional pixel points e0 and 0 requires waiting for loading of 8 lines of integer pixel points by a chip, wherein the related integer pixel point areas are areas corresponding to A-3, -3-A4, -3, A-3, -3-A-3, 4, A-3, 4-A4, 4 and A4, 4-A4, -3; under the condition of adopting the asymmetric tap coefficient 6+4 shown in the embodiment, calculating the fractional pixel points e0 and 0 only needs to wait for loading 4 lines of integer pixel points on a chip, and the related integer pixel point areas are areas corresponding to A-2, 1-A3, -1, A-2, 1-A-2, A-2, 2-A3, 2 and A3, 2-A3, -1, compared with the symmetric tap coefficient, the interpolation scheme adopting the '6+4' can reduce a large number of calculation logics and registers.

Step 303, performing image coding based on the row fraction pixels and the column fraction pixels.

The implementation of step 303 may refer to the above embodiments, which are not described herein.

In this embodiment, by setting the number of coefficients of the row tap coefficients to be greater than the number of coefficients of the column tap coefficients in the asymmetric tap coefficients, under the premise of performing fractional interpolation operation on the row tap coefficients, accuracy of fractional pixel estimation can be ensured to the maximum extent while reducing the amount of interpolation operation.

Although the asymmetric tap coefficient of 6+4 is a preferred choice for both interpolation and motion estimation accuracy, in some image coding scenarios, either one of the two may be more emphasized, for example, the motion estimation accuracy may be more emphasized, or the interpolation may be more emphasized and the interpolation may be reduced. In other possible embodiments, the required asymmetric tap coefficients may also be dynamically adjusted according to different requirements of the image coding scene (video coding scene).

Referring to fig. 6, a flowchart of an image encoding method according to an exemplary embodiment of the present application is shown, where the embodiment of the present application is described by taking application of the method to a terminal as an example, and the method includes:

in step 601, a quality requirement level for image coding quality in an image coding scene is determined.

The higher the quality demand level is, the more important the picture coding quality is in the picture coding scene; the lower the quality requirement level, the more emphasis may be placed on image coding efficiency in an image coding scene. Because the image coding requirements focused on different image coding scenes are different, the image coding quality may be focused more, and the image coding efficiency may also be focused more, in order to adapt to the image coding requirements on different image coding scenes, in this embodiment, asymmetric tap coefficients with different combination relationships may be preset in the terminal, so that in an actual image coding scene, an appropriate asymmetric tap coefficient may be selected according to the image coding requirements of the image coding scene.

Optionally, the terminal presets asymmetric tap coefficients corresponding to different quality requirement levels according to quality requirement levels of image coding quality in different image coding scenes.

Step 602, performing fractional interpolation operation on integer pixel points in the reference image frame based on the first asymmetric tap coefficient to obtain row fractional pixels and column fractional pixels when the quality requirement level is the first quality level.

Optionally, when the terminal determines that the quality requirement level in the image coding scene is the first quality level, the asymmetric tap coefficient corresponding to the quality requirement level matched with the first quality level may be obtained from the corresponding relationship (the corresponding relationship between the quality requirement level and the asymmetric tap coefficient) according to the first quality level, and used as the first asymmetric tap coefficient, and further, based on the first asymmetric tap coefficient, the fractional interpolation operation may be performed on the integer pixel point in the reference image frame, so as to obtain the row fractional pixel and the column fractional pixel.

Step 603, performing fractional interpolation operation on integer pixel points in the reference image frame based on the second asymmetric tap coefficient to obtain row fractional pixels and column fractional pixels when the quality requirement level is the second quality level.

Optionally, when the terminal determines that the quality requirement level in the image coding scene is the second quality level, the asymmetric tap coefficient corresponding to the quality requirement level matched with the second quality level may be obtained from the corresponding relationship (the corresponding relationship between the quality requirement level and the asymmetric tap coefficient) according to the second quality level, and used as the second asymmetric tap coefficient, and further, based on the second asymmetric tap coefficient, the fractional interpolation operation may be performed on the integer pixel point in the reference image frame, so as to obtain the row fractional pixel and the column fractional pixel.

The second quality level is lower than the first quality level, that is, the image coding scene corresponding to the first quality level has higher requirements on the image coding quality, so that in order to improve the image coding quality, the coefficient number of the line tap coefficients in the first asymmetric tap coefficient is greater than the coefficient number of the line tap coefficients in the second asymmetric tap coefficient. Illustratively, the first asymmetric tap coefficient may be 8+4 and the second asymmetric tap coefficient may be 6+4. So that the interpolation accuracy of the row fractional pixels and the column fractional pixels can be improved by increasing the number of coefficients of the row tap coefficients.

Optionally, for a scenario in which the second quality level is lower than the first quality level, the number of coefficients of the column tap coefficients in the first asymmetric tap coefficient may be set to be greater than the number of coefficients of the row tap coefficients in the second asymmetric tap coefficient; alternatively, the number of coefficients of the line tap coefficients in the first asymmetric tap coefficient may be set to be greater than the number of coefficients of the line tap coefficients in the second asymmetric tap coefficient, and the number of coefficients of the column tap coefficients in the first asymmetric tap coefficient may be set to be greater than the number of coefficients of the line tap coefficients in the second asymmetric tap coefficient.

Step 604, image encoding is performed based on the row fractional pixels and the column fractional pixels.

The implementation of step 604 may refer to the above embodiments, which are not described herein.

In this embodiment, by acquiring the quality requirement level of the image coding quality in the image coding scene to select the asymmetric tap coefficient matched with the quality requirement level, the asymmetric tap coefficient with higher interpolation accuracy can be used for the image coding scene with higher quality requirement level, so as to meet the requirements of different image coding scenes on the image coding quality.

In other possible embodiments, the required asymmetric tap coefficients may also be dynamically adjusted according to the rate requirement level for the image coding rate in the image coding scene. Referring to fig. 7, a flowchart of an image encoding method according to an exemplary embodiment of the present application is shown, where the embodiment of the present application is described by taking application of the method to a terminal as an example, and the method includes:

Step 701, determining a rate requirement level for an image coding rate in an image coding scene.

In the fractional interpolation operation process, when the column fractional pixel interpolation operation is performed, the waiting time is determined by the coefficient number of the column tap coefficients, and the waiting time can influence the image coding rate of the coding chip. The image coding requirements that are emphasized in different image coding scenarios may be different, may emphasize image coding quality more, and may emphasize image coding rate more. Therefore, in order to adapt to the requirements of the image coding rate in different image coding scenes, in this embodiment, asymmetric tap coefficients corresponding to different rate requirement levels are preset in the terminal according to the rate requirement levels of the image coding rate in different image coding scenes.

Step 702, performing fractional interpolation operation on integer pixel points in the reference image frame based on the third asymmetric tap coefficient to obtain row fractional pixels and column fractional pixels when the rate requirement level is the first rate level.

Optionally, after determining a rate level requirement for an image coding rate in the image coding scene, the terminal obtains a relationship between the rate requirement level and an asymmetric tap coefficient, determines an asymmetric tap coefficient corresponding to the first rate level as a third asymmetric tap coefficient under the condition that the rate requirement level is determined to be the first rate level, and performs fractional interpolation operation on integer pixel points in the reference image frame according to the third asymmetric tap coefficient to obtain row fractional pixels and column fractional pixels.

In step 703, in the case that the rate requirement level is the second rate level, fractional interpolation is performed on the integer pixel points in the reference image frame based on the fourth asymmetric tap coefficient, so as to obtain row fractional pixels and column fractional pixels.

Optionally, after determining a rate level requirement for an image coding rate in the image coding scene, the terminal obtains a relationship between the rate requirement level and an asymmetric tap coefficient, determines an asymmetric tap coefficient corresponding to the second rate level as a fourth asymmetric tap coefficient under the condition that the rate requirement level is determined to be the second rate level, and performs fractional interpolation operation on integer pixel points in the reference image frame according to the fourth asymmetric tap coefficient to obtain row fractional pixels and column fractional pixels.

The first rate level is greater than the second rate level, that is, the image coding scene corresponding to the first rate level has a higher requirement on the image coding rate, so that in order to improve the image coding rate, the number of coefficients of the line tap coefficients in the third asymmetric tap coefficient is smaller than the number of coefficients of the line tap coefficients in the fourth asymmetric tap coefficient. Illustratively, the third asymmetric tap coefficient may be 8+4 and the second asymmetric tap coefficient may be 8+6. The interpolation waiting time of the column fraction pixels can be reduced by reducing the coefficient number of the column tap coefficients, and the operation speed of the fraction interpolation operation is further improved.

Step 704, image encoding is performed based on the row fractional pixels and the column fractional pixels.

The implementation of step 704 may refer to the above embodiments, which are not described herein.

In this embodiment, by acquiring the rate requirement level of the image coding rate in the image coding scene to select the asymmetric tap coefficient matched with the rate requirement level, the asymmetric tap coefficient with higher interpolation rate can be used for the image coding scene with higher rate requirement level, so as to meet the requirements of different image coding scenes on the image coding rate.

Referring to fig. 8, a diagram of a fractional pixel motion estimation architecture according to an exemplary embodiment of the present application is shown. The fractional pixel motion estimation architecture diagram includes a fractional pixel motion estimation control module 801, a fractional pixel motion estimation module 802, and a data buffer module 803. After the integer pixel motion estimation determines the integer pixel points in the reference image frame, fractional pixel interpolation can be performed on the integer pixel points according to the asymmetric tap coefficients. First, the fractional pixel motion estimation control module 801 controls to perform row pixel interpolation on integer pixel points using 6 tap coefficients, and then performs column pixel interpolation on integer pixel points using 4 tap coefficients. After the fractional pixel points corresponding to the integer pixel points are obtained, the fractional pixel motion estimation calculation module 802 can be controlled to perform fractional pixel motion estimation so as to determine the optimal fractional pixel point in the reference image frame. Optionally, the data buffering module 803 is configured to buffer integer pixel points involved in the fractional pixel motion estimation process.

Referring to fig. 9, a block diagram of an image encoding apparatus according to an embodiment of the present application is shown. The device comprises:

the fractional interpolation module 901 is configured to perform fractional interpolation operation on integer pixel points in a reference image frame based on asymmetric tap coefficients, so as to obtain row fractional pixels and column fractional pixels, where the asymmetric tap coefficients include row tap coefficients and column tap coefficients, and the number of coefficients of the row tap coefficients is different from the number of coefficients of the column tap coefficients;

an image encoding module 902, configured to perform image encoding based on the row fractional pixels and the column fractional pixels.

Optionally, the fractional interpolation module 901 is further configured to:

performing row fraction pixel interpolation operation on the integer pixel points based on the row tap coefficients to obtain the row fraction pixels;

and performing column fractional pixel interpolation operation on the integer pixel points based on the column tap coefficients to obtain column fractional pixels, wherein the coefficient number of the row tap coefficients is larger than that of the column tap coefficients.

Optionally, the number of coefficients of the row tap coefficients is 6, and the number of coefficients of the column tap coefficients is 4.

Optionally, the fractional interpolation module 901 is further configured to:

acquiring a first integer pixel set from a row where the integer pixel points are located in the reference image frame based on the coefficient number of the row tap coefficients, wherein the number of the integer pixel points in the first integer pixel set is the same as the coefficient number;

the row fractional pixels are determined based on the row tap coefficients and the first set of integer pixels.

Optionally, the pixel positions of the row fractional pixels at least include a first fractional pixel position, a second fractional pixel position and a third fractional pixel position, the horizontal distance between the first fractional pixel position and the integer pixel point is smaller than the horizontal distance between the second fractional pixel position and the integer pixel point, and the horizontal distance between the second fractional pixel position and the integer pixel point is smaller than the horizontal distance between the third fractional pixel position and the integer pixel point;

the fractional interpolation module 901 is further configured to:

determining a first sub-tap coefficient for the first fractional pixel location, a second sub-tap coefficient for the second fractional pixel location, and a third sub-tap coefficient for the third fractional pixel location from the row tap coefficients;

Determining a row fractional pixel at the first fractional pixel location based on the first sub-tap coefficient and the first set of integer pixels;

determining a row fractional pixel at the second fractional pixel location based on the second sub-tap coefficient and the first set of integer pixels;

a row fractional pixel at the third fractional pixel location is determined based on the third sub-tap coefficient and the first set of integer pixels.

Optionally, the fractional interpolation module 901 is further configured to:

acquiring a second integer pixel set from the column of the integer pixel points in the reference image frame based on the coefficient number of the column tap coefficients, wherein the number of the integer pixel points in the second integer pixel set is the same as the coefficient number;

the column fractional pixel is determined based on the column tap coefficients and the second set of integer pixels.

Optionally, the pixel positions of the column fractional pixels include at least a fourth fractional pixel position, a fifth fractional pixel position, and a sixth fractional pixel position, a vertical distance between the fourth fractional pixel position and the integer pixel point is smaller than a vertical distance between the fifth fractional pixel position and the integer pixel point, and a vertical distance between the fifth fractional pixel position and the integer pixel point is smaller than a vertical distance between the sixth fractional pixel position and the integer pixel point;

The fractional interpolation module 901 is further configured to:

determining a fourth sub-tap coefficient for the fourth fractional pixel location, a fifth sub-tap coefficient for the fifth fractional pixel location, and a sixth sub-tap coefficient for the sixth fractional pixel location from the column tap coefficients;

determining a column fractional pixel at the fourth fractional pixel location based on the fourth sub-tap coefficient and the second set of integer pixels;

determining a column fractional pixel at the fifth fractional pixel location based on the fifth sub-tap coefficient and the second set of integer pixels;

a column fractional pixel at the sixth fractional pixel location is determined based on the sixth sub-tap coefficient and the second set of integer pixels.

Optionally, the fractional interpolation module 901 is further configured to include:

determining a quality requirement level of image coding quality in an image coding scene;

under the condition that the quality requirement level is a first quality level, performing fractional interpolation operation on the integer pixel points in the reference image frame based on a first asymmetric tap coefficient to obtain the row fractional pixels and the column fractional pixels;

performing fractional interpolation operation on the integer pixel points in the reference image frame based on a second asymmetric tap coefficient under the condition that the quality requirement level is a second quality level to obtain the row fractional pixels and the column fractional pixels;

Wherein the second quality level is lower than the first quality level, and the number of coefficients of the line tap coefficients in the first asymmetric tap coefficient is greater than the number of coefficients of the line tap coefficients in the second asymmetric tap coefficient.

Optionally, the fractional interpolation module 901 is further configured to:

determining a rate requirement level of an image coding rate in an image coding scene;

performing fractional interpolation operation on the integer pixel points in the reference image frame based on a third asymmetric tap coefficient under the condition that the rate requirement level is the first rate level to obtain the row fractional pixels and the column fractional pixels;

performing fractional interpolation operation on the integer pixel points in the reference image frame based on a fourth asymmetric tap coefficient under the condition that the rate requirement level is a second rate level to obtain the row fractional pixels and the column fractional pixels;

wherein the first rate level is greater than the second rate level and the number of coefficients of the line tap coefficients in the third asymmetric tap coefficient is less than the number of coefficients of the line tap coefficients in the fourth asymmetric tap coefficient.

Referring to fig. 10, a block diagram of a terminal 1000 according to an exemplary embodiment of the present application is shown. Terminal 1000 can include one or more of the following components: a processor 1010 and a memory 1020.

Processor 1010 may include one or more processing cores. Processor 1010 utilizes various interfaces and lines to connect various portions of terminal 1000 in order to perform various functions of terminal 1000 and to process data by executing or executing instructions, programs, code sets, or instruction sets stored in memory 1020, and invoking data stored in memory 1020. Alternatively, the processor 1010 may be implemented in hardware in at least one of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 1010 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 1010 and may be implemented solely by a baseband chip.

The Memory 1020 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (ROM). Optionally, the memory 1020 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). Memory 1020 may be used to store instructions, programs, code, sets of codes, or instruction sets. The memory 1020 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, which may be an Android (Android) system (including a system developed based on an Android system), an IOS system developed by apple corporation (including a system developed based on an IOS system depth), or other systems, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, and the like. The storage data area may also store data created by terminal 1000 in use (e.g., phonebook, audiovisual data, chat log data), and the like.

Optionally, terminal 1000 can further include a video encoding chip, where the video encoding chip can be located on the same chip as the processor or be a different chip assembly. The video coding chip is used for video coding of video frames or image frames.

In addition, those skilled in the art will appreciate that the configuration of terminal 1000 shown in the above-described figures does not constitute a limitation of terminal 1000, and that an electronic device may include more or less components than those illustrated, or may combine certain components, or may have a different arrangement of components. For example, terminal 1000 can further include radio frequency circuitry, a camera module, a sensor, an audio circuit, a wireless fidelity (Wireless Fidelity, wiFi) module, a power supply, a bluetooth module, etc., which are not described herein.

Embodiments of the present application also provide a chip including programmable logic circuits and/or program instructions for implementing the image encoding method as described in the above aspects when the chip is operated.

The present application also provides a computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which are loaded and executed by a processor to implement the image encoding method provided by any of the above-described exemplary embodiments.

Embodiments of the present application provide a computer program product comprising computer instructions stored in a computer-readable storage medium. The processor of the terminal reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the terminal performs the image encoding method provided in the above-described alternative implementation.

Those skilled in the art will appreciate that in one or more of the examples described above, the functions described in the embodiments of the present application may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, these functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

The foregoing description of the preferred embodiments is merely exemplary in nature and is in no way intended to limit the invention, since it is intended that all modifications, equivalents, improvements, etc. that fall within the spirit and scope of the invention.

Claims

1. An image encoding method, the method comprising:

2. The method according to claim 1, wherein the performing fractional interpolation on integer pixel points in the reference image frame based on the asymmetric tap coefficients to obtain row fractional pixels and column fractional pixels includes:

3. The method of claim 2, wherein the number of coefficients for the row tap coefficients is 6 and the number of coefficients for the column tap coefficients is 4.

4. The method of claim 2, wherein performing a row fractional pixel interpolation operation on the integer pixel points based on the row tap coefficients to obtain the row fractional pixels comprises:

5. The method of claim 4, wherein the pixel locations of the row fractional pixels comprise at least a first fractional pixel location, a second fractional pixel location, and a third fractional pixel location, the horizontal distance of the first fractional pixel location from the integer pixel point is less than the horizontal distance of the second fractional pixel location from the integer pixel point, and the horizontal distance of the second fractional pixel location from the integer pixel point is less than the horizontal distance of the third fractional pixel location from the integer pixel point;

the determining the row fractional pixel based on the row tap coefficients and the first set of integer pixels includes:

6. The method according to claim 2, wherein performing column fractional pixel interpolation on the integer pixel points based on the column tap coefficients to obtain the column fractional pixels comprises:

7. The method of claim 6, wherein the pixel locations of the column fractional pixels comprise at least a fourth fractional pixel location, a fifth fractional pixel location, and a sixth fractional pixel location, the fourth fractional pixel location being at a smaller vertical distance from the integer pixel point than the fifth fractional pixel location, and the fifth fractional pixel location being at a smaller vertical distance from the integer pixel point than the sixth fractional pixel location;

The determining the column fractional pixel based on the column tap coefficients and the second set of integer pixels comprises:

8. The method according to any one of claims 1 to 7, wherein performing fractional interpolation on integer pixel points in the reference image frame based on the asymmetric tap coefficients to obtain row fractional pixels and column fractional pixels includes:

9. The method according to any one of claims 1 to 7, wherein performing fractional interpolation on integer pixel points in the reference image frame based on the asymmetric tap coefficients to obtain row fractional pixels and column fractional pixels includes:

10. An image encoding apparatus, the apparatus comprising:

11. A chip comprising programmable logic circuits and/or program instructions for implementing the image encoding method of any one of claims 1 to 9 when the chip is run.

12. A terminal comprising a processor and a memory, wherein the memory has stored therein at least one program that is loaded and executed by the processor to implement the image encoding method of any of claims 1 to 9.

13. A computer readable storage medium having stored therein at least one program code loaded and executed by a processor to implement the image encoding method of any one of claims 1 to 9.

14. A computer program product, characterized in that it comprises computer instructions stored in a computer-readable storage medium, from which a processor reads and executes them to implement the image encoding method according to any of claims 1 to 9.