WO2012065447A1

WO2012065447A1 - Sub pixel interpolation method and system

Info

Publication number: WO2012065447A1
Application number: PCT/CN2011/076352
Authority: WO
Inventors: 舒倩
Original assignee: 深圳市融创天下科技股份有限公司
Priority date: 2010-11-19
Filing date: 2011-06-25
Publication date: 2012-05-24
Also published as: CN102469310B; CN102469310A

Abstract

A sub pixel interpolation method is disclosed in the present invention, wherein the method includes the following steps of: constructing a 1/8, 2/8, 3/8, 5/8, 6/8, 7/8 sub pixel 6 tap interpolation filter based on quartic convolution and a 4/8 sub pixel 6 tap interpolation filter based on cubic convolution; obtaining an interpolation filter group; obtaining pixel values of the 1/8, 2/8, 3/8, 4/8, 5/8, 6/8, 7/8 sub pixels which are line-expanded by horizontal interpolation and row-expanded by vertical interpolation. The present invention also relates to a sub pixel interpolation system. The present invention can realize interpolation of the 1/8, 2/8, 3/8, 4/8, 5/8, 6/8, 7/8 sub pixel at once and use different filter prototypes on sub pixels with different accuracy, which makes the property of different filters reinforce each other, therefore improves the comprehensive performance of the filter group and improves rate-distortion performance of a encoder more obviously at low bit rate.

Description

Technical field

The present invention relates to the field of video coding inter-frame prediction, and in particular, to a sub-pixel interpolation method and system. Background technique

In order to improve compression performance in video coding, sub-pixel precision is used in motion search. The integer pixel refers to the original pixel of the original image or the image after encoding and decoding. The so-called sub-pixel point refers to the pixel point between the whole pixel points obtained by processing a plurality of integer pixel points, that is, in the middle of two integer pixel points. A sub-pixel can be called a half sub-pixel, and a sub-pixel between two 1/2 sub-pixels or an integer pixel and a 1 I 2 sub-pixel can be called a quarter. A sub-pixel, the process of obtaining sub-pixels can be called interpolation filtering. The sub-pixel does not actually exist. It is obtained by the interpolation technique of the whole pixel. Since the sub-pixel precision-based search can obtain a better image matching block, the redundant information of the current coded image block can be further reduced, and the coding efficiency is improved. This makes the performance of motion search closely related to the interpolation filter. What kind of interpolation filter is used for sub-pixels with different precision is also a problem to be considered.

Currently used are 4tap, 6tap spline interpolation filter, bilinear interpolation filter, etc.; in 1/4 sub-pixel interpolation, there are further interpolation by 1/2 sub-pixel, and also directly obtained by integer pixel interpolation. of. The former is not conducive to parallel computing, and the current latter technology generally uses the same type of interpolation filter, so that its performance is not satisfactory. Summary of the invention

The purpose of the embodiment of the present invention is to provide a method for sub-pixel interpolation, which aims to solve the problem that the interpolation method in the prior art is not conducive to parallel computing. Different types of interpolation filters are not used for sub-pixels with different precisions, and the coding performance is not very high. problem.

The method of the embodiment of the present invention is implemented by a sub-pixel interpolation method, which comprises step building 1/8, 2/8, 3/8, 5/8, 6/8, 7 based on four convolutions. /8 sub-pixel 6 tap interpolation filter and 3/8 sub-pixel 6 tap interpolation filter based on cubic convolution;

Obtaining an interpolation filter bank;

Get 1/8, 2/8, 3/8, 4/8, 5/8 after horizontal interpolation column expansion and vertical interpolation line expansion,

6/8, 7/8 pixel subpixel values.

The "building a 4/8, 2/8, 3/8, 5/8, 6/8, 7/8 sub-pixel 6 tap interpolation filter based on four convolutions" is specifically: constructing four convolutions Base function:

-19/240*w ⁴ -209/240*w ³ -855/240*w ² -1539/240*w -1026/240 -3 u<-2 61/240*w ⁴ +503/240*w ³ +1497/240*w ² +1885/240* w + 830/240 -2 u<-\

-l/5*w ⁴ -69/40*w ³ -5/2* w ² +1/40* w + 1 -l- w≤0 -2/15*w ⁴ +193/120* w ³ - 5/2* w ² +l/40*w+l 0- w≤l 59/240*w ⁴ -497/240*w ³ +1503/240*w ² -1915/240*w + 850/240 \ u<2 -7/80*w ⁴ +77/80*w ³ -315/80*w ² +567/80*w-378/80 2- w- 3

0 u≥3 0 u<-3 where w is the base function /( ) is self-variant]

The expressions of the basis functions of the four convolutions are respectively equal to = 1, 2, 3, 5, 6, and 7,

Interpolation filters for 1/8, 2/8, 3/8, 5/8, 6/8, 7/8 sub-pixels: Dance _/8 = [/(- 2 - X), (-1 - X), f(-x), f(l- x), f(2-x), f(3-x)] , = k / 8.

The "building a 3/8 sub-pixel 6 tap interpolation filter based on a cubic convolution" is specifically: constructing a cubic convolution basis function: 19/20* I u I ³ -39/20* I u I ² +1 , 0 -<lu l< 1

-31/40*1 u I ³ +83/20* I u I ² -281/40* I u I +73/20 , 1 -<lu l< 2

/'(")

11/40* I u I ³ -11/5*1 u I ² +231/40* I u I -99/20 , 2 -lu l< 3

0 , I u I 3 where w is the independent function of the basis function / '( ); in conjunction with the expression of the cubic function of the cubic convolution, an interpolation filter of 4/8 sub-pixel is obtained: Dance ¹ ' 4 =

- ], X = 4/8.

The ''obtaining interpolation filter bank'" is specifically:

Dance ^r " ₈ = [/(- 2 - X), (-1 - X), f(-x), f(l - x), f(2 - -x) - )], X: = 1/ 8 Dance 1⁄2/ ₈ = [/(- 2 - ), (-1 - X), f(-x), fCi - X), f(2 - -x) - )], X = 2/8 Dance r ₃ / ₈ = [/(- 2 - Λ-), /(- 1 - X), f(-x), /(1 - Λ-), (2 - -x) - )], X = 3/ 8 r4/ ₈ = [f'(2 + x),fi + x),fx),f'(lx),f 2- ) ' - - )], X = 4/8 Dance r ₅ / ₈ = [ /(- 2 - X), (-1 - X), f(-x), f(l - x), f(2 - -x) - )], X = 5/8 Dance r ₆ / ₈ = [/(- 2 - X), (-1 - X), f(-x), f( - x) (2 - -x) - )], X = 6/8 Dance r ₇ / ₈ = [/ (- 2 - X), /(- 1 - X), f(-x), f{\ - ), f(2 - -x) - )], X = 7/8 "Get horizontal interpolation Column expansion and vertical interpolation lines are expanded by 1/8, 2/8, 3/8, 4/8,

6/8, 7/8 sub-pixel pixel values" specifically:

3⁄4 ₈ ( y) = ') *

I weight(k) The frame after extending the sub-pixel through the vertical interpolation line is: F _kls (x,y, = (filter _k/ , *F^(x,y))/weight(k) where ₈ ( ) is After the column is expanded, the filter,, is /8 pixel interpolation filter, k = 1,2,3,4,5,6,7, filter^, is the transpose matrix of filter^, which is the original frame, ' gfeW is the subpixel interpolation normalization factor, wdgh = sum( ilter^ ₈ ), where, for the summation calculation, _A/8 ( , y) The last filtered interpolation frame is obtained.

Another object of the embodiments of the present invention is to provide a sub-pixel interpolation system, the system comprising: an interpolation filter construction device: for constructing 1/8, 2/8, 3/8, 5 based on four convolutions /8, 6/8, 7/8 sub-pixel 6tap interpolation filter and 3/8 sub-pixel 6 tap interpolation filter based on cubic convolution; interpolation filter bank acquisition device: based on four convolutions based on construction 1/8, 2/8, 3/8, 5/8, 6/8, 7/8 subpixel 6 tap interpolation filters and 3/8 subpixel 6 tap interpolation filters based on cubic convolution, forming filtering Group

Sub-pixel pixel value obtaining means for acquiring 1/8, 2/8, 3/8, 4/8, 5/8, 6/8, 7/8 sub-pixels after horizontal interpolation column expansion and vertical interpolation line expansion The pixel value.

The embodiment of the invention provides a sub-pixel interpolation method and system, which can realize interpolation of 1/8, 2/8, 3/8, 4/8, 5/8, 6/8, 7/8 sub-pixels at a time. When designing the filter bank, sub-pixels of different precisions use different filter prototypes, so that the performance of different filters complement each other, thus improving the overall performance of the filter bank. Experiments have shown that sub-pixel interpolation using the filter bank of the present invention can improve the rate-distortion performance of the encoder, which is more significant at low bit rates.

DRAWINGS

1 is a flow chart of a method of a preferred embodiment of a sub-pixel interpolation method of the present invention;

2 is a schematic block diagram of a preferred embodiment of a sub-pixel interpolation system of the present invention.

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. For the convenience of description, only the parts related to the embodiments of the present invention are shown. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The embodiment of the invention provides a sub-pixel interpolation method and system, which can realize interpolation of 1/8, 2/8, 3/8, 4/8, 5/8, 6/8, 7/8 sub-pixels at a time. When designing a filter bank, different sub-pixels of different precisions use different filter prototypes, so that the performance of different filters complement each other, thereby improving the overall performance of the filter bank. Sub-pixel interpolation using the filter bank of the present invention can improve the rate-distortion performance of the encoder, which is more significant at low bit rates.

FIG. 1 is a flowchart of a sub-pixel interpolation method according to an embodiment of the present invention, where the method includes the following steps:

Sl, build 1/8, 2/8, 3/8, 5/8, 6/8, 7/8 subpixel 6 tap interpolation filters based on four convolutions and 4/8 subpixels based on cubic convolution 6 tap interpolation filter.

S1K is based on four convolution basis functions, combined with 1/8, 2/8, 3/8, 4/8, 5/8, 6/8, 7/8 sub-pixel interpolation points and integer pixel distribution characteristics to construct filters . The base function of the four convolutions is constructed as follows:

-19/240*w -209/240*w ³ -855/240*w ² -1539/240 *w -1026/240 -3-<u< -: 61/240*w ⁴ +503/240*w ³ +1497/240*w ² +1885/240*w + 830/240 -2 u <- -l/5*w ⁴ -69/40*w ³ -5/2*u ² +l/40*w +l -l- w≤0 -2/15*w ⁴ +193/120*w ³ -5/2*u ² +l/40*w + l 0- w≤l 59/240*w ⁴ -497 /240*w ³ +1503/240*w ² -1915/240*w + 850/240 \ u<2 -7/80*w ⁴ +77/80*w ³ -315/80*w ² +567/ 80*w-378/80 2- w- 3

0 u≥3 0 u <-3 where M is the independent variable of the basis function / ( ); in conjunction with the expression of the basis function of the four convolutions, an interpolation filter of 1/8 sub-pixel is obtained: /ilter _{1 /8} = [f(-2 - X), f(-l - x) (-x), f(l - x), f(2 - x), f(3 - x)] , = l/8 Similarly, the expressions of the basis functions of the four convolutions are equal to = 1, 2, 3, 5, 6, and 7, respectively, and 1/8, 2/8, 3/8, 5/8 are obtained. , 6/8, 7/8 sub-pixel interpolation filter: dance r _k/8 = [f(-2 - X), f(-l - X), f(-x), f(l- x) , f(2- x), f(3-x)] , = k/8 In practical applications, in order to reduce the amount of calculation, floating-point operations can be directly converted into integer operations. In addition, the lower-order filter can be constructed by using the approximation method to cut off the data: that is, the floating-point value is replaced by an approximate integer, using the same basis function / ( _W ), construct a low-order "order filter, n ≤ 6.

S12. Based on the cubic convolution basis function, a filter is constructed by combining 4/8 sub-pixel interpolation points and integer pixel distribution characteristics.

The cubic function of the cubic convolution is constructed as follows:

' 19/20* lul ³ -39/20* lul ² +1 , 0 -lul≤ 1

, -31/40*1 u I ³ +83/20* I u I ² -281/40* I u I +73/20 , 1 -lul≤ 2

^U 11/40* lul ³ -11/5* lul ² +231/40* I u I -99/20 , 2 -lul≤ 3

0 , lul 3 combined with the expression of the cubic convolution basis function, can obtain 4/8 sub-pixel interpolation filter:

/ilter _4/8 =[ '(2 + ), '(l + ), '( ), '(l- ), '(2- ), '(3- )], = 4/8.

In practice, in order to reduce the amount of calculation, floating-point operations can be directly converted into integer operations; in addition, the approximation method can be used to construct the low-order filter, that is, the floating-point value is replaced by an approximate integer, using the same basis function / '( ), construct a low-order "order filter bank, n ≤ 6. The motion estimation is limited by the principle of the search algorithm, so that the sub-pixels with different precisions have different influences on the motion estimation. Therefore, the embodiment of the present invention uses a higher-order filter on the 4/8 sub-pixel to improve the interpolation effect. .

S2. Acquire an interpolation filter bank.

Dance 1⁄2 ; =[/(- 2 - x) (-i- -x),f(-x),f(l- -x) (2- -x) - )], x : = 1/8 Dance r _2/: ₈

x),f(-x),f(l -x),f(2- -x) - - )], X = 2/8 Dance 1⁄2 , =[ (-2- ),/(- 1- - x),f(-x),f(l -x),f(2- -x) - )], X = 3/8 r _4/: ₈ =[/'(2 + x) x) \- x),f 2- ) ' - - )], X = 4/8

/liter ₃ =[ (-2- ), (-ΐ- -x),f(-x),f(l -x),f(2- -x) - )], X = 5/8 /ilter _6/8 = [f(-2 - x), f(-l - x), f(-x), f(l- x),f(2 - x), f(3 - x)] , ^ = 6/8

/ilter _7/8 = [f(-2 - x), f(-l - x), f(-x), f(l - x), f(2 - x), f(3- x)] , ^ = 7/8

S3. Obtain pixel values of 1/8, 2/8, 3/8, 4/8, 5/8, 6/8, 7/8 sub-pixels after horizontal interpolation column expansion and vertical interpolation line expansion.

S31: horizontal interpolation, extended column sub-pixel; 3⁄4 ₈ ( , y) = (F(x, y) *

) I weight(k),

Where /Zfer _/8 is the pixel interpolation filter, k=l or 2 or 3 or 5 or 6 or 7, filter ^ is

The transposed matrix, ^F is the original frame, ₈ ,) is the frame after the column expansion, weigh is: subpixel interpolation normalization factor, w gfe(fc) = sum(yilte _/8 ;), where TMm For the summation calculation.

S32: vertical interpolation, extending row sub-pixels;

F _kli (x, y) = ( filter _kli * (x, y ) I weight(k) where F (x, y) is the last obtained interpolated filtered frame, where filte ^ is an 8-subpixel interpolation filter, k=l or 2 or 3 or 5 or 6 or 7, ₈ (; is the frame after column expansion, the value output after SPS31 processing, weighted. Subpixel interpolation normalization factor, w gfe (W = sum(/ Ilter ), where TMm is the summation calculation. Since each interpolated sub-pixel is only related to the whole pixel, and is independent of the previous sub-pixel, each precision sub-pixel interpolation can be calculated in parallel. Pixel interpolation method, which can achieve 1/8, 2/8 at a time,

Interpolation of 3/8, 4/8, 5/8, 6/8, 7/8 sub-pixels. When designing the filter bank, sub-pixels of different precisions use different filter prototypes, so that the performance of different filters complement each other, thus improving the overall performance of the filter bank. Sub-pixel interpolation using the filter bank of the present invention can improve the rate-distortion performance of the encoder, which is more significant at low bit rates.

FIG. 2 is a schematic structural diagram of a sub-pixel interpolation system according to an embodiment of the present invention. The system package Includes:

Interpolation filter construction device: used to construct 1/8, 2/8, 3/8, 5/8, 6/8, 7/8 subpixel 6 tap interpolation filters based on quadratic convolution and based on cubic convolution 4/8 sub-pixel 6 tap interpolation filter; interpolation filter bank acquisition device: 1/8, 2/8, 3/8, 5/8, 6/8 based on four convolutions constructed according to a 7/8 sub-pixel 6 tap interpolation filter and a 3/8 sub-pixel 6 tap interpolation filter based on a cubic convolution to form a filter bank;

Sub-pixel pixel value obtaining means for acquiring 1/8, 2/8, 3/8, 4/8, 5/8, 6/8, 7/8 sub-pixels after horizontal interpolation column expansion and vertical interpolation line expansion The pixel value. The embodiment of the invention provides a sub-pixel interpolation system, which can realize interpolation of 1/8, 2/8, 3/8, 4/8, 5/8, 6/8, 7/8 sub-pixels at a time. When designing the filter bank, sub-pixels of different precisions use different filter prototypes, so that the performance of different filters complement each other, thus improving the overall performance of the filter bank. Sub-pixel interpolation using the filter bank of the present invention can improve the rate-distortion performance of the encoder, which is more significant at low bit rates.

It will be understood by those skilled in the art that all or part of the steps of the foregoing embodiments may be implemented by a program instruction related hardware, and the program may be stored in a computer readable storage medium. The storage medium may be a ROM, a RAM, a magnetic disk, an optical disk, or the like. The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. Within the scope.

Claims

Rights request

A sub-pixel interpolation method, the method comprising the steps of:

Construct a 1/8, 2/8, 3/8, 5/8, 6/8, 7/8 subpixel 6 tap interpolation filter based on four convolutions and a 4/8 subpixel 6 tap based on cubic convolution Interpolation filter

Obtaining an interpolation filter bank;

6/8, 7/8 pixel subpixel values.

2. The sub-pixel interpolation method according to claim 1, wherein said "building is based on four convolutions of 1/8, 2/8, 3/8, 5/8, 6/8, 7/ 8 sub-pixel 6 tap interpolation filter "Specifically: Construct a base function of four convolutions:

-19/240*w ⁴ -209/240*w ³ -855/240*w ² -1539/240*w -1026/240 -3 u<-2 61/240*w ⁴ +503/240*w ³ +1497/240*w ² +1885/240*w + 830/240 -2 u<-\

-l/5*w ⁴ -69/40*w ³ -5/2* w ² +1/40* w + 1 -l- w≤0 -2/15*w ⁴ +193/120*w ³ - 5/2* w ² +1/40* w + 1 0- w≤l 59/240*w ⁴ -497/240*w ³ +1503/240*w ² -1915/240*w + 850/240 \ u<2 -7/80*w ⁴ +77/80*w ³ -315/80*w ² +567/80*w-378/80 2- w- 3

0 u>3 0 u<-3 where "the self-change of the basis function /( )]

Combining the expressions of the basis functions of the four convolutions, respectively, is equal to fc= 1, 2, 3, 5, 6, 7 to obtain 1/8, 2/8, 3/8, 5/8, 6/ 8, 7/8 sub-pixel interpolation filter: Dance r _k/8 = [f(-2 - X), f(-l - X), f(-x), f(l- x), f( 2-x),f(3-x)] , = k / 8.

3. The sub-pixel interpolation method according to claim 1, wherein the "building a 3/8 sub-pixel 6 tap interpolation filter based on a cubic convolution" is specifically: constructing a cubic convolution basis function:

Where w is the independent variable of the basis function /; in conjunction with the expression of the cubic function of the cubic convolution, obtain an interpolation filter of 4/8 sub-pixels: Altcr _4/ , =[f 2 + x),f (\ + xlf (xlf (lx),f(2-x),f(3-x)] , _τ = 4/8.

4. The sub-pixel interpolation method according to claim 1, wherein the "acquisition interpolation filter bank" is specifically: 1 ^tel ' _1/8 = U(-2- ), (-ι- -x ) {-x) (\- -x),f(2- -x) - )], X = 1/8

;[/(- 2 - "), (-l- -x) {-x) {\ -x),f(2 -x),f(3- - ], X = 2/8

/ilter _3/8 ; =[/(- 2- x) - - -x) {-x) (\ -x),f(2- -x),f(3- )], X = 3/8

r _{4 s} =[ '(2 + x) '( + x),fx) \- x),f 2- -x),f'(3- - )], X = 4/8

/ilter _5/8 ; =[/(- 2 - ), (-ΐ- -x) (-x) (\ -x),f(2- -x) - )], X = 5/8

s =[ (-2- ), (-ι- -x),f{-x) (\ -x) - -x) - )], X = 6/8

;[/(- 2 - x) (- - -x) {-x) (\ -x),f(2- -x),f(3- - ], X = 7/8.

5. The sub-pixel interpolation method according to claim 1, wherein said "acquiring 1/8, 2/8, 3/8, 4/8 after horizontal interpolation column expansion and vertical interpolation row expansion, 5/8, 6/8, 7/8 sub-pixel pixel values" Specifically: The frame after the sub-pixel is expanded by the horizontal interpolation column is:

I weight(k) The frame after extending the sub-pixel through the vertical interpolation line is: ^ _/8 ,v) = (filter _k/& *F^ (x,y))/ weigh†(k) Where ₈ (; is the frame after column expansion, 3⁄4^ _/8 is /8 pixel interpolation filter, k = 1,2,3,4,5,6,7, 3⁄4^ _/8 is 3⁄4^ _/8 Transpose matrix, ^F(JC , is the original frame, g/^W is the subpixel interpolation normalization factor, w gfe(fc) =

), where TMm is the summation calculation, and F _k x, y) is the last acquired interpolated filtered frame.

The sub-pixel interpolation method according to claim 2, wherein the floating point operation is converted into an integer operation in the /^).

7. The sub-pixel interpolation method according to claim 3, wherein the floating point operation is converted into an integer operation in the /).

The sub-pixel interpolation method according to claim 2, wherein the floating point value in the /^) is replaced by an approximate integer, and the same basis function / ) is used to construct a "order filter, M ≤ 6.

The sub-pixel interpolation method according to claim 3, wherein the / mid-floating point value is replaced by an approximate integer, and the same basis function / ) is used to construct a "order filter, M ≤ 6.

10. A sub-pixel interpolation system, wherein the system comprises:

Interpolation filter construction device: used to construct 1/8, 2/8, 3/8, 5/8, 6/8, 7/8 sub-pixel 6tap interpolation filters based on four convolutions and based on cubic convolution 4/8 sub-pixel 6 tap interpolation filter; interpolation filter bank acquisition device: 1/8, 2/8, 3/8, 5/8, 6/8, 7 based on four convolutions constructed a /8 sub-pixel 6 tap interpolation filter and a 3/8 sub-pixel 6 tap interpolation filter based on cubic convolution to form a filter bank;