WO2007100128A1

WO2007100128A1 - Method and system for picture up-sampling

Info

Publication number: WO2007100128A1
Application number: PCT/JP2007/054139
Authority: WO
Inventors: Shijun Sun
Original assignee: Sharp Kabushiki Kaisha
Priority date: 2006-02-28
Filing date: 2007-02-27
Publication date: 2007-09-07

Abstract

A picture up-sampling method uses plural up-sampling filters with phase-related coefficients, for image interpolation, the relative influence of which is controlled by a weighting factor to realize desirable filter selection in spatial scalable video coding.

Description

DESCRIPTION

METHOD AND SYSTEM FOR PICTURE UP-SAMPLING

TECHNICAL FIELD

The present invention generally relates to an up-sampling method and an up-sampling system, and more particularly relates to an up-sampling method and an up-sampling system which realize a desirable filter selection in spatial scalable video coding.

BACKGROUND ART

Some embodiments of the present invention are related to the Scalable Video Coding (SVC) extension of H .264 /Advanced Video Coding (AVC) . In the current SVC extension of H .264 (in Joint Draft version 4 , Joint Video

Team (JVT) -Q202) , the texture signal of a base layer is Team (JVT)-Q202) , the texture signal of a base layer is upsampled using a set of 6-tap filters before it is used as a prediction signal for the enhancement layer. The 6-tap filters are derived from the Lanczos-3 function and defined in a pre-fixed filter table.

DISCLOSURE OF THE INVENTION

Some embodiments of the present invention are related to the Scalable Video Coding (SVC) extension of H.264/AVC. More specifically, some embodiments comprise a filter design related to the texture up-sampling in spatial scalable video coding.

Embodiments of the present invention comprise one or more up-sampling filters for image interpolation. Some embodiments comprise a matrix-based representation of a set of 6-tap filters, which have a very similar frequency response to that of Lanczos3 filter.

Some embodiments may also comprise a matrix-based representation of a new set of 4-tap filters, which may obtain a wider pass-band than the popular Catmull-Rom

• filter. Other embodiments comprise a combination of filters controlled by a weighting factor. In some embodiments a combination of filters with phase-related coefficients may be used. The foregoing and other objectives, features, and advantages of the invention will be more readily understood upon consideration of the following detailed description of the invention taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

Fig. 1 is a diagram showing the geometric relationship between a base spatial layer and an enhancement spatial layer in some embodiments of the present invention;

Fig. 2 is a diagram showing the frequency response of a cubic B-spline and a Catmull-Rom cubic at phase position of 1 / 2;

Fig. 3 a diagram showing a comparison between filter coefficients;

Fig. 4 is a diagram showing a frequency response of a 6-tap cubic filter and Lanczos-3 filter as well as the 4-piece cubic filters at phase position of 1 / 2; and

Fig. 5 is a diagram showing a frequency response of a 4-tap cubic filter, a 6-tap cubic filter and a

^■ Catmull-Rom filter at phase position of 1 / 2.

Fig. 6 is a block diagram showing an up-sampling system of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be best understood by reference to the drawings, wherein like parts are designated by like numerals throughout. The figures listed above are expressly incorporated as part of this detailed description. It will be readily understood that the components of the present invention, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of the embodiments of the methods and systems of the present invention is not intended to limit the scope of the invention, but it is merely representative of the presently- preferred embodiments of the invention.

Elements of embodiments of the present invention may be embodied in hardware, firmware and/ or software.

While exemplary embodiments revealed herein may only describe one of these forms, it is to be understood that one skilled in the art would be able to effectuate these elements in any of these forms while resting within the scope of the present invention.

H.264/ MPEG-4 AVC [Joint Video Team of ITU-T VCEG and ISO/IEC MPEG, "Advanced Video Coding (AVC) - 4th Edition," ITU-T Rec. H.264 and ISO/ IEC 14496- 10 (MPEG4-Part 10) , January 2005] , which is incorporated by reference herein, is a video codec specification that is related to embodiments of the present invention.

Spatial scalability is supported by the Scalable Video Coding (SVC) extension of H.264/ MPEG-4 AVC.

The SVC extension of H.264/ MPEG-4 AVC [Working Document 1.0 (WD- 1.0) (MPEG Doc. N6901 ) for the Joint

Scalable Video Model (JSVM)] , which is incorporated by reference herein, is a layered video codec in which the redundancy between spatial layers is exploited by inter-layer prediction mechanisms. Some embodiments of the present invention relate to the Scalable Video Coding Extension of H.264/AVC. Some embodiments relate to filtering to address a problem of picture up-sampling for spatial scalable video coding. More specifically, some embodiments of the present invention provide an up-sampling procedure that is designed for the Scalable Video Coding extension of H.264/MPEG-4 AVC, especially for the Extended Spatial Scalable (ESS) video coding feature adopted in April 2005 by JVT (Joint Video Team of MPEG and VCEG) . Currently, JSVM WD- 1.0 [MPEG Doc. N6901 ], which

^■ is incorporated by reference herein, only addresses dyadic spatial scalability, that is, configurations where the ratio between picture width and height (in terms of number of pixels) of two successive spatial layers equals 2. This obviously will be a limitation on more general applications, such as SD to HD scalability for broadcasting.

For the purposes of this specification and claims, the term "picture" may comprise an array of pixels, a digital image, a subdivision of a digital image, a data channel of a digital image or another representation of image data.

Figure 1 shows two pictures corresponding to an image picture:

Embodiments of the present invention relate to two or more successive spatial layers, a lower layer

(considered as base layer) 253 and a higher layer

(considered as enhancement layer) 251. These layers may be linked by the following geometrical relations

(shown in Figure 1 ) . Width 250 and height 252 of enhancement layer pictures may be defined as w_enh and henh, respectively. In the same way, dimensions of a base layer picture may be defined as Wbase 254 and hbase

256. The base layer 253 may be a subsampled 264 version of a sub-region of an enhancement layer picture 251 , of dimensions Wextract 258 and h_extract 260, positioned

^■ at coordinates 262 (x_Ori_g , yorig) in the enhancement layer picture coordinate system. Parameters (x_ori_g , yorig ,

Wextract , hextract , Wbase , hbase) define the geometrical relations between a higher layer picture 251 and a lower layer picture 253. Here, since the resolutions are different between the higher layer picture 251 and the lower layer picture 253 , the respective pixels of the higher layer picture 251 and the lower layer picture 253 do not have one to one correspondence, and one pixel in the lower layer picture

353 overlap multiple pixels in the higher layer picture 251 . This creates an offset between pixel edges in the lower layer picture 253 and the pixel edges in the higher layer picture 251 , which is known as "phase offset" . Cubic Splines

Splines are piecewise polynomials. Typically, cubic spline filters with four pieces or intervals have been applied in many applications. One such filter is known as the "B-spline" filter as represented in Eq. 1 . Among piecewise cubic functions, the B-spline is special because it has continuous first and second derivatives.

Another popular piecewise cubic filter, the Catmull-Rom filter, has the value zero at x = -2, - 1 , 1 , and 2, which means it will interpolate the samples when used as a reconstruction filter.

For the application of resampling images, Mitchell and Netravali recommended one partway between the previous two filters. It is simply a weighted combination of the previous two filters with b and c as the weighting factors (b+c= l ) .

f_M(x) = b -f_B(x) + c -f_c(x) (₃)

Adaptive Upsampling

Adaptive up-sampling may be applied for spatial scalability video coding. The Mitchell-Netravali filter in adaptive image up-sampling has been proposed for the

SVC standard. The adaptive filter selection can be achieved by adjusting the weighting factors. The weighting factors may be selected based on, for example, image noise, or the proximity to an image block boundary.

^■ As shown in Figure 2, the cubic B-spline tends to blur the signals more than the Catmull-Rom cubic does. For example, at a normalized frequency of 0.7, the B-spline is roughly 4.5 dB below the Catmull-Rom. And the size of this gap can be used to represent the flexibility or dynamic range of the adaptive filter design.

6-tap Cubic-Spline Interpolation Filter In the current SVC extension of H.264 (in Joint Draft version 4, JVT-Q202) , the texture signal of a base layer is upsampled using a set of 6-tap filters before it is used as a prediction signal for the enhancement layer. The 6-tap filters are derived from the Lanczos-3 function and defined in a pre-fixed filter table.

Inspired by the 4-piece cubic functions, which give us the 4-tap filters, some 6-piece cubic splines were studied. These splines can yield 6-tap filters that have similar frequency response with that of the Lanczos-3 filter.

The 6-piece function may be described as:

Z₁ (X) | x |≤ l ^■

/₂ (x) l ≤| x |< 2 f S6 (x) = Z₃ (x) 2 <| x |< 3 (4)

0 otherwise

By requiring me ioπowmg conditions including C¹ and C² conditions between pieces of splines,

/J(0) = l,^(l) = 0,/₂(2) = 0, /₃(3) = 0,

./T(O) = O, /₃ '(3) = 0,

■/ϊα) = /₂α)>/₂(2) = /3(2), (5)

/ϊ '(I) = Z₂ 'α),/₂ '(2) = /3 '(2),

Λ "(I) =Z₂ ¹¹GXZ₂ "(2) =Z₃ "(2) we can get the following solution for the 6-piece spline as an interpolation filter

For a relative phase offset position 0<=x< l , this kernel produces a 6-tap FIR filter with tap values given by the following matrix equation

Actually, it is sufficient to consider only the range of x from 0 to 1 / 2 , since the FIR filter kernel for x is simply the FIR filter kernel for 1 - x in reverse order. It is clearly shown in Figure 3 (a comparison between the filter coefficients based on Eq-7 and Lanczos-3) that Eq-7 is a very good approximation of the Lanczos-3 function.

As shown in Figure 4 , the new 6-piece cubic filter gives less-blurred signals than the Catmull-Rom filter. For example, at normalized frequency of 0.7, the new 6-tap filter is roughly 2 dB above the Catmull-Rom. And it has been observed that the filters given in Eq-7 have very similar frequency response with the existing 6-tap Lanczos-3 filters. So, Eq-7 can potentially be used as a closed-form representation for the up-sampling filters in the SVC extension.

Embodiments of the present invention may comprise a weighted combination of the three cubic spline functions.

F_s(x) =b -f_B(x) + c-f_c(x) + s -f_S6(x) (8) with (b+c+s)=l.

Since the new 6-tap filter potentially gives sharper images, the new combination as in Eq-8 potentially can provide more flexible filter design solutions with increased dynamic range.

One special option is to have c=0 in Eq-8 , so Eq-8 can become a weighted combination of the B-spline and the newly proposed filter. When s=0, Eq-8 will simply become the Mitchell-Netravali filter. When b=0, Eq-8

^• becomes a weighted combination of Catmull-Rom and the new 6-tap filter.

Integerization and Dynamic Range Control Meanwhile, there is also a simpler option. First, we can pre-calculate the cubic filters for various phases as fixed-point numbers (for example 8-bit numbers) and stored in look-up-tables. Tables 1 -3 show the filters derived for 16 phase positions from the three cubic functions, respectively. We can also represent the weighting parameters as fixed-point numbers (for example 6-bit numbers) and signal them in the bitstreams . The desired filter coefficients can then be calculated and rounded to fixed-point numbers (for example 6-bit numbers) for the interpolation process.

Table- 1 Filter coefficients based on 4-piece cubic B-Spline

Table-2 Filter coefficients based on Catmull-Rom spline

Table-3 Filter coefficients based on the new 6-piece cubic Spline

4-tap Cubic Spline Interpolation Filter Comparing to the 6-tap filter, the advantage of the 4-tap filter is the lower complexity requirement. We have observed that by changing the constraints in the cubic functϊons, a new set of 4-tap filters can be derived with wider pass band than the Catmull-Rom filter.

A 4-piece spline function may be defined as:

By requiring the following conditions,

/₁(0) = l,/₁(l) = 0,/₂(2)-0

Z₁(I)

"(i)

We can get the following solution for the 4-piece spline as an interpolation filter

For a relative phase offset position 0<=x< l , this kernel produces a 4-tap FIR filter with tap values given by the following matrix equation

As shown in Figure 4, the new cubic filter gives less-blurred signals than the Catmull-Rom filter although it still tends to blur more than the 6-tap filters. For example, at normalized frequency of 0.7, the new 4-tap filter is roughly 1 dB above the Catmull-Rom while roughly 1 dB below the new 6-tap cubic filter.

Table-4 shows the filter coefficients as fixed-point numbers for various phases. Some embodiments of the present invention may comprise an adaptive filter design as a weighted combination of several basis functions as shown in the following equation.

F_s(x)=b-f_B(x)+c-f_c(x)+s-f_S4(x) (13) with (b+c+s) = l . And obviously, the new 4-tap filter can enable larger filter dynamic range in adaptive filter design than the Catmull-Rom case [I] .

Table-4

Filter coefficients based on the new 4-piece cubic Spline

In some embodiments, the 4-tap filter alone can be applied to the up-sampling of chroma signals to reduce the complexity while maintaining reasonable coding quality comparing to the current SVC design. SVC Syntax

For SVC design embodiments, a signal may be sent to indicate whether the default up-sampling filter should be applied or the adaptive filter derivation process be

^■ invoked. When the adaptive filter option is selected, the filter weighting parameters (s and/or c in Eq-8 or Eq- 13) can be signaled in the slice header. In some embodiments, the weighting parameters can be signaled separately for vertical and horizontal directions. In some embodiments, the parameters for luminance and chroma channels can be signaled separately. For the luminance channel, the filter definition is preferred to follow Eq-8. However, for the chroma channel, there is certain benefit (in terms of reduced complexity) to apply Eq- 13 (instead of Eq-8) so the up-sampling filter is always 4-tap.

In some embodiments, depending on the frequency response of desired filters in a typical application, various combinations of the discussed filter functions can be defined and applied. In some downsampling embodiments, a weighted combination of several basis filter functions can also be applied. For embodiments with adaptive interpolation filter design in motion compensation, a weighted combination of these basis filter functions can also be applied.

In the current SVC extension of H.264, the 6-tap filters are derived from the Lanczos-3 function and defined in a pre-fixed filter table. Coding performances are reported here using a 4-tap cubic-spline based filter.

^■ The results show a degradation of 0.04 dB on average

(and up to 0.09 dB) for all Intra picture coding. Coding results are also provided for the 4-tap Catmull-Rom (also cubic-spline based) filter, which gives a degradation of 0.09 dB on average (and up to 0.22 dB) . The degradation in coding performance for typical long-delay configurations is negligible for both 4-tap cubic splines. The current JSVM downsampling filters are applied in all experiments. Embodiments of the present invention adopt the new spline-based filter (JVT-SO 16) for luminance texture (picture values) up-sampling in order to reduce the computational complexity.

A new cubic-spline function is given in the following equation.

For a relative phase offset position 0< =x< l , this kernel produces a 4-tap FIR filter with tap values given by the following matrix equation:

In some embodiments of the present invention, the filter coefficients are pre-calculated and stored in filter look-up tables as in Table-5 and Table-6. The normalization factor of the filters is 32, which is consistent with that of the current filter design. JVT-R066 outlined a basic procedure for deriving filter coefficients, which can be a good option for specific implementation.

Table-5 : Filter coefficients derived from the 4-tap cubic spline function (JVT-SO 16)

Table-6 : Filter coefficients derived from the

Catmull-Rom function (Eq. 2)

All experimental results (except interlace coding tests) are based on the JSVM_5_9 software, which includes all ESS related adoptions in previous meetings.

The current JSVM downsampling filters (based on Sine-windowed Sine functions) are applied in all experiments.

Dyadic spatial scalability

Experiments are first conducted to compare the up-sampling filters in ESS-dyadic coding performance.

All-Intra configuration

For intra only configuration, the QP at base layer is set to 24 , 30, and 36, respectively. The QP difference between a spatial layer and its immediate enhancement layer is "-4" . As shown in Table 3 , the degradations in coding performance are not very significant, with the average (of eight test sequences) at 0.04 dB for the JVT-SO 16 spline function and 0.09 dB for the Catmull-Rom spline. The average SNR differences are calculated based on the approach introduced in VCEG-M33 by Gisle Bjontegaard. Detailed experimental results are available in JVT-TOxx.xls. The average PSNR differences in Table 7 are calculated for the layer with the original (or highest) resolution.

Table 7 : Performance difference for all-intra coding between the JSVM and the 4-tap spline-based

Long-delay configuration

For typical long-delay configuration, the encoder parameters and rate points are based on the Spatial Scalability section in the common test conditions as defined in JVT-Q205. Additionally, "intra_ρeriod" is set to "64" for the 4CIF sequences or "32" for the CIF sequences. As shown in Table 8 , the degradations in coding performance are negligible for both 4-tap spline functions.

Table 8 : Performance difference for long-delay coding between the JSVM and the 4-tap spline-based up-sampling filters (JVT-SO 16 and Catmull-Rom)

Non-dyadic spatial scalability

For ESS non-dyadic tests, the picture resolutions and encoder parameters and rate points for the long-delay configurations are based on the earlier ESS core experiments (as in Poznan and Nice meetings) . Additionally, various combinations of scaling ratios and picture QP's are tested for all-intra configuration. The results for the all-intra configuration are summarized in Table-9 , which indicates no significant difference in coding performance. The luminance PSNR was improved by 0.009 dB while the bitrate increased by 0.29%.

The results for the long-delay configuration are summarized in Table- 10, which also indicates no significant difference in coding performance . The luminance PSNR dropped 0.015 dB . Interlace Coding

Experiments are also conducted following the test conditions defined in CE2 for interlace SVC . The software distributed among the CE participants was used for the tests. The results are summarized in Table-11 for the four test configurations defined in CE2. Similar to the non-interlace ESS tests, no significant difference in coding performance is. observed either for interlace coding configurations.

Table 9 : Performance difference for all-intra coding between the JSVM and the 4-tap spline-based up-sampling filter (JVT-SOl 6) for non-dyadic tests

Table 10 : Performance difference for long-delay coding between the JSVM and the 4-tap spline-based up-sampling filter (JVT-SOl 6) for non-dyadic tests W 2

- 25 -

Table 11 : Performance difference between the JSVM and "the 4-tap spline-based up-sampling filter (JVT-SOl 6) foτ 4 different interlace coding configurations:

Coding performances are reported for the 4-tap cubic-spline based filter introduced in JVT-SO 16. The results show a degradation of 0.04 dB on average for all-Intra picture coding. The degradation in coding performance for typical long-delay configurations (including interlace configurations) is negligible. During the experiments, no significant visual quality degradation is observed. Embodiments of the present invention comprise a new spline-based filter as described in JVT-SO 16 and Table-5) for luminance texture up-sampling in order to reduce the computational complexity of the texture up-sampling process.

The system for picture up-sampling from the lower resolution picture to the higher resolution picture, each of the pictures being made up of a plurality of pixels will be explained with reference to the block diagram of Figure 6. As shown in Figure 6, the picture up-sampling system

100 includes a position processor 1 1 , a phase calculator 12, interpolation filters 13, 14 (up-sampling filters for image interpolation), a first coefficient selector 15, a second coefficient selector 16, a weighting factor calculator 17, and a filter application 18. The position

^■ processor 1 1 determines the lower resolution picture location corresponding to the sample position in the higher resolution picture. The phase calculator 12 determines the phase offset position for the picture location, based on the relative location of the sample position in the higher resolution picture with respect to the lower resolution picture location. The respective coefficients of the two interpolation filters 13, 14 (first and second interpolation filters) are selected respectively by the first coefficient selector 15 and the second coefficient filter 16 based on the phase offset position. The weighting factor calculator 17 selects the weighting factor(s) to control the relative influence of the first interpolation filter 13 and the second interpolation filter 14. The filter application 18 calculates the picture value for the sample position, using the weighted combination of the first interpolation filter 13 and the second interpolation filter 14, which is controlled by the weighting factor(s) as selected by the weighting factor calculator 17.

In the present invention, the picture value refers to the intensity value for a pixel or a sub-pixel. In an exemplary application, the picture value can be determined by a) determining a first picture value with an application of the first interpolation filter; b) determining

^• a second picture value with an application of the second interpolation filter; and c) multiplying the resulting first and second picture values by one or more weighting factors to obtain a final picture value, as can be performed for filters in any direction or combination of directions.

As described, the method for picture up-sampling from a lower resolution picture to a higher resolution picture, each of the pictures being made up of a plurality of pixels (pixel array) includes the steps of determining a lower resolution picture location corresponding to a sample position in the higher resolution picture; determining a phase offset position for the picture location; selecting a first filter coefficient for a first interpolation filter based on the phase offset position; selecting a second filter coefficient for a second interpolation filter; selecting a weighting factor to control the relative influence of the first interpolation filter and the second interpolation filter; and calculating a picture value for the sample position using a weighted combination of the first interpolation filter and the second interpolation filter, the weighted combination being controlled by the weighting factor.

According to the foregoing method, the relative influence of the first interpolation filter and the second

^■ interpolation filter for up-sampling is controlled by the weighting factor, which can be selected based on, for example, image noise, or the proximity to an image block boundary. Here, since the resolutions are different between the lower resolution picture and the corresponding higher resolution picture, the respective pixels in the higher resolution picture and the lower resolution picture do not have one to one correspondence, and one pixel in the lower resolution picture overlap multiple pixels in the higher resolution picture. This creates an offset between pixel edges in the lower resolution picture and the pixel edges in the higher resolution picture, which is known as "phase offset". Therefore, the phase offset position for the picture location can be determined based on the relative location of the sample position in the higher resolution picture with respect to the lower resolution picture location.

It is preferable that the foregoing method be arranged such that the second filter coefficient is selected based on the phase offset position.

It is preferable that the foregoing method be arranged such that the first interpolation filter and the second interpolation filter are controlled by separate weighting factors.

In the foregoing method, the 4-tap filter based on a 4-piece cubic spline may be used for one of the first interpolation filter and the second interpolation filter is .

The foregoing method of the present invention may be arranged such that the first interpolation filter is a 4-tap filter based on a 4-piece cubic spline and the second interpolation filter is derived from the Catmull-Rom function.

The foregoing method of the present invention may be arranged such that the phase offset position and the first and second filter coefficients are determined and selected independently for a horizontal direction and a vertical direction.

The foregoing method of the present invention may be arranged such that one of the first interpolation filter and the second interpolation filter is a 4-tap filter with coefficients defined by the following table:

The foregoing method of the present invention may be arranged such that one of the first interpolation filter and the second interpolation filter is a 4-tap FIR filter with tap values given by the following matrix equation:

0 4 0 0 -3 0 3 0

1 x x² x³ *

A L 6 -9 6 -3 (15)

-3 5 -5 3

wherein the phase offset position is defined as x, where 0< =x< l .

As described, another method for picture up-sampling from a lower resolution picture to a higher resolution picture, each of the pictures being made up of a plurality of pixels, is arranged so as to include the steps of determining a lower resolution picture location corresponding to a sample position in the higher resolution picture; determining a phase offset position for the picture location; selecting a first filter coefficient for a first interpolation filter; selecting a second filter coefficient for a second interpolation filter; selecting a

^• third filter coefficient for a third interpolation filter; selecting at least one weighting factor to control the relative influence of the first interpolation filter, the second interpolation filter and the third interpolation filter; and calculating a picture value for the sample position using a weighted combination of the first interpolation filter, the second interpolation filter and the third interpolation filter, the weighted combination being controlled by the at least one weighting factor. The foregoing method of the present invention may^¬ be arranged such that one of the first interpolation filter, the second interpolation filter and the third interpolation filter is a 4-tap filter based on a 4-piece cubic spline.

The foregoing method of the present invention may be arranged such that one of the first interpolation filter, the second interpolation filter and the third interpolation filter is a 4-tap FIR filter with tap values given by the following matrix equation

wherein the phase offset position is defined as x, where 0<=x< l . The foregoing method of the present invention may

^• be arranged such that one of the first interpolation filter, the second interpolation filter and the third interpolation filter is a 4-tap filter with phase-related coefficients taken from the following table:

The foregoing method of the present invention may be arranged such that one of the first interpolation filter, the second interpolation filter and the third interpolation filter is a 4-tap filter with phase-related coefficients taken from the following table:

The foregoing method of the present invention may^¬ be arranged such that one of the first interpolation filter, the second interpolation filter and the third interpolation filter is a 6-tap filter with phase-related coefficients taken from the following table:

The foregoing method of the present invention may be arranged such that the first interpolation filter is a 4-tap filter based on a 4-piece cubic spline and the second interpolation filter is derived from the

Catmull-Rom function.

The foregoing method of the present invention may be arranged such that the first interpolation filter is a 4-tap filter based on a 4-piece cubic spline, the second interpolation filter is derived from the Catmull-Rom function and the third interpolation filter is a 6-tap filter based on a 6-piece cubic spline.

The foregoing method of the present invention may be arranged such that the first interpolation filter is a 4-tap filter with phase-related coefficients taken from the Table I below:

Table I

wherein the second interpolation filter is derived from the Catmull-Rom function and the third interpolation filter is a 6-tap filter with phase-related coefficients taken from the Table II below: Table II

10

The terms and expressions which have been employed in the forgoing specification are used therein as

W terms of description and not of limitation, and there is no intention in the use of such terms and expressions of excluding equivalence of the features shown and described or portions thereof.

,0 INDUSTRIAL APPLICABILITY

The picture up-sampling method and system of the present invention are applicable to the filter design related to the texture up-sampling in spatial scalable video coding.

Claims

1. A method for picture up-sampling from a lower resolution picture to a higher resolution picture, each of said pictures being made up of a plurality of pixels, said method comprising the steps of: a. determining a lower resolution picture location corresponding to a sample position in said higher resolution picture; b. determining a phase offset position for said picture location, based on the relative location of said sample position in said higher resolution picture with respect to said lower resolution picture location; c. selecting a first filter coefficient for a first interpolation filter based on said phase offset position; d. selecting a second filter coefficient for a second interpolation filter; e. selecting a weighting factor to control the relative influence of said first interpolation filter and said second interpolation filter; and f. calculating a picture value for said sample position using a weighted combination of said first interpolation filter and said second interpolation filter, said weighted combination being controlled by said weighting factor.

2. The method as described in claim 1 wherein said second filter coefficient is selected based on said phase offset position.

3. The method as described in claim 1 , wherein: said weighting factor is selected based on image noise .

4. The method as described in claim 1 , wherein: said weighting factor is selected based on proximity to an image block boundary.

5. The method as described in claim 1 , wherein: said first interpolation filter and said second interpolation filter are controlled by separate weighting factors.

6. The method as described in claim 1 wherein: one of said first interpolation filter and said second

^■ interpolation filter is a 4-tap filter based on a 4-piece cubic spline.

7. The method as described in claim 1 wherein: said first interpolation filter is a 4-tap filter based on a 4-piece cubic spline and said second interpolation filter is derived from the Catmull-Rom function.

8. The method as described in claim 1 wherein: said phase offset position and said first and second filter coefficients are determined and selected independently for a horizontal direction and a vertical direction.

9. The method as described in claim 1 wherein: one of said first interpolation filter and said second interpolation filter is a 4-tap filter with coefficients defined by the following table:

10. The method as described in claim 1 wherein: one of said first interpolation filter and said second interpolation filter is a 4-tap FIR filter with tap values given by the following matrix equation:

wherein said phase offset position is defined as x, where 0<=x< l .

1 1. A method for picture up-sampling from a lower resolution picture to a higher . resolution picture, each of said pictures being made up of a plurality of pixels, said method comprising the steps of: a. determining a lower resolution picture location corresponding to a sample position in said higher resolution picture; b. determining a phase offset position for said picture location, based on the relative location of said sample position in said higher resolution picture with

^■ respect to said lower resolution picture location; c. selecting a first filter coefficient for a first interpolation filter; d. selecting a second filter coefficient for a second interpolation filter; e. selecting a third filter coefficient for a third interpolation filter; f. selecting at least one weighting factor to control the relative influence of said first interpolation filter, said second interpolation filter and said third interpolation filter; and g. calculating a picture value for said sample position using a weighted combination of said first interpolation filter, said second interpolation filter and said third interpolation filter, said weighted combination being controlled by said at least one weighting factor.

12. The method as described in claim 1 1 , wherein: one of said first interpolation filter, said second interpolation filter and said third interpolation filter is a 4-tap filter based on a 4-piece cubic spline.

13. The method as described in claim 1 1 , wherein: one of said first interpolation filter, said second interpolation filter and said third interpolation filter is a 4-tap FIR filter with tap values given by the following matrix equation

wherein said phase offset position is defined as x, where 0<=x< l .

14. The method as described in claim 1 1 wherein: one of said first interpolation filter, said second interpolation filter and said third interpolation filter is a

4-tap filter with phase-related coefficients taken from the following table:

15. The method as described in claim 1 1 wherein: one of said first interpolation filter, said second interpolation filter and said third interpolation filter is a 4 -tap filter with phase-related coefficients taken from the following table:

16. The method as described in claim 11 -wherein one of said first interpolation filter, said second interpolation filter and said third interpolation filter is a 6-tap filter with phase-related coefficients taken from the following table :

17. The method as described in claim 1 1 wherein: said first interpolation filter, is a 4-tap filter based on a 4-piece cubic spline and said second interpolation filter is derived from the Catmull-Rom function.

18. The method as described in claim 1 1 wherein: said first interpolation filter is a 4-tap filter based on a 4-piece cubic spline, said second interpolation filter is derived from the Catmull-Rom function and said third interpolation filter is a 6-tap filter based on a 6-piece cubic spline.

19. The method as described in claim 1 1 wherein said first interpolation filter is a 4 -tap filter with phase-related coefficients taken from the Table 1 below:

Table I

wherein said second interpolation filter is derived from the Catmull-Rom function and said third interpolation filter is a 6-tap filter with phase-related coefficients taken from the Table II below:

Table II

20. A system for picture, up-sampling from a lower resolution picture to a higher resolution picture, each of said pictures being made up of a plurality of pixels, said system comprising: a position processor for determining a lower resolution picture location corresponding to a sample position in said higher resolution picture; a phase calculator for determining a phase offset position for said picture location, based on the relative location of said sample position in said higher resolution picture with respect to said lower resolution picture location; a first interpolation filter; a second interpolation filter; a first coefficient selector for selecting a first filter coefficient for said first interpolation filter based on said phase offset position; a second coefficient selector selecting a second filter coefficient for said second interpolation filter; a weighting factor calculator for selecting a weighting factor to control the relative influence of said first interpolation filter and said second interpolation filter; and a filter application for calculating a picture value for said sample position, using a weighted combination of said first interpolation filter and said second interpolation filter, said weighted combination being controlled by said weighting factor.