AU729482B2

AU729482B2 - Image interpolation with a continuous two-dimensional kernel

Info

Publication number: AU729482B2
Application number: AU63022/99A
Authority: AU
Inventors: Dominic Yip
Original assignee: Canon Inc
Current assignee: Canon Inc
Priority date: 1998-12-18
Filing date: 1999-12-01
Publication date: 2001-02-01
Anticipated expiration: 2019-12-01
Also published as: AU6302299A

Description

S&F Ref: 482502

AUSTRALIA

PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT

ORIGINAL

Name and Address of Applicant: Canon Kabushiki Kaisha 30-2, Shimomaruko 3-chome Ohta-ku Tokyo Japan Actual Inventor(s): Address for Service: Invention Title: Dominic Yip Spruson Ferguson St Martins Tower 31 Market Street Sydney NSW 2000 Image Interpolation with a Continuous Twodimensional Kernel ASSOCIATED PROVISIONAL APPLICATION DETAILS [33] Country [31] Applic. No(s) PP7801 [32] Application Date 18 Dec 1998 The following statement is a full description of this invention, including the best method of performing it known to me/us:- 5815c -1- IMAGE INTERPOLATION WITH A CONTINUOUS TWO-DIMENSIONAL

KERNEL

Field of Invention The present invention relates to the field of resolution conversion of multidimensional digital data and in particular digital image data. The present invention is particularly advantageous for the interpolation of image data that contains abrupt changes in values, (eg. edges), where these changes are 2-dimensional in nature forming lines and boundaries at oblique angles to the underlying pixel layout. Examples of such lines and boundaries are in digital images containing text, graphics, or other high contrast edges between objects. The invention also relates to a computer program product including a S" computer readable medium having recorded thereon a computer program for resolution conversion of multi-dimensional digital data and in particular digital image data.

Background of Invention In the field of digital data resolution conversions, there are numerous 1: architectures and methods that take into account the abrupt changes in neighbouring pixel values when performing interpolation on an input image. Such architectures and methods usually apply some sort of edge detection filter on the input image to detect the existence of edges, and then modify the interpolation kernel used to compute the output image data.

For example, one such architecture may switch from using the cubic kernel to using the nearest neighbour (NN) kernel when it detects the existence of an edge in the input image, as the nearest neighbour kernel retains the original contrast between the neighbouring pixels. These architectures and methods either apply a 1-dimensional kernel first to the rows of the image and then to the columns, known as a separable implementation, or apply a 2-dimensional kernel directly to the image data.

These are then termed bilinear and bicubic implementations. The disadvantage of the conventional interpolation kernels is that they interpolate mainly using pixel values from the horizontal and vertical directions relative to the interpolated pixel. Therefore, when interpolating an oblique edge, pixel values on either side of the edge are primarily used in the interpolation rather than pixels along the edge. This results in an interpolated CFP1536AU (482502) [0:\CISRA\RC\RC051482502.doc:SaF (i -2edge that no longer appears to be smooth, but appears to be jagged. These artefacts are commonly referred to as jaggies. In addition to the edge jaggies, the conventional methods and architectures also create a blurring effect on edges in the output image. Fig.

1 shows the effect ofjaggies on an interpolated edge using different interpolation kernels.

The present invention discloses an architecture and a method that, particularly on oblique edges, reduces both the edge j aggy and blurring effects.

A method for interpolating digital image data with bilinear interpolation is known. The bilinear interpolation equation, however, contains weighting factors that are controlled by the horizontal, vertical, and diagonal edge strength. This method, however, 0 10 still creates jaggies in the interpolated output image. Furthermore, it may move edges 1/2 S.i pixel to the right or left of its original position in the interpolated output image. As a result of this, text on the output image may appear to have non-uniform width.

Another problem in the design of conventional resolution conversion systems is the increase in effort in generating the interpolation kernel coefficients as one wants to 15 speed up the operation by performing a number of multiplications between pixel values and the interpolation kernel coefficients. If the interpolation kernel coefficients are fetched from look-up tables (LUT's) for example, the increase in the number of S"multiplications in parallel means an equal increase in the number of read ports in the look-up tables. If the interpolation kernel coefficients are calculated by arithmetic units, the increase in the number of multiplications in parallel means an equal increase in the number of such arithmetic units. It is therefore an object of the present invention to reduce the effect in generating kernel coefficients (in terms of the number of ports in the look-up tables or the number of arithmetic units required to calculate the coefficients) while a number of multiplications between pixel values and interpolation kernel coefficients are executed in parallel.

Summary of the Invention According to one aspect of the invention there is provided a method of performing interpolation on a pixel-based image, the method comprising the following CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF

((I

-3steps: determining a scaling factor of a first image compared to a second image; (ii) detecting incoming pixel values of said first image; (iii) storing said incoming pixel values in a plurality of line stores; (iv) determining a current plurality of filter function coefficients for a filter function based on said scaling factor; storing said current plurality of filter function coefficients in a plurality of coefficient stores; (vi) manipulating said coefficient stores to generate one column of filter function coefficients for said filter function; :(vii) multiplying said incoming pixel values with said one column of filter function coefficients to form a plurality of product values; S.(viii) adding said plurality of product values to form an accumulated sum; (ix) repeating steps (iv) to (viii) for each column of said filter function; S• 15 generating an interpolated pixel value based on said accumulated sum; and (vii) storing said interpolated pixel value in one of a plurality of lines stores to generate said second image.

oooo° According to another aspect of the invention there is provided an apparatus for i performing interpolation on a pixel-based image, the apparatus comprising: means for determining a scaling factor of a first image compared to a second image; means for detecting incoming pixel values of said first image; means for storing said incoming pixel values in a plurality of line stores; means for determining a current plurality of filter function coefficients for a filter function based on said scaling factor; means for storing said current plurality of filter function coefficients in a plurality of coefficient stores; means for manipulating said coefficient stores to generate one column of filter function coefficients for said filter function; CFP1536AU (482502) [O:\CISRA\RC\RC05j482502.doc:SaF means for multiplying said incoming pixel values with said one column of filter function coefficients to form a plurality of product values; means for adding said plurality of product values to form an accumulated sum; means for generating an interpolated pixel value based on said accumulated sum; and means for storing said interpolated pixel value in one of a plurality of lines stores to generate said second image.

According to still another aspect of the invention there is provided a method of performing interpolation on a pixel-based image, the method comprising the steps of: 10 determining a scaling factor of a first image compared to a second image; (ii) detecting incoming pixel values of said first image; •o -(iii) storing incoming pixel values in a plurality of line stores; (iv) determining a plurality of filter function coefficients for a first filter function based on said scaling factor; storing said plurality of filter function coefficients in a first plurality of coefficient stores; (vi) determining a current filter function coefficient for a second filter function; S"(vii) storing said current filter function coefficient in a second plurality of coefficient stores; (viii) multiplying said filter function coefficients in said first plurality of coefficient stores with said current filter function coefficient in said second plurality of coefficient stores to form a first plurality of product values; (ix) multiplying incoming pixel values with said first plurality of product values to form a second plurality of product values; adding said second plurality of product values to form an accumulated sum; (xi) repeating steps (vi) to for all coefficients of said second filter function to generate an interpolated pixel value based on said accumulated sum; and (xii) storing said interpolated pixel value in one of a plurality of lines stores to generate said second image.

CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF According to still another aspect of the invention there is provided an apparatus for performing interpolation on a pixel-based image, the apparatus comprising: means for determining a scaling factor of a first image compared to a second image; means for detecting incoming pixel values of said first image; means for storing incoming pixel values in a plurality of line stores; means for determining a plurality of filter function coefficients for a first filter function based on said scaling factor; means for storing said plurality of filter function coefficients in a first plurality of coefficient stores; .i means for determining a current filter function coefficient for a second filter function; means for storing said current filter function coefficient in a second plurality of coefficient stores; means for multiplying said filter function coefficients in said first plurality of coefficient stores with said current filter function coefficient in said second plurality of coefficient stores to form a first plurality of product values; means for multiplying incoming pixel values with said first plurality of product values to form a second plurality of product values; means for adding said second plurality of product values to form an accumulated sum; means for generating an interpolated pixel value based on said accumulated sum; and means for storing said interpolated pixel value in one of a plurality of lines stores to generate said second image.

According to still another aspect of the invention there is provided a method of performing interpolation on a pixel-based image, the method comprising the steps of: determining a scaling factor of a first image compared to a second image; determining a plurality of filter function coefficients based on said scaling factor; CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -6detecting incoming pixel values of said first image; storing said incoming pixel values in a plurality of line stores; interpolating said pixel values of said first image with one of a plurality of filter functions, utilising said filter function coefficients, for each pixel value of said first image; and generating a second image using said interpolated pixel values.

According to still another aspect of the invention there is provided an apparatus for performing interpolation on a pixel-based image, the apparatus including: means for determining a scaling factor of a first image compared to a second image; means for determining a plurality of filter function coefficients based on said scaling factor; means for storing said filter function coefficients; means for detecting incoming pixel values of said first image; means for storing said incoming pixel values in a plurality of line stores; means for interpolating said pixel values of said first image with one of a plurality of filter functions, utilising said stored filter function coefficients, for each pixel S•value of said first image; and means for storing pixel values of said second image including said interpolated o 20 pixel values.

According to still another aspect of the invention there is provided a method of performing interpolation on a pixel-based image, the method including the following steps: determining a scaling factor of a first image compared to a second image; determining a plurality of filter function coefficients based on said scaling factor; detecting incoming pixel values of said first image; storing said incoming pixel values in a plurality of line stores; determining an orientation angle of said detected incoming pixel values; CFP1536AU (482502) [0:\CIS RA\RC\RC05]482502.doc: SaF

C.

-7interpolating said pixel values of said first image with one of a plurality of filter functions utilising said stored filter function coefficients, depending on said orientation of the corresponding input pixel value, for each pixel value of said first image; and generating a second image using said interpolated pixel values.

According to still another aspect of the invention there is provided a computer readable memory medium for storing a program for an apparatus which processes data, said processing comprising a method of performing interpolation on a pixel-based image, said program comprising code for each of the following steps: determining a scaling factor of a first image compared to a second image; (ii) detecting incoming pixel values of said first image; .(iii) storing said incoming pixel values in a plurality of line stores; (iv) determining a current plurality of filter function coefficients for a filter function based on said scaling factor; storing said current plurality of filter function coefficients in a plurality of coefficient stores; (vi) manipulating said coefficient stores to generate one column of filter function coefficients for said filter function; oC...

S"(vii) multiplying said incoming pixel values with said one column of filter function coefficients to form a plurality of product values; (viii) adding said plurality of product values to form an accumulated sum; (ix) repeating steps (iv) to (viii) for each column of said filter function; generating an interpolated pixel value based on said accumulated sum; and (vii) storing said interpolated pixel value in one of a plurality of lines stores to generate said second image.

According to still another aspect of the invention there is provided a computer readable memory medium for storing a program for an apparatus which processes data, said processing comprising a method of performing interpolation on a pixel-based image, said program comprising code for each of the following steps: determining a scaling factor of a first image compared to a second image; CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -8- (ii) detecting incoming pixel values of said first image; (iii) storing incoming pixel values in a plurality of line stores; (iv) determining a plurality of filter function coefficients for a first filter function based on said scaling factor; storing said plurality of filter function coefficients in a first plurality of coefficient stores; (vi) determining a current filter function coefficient for a second filter function; (vii) storing said current filter function coefficient in a second plurality of coefficient stores; (viii) multiplying said filter function coefficients in said first plurality of :coefficient stores with said current filter function coefficient in said second plurality of "coefficient stores to form a first plurality of product values; (ix) multiplying incoming pixel values with said first plurality of product values to form a second plurality of product values; 15 adding said second plurality of product values to form an accumulated sum; (xi) repeating steps (vi) to for all coefficients of said second filter function to generate an interpolated pixel value based on said accumulated sum; and (xii) storing said interpolated pixel value in one of a plurality of lines stores to generate said second image.

According to still another aspect of the invention there is provided a computer readable memory medium for storing a program for an apparatus which processes data, said processing comprising a method of performing interpolation on a pixel-based image, said program comprising code for each of the following steps: determining a scaling factor of a first image compared to a second image; determining a plurality of filter function coefficients based on said scaling factor; detecting incoming pixel values of said first image; storing said incoming pixel values in a plurality of line stores; CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF interpolating said pixel values of said first image with one of a plurality of filter functions, utilising said filter function coefficients, for each pixel value of said first image; and generating a second image using said interpolated pixel values.

Brief Description of the Drawings Embodiments of the invention are described with reference to the drawings, in which: Figs. l(a) to l(d) shows the effect of jaggies on an interpolated edge using different interpolation kernels; Fig. 2 shows a contour plot for a conventional 2-dimensional kernel, with parameters b 0 and c o* Fig. 3 shows offset values to the sum of the fractional parts of the x and y o. ,o coordinates at various positions in the 2-dimensional interpolation kernel in accordance with an embodiment of the present invention; 15 Fig. 4 shows offset values to the difference of the fractional parts of the x and y coordinates at various positions in the 2-dimensional interpolation kernel in accordance with a preferred embodiment of the present invention; Fig. 5 is a schematic block diagram showing an interpolator according to the preferred embodiment of the present invention; Fig. 6 shows the logic and control unit ofFig. 5 in more detail; Fig. 7 is a flow diagram showing the overall control flow of the logic and control unit of Fig. 6; Fig. 8 shows the line stores of Fig. 5 in more detail; Fig. 9 shows the pixel interpolator of Fig. 5 in more detail; Fig. 10 is a flow diagram showing the operation of the pixel interpolator of Fig. 9 when it is interpolating with normal kernels; Fig. 11 is a flow diagram showing the operation of the pixel interpolator of Fig. 9 when it is interpolating with a steerable kernel; CFP1536AU (482502) [0:\CISRA\RC\RC051482502.doc:SaF Fig. 12 is a diagram showing a general purpose computer upon which the interpolator of Fig. 5 can be implemented.

Detailed Description Where reference is made in any one or more of the drawings to steps and/or features, which have the same reference numerals, those steps and/or features are for the purposes of the description the same, unless specifically indicated.

The preferred embodiment is an architecture and a method of performing a twodimensional interpolation using steerable kernel with less hardware working in parallel than a conventional system. The preferred embodiment, particularly on oblique edges, reduces both the edge jaggy and blurring effects. The preferred embodiment interpolates the input pixel values by first storing the pixels in a plurality of line stores, and then feeding pixel values from these line stores to a configurable network of latches, adders and multipliers. The configuration of the latches/adder/multipliers network depends on the type of steerable kernel used. The method described can work with quadratic, cubic, S 15 sinc or similar continuous interpolation kernel.

The preferred embodiment is based on the observation that some of the subcoefficients that are multiplied together to obtain the steerable kernel coefficients form a shifting pattern in the 2-dimensional interpolation kernel. By making use of this observation, the effort of generating the interpolation kernel coefficients needs only to be doubled to handle a steerable kernel of any size.

When interpolating in an edge region of an image it is important to smooth along the edge, but not across the edge. In this way the edge contour is kept smooth, whilst the edge transition is kept sharp.

Initially the case of a 1-dimensional continuous kernel convolution will be described.

Letj(kAt) (k 2, 0, 1, 2, be the samples of a continuous function where At is the (constant) sampling rate. Interpolation with a continuous kernel gives a continuous function that is an approximation of and is given by the finite convolution sum: CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -11 g(t) F f(kAt)h(t kAt) (1) where h(t) is the continuous interpolation kernel. In the preferred embodiment, the cubic interpolation kernel defined in Equation is used, though quadratic, linear, and nearest neighbour kernels can also be used. The two-parameter Catmull-Rom cubic has a kernel of the form: b-c)s 3 2b+c)Is1 2 sl 2 3 h(s) c)s 3 5c)s| 2 (-2b 8c)Is|+(4b+4c), l<js 2 6 3 0, Otherwise (2) where s tlAt is a normalised coordinate that has integer values at the original sample points. Choosing the parameters b and c (b 0, c the interpolating cubic agrees with the first three terms of the Taylor series expansion of the original function, and (b 10 1, c 0) which is the approximating cubic B-spline. All of the kernels used are, by definition, of finite support and symmetrical about s 0. With b 0 the cubic kernel can be shown to be an interpolating kernel, ie. the interpolated function passes through the sample points of the original function, as h(0) 1, while h(kAt) 0 for k 0 which ensure that g(k-t) f(kAt). From Equation any sample of g(t) can be calculated using the 15 continuous kernel, and a finite number (say 4) of original sample points, f(k At).

For example, the conventional 1-dimensional cubic kernel (b 0, c 0.5) requires the two nearest original samples on either side of the point being interpolated are required.

The extension of the cubic kernel to 2-dimensions is the separable extension.

This means that the convolution kernel can be applied to the image data in one of two ways: applying the 1-dimensional convolution kernel to the rows of the image and then using these interpolated values to interpolate along the columns of the image (or vice-versa). For example, the conventional cubic kernel has four coefficients and requires the four nearest samples for both the rows and the columns in turn. This technique however, has the disadvantage of requiring intermediate storage of interpolated values, which may make it unsuitable for a hardware implementation.

CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -12- (ii) a 2-dimensional kernel can be generated and then convolved directly with the image data. The 2-dimensional kernel is generated using matrix multiplication of the coefficient values calculated for the rows and the columns separately. For example, the conventional bi cubic kernel has 16 (4x4) coefficients while the bilinear kernel has four, (2x2). These 2-dimensional blocks of coefficients are then convolved directly with the same size block of nearest neighbour original image pixels around the sample being interpolated. This method has the advantage of not requiring any intermediate storage, but does require more multiplication operations. The method also has the advantage of allowing the implementation of non-separable kernels, which could not be implemented on the rows and columns separately.

.i A contour plot for the conventional 2-dimensional cubic kernel, with parameters ooo O, c is shown in Fig. 2. It can be seen that the kernel primarily uses pixel values from the horizontal and vertical directions relative to the interpolated pixel (which would be at the centre of the kernel). That is, the coefficient values from any diagonal of 15 the kernel are approximately zero and therefore pixel values in these areas have little effect in the convolution sum of Equation This results in the jaggies and blurring effects already discussed.

o Unless otherwise stated the following discussion shall refer to an interpolating cubic kernel with parameter b 0. For the cases of horizontal or vertical edges we can effectively use different cubic kernels in each of the directions by choosing different values of the c parameter. For example, a kernel suitable for interpolating vertical edges can be obtained if values of c x 0 in the horizontal direction and Cy -0.5 in the vertical direction are used. This kernel is two coefficients wide and four coefficients high and is the orientation 7r/2 (900) kernel shown in Fig. 2. A kernel suitable for interpolating horizontal edges is obtained by using values c x -0.5 and Cy 0.

The preferred embodiment utilises interpolation kernels not only for horizontal and vertical (r7c/2) edges, but that can also be oriented at angles of 450 (7n/4) and 135' (37t/4). Steerable kernels with orientations of 0, 7c/2, rt/4 and 37r/4 are: CFP1536AU (482502) [0:\CISRA\RC\RC05]482502.doc:SaF -13h(sxsY)OIh(-Sx)c=0 .h(sy)c=o} Y =-J2} h~sxs Y0=72 =I Sx)c=L h(sY )c=0.51 12--J h(sx,sy)0=/7 4 7L h(Y 2 c=0.5

C=

S

*5 S

S

h(s,sy)0=370 I .x s h T2 V/ 2 c= 0 where sx and Sy are re-sampling distances in the horizontal and vertical directions, respectively, and. indicates multiplication.

From the Catmull-Rom cubic equation any output pixel values can be calculated using one of the continuous kernels above, and a finite number (say 16) of original simple points. Let (x 0 ut, Yout) be the coordinates of the output pixel that we want to 10 estimate by interpolation with one of the kernels above, and let Pout be the desired output pixel value. Let xjfntout be the integer part of x,,u and y into,, be the integer part of yout. Let xffracout be the fractional part of xut, and yfracout be the fractional part of Yout. Let Pi(xy) be the input pixel value of coordinate Applying the steerable kernel on the input pixel values, the following equation is obtained: Pout h(xfracout lyJracout l)pj(xjintut lyintout 1) h(xnracout 1 ,yfacot)pin(x int 0 ut 1yintout) h(xnracout lyfracout 1)p 1 n(x int 0 ut ,yintout h(xfracout 1,yfracout 2)pn(x intOut 1,yintout h(xnracut,yrac 0 ut l)pi,(x intutyintout 1) h(x-fracutyfracoudxinx intou h(xJ'racOut,yfracOut l)pin(xintutyint 0 ut 1) h(xfracOut,yfraCut 2lPin()_intutyintut 2) h(xfracout 1 y-fracut 1)pin(x-int 1+ ,Y intout 1) h(x.fracout 1,Yjfacout)pin(x ntout 1y-intout) CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF 14 h(xjfracout 1 ,YJ'racout 1)Pin(xintout ly intout 1) h(x~fracout 1 yjfracout 2)Pin(xi _ntout 1 y jfltout 2) h(xfracout 2 ,y~racout l )Pi&n~tout 2,yintout 1) h(x~fracout 2 ,yjracout)pn(x ifltout 2,yintout) h(x~fracout 2,y~fracout +lI)pi(x intout 2,y ifltout +1) h(xfacout 2 y~fracout 2)pin(xjntout 2,y_intout For orientations of 0 and 7E/2, the above equation can be regrouped and becomes the following equation: Pout 10 0**Q h(x~fracout 1) h(y~fracout 1)Pin(xintout lYjfltout -1) h(y-fracout)Pin(x int 0 t1,y not h(y~fracout 1 Pin@ ifltout lYjfltout 1) h(y~fracout 2)Pin(xintout lyinou 2) h(xJ'racout) I h(y~fracout 1 )Pin(x~J fltout jYjntout 1) h(y-fraC 0 otdPin(xintot ,Yifltout) h(y~fracout 1 )pin(x intout ,y~intout 1) h(y~fraco ut 2 )pinox intout ,y-intout 2) h(xJ'racout+1) h(y~fracout 1)pi,(xintout 1,yiltout) 1) h(y-fraCoutdPin(xintout 1 y intout) h(yJ'racout l )Pin(x-intout IJ'ylintout 1) h(yfracout 2 )Pin(x intout lY -fltout 2 h(x-fracout+2) h(y~fracout I)pin(Xifltout 2,y lfltout 1) h(y-fraC 0 odPin(xintout 2,y-intout) h(yjfracout Opn- Jtu 2,y_intout 1 h(y~fracout 2 )Pin(x) intout 2,y_intout and for orientations of iT/4, the above equation becomes the following equation: Pout h'(xjfracout yJ'racout 2) h "(xfracout Yjfracout)Pinoxintout 1 lflitout ht(xjfracout yfracout 1) h "(xjfracout y~fracout D)Pin(xifltout ,yinto CFP1536AU (482502) [0:\CISRA\RC\RC0514825 1) ut) 02.doc:SaF h'(xjrac,,t yJrac.t) h'(xfracut yfracout 1) h'(xfracut yjracout 1) h'(xfracut yJracout) h'(xfracut yjracout 1) h'(xfracut yjracout 2) h'(xfrac,,t yracout) h'(xfracut yjracout 1) h'(xfracut yracout 2) 10 h'(xfracut yjracout 3) h'(xfracut yjracout 1) hY(xjracout y-racout 2) h'(xfrac 0 ut yjracout 3) h'(xfracut y-racout 4) h "(xracout yracout 2)pj,(xjintout 1,y intout 1) h "(xjracout yjracout 3)pin(xintout 1 ,y intout 2) h "(xfracout yJracout +1 pin(xjint 0 ty int 0 ut h "(xracout y-ract)pin((x intoutyintout) h "(xracout yjracout 1)pin(x int 0 uty int 0 ut 1) h "(xracout yjracout 2lPin(__intutyint ut 2) h "(xracout yJracout 2)pin(x-int 0 ut 1 yintout h "(xracout yJracout 1)p (x int 0 ut ,yintot) h "(xracout y-rac 0 otuin(x int 0 ut lyintout 1)+ h "(xjracout yjracout 1 p 1 n(x intout 1 yintout 2) h "(xjracout y-racout 3)pin(xjint 0 ut 2,yintout 1) h "(xracout yJracout 2)pin(xintout 2,yintot) h "(xjracout yjracout 1)pin(xintout 2,yintout 1) h "(xjracout yfacout)pin(x intout 2 ,yintout 2),

'S

*5

S

55

S

S.

S

*5r*

C

*55* where is h(x/2) and is h(xI\2). Similar expression can be derived for interpolation with steerable kernel with orientation of 37c/4. Fig. 3 shows what the offsets to the sum of the fractional parts of the x and y coordinates are at various positions in the 2-dimensional interpolation kernel, and the arrows show how the offsets are shifted one place up as we move 20 one column to the right. Fig. 4 shows what the offsets to the difference of the fractional parts of the x and y coordinates are at various positions in the 2-dimensional interpolation kernel, and the arrows show how the offsets are shifted one place down as we move one column to the right.

Fig. 5 is a schematic block diagram showing an interpolator according to the present invention. The interpolator 500 receives digital image pixel data, for example, from a frame store memory (not shown), one row at a time via line 513, and produces interpolated digital image data one row at a time which can be supplied to a printer or a monitor via line 515. The interpolator 500 includes a logic and control unit 502 which calculates a horizontal sampling increment hH and vertical sampling increment hv for the CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -16output image and controls the overall operation of the interpolator 500 via a control bus 521. The sampling increments h are determined by the input image width and height, and the output image width and height. The interpolator 500 also includes a pixel interpolator 504, a plurality of line stores 508, an interpolation kernel generator 510, and an output buffer 512. In operation, the logic and control unit 502 receives instructions, for example, from a central processing unit (see Fig. 12) or graphics processor (not shown) via lines 517 and/or 519, regarding the sizes of input and interpolated output images, and pixel-bypixel interpolation kernel selection scheme. The line stores 508 accept incoming pixel data row-by-row and store them in internal buffers in such a fashion that the last one or S 10 more consecutive lines are in the internal buffers. Internal logic in the line stores 508 also retrieves pixel values in the required horizontal coordinate in those buffers containing the ooe• last plurality of consecutive lines and provides them to the pixel interpolator 503. At the 9 same time, the interpolation kernel generator 510 accepts the fractional parts of the x and too* y coordinates of the interpolated pixel and kernel selection information from the logic and S 15 control unit 502, via line 523, and generates the required interpolation kernel coefficients.

9* The pixel interpolator 504 receives raw pixel data from the line stores 508, via lines 525-1 to 525-n, and the interpolation kernel coefficients from the interpolation kernel So generator 510. The pixel interpolator 504 then performs interpolation on the raw pixel data, and produces rows of interpolated values to the output buffer 512 via line 535, •o o 20 which is double buffered so that a row of interpolated pixels values can be read out while the next row is being read into the output buffer 512. Double buffering is produced as is known in the art by a pair of buffers and multiplex switches.

The structure and operation of the logic control unit 502 will now be described with reference to Figs. 6 and 7. The logic and control unit 502, seen in Fig. 6, includes a control logic module 602, a row counter 604, a pixel counter 603, a sampling increment calculator 608, an X coordinate calculator 610, and an Y coordinate calculator 612.

Operation is shown in Fig. 7 and begins at steps 702 and 704 where all modules are stalled by control logic 602 via a signal on line 614. The contents of pixel counter 603, row counter 604, X coordinate calculator 610 and Y coordinate calculator 612 are reset to CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -17- 0, and the horizontal sampling increment hH and the vertical sampling increment h v are calculated using the following equations: hH input image width/output image width h v input image height/output image height.

Next, at step 706, when enough valid pixels have been received to perform interpolation (indicated by deassertion of the stall signals from line stores 508 and valid pixel signal), the control logic module 602 signals the pixel interpolator 504, via line 521, to begin interpolation, and the kernel generator 510 to begin generating kernel coefficients. After both the modules 504 and 510 signal that they have finished, the 10 control logic module 602 deasserts the stall signal to all modules. The process continues go at step 708, where the pixel counter 603 asks the X coordinate calculator 610, via line 0S*S oooo °o6o 616, to add hH to the current x coordinate via line 618. When the number of output S. interpolated pixels reaches the output image width at step 710, the pixel counter 606 asks sees the row counter 604 to increment its content, via line 620. At the same time the X 4.00:6,

OS@S

o 15 coordinate calculator 610 is reset to 0, and h v is added to the content of Y coordinate °64:4 o o0 calculator 612 via line 622, at step 712. This process continues until the content of row counter 604 reaches output image height, at step 714. Next at step 716, the control logic ooooo S° 602 detects whether there is any more incoming pixel data. If there more incoming pixel data, the contents of row counter 604, pixel counter 603 and X and Y coordinate 0050 ooo• 20 calculators 610,612 are reset to 0, and the whole process is repeated again. If there is no more pixel data coming in, the whole logic and control unit 602 becomes idle.

The line stores 508 will now be described in more detail with reference to Fig. 8.

The line stores include a plurality of buffers 801-1 to 801-m, an input de-multiplexer 810, an output multiplexer 812 and a modulo calculator 814. The modulo calculator 814 calculates the value of current_row modulo k+l, where currentrow is provided from the logic and control unit 102 and k is the number of columns required to perform interpolation. The m output of the modulo calculator 814 is also input, via line 818, to the input de-multiplexer 810 and the output multiplexer 812. The input de-multiplexer 810 CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -18writes the incoming pixels to the mth buffer. The nth output of the output multiplexer selects the output of the pth buffer, where p is modulo k.

The buffer controller 816 controls the read/write operations to the internal buffers, 801-1 to 801-m, via line 820. When the buffer controller 816 receives a signal from the logic and control unit 502 to start via line 521, the buffer controller 816 reads from the internal buffers 801-1 to 801-m the first column of pixel values that are needed to interpolate the output pixel at the current row and current column. (In the case of cubic interpolation, the first column of pixel values needed are at the (current_column-)th column in the internal buffers 801-1 to 801-m). At the same time, the buffer controller 10 816 asserts its stall signal to the logic and control unit 502, via line 521, indicating that the buffer controller 816 is busy processing the current request. After that, the buffer O: controller 816 will continue to fetch the second column of pixel values needed for interpolation, and so on, until all pixel values required for interpolating the output pixel at the current row and current column are supplied to the pixel interpolator 504. Then the 15 buffer controller 816 will deassert the stall signal to the logic and control unit 502, indicating that the buffer controller 816 has finished, and is ready to accept the next request.

Concurrently, the buffer controller 816 determines where the next empty slot is in the incoming line buffer, and writes the incoming pixel into that slot. Also, the buffer controller 816 determines whether all rows required to interpolate the output pixel at the current row have arrived in the line stores. (In the case of cubic interpolation, the buffer controller 816 checks whether the (current_row-l)th, current_rowth, (current_row+l)th, and (currentrow+2)th rows are there in the line stores). If not all the rows are there, it will assert its stall signal to the logic and control unit 502.

The pixel interpolator 504 will now be described in more detail with reference to Fig. 9. The pixel interpolator 504 includes 2k multipliers, 901-1 to 901-k and 902-1 to 902-k, configured in two columns, where k is the number of columns in the interpolation kernel. The pixel interpolator 504 also includes 2k coefficient latches, 938-1 to 938-k and 950-1 to 950-k, connected to the inputs of the first column of multipliers 901-1 to 901-k, CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -19which feeds kernel coefficients into the multipliers 901-1 to 901-k, and 2k-2 multiplexers 920-1 to 920-(2k-2) connected to the input of the coefficient latches. The second column of multipliers 902-1 to 902-k are connected to the input of an adder 930. The output of the adder 930 is connected to an accumulator register 932. A multiplexer 934 is connected to one input of the adder 930 with the output of the accumulator register 932 feeding back into one input of the multiplexer 934. The pixel interpolator 504 also includes a pixel interpolator control unit 936 which controls the overall operation of the pixel interpolator 504, via the control line 521. Initially, the pixel interpolator control unit (PICU) 936 waits for the logic and control unit 502 to signal the start of pixel interpolation, via line S 10 521. When the pixel interpolator control unit (PICU) 936 has started, the PICU 936 *configures the multiplexers, 920-1 to 920-(2k-2), according to the kernel selection S•information. Depending on the type of steerable kernel that it is interpolating with, the multiplexers, 920-1 to 920-(2k-2), can be configured so that all the coefficient latches 938- 1 to 938k, and 950-1 to 950-k, get the same value from the kernel generator 510, or they S 15 would form a shift register. Multipliers in the first column 901-1 to 901-k multiply the kernel coefficients in the first column of kernel coefficient latches 938-1 to 938-k with the kernel coefficients in the second column of kernel coefficient latches 950-1 to 950-k.

Multipliers in the second column 902-1 to 902-k multiply the products from multipliers 901-1 to 901-k in the first column with pixels (Po-Pk-l) from the line stores 508. The adder 932 adds all products from the multipliers 902-1 to 902-k in the second column together and the output of the accumulator register 932. The output of the adder 930 is stored in the latch 934, which is fed back as one of the inputs of the adder 930, forming an accumulator adder.

The multiplexers 920-1 to 920-(2k-2) before the coefficient latches, 938-1 to 938k and 950-1 to 950-k,,can be configured in one of two ways. When the interpolator 504 is interpolating for normal kernels that are separable in the horizontal and vertical directions, multiplexers in the first column select the outputs from the next kernel coefficient latches 938-1 to 938-k, while multiplexers in the second column select the inputs from the kernel coefficients generator 510. When the interpolator 504 is interpolating with a steerable CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF kernel with orientation of 7t/4 or 37c/4, both columns of the kernel coefficient latches, 938- 1 to 938-k and 950-1 to 950-k, are configured to become shift registers.

The operation of the pixel interpolator 504 when interpolating with normal kernels that are separable in the horizontal and vertical directions will now be further described with reference to Fig. 10. When the pixel interpolator 10 is signalled to start, at step 1002, by the logic and control unit 502, the pixel interpolator 504 will wait for the kernel generator 510 to generate kernel coefficients for the vertical kernel, and latch them into the first column of the coefficient latches 938-1 to 938-k at step 1004. The 1St coefficient generated from the kernel generator 510 should correspond to the 1St kernel 10 coefficient in the vertical kernel matrix. (In the case of cubic interpolation, the 1 st coefficient corresponds to the value of the kernel when the pixel offset is fractional part of S y coordinate When all the coefficients in the vertical kernel are latched into the 1 st column at step 1006, the kernel generator 510 switches to generate the first kernel coefficient in the horizontal kernel, with the fractional part of the current x coordinate.

(In the case of cubic interpolation, the first coefficient corresponds to the fractional part of :the current x coordinate These coefficients are latched into all latches 950-1 to 950-k in the second column at step 1008. At the same time, the pixel interpolator 504 latches pixels from the line stores 508. The coefficients Bo-Bkl 1 in the coefficient latches 938-1 to 938-k in the first row are multiplied with the coefficients Co-Ck_ 1 in the coefficient latches 950-1 to 950-k in the second row to get the actual coefficients in first column in the 2-dimensional interpolation kernel, and the products are multiplied with the pixel data Po-Pk-1l These products are then added together to form the first partial sum in the accumulator.

At the second cycle the kernel generator 510 calculates the second coefficient in the horizontal kernel. The pixel interpolator 504 latches it and interpolates the pixels from the line stores 508 with kernel coefficients in the second column. The accumulator multiplexer 934 selects the output of the latch 932 at step 1010, and the accumulator contains the partial result of the interpolated output with the first two columns in the 2- CFP1536AU (482502) [0:\CISRA\RC\RC05]482502.doc:SaF -21 dimensional kernel. This process is repeated for all the coefficients in the horizontal kernel at step 1012. After that the accumulator latch 932 contains the interpolated pixel.

The operation of the pixel interpolator 504 when it is interpolating with a steerable kernel with orientation of n/4 or 37r/4 will now be further described with reference to Fig. 11. When the pixel interpolator 504 is signalled to start by the logic and control unit 502, the pixel interpolator will wait for the kernel generator to generate kernel coefficients for the coefficient latches on both columns at steps 1102 and 1104. The 1 st pair of coefficients should correspond to the left most sampling points of the kernel when it is evaluated forfrac x +fracy andfrac x -fracy, wherefrac, is the fractional part of the 10 current x coordinate, and fracy is the fractional part of the current y coordinate. (In the case of cubic interpolation, the 1 st pair of coefficients corresponds to the values of the kernel when the pixel offset is (frac x +fracy 2)/2 and (frac x -fracy-3)/42)). In the next cycle, at steps 1106 and 1114, the kernel generator 510 increments the pixel offsets by one, and generates a pair of new coefficients. The coefficient generated from the sum of 15 the fractional parts of the current x and y coordinates is latched at the bottom of the shift register 952 in the first column, and the old values are shifted up. The coefficient generated from the difference of the fractional parts of the current x and y coordinates is latched at the top of the shift register 932 in the second column, and the old values are shifted down. This continues until all the coefficient latches in the 1 st and 2 nd columns have values for the current x and y coordinates. At that time, the pixel interpolator 504 also latches the values from the line stores 508, and enables the multipliers 901-1 to 901k, adder 930, and the accumulator latch 934 to start interpolating the input pixel values.

The first column of multipliers 901-1 to 901-k multiply the two sets of kernel coefficients together to get the kernel coefficients in the 1 st column of the 2-dimensional steerable kernel with the specified orientation. The second column of multipliers multiply the 2dimensional interpolation kernel values with the pixel value, and the adder 930 adds all the products together. The output of the adder 930 is the partial output result when the input pixel values mapped to the first column of the kernel are interpolated with the coefficients in the first columns. The partial sum is stored in the accumulator.

CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -22- At the next cycle, steps 1106 to 1114, the kernel generator 510 continues to increment the pixel offsets and calculate new coefficients, and the new coefficients are latched into the shift registers. The pixel interpolator 504 then performs the same calculation to interpolate the pixel values from the line stores with kernel coefficients in the second column of the 2-dimensional kernel. The accumulator multiplexer 934 selects the output of the latch 932, and the accumulator contains the partial result of the interpolated output with the first two columns in the 2-dimensional interpolation kernel.

This process is repeated until all coefficients in the 2-dimensional interpolation kernel are multiplied with the corresponding pixel values in the input image. After that the 10 accumulator latch 932 contains the interpolated pixel.

The operation of the interpolation kernel generator 510 will now be described in more detail. The module accepts the current x and y coordinates and the kernel selection information from the logic and control unit 502 and evaluates the kernel coefficients according the factional parts of both coefficients. The kernel generator 510 works in two 15 mode. In the normal mode, where the interpolation kernel is 2-dimensional separable, the kernel generator 510 generates the coefficients in the vertical kernel first, and then the coefficients in the horizontal kernel. In the other mode, when the interpolation kernel is a steerable kernel with orientation of 7/4 or 37/4, the kernel generator 510 evaluates two .coefficients in parallel, one with the operand offrac x +fracy offsetO, and the other with frac x -fracy offsetl, wherefrac x is the fractional part of the current x coordinate, fracy is the fractional part of the current y coordinate, and offsetO and offsetl depend on the width and height of the kernel matrix. (For example, for cubic interpolation, offsetO begins from and continues to be incremented until it is 4. The value of offsetl begins from and continues to be incremented until it is The kernel selection information selects which kernel is to be used to interpolate the current output pixel, and determines the mode under which the kernel generator 510 works. If the module is implemented as a LUT, the kernel selector may select a particular LUT to be used. The fraction parts of the x and y coordinates are then combined to form an address to look up that LUT. If the kernel coefficients are generated by a plurality of multipliers and adders on the fly, the kernel CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -23selector 510 may select some of the inputs of these multipliers/adders. The fraction parts of the x and y coordinates are then fed into multipliers/adders.

Preferred Embodiment of Apparatus(s) The preferred embodiment is preferably implemented as part of a conventional general-purpose computer system, such as the system 1200 shown in Fig. 12, wherein the interpolator 500 of Fig 5 may be implemented as part of a plug-in board. Alternatively, the interpolator 500 may be implemented as part of a video interface 1210 or graphics processor (not shown).

The process described with reference to Fig 5 can also be implemented as 10 software executing on the computer system 1200. In particular, the steps of the method .'.are effected by instructions in the software that are carried out by the computer. The :oo osoftware can be divided into two separate parts; one part for carrying out the method of the preferred embodiment; and another part to manage the user interface between the latter and the user. The software can be stored in a computer readable medium, including the storage devices described below, for example. The software is loaded into the :computer from the computer readable medium, and then executed by the computer. A oo* *computer readable medium having such software or computer program recorded on it is a computer program product. The use of the computer program product in the computer preferably effects an advantageous apparatus for orientating a character stroke or ndimensional finite space curves in accordance with the embodiments of the invention.

The computer system 1200 has a computer module 1202, a video display 1216, and input devices 1218, 1220. In addition, the computer system 1200 can have any of a number of other output devices including line printers, laser printers, plotters, and other reproduction devices connected to the computer module 1202. The computer system 1200 can be connected to one or more other computers via a communication interface 1208c using an appropriate communication channel 1230 such as a modem communications path, a computer network, or the like. The computer network may CFP1536AU (482502) [0:\CISRA\RC\RC05]482502.doc:SaF -24include a local area network (LAN), a wide area network (WAN), an Intranet, and/or the Internet.

The computer module 1202 has a central processing unit(s) (simply referred to as a processor hereinafter) 1204, a memory 1206 which may include random access memory (RAM) and read-only memory (ROM), input/output (IO) interfaces 1208, a video interface 1210, and one or more storage devices generally represented by a block 1212 in Fig. 12. The storage device(s) 1212 can include one or more of the following: a floppy disc, a hard disc drive, a magneto-optical disc drive, CD-ROM, magnetic tape or any S 10 other of a number of non-volatile storage devices well known to those skilled in the art.

S..-Each of the components 1204 to 1212 and the interpolator 500 are typically connected to *OoO* one or more of the other devices via a bus 1214 that in turn includes data, address, and control buses.

The video interface 1210 is connected to the video display 1216 and provides :video signals from the computer 1202 for display on the video display 1216. User input ooo to operate the computer 1202 can be provided by one or more input devices 1208. For example, an operator can use the keyboard 1218 and/or a pointing device such as the mouse 1220 to provide input to the computer 1202. As discussed above, the interpolator 500 may be implemented as part of a video interface 1210.

The system 1200 is simply provided for illustrative purposes and other configurations can be employed without departing from the scope and spirit of the invention. Exemplary computers on which the preferred embodiment can be implemented include the IBM-PC or compatibles, one of the Macintosh (TM) family of PCs, Sun Sparcstation arrangements evolved therefrom or the like. The foregoing are merely exemplary of the types of computers with which the embodiments of the invention may be practiced. Typically, the processes of the embodiments, described herein, are resident as software or a program recorded on a hard disk drive (generally CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF depicted as block 1212 in Fig. 12) as the computer readable medium, and read and controlled using the processor 1204. Intermediate storage of the program and pixel data and any data fetched from the network can be accomplished using the semiconductor memory 1206, possibly in concert with the hard disk drive 1212. Intermediate storage of the program and pixel data and any data fetched from the network may be accomplished using the semiconductor memory 1206, possibly in concert with the hard disk drive 1212.

In some instances, the program can be supplied to the user encoded on a CD-ROM or a floppy disk (both generally depicted by block 1212), or alternatively could be read by the user from the network via a modem device connected to the computer, for o• 0 example. Still further, the software can also be loaded into the computer system 600 from other computer readable medium including magnetic tape, a ROM or integrated circuit, a magneto-optical disk, a radio or infra-red transmission channel between the computer and another device, a computer readable card such as a PCMCIA card, and the Internet and Intranets including e-mail transmissions and information recorded on websites and the like. The foregoing are merely exemplary of relevant computer readable mediums. Other S °computer readable mediums can be practiced without departing from the scope and spirit •of the invention.

The foregoing only describes one embodiment of the present invention, however, modifications and/or changes can be made thereto by a person skilled in the art without departing from the scope and spirit of the invention.

In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including" and not "consisting only of'. Variations of the word comprising, such as "comprise" and "comprises"have corresponding meanings.

CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF

Claims

1. A method of performing interpolation on a pixel-based image, the method comprising the following steps: determining a scaling factor of a first image compared to a second image; (ii) detecting incoming pixel values of said first image; (iii) storing said incoming pixel values in a plurality of line stores; (iv) determining a current plurality of filter function coefficients for a filter function based on said scaling factor; storing said current plurality of filter function coefficients in a plurality of 10 coefficient stores; S (vi) manipulating said coefficient stores to generate one column of filter function coefficients for said filter function; (vii) multiplying said incoming pixel values with said one column of filter function coefficients to form a plurality of product values; (viii) adding said plurality of product values to form an accumulated sum; (ix) repeating steps (iv) to (viii) for each column of said filter function; generating an interpolated pixel value based on said accumulated sum; and (vii) storing said interpolated pixel value in one of a plurality of lines stores to generate said second image.

2. The method according to claim 1, wherein step (vi) includes the following substeps: accessing a plurality of filter function coefficients from said plurality of coefficient stores; and combining said plurality of filter function coefficients from said plurality of coefficient stores with said current plurality of filter function coefficients to generate said one column of filter function coefficients for said filter function. CFP1536AU (482502) [O:\CISRA\RC\RC051482502.doc:SaF -27-

3. The method according to claim 1, comprising the further step of determining an angle of orientation of a plurality of incoming pixel values.

4. The method according to claim 3, wherein said plurality of incoming pixel values defines an edge. The method according to claim 3, wherein said filter function is dependent on said angle of orientation. 10 6. The method according to claim 1 wherein said filter function is of the form: h(sx,sy)O -I h(Sx)c=0.5 h(sy)c=o 1 h s s y)=l2 h s 1 +s (sx s 1 S 1l 1 sx +y sx -Sy h c=0 h(sx'Sy)0=37t/4 42[ h( c=O 2h( 15 and wherein s t At and 0 is the angle of orientation.

7. The method according to any one of claims 1 to 6, further including the step of inputting dimension values of said first image and said second image to be utilised in determining said scaling factor.

8. The method according to claim 7, further including the step of calculating a first sampling increment value, a second sampling increment value, a first sampling limit value and a second sampling limit value, for said first image, using said dimension values.

9. The method according to any one of the preceding claims, further including the step of calculating first and second fractional coordinate values. CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -28- The method according to any one of the preceding claims, wherein said plurality of filter coefficients are calculated simultaneously.

11. The method according to any one of claims 1 to 10, wherein said filter function is a linear function.

12. The method according any one of claims 1 to 10, wherein the filter function is a quadratic function. o:

13. The method according to any one of claims 1 to 10, wherein the filter function is a weighted sinc function. .V 14. The method according to any one of claims 1 to 10, wherein said first image is a 15 colour image.

15. An apparatus for performing interpolation on a pixel-based image, the apparatus •comprising: means for determining a scaling factor of a first image compared to a second image; means for detecting incoming pixel values of said first image; means for storing said incoming pixel values in a plurality of line stores; means for determining a current plurality of filter function coefficients for a filter function based on said scaling factor; means for storing said current plurality of filter function coefficients in a plurality of coefficient stores; means for manipulating said coefficient stores to generate one column of filter function coefficients for said filter function; CFP1536AU (482502) [0:\CISRA\RC\RC05]482502.doc:SaF -29- means for multiplying said incoming pixel values with said one column of filter function coefficients to form a plurality of product values; means for adding said plurality of product values to form an accumulated sum; means for generating an interpolated pixel value based on said accumulated sum; and means for storing said interpolated pixel value in one of a plurality of lines stores to generate said second image.

16. The apparatus according to claim 15, further comprising means for accessing a plurality of filter function coefficients from said plurality Vote to* of coefficient stores; and means for combining said plurality of filter function coefficients from said plurality of coefficient stores with said current plurality of filter function coefficients to generate said one column of filter function coefficients for said filter function. o9*

17. The apparatus according to claims 15 or 16, further comprising means for S°determining an angle of orientation of a plurality of incoming pixel values. S18. The apparatus according to claim 17, wherein said plurality of incoming pixel values defines an edge.

19. The apparatus according to claim 17, wherein said filter function is dependent on said angle of orientation.

20. The apparatus according to claim 15, further including means for inputting dimension values of said first image and said second image for use in determining said scaling factor. CFP1536AU (482502) [0:\CIS RA\RC\RC05]482502. doc:Sa F

21. The apparatus according to claim 20, further including means for determining a first sampling increment value, a second sampling increment value, a first sampling limit value and a second sampling limit value, for said first image, using said dimension values.

22. The apparatus according to any one of claims 15 to 21, further including means for determining first and second fractional coordinate values.

23. The apparatus according to any one of claims 15 to 22, wherein said plurality of filter coefficients are calculated simultaneously.

24. The apparatus according to any one claims 15 to 23, wherein the filter function is a linear function. *606 6S06

25. The apparatus according to any one of claims 15 to 23, wherein the filter @ool function is a quadratic function. *6 S S• 26. The apparatus according to any one of claims 15 to 23, wherein the filter function is a weighted sinc function. 603*

27. The method according to any one of claims 15 to 26, wherein said first image is a colour image.

28. A method of performing interpolation on a pixel-based image, the method comprising the steps of: determining a scaling factor of a first image compared to a second image; (ii) detecting incoming pixel values of said first image; (iii) storing incoming pixel values in a plurality of line stores; (iv) determining a plurality of filter function coefficients for a first filter function based on said scaling factor; CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -31- storing said plurality of filter function coefficients in a first plurality of coefficient stores; (vi) determining a current filter function coefficient for a second filter function; (vii) storing said current filter function coefficient in a second plurality of coefficient stores; (viii) multiplying said filter function coefficients in said first plurality of coefficient stores with said current filter function coefficient in said second plurality of coefficient stores to form a first plurality of product values; multiplying incoming pixel values with said first plurality of product values 10 to form a second plurality of product values; adding said second plurality of product values to form an accumulated sum; (xi) repeating steps (vi) to for all coefficients of said second filter function to generate an interpolated pixel value based on said accumulated sum; and (xii) storing said interpolated pixel value in one of a plurality of lines stores to generate said second image.

29. The method according to claim 28, further including the step of inputting dimension values of said first image and said second image for use in determining said scaling factor. The method according to claim 28, further including the step of calculating a first sampling increment value, a second sampling increment value, a first sampling limit value and a second sampling limit value, for said first image, using said dimension values.

31. The method according to any one of claims 28 to 30, further including the step of calculating first and second fractional coordinate values.

32. The method according to any one of claims 28 to 31, wherein said plurality of filter coefficients are calculated simultaneously. CFP1536AU (482502) [O:\CISRA\RC\RC051]482502.doc:SaF 32-

33. The method according to any one of claims 28 to 32, wherein said filter function is a linear function.

34. The method according to any one of claims 28 to 32, wherein the filter function is a quadratic function. The method according to any one of claims 28 to 32, wherein the filter function is a weighted sinc function. o.

36. The method according to any one of claims 28 to 35, wherein said first image is a .colour image. An apparatus for performing interpolation on a pixel-based image, the apparatus comprising: e• S" means for determining a scaling factor of a first image compared to a second image; means for detecting incoming pixel values of said first image; means for storing incoming pixel values in a plurality of line stores; means for determining a plurality of filter function coefficients for a first filter function based on said scaling factor; means for storing said plurality of filter function coefficients in a first plurality of coefficient stores; means for determining a current filter function coefficient for a second filter function; means for storing said current filter function coefficient in a second plurality of coefficient stores; CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -33- means for multiplying said filter function coefficients in said first plurality of coefficient stores with said current filter function coefficient in said second plurality of coefficient stores to form a first plurality of product values; means for multiplying incoming pixel values with said first plurality of product values to form a second plurality of product values; means for adding said second plurality of product values to form an accumulated sum; means for generating an interpolated pixel value based on said accumulated sum; and 10 means for storing said interpolated pixel value in one of a plurality of lines stores to generate said second image. o

38. The apparatus according to claim 37, further including means for inputting dimension values of said first image and said second image for use in determining said scaling factor.

39. The apparatus according to claim 38, further including means for determining a S"first sampling increment value, a second sampling increment value, a first sampling limit value and a second sampling limit value, for said first image, using said dimension values. The apparatus according to any one of claims 37 to 39, further including means for determining first and second fractional coordinate values.

41. The apparatus according to any one of claims 37 to 40, wherein said plurality of filter coefficients are calculated simultaneously.

42. The apparatus according to any one claims 37 to 41, wherein the filter function is a linear function. CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -34-

43. The apparatus according to any one of claims 37 to 41, wherein the filter function is a quadratic function.

44. The apparatus according to any one of claims 37 to 41, wherein the filter function is a weighted sinc function. The apparatus according to any one of claims 37 to 44, wherein said first image is a colour image. 10 46. A method of performing interpolation on a pixel-based image, the method •comprising the steps of: determining a scaling factor of a first image compared to a second image; determining a plurality of filter function coefficients based on said scaling factor; -detecting incoming pixel values of said first image; storing said incoming pixel values in a plurality of line stores; interpolating said pixel values of said first image with one of a plurality of filter functions, utilising said filter function coefficients, for each pixel value of said first image; and generating a second image using said interpolated pixel values.

47. The method according to claim 46, further including the step of inputting dimension values of said first image and said second image for use in determining said scaling factor.

48. The method according to claim 46, further including the step of calculating a first sampling increment value, a second sampling increment value, a first sampling limit value and a second sampling limit value, for said first image, using said dimension values. CFP1536AU (482502) [0:\CIS RA\RC\RC05]482502.doc: SaF

49. The method according to any one of claims 46 to 48, further including the step of calculating first and second fractional coordinate values using said positional indicators. The method according to any one of claims 46 to 49, said method including the further step of convolving each pixel of said first image with said filter function.

51. The method according to any one of claims 46 to 49, wherein said plurality of filter coefficients are calculated simultaneously. 10 52. The method according to any one of claims 46 to 51, wherein the filter functions are linear functions.

53. The method according to any one of claims 46 to 51, wherein the filter functions are quadratic functions. S" 54. The method according to any one of claims 46 to 51, wherein the filter functions are weighted sinc function. The method according to any one of claims 46 to 54, wherein said first image is a colour image.

56. An apparatus for performing interpolation on a pixel-based image, the apparatus including: means for determining a scaling factor of a first image compared to a second image; means for determining a plurality of filter function coefficients based on said scaling factor; means for storing said filter function coefficients; means for detecting incoming pixel values of said first image; CFP1536AU (482502) [O:\CISRA\RC\RC051482502.doc:SaF -36- means for storing said incoming pixel values in a plurality of line stores; means for interpolating said pixel values of said first image with one of a plurality of filter functions, utilising said stored filter function coefficients, for each pixel value of said first image; and means for storing pixel values of said second image including said interpolated pixel values.

57. The apparatus according to claim 56, further including means for inputting dimension values of said first image and said second image for use in determining said 10 scaling factor.

58. The apparatus according to claim 56, further including means for determining a first sampling increment value, a second sampling increment value, a first sampling limit value and a second sampling limit value, for said first image, using said dimension values.

59. The apparatus according to any one of claims 56 to 58, further including means for determining first and second fractional coordinate values. The apparatus according to any one of claims 56 to 59, further including means for convolving each pixel of said first image with said filter function.

61. The apparatus according to any one of claims 56 to 60, wherein said plurality of filter coefficients are calculated simultaneously.

62. The apparatus according to any one claims 56 to 61, wherein the filter functions are linear functions.

63. The apparatus according to any one of claims 56 to 61, wherein the filter functions are quadratic functions. CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -37-

64. The apparatus according to any one of claims 56 to 61, wherein the filter functions are weighted sinc function.

65. The apparatus according to any one of claims 56 to 64, wherein said first image is a colour image.

66. A method of performing interpolation on a pixel-based image, the method including the following steps: 10 determining a scaling factor of a first image compared to a second image; determining a plurality of filter function coefficients based on said scaling factor; detecting incoming pixel values of said first image; storing said incoming pixel values in a plurality of line stores; determining an orientation angle of said detected incoming pixel values; interpolating said pixel values of said first image with one of a plurality of filter functions utilising said stored filter function coefficients, depending on said orientation of the corresponding input pixel value, for each pixel value of said first image; and generating a second image using said interpolated pixel values.

67. The method according to claim 66, wherein said filter function is of the form: h(sy)c=o h(sx,sy)eo )c h(sxsy) 0 c=0.5 c=0 1 hisx+sy. .h0sxS-Y h(sx,sy)O=3/4 2h c=O and wherein s t t and is the angle of and wherein s t At and 9 is the angle of orientation. CFP1536AU (482502) [0:\CIS RA\RC\RC05]482502.doc: SaF -38-

68. A computer readable memory medium for storing a program for an apparatus which processes data, said processing comprising a method of performing interpolation on a pixel-based image, said program comprising code for each of the following steps: determining a scaling factor of a first image compared to a second image; (ii) detecting incoming pixel values of said first image; (iii) storing said incoming pixel values in a plurality of line stores; (iv) determining a current plurality of filter function coefficients for a filter function based on said scaling factor; storing said current plurality of filter function coefficients in a plurality of 10 coefficient stores; (vi) manipulating said coefficient stores to generate one column of filter function coefficients for said filter function; (vii) multiplying said incoming pixel values with said one column of filter function coefficients to form a plurality of product values; (viii) adding said plurality of product values to form an accumulated sum; (ix) repeating steps (iv) to (viii) for each column of said filter function; 0:00 0• generating an interpolated pixel value based on said accumulated sum; and (vii) storing said interpolated pixel value in one of a plurality of lines stores to 0000 generate said second image.

69. The computer readable memory medium according to claim 68, wherein step (vi) includes the following substeps: accessing a plurality of filter function coefficients from said plurality of coefficient stores; and combining said plurality of filter function coefficients from said plurality of coefficient stores with said current plurality of filter function coefficients to generate said one column of filter function coefficients for said filter function. CFP1536AU (482502) [0:\CISSRA\RC\RC05482502.doc:Sa F -39- The computer readable memory medium according to claim 68, said program comprising code for the further step of determining an angle of orientation of a plurality of incoming pixel values.

71. The computer readable memory medium according to claim 70, wherein said plurality of incoming pixel values defines an edge.

72. The computer readable memory medium according to claim 70, wherein said filter function is dependent on said angle of orientation.

73. The computer readable memory medium according to claim 68, wherein said filter function is of the form: h(sx,sy)e= I {h(sx)c=0.5 h(Sy)c= h(sx,)O= (Sx) C1 }y)1c=O 2 :h(sx,sy)=3n/2= sx)c=sy c=0.5h( S" sx syi sx sy. h(sx,sy)O=t/ 4 h 2 h h(sx, sy)O=3t/4 h c= y2 c and wherein s t At and 0 is the angle of orientation.

74. The computer readable memory medium according to any one of claims 68 to 73, said program comprising code for the further step of inputting dimension values of said first image and said second image for use in determining said scaling factor. The computer readable memory medium according to claim 74, said program comprising code for the further step of calculating a first sampling increment value, a second sampling increment value, a first sampling limit value and a second sampling limit value, for said first image, using said dimension values. CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF

76. The computer readable memory medium according to any one of claims 68 to said program comprising code for the further step of calculating first and second fractional coordinate values.

77. The computer readable memory medium according to any one of claims 68 to 76, wherein said plurality of filter coefficients are calculated simultaneously.

78. The computer readable memory medium according to any one of claims 68 to 77, 10 wherein said filter function is a linear function. ooo•

79. The computer readable memory medium according claim 68, wherein the filter function is a quadratic function.

80. The computer readable memory medium according to claim 68, wherein the filter function is a weighted sinc function. *81. The computer readable memory medium according to claim 68, wherein said first image is a colour image.

82. A computer readable memory medium for storing a program for an apparatus which processes data, said processing comprising a method of performing interpolation on a pixel-based image, said program comprising code for each of the following steps: determining a scaling factor of a first image compared to a second image; (ii) detecting incoming pixel values of said first image; (iii) storing incoming pixel values in a plurality of line stores; (iv) determining a plurality of filter function coefficients for a first filter function based on said scaling factor; CFP1536AU (482502) [0:\CISRA\RC\RC05]482502.doc:SaF -41- storing said plurality of filter function coefficients in a first plurality of coefficient stores; (vi) determining a current filter function coefficient for a second filter function; (vii) storing said current filter function coefficient in a second plurality of coefficient stores; (viii) multiplying said filter function coefficients in said first plurality of coefficient stores with said current filter function coefficient in said second plurality of coefficient stores to form a first plurality of product values; (ix) multiplying incoming pixel values with said first plurality of product values 10 to form a second plurality of product values; adding said second plurality of product values to form an accumulated sum; (xi) repeating steps (vi) to for all coefficients of said second filter function to generate an interpolated pixel value based on said accumulated sum; and (xii) storing said interpolated pixel value in one of a plurality of lines stores to generate said second image. 83 The computer readable memory medium according to claim 82, said program •comprising code for the further step of inputting dimension values of said first image and said second image for use in determining said scaling factor. 84 The computer readable memory medium according to claim 82, said program comprising code for the further step of calculating a first sampling increment value, a second sampling increment value, a first sampling limit value and a second sampling limit value, for said first image, using said dimension values. The computer readable memory medium according to any one of claims 82 to 84, said program comprising code for the further step of calculating first and second fractional coordinate values. CFP1536AU (482502) [0:\CISRA\RC\RC05]482502.doc:SaF 1 -42- 86 The computer readable memory medium according to any one of claims 82 to wherein said plurality of filter coefficients are calculated simultaneously. 87 The computer readable memory medium according to any one of claims 82 to 86, wherein said filter function is a linear function. 88 The computer readable memory medium according to any one of claims 82 to 86, wherein the filter function is a quadratic function. o 10 89 The computer readable memory medium according to any one of claims 82 to 86, •go• S o wherein the filter function is a weighted sinc function. The computer readable memory medium according to any one of claims 82 to 86, wherein said first image is a colour image.

91. A computer readable memory medium for storing a program for an apparatus which processes data, said processing comprising a method of performing interpolation on a pixel-based image, said program comprising code for each of the following steps: determining a scaling factor of a first image compared to a second image; determining a plurality of filter function coefficients based on said scaling factor; detecting incoming pixel values of said first image; storing said incoming pixel values in a plurality of line stores; interpolating said pixel values of said first image with one of a plurality of filter functions, utilising said filter function coefficients, for each pixel value of said first image; and generating a second image using said interpolated pixel values. CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -43-

92. The computer readable memory medium according to claim 91, said program comprising code for the further step of inputting dimension values of said first image and said second image for use in determining said scaling factor.

93. The computer readable memory medium according to claim 91, said program comprising code for the further step of calculating a first sampling increment value, a second sampling increment value, a first sampling limit value and a second sampling limit value, for said first image, using said dimension values. S10 94. The computer readable memory medium according to any one of claims 91 to 93, oo o *-*,said program comprising code for the further step of calculating first and second fractional coordinate values using said positional indicators. The computer readable memory medium according to any one of claims 91 to 94, said program comprising code for the further step of convolving each pixel of said first S. S image with said filter function. The computer readable memory medium according to any one of claims 91 to 94, o• wherein said plurality of filter coefficients are calculated simultaneously.

97. The computer readable memory medium according to any one of claims 91 to 96, wherein the filter functions are linear functions.

98. The computer readable memory medium according to any one of claims 91 to 96, wherein the filter functions are quadratic functions.

99. The computer readable memory medium according to any one of claims 91 to 96, wherein the filter functions are weighted sinc function. CFP1536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF -44-

100. The computer readable memory medium according to any one of claims 91 to 99, wherein said first image is a colour image.

101. A method of performing interpolation on a pixel-based image, substantially as herein described with reference to Figs. 3 to 12.

102. An apparatus for performing interpolation on a pixel-based image, substantially as herein described with reference to Figs. 3 to 12. 10 103. A computer readable memory medium for storing a program for an apparatus o which processes data, said processing comprising a method of performing interpolation O S on a pixel-based image, substantially as herein described with reference to Figs. 3 to 12. 68 Dated 1 December, 1999 Canon Kabushiki Kaisha Patent Attorneys for the Applicant/Nominated Person SPRUSON FERGUSON CFP536AU (482502) [O:\CISRA\RC\RC05]482502.doc:SaF