AU6542500A - Universal linear kernal interpolation - Google Patents

Description

SS&F Ref: 522734

AUSTRALIA

PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT

ORIGINAL

Name and Address of Applicant: Actual Inventor(s): Address for Service: Invention Title: Canon Kabushiki Kaisha 30-2, Shimomaruko 3-chome, Ohta-ku Tokyo 146 Japan Andrew James Dorrell Spruson Ferguson St Martins Tower 31 Market Street Sydney NSW 2000 Universal Linear Keral Interpolation ASSOCIATED PROVISIONAL APPLICATION DETAILS [33] Country [31] Applic. No(s) AU PQ3776 [32] Application Date 29 Oct 1999 The following statement is a full description of this invention, including the best method of performing it known to me/us:- 5815c [R:\LIBW]07456.doc:vsg UNIVERSAL LINEAR KERNAL INTERPOLATION Field of Invention The present invention relates to a method and apparatus for resolution conversion for multi-dimensional image data and in particular digital image data. The invention also relates to a computer program product including a computer readable medium having recorded thereon a computer program for resolution conversion of multidimensional image data and in particular digital image data.

Background There are a number of known methods available for digital data resolution conversion, these including, for example, transform domain methods such as the fractional Fourier transform (fractional FFT or Chirp-Z transform), the discrete cosine transform (DCT), and the discrete wavelet transform (DWT). In addition, there are a number of spatial domain methods such as re-sampling and digital filtering with finite- *."*impulse response (FIR) and infinite-impulse response (IIR) filters and interpolation with 15 continuous, usually cubic, splines.

A significant problem encountered in resolution conversion and in particular, resolution enhancement, or interpolation, is retaining the greatest possible sharpness in an output image. In order to retain the greatest possible sharpness in an output image, the sharpness of edges in the output image must be maintained. A problem occurs where computer generated graphics such as text, icons and web graphics are to be processed, as these frequently contain abrupt edges. Known methods of maintaining edge sharpness include applying a plurality of interpolation techniques to an image, which are adapted to the image content either manually, semi- or fully-automatically. Edge sharpness can be increased by making use of a plurality of interpolation kernels.

i 25 One particular known method of interpolation, which is directed to minimising the degree of blurring of a particular orientation, utilises kernels which are matched to the orientation of dominant edges in the image data such that minimal blurring occurs across the edge, while jagged features which occur along the edge are smoothed. The jagged edges result from resampling. A problem with the method is that the quality of the result is dependent on accurate kernel selection, which is in turn dependent on accurate measurement of edge strength and orientation. The method requires a region of 4x4 pixels to be analysed. The method measures edge strength separately in the horizontal and vertical directions and takes a vector sum to determine the general edge direction within the region. In practice, the method is unable to correctly characterise very fine or rapidly changing image features such as are found in text, computer graphics and icons.

522734.doc In the presence of such features, the method is prone to misclassification leading to a visually inferior interpolation. Australian Patent Application No. PQ3778, filed 29 October 1999, the corresponding complete patent specification of which being filed concurrently with the present specification, discloses a method of minimising the misclassification in two ways. Firstly, an attempt is made to identify and independently process areas of the image which exhibit abrupt edges. Secondly, these areas are interpolated using a kernel which minimises the amount of blurring in all orientations.

While the method does avoid the introduction of artefacts due to incorrect kernel selection, the result represents a poor alternative as the non-directed kernel is neither capable of removing the jagged appearance of output edge features, nor maintain the original edge sharpness. A second problem with the method and similar approaches is the added complexity of the processing pipeline which can make the method inappropriate for certain applications.

Additionally, all of the above mentioned methods involve the application of a 15 linear interpolating operator to local regions of the input image data, which has the disadvantage that new colours are introduced as a result of blurring. When the interpolating kernel is cubic, in addition to blurring, a halo effect can be introduced around interpolated edges. The halo effect is particularly visible when interpolation is performed on an area of image data containing abrupt edges but a flat (near constant eo coloured) background and further detracts from the perceived quality of the interpolated image. Computer "desktop" and slide presentations are typical examples. To solve the problem, further logic is usually required in regions exhibiting more than a certain number of colours in order to maintain the sharpness of natural images. However, the approach of including further logic adds further to the complexity of the algorithm.

25 A computationally efficient alternative is to use a steerable linear kemrnel. A o steerable linear kernel has the advantage of being short, and inexpensive to apply, as well as having the property that it cannot generate intensities which lye outside the bounds of the input intensities. However, effective use of a steerable kernel is prevented by the absence of a method for determining local edge orientation, as discussed above.

It is an object of the present invention to ameliorate one or more of the limitations of the methods described above.

Summary of the Invention According to one aspect of the present invention there is provided a method of determining an orientation angle for a set of discrete sample values, said method comprising the steps of: 522734.doc -3accessing at least one portion of said set; calculating an absolute difference between at least one discrete sample value of said portion and each remaining discrete sample value of said portion; comparing said absolute difference for each remaining discrete sample value of said portion to a threshold value; providing an indicator for each remaining discrete sample value depending on said comparison; and matching said indicator to one of a plurality of orientation angles.

According to another aspect of the present invention there is provided a method of determining pixel edge orientation for a pixel-based image, said method comprising the steps of: determining neighbouring pixels for at least one non-boundary pixel of said pixel-based image; *...calculating an absolute difference between said non-boundary pixel and each o 15 neighbouring pixel; comparing said absolute difference for each neighbouring pixel to a threshold; providing an indicator for each neighbouring pixel depending on said comparison; and matching said indicator to one of a plurality of edge orientations.

20 According to still another aspect of the present invention there is provided a method of determining pixel edge orientation for a pixel-based image, said method comprising the steps of: determining neighbouring pixels for at least one non-boundary pixel of said pixel-based image; i 25 calculating an absolute difference between said non-boundary pixel and each °neighbouring pixel; comparing said absolute difference for each neighbouring pixel to a threshold; providing an indicator for each neighbouring pixel depending on said comparison; and matching said indicator to one of a plurality of edge orientations.

According to still another aspect of the present invention there is provided a method of determining pixel edge orientation for a pixel-based image, said method comprising the steps of: determining neighbouring pixels for each non-boundary pixel of said pixel-based image; 522734.doc determining a difference map using said neighbouring pixels; determining a key for each said non-boundary pixel by applying a threshold to said difference map; and matching said key to one of a plurality of edge orientations.

According to still another aspect of the present invention there is provided a method of performing interpolation on a pixel-based image, said method comprising, for each output pixel location of said pixel-based image, the steps of: determining input pixels which contribute to an output pixel value at said output pixel location; (ii) calculating a kernel value for each of said input pixels using a kernel selected from one of a plurality of kernels, wherein for each non-boundary pixel of said input pixels, said calculating comprising the sub-steps of: determining neighbouring pixels of said non-boundary pixel; determining a difference map using said neighbouring pixels; 15 determining a key by applying a threshold to said difference map; and selecting said kernel for said non-boundary pixel using said key; and (iii) calculating said output pixel value utilising said kernel values.

20 According to still another aspect of the present invention there is provided an apparatus for determining an orientation angle for a set of discrete sample values, said apparatus comprising: access means for accessing at least one portion of said set; calculation means for calculating an absolute difference between at least one 25 discrete sample value of said portion and each remaining discrete sample value of said o portion; comparison means for comparing said absolute difference for each remaining discrete sample value of said portion to a threshold value; indicator means for providing an indicator for each remaining discrete sample value depending on said comparison; and means for matching said indicator to one of a plurality of orientation angles.

According to still another aspect of the present invention there is provided an apparatus for determining pixel edge orientation for a pixel-based image, said apparatus comprising: 522734.doc means for determining neighbouring pixels for each non-boundary pixel of said pixel-based image; means for determining a difference map using said neighbouring pixels; means for determining a key for each said non-boundary pixel by applying a threshold to said difference map; and means for matching said key to one of a plurality of edge orientations.

According to still another aspect of the present invention there is provided an apparatus for performing interpolation on a pixel-based image, said apparatus comprising, for each output pixel location of said pixel-based image, the steps of: means for determining, for each output pixel location of said pixel-based image, input pixels which contribute to an output pixel value at said output pixel location; means for calculating a kernel value for each of said input pixels using a kernel selected from one of a plurality of kernels, wherein for each non-boundary pixel of said input pixels; 15 means for determining neighbouring pixels of said non-boundary pixel; •means for determining a difference map using said neighbouring pixels; means for determining a key by applying a threshold to said difference map; means for selecting said kernel for said non-boundary pixel using said key; and means for calculating said output pixel value utilising said kernel values.

20 According to still another aspect of the present invention there is provided a computer readable medium, having a program recorded thereon, where the program is configured to make a computer execute a procedure to determine an orientation angle for a set of discrete sample values, said program comprising: code for accessing at least one portion of said set; i 25 code for calculating an absolute difference between at least one discrete sample value of said portion and each remaining discrete sample value of said portion; code for comparing said absolute difference for each remaining discrete sample value of said portion to a threshold value; code for providing an indicator for each remaining discrete sample value depending on said comparison; and code for matching said indicator to one of a plurality of orientation angles.

According to still another aspect of the present invention there is provided a computer readable medium, having a program recorded thereon, where the program is configured to make a computer execute a procedure to determine pixel edge orientation for a pixel-based image, said program comprising: 522734.doc code for determining neighbouring pixels for each non-boundary pixel of said pixel-based image; code for determining a difference map using said neighbouring pixels; code for determining a key for each said non-boundary pixel by applying a threshold to said difference map; and code for matching said key to one of a plurality of edge orientations.

According to still another aspect of the present invention there is provided a computer readable medium, having a program recorded thereon, where the program is configured to make a computer execute a procedure to interpolate a pixel-based image, said apparatus comprising, for each output pixel location of said pixel-based image, the steps of: means for determining, for each output pixel location of said pixel-based image, input pixels which contribute to an output pixel value at said output pixel location; means for calculating a kernel value for each of said input pixels using a kernel o, 15 selected from one of a plurality of kernels, wherein for each non-boundary pixel of said So. input pixels; means for determining neighbouring pixels of said non-boundary pixel; means for determining a difference map using said neighbouring pixels; means for determining a key by applying a threshold to said difference map; means for selecting said kernel for said non-boundary pixel using said key; and co means for calculating said output pixel value utilising said kernel values.

0 Brief Description of the Drawings A number of preferred embodiments of the present invention will now be 25 described with reference to the drawings, in which: Fig. l is a flow diagram of a method of an interpolation process; Fig. 2 shows a modified form of a linear interpolating kernel; Fig. 3 shows a flowchart of a method of determining pixel edge orientation for a pixel-based image in accordance with the preferred embodiment of the present invention; Fig. 4 is a flowchart showing a method of performing interpolation on a pixelbased image utilising the method of detennrmining pixel edge orientation of Fig. 3; and Fig. 5 is a schematic block diagram of a general purpose computer upon which the preferred embodiment of the present invention can be practiced; and Fig. 6 shows an example of up-sample and input grids in accordance with the preferred embodiment.

522734.doc Detailed Description including Best Mode Where reference is made in any one or more of the accompanying drawings to steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears.

The preferred embodiment is a method of determining pixel edge orientation for a pixel-based image. The preferred embodiment can be utilised to interpolate image data by adapting an interpolating kernel to local image data and selecting the most appropriate interpolation kernel from a number of available oriented interpolation kernels. The preferred embodiment can be utilised in an interpolation method using a 2-dimensional form of convolution with a continuous kernel. A further interpolation method is described whereby a non-linear weighted mode filter is used and weights are drawn from a continuous kernel definition. Both interpolation methods preferably make use of a look-up table for storing pre-calculated kernels. A kernel set used which is preferably a 15 steerable linear set containing 8 oriented kernels and 2 non-directional kernels.

Generation of the kernel set will be detailed below.

Fig. 1 is a flowchart showing an interpolation process. The process beings at step 101, where the input image is processed in order to generate a "map" image in which intensity values are used to refer to specific edge orientations and consequently, interpolation kernels. The processing of step 101, as it relates to the preferred embodiment, is detailed later in this document. The map image, hereinafter referred to as the "edge map", is then used by the interpolation engine to interpolate the input image at :i the next step 103. The process concludes when the entire input image has been interpolated.

25 The preferred method of determining pixel edge orientation for a pixel-based image in accordance with the present invention is a fast, highly localised method which has sufficient classification resolution to be used with a set of steered linear kernels on detailed regions such as text and graphics. The preferred method results in very low computational overhead in both software and hardware. The preferred method provides a result which is more consistent with what a casual viewer expects to see in an interpolated graphic. Also, the non-linear interpolation method is able to perform a morphologically smooth interpolation without introducing new colours into the output image.

Initially the case of a 1-dimensional continuous kernel convolution will be described. LetflkAt) (k 2, 0, 1, 2, be the samples of a continuous functionfj(t), 522734.doc -8where At is the sampling rate. Interpolation with a continuous kernel gives a continuous function that is an approximation off(t), and is given by the finite convolution sum: g(t) Y f(kAt)h(t kAt) (1) where h(t) is the continuous interpolation kernel. From equation any sample of g(t) can be calculated using the continuous kernel, and a finite number (say 4) of original sampled points, f(kAt).

In accordance with the preferred embodiment, kernels are derived from a modified form of the linear interpolating kernel, depicted in Fig. 2 and defined in equation The kernel is parameterised to include a variable dominant-zone. The parameter d defines the positive width of the zone in which the kernel magnitude is always unity. From the dominant zone the kernel ramps linearly to zero with a slope that guarantees an overall gain of unity. It is worth noting that when d 0, a continuous convolution with the kernel according to equation results in linear interpolation of the input data. When d =0.5 the same convolution results in nearest neighbour 15 interpolation.

1 where s d, 0 where s (2) s d s- d otherwise.

***1-2d A two-dimensional steered linear kernel is defined based on a separable combination of the modified linear kernel defined in equation By setting the d parameter differently in each plane, a directional preference is introduced as the resulting 20 2D kernel has different extent and performs different degrees of smoothing in each plane.

Steered linear kernels can then be calculated for any preferred edge orientation by rotating the basic, directionally biased kernel by any required angle according to equation The kernels that result are non-separable.

hdd(sx,s, W)hd ((1-20/r)s x +(20/7)sy)xhd((20/)s, +(2/7-l)sy) (3) where w(0) 1+ Icos(40) (4) 2 2 The parameter w(O) is referred to as the cross edge weighting factor, and is required to ensure unity gain for the rotated filter.

522734.doc -9- Ten different kernels are preferably used in accordance with the preferred embodiment. Eight of the kernels are preferably steered kernels having d 0 in the direction of the edge and d 0.3 in the direction perpendicular to the edge and steered to orientations in the set: Steer angles 8 8' 2, 58,' /4,78 The remaining two kernels are preferably non-directional: the first, hereinafter referred to as the short kernel, uses d 0.3 in both image planes; the second, hereinafter referred to as the background or long kernel, uses d 0 in both image planes. The short kernel is used in regions where sharp transitions are present but no definitive direction can be identified. The background kernel is used in regions where no edges are found.

The above kernels are exemplary and are not intended to limit the invention.

Different values for the parameter d can be chosen for each kernel direction and, in fact, can be selected in accordance with previously calculated image features, local or global.

The preferred embodiment preferably uses linear kernels in order to minimise 15 computational cost. However, kernels from other families can be utilised. For example, in accordance with another embodiment, a cubic interpolating kernel can be utilised instead of the linear background kernel described above.

1. Fig. 3 shows a flowchart of a method of determining pixel edge orientation for a pixel-based image in accordance with the preferred embodiment of the present invention.

S

20 The process begins at step 310 where (output) boundary pixels are initialised. As there is no complete neighbourhood for boundary pixels the output for these pixels is set to have preferred direction at step 310. Further, to minimise smoothing at image boundaries, a short kernel is preferably used for all boundary pixels. Subsequently, for each pixel that is not a boundary pixel, an immediate neighbourhood region is extracted for processing at the next step 320. The process continues at step 330, where the immediate neighbourhood region is used to calculate a difference map by taking the absolute difference between each neighbour pixel and the centre pixel. The difference map E(ij) in accordance with the preferred embodiment is as follows: E(ij)= x(1,1) (6) for all i and j in the range and (ij) At the next step 340, a threshold is applied to the difference map to provide an 8 bit key. The key contains one bit of information for each of the eight neighbouring pixels of the immediate neighbourhood region, indicating whether the neighbour differs from the centre pixel by the threshold.

The selection of a threshold can be dynamic and preferably adapted to local image 522734.doc properties including those drawn from a larger surrounding area. In accordance with the preferred embodiment, a static threshold of 30 is used (with 8 bit pixel data) and steps 330 and 340 are preferably performed in a single operation.

Once obtained, the 8-bit key is matched to one of a predefined set of edge orientations at the next step 350. The matching can preferably utilise a look-up table.

Alternatively, matching can be implemented using a logic circuit. In accordance with the preferred embodiment, the logic circuit alternative is used, the behaviour of which is defined in Table 1 below. The process continues at the next step 360, where a check is made to see if all non-boundary pixels and their surrounding regions have been processed in the input image.

Orientation Matching keys I 1 1 1i xI 1x I X X 0 o o o 0- 0 011 1 0 0 00 00 0 S1 111 1 x xI x x I III X O 0 X 11 0 000 8 1 0 0 1I0 I1 0 0 0 0 0 0 X I I I X 0 0 0 0 0 1 ox I l lX0I 10 000011 0 a 1 1 0 110 1i i X i XXO 100 xo 4 1 1 1 i x 1 0 0 0 0 153 mtx i o x l1i o o lo oo SX 1 0 1 1 X 0 0 1 1 0 011 oiX 0X i 00 001 000 S1 i 1 1 X x X 0 0 2/ I I I I x x 1 1 1 I I 1 0 1 X X 1 X X 5i ox 1 ,110 oix o

I

o o0 8 1 X ox 10 1 1 0 0 1 o 110 x10 xo1 001 100 000 S3,/ o01 1 x i 0xx o x ol oox 4 1 1 1 1 1 Ox I 0 0 0 0 /4I I I 1 0 X 0 ox o 1 o i7n clx 110o o x IOo o 100 8 0 1 0 1 0 1 0 0 0 0 0 SxXO 00011 X 1 11 000 Table 1 Table 1 also defines the rules preferably used to determine an edge orientation given the eight bit key described above. For clarity, the keys are represented as a square matrix with the position of each bit of information reflecting the relative position of the input (neighbour) pixel from which it is drawn. The contents of the matrix are 1, indicating that a particular bit is set; 0, indicating that the bit is unset; or X, indicating that the significance of the bit is not tested. In addition, an all-zeros key matches to a nondirectional long kernel, as described above. If a non-zero key matches none of the patterns in the table it is preferably assigned a non-directional short kernel.

522734.doc -11 Accordingly, the method of Fig. 3 provides for the selection of an appropriate kernel for the pixel in question, the kemrnal being one of the eight steered kernels dependent upon edge orientation angle, a short kernel or a long kernel.

Fig. 4 is a flowchart showing a method of performing interpolation on a pixelbased image, which utilises the method of determining pixel edge orientation in accordance with the preferred embodiment of the present invention. The preferred method utilises a linear convolution process and is used for arbitrary up-sampling factors when it is necessary to introduce new colours into an output.

The process begins at step 401, where for each output pixel location the contributing neighbouring pixels are determined. The neighbours contributing to each output point are simply those that lie on and within the perimeter of a square centred at the output point with sides equal to the largest interpolating kernel width. At the next step 403, the contribution of each of the neighbours to the output pixel is calculated by sampling continuous kernels, centred at each of the input points at the relative location of the output point. The kernel used at each input point is one determined by the kernel selection method previously described with refemce to equation Fig. 3 and Table 1.

Kernel samples are weighted (multiplied) by the sample value at the input points to give their contribution to the output. The process continues at step 405, where the output is formed as the sum of all contributions of the selected output pixel location. The method described does not guarantee that the kernel samples used at any output location will sum to unity. To ensure unit gain for the interpolation process a value equal to the sum of the unweighted kernel samples used to calculate each output is preferably maintained during the calculation of each output point. The sum of input contributions is divided by the value equal to the sum of the unweighted kernel samples (the sum of kernel samples) to 25 ensure unity gain.

In a further embodiment of the interpolation process described in Fig. 4, a single interpolating kernel can be selected for use at each output point based on a nearest neighbour interpolation of the previously determined kernel selection map (or edge map).

The interpolation factor used in the interpolation process described in Fig. 4 is preferably described as a rational number resulting in a fixed number of sampled versions of each of the interpolating kernels ever being used. Specifically, after referring to the numerator of the rational up-sample factor as the "up-sample rate" and denoting it as u, an "up-sample grid" can be generated by inserting u -1 new locations between each of the input points. The grid defined by the combination of input and new locations is the upsample grid. The grid defined by the original (input) image sample points is referred to as 522734.doc -12the input grid. Fig. 6 shows an example of up-sample 601 and input grids 603. Unit distance is defined with respect to the input grid 603. By placing each kernel to be used in the interpolation process at each location on the up-sample grid 601, then sampling the selected kernel, for each position on the input grid 603, a set of samples corresponding to all of the possible sampling of the kernels required for the interpolation process described above is obtained. The sample values can preferably be stored in a look-up-table for fast reference.

In a further embodiment of the present invention, the linear sum previously used to calculate the pixel intensity at each output point is replaced with a non-linear process.

The further embodiment is used for integer up-sample rates where it is possible to interpolate without introducing new colours. The further embodiment differs from the interpolation process described in Fig. 4 in that the output pixel value (calculated at step 405) is calculated by selecting the input pixel intensity that has the greatest weight as defined by the kernel samples. For each set of input pixel and corresponding kernel 15 coefficients, a bin is preferably created for each unique input value. Then, for each input pixel location, the corresponding kernel weight is added to the bin corresponding to the oo..

pixel value. The output value is the (input) value with the highest score in the associated bin after all locations have been processed in this way. Note that no normalisation step is required using the non-linear method of the further embodiment.

20 The further embodiment can be modified such that a single kernel is selected based on a nearest neighbour interpolation of the edge map image. Rational interpolation factors and a look-up-table for determining kernel samples can preferably be utilised with the further embodiment.

It is important to note that the result of non-linear interpolation is dependent on 25 the kernel size used. Specifically, for even kernel sizes, only odd up-sample rates lead to visually pleasing results. Where even up-sample rates are required, odd length kernels are preferably used.

The aforementioned preferred methods comprise a particular control flow. There are many other variants of the preferred methods which use different control flows without departing the spirit or scope of the invention. Furthermore one or more of the steps of the preferred methods can be performed in parallel rather sequential.

The method of determining pixel edge orientation for a pixel-based image is preferably practiced using a conventional general-purpose computer system 500, such as that shown in Fig. 5 wherein the processes of Figs. 1 to 4 can be implemented as software, such as an application program executing within the computer system 500. In 522734.doc 13particular, the steps of method of Fig. 3 are effected by instructions in the software that are carried out by the computer. The software can be divided into two separate parts; one part for carrying out the method of determining pixel edge orientation for a pixel-based image; and another part to manage the user interface between the latter and the user. The software may be stored in a computer readable medium, including the storage devices described below, for example. The software is loaded into the computer from the computer readable medium, and then executed by the computer. A computer readable medium having such software or computer program recorded on it is a computer program product. The use of the computer program product in the computer preferably effects an advantageous apparatus for determining pixel edge orientation for a pixel-based image in accordance with the embodiments of the invention.

The computer system 500 comprises a computer module 501, input devices such as a keyboard 502 and mouse 503, output devices including a printer 515 and a display device 514. A Modulator-Demodulator (Modem) transceiver device 516 is used by the 15 computer module 501 for communicating to and from a communications network 520, for example connectable via a telephone line521 or other functional medium. The modem 516 can be used to obtain access to the Internet, and other network systems, such as a Local Area Network (LAN) or a Wide Area Network (WAN).

The computer module 501 typically includes at least one processor unit 505, a 20 memory unit 506, for example formed from semiconductor random access memory (RAM) and read only memory (ROM), input/output interfaces including a video interface507, and an I/O interface513 for the keyboard502 and mouse503 and optionally ajoystick (not illustrated), and an interface 508 for the modem 516. A storage device 509 is provided and typically includes a hard disk drive 510 and a floppy disk i 25 drive 511. A magnetic tape drive (not illustrated) may also be used. A CD-ROM drive 512 is typically provided as a non-volatile source of data. The components 505 to 513 of the computer module 501, typically communicate via an interconnected bus 504 and in a manner which results in a conventional mode of operation of the computer system 500 known to those in the relevant art. Examples of computers on which the embodiments can be practised include IBM-PC's and compatibles, Sun Sparcstations or alike computer systems evolved therefrom.

Typically, the application program of the preferred embodiment is resident on the hard disk drive 510 and read and controlled in its execution by the processor 505.

Intermediate storage of the program and any data fetched from the network 520 may be accomplished using the semiconductor memory 506, possibly in concert with the hard 522734.doc 14disk drive 510. In some instances, the application program may be supplied to the user encoded on a CD-ROM or floppy disk and read via the corresponding drive 512 or 511, or alternatively may be read by the user from the network 520 via the modem device 516.

Still further, the software can also be loaded into the computer system 500 from other computer readable medium including magnetic tape, a ROM or integrated circuit, a magneto-optical disk, a radio or infra-red transmission channel between the computer module 501 and another device, a computer readable card such as a PCMCIA card, and the Internet and Intranets including email transmissions and information recorded on websites and the like. The foregoing is merely exemplary of relevant computer readable mediums. Other computer readable mediums may be practiced without departing from the scope and spirit of the invention.

The method of determining pixel edge orientation for a pixel-based image can alternatively be implemented in dedicated hardware such as one or more integrated circuits performing the functions or sub functions of Fig. 3. Such dedicated hardware can 15 include graphic processors, digital signal processors, or one or more microprocessors and Sg. associated memories.

The foregoing describes only one embodiment/some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiment(s) being illustrative and not oo 20 restrictive.

In the context of this specification, the word "comprising" means "including ***principally but not necessarily solely" or "having" or "including" and not "consisting only of'. Variations of the word comprising, such as "comprise" and "comprises" have corresponding meanings.

,oooo* oo*° 522734.doc

Claims (7)

1. A method of determining an orientation angle for a set of discrete sample values, said method comprising the steps of: accessing at least one portion of said set; calculating an absolute difference between at least one discrete sample value of said portion and each remaining discrete sample value of said portion; comparing said absolute difference for each remaining discrete sample value of said portion to a threshold value; providing an indicator for each remaining discrete sample value depending on said comparison; and matching said indicator to one of a plurality of orientation angles.

2. The method according to claim 1, wherein said one discrete sample value is a 15 non-boundary discrete sample value. oeeo The method according to any one of claims 1 or 2, comprising the further step of initialising boundary discrete sample values of said image data to have no preferred orientation.

4. The method according to any one of the preceding claims, wherein said threshold value is predetermined. The method according to claim 4, wherein said threshold value is equal to

6. The method according to any one of claims 1 to 3, wherein said threshold value is determined dynamically.

7. The method according to any one of the preceding claims, wherein said edge orientations are predetermined.

8. The method according to any one of the preceding claims, wherein said matching of said indicators is dependent on a set of predetermined rules.

522734.doc -16- 9. The method according to any one of the preceding claims, wherein a look-up table is used to perform said matching. The method according to any one of claim 1 to 9, wherein a logic circuit is used to perform said matching. 11. The method according to any one of the preceding claims, wherein said plurality of orientation angles include the set t7c/8, 37t/8, t/2, 5t/8, 37t/4 and 77/8}. 12. The method according to claim any one of the preceding claims, wherein the logic table illustrated in Table 1 is used to perform said matching. 13. A method of selecting a kernel for interpolation utilising the method of determining an orientation angle for a set of discrete sample values according to claim 1. S. o, 14. A method of performing resolution conversion utilising the method of determining an orientation angle for a set of discrete sample values according to claim 1. S.. A method of determining pixel edge orientation for a pixel-based image, said 20 method comprising the steps of: 6666 ~determining neighbouring pixels for at least one non-boundary pixel of said pixel-based image; calculating an absolute difference between said non-boundary pixel and each neighbouring pixel; s:*I 25 comparing said absolute difference for each neighbouring pixel to a threshold; providing an indicator for each neighbouring pixel depending on said comparison; and matching said indicator to one of a plurality of edge orientations. 16. A method of determining pixel edge orientation for a pixel-based image, said method comprising the steps of: determining neighbouring pixels for each non-boundary pixel of said pixel-based image; determining a difference map using said neighbouring pixels; 522734.doc -17- determining a key for each said non-boundary pixel by applying a threshold to said difference map; and matching said key to one of a plurality of edge orientations. 17. The method according to any one of claims 15 or 16, comprising the further step of initialising boundary pixels of said pixel-based image to have no preferred orientation. 18. The method according to any one of claims 15 or 16, wherein said difference map is determined by calculating an absolute difference value representing a difference between each neighbouring pixel and said non-boundary pixel. 19. The method according to any one of claims 15 to 18, wherein said threshold is predetermined. 20. The method according to claim 19, wherein said threshold is equal to S 21. The method according to any one of claims 15 to 18, wherein said threshold is determined dynamically. 0o0o0 20 22. The method according to any one of claims 15 to 21, wherein said key includes one bit of information for each neighbouring pixel of said non-boundary pixel. o00.0: 23. The method according to any one of claims 15 to 22, wherein said edge orientations are predetermined. 24. The method according to any one of claims 15 to 23, wherein said matching of said keys is dependent on a set of predetermined rules. The method according to any one of claims 15 to 24, wherein a look-up table is used to perform said matching. 26. The method according to any one of claims 15 to 24, wherein a logic circuit is used to perform said matching. 522734.doc -18- 27. The method according to any one of claims 15 to 25, wherein said plurality of edge orientations include the set n/8, 37/8, 7/2, 57r/8, 3n/4 and 77/8}. 28. The method according to any one of claims 15 to 26, wherein the logic table illustrated in Table 1 is used to perform said matching. 29. A method of selecting a kernel for image interpolation utilising the method of determining pixel edge orientation according to any one of claims 15 to 28. 30. A method of performing resolution conversion utilising the method of determining pixel edge orientation according to any one of claims 15 to 28. 31. A method of performing interpolation on a pixel-based image, said method g comprising, for each output pixel location of said pixel-based image, the steps of: determining input pixels which contribute to an output pixel value at said output pixel location; S (ii) calculating a kernel value for each of said input pixels using a kernel selected from one of a plurality of kernels, wherein for each non-boundary pixel of said input pixels, said calculating comprising the sub-steps of: S: 20 determining neighbouring pixels of said non-boundary pixel; determining a difference map using said neighbouring pixels; determining a key by applying a threshold to said difference map; and selecting said kernel for said non-boundary pixel using said key; and (iii) calculating said output pixel value utilising said kernel values. 32. The method according to claim 31, wherein step (iii) comprises the sub-steps of: multiplying said kernel values with said input pixel values to produce a weighted pixel value for each said input pixel value; and summing said weighted values to calculate said output pixel value. 33. The method according to claim 31, wherein step (iii) comprises the sub-steps of: creating an associated bin for each unique said input pixel value; 522734.doc -19- adding said kernel values for each of said input pixel values to said associated bins to create a bin value for each said input pixel value; and selecting said input pixel value with said largest bin value as said output pixel value. 34. The method according to any one claim 31 to 33, wherein said difference map is determined by calculating an absolute difference value representing a difference between each neighbouring pixel and said non-boundary pixel. 35. The method according to any one of claims 31 to 34, comprising the further step S: "of initialising boundary pixels of said input pixels to be non-directional. 36. The method according to any one of claims 31 to 35, wherein said threshold is predetermined. o *o* 37. The method according to claim 36, wherein said threshold is equal to o*o* 38. The method according to any one of claims 31 to 37, wherein said threshold is determined dynamically. 39. The method according to any one of claims 31 to 38, wherein said key includes one bit of information for each neighbouring pixel of said non-boundary pixel. The method according to any one of claims 31 to 39, wherein step (ii)(d) comprises the further step of matching said key to a one of a plurality of edge orientations. 41. The method according to any one of claims 31 to 40, wherein said edge orientations are predetermined. 42. The method according to any one of claims 40 or 41, wherein said matching of said keys is dependent on a set of predetermined rules. 43. The method according to any one of claims 40 to 42, wherein a look-up table is used to perform said matching. 522734.doc 44. The method according to any one of claims 40 to 42, wherein a logic circuit is used to perform said matching. 45. The method according to claim 40, wherein the logic table illustrated in Table 1 is used to perform said matching. 46. The method according to any one of claims 31 to 45, wherein steps and are replaced by a further step of selecting said kernel for said non-boundary pixels based on a nearest neighbour interpolation of a previously determined difference map. 47. The method according to any one of claims 31 to 46, wherein said plurality of kernels are of the form: hdxd,,0 (Sxsy) h d.((1-20/ir)sx +(20/i)s,)xhd ((20/r)s x -1)sy), wherein os(40), and wherein s t At, 0 is the angle of S2 2 orientation of an input pixel, and wherein 1 where s <d, h d 0 wheres d, \sl-d S* s-d otherwise. 1-2d 48. An apparatus for determining an orientation angle for a set of discrete sample values, said apparatus comprising: access means for accessing at least one portion of said set; calculation means for calculating an absolute difference between at least one discrete sample value of said portion and each remaining discrete sample value of said portion; comparison means for comparing said absolute difference for each remaining discrete sample value of said portion to a threshold value; indicator means for providing an indicator for each remaining discrete sample value depending on said comparison; and means for matching said indicator to one of a plurality of orientation angles. 522734.doc -21- 49. The apparatus according to claim 48, wherein said one discrete sample value is a non-boundary discrete sample value. 50. The apparatus according to any one of claims 48 or 49, further comprising means for initialising boundary discrete sample values of said image data to have no preferred onrientation. 51. The apparatus according to any one of claims 48 to 50, wherein said threshold value is predetermined. 52. The apparatus according to claim 51, wherein said threshold value is equal to 0% 53. An apparatus for determining pixel edge orientation for a pixel-based image, said apparatus comprising: means for determining neighbouring pixels for each non-boundary pixel of said pixel-based image; means for determining a difference map using said neighbouring pixels; ~means for determining a key for each said non-boundary pixel by applying a 20 threshold to said difference map; and means for matching said key to one of a plurality of edge orientations. 54. The apparatus according to claim 53, further comprising means for initialising boundary pixels of said pixel-based image to have no preferred orientation. The apparatus according to any one of claims 53 or 54, wherein said difference map is determined by calculating an absolute difference value representing a difference between each neighbouring pixel and said non-boundary pixel. 56. The apparatus according to any one of claims 53 to 55, wherein said threshold is predetermined. 57. The apparatus according to claim 56, wherein said threshold is equal to 522734.doc Is-22- 58. An apparatus for performing interpolation on a pixel-based image, said apparatus comprising, for each output pixel location of said pixel-based image, the steps of: means for determining, for each output pixel location of said pixel-based image, input pixels which contribute to an output pixel value at said output pixel location; means for calculating a kernel value for each of said input pixels using a kernel selected from one of a plurality of kernels, wherein for each non-boundary pixel of said input pixels; means for determining neighbouring pixels of said non-boundary pixel; means for determining a difference map using said neighbouring pixels; means for determining a key by applying a threshold to said difference map; "means for selecting said kernel for said non-boundary pixel using said key; and means for calculating said output pixel value utilising said kernel values. 0% 59. The apparatus according to claim 58, further comprising: :.7 means for multiplying said kernel values with said input pixel values to produce a weighted pixel value for each said input pixel value; and 0" means for summing said weighted values to calculate said output pixel value. The apparatus according to claim 58, further comprising: o 20 means for creating an associated bin for each unique said input pixel value; means for adding said kernel values for each of said input pixel values to said associated bins to create a bin value for each said input pixel value; and means for selecting said input pixel value with said largest bin value as said output pixel value. 61. The apparatus according to claim 58, wherein said difference map is determined by calculating an absolute difference value representing a difference between each neighbouring pixel and said non-boundary pixel. 62. The apparatus according to claim 58, comprising the further step of initialising boundary pixels of said input pixels to be non-directional. 63. A computer readable medium, having a program recorded thereon, where the program is configured to make a computer execute a procedure to determine an orientation angle for a set of discrete sample values, said program comprising: 522734.doc 23 code for accessing at least one portion of said set; code for calculating an absolute difference between at least one discrete sample value of said portion and each remaining discrete sample value of said portion; code for comparing said absolute difference for each remaining discrete sample value of said portion to a threshold value; code for providing an indicator for each remaining discrete sample value depending on said comparison; and code for matching said indicator to one of a plurality of orientation angles. 64. The computer readable medium according to claim 63, wherein said one discrete sample value is a non-boundary discrete sample value. o* 65. The computer readable medium according to claim 63, said program further comprising code for initialising boundary discrete sample values of said image data to have no preferred orientation. S 66. The computer readable medium according to claim 63, wherein said threshold value is predetermined. *•go 20 67. The computer readable medium according to claim 63, wherein said threshold value is equal to o•••o 68. A computer readable medium, having a program recorded thereon, where the program is configured to make a computer execute a procedure to determine pixel edge orientation for a pixel-based image, said program comprising: code for determining neighbouring pixels for each non-boundary pixel of said pixel-based image; code for determining a difference map using said neighbouring pixels; code for determining a key for each said non-boundary pixel by applying a threshold to said difference map; and code for matching said key to one of a plurality of edge orientations. 69. The computer readable medium according to claim 68, further comprising means for initialising boundary pixels of said pixel-based image to have no preferred orientation. 522734.doc 24 The computer readable medium according to claim 68, wherein said difference map is determined by calculating an absolute difference value representing a difference between each neighbouring pixel and said non-boundary pixel. 71. The computer readable medium according to claim 68, wherein said threshold is predetermined. 72. The computer readable medium according to claim 68, wherein said threshold is equal to 73. A computer readable medium, having a program recorded thereon, where the .:program is configured to make a computer execute a procedure to interpolate a pixel- based image, said apparatus comprising, for each output pixel location of said pixel-based image, the steps of: means for determining, for each output pixel location of said pixel-based image, o °-input pixels which contribute to an output pixel value at said output pixel location; means for calculating a kernel value for each of said input pixels using a kernel selected from one of a plurality of kernels, wherein for each non-boundary pixel of said "input pixels; 20 means for determining neighbouring pixels of said non-boundary pixel; means for determining a difference map using said neighbouring pixels; oo.ooi 9 means for determining a key by applying a threshold to said difference map; means for selecting said kernel for said non-boundary pixel using said key; and means for calculating said output pixel value utilising said kernel values. 74. The apparatus according to claim 73, further comprising: means for multiplying said kernel values with said input pixel values to produce a weighted pixel value for each said input pixel value; and means for summing said weighted values to calculate said output pixel value. The method according to claim 73, further comprising: means for creating an associated bin for each unique said input pixel value; means for adding said kernel values for each of said input pixel values to said associated bins to create a bin value for each said input pixel value; and 522734.doc means for selecting said input pixel value with said largest bin value as said output pixel value. 76. The method according to claim 73, wherein said difference map is determined by calculating an absolute difference value representing a difference between each neighbouring pixel and said non-boundary pixel. 77. The method according to claim 73, comprising the further step of initialising boundary pixels of said input pixels to be non-directional. 78. An interpolation method substantially as described herein with reference to any S* one of the embodiments described. DATED this Eleventh Day of October, 2000 1i Canon Kabushiki Kaisha S Patent Attorneys for the Applicant SPRUSON FERGUSON S S S 522734.doc