AU735557B2 - Method and apparatus for highly efficient representation and compression of images - Google Patents

Description

S F Ref: 452377

AUSTRALIA

PATENTS ACT 1990 COMPLETE SPECIFCATION FOR A STANDARD PATENT

ORIGINAL

r r r i Name and Address of Applicant: Actual Inventor(s): Address for Service: Invention Title: Canon Kabushlki Kaisha 30-2, Shimomaruko 3-chome Ohta-ku Tokyo 146

JAPAN

Kieran Gerard Larkin Spruson Ferguson, Patent Attorneys Level 33 St Martins Tower, 31 Market Street Sydney, New South Wales, 2000, Australia Method and Apparatus for Highly Efficient Representation and Compression of Images *.aa a ASSOCIATED PROVISIONAL APPLICATION DETAILS [31] Application No(s) [33] Country PP3406 AU [32] Application Date 7 May 1998 The following statement is a full description of this invention, including the best method of performing it known to me/us:- 5815 d r 1- METHOD AND APPARATUS FOR HIGHLY EFFICIENT REPRESENTATION AND COMPRESSION OF IMAGES Field of the Invention The present invention relates to the digital compression of images and, in particular, to an efficient method of representing the essence of an image as a phase modulated structure. The present invention finds application in the coding of a variety of images, and is of particular relevance to the representation of "fingerprint" 10 type images.

Background Art The compression of grey-scale fingerprint images is an important problem in the field of digital image compression. In one recent proposal utilised by the United 15 States Federal Bureau of Investigation (FBI), an algorithm based on a scalar quantisation of a discrete wavelet transform decomposition of the image, followed by run length encoding and Huffman encoding has been implemented.

For an example of discussion of the technique utilised, see 20 "Compression of Gray-Scale Fingerprint Images" by Tom Hopper appearing in SPIE Vol. 2242, entitled Wavelet Applications (1994) at pp 180-187. The reported compression factors for the wavelet based compression were typically 15:1, and at best about 20:1.

Fingerprint ridges are not necessarily continuous across an impression but suffer interruptions such as ridge endings or bifurcations. Collectively, these interruptions are known as minutiae. The definitive information utilised to determine whether one fingerprint matches another is the fine detail in these ridges the minutiae and their relationships. Hence, it is normally considered important CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISRA\MMEDIA\MMEDIA06\45237 7 .doc d -2that for fingerprint compression applications that the fidelity of the ridge detail is retained in the compression and decompression transformations.

The process of fingerprint identification (matching) and fingerprint image compression are based on two unrelated methods. The typical compression scheme utilised is wavelet compression. The wavelet compression schemes are essentially general image compression schemes and do not take advantage of the particular structure of a fingerprint image. The matching schemes characterise the ooooo "minutiae by type, orientation and position. These features then catalogued and stored as essentially separate data to the compressed image. Automatic fingerprint identification is achieved by comparing a list of the S 15 catalogued features with a data base of stored lists. When a match is detected, the "matching" fingerprint is decompressed for direct comparison with the fingerprint in e oo ::question.

Whilst such an arrangement affords fast high-level 20 matching, the fine detail that characterises the minutiae oeooe is not considered until the more time consuming direct comparison. Significantly, although the catalogued features may be thought of as compressed data, such does not represent the compressed image and it is desirable for matching to be based on the image, rather than some catalogued summary thereof. It is thus desirable to provide for a system that provides for high compression ratios whilst permitting the automatic classification of minutiae, thus promoting automatic matching systems.

Summary of the Invention In accordance with a first aspect of the present CFP1270AU Mmedia 06 452377 \\SAF\DATA\HOME\ELEC\CISRA\MMEDIA\MMEDIA06452377doc invention, there is provided a method of encoding a fringe pattern image, said method comprising the steps of: demodulating said image to provide a phase map structure of said image; identifying and removing any singularities in said phase map structure to create a continuous phase map structure; removing 2r phase discontinuities from said continuous phase map structure; and separately encoding said singularities and said continuous phase map structure as an encoded representation of said image.

In accordance with a second aspect of the present invention, there is provided a method of encoding a fringe pattern image, said method comprising the steps of: providing input data of said image; demodulating said input data to determine at least a phase mapping structure; identifying singularities within said phase mapping structure and retaining a separate record of said singularities; removing said singularities from said phase mapping structure to provide a continuous phase mapping structure; removing 2n phase discontinuities from said continuous phase mapping structure; encoding said continuous phase mapping structure; and associating said separate record of said singularities with said encoded continuous phase mapping structure to provide an encoded representation of said image.

In accordance with another aspect of the present invention, there is provided a \nethod of handling singularities forming parts of fringe maps; said method comprising the 452377.doc -3asteps of: determining a phase mapping structure of said fringe map; identifying each singularity within said phase mapping structure and forming a corresponding record thereof; and subtracting each said singularity from said phase mapping structure to form a continuous phase mapping structure.

In accordance with another aspect of the present invention, there is provided a method of determining a match between an input fringe pattern image and one of a plurality of stored fringe pattern images, each of said images being represented in an encoded form comprising a residual phase mapping and a record of singularities in which each said singularity is associated with a residual phase value of said mapping at a location S.:i corresponding to said singularity, said method comprising the steps of: comparing the record of singularities of said input image with the records of Sosingularities of each said stored image to determine an output group of matching images.

o S 15 In accordance with another aspect of the present invention, there is provided a method of decoding an encoded representation of a fiinge pattern image, said method comprising the steps of: ooooe S(a) decoding said encoded representation to provide a phase map structure of oooo said image; and inverse transforming said phase map structure to form said image.

In accordance with another aspect of the present invention, there is provided an apparatus for encoding a fringe pattern image, said apparatus comprising: first means for demodulating said image to provide a phase map structure of said 452377.doc 3b second means for identifying and removing any singularities in said phase map structure to create a continuous phase map structure; third means for removing 27r phase discontinuities from said continuous phase map structure; and fourth means for separately encoding said singularities and said continuous phase map structure as an encoded representation of said image.

In accordance with another aspect of the present invention, there is provided an apparatus for encoding a fringe pattern image, said apparatus comprising: a demodulator for demodulating said input data to determine at least a phase mapping structure; means for identifying singularities within said phase mapping structure and retaining :i a separate record of said singularities; means for removing said singularities from said phase mapping structure to provide a continuous phase mapping structure; o* means for removing 271 phase discontinuities from said continuous phase mapping structure; means for encoding said continuous phase mapping structure; and means for associating said separate record of said singularities with said encoded o* continuous phase mapping structure to provide an encoded representation of said image.

In accordance with another aspect of the present invention, there is provided an apparatus for handling singularities forming parts of fringe maps, said apparatus comprising: means for determining a phase mapping structure of said fringe map; means for identifying each singularity within said phase mapping structure and forming a corresponding record thereof; and 452377.doc -3c- 0 000 C 0 0 00 means for subtracting each said singularity from said phase mapping structure to form a continuous phase mapping structure.

In accordance with another aspect of the present invention, there is provided an apparatus for determining a match between an input image and one of a plurality of stored images, each of said images being represented in an encoded form comprising a residual phase mapping and a record of singularities in which each said singularity is associated with a residual phase value of said mapping at a location corresponding to said singularity; said apparatus comprising: means for comparing the record of singularities of said input image with the records of singularities of each said stored image to determine an output group of matching images.

In accordance with another aspect of the present invention, there is provided an apparatus for decoding an encoded representation of a fringe pattern image, said apparatus comprising: means for decoding said encoded representation to provide a phase map structure of 15 said image; and means for inverse transforming said phase map structure to form said image.

In accordance with another aspect of the present invention, there is provided a fringe pattern image representation system comprising: an encoding arrangement for encoding said image by determining a phase map structure of said image and compressing the phase map structure to thereby provide a compressed representation of said image; and a decoding arrangement complementing said encoding arrangement for decompressing said compressed representations.

In accordance with another aspect of the present invention, there is provided a fringe

S

01F 452377.doc 3d pattern image representation system comprising an encoding arrangement for encoding said image by providing input data of said image; demodulating said input data to determine at least a phase mapping structure; identifying singularities within said phase mapping structure and retaining a separate record of said singularities; removing said singularities from said phase mapping structure to provide a continuous phase mapping structure; removing 27t phase discontinuities from said continuous phase mapping structure; encoding said continuous phase mapping structure; and associating said separate record of said singularities with said encoded continuous phase mapping structure to provide an encoded representation of said image; and a decoding arrangement substantially complementing said encoding arrangement.

In accordance with another aspect of the present invention, there is provided a fringe pattern image matching system for matching an input image with one or more of a plurality of stored images, said system comprising means for representing each of said images using an encoded phase map structure in which phase singularities in each said structure are associated with said structure, said system comprising: means for comparing the phase singularities of said input image with the phase singularities of said stored images.

In accordance with another aspect of the present invention, there is provided a fringe pattern image matching system, said system comprising: a database of compressed image data representing a plurality of frinje pattern images, the compressed image data for each said image including: a compressed residual phase map obtained by compressing a phase map 452377.doc -3efrom which: singularities have been identified and removed; and 2n7 phase discontinuities have been removed; and (ii) a record of each said singularity removed from said phase map, said record including a position, order and orientation of each said singularity; means for extracting from said database for any one said image said record and associating with said position, order and orientation, a value of the local residual phase associated with said singularity; means for comparing the extracted position, order, orientation and local residual phase of said one image with corresponding values of an input image to determine a match therebetween.

In accordance with another aspect of the present invention, there is provided a database of encoded images, each said image being represented by: a compressed residual phase map obtained by compressing a phase map from which singularities have been identified and removed; and 27n phase discontinuities have been removed; and; and (ii) a record of each said singularity removed from said phase map, ""said record including a position, order and orientation of each said singularity.

*oeo 9 Preferably, the singularity encoding includes encoding a position of each singularity in addition to its order and orientation. The phase map structure can be of the form: f(x, y) a(x, y) b(x, y) cos((x, c(x, y) n(x, y) The magnitude of the phase modulated component can be separately encoded as what is termed herein a "significance map". The significance map represents the relative S importance of all phase components. Advantageously the image is a fingerprint-type image.

452377.doc -3f- Brief Description of the Drawings Notwithstanding any other forms which may fall within the scope of the present invention, preferred forms of the invention will now be described, by way of example only, with reference to the accompanying drawings in which: Fig. 1 illustrates a flow chart of the steps of the preferred embodiment; Fig. 2 illustrates an example synthetically created "fingerprint"; Fig. 3 illustrates a phase map of the "fingerprint" of e* .o *e eooo* e *ee 452377.doc -4 Fig. 2; Fig. 4 illustrates the process of locating discontinuities in the phase map; Fig. 5 illustrates the resultant phase map after removal of the singularities; Fig. 6 depicts a prior art compression and matching scheme; Fig. 7 depicts a compression and matching scheme according to the preferred embodiment; and 10 Fig. 8 is a block diagram of a general purpose computer upon which the preferred embodiment of the present invention can be practiced.

Detailed Description The preferred embodiment can be understood through a s 15 number of important initial observations in respect of fingerprint images, those being: Fingerprints generally consist of a series of nearly equally spaced ridges and valleys. This structure is believed to be related to the reaction-diffusion 20 dynamics occurring at morphogensis; and Fingerprints resemble optical interference patterns (fringe patterns).

Important features of fingerprints often correspond to dislocations in the fringe patterns (for example: bifurcations of fringes, fringe endings, spirals or whorls).

The preferred embodiment involves harnessing these various observations in the production of an efficient compression system. The present inventor has determined that fingerprint images and their minutiae can be compactly represented by an equation of the form: CFP127AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISRA\MMEDIA\MEDIA 6 4 5 2 37 7 .doc f(x,y) a(x,y) b(x,y) cos( c(x,y) n(x,y) Equation (1) In Equation f(x,y) represents the intensity of the fingerprint image and includes four main terms. Position coordinates within the image may be continuous for an analog image and discrete for digital images. A slowly varying background level is denoted by a(x,y) while the amplitude modulation term is Perhaps the most important term is which represents the phase of the S"fringe" pattern associated with the fingerprint. Real 10 fingerprints need an additional term to represent them accurately. This term may be called noise n(x,y) because it S: contributes little useful information to the image, and contains such features as blurring, harmonics, smudging, scratches, cuts and dust. Finally, there is a term c(x,y) to represent additional features (such as pores) that are potentially significant features, but cannot be can be contained in the first two terms.

An idealised fingerprint (without pores and defects) can be represented in a simplified form by the first two terms in equation with: ab l1,ie.: f(x,y) 1+cos( Equation (2) In this situation, all the useful information regarding the fingerprint is contained in the phase function Although f(x,y) is (generally) a rapidly varying function of the phase is (generally) a slowly varying function of This factor can therefore be used in the compression process. If f(x,y) can be (accurately) demodulated to reveal the phase then high compression ratios are attainable for the slowly varying and therefore for which can be CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISRA\MMEDIA\MMEDIA06\ 4 5 2377.doc -6reconstructed (or decompressed/decoded) using Equation (2) It should be noted is only a slowly varying function if it has been "unwrapped" correctly, and the phase singularities identified and removed. Assuming it is possible to extract the phase, characterise and remove any phase singularities, and correctly unwrap the residual phase (a process which will be described hereinafter), then it is possible to: 0 compress the residual phase using conventional (eg.

wavelet) compression techniques or parametrically compress using 2D polynomial or spline fitting; and catalogue the phase singularities which correspond directly to the so called minutiae of conventional fingerprint analysis. This catalogue has entries with just 4 parameters for each singularity.

1. singularity x location 2. singularity y location 3. singularity "charge" or order 4. singularity phase (or orientation).

20 The catalogue is an integral part of the compression data, but also contains the data for a matching process.

At this point it should be mentioned that the residual phase is also useful for matching as it contains such information as the number of ridges (and their orientation) between minutiae. The information will be apparent in the residual phase and does not need to be decompressed to obtain this data if the local residual phase is noted and catalogued at each singularity location. This allows matching to be performed in the compressed domain if desired. Such matching may be performed using the singularities and the singularity locations within the CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISRA\MMEDIA\MMEDIA06\ 4 5 2 377doc -7residual phase without the need to decompress the residual phase.

Turning now to Fig. 1 there is shown in more detail the main steps of a method 1 in accordance with the preferred embodiment. The steps are now- described in detail.

The method 1 starts with the digitizing 2 of an image, such as that of a fingerprint. This may include a scan from a fully automatic fingerprint imaging system, or a system for digitising ink on paper prints. The method may also use latent fingerprints which have been developed using forensic techniques. Present optically based fingerprinting devices, however, give much higher quality images with which the method 1 may commence.

15 A preprocessing stage 3 follows the digitizing step 2 and is utilised to remove gross image defects such as smearing or over-inking. The amount of preprocessing performed is dependent upon the input data source and some amount of preprocessing is likely with most input images.

20 However, for the purposes of discussion of the preferred embodiment, a relatively high quality image from an optical input device is assumed.

A demodulation stage 4 follows to extract at least a phase component of the image. Demodulation preferably uses compact kernel algorithms for spatial carrier demodulation methods such as those disclosed in M. Kujawinska, "Spatial Phase Measurement Methods" in Interferogram Analysis: Digital Fringe Pattern Measurement Techniques, D.W.

Robinson and G. T. Reid, eds (Institute of Physics, Bristol, U.K. 1993). There are many algorithms which could be used, both 1-dimensional and 2-dimensional. In CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISR\A\MMEDIA\MMEDIA0 6 4 5 2 3 7 7 .doc -8principle, methods such as the Fourier (Hilbert) Transform Method may also be used.

In the preferred embodiment a simple 2D adaptive demodulator is chosen, but other algorithm may be chosen to suit the characteristics of the data. The 1D version of this demodulator is derived and explained in, for an example, "Efficient Nonlinear Algorithm for Envelope Detection in White Light Interferometry", Larkin, K.G.; J. Opt. Soc. Am. A/Vol. B, No. 4, April 1996, pp 832-843.

10 Consider the basic fringe pattern structure of the fingerprint: a(x,y)+b(x,y)cos(4(x,y)) Equation (3) The observation that the fringe spacing is nearconstant can be written mathematically as the phase 15 derivative (or frequency) having two components, one of which has a constant magnitude o,i.e.

27(ux+vy) +V Equation (4) and +v2 =O2 Equation 20 The nominal orientation of the fringe is defined by the angle P,where: v tanP=- Equation (6)

U

A typical case of a simulated fingerprint image illustrating the angle orientation P of the fringe is illustrated in Fig. 2.

As seen in Fig. 2, the separation between fringes 1/a and the angle P (normal to the fringes) provide two values that may be utilized for fingerprint analysis.

Significantly the present inventor has realised that the information implicit in the combination of these two values CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISR\MMEDIA\MMEDIA06\452377.doc -9may be encapsulated in a single representation based on the phase of the fringe pattern.

The objective of the demodulation step 4 is to recover the phase from the fringe pattern f. Conventional spatial carrier phase-shifting algorithms can demodulate the phase over a small range of frequencies (phase derivative). However, a fingerprint pattern represents a fringe structure which has x and y components of frequency which vary over a wide range. A useable demodulation algorithm must adapt to the local fringe pattern.

A convenient approach may be based upon a five sample non-linear phase-shifting algorithm. Consider five successive samples of the digitised fingerprint: -2 f(x 2 ,y) 15 I i f(x ,y) Io f(x,y) Equation (7) fh(x 1,y) I+2 =fh(x+ 2 ,y) From this base data it is possible to _construct 20 symmetrically filtered components as follows: c, 210 o- I+1 c 2 +21 -1+2 Equation (8) s,

I+

s 2 -12 1+2 From this, and according to step 5 of Fig. 1, it is possible to extract the phase, modulation and frequency parameters. One particular form is a robust estimator (which avoids the conventional problem of zero-by-zero division), for fringe patterns with between 3 and 8+ pixels per fringe: CFP1270AU Mmedia 06 452377 \\SAF\DATA\HOME\ELEC\CISRA\MMEDIA\MEDIA0 6 4 52377 .doc S=2 arccos -s 2 Equation (9) a=2 arccos 22 2 s, C Thus, the actual phase may be recovered in a number of ways. One method integrates a with respect to x to get 4 In general, this can be combined with a corresponding y integration to get all components of 4 An alternative is to substitute back into and to get: sgn(s, sf+4s bsin()= sin(a) cos2() Equation 4 sin(a) 1+ cos 2 c 4c bcos( 2 Equation (11) c s()16sin2(a/2)[ cos 2 Equation and bsin( tan( Equation (12) bcos()) There are certain advantages to initially calculating the phase derivatives with respect to x and y. Primarily, the next step 6 of Fig. 1, of unwrapping the phase, is simplified by the use of phase derivatives. Unwrapping the 15 phase involves removing 2n phase discontinuities due to the modulo 27 character of the inverse trigonometric functions used to calculate in Equation (12) for example. The phase calculated from Equation (12) can have ambiguities owing to an assumption about the orientation of the fringes. This means that the sign of the numerator of Equation (12) does not always change as required as the orientation angle P is moved in the range zero to 2 radians. Consequently the phase output from Equation (12) can have regions of 7 radian wrapping. This ambiguity can be resolved in several ways. One way is to consider continuity of phase over regions and negate the numerator CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CIS\M

MEDIA\MEDIAOG\

4 23 77 .doc -11as required to reduce the overall discontinuities. An alternative method is to negate the numerator in a systematic manner to formally minimize the "global" (summed over the entire image) number of discontinuities. A global minimum in the discontinuities also ensures a well defined solution with maximum compressibility of the final residual phase map. In both cases the output is a 2n radian wrapped phase map derived from a n radian wrapped phase map.

There are numerous schemes for unwrapping the 27 radian phase maps. For example, see "Noise Immune Phase Unwrapping by Use of Calculated Wrap Regions", Stetson et al, Applied Optics Vol. 36, No. 20, 10 July, 1997 pp 4830- 4839. The phase demodulated and unwrapped version of the image of Fig. 2 is shown in Fig. 3.

15 The phase singularities (or dislocations) are identified in Fig. 3 by white circles containing the value of the spiral phase "charge". A spiral phase of charge (or order) located at (X o ,yo) can be represented mathematically by where the modulo 2n arctangent 20 function is used as follows: h(x,y)=C.arctan(y-yo,x-Xo) Equation (13) The charge is usually an integer, positive or negative, and represents the number of 2n phase shifts which occur in one polar rotation. The example shown in Fig. 3 actually contains a one half order discontinuity which has a characteristic misalignment, or shear, of the fringes. Integer order charges do not exhibit, this shearing effect but do have the effect of bifurcating or terminating fringes, this being best seen in Fig. 2. A charge of ±1 introduces a bifurcation. A charge of ±2 CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CIS A\MEDIA\MM1EDIA0\ 4 23 7 7 .doc -12introduces tri-furcation, and so on. The presence of a phase spiral is readily detected in a locality by choosing any closed path and summing the phase differences (each modulo 2n as the loop is traced. Any region containing just one singularity will have a total equalling 27 times the charge. Regions with no singularities have zero total charge. Regions with more than one singularity give a total equal to the total charge. This means that positive and negative charges cancel for loops containing both 10 singularities.

Having unwrapped the phase at step 6, the detection of singularities can now be performed at step 7.

The essential details of a preferred singularity detection scheme are described below, but any method which 15 detects phase spirals may be substituted. The simplest detector uses just 3x3 pixels in a typical convolution kernel. The central value is ignored and the 8 surrounding pixels are used to estimate the spiral phase component.

Working around these 8 pixels counter-clockwise, the phase increments from pixel to pixel are calculated modulo 2x.

Fig. 4 shows a suitable 3x3 kernel arrangement. To avoid unintentional or additional wraparound, the following relation can be used: sin9,,,, cos9,, sin9,, cos9 A+ mod 2 rctan n cos,,- sinO,, cos,, 1 Equation (14) cosO,,+, cosO,, sinO,, sinm,+1 From this expression the total of the 8 phase increments is found. To calculate the first increment the pixel order is wrapped and the pixel n=8 is used to represent n=0. The total is used to estimate the presence and strength of any spiral phase. For example, a charge +1 spiral will have a total of 2n if the system is CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISRA\MMEDIA\MMEDIA06\ 4 5 2 3 77 doc -13noise free. The procedure can be repeated for larger loops the outside loop of the 5x5 or 7x7 kernel) to reduce noise sensitivity.

Once all the singularities have been identified with a predefined confidence level, they can be characterised and catalogued as outlined earlier in the following manner: 1. singularity x location 2. singularity y location 3. singularity "charge" or order 10 4. singularity offset or orientation).

Once this is accomplished, the pure singularities can be subtracted from the phase, leaving a residual phase map with no singularities. The singularity removal may proceed as follows. Assuming a singularity of charge +1 has been detected at position (x 0 ,y 0 then it is possible to add a phase spiral of opposite charge: arctan(y y, x x 0 Equation This has the effect of cancelling (or removing) the effect of the phase singularity over the entire phase map.

This contrasts prior art methods which have been local in their processing and which, having acknowledged the existence of singularities, have been unable to practically deal with them. The present inventor has determined that global removal is a way to remove singularities without introducing nearby (local) artefacts and errors.

Alternatively, and advantageously, steps '6 and 7 of Fig. 1 may be swapped in their execution, this providing singularity detection and removal before 21T phase unwrapping is performed. This simplifies and accelerates the unwrapping process.

The remaining phase map is characterised by a gradient CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISRA\MM EDIA\MMEDIAO\ 4 5 23 7 7 .doc -14of almost constant magnitude, this being a mathematical way of saying that the original fingerprint was characterised by ridges and valleys of almost constant pitch or separation. A consequence of this is that the residual phase map may be well represented by a simple (smooth) polynomial or spline type function. The function may be found, using a least square fit process for.example, and the residual phase map can be represented by a function with a small number of parameters as depicted in step 8 of 10 Fig. 1. These parameters, in fact, constitute the compressed, or encoded residual phase map.

Fig. 5 shows an example of the residual phase map derived from the fingerprint of Fig; 2, which is seen to be almost devoid of information, other than an (almost 15 constant) change in shading from the centre to the extremities. As a consequence, in this particular case, the residual phase map can be represented by (compressed to) less than half a dozen floating point numbers. A more robust method of compression at this point may be to use 20 wavelet compression for the residual phase map. Because the singularities have been removed (which compress very poorly with wavelets), the residual phase allows very high compression ratios combined with high quality, due to the near constant gradient detailed earlier. In combination with the singularity characteristics calculated earlier, these form the compressed data-set. Significantly, the combination of the singularities and the residual phase at each singularity location provide a basis by which matching may be performed on a compressed data structure according to the preferred embodiment.

As seen in step 9 of Fig. 1, the location and CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISRA\MMEDIA\MMEDIA06\ 4 52377 .doc separation of spiral singularities may be catalogued in pixels, or, more meaningfully as a phase, in multiples of 27i radians. The second method is more robust to distortions because it is inherently an invariant of the fingerprint.

Further, the latter actually corresponds to the number of ridge/valley pairs between singularities.

It should be noted that real fingerprints will have regions of very low (or zero) modulation. This information is implicit in the amplitude variable b(x,y) of Equation 3.

Fig. 1 also shows one method of a demodulation scheme that calculates and processes The utilisation of the variable b(x,y) determines whether or not image regions are useable or not. Firstly, the demodulation kernel process 4 can also output an amplitude component 11 at each X and Y 99 position. A test 12 is conducted to determine if the amplitude is significant and those blocks determined to be significant 13 are utilised in the parameterisation process 8. The significance map can also be separately compressed 14 (for example, by run length encoding) and stored 15 with the other phase data.

The foregoing schema provides for high levels of compression of fingerprint type images. With the artificial fingerprint of Fig. 2, the preferred embodiment can provide for a compression ratio of 1000:1 or better.

However, for real fingerprints, because of the presence of discontinuities caused by dirt, skin damage and the like, compression ratios typically better than 100:1 are expected. Further, the compression scheme utilised- leads naturally to a system for matching minutiae as the compressed data is naturally divided into a phase map and the singularities which are precisely the singularities of CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISRA\MM MMIA\MEDIA06\452377. doc -16significance. Hence, in the matching process it is not necessary to decompress the data.

Fig. 6 shows a prior art fingerprint storage and matching system 60 where a first input image 50 is desired to be stored as part of a database for subsequent matching with other fingerprint images such as a second input image 52. In this prior art arrangement, the input image is compressed to give compressed image data 62. Also, catalogue data 64 as discussed previously is derived, sees 10 either manually or through automatic means. Together, the

S

catalogue data 64 and the compressed image data 62 form an individual record 66 that may be stored in a database (not illustrated). When a matching or comparison process is desired, a corresponding data record 68 of the second input 15 image 52 is extracted. Initially, the catalogue data of °5° e each record are compared in a match detector 70. Where a match occurs, each of the corresponding compressed images 5@50 are decompressed 72 and a match detection 74 applied between the actual images 50 and 52. The match S 20 detection 74 is typically manually performed although oooo recently automated methods have been developed to reduce the amount of manual comparison required.

Fig. 7 shows how the encoding method of the preferred embodiment can be used to simplify the matching process.

As seen, the first input image is processed to form a phase map 80 including a residual phase component 82 and singularities 84. The residual phase component 82 is compressed (encoded) by an appropriate process to give a compressed residual 86. The singularities 84 and compressed residual 86 together form a single (compressed) record 88 of the image 50 that may be stored in a database.

CFP1270AU Mmedia 06 452377 \\SAF0\DATA\HOME\ELEC\CISRA\MMEDIA\MMEDIA06 \452377.doc -17- When it is desired to match the first image 50 with a second image 52, having a corresponding compressed record 90, each of the records 88 and 90 may be input to a matching process 92 which operates on the combination of the singularity information and the residual phase at the singularity location of each record 88,90 to perform matching without a need to decompress either or both of the records 88,90. Through performing matching based upon the records 88,90, the matching process can be greatly 10 accelerated compared to the arrangement of Fig. 6. This is oo.

due to the match 90 being more accurately performed on the Scompress data set 88 than that performed on the prior art data set 66. Once a match is identified each of the records 88,90 may be decompressed to allow for manual 15 confirmation of the match if desired.

Decompression of the records 88, 90 may be performed by reversing the encoding process. Specifically, the encoded residual phase is decompressed using the complement of it's encoding process, to which is added the 20 signularities obtained directly from the corresponding record 88, 90, to provide the original phase map structure.

The phase is then combined with the amplitude components and inverse transformed back to Cartesian coordinates having a displayable amplitude, thus representing the original image, which may then be displayed. Optimally, the phase is combined with a "smoothed" version of the amplitude components, obtained by removing noise and artefact components c(x,y) and n(x,y).

The preferred embodiment of the present invention incorporating the method 1 and the above noted equations may be implemented as a computer application program hosted CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISRA\MMEDIA\MMEDIA06\ 4 52377.doc -18by an operating system forming part of a general purpose computer system. The application program has a user interface which includes menu items and controls that respond to mouse and keyboard operations. The application program has the ability to transmit data to one or more printers either directly connected to a host computer or accessed over a network. The application program also has the ability to transmit and receive data to a connected digital communications network (for example the "Internet").

The preferred embodiment of the invention can be :practised using a conventional general-purpose (host) computer system, such as the computer system 100 shown in Fig. 8, wherein the application program discussed above and 15 to be described with reference to the other drawings is implemented as software executed on the computer system 100. The computer system 100 comprises a computer "oo" module 101, input devices such as a keyboard 102 and mouse 103, output devices including a printer 115 and a 20 display device 114. A Modulator-Demodulator (Modem) transceiver device 116 is used by the computer module 101 for communicating to and from a communications network 120, for example connectable via a telephone line or other functional medium. The modem 116 can be used to obtain access to the Internet, and other network systems.

The computer module 101 typically includes at least one processor unit 105, a memory unit 106, for example formed from semiconductor random access memory (RAM) and read only memory (ROM), input/output interfaces including a video interface 107, and an I/O interface 113 for the keyboard 102 and mouse 103 and optionally a CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISRA\MMEDIA\MMEDIA06\ 4 5 2 377 .doc -19joystick (not illustrated), and an interface 108 for the modem 116. A storage device 109 is provided and typically includes a hard disk drive 110 and a floppy disk drive 111.

A CD-ROM drive 112 is typically provided as a non-volatile source of data. The components 105 to 113 of the computer module 101, typically communicate via an interconnected bus 104 and in a manner which results in a conventional mode of operation of the computer system 100 known to those in the relevant art. Examples of computers on which the oooo 10 embodiments can be practised include IBM-PC's and .compatibles, Sun Sparcstations or alike computer systems *'"':evolved therefrom. Typically, the application program of S"the preferred embodiment is resident on the hard disk drive 110 and read and controlled in its execution by the 15 processor 105. Intermediate storage of the program and any data fetched from the network 120 may be accomplished using .'the semiconductor memory 106, possibly in concert with the hard disk drive 110. In some instances, the application program may be supplied to the user encoded on a CD-ROM or 20 floppy disk, or alternatively could be read by the user from the network via the modem device 116.

As seen in Fig. 8, the computer system includes a scanner 130 connected to the computer module 101 via the I/O interface 113 which may act as a source of image data, such a fingerprint data, for processing by the application program. The scanner 113 may be specifically configured for fingerprint scanning or alternatively may be a general purpose device for hardcopy images. Other sources may typically include the network 120 and data supplied via the CD-ROM drive 112 or the floppy disk drive 111. The database of compressed image data may be formed on the hard CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISR A\MM EDIA\MMEDIA 6 4 52377 .doc disk drive 110 or alternatively/supplementally on the computer network 120. The computer module 101 may also be supplemented by dedicated compression hardware (not illustrated) in those cases where it is not desired for such processing to be handled by the processor 105.

Although the preferred embodiment has been described with reference to fingerprint type images, the inventive principles disclosed herein may be applied to a variety of image types obtained from a variety of sources for a 0 variety of purposes. Such image types are typically characterised by those which display substantial phase uniformity and thus lend themselves to high compression ratios for all image parts excepting where singularities occur. Examples of such images, in addition to 15 fingerprints include synthetic aperture radar images, ultrasonic and sonar images, interferometry images, and images obtained from astronomical, seismic and nondestructive evaluation examinations. Such images essentially form contour maps of phase change and thus I 20 exhibit large areas of slow varying phase gradient. This is to be contrasted with images such as real-life bit-maps which display no phase continuity (ie. they exhibit substantial phase noise) It would be further appreciated by a person skilled in the art that numerous variations and/or modification may be made to the present invention without departing from the spirit or scope of the invention as broadly described. The present embodiment is, therefore, to be considered in all respects to be illustrative and not restrictive.

In the context of this specification and claims, the word "comprising" means "including principally but not CFP1270AU Mmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISRA\MMEDIA\MMEDIA06\42377 .doc -21necessarily solely". Variations of the word "comprising", such as "comprise" and "comprises" have correspondingly varied meanings.

0

*V

CFP1270AU rMmedia 06 452377 \\SAFO\DATA\HOME\ELEC\CISRA\MM1EDIA\MMEIDIA06\ 4 52377 .doc

Claims (10)

1. A method of encoding a fringe pattern image, said method comprising the steps of: demodulating said image to provide a phase map structure of said image; identifying and removing any singularities in said phase map structure to create a continuous phase map structure; removing 2r phase discontinuities from said continuous phase map structure; and separately encoding said singularities and said continuous phase map structure as an encoded representation of said image. I0 S

2. A method according to claim 1, wherein encoding of said singularities includes *o .0 encoding at least a position and order of each said singularity. e.

3. A method according to claim 1, wherein removal of said singularities comprises :adding to said phase map structure a phase spiral of corresponding order and opposite orientation for each said singularity.

4. A method according to claim 1, further comprising encoding a significance map of a magnitude of said phase map structure.

A method according to claim 4, wherein said continuous phase map structure is separately encoded from significance map.

6. A method according to claim 4, wherein encoding of said continuous phase map

452377.doc -23- structure is performed using said magnitude.

7. A method according to claim 1, wherein said image is of the form: f(x,y) a(x,y) b(x, y) cos(q(x,y)) c(x,y) n(x,y), q(x,y) being said phase map structure. 10 15

8. A method according to claim 1, wherein said image is selected from the group consisting of a fingerprint image, a synthetic aperture radar image, an ultrasonic image, a sonar image, an interferometry image, an astronomical image, a seismic image, and an image obtained from a non-destructive evaluation procedure.

9. A method of encoding a fringe pattern image, said method comprising the steps of: providing input data of said image; demodulating said input data to determine at least a phase mapping structure; identifying singularities within said phase mapping structure and retaining a separate record of said singularities; removing said singularities from said phase mapping structure to provide a continuous phase mapping structure; removing 27n phase discontinuities from said continuous phase mapping structure; encoding said continuous phase mapping structure; and associating said separate record of said singularities with said encoded continuous phase mapping structure to provide an encoded representation of said image. 452377.doc -24- A method according to claim 9, wherein said demodulating further comprises determining an amplitude mapping of said image, and said method comprises the further steps of: determining a significance mapping based on said amplitude mapping; encoding said significance mapping; and associating said encoded amplitude mapping with said separate record and said encoded continuous phase mapping to form part of said encoded representation. 11. A method according to claim 10, wherein step is performed using said significance mapping. 12. A method according to claim 9, wherein each said singularity is represented by a corresponding position, order and orientation. s.ee: 13. A method according to claim 12, wherein step comprises adding to said phase map structure a phase spiral of corresponding order and opposite orientation for each said singularity. 20 14. A method according to claim 9, wherein step comprises performing a discrete wavelet transform upon said continuous phase mapping. 15. A method according to claim 10, wherein step comprises performing a discrete ^/eST wavelet transform upon said significance mapping. 452377.doc 25 16. A method according to claim 11, wherein step comprises parametrically compressing said continuous phase mapping. 17. A method according to claim 16, wherein said parametrically compressing utilizes one of an n-th order 2-dimensional polynomial on a spline fitting arrangement. 18. A method according to claim 9, wherein step removes 2n phase discontinuities from said phase mapping structure; and precedes steps and 19. A method of handling singularities forming parts of fringe maps, said method comprising the steps of: determining a phase mapping structure of said fringe map; identifying each singularity within said phase mapping structure and forming a 15 corresponding record thereof; and oooo subtracting each said singularity from said phase mapping structure to form a continuous phase mapping structure. 20. A method according to claim 19, further comprising associating each said °oo• singularity record with a phase value from said continuous phase mapping structure at a location of said singularity record. S S 21. A method of processing a fringe map, said fringe map including at least S one singularity, said method comprising: 452377.doc -26- performing the method of claim 20; and processing at least one of: said corresponding records; said continuous phase mapping structure; and said associated records and said phase values. 22. A method of determining a match between an input fi-inge pattern image and one of a plurality of stored fringe pattern images, each of said images being represented in an encoded form comprising a residual phase mapping and a record of singularities in which 10 each said singularity is associated with a residual phase value of said mapping at a location corresponding to said singularity, said method comprising the steps of: comparing the record of singularities of said input image with the records of *5 V singularities of each said stored image to determine an output group of matching images. 23. A method according to claim 22, wherein encoding of said singularities includes encoding at least a position and order of each said singularity. 24. A method according to claim 22, further comprising the steps of: decoding said input image and each image of said output group; and directly comparing the decoded images. A method according to claim 22, further comprising encoding a significance map of a magnitude of said phase map structure. 452377.dSc 452377.doc 27 26. A method of decoding an encoded representation of a fringe pattern image, said method comprising the steps of: decoding said encoded representation to provide a phase map structure of said image; and inverse transforming said phase map structure to form said image. a a. 0 0 a..1 27. A method according to claim 26, wherein said encoded representation comprises at least one phase singularity and said method further comprises: (ab) adding said singularities to said phase map structure. 28. A method according to claim 27, wherein said encoded representation comprises encoded amplitude components and said method comprises the steps of: (ac) decoding said encoded amplitude components; and (ad) combining said amplitude component with said phase map structure. A method according to claim 28, wherein said image prior to step is of the form: f(x, y) a(x, y) b(x, y) cos(b(x, Apparatus for encoding a fringe pattern image, said apparatus comprising: first means for demodulating said image to provide a phase map structure of said image; second means for identifying and removing any singularities in said phase map structure to create a continuous phase map structure; third means for removing 27t phase discontinuities from said continuous phase map 452377.doc -28- structure; and fourth means for separately encoding said singularities and said continuous phase map structure as an encoded representation of said image. 31. Apparatus according to claim 30, wherein said fourth means comprises fifth means for encoding said singularities including encoding a position, order and orientation of each said singularity. 32. Apparatus according to claim 30, wherein said second means is configured to add to 10 said phase map structure a phase spiral of corresponding order and opposite orientation for each said singularity. 33. Apparatus according to claim 30, further comprising six means for encoding a S significance map of a magnitude of said phase map structure. 34. Apparatus according to claim 33, wherein said fourth means comprises seventh 36. Apparatus according to claim 30, wherein said image is of the form: f a(x,y) b(x,y) cos( c(x,y) n(x,y), 452377.doc -29- q(x,y) being said phase structure. 37. Apparatus according to claim 30, wherein said image is selected from the group consisting of a fingerprint image, a synthetic aperture radar image, an ultrasonic image, a sonar image, an interferometry image, an astronomical image, a seismic image, and an image obtained from a non-destructive evaluation procedure.

10 S. S S 1 38. Apparatus for encoding a fringe pattern image, said apparatus comprising: a demodulator for demodulating said input data to determine at least a phase mapping structure; means for identifying singularities within said phase mapping structure and retaining a separate record of said singularities; means for removing said singularities from said phase mapping structure to provide a continuous phase mapping structure; means for removing 27r phase discontinuities from said continuous phase mapping structure; means for encoding said continuous phase mapping structure; and means for associating said separate record of said singularities with said encoded continuous phase mapping structure to provide an encoded representation of said image. 39. Apparatus according to claim 38, wherein said demodulator is configured to determine an amplitude mapping of said image, said apparatus further comprising: means for determining a significance mapping based on said amplitude mapping; means for encoding said significance mapping; and 452377.doc means for associating said encoded amplitude mapping with said separate record and said encoded continuous phase mapping to form part of said encoded representation. Apparatus according to claim 39, wherein said means for encoding said continuous phase mapping structure utilizes said significance mapping. 41. Apparatus according to claim 38, wherein each said singularity is represented by a corresponding position, order and orientation. 10 S S 9 S5 S 42. Apparatus according to claim 41, wherein said means for removing adds to said phase map structure a phase spiral of corresponding order and opposite orientation for each said singularity. 43. Apparatus according to claim 38, wherein said means for encoding said continuous phase mapping structure performs a discrete wavelet. 44. Apparatus according to claim 39, wherein said means for encoding said significance mapping performs a discrete wavelet transform upon said significance mapping. 45. Apparatus according to claim 40, wherein the encoding of said continuous phase mapping structure comprises parametrically compressing said continuous phase mapping. 46. Apparatus according to claim 45, wherein said parametrically compressing utilizes one of an n-th order 2-dimensional polynomial on a spline fitting arrangement. 452377.doc -31- 0 S 0 S 15 a 47. Apparatus for handling singularities forming parts of fiinge maps, said apparatus comprising: means for determining a phase mapping structure of said fringe map; means for identifying each singularity within said phase mapping structure and forming a corresponding record thereof; and means for subtracting each said singularity from said phase mapping structure to form a continuous phase mapping structure. 48. Apparatus according to claim 47, further comprising means for associating each said singularity record with a phase value from said continuous phase mapping structure at a location of said singularity record. 49. Apparatus for processing a fringe map, said fringe map including at least one singularity, said apparatus comprising: apparatus according to claim 47; and means for processing at least one of: said corresponding records; said continuous phase mapping structure; and said associated records and said phase values. Apparatus for determining a match between an input image and one of a plurality of stored images, each of said images being represented in an encoded form comprising a residual phase mapping and a record of singularities in which each said singularity is 452377.doc -32- associated with a residual phase value of said mapping at a location corresponding to said singularity, said apparatus comprising: means for comparing the record of singularities of said input image with the records of singularities of each said stored image to determine an output group of matching images. 51. Apparatus according to claim 50, further comprising: means for decoding said input image and each image of said output group; and means for directly comparing the decoded images. :10 0 0 0 52. Apparatus for decoding an encoded representation of a fringe pattern image, said apparatus comprising: means for decoding said encoded representation to provide a phase map structure of said image; and means for inverse transforming said phase map structure to form said image. 53. Apparatus according to claim 52, wherein said encoded representation comprises at least one phase singularity, said apparatus further comprising: means for adding the singularities to said phase map structure. 54. Apparatus according to claim 53, wherein said encoded representation comprises encoded amplitude components and said apparatus further comprises: means for decoding the encoded amplitude components; and means for combining the amplitude components with said phase map structure. 452377.doc -33- A computer readable medium incorporating a computer program product for encoding a fringe pattern image, said computer program product including a series of: computer interpretable instructions for performing the method of any one of claims 1 to 54. A computer readable medium incorporating a computer program product for handling singularities forming parts of a fringe map, said computer program product including a series of computer interpretable instructions for performing the method of any one of claims 21 to 1 S S. 0 0* S S p 0 0 S 57. A computer readable medium incorporating a computer program product for decoding a fringe pattern image, said computer program product including a series of: computer interpretable instructions for performing the method of any one of claims 26 to 58. A fringe pattern image representation system comprising: an encoding arrangement for encoding said image by determining a phase map structure of said image and compressing the phase map structure to thereby provide a compressed representation of said image; and a decoding arrangement complementing said encoding arrangement for decompressing said compressed representations. 59. A fringe pattern image representation system comprising an encoding arrangement for encoding said image by providing input data of said image; 452377.doc -34- demodulating said input data to determine at least a phase mapping structure; identifying singularities within said phase mapping structure and retaining a separate record of said singularities; removing said singularities from said phase mapping structure to provide a continuous phase mapping structure; removing 27t phase discontinuities from said continuous phase mapping structure; encoding said continuous phase mapping structure; and associating said separate record of said singularities with said encoded continuous phase mapping structure to provide an encoded representation of said image; and 10 a decoding arrangement substantially complementing said encoding arrangement. 0 a 0 oo 60. A fringe pattern image matching system for matching an input image with one or more of a plurality of stored images, said system comprising means for representing each of i said images using an encoded phase map structure in which phase singularities in each said 15 structure are associated with said structure, said system comprising: means for comparing the phase singularities of said input image with the phase *oo singularities of said stored images. 61. A fringe pattern image matching system, said system comprising: a database of compressed image data representing a plurality of fringe pattern images, the compressed image data for each said image including: a compressed residual phase map obtained by compressing a phase map from which: Ssingularities have been identified and removed; and 452377.doc 27i phase discontinuities have been removed; and (ii) a record of each said singularity removed from said phase map, said record including a position, order and orientation of each said singularity; means for extracting from said database for any one said image said record and associating with said position, order and orientation, a value of the local residual phase associated with said singularity; means for comparing the extracted position, order, orientation and local residual phase of said one image with corresponding values of an input image to determine a match therebetween. 0 9 0 9 99 0 0 0 go o 0 9*99 9 9 .0.0 0 9 0 9 9 9**9 62. A database of encoded images, each said image being represented by: a compressed residual phase map obtained by compressing a phase map from which singularities have been identified and removed; and 2 7 phase discontinuities have been removed; and; and (ii) a record of each said singularity removed from said phase map, said record including a position, order and orientation of each said singularity. 63. A method of encoding a fringe pattern image substantially as described herein with reference to the drawings excluding Fig. 6. 64. Apparatus for encoding a fringe pattern image substantially as described herein with reference to the drawings excluding Fig. 6. A fringe pattern image representation system substantially as described herein with 452377.doc -36- reference to the drawings excluding Fig. 6. 66. A method of handling singularities in a phase map of a fringe pattern image substantially as described herein with reference to the drawings excluding Fig. 6. DATED this Third Day of May, 2001 Canon Kabushiki Kaisha Patent Attorneys for the Applicant SPRUSON FERGUSON *9* S S S S. S. *S S S S 9* 9S S S S*SS S 555. S .9 5* S S S S S 55.. S S 55.. S. S S S 452377.doc