AU2005201497B2 - Robust traceable marks - Google Patents

Description

S&F Ref: 694684

AUSTRALIA

00 O r, PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT Name and Address of Applicant: Actual Inventor(s): Address for Service: Invention Title: Canon Kabushiki Kaisha, of 30-2, Shimomaruko 3-chome, Ohta-ku, Tokyo, 146, Japan Matthew Guy Partridge Stephen Edward Ecob Spruson Ferguson St Martins Tower Level 31 Market Street Sydney NSW 2000 (CCN 3710000177) Robust traceable marks The following statement is a full description of this invention, including the best method of performing it known to me/us:- 5845c -1- ROBUST TRACEABLE MARKS SField of the Invention The present invention relates generally to watermarking of documents and, in particular, to adding a watermark to a document that is robust to photocopying.

Background Even in the current "digital age" there is still a significant reliance on the photocopying of documents. Photocopiers, also named copiers, are standard equipment N, in any business. An original of a document is typically printed using a laser printer, or an inkjet printer, but subsequent copies are typically reproductions of that original document made using the photocopier. In fact, it often occurs that photocopies are made of the photocopies of the original document.

Photocopiers themselves have evolved over many years and use a high intensity light reflected from the original document in order to selectively charge a drum. The charge on the drum in turn deposits toner onto a blank sheet of paper to form the copy.

Many modem photocopiers are essentially a scanner combined with a laser printer. The scanner scans the original document and forms electronic image data of that document.

The electronic image data is then printed using the laser printer.

Regardless of the generation of the photocopier used, the copying process is imperfect. Starting with the printing of the original document, in order to create the illusion of different shades of grey in the case of a monochrome printer, the image to be printed is often dithered or halftoned. When the original document is then scanned, the image formed on the document is resampled, filtered and quantised to form an electronic image. The filtering typically includes beta correction. Each of the resampling, filtering and quantising steps introduces noise. Also, the optics of the scanner introduces 694684 -2additional noise. When the electronic image formed by the scanner is printed, yet more noise is added when the pixel values of the image are again dithered.

Each time noise is added, the photocopy becomes a further degraded version of 00 0the original document. Accordingly, the copies that are a number of generations removed r 5 from the original document are significantly degraded when compared with the original document.

,Not only is the image on the photocopy a degraded version of the original, photocopying also applies various transforms to the image. Such transforms typically include translation, anamorphic scaling, shearing and rotation, all of which are affine transforms, as well as others such as cropping.

Adding a watermark to an image is a known technique used to embed information into that image. For example, the information may identify the owner of the image. Watermarks typically fall in two classes, those relying on human detection and those relying on automatic detection, typically using a computer. The watermarks intended for human detection are necessarily visible, whereas the watermarks intended for automatic detection may be visible or invisible to the human eye.

A weakness of most watermarks intended for automatic detection is that such watermarks are very fragile. Firstly, most of those watermarks are extremely sensitive to any transforms that may have been applied during the photocopying process. Also, the watermark itself degrades significantly as a result of the photocopy process, making the watermark hard or impossible to automatically detect in the photocopy.

Regardless of the degradation resulting from, and the affine transforms applied by, the photocopy process, text and line drawings on the photocopy are still legible and therefore still usable, even after significant degradation. However, the watermarks are typically undetectable after some degradation and transformation. A need therefore exists for a watermark that is more robust against the photocopying process. Preferably the 694684 -3- O watermark should be detectable in degraded copies as long as the content on the document is legible.

O Summary It is an object of the present invention to substantially overcome, or at least ameliorate, one or more disadvantages of existing arrangements.

According to a first aspect of the present invention, there is provided a method of Sprinting a watermark onto a document, said method comprising the steps of: N receiving an image of said document; generating the watermark from a spatially dispersed basis pattern; adding said watermark to said image to form a watermarked image such that the peak values of said watermark are substantially the same as that of the peak image values; and printing said watermarked image.

According to a second aspect of the present invention, there is provided a method of watermarking an image, said method comprising the steps of: generating a watermark; and adding said watermark to said image to form a watermarked image, wherein said watermark is recoverable in a degraded version of the printed watermarked image when the content of the image is no longer legible due to the degradation.

According to another aspect of the present invention, there is provided an apparatus for implementing any one of the aforementioned methods.

According to another aspect of the present invention there is provided a computer program product including a computer readable medium having recorded thereon a computer program for implementing any one of the methods described above.

Other aspects of the invention are also disclosed.

694684 -4- Brief Description of the Drawings One or more embodiments of the present invention will now be described with reference to the drawings, in which: 00 C)Fig. 1 shows a schematic flow diagram of a method of printing a watermark onto r- 5 a document; Fig. 2 shows a schematic block diagram of a general purpose computer upon which arrangements described can be practiced; Fig. 3 shows a schematic flow diagram of a method of detecting the watermark in a copy of a watermarked image; Fig. 4A shows a straight-line in an image; Fig. 4B illustrates the parameter pair defining the straight-line shown in Fig. 4A in the polar space; Fig. 5A shows the parameter pairs of 3 lines in the polar space before a transform is applied; Fig. 5B shows the parameter pairs of the lines shown in Fig. 5A after a transform is applied; Fig. 6 shows the 4 lines arranged in such a fashion that the lines are uniformly distributed over the full angular range; Fig. 7 shows the length ratios of line segments as a function of a parameter e; Fig. 8 illustrates an example pattern; Fig. 9 shows the placement of registration marks and message marks; Fig. 10 shows a schematic flow diagram of a method of generating message pattern; Fig. 11 shows a schematic flow diagram of the preferred sub-steps for determining the number of marks for encoding a number with a predetermined number of available grid positions; 694684 Fig. 12 shows a schematic flow diagram of the preferred sub-steps for Sdetermining the grid positions for placing the marks for encoding the number; Figs. 13A to 13F illustrate the steps of encoding a message into an image by way 00 of a simplified example; Figs. 14A and 14B show the real and imaginary parts respectively of a typical basis pattern; N Fig. 15A shows an example random noise pattern; Fig. 15B shows a magnified version of the random noise pattern shown in Fig.

B;

Fig. 16 sets out a process of generating and applying a clarity mask by way of example; Fig. 17 shows a schematic flow diagram of a method of estimating, and compensating for, transforms that have been applied to a printed copy of the watermarked image; Fig. 18 shows a schematic flow diagram of a method of detecting the message patterns and decoding the message; Fig. 19 illustrates the projection of the undistorted example pattern illustrated in Fig. 8, plotted in the quasi-polar space; and Fig. 20 shows a schematic flow diagram of a method of converting an array of positions to a number.

Detailed Description including Best Mode Where reference is made in any one or more of the accompanying drawings to steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears.

694684 -6- Fig. 1 shows a schematic flow diagram of a method 200 of printing a watermark onto a ddcument. The method 200 is preferably practiced using a general-purpose computer system 100, such as that shown in Fig. 2 wherein the steps of the method 200 00 Sare implemented as software executing within the computer system 100. In particular, the t 5 steps of the method 200 are effected by instructions in the software that are carried out by the computer system 100. The software may be stored in a computer readable medium, Nincluding the storage devices described below, for example. The software is loaded into the computer from the computer readable medium, and then executed by the computer system 100. A computer readable medium having such software or computer program recorded on it is a computer program product.

The computer system 100 is formed by a computer module 101, input devices such as a keyboard 102, a mouse 103 and an imaging device 122, and output devices including a printer 115 and a display device 114. A Modulator-Demodulator (Modem) transceiver device 116 is used by the computer module 101 for communicating to and from a communications network 120, for example connectable via a telephone line 121 or other functional medium.

The computer module 101 typically includes at least one processor unit 105, and a memory unit 106, for example formed from semiconductor random access memory (RAM) and read only memory (ROM). The module 101 also includes a number of input/output interfaces including a video interface 107 that couples to the video display 114, an I/O interface 113 for the keyboard 102, imaging device 122 and mouse 103, and an interface 108 for the modem 116 and printer 115. A storage device 109 is provided and typically includes a hard disk drive 110 and a floppy disk drive 111. A CD-ROM drive 112 is typically provided as a non-volatile source of data.

The components 105 to 113 of the computer module 101, typically communicate via an 694684 -7interconnected bus 104 and in a manner which results in a conventional mode of Soperation of the computer system 100 known to those in the relevant art.

Typically, the software is resident on the hard disk drive 110 and read and 00 Scontrolled in its execution by the processor 105. In some instances, the software may be supplied to the user encoded on a CD-ROM or floppy disk and read via the corresponding drive 112 or 111, or alternatively may be read by the user from the network 120 via the N modem device 116. Still further, the software can also be loaded into the computer Ssystem 100 from other computer readable media.

Referring again to Fig. 1, the method 200 of printing the watermark onto the document starts in step 210 where the document is received by the computer system 100.

Also in step 210 a message to be encoded in the watermark is received by the computer system 100.

In one implementation the document comprises instructions for rendering at least one page. The instructions are for rendering on the page a number of splines at a given intensity. Typically that intensity is a maximum intensity. For example, the instructions may be to render black curves on a white background. A person skilled in the art would understand that characters and symbols are also made up from splines. The processor 105 then in step 220 forms an image of the document to which the watermark is to be added by rendering the document.

The document may also be received as a bitmap image containing, for example, text and lines in a bitmap format. In this case step 220 is not performed as an image of the document already exists.

The method 200 next continues in step 230 where the processor 105 generates the watermark by encoding the message through one or more spatially dispersed basis patterns. The message is typically encoded by the placement of respective basis patterns.

Each basis pattern is typically generated from a mathematical equation. However, the 694684 -8basis patterns may also be generated in advance, and stored as images on the storage Sdevice 109 or a CD-ROM readable by CD-ROM drive 112. The composition of the preferred watermark is described below.

00 SFollowing the generation of the watermark in step 230, in step 240 the watermark is then added to the image rendered from the document to form a watermarked image. In particular, the peak values of the watermark when added to the image are substantially the same as that of the peak values of the document image.

SBecause the watermark comprises spatially dispersed basis patterns which cover a large area, when such patterns are added to the document image containing for example text, the text is more difficult to read by humans. In order to improve clarity, a clarity mask is preferably applied during step 240 to edge regions of splines appearing in the document image.

In the generation of clarity masks, some information about the original content may be determined. For example, whether or not the image contains text and if so, information regarding the font size may be used. Such information may be used to determine the size of the clarity mask. However, for typical usage, an arbitrary mask size of around 1mm will normally suffice.

Fig. 16 sets out a process 260 of generating and applying a clarity mask by way of example. The process 260 receives the document image and watermark as inputs. A section of the document image 261 and a section of the watermark 262 are illustrated in Fig. 16. The process 260 starts by determining the edge regions of the splines appearing in the document image 261. Persons skilled in the art will appreciate that edge filters, such as Roberts, Prewitt, Sobel and Laplacian filters, may be appropriate for this purpose.

In the case of splines, it may be sufficient to simply regard the entire black region as edges.

694684 -9- O Next, the size 264 of a mask to be formed around the edges is determined. In one implementation a set value of 1mm may be used for the size 264. The edges are then expanded to meet the size 264. In the preferred implementation a morphological 00 operation, such as erosion/dilation, is used for expanding the edges. Section 265 shows the relationship between the text and the size 264 of the mask, whereas section 266 shows only the mask formed by the expanded edges.

Finally, the mask 266 is applied to the watermark 262 such that the watermark 262 is attenuated in regions corresponding to the mask 266. That is, the watermark 262 is applied weakly, or not at all, in the vicinity of the edges in the image document 261. In the preferred implementation the watermark 262 is not applied in regions in the vicinity of the edges in the document 261, and those regions are filled with the colour grey as shown in section 280. When the document image 261 and the attenuated watermark 280 are added, section 282 results. Grey is chosen as the flat tone region around the edges enhances content visibility to the human observer.

White is an alternative that may be used in the flat tone region. However, the colour grey is preferred as that colour assist in the degradation of text when copied.

Without any clarity mask, the intensity of the watermark would typically have to be reduced as the busyness of the strong watermarks would make the watermarked document very hard to read. However, if the intensity of the watermark is reduced, then the robustness of the watermark is reduced following a number of generations of copying.

Referring again to Fig. 1, the method 200 ends in step 250 where the pixel values of the watermarked image are sent to the printer 115 for printing. As described in the background section, in order to print the watermarked image now containing pixel values ranging from 0 to a maximum level, the printer 115 applies dithering to create the illusion of those pixel values. A paper copy of the watermarked image is formed.

694684 O The paper copy may be photocopied a number of times to produce further copies of the watermarked image. For reasons discussed in the background section, each generation of copies is a degraded version of its parent copy.

00 Fig. 3 shows a schematic flow diagram of a method 300 of detecting the watermark in a copy of the watermarked image. The method 300 is also preferably practiced by the general purpose computer 100 shown in Fig. 2, wherein the steps of the method 300 are implemented through software executing within the computer system 100.

The method 300 starts in step 310 where an image of the copy is captured. In the preferred implementation the imaging device 122 is a scanner, and the (degraded) copy of the watermarked image is scanned to form the image.

Method 300 then continues in step 320 where the embedded watermarks are detected. Because the embedded watermark is based on spatially dispersed basis patterns, and due to the intensity of the watermark added to the document, the watermark is detectable even in the degraded copy of the watermarked image. Generally, only after substantial degradation does step 320 fail to detect the watermark correctly, or fail to detect one or more of the composition basis patterns. However, as the peak values, such as peak intensity, of the watermark when added to the document image are substantially the same as that of the peak values of the document image, the extent of degradation that would lead to the failure to correctly detect the watermark would generally be similar to when the document is no longer usable. For example, a person would no longer be able to read, or have great difficulty in deciphering, the content on the copy.

Finally, in step 330, the message embedded in the watermark is decoded and output to the user.

Referring again to Fig. 1, the generation of the watermark performed in step 230 is now described in more detail. As explained in the background section, various 694684 -11transforms are also applied to the watermarked image during photocopying, with the transforms typically include cropping, translation, anamorphic scaling, shearing and rotation. Most watermarks are no longer detectable when transformed, or an extensive 00 Ssearch through a large search space has to be performed to detect such watermarks.

S 5 The preferred watermark comprises separate alignment patterns and message patterns. The alignment patterns are still detectable after transforms, and are used to Sdetermine the transforms. With the transforms determined, the transforms may be (reversed, or compensated for. The message patterns are then detected from which the embedded message is decoded in step 330 (Fig. 3).

The alignment patterns make use of the property of affine transforms that a straight line remains a straight line after any number of affine transforms is applied to that line. Parallelism of lines is also preserved. The ratio of two segments of any straight-line is also invariant with respect to affine transforms, even though lengths and angles change.

Consider a straight-line n in an image, an example of which is illustrated in Fig.

4A. The straight-line n is at an angle 83 with the vertical Cartesian axis (Y-axis) and a distance rn from the Cartesian origin in a direction perpendicular to the straight-line n.

The straight-line n is uniquely described by the parameter pair Fig. 4B illustrates the parameter pair defining the straight-line n in the polar space.

Referring again to Fig. 4A, a point (xy) on the straight-line n and a distance along the straight-line n may be written in canonical form as: x r cos,8n A sin, y r, sinf,, 2, cosf, Now consider a general affine transform applied to the image containing the straight-line n. The affine transform may include one or more of rotation, scaling, shearing, translation, and anamorphic scaling. Let 35) be the transformed coordinates 694684 12- §O in the image of the original point after the affine transform, then the point Smay be written as: 00 I 1 a al2x+ix O (2) y a 2 1 a22 (2) wherein a, are parameters defining rotation, scaling, shearing, and anamorphic scaling, and (xo,yo) defines a translation.

SIt is assumed that det a 0. Furthermore, it is noted that Sa 21 a22 det a a 12 0 indicates that a reflection has occurred.

a 21 a 2 2 The total transform applied by the parameters aij may be decomposed into convenient combinations of the prime transforms, namely rotation, anamorphic scaling and shear as follows: a, a2 (A 0l R cosc sin Transpose (3) a 21 a 22 0 B 0 1 -sin o cos wj wherein rotation by an angle o is given by matrix: (cosow sin (4) -sinco cosw c anamorphic scaling along the X- and Y-axes is sometimes called aspect ratio change and has the form: (i A B [O B) shear in the x direction has the form: I1 R (6) (0 1 694684 13- 0 and the transposition, which is applied to place the rotation transform in the correct Squadrant so that co 45°, is selected from one of the four options: 00 1 -ooi o0 -1 1 0 0 Transpose 1, 1 ,or (7) -1 0 0 -1 1 0 0 1 SThe straight-line n in the image is transformed to a straight-line in the deformed image being defined by the parameter pair n, f/ which is typically different from the Sparameter pair of the straight-line n before the affine transforms.

From Equations and and using the abbreviations c, cos sin the following relations exist: S(allsn -a 2 C) (8) (tan a 2 lsn +a 22 c.) and (a c, a 1 2 X-a 21 Sn+ a 22 c 2 c. a 22 sX- a, Is,, +a 12 c. a, 1 )2s+(ai C +2 S (9) X(-a 2 sn a 22 cn)+ yO(a 1 ,cn a 12 s) /(acI +al2S a12s 2 With Equations and including 6 unknown parameters {a 1 l, al2, a21, a22, x 0 yo} defining the affine transform, by embedding 3 straight-lines into the image, with each straight line n being defined by the parameter pair by detecting the 3 straight-lines in the affine transformed image, by determining the parameter pair for each affine transformed line, and by solving Equations and the parameters {a 1 1 al2, a21, a 2 2, xo, Y0} may be calculated.

694684 -14- However, from Equations and it can be seen that, when an affine transform is applied to a straight-line, then the parameter pair is generally 00 transformed in a rather complicated and ambiguous manner. The ambiguity occurs because a translation or scaling in the direction parallel to the straight-line n has no effect on the parameter pair {r, n fl Also, identifying the transformed straight-line corresponding to the straight-line t'n in the image before affine transform is non-obvious. To illustrate this, Fig. 5A shows the parameter pairs of 3 lines in the polar space before a transform is applied, while Fig. 5B shows the parameter pairs of the same 3 lines after the transform is applied. The correspondence between parameter pairs and before and after transform is potentially ambiguous, leading to degeneracy and possible uncertainty.

A number of techniques may be used to uniquely identify the correspondence between each straight-line before and after affine transform. One technique is to embed the line uniquely, and where the uniqueness of the lines is not destroyed by affine transforms. For example, each line may be embedded using a different basis pattern.

In the preferred implementation the property of affine transforms that the ratio of two segments of a straight-line is invariant is used to uniquely identify the correspondence between each straight-line before and after affine transform. Consider N straight-lines embedded into an image, each of the straight-lines again being uniquely described by the parameter pair {jr, f/i n =1 N, with angles 8n#n,8m when nm. Hence the straight-lines are non-parallel.

Because the straight-lines are non-parallel, each straight-line intersects each other straight-line along its length. The maximum number of intersection points for N 694684 (non-parallel) straight-lines is the triangular numberN(N-1)/2. The point (Xkm,ykm) Swhere straight-line m intersects straight-line k is: 00 Xkm m Xk Frmcos/ m -mk sinm k Cos k -km Ykm =Ym =Yk irm sin ,m Amk COS im k sin 3 k Akcosi k. O wherein ALk is the distance along straight-line k where straight-line m intersects.

5 Similarly, distance 2 1mk is the distance along straight-line m where straight-line k intersects.

Solving the simultaneous equations in Equation the distance 2 Ak is given by: rk COS -k (1 1) kIn sin(-k Consider the case where the number of embedded straight-lines N=4. Each line k would have 3 intersection points (xkm,ykm) with the other 3 lines m. At this stage it is useful to order the distances Akm along straight-line k to the respective intersection points (Xkm,ykm) by size as follows: {'Akm max {km mid {km}min m 1 4,m k (12) With segment lengths kl and k2 being defined as follows: {kl lkm }max km mid; and (13) k 2 {km mid I-{km} min (14) a length ratio Rk for line k is then defined as: Rk =min[ kl <1 L k2 k1 j 694684 -16- 0 With the parameter pairs of the embedded straight-lines suitably chosen, Sthe ratio Rk of each line k is distinct from that of every other line m. Because the ratio Rk is invariant with respect to affine transforms, the correspondence between each straight- 00 line before and after affine transform can be uniquely determined.

5 From the correspondence between each straight-line before and after affine transform, the correspondence between the parameter pairs and is uniquely determined. Substituting Equation (11) into Equation the intersection point N (Xkm,ykm) is given by: km I sin k -sin r (16) Ykm sin(p- ik m) cos k COSo rk Because the straight-lines are non-parallel, sin(i3 0. Respective angles 8k and are preferably chosen such that sin 2 0.25 holds. Equivalently, the angle at which lines k and m intersect preferably satisfies 300 k/ If <150'.

Similar to Equation the transformed point (km'ykm) corresponding to the intersection point (Xkm,ykm) is given by: S(km 1 sin sin (17) km sin/ k m COS k cos By substituting the parameter pairs and into Equations (16) and (17) respectively, and then solving Equation using the corresponding intersection points (xkm,Ykm) and (kYkI), the affine transform parameters {all, a12,a21, a2z2,X0,yo} may be obtained as is described below.

The preferred method of affine parameter estimation uses least squares fitting (LSF) between the intersection points (xkm,ykm) of the lines and the intersection points 694684 -17after affine transform. This LSF is greatly eased because the correspondence between the intersection points (Xkm,ykm) and (Xkm, km) is known. Without the unique 00 identification of the correspondence between intersection points (Xkm,ykm) and (xkm, km) it is necessary to evaluate the LSF for all possible permutations, in this case 6!=720.

Let the detected and (uniquely) ordered intersection points be denoted by 0 (Xkm km). As before the original (untransformed) intersection points are (xkm,y), while the transformed intersection points are (km, ym). An error energy is defined as the Euclidean norm measure E: k-I N E Ei km km)2 (.km 2 km)2 (18) m=1 k=1 Error energy minimization with respect to a transform parameter p gives six equations: 2m km Ykm) km=0, atminima, (19) p I k=1p apf with the transform parameter p being one of the parameters all, al2, a21, a22, xo andyo (Equation defining the transform.

Fortunately the 6 equations represented in Equation (19) simplify greatly because of the partial derivatives with respect to p aj: -km km 0 km 'km km km 0 a, aa 12 'a 21 aa 22 0 0 0-5km km kn k km km k 0, k km, 0, aa, aal 2 aa 2 1 aa 22 ax 0 The six equations represented in Equation (19) are then essentially equivalent to matching the following summations: 694684 18 00 ZjkmXkm k-I N I I m.Ym k-I N I- >LXkXkm m=1 k=1 k-I N ZkYkm m=1 k=I k-I N I I kmXkm m=1 k=1 (21) ~Zkm- k N k-I N 11 myk =ZZI k Ykm ZXIkm k-I N I I i EZI km m=1 k= m=1 k=1 ZX- krN k-I N Zj~m ZZ~km I= m=1 k=1 Filling in the full transform parameters from Equations (19) and (20) gives a pair of 3x3 matrix equations with the same symmetric matrix multiplier M: I I mXkm m=I k=1 k-I N Y1XkmXkm k-I N I I YknXkm m=1 k=1 ElXkmYfl j: 1 YkmYkm m1 k= k-I N 11ZXkn k-I N IXYkn m= k1 ra 1 2 M a 12 (22) xo IYO Z YkmXkm m =1= k-I N Y Y Ykmk mI k=1 (a2Il =M a 2

YO)

(23)

(S"

M=SK

k-I N 11XkmXkm k-I N Z Xkn m=I k=1 Y1 kmYkm I IYkmrn m= =1 k-I N LIXkm k-I N Z ZYkn m=1 k1 (24) The inverse of matrix M is another symmetric matrix M-1, where 694684 oo 0 -19- SSxy Sx S, SSx SSxx SxSxy SxxSy M-1 I SS SXS Y IMI SX .sY SxyyS SxSyy SxSxy SxxS SxySxy SxSyy MI detM -SSxySxy +2SxSxyS, -SxxSyS SxSxSy +SSxxSy, (26) Inverting Equations (22) and the final solution for the least squares estimate of all six transform parameters is explicitly: a 1 1 ao

M-'

vX0 k-1 N XkmXkm m=l k=l k-1 N SYkn. Ykm m=1 k=1 k-1 N m=1 k=1 k-1 N Skm kmk m=1 k=1 k-1 N I kI Ykm m=1 k=l k-1 N SZ km Sm=1 k=1 (27) (28) a y2 M-1 The minimum error energy E may be calculated using Equation and may be compared with an allowable limit. If the error energy E is greater than the allowable limit, then it is assumed that an affine match has not been found. It is also possible to calculate maximum point deviations by comparing intersection points and (Xkm which is useful for establishing a reliability measure of the estimated transform. Another useful parameter, especially with regard to reliability estimation, is the normalized energy; namely the error energy E divided by the centred mean square point distribution G, defined as follows for any set of points n=l->M: 694684 8 )2 -)2 cE (29) k

G

n=l 00 where the centroid of the measured point distribution is defined as (xi, y).

When choosing the parameter pair {rn,f3n} of the lines to be embedded, one Soption is to choose 4 lines having angles f3n that are uniformly distributed over the full angular range, i.e. at 00, 450, 90, 135°. Fig. 6 shows the 4 lines arranged in such a fashion.

A number of arrangements of the four straight-lines give highly symmetric results. The arrangement shown is less symmetric. It contains six intersection points (XkmYkm) that can be distinguished from a transformed version of itself. Fewer intersections are possible in degenerate arrangements where more than two lines meet at a point, but are not considered here.

Also shown on Fig. 6 are the lengths of the line segments, defined with reference to the length p of segment 600, in terms of a further parameter e. The length ratios R,, defined in Equation of line segments as a function of the parameter e are calculated as shown in Table 1: Table 1 694684 -21- Fig. 7 shows the length ratios R, of line segments as a function of the parameter Se. It can be seen from Fig. 7 that the particular value e 0.31 gives 4 quite different and 0 distinct length ratios In the preferred implementation (e=0.31) 4 straight-lines are embedded having parameters as follows: "fl =-67.5,r 1 f12 +22.5',rz p(l tfl 3 =-22.5,r 3 =p/2 +T) 64 =+67.5 ,r 4 where p is set to some fraction of the image width. Note that the negative values of r, are equivalent to their positive counterparts, provided the corresponding angle 83, is rotated by 1800.

The preferred choice of orientation angles avoids any coincidence between any one straight-line n and image boundaries by the largest amount.

In order to use the properties of straight lines to detect the parameters of the affine transform applied to the copy of the watermarked image while embedding the watermark in the image of the document using spatially dispersed basis patterns, each straight line is embedded as a pattern. Each pattern has to have the property that it can be detected even after the image into which it is embedded has been affine transformed. According to the preferred implementation each pattern has variation in one direction only, that direction being perpendicular to the straight-line n. The pattern also has an axis, which may be an axis of (odd or even) symmetry or a nominated axis, which coincides with the embedded line. The patterns are typically generated from a one-dimensional basis function applied at the direction of variation and repeated parallel to the axis.

In a preferred implementation, the basis function is a complex homogeneous function of the form: 694684 -22- Sg(v)= IvQ+i IVI exp(ia logvD (31) where v is a one-dimensional coordinate, which is not necessarily positive, while a and q are constants. The basis function g(v) is preferably attenuated in areas where the basis function has a frequency above the Nyquist frequency of the image to which it is to be embedded. Equation (31) may be thought of as an amplitude function, amplitude modulating a phase function, with the phase function having a logarithmic phase. When such a complex homogeneous function g(v) is scaled, say by a factor a, the scaling only introduces a complex constant factor as follows: g(av)= aq+iag(v) (32) The advantage of the complex homogeneous function is that the auto-correlation of the complex homogeneous function is directly proportional to the cross-correlation of the complex homogeneous function with a scaled version of the complex homogeneous function. This 'scale-invariant' property allows a watermark comprising complex homogeneous functions to be detected in an image even after a scale transformation by correlating the image with the basis function.

Fig. 8 illustrates an example pattern generated using a real part of the basis function of Equation (31) as follows: Re{g(xcos ysin,6 (33) Note that masking has been applied at the area adjoining the axis of symmetry to remove values with a frequency above the Nyquist frequency of the pattern image. Also illustrated is the relationship of the pattern with the straight-line it embeds. It would be understood that the straight-line itself does not form part of the pattern, and would not be added to the image in the steps that follow. In illustrating the example pattern, pattern 694684

I

-23values have been mapped to values in the range of 255], with a value of 0 being represented as the colour black and a value of 255 being represented as the colour white.

The above describes the generation of the alignment patterns of the watermark.

The generation of the message patterns are now described in more detail with reference to Fig. 10 where a schematic flow diagram of a process 500 of generating message patterns is shown. The message patterns are preferably mathematically generated by the processor 105 through the use of a basis pattern equation and parameters.

The message to be embedded may be any message representable as a bit string, and is preferably encoded into the image by the placement of the respective message patterns such that a binary is encoded by the presence of the message basis pattern centred at a particular position, whereas a binary is encoded by the absence of the message basis pattern centred at that position. Accordingly, the message received in step 210 (Fig. 1) is used by the processor 105 to determine the translations (xkyk) of the copies of the basis pattern. As shall be set out in detail below, correlation of the watermarked image with the message basis pattern results in correlation magnitude peaks at those translations (xk,yk) where the message basis patterns were embedded. Accordingly, by adding a message basis pattern to the image at such a translation (xkyk), a "mark" is added in the image obtained through correlation. Therefore, adding the message basis pattern at translation (xkyk) is hereafter referred to as adding a mark at image coordinate (xkyk).

Process 500 starts in step 501 where the processor 105 converts the message into a number B of length b bits. For example, the message "2Bit" may be converted into the number: 00110010 01000010 01101001 011101002 (34) 694684 -24using the conventional 8-bit ASCII code, which translates to 843213172 decimal. In such a case where the 8-bit ASCII code is used, a complementary decoder will know that each letter of the message consists of an 8 bit number.

00 SIn an alternative implementation, the message is converted into a string of d bits which is then treated as a number containing d bits. In such a case the bit string has a variable length and such a representation of the message does not distinguish between bitstrings with differing numbers of leading zeroes. For example, the two binary messages 101101 and 00101101 cannot be distinguished, as both binary messages, when treated as numbers, have the value of 4510.

In order to encode a variable length message as a number, the string of d bits is simply pre-pended with a binary 1, with the pre-pended message treated as a number.

During decoding, the message may be decoded by reading the marks as a number, and then removing the leading binary 1 from the number to retrieve the encoded string of d bits.

Next, in step 503, the processor 105 defines a set of positions on the image, which corresponds to translations (xk,yk) where marks may be added to the image. A simple scheme for defining the set of positions for the translations (xkyk) is to define an KxK grid in the image, with the positions in the grid being identified by their respective array values with and grid position being the lower left corner grid position. In the preferred implementation, grid positions and are reserved for alignment marks, and grid position is kept open, as a mark at that position will be confused for an alignment mark. There are thus K 2 -4 positions provided for adding marks to encode the message.

The alignment marks form three corners of the KxK grid, and are used for the fine registration of the grid positions. Fig. 9 shows three registration marks 406 placed at 694684 grid positions and and forming an 'L'-shape. The positions of the k-alignment marks define the coordinate system. The remainder of the marks, such as marks 407 and 408, are message marks.

00 SStrictly speaking the use of alignment marks is unnecessary given the alignment patterns discussed above with reference to Figs. 4A to 8. In the case where only small scale or rotation transformations are expected, the addition of alignment patterns may be neglected and only the alignment marks may be used. This would require a basis pattern (to be used in at least the alignment mark positions that is scale and rotation invariant, in that the basis pattern, when correlated with a scaled and rotated version thereof, would form a magnitude peak at that position.

In the preferred implementation, both the alignment patterns and the alignment marks forming part of the message patterns are used, with the former providing a coarse but robust alignment, whereas the latter gives more sensitive but much finer alignment.

With the set of positions where marks may be added to the image defined, the processor 105 then determines in step 505 the number of marks 1 for encoding the number B. As set out above, an implementation for encoding a number B with length b bits into the image is to allocate a predetermined grid position (ij) to each of the b bits of the number B. If a bit of the number B is an 1, then a mark is embedded in translation (xkyk) that corresponds with the grid position, whereas no marks are embedded in translations (XkYk) that correspond with the grid position which in turn corresponds to O's in the number B. Thus, with this method, the number of marks I for encoding the message is equal to the number of 1 bits in the number B.

In order to improve reliability, a single extra mark and an extra set of grid positions may be used to indicate the number of 1 bits in the number B. For example, to encode an 8 bit message, a group of 8 grid positions may be used to encode the message, and a single mark may be embedded at one of 9 separate grid positions to encode the 694684 -26number of 1 bits used in the 8 bit message, thereby using between 1 and 9 marks to Sencode 8 bits. Thus, together with the three alignment marks, the total number of marks embedded into image is between 4 and 12.

00 SThe above implementation for encoding the number B into the image, while 5 effective, is not very efficient. The number of marks 1 required for encoding the number _B is proportional to the length of the number b in bits. As multiple copies of the basis pattern decreases the signal to noise ratio of the patterns, detectability of the patterns is (reduced as the number of copies increases. So, it is desirable to encode the number B using the minimum number of marks 1, i.e. adding the minimum number of copies of the message basis pattern to the image.

To use the minimum number of marks possible, numbers are assigned to combinations of marks starting from all combinations with 0 marks, all combinations with 1 mark, and so on while the number of marks r is increased. If the number of positions z available for placing marks is much larger than the message length b of the number B, only a small number of positions (ij) need to be marked.

Using I marks with z available grid positions (ij) for embedding those marks, by combinatorial analysis it can be shown that there are: z! l!(z combinations of grid positions (ij) available for embedding those marks. When the number of available grid positions z is large and the number of message marks 1 is small, then the combinations of grid positions (ij) available approximates z t which is significantly larger than either the number of available grid positions z or the number of message marks 1.

694684 -27- By assigning a separate number to each possible combination for embedding 1 or Sfewer marks, any number between 1 and 0( z! (36) i! (z may be encoded. For example, with 1600 available grid positions and up to marks, there are 29,643,848,128,206,309,473,068,360 combinations of marks, which is an number. Thus, using up to 10 marks and 1600 available grid positions, any number B containing between 0 and 84 bits can be encoded into an image.

Fig. 11 shows a schematic flow diagram of the preferred sub-steps for step 505 where the number of marks 1 for encoding the number B with z available grid positions is determined by the processor 105. In this implementation, the number B may be formed by addition of a lower bound number and an offset number, with the lower bound number being encoded by the number of message marks 1 such that the lower bound number is simply: 1Z! (37) The offset number is then encoded by the specific placement of the 1 message marks.

In sub-step 521 the number of marks 1 is initially set to 0. The processor 105 then determines in sub-step 523 whether the number B is smaller than zC, which is the number of combinations of grid positions (ij) available for embedding the I marks in z possible positions, or the largest number that can be encoded with the 1 marks. If it is determined in sub-step 523 that the number B is not smaller than i.e. that the number B is larger than, or equal to, the largest number that can be encoded with the 1 marks, then sub-step 525 recalculates the number B by subtracting the largest number that can be 694684 -28encoded with the 1 marks, i.e. from the number B, and sub-step 527 increments the Snumber of marks 1. The processor 105 then returns control to sub-step 523 from where 00 sub-steps 523 to 527 are repeated until the recalculated number B is smaller than ZC In sub-step 529 the number marks 1 and the recalculated number B is then stored by the processor 105 in memory 106 for later use. The sub-steps of step 505 end in sub-step 529. The recalculated number B now represents the offset number.

Referring again to Fig. 10, step 505 is followed by step 509 where the grid N positions (i 1 for placing the marks for encoding the (offset) number B are determined.

Step 509 also includes a number of sub-steps, which are shown in a schematic flow diagram in Fig. 12. The sub-steps of step 509 start in sub-step 530 where the following counters are set by the processor 105: c=l; t=0, and d=0. A counterfis also set equal to the number of marks required to encode the (offset) number B, and counter 6 is set equal to the number of positions z available for placing the 1 marks.

The processor 105 determines in sub-step 532 whether the counterfis equal to 0.

If it is determined that the counterfis not equal to 0, then the positions of all 1 marks have not been determined and the processor 105 proceeds to sub-step 534 where the counterf is decremented.

The processor 105 then decrements counter e and increments counter t in substep 536. The processor 105 also calculates in sub-step 538 a new value for counter d by adding 'C I to the previous value of counter d. This is followed by the processor 105 in sub-step 540 determining whether the counter d is smaller or equal to the (offset) number B. If the counter d is smaller or equal to the (offset) number B, then sub-steps 536 to 540 are repeated until the processor 105 determines in sub-step 540 that the counter d is larger than the (offset) number B, in which case the processor 105 in sub-step 542 calculates a new value for counter d by subtracting 'Cf from the previous value of counter d. This is 694684 -29- Sfollowed by sub-step 544 where the processor 105 sets a value for the hth position of a one dimensional array pos[h] as counter t. Thus, the hth mark of the I marks required to encode the number B is placed in position t of z possible positions. The processor 105 00 Sthen increments counter h in sub-step 546 and passes control again to sub-step 532 from where steps 534 to 546 are repeated until it is determined in step 532 that counterf-0, i.e.

that the positions pos[h] of all 1 marks have been determined, in which case the grid positions (ij) are determined by the processor 105 from the one-dimensional array positions pos[h] in sub-step 548. The grid position (ij) of the hth mark is: (pos[h],0) if (pos[h]<K-1) (remainder(pos[h]+l,K), quotient(pos[h]+l,K)) if (pos[h]<(K 2 and pos[h]>K-2 (remainder(pos[h]+1,K)+1, K-1) if (pos[h]>(k 2 (18) This concludes the description of the sub-steps of step 509.

Returning to Fig 10, in a final step 511 of process 500 marks are also added at the alignment mark grid positions and This then also concludes the description of process 500.

Figs. 13A to 13F illustrate the steps of encoding a message into an image by way of a simplified example. Fig. 13A shows an image into which a message "2Bit" is to be encoded by adding basis patterns to the image. Fig. 13B shows a simple basis pattern.

The message "2Bit" is used to determine the translations (xk,yk) where copies of the basis pattern 102 are to be added. Referring also to process 500 shown in Fig. 10, in step 501, the message "2Bit" is converted into the decimal number 843213172.

Step 503 follows where the processor 105 defines a set of positions where the basis pattern is to be added to the image. Fig. 13C shows a 10x10 grid, with the positions in the grid being identified by their respective array values with i,je0,..,9 and grid position being the lower left corer grid position. In the example alignment marks 694684 will not be used for simplicity. There are thus 100 positions provided for adding marks to Sencode the message.

With 100 positions, the number of marks required to encode the message is 00 Scalculated by the processor 105 in step 505 to be 6. Step 509 follows where the grid positions (ij) for placing the 6 marks for encoding the message are determined. The processor 105 calculates these positions to be and Those positions are indicated in Fig. 13D. Fig. 13 E shows a copy of the basis pattern (placed at each of the positions. Finally, the basis patterns are added to the image as shown in Fig. 13F.

Having described the process 500 of generating message patterns, the preferred basis pattern used in the message patterns is now described. The detection of the possibly transformed translations of the alignment marks is performed by using process 900 described below in which a basis pattern g is correlated with an image containing the alignment mark. In the case where the alignment pattens are not used in order to compensate for any transforms applied to the image in the copying process, the basis pattern g used for the alignment marks has to have the property that correlation peaks will still be formed at the transformed translations of the alignment marks even if the image containing the marks is scaled and or rotated. Accordingly, the basis pattern g must correlate with a scaled and rotated version of itself such that the basis pattern equation y from which the basis pattern g is formed has the property: y(r,O) y(a.r,O 0)0 (39) wherein are the polar co-ordinates of a pixel, a is a positive real scaling factor, q9 is a rotation angle, c is a complex number not dependent on either of the coordinates r or 0. The basis pattern g of the preferred implementation is in the form: 694684 -31wP/k w (x,y).X2 +y,2 eikO eqe with k, s and a. being parameters of the basic patternm and w(xy) is a window 00 function. The window function w(x,y) may be used to remove or de-emphasise a central region of the basis pattern g having frequencies above a predetermined frequency. Figs.

14A and 14B show the real and imaginary parts respectively of a typical basis pattern Umk with parameters k=50 and am=50. As the basis pattern ,Umk has negative values in its real and imaginary parts, the values of the basis pattern ,Umk have been normalised around a value of 127, with all values ranging from 0 to 255, 0 represented in black, 255 represented in white, and intermediate levels represented in levels of grey. Only the real part of the basis pattern umk is used to form the basis pattern.

In the case where the alignment patterns are used in order to compensate for any transforms applied to the image in the copying process, the preferred basis pattern used for the alignment marks and the message marks is a random noise pattern, such as that shown in Fig. 15A. Fig. 15B shows a magnified version of the random noise pattern shown in Fig. 15 B.

The watermark is then formed from the alignment patterns representing the respective straight lines and message patterns (which preferably include alignment marks) by summing all the patterns together. As set out above, the watermark so formed is then added to the image in step 240 (Fig. Requantisation into suitable values in a fixed range 0 to 255) is typically required to avoid clipping. Direct rounding or some form of dithering, e.g. Floyd-Steinberg error propagation, is suitable for the requantisation.

The detection of the various patterns in the watermark performed in step 320 (Fig. 3) is now described in more detail. In particular, the detection of the various patterns forming the preferred watermark described with reference to step 230 is described.

694684 -32- SFirstly, the alignment patterns representing the 4 lines are detected, and the affine transform parameters all, al2, a21, a22, xo and yo (Equation representing the various transforms that have been applied to the printed copy to derive the copy obtained in step 00 S310 (Fig. 3) are estimated in method 700, a schematic flow diagram of which is shown in 1 5 Fig. 17.

Method 700 starts in step 710 where a projective transform is applied to the image of the copy obtained in step 310. The projective transform accumulates energy by Ssumming values along straight lines in the image. The Radon (or equivalently Hough) transform is one such projective transform that may be used in step 710 and is defined as: +D +M J f(x,y)S(r xcos- ysin O)dxdy (41) -cO -co wherein (xy) is a function representing the image of the copy. In order to derive a convenient implementation of the Radon transform for a discrete dataset, a useful correspondence between the projection of the image function and the slices of the function's Fourier transform is used, that correspondence being known as the "projectionslice theorem".

The projection-slice theorem states that the one-dimensional Fourier transform of a projection of a two dimensional function is equal to a radial slice of the twodimensional Fourier transform of that function. Note that +o +0 J f(x,y)exp[- 2i(ux vy)]dxdy (42) -Oo -O wherein X(u,v) is the 2-D Fourier transform of image A "quasi polar space" is defined in which the angles are in the range and in which signed distance is in the range By defining quasi-polar coordinates in the 694684 -33- Fourier domain, the coordinate transform is u q cos v q sin and one form of the projection-slice theorem is obtained for the Fourier polar angle corresponding to the 00 Radon projection angle 0 0 as follows: Ro (r)exp[- 2i(rq)ldr X(q cos 0, q sin (43) ^1-m Equation (43) is useful because it allows estimation of (the Fourier transform of) a Radon projection as a radial slice of the 2-D-FFT of a discrete image. This suggests that a discrete Radon transform may be evaluated by first performing a 2-D FFT followed by a Cartesian to polar remapping, using a suitable interpolation such as bicubic, or chirp-z to perform the resampling.

The image X(x,y) having embedded therein N patterns based on one-dimensional basis function g(v) having an orientation angle and perpendicular displacement r, and ignoring the pixel values of the image itself, may be written as: g(xcos +ysin,,) (44) n=l When the projection transform is applied to a pattern having a variation in only one direction with that direction being at an orientation angle then the values of the projection are significantly higher when the angle 0 is equal to the orientation angle compared to all other values of angle 0. Fig. 19 illustrates the projection of the undistorted example pattern illustrated in Fig. 8, plotted in the quasi-polar space. It can be seen that the values of the projection are significantly higher when the angle 0 is equal to the angle The projection also has symmetry about the offset distance rn.

694684 -34tt Hence, by applying the Radon transforms on the functions in Equation it can be shown that the Radon transform of such a function is constrained to a line where 00 fi8 as shown in the following:

N

g(r -i5 n=l 5 Having concentrated the image function X(x,y) having the embedded patterns (Ni g(v) onto N single lines in quasi-polar space by the use of the Radon transform, it is further possible to concentrate the energy contained in each of the lines into a single point (or into a small region near a point) by using l-D quasi-radial correlations (in coordinate r) detection for all values of the polar angle 0.

Accordingly, step 720 follows where the processor 105 performs 1-D correlations between the projection and the basis function g(v) in the quasi-radial coordinate r for all possible values of the polar angle 0. The term "correlation" also includes phase correlation and phase correlation scaled by energy. The resulting correlations have peaks at quasi-polar coordinates In step 730 the processor 105 then finds the absolute peaks of the correlation.

The orientation angle fin and perpendicular displacement of each distorted embedded pattern is directly available from the quasi-polar coordinates of the peaks. Also, the parameters pairs f, of the embedded straight-lines are available.

The pre-processing stage of the detector finds a number of candidate lines, typically about 64, based on the highest correlation magnitude peaks. From this number, all combinations of four lines are scrutinized. The number of possible combinations is 64!/(60!4!)=635376. If the angles of the four lines do not all differ by at least 15', then that combination is dismissed and does not pass on to the next stage. Assuming a 694684 combination of lines passes this stage, then the six intersection points are calculated using Equation (10) and the four sets of line segment ratios R, evaluated. The ratios R, are 00 then placed in order of increasing size. A merit function for the closeness of fit to the expected line ratios (also placed in order of increasing size) is then evaluated as: n=1 R (46) All combinations with a merit function below a certain threshold (preferably 0.1) are labelled as "candidate combinations" for further processing.

Note that other merit functions could be used, for example the above merit function could be weighted to reflect the line strength (in terms of the line correlation magnitude) and thus reduce contributions from weakly detected lines.

The following processing is then carried out for each candidate combination.

In step 740 the processor uniquely identifies the correspondence between each straight-line n before and after affine distortion. In the preferred implementation the length ratios of the lines after distortion are matched with the length ratios Rn before transformation to uniquely identify the correspondence between each straight-line before and after affine transformation.

Using Equation (17) and in step 750 the processor 105 next calculates the intersection points (km, kn). The affine transform parameters {all, a12, a21, a22, Xo, Yo} are then estimated in step 760 using LSF described above with reference to Equations (18) to The final affine transform parameters are chosen to be those producing the lowest minimum error energy E (Equation over all candidate combinations.

Method 700 terminates in step 770 where the estimated affine transform parameters {all, a12, a 2 1, a22, xo, Yo} are used to invert the affine transforms.

694684 -36- Now that the affine transforms are inverted, the message patterns are detected and the message is decoded from the location of the message patterns. Fig. 18 shows a schematic flow diagram of a method 900 for detecting the message patterns and decoding 00 the message.

Method 900 starts in step 940 where the image 910 resulting from method 700, where the affine transforms are estimated and inverted, is correlated with the basis pattern 935 used to form the alignment patterns and the message patterns. The basis pattern is preferably mathematically generated by the processor 105 through the use of the basis pattern equation and parameters of the basis pattern. As the preferred basis pattern is a wideband signal, correlation between the basis pattern and the image resulting from method 700 produces another image containing sharp peaks at translations (xk,yk)' where the basis pattern exists in the image.

The correlation magnitude peaks at translations (xk,yk)' include those added as three alignment marks, 1 message marks, and a mark indicating the value ofl 1. In step 950 the translations (xk,yk)' of the alignment marks are found. By ensuring that the I message marks and the mark indicating the value of 1 are kept completely inside the square defined by the three alignment marks when adding those to the image, the alignment marks may easily be distinguished from any other marks by finding a bounding box around the detected correlation magnitude peaks, and then selecting the three points furthest from the centre of the bounding box.

Even though the method 700 described with reference to Fig. 17 inverted affine transforms applied during the copying process, the alignment marks added as part of the message patterns allow for even finer alignment, thereby allowing scale, rotation and translation transforms still remaining after the affine transforms have been inverted to be compensated for with higher precision. There is a price to be paid for this additional 694684 -37processing, but the reward is higher accuracy when detecting the message marks. The increased accuracy allows for a higher density of message patterns to be distinguished from each other.

00 SAccordingly, in step 952 that follows the processor 105 determines the linear transformation required to register translations (Xkk)' with the translations (xkyk) of the message patterns added to the document image. In particular, a rotation angle may be V estimated using the vector between the two detected alignment mark translations which are furthest apart. Similarly, a total scaling factor may be estimated using the distance between these two alignment mark translations A shear factor may be io estimated by measuring the angle between the three alignment mark translations (xn,yn)'.

A change in aspect ratio may be estimated by measuring the difference in the length of the horizontal side and the vertical side of the shape formed by the three alignment marks. The middle point of the shape may be used to define the coordinate system origin of the grid.

The six parameters (angle, scaling, shear, aspect ratio, and horizontal and vertical translation) define the linear transformation. In step 955 registration is performed by inverting the linear transformation.to restore the transformed image to its original size and orientation. Scaling and rotation may be performed by a variety of resampling algorithms. Typically a high quality resampling using bi-cubic interpolation, Fourier interpolation, or even a non-linear resampling scheme may be used.

Registration may also include focusing because the correlation magnitude peaks provide a nice smooth variation in magnitude with respect to changes in the focus of imaging systems. Focus can be estimated by correlation peak width. An advantage of a width based measure of focus is that it is normalised and does not depend upon the 694684 -38absolute peak level. Registration may further include aspect ratio correction and shearing Sto some extent, due to the strength of the correlation magnitude peaks.

With the translations of the correlation peaks registered, the translations of the 00 detected alignment marks corresponds to grid positions and of the KxK grid. The grid positions of the r message marks are then found in step 959.

Because the spatial transformation of the image registered in step 955 may affect V) the accuracy of detected translations (xn,yn)' of the correlation magnitude peaks, it may be useful to register the message-marked image 910 using the computed transformation parameters, and to correlate the resulting image with the basis pattern to find the translations of the 1 message marks more accurately.

Method 900 continues in step 960 where the processor 105 converts the grid positions (ij) of the 1 message marks to a one-dimensional array pos'[h], with hl For example, with the hth message mark being on grid position the value of array pos'[h]is: i ifj=0 jxK+i-1 if 0<j<K-1 jxK+i-2 ifj K-1 (47) The above numbering system explicitly excludes the alignment marks from the numbering.

The processor 204 then converts the array pos'[h] to a number B' in step 962.

Step 962 is described in detail below with reference to Fig. 20. In step 964 the number B' is converted to a string. For example, the binary number: 01000010 01101111 011000102 (48) 694684 -39and using the ASCII code, is converted to the string "Bob". The result is reported in step 965 by the processor 105 by displaying the string on the display device 114.

00 SStep 962 is now described with reference to the schematic flow diagram in Fig.

20. Step 962 converts the array pos'[h] to a number and is complementary to step 505 shown in Fig. 11 where only an offset is encoded by the placement of the marks, while a n lower bound is encoded in the number of marks r. Step 962 starts in sub-step 830 where N the following counters are set by the processor 105: h=0; t=0, and d=O. A counterfis also set equal to the number of marks 1, and counter e is set equal to the number of positions z available for placing marks.

The processor 105 determines in sub-step 832 whether the counter t is equal to the number of message marks 1. If it is determined that the counter t is not equal to the number of message marks 1, then the processor 105 proceeds to sub-step 834 where the counter t is incremented. The processor 105 also calculates a new value for counter d in sub-step 835 by adding to the previous value of counterf, and returns control to step 832.

Sub-steps 832 to 835 are repeated until the processor 105 determines in sub-step 832 that the counter t is equal to the number of message marks 1. Counter d now has the value of the lower bound of number The processor 105 then proceeds to sub-step 838 where the counter t is set to 0. This is followed by the processor 105 in sub-step 840 determining whether the counterfis equal to zero. If the counter a is not equal to zero, then the processor 105 proceeds to sub-step 842 where counter f is decremented and counter c is incremented.

This is followed by sub-step 844 where the processor 105 increments counter t and decrements counter The processor 105 determines then in sub-step 846 whether 694684 the value for the hth position of the one dimensional array pos'[h] is equal to the counter t.

If the value for the hth position of the one dimensional array pos'[h] is equal to the counter t, then control is returned to sub-step 840. Alternatively, if the value for the hth position 00 Sof the one dimensional array pos'[h] is not equal to the counter t, then the processor 105 calculates a new value for counter a in step 848 by adding 'Cf to the previous value of counter d, and returns control to step-sub 844.

t Once the processor 105 determines in sub-step 540 that counterf is equal to zero, ,I the sub-steps of step 962 ends in sub-step 850. The value of counter d now represents the number B'.

A number of refinements may be applied to methods 500 (Fig. 10) and 900 (Fig.

18).

In a first refinement, the pixel values of each basis pattern p is multiplied by a separate scaling constant when the message marks are formed. The separate scaling constant for each copy of the basis pattern p is chosen such that the detectability of all the copies are normalised. Accordingly, copies of the basis pattern p to be added closer to the edge of the document image and which would therefore be cropped when it is added onto the document image may be scaled slightly higher than those copies that would be complete when added to the document image.

It is desirable for the 1 message marks not to be placed too close to each other, thereby ensuring that the marks are more distinguishable after the correlation step 940. In a further refinement a neighbourhood of subsequent grid positions is excluded when the grid positions (ij) are determined by the processor 105 from the one-dimensional array positions pos[h] in step 448 (Fig. 12). For example, if the 10 grid positions (ij) surrounding a mark grid positions (ij) are excluded, then the number of combinations available is reduced from ZCT to z- 9 r'C. This exclusion has a negligible effect on the total 694684 -41length of message that may be encoded. The positions that are excluded during encoding must also be excluded during decoding.

In a fourth refinement, multiple basis patterns are used, with each basis pattern 00 orthogonal to all other basis patterns. For example, if the basis patterns used have the 5 form of Equation different basis patterns that are orthogonal to all other basis patterns may be generated by choosing different parameters for each pattern. In this case, multiple marks but from different basis patterns may be added to the same grid position while remaining separately detectable.

In the implementation described, a rather trivial mapping is used between the rectangular grid positions and the one dimensional array pos[h], in that the mapping is sequential. The message may be encrypted by employing a more sophisticated mapping between the rectangular grid positions and the one dimensional array pos[h]. For example, the mapping may be pseudo-random, with the seed (or key) and the random number generator known only to the encoder and the decoder of the message.

Furthermore, the positions for placing the marks do not have to be on a rectangular grid. Some examples of different position definitions are: a space filling curve, e.g. the Peano curve; and a spiral, with array pos[1] in the center of the spiral and array pos[p] for larger p values occurring nearer the outside of the spiral.

In yet another implementation a hexagonal position grid is used, thereby ensuring adjacent positions are spaced more optimally, thus allowing a larger number of positions to be packed into a smallest space.

In yet further refinements, provision is made for message verification and error correction. One method of verifying that the message has been decoded accurately is to incorporate a checksum into the number B when the message is converted to the number B in step 501 (Fig. 10). This will increase the length of the number B somewhat, but 694684 -42provides a way for a decoder to increase the certainty that the message has been decoded accurately. For example, by adding a checksum to a 10 bit number B, a successful decoding will have a likelihood of error of only 2 10, or 10 3 00 Adding a checksum will not increase the robustness of the message encoded using the message marks, but only increases the confidence that the message has been decoded successfully. To increase the robustness of the message, error correcting codes tt can be used.

NBecause the method 500 of encoding the message into the document image does not store the message directly in binary, it is not possible to use standard error correcting codes. However, by encoding several separate messages using disjoint subsets of positions, it is possible to introduce redundancy into the encoding so that if one or more of the sub-messages is corrupted, then the full message may still be recovered.

For example, assume that the message has been converted to a number B having a length of 64 bits. With a 45x45 grid, and using 2000 possible position of that grid, step 505 (Fig. 10) calculates that 7 message marks are required. However, if any error were to occur with any of the 7 message marks, the message would not be recoverable. By storing the message redundantly across several groups of the 2000 possible positions, the chances of both detecting and correcting errors correctly are greatly increased.

Two methods can be used for partitioning groups of the possible positions.

If the number of possible positions z is large, disjoint subsets of these possible positions may be assigned to different groups of positions. Using a single basis pattern pu for encoding, the message, or parts of the message, may then be repeated in the different groups of possible positions. For example, with 2000 possible positions, 4 groups of 500 positions may be formed. If it is further decided to only use three message marks within each group, thereby using a total of 12 message marks instead of 7 message marks will be 694684 -43used. Three message marks in 500 possible positions allows 25 bits to be encoded in each Sgroup. If the 64 bit number B is split into three parts A, B, and C having lengths of 20, 22 and 22 bits respectively, within the four groups A, B, C, and A*B*C (where is the 00 exclusive-or operator) may be encoded, with each group having a 3-bit checksum added.

This checksum will give around 88% probability of detecting an error in any of the four groups.

The number B may be retrieved from any three of the four groups as follows: A, B, C number B is (ABC) A, B, A*B*C C A, A*B*C, C: B A*B*C, B, C: A If no checksum error is detected in any of the groups, then the message may be decoded from any three of the groups. By decoding the message twice, using two sets of groups, it is possible to verify that the decoding was accurate: it is extremely unlikely (chances smaller than 10 7 that errors occurred in two or more groups, with correct checksums, with two erroneous messages decoding to identical messages.

If a checksum error is detected in one of the groups, the other three groups may be used for decoding the message. In this case, an error in any of the three remaining groups will have a relatively large chance of being undetected, and hence causing an undetected decoding error.

If a checksum error is detected in more than one group, it will not be possible to decode any message.

Alternatively, by using multiple different basis patterns gi, the same set of grid positions may be used with each basis pattern gi for encoding the message multiple times.

Each copy of the message is independently decoded by correlating the watermarked image with the respective basis patterns /i.

694684 -44- In yet another refinement, separate basis patterns are used for the alignment Smarks and the message marks. If the alignment marks are encoded using their own basis 00 pattern, then the decoder may simply look for exactly three peaks in the alignment Spattern, removing the chance of message marks being mistaken for alignment marks.

If decoding time is not an important consideration, a separate basis pattern ,Ui may be used for each individual message mark. This would completely remove any interference between message marks, giving a slightly improved chance of watermark Sdetection and decoding the message.

The foregoing describes only some implementations, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the implementation(s) being illustrative and not restrictive.

In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including", and not "consisting only of'. Variations of the word "comprising", such as "comprise" and "comprises" have correspondingly varied meanings.

694684

Claims (13)

1. A method of printing a watermark onto a document, said method comprising the steps of: receiving an image of said document; generating the watermark from a spatially dispersed basis pattern; adding said watermark to said image to form a watermarked image such that the peak values of said watermark are substantially the same as that of the peak image values; and printing said watermarked image.

2. The method as claimed in claim 1 further comprising the steps of: determining edge regions of splines appearing in said document image; expanding said edge regions to form a mask; and attenuating said watermark in regions corresponding with said mask before said adding step.

3. The method as claimed in claim 2 wherein the extent of expansion of said edge regions is dependent upon a feature of said splines.

4. The method as claimed in claim 2 wherein said splines represent text, and the extent of expansion of said edge regions is dependent upon the font size of said text.

The method as claimed in any one of claims 1 to 4 wherein said watermark comprises: 694684 46 alignment patterns for allowing transforms applied to the printed watermarked image to be determined in a detection process; and message patterns for encoding a message. 00

6. The method as claimed in claim 5 comprising the further steps of: obtaining a digital copy of a degraded version of the printed watermarked image; detecting said alignment patterns; determining transforms applied to the printed watermarked image using said alignment patterns; inverting said transforms; detecting said message patterns; and decoding said message from said message patterns.

7. A method of watermarking an image, said method comprising the steps of: generating a watermark; and adding said watermark to said image to form a watermarked image, wherein said watermark is recoverable in a degraded version of the printed watermarked image when the content of the image is no longer legible due to the degradation.

8. Apparatus for printing a watermark onto a document, said apparatus comprising: means for receiving an image of said document; means for generating the watermark from a spatially dispersed basis pattern; means for adding said watermark to said image to form a watermarked image such that the peak values of said watermark are substantially the same as that of the peak image values; and 694684 47 O means for printing said watermarked image.

9. Apparatus for watermarking an image, said apparatus comprising: 00 means for generating a watermark; and means for adding said watermark to said image to form a watermarked image, wherein said watermark is recoverable in a degraded version of the printed watermarked image when the content of the image is no longer legible due to the degradation.

A computer program product comprising machine-readable program code recorded on a machine-readable recording medium, for controlling the operation of a data processing apparatus on which the program code executes to perform a method of printing a watermark onto a document, the method comprising the steps of: receiving an image of said document; generating the watermark from a spatially dispersed basis pattern; adding said watermark to said image to form a watermarked image such that the peak values of said watermark are substantially the same as that of the peak image values; and printing said watermarked image.

11. A computer program product comprising machine-readable program code recorded on a machine-readable recording medium, for controlling the operation of a data processing apparatus on which the program code executes to perform a method of watermarking an image, the method comprising the steps of: generating a watermark; and 694684 -48- Sadding said watermark to said image to form a watermarked image, wherein said ,watermark is recoverable in a degraded version of the printed watermarked image when the content of the image is no longer legible due to the degradation. 00 5

12. A method of printing a watermark onto a document, the method being substantially as described herein with reference to the accompanying drawings.

13. Apparatus for printing a watermark onto a document, the apparatus being substantially as described herein with reference to the accompanying drawings. DATED this 8th Day of APRIL 2005 CANON KABUSHIKI KAISHA Patent Attorneys for the Applicant SPRUSON&FERGUSON 694684