AU2005202688A1 - Anti Collusion Signal Embedding Method - Google Patents

Description

S&FRef: 725004

AUSTRALIA

PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT 00 00

\O

(N

0', Name and Address of Applicant: Actual Inventor(s): Address for Service: Invention Title: Canon Kabushiki Kaisha, of 30-2, Shimomaruko 3-chome, Ohta-ku, Tokyo, 146, Japan Kieran Gerard Larkin Peter Alleine Fletcher Stephen James Hardy Spruson Ferguson St Martins Tower Level 31 Market Street Sydney NSW 2000 (CCN 3710000177) Anti Collusion Signal Embedding Method Associated Provisional Application Details: [33] Country:

AU

[31] Appl'n No(s): 2004904704 [32] Application Date: 18 Aug 2004 The following statement is a full description of this invention, including the best method of performing it known to me/us:- 5845c -1- ANTI COLLUSION SIGNAL EMBEDDING METHOD SField of the Invention The present invention relates generally to the imperceptible watermarking of images and audio and, in particular, to the creation of watermarks which are not 00 00 vulnerable to a collusion attack in which an attacker attempts to recover a watermark by examining identically watermarked images or audio.

SBackground Several watermarking systems have been developed for imperceptibly embedding information in an image. The embedded information may later be retrieved, and is commonly used for determining the provenance of the image, recording information about the image that is not directly visible, or even storing information unrelated to the image.

For most applications, it is convenient for the watermarks to be made robust to image corruption. Robustness of the watermarks allows for the embedded information to be retrieved even after substantial processing of the image. The processing of the image may include both common image transformations, such as cropping, rotation, and contrast enhancement, and also malicious attacks directed explicitly at removing or damaging the watermark.

Under these constraints, it is difficult to reliably embed watermarks in a manner that the watermarks are both imperceptible to observers of the image, and also carry enough information to be useful in their intended application area. In general, as more information is embedded through the embedding of the watermark, either the perceptibility of the watermark increases, or the detectibility of the watermark is reduced following corruption to the image.

725004 -2- SA common feature of most robust watermarking systems is their use of a correlation operation at some stage during watermark detection. Correlation has two Sproperties which are extremely desirable for watermark detection. Firstly, because every value in a correlation image is a sum of the product of every pixel in the input images, it 00o 5 is possible for correlation to concentrate energy spread across a whole image into a single point, thus making it possible for correlation to detect a signal embedded at very low t signal strength. Secondly, there are efficient implementations of correlation using the r, Fast Fourier Transform, making watermark detection using correlation relatively fast.

When the watermark is made up from the summation of a number of basis patterns, the basis pattern is used for detection. For example, when the detection process utilises correlation, the watermarked image is correlated with the basis pattern to form a number of correlation peaks.

Another common feature of robust watermarking systems is their use of a perceptual mask. A perceptual mask modulates a watermark for a specific image to embed the watermark at a higher signal strength in regions of the image where the watermark is likely to be less visible, such as in a highly textured piece of granite, and the watermark at a lower signal strength in regions where the watermark is likely to be highly visible, such as a flat piece of sky.

In addition to robustness, imperceptibility, and the information carrying capacity of the watermark, it is further desirable for the watermark to be protected against collusion attacks.

There are two distinct forms of collusion attack. A first form of collusion attack is typically mounted in the case where multiple copies of a single image are available, but with each copy of the image containing a distinct watermark. In this case, by obtaining and averaging the multiple copies of the image, an attacker may reduce the signal strength 725004 -3- 0 of each distinct watermark, resulting in a copy of the single image in which none of the Sdistinct watermarks is detectible. This first form of collusion attack may be called "watermark corruption collusion attack".

A prior art method exists for preventing watermark corruption collusion attacks 00 5 by modifying each copy of an image to be watermarked. When the watermarked images N are combined, phase cancellations will occur between the different watermarked images, tn and the resulting image will be perceptibly worsened. Although effective, the method Ssuffers from the necessity to modulate the phase of, and hence corrupt, the input image.

This typically has perceptual effects in the images.

In a second form an attacker gathers many watermarked images, and uses common properties of the images in an attempt to determine information of a watermark common to the images. Using this information, an attacker may be able to remove or corrupt a similar watermark in another image, or at least further examine the watermark in an attempt to find weaknesses. This second form of collusion attack may be called "watermark reinforcement collusion attack".

An example of a watermark reinforcement collusion attack is for the attacker to obtain several images of identical size, the images containing different content but an identical watermark. By averaging corresponding pixels in each image, a new image is created. Because the images contain an identical watermark, the strength of the watermark in the combined image is relatively increased. The uncorrelated values in the images are relatively decreased. Accordingly, the new image contains substantially the watermark only. By multiplying the new image by a small value and subtracting the resulting image from one or more watermarked images, the watermark may be damaged, or even destroyed. In particular, the watermark may be damaged in the manner described above without substantially affecting the quality of the watermarked images. Indeed, 725004 -4- O because the attack removes all or part of the watermark, the attack may even improve the Sperceived quality of the watermarked images.

Watermark reinforcement collusion attack is possibly the most common form of attack used. Accordingly, the use of a watermarking method which is not vulnerable to 00 00 5 watermark reinforcement collusion attack would greatly improve its robustness. oO Summary It is an object of the present invention to provide a watermarking method which substantially reduces the vulnerability of the watermark to watermark reinforcement collusion attack.

According to a first aspect of the present disclosure, there is provided a method of embedding a watermark into an image or audio segment, said method comprising the steps of: maintaining a complex basis pattern; combining a predetermined number of instances of said complex basis pattern together to form a watermark, wherein each instance is multiplied by a different complex constant before combining; and adding said watermark to said image or audio segment.

According to a second aspect of the present disclosure, there is provided a method of embedding a watermark into an image or audio segment, said method comprising the steps of: maintaining a complex basis pattern; combining a predetermined number of instances of said complex basis pattern together to form a watermark; affine transforming said watermark; and adding said watermark to said image or audio segment.

725004 O According to a third aspect of the present disclosure, there is provided a method of embedding a watermark into a video sequence, said method comprising the steps of: Smaintaining a complex basis pattern; combining a predetermined number of instances of said complex basis pattern 00 00 5 together to form a watermark; and

IND

\O

Sadding instances of said watermark to frames of said video sequence, wherein Seach instance is multiplied by a different complex constant before adding to a respective Sframe.

According to a fourth aspect of the present disclosure, there is provided a method of embedding a watermark into a video sequence, said method comprising the steps of: maintaining a complex basis pattern; combining a predetermined number of instances of said complex basis pattern together to form a watermark; and adding instances of said watermark to frames of said video sequence, wherein each instance is affine transformed before adding to a respective frame.

According to a fifth aspect of the present disclosure, there is provided a method of detecting a watermark in a video sequence, wherein instances of a watermark are multiplied by a different complex constant before embedding the watermark to frames of said video sequence, said method comprising the steps of: multiplying each of a predetermined number of frames of said video sequence with a predetermined sequence of complex constants to form a modified sequence of frames; averaging corresponding pixels of said modified sequence of frames to form a candidate image; and detecting said watermark in said candidate image.

725004 -6- According to a sixth aspect of the present disclosure, there is provided a method Sof removing a tiled watermark from an image, said method comprising the steps of: determining a tile size of said tiled watermark; averaging corresponding pixels from a plurality of tiles to form a 00 5 watermark tile;

\O

S(c) replicating and tiling said watermark tile together to form a tiled image; t~t and subtracting a multiple of said tiled image from said. image.

According to another aspect of the present disclosure, there is provided an apparatus for implementing any one of the aforementioned methods.

According to yet another aspect of the present disclosure there is provided a computer program product including a computer readable medium having recorded thereon a computer program for implementing any one of the methods described above.

Other aspects of the invention are also disclosed.

Brief Description of the Drawings One or more embodiments of the present invention will now be described with reference to the drawings, in which: Fig. 1 shows a flow diagram of a method of removing a tiled watermark from an input image; Fig. 2 is a schematic block diagram of a general purpose computer upon which arrangements described can be practiced; Fig. 3 shows a flow diagram of a method of embedding a watermark into an input image according to the present disclosure; Fig. 4 shows a flow diagram of a method of detecting a message embedded in a watermarked image; 725004 -7- 8 Fig. 5 shows a flow diagram of a method of embedding a watermark in a video Ssequence in a manner which avoids watermark reinforcement collusion attack; and Fig. 6 shows a flow diagram of a method of detecting a message embedded in a video sequence, where the message has been embedded using the method of Fig. 00 oO 00 5 Detailed Description

(NO

Watermark reinforcement collusion attacks exploit a weakness common to many watermarking systems, the weakness being that the same watermark is embedded in multiple different images. This weakness allows for the images to be combined causing watermark reinforcement, thereby producing an approximation of the embedded watermark.

Because it is common for a perceptual mask to be used to modulate the watermark before the watermark is embedded in each of the images, the watermark as applied to each of the different images is not likely to be identical. Nevertheless, the uncorrelated perceptual masks, when added together, will tend to produce a relatively flat watermark. Without knowledge of the perceptual mask, it is not possible to remove the watermark from any of the images entirely. However, it may be possible to damage a watermark beyond detection.

An improvement in the above described watermark reinforcement collusion attack may be obtained by guessing the perceptual masking algorithm used during watermark embedding. This allows for each of the watermark images to be divided by their respective estimated perceptual mask before the images are averaged, thereby at least partially removing the effect of the perceptual masks and resulting in a better approximation of the embedded watermark.

As well as providing a way to attack watermarked images directly, the combining of images to reinforce the constituent watermark provides an attacker with a 725004 -8stronger watermark to analyze, helping the attacker to better ascertain the original Swatermarking method in order to recreate, read, or remove arbitrary watermarks using the same watermarking system.

Watermark reinforcement collusion attack is also effective when used against 00 00 5 video watermarking, in which every frame in a video sequence is watermarked. In this case, there are hundreds of watermarked images available for use by an attacker.

Although nearby frames are strongly correlated, and the averaging attack will tend to Sreinforce the signal strength of the content as well as the strength of the watermark, over the course of several scenes of the video sequence the content will tend to cancel each other.

A method has been suggested for reducing the effectiveness of watermark reinforcement collusion attack by using two different watermarks, where one watermark is the mathematical negation of the other. The two watermarks are then embedded in the images alternately or randomly. When a selection of images is combined together, the two different watermarks will cancel each other very effectively, and little, if any, of the watermark will be recoverable from the combined image.

However, this form of watermarking may be vulnerable to a more complicated attack, still allowing an attacker to combine images to produce a strengthened watermark which can be used for watermark removal or analysis. If a pair of unrelated images contain either an identical or negated pair of watermarks, the correlation, or better, phase correlation of the two images is likely to show a peak in the centre of the correlation image. The value of the peak will be positive if the two watermarks are identical, and will be negative if the two watermarks are negated with respect to each other.

Using this information, a sequence of watermarks may be combined by addition and subtraction, as with the previous watermark reinforcement collusion attack, thereby 725004 -9- O reinforcing the watermark and reducing the signal strength of the content of the images.

SIf the images come from a video stream, and the watermarks alternate in sign, then the Ssignal strength of the content of the images will be reduced much more quickly than it would if straightforward averaging were used. Thus, even though the intent of this 005 method of using two different watermarks is to reduce the effectiveness of watermark

INO

N reinforcement collusion attack, the result is that the attack may be even more effective at t recovering, and hence attacking, the watermark.

Because of the difficulty of obtaining a reference coordinate system in a watermarked image that may have been rotated, translated and/or scaled before detection commences, the watermark typically includes some registration information to allow detection of the complete watermark even after such image corruption. In many cases, this registration information is included by the use of a regular grid of tiles repetitively inserted into an image. During detection, the detector performs auto-correlation on the watermarked image, and examines the resulting pattern of correlation peaks. The relative scale, angle and position of these peaks are used to determine the original grid of tiles, allowing the watermarked image to be re-registered, or simply to provide a guide for detecting another watermark in the image.

Unfortunately, use of a regular grid of tiles in an image for embedding a watermark allows a collusion attack to be mounted using only the information available from a single watermarked image. Fig. 1 shows a flow diagram of a method 600 of removing a tiled watermark from an input image 601, with a tiled watermark being a watermark comprising basis patterns summed on a regular grid.

The method 600 is preferably practiced using a general-purpose computer system 100, such as that shown in Fig. 2 wherein the method 600 may be implemented as software, such as an application program executing within the computer system 100. In 725004 O particular, the steps of method 600 are effected by instructions in the software that are Scarried out by the computer system 100. The software may be stored in a computer readable medium, including the storage devices described below, for example. The software is loaded into the computer system 100 from the computer readable medium, 00 oO and then executed by the computer system 100. A computer readable medium having

IND

\O

such software or computer program recorded on it is a computer program product. The t use of the computer program product in the computer preferably effects an advantageous CI apparatus for removing a watermark from the input image 601 having a tiled watermark embedded therein.

The computer system 100 is formed by a computer module 101, input devices such as a keyboard 102, a mouse 103 and digital video camera 118, and output devices including a printer 115, a display device 114 and loudspeakers 117. A Modulator- Demodulator (Modem) transceiver device 116 is used by the computer module 101 for communicating to and from a communications network 120, for example connectable via a telephone line 121 or other functional medium. The modem 116 can be used to obtain access to the Internet, and other network systems, such as a Local Area Network (LAN) or a Wide Area Network (WAN), and may be incorporated into the computer module 101 in some implementations.

The computer module 101 typically includes at least one processor unit 105, and a memory unit 106, for example formed from semiconductor random access memory (RAM) and read only memory (ROM). The module 101 also includes a number of input/output interfaces including an audio-video interface 107 that couples to the video display 114, the video camera 118 and loudspeakers 117, an 1/0 interface 113 for the keyboard 102 and mouse 103, and an interface 108 for the modem 116 and printer 115. In some implementations, the modem 116 may be incorporated within the 725004 -11- O computer module 101, for example within the interface 108. A storage device 109 is provided and typically includes a hard disk drive 110 and a floppy disk drive 111. A CD- ROM drive 112 is typically provided as a non-volatile source of data. The components 105 to 113 of the computer module 101, typically communicate via an 00 00 interconnected bus 104 and in a manner which results in a conventional mode of

IND

Soperation of the computer system 100 known to those in the relevant art.

t Typically, the application program is resident on the hard disk drive 110 and r read and controlled in its execution by the processor 105. Intermediate storage of the program and any data fetched from the network 120 may be accomplished using the semiconductor memory 106, possibly in concert with the hard disk drive 110. In some instances, the application program may be supplied to the user encoded on a CD-ROM or floppy disk and read via the corresponding drive 112 or 111, or alternatively may be read by the user from the network 120 via the modem device 116. Still further, the software can also be loaded into the computer system 100 from other computer readable media.

The term "computer readable medium" as used herein refers to any storage or transmission medium that participates in providing instructions and/or data to the computer system 100 for execution and/or processing.

The method 600 of removing a tiled watermark from the input image 601 may alternatively be implemented in dedicated hardware such as one or more graphic processors, digital signal processors, or one or more microprocessors and associated memories.

Referring again to Fig. 1, the method 600 starts by calculating in step 602 an estimated perceptual mask of the input image 601. The calculated perceptual mask has to match as closely as possible that used during watermark embedding. For example, the 725004 -12absolute gradient over a 4x4 pixel window may be averaged to determine values of the Sperceptual mask.

SNext, in step 603, the processor 105 divides the input image 601 by the perceptual mask. The result from step 603 is then autocorrelated in step 604 to form an 00 00 5 autocorrelation image. In calculating the autocorrelation image, the result from step 603 may be weighted by a conical frequency filter. The autocorrelation image is then Sexamined in step 605 to find a grid, centered at the origin of the autocorrelation image, N indicating the spacing and rotation angle of the watermarking tiles. Note that steps 604 and 605 are not required if the watermark tile size for the input image 601 is already known 128x128 pixels).

With the size of the tiles known, the corresponding pixels in respective tiles in the result from step 603 are averaged in step 606 to create an estimate of a tile of the watermark. Next in step 607 an estimate watermark image of the same size as the input image 601 is formed from the estimate of the watermark tile by replicating the estimate of the watermark tile, taking care to ensure that the estimate of the watermark tile exactly overlays the grid determined in step 605.

In step 608 which follows the estimate watermark image is multiplied by the estimated perceptual mask (calculated in step 602) to yield an estimated normalized watermark. Method 600 ends in step 609 where the estimated normalized watermark is subtracted from the input image 601 to yield a "clean" image 610 in which the strength of the embedded watermark has been reduced, typically to undetectable levels.

It is possible to avoid reinforcement collusion attacks by using a combination of many different watermarks. However, such avoidance of reinforcement collusion attacks suffers from the need to perform multiple detection steps to recover each of the watermarks. In the case where enough images are available, the method may still be 725004 -13vulnerable to analysis by cross-correlation between the images to find those images using Sthe same watermark, and performing reinforcement collusion attack using images Scontaining the same watermark.

Fig. 3 shows a flow diagram of a method 200 of embedding a watermark into an 00 00 5 input image 201 according to the present disclosure. The method 200 is also preferably

IND

practiced using the general-purpose computer system 100 (Fig. 2).

t The watermark used in the method 200 is a type of watermark where a complex basis pattern is summed at different positions, and the real part of the summation forms the watermark. Also, scaled and/or rotated versions of the real basis pattern in the watermark may be detected using correlation with the complex version of the original basis pattern. This may also be viewed as a self-similarity property. Self-similar functions produce correlation magnitude peaks even when one of the correlated functions is rescaled. The complex basis pattern is maintained in the storage device 109.

An example of such a complex basis pattern is the scale and rotation invariant pattern g, which has a circular harmonic phase defined by the parameter k, where parameter k is an integer. Such a function is sometimes referred to as a logarithmic radial harmonic function [LRHF], and has the form: gmk 0) r r r ia e ikO (1) When the function gmk,,(r, 0) in Equation is scaled by a factor and rotated by a factor r, the scaling and rotation only introduce a complex constant factor as follows: gmk 0) P t ia eik l gmk (2) 725004 -14o This "scale and rotation-invariant" property allows the basis pattern to be Sdetected in an image even after a scale and/or rotation transformation has been applied thereto by correlating the image with the complex basis function.

In practice, a basis function useable as a scale and rotation invariant function only 00 5 has to approximate this ideal property sufficiently so as to provide a dominant correlation

IND

Speak.

l An example of a watermarking scheme that is inherently resistant to affine

O

C transformations (translation, scaling, rotation, anamorphic scaling and shearing), as well as complex constant multiplication (or "phase-shifting) takes as a starting point the twodimensional basis function derived from a one-dimensional complex homogeneous function p: xp+a (3) The derived two-dimensional basis function is a family of functions defined by Radon parameters

P):

hPp(x,y)= ,u(x cos f +y sin/f-p)= xcosp +ysin pp+,ia" (4) The function y) has the favourable property that any affine transformation thereof results in another function in the family it is closed under affine transformation), apart from a complex constant multiplier, or factor.

The overall watermarking scheme consists of the following parts: i. Embedding of a plurality of the real parts of basis patterns in a real image; ii. First detection step of Radon transform of image; iii. Second detection step of 1-D cross-correlation with the complex p function; and 725004 iv. Identification of embedded patterns from magnitude peaks of cross- Scorrelation.

The aforementioned scheme has the overall property that it detects the occurrence of embedded basis functions in an image. The detection is independent of any affine 00 00 5 transformations of the image (and therefore, basis functions), as well as independent of

IND

0 any complex constant multiplication factor of the basis functions (before the real part is tt embedded in the image). Such watermarking schemes may be termed as "affinity-factor invariant".

Other implementations are conceivable. For example, the cross-correlation may be replaced by another expectation maximization technique, which is not necessarily formally equivalent to cross-correlation, because of non-linear mechanisms. The important point is that the combination of affine transformation and multiplication by a complex constant factor has no substantial effect on the detection of the correlation peak magnitude. The aforesaid combination, for any collection of basis functions can be classified as a "Factored Affine Transformation". So, for example, multiplying a collection of (different) basis functions by different complex constants, then combining the functions and affine transforming the combination may be called a factored affine transformation of the combined basis functions. The affine transformations and factor multiplications of the bases may occur in any order.

The method 200 may use the watermark to encode a message 202 into the input image 201. The message 202 is encoded in step 203 as a sequence of parameters of complex basis patterns Bk making up the watermark, where the parameters may include, but are not limited to, centre positions of basis patterns Bk; functional form of the basis pattern Bk; 725004 -16- S- any parameters for substitution into the functional form; F1- orientation of basis patterns B amplitude scaling factor of the basis patterns Bk; etc.

00 Step 204 follows where each basis pattern Bk is then multiplied by a separate random or predefined complex constant Zk, including the constant 1, to yield modified S(message) basis patterns It is noted that the detectability of basis patterns such as 1-dimensional and 2-dimensional scale-invariant basis patterns, is not materially affected by multiplication by a complex constant zk, where Iz.l 1, because the real part of the basis pattern Bk' is taken to construct the watermark. It is further noted that this multiplication results in a constant phase change in the complex basis pattern Bk, but need not change the modulus thereof.

Step 206 follows where the processor 105 creates alignment marks A, to provide image alignment information by computing another sequence of parameters of complex basis patterns Bk. Each alignment mark A, is then also multiplied in step 207 by a separate random or predefined complex constant zj, including the constant 1, to yield modified alignment basis patterns A;.

A complex watermark is then formed in step 208 by the summation of the message and alignment basis patterns B' and Next, in step 210, the (complex) watermark is modified by a single random or predefined affine transformation, including the identity transformation, to yield a modified watermark. The affine transform may include one or more of rotation, scaling, translation, anamorphic scaling, and shearing).

If this transformation is not the identity transformation, then any alignment information 725004 -17available from the alignment basis patterns A; is not usable as a reference for restoring Sthe geometry of the watermarked image with respect to the original, but may be useful for recovering the message 202.

Method 200 then proceeds by, in step 211, calculating a perceptual mask from 00 oO 00 5 the input image 201, and in step 212 by multiplying the perceptual mask with the real part Sof the modified watermark formed in step 210. The method 200 ends in step 213 where a watermarked image 214 is formed by adding the result from step 212 to the input image S201.

Fig. 4 shows a flow diagram of a method 300 of detecting the message embedded in a watermarked image 301, where the message has been embedded using the method 200 (Fig. 3) described above. The method 300 is also preferably practiced using the general-purpose computer system 100 (Fig. 2).

The method 300 starts in step 302 where the processor 105 calculates a perceptual mask of the input image 301. The calculated perceptual mask is not exactly the same as the perceptual mask 211 (Fig. 3) used in the embedding method 200 to form the watermarked image 214, but if the same algorithm is used as that of the embedding method 200, then the perceptual mask calculated in step 302 should closely match the perceptual mask 211 used during the method 200 of embedding the watermark.

Next in step 303 the processor 105 divides the input image 301 by the perceptual mask. From the result of step 303 the alignment basis patterns A; are detected in step 304. In step 305 the message basis patterns B' are detected from the result of step 303 and using the alignment information available from the positions of the alignment basis patterns The positions of the alignment and message basis patterns A; and B' are typically detected using correlation with the complex version of the basis pattern used in 725004 -18- O the watermark. An embedded message 307 is then decoded in step 306 from the positions F1and parameters of the message basis patterns B k Because the affine transformed and/or phase modified versions of the basis pattern correlate with the complex version of the original basis pattern, the detection 00 oO 00 5 method 300 is identical to that used in detection of a watermark without the affine transform applied thereto, and/or without the basis patterns multiplied by complex Sconstants zi. Also, even though the respective basis patterns are modified such that they do not constructively add together, the same detection process detects all the basis patterns using the same complex basis pattern. As a result it is not necessary for the detector to reverse, or to even be aware of, the modifications made to the watermark.

The method 200 (Fig. 3) of inserting a watermark in an input image 201 and the method 300 (Fig. 4) of detecting the message embedded in a watermarked image 301 may be used for embedding a watermark into audio. In that case the basis patterns have a scale invariant property, and the affine transform includes scaling.

The method 200 (Fig. 3) of inserting a watermark in an input image 201 and the method 300 (Fig. 4) of detecting the message embedded in a watermarked image 301 may also be used for embedding a watermark into a video sequence, wherein the watermark is embedded into each frame of the video sequence. However, embedding the same watermark in successive frames of the video sequence still allows for watermark reinforcement collusion attack to be launched.

Fig. 5 shows a flow diagram of a method 400 of embedding a watermark in a video sequence in order to avoid watermark reinforcement collusion attack. The method 200 is also preferably practiced using the general-purpose computer system 100 (Fig. 2), with the source of the video sequence being the video camera 118, or a video sequence store on the storage device 109 or CD-ROM 112.

725004 -19- O The method 400 may also use the watermark to encode a message 401 into the g frames of the video sequence. In particular, the same message 401 is embedded in each Sframe of the video sequence. The message 401 is encoded in step 403 as a sequence of parameters of complex basis patterns making up the watermark in the manner described 00 00 5 with reference to step 203 (Fig. 3).

IND

SStep 404 follows where the processor 105 creates alignment marks to provide t image alignment information by computing another sequence of parameters of complex (basis patterns. A complex watermark is then formed in step 405 by the summation of the message and alignment basis patterns. In step 406 the complex watermark is rotated by an angle of (360xjxa) about the watermark image origin to yield a modified watermark, wherej is the frame number of the frame into which the watermark is to be embedded, and a is a constant such that 0_ a 1. If a watermark reinforcement collusion attack is attempted on sequential or randomly selected frames of the video sequence, then the watermarks will not reinforce each other due to the rotation introduced to each instance of the watermark.

The (complex) modified watermark is then, in step 407, multiplied by a complex constant eb, 2 i, wherein i 1, and b is a constant such that 0 b to yield a doubly modified watermark. Again, due to the fact that sequential or randomly selected frames of the video sequence differ by the multiplication by the complex constant e 2 and even without the rotation introduced in step 406, watermark reinforcement collusion attack will not reinforce the watermark.

Method 400 then proceeds, in step 408, by calculating a perceptual mask from input framej 402, and in step 409, by multiplying the perceptual mask with the real part of the doubly modified watermark formed in step 407. The method 400 ends in step 410 725004 O where a watermarked frame j 411 is formed by adding the result from step 409 to the Sinput framej 402.

Because the watermarks embedded in the respective frames of the video sequence are not identical, but rather a transformed version thereof, the watermark is not 00 005 visible to any viewer of the video sequence as a static watermarking pattern video.

IND

Fig. 6 shows a flow diagram of a method 500 of detecting the message Sembedded in a video sequence, where the message has been embedded using the method r 400 (Fig. 5) described above. The method 500 is also preferably practiced using the general-purpose computer system 100 (Fig. 2).

The method 500 starts in step 502 where the processor 105 calculates a perceptual mask of thej-th input frame 501. Because a transformed version of the same watermark has been embedded in each frame of the video sequence, the detection may start at any framej in the video sequence. The processor 105 then in step 503 divides the j-th input frame 501 by the perceptual mask.

In step 504 the result from step 503 is rotated by an angle of (-360xjxa) 0 about the frame image origin, wherein a is the constant used in step 406 (Fig. The result from step 504 is then, in step 505, multiplied by the complex constant e b 2 4, wherein b is the constant used in step 407 (Fig. Steps 504 and 505 operate to undo the rotation and multiplication with the complex constant ebi2' 7 i implemented in steps 406 and 407 respectively. It is noted that the frame image is also rotated and multiplied with the complex constant.

Because video frames are generally of lesser resolution than still images, less pixels are available to add a spread spectrum watermark. Video frames also generally contain more quantisation noise. As a result, such watermark is difficult to detect in a single frame when compared with watermark detection in a high-quality still image. To 725004 -21- O overcome the above, the corresponding pixels from the output from step 505 of several frames are averaged in step 506 to form a candidate image 507. Even though the content of successive video frames are highly correlated, because the content has been rotated and multiplied by the complex constant, the content of successive video frames now tend to 00 00 5 cancel each other out, leaving an image containing a watermark with relatively high

IND

Ssignal strength. The averaging also tends to reduce quantisation noise in the watermark.

t' In- the case where synchronization is lost between the frame number of the N embedding and that of the detection process, steps 504 and 505 introduce a constant rotation error, which is a constant multiple of (360xa) and a constant complex multiplication error of a power of e b 2 However, the watermark detection will still operate successfully if the detection method is given the frame number or a time-code from the frame, so that the decoding is effectively re-synchronized with the partial sequence.

From the candidate image 507 the alignment basis patterns are detected in step 508. In step 509 the message basis patterns are detected from the candidate image 507 and using the alignment information available from the positions of the alignment basis patterns. The positions of the alignment and message basis patterns are detected using correlation with the complex version of the basis pattern used in the watermark. An embedded message 511 is then decoded in step 510 from the positions and parameters of the message basis patterns.

The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive.

In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including", and not "consisting 725004 -22- Sonly of'. Variations of the word "comprising", such as "comprise" and "comprises" have 0 correspondingly varied meanings.

O 00 00 725004

Claims (16)

1. A method of embedding a watermark into an image or audio segment, said method comprising the steps of: 00 00 5maintaining a complex basis pattern; Scombining a predetermined number of instances of said complex basis pattern together to form a watermark, wherein each instance is multiplied by a different complex constant before combining; and adding said watermark to said image or audio segment.

2. The method according to claim 1 wherein said method comprises the further step of affine transforming said watermark prior to adding to said image or audio segment.

3. A method of embedding a watermark into an image or audio segment, said method comprising the steps of: maintaining a complex basis pattern; combining a predetermined number of instances of said complex basis pattern together to form a watermark; affine transforming said watermark; and adding said watermark to said image or audio segment.

4. A method of embedding a watermark into an image or audio segment, said method comprising the steps of: maintaining a complex basis pattern; 725004 -24- combining a predetermined number of instance of said complex basis patterns together to form a watermark, wherein the combination process entails a Factored Affine Transformation; and adding said watermark to said image or audio segments A method of embedding a watermark into a video sequence, said method comprising the steps of: maintaining a complex basis pattern; combining a predetermined number of instances of said complex basis pattern together to form a watermark; and adding instances of said watermark to frames of said video sequence, wherein each instance is multiplied by a different complex constant before adding to a respective frame.

6. The method according to claim 5 wherein said method comprises the further step of affine transforming instances of said watermark prior to adding to the respective frame.

7. A method of embedding a watermark into a video sequence, said method comprising the steps of: maintaining a complex basis pattern; combining a predetermined number of instances of said complex basis pattern together to form a watermark; and adding instances of said watermark to frames of said video sequence, wherein each instance is affine transformed before adding to a respective frame. 725004 S8. The method according to claim 7 wherein said method comprises the further step of multiplying instances of said watermark by a different complex constant prior to adding to the respective frame. 00 00 S9. A method of detecting a watermark in a video sequence, wherein instances of a watermark are multiplied by a different complex constant before embedding the C watermark to frames of said video sequence, said method comprising the steps of: multiplying each of a predetermined number of frames of said video sequence with a predetermined sequence of complex constants to form a modified sequence of frames; averaging corresponding pixels of said modified sequence of frames to form a candidate image; and detecting said watermark in said candidate image. The method according to claim 9 wherein instances of said watermark are affine transformed before embedding the watermark to frames of said video sequence, said method comprising the further step of reversing said affine transforms to each frame before said averaging step.

11. A method of removing a tiled watermark from an image, said method comprising the steps of: determining a tile size of said tiled watermark; averaging corresponding pixels from a plurality of tiles to form a watermark tile; 725004 -26- replicating and tiling said watermark tile together to form a tiled image; Sand S(d) subtracting a multiple of said tiled image from said image. 00 00 5 12. The method according to claim 11 comprising the initial steps of: IND Scalculating a perceptual mask of said image; and t dividing said image by said perceptual mask.

13. The method according to claim 11 or 12, wherein step comprises the sub-step of: (al) autocorrelating said image to form a correlation image; and (a2) determining said tile size from the spacing of peaks in said correlation image.

14. The method according to claim 11, wherein said multiple is determined by the step of calculating a perceptual mask of said image, wherein said multiple is values of corresponding pixels in said perceptual mask. Apparatus for embedding a watermark into an image or audio segment, said apparatus comprising: means for maintaining a complex basis pattern; means for combining a predetermined number of instances of said complex basis pattern together to form a watermark, wherein each instance is multiplied by a different complex constant before combining; and means for adding said watermark to said image or audio segment. 725004 -27-

16. Apparatus for embedding a watermark into an image or audio segment, O said apparatus comprising: means for maintaining a complex basis pattern; 00 00 5 means for combining a predetermined number of instances of said complex basis pattern together to form a watermark; t means for affine transforming said watermark; and C means for adding said watermark to said image or audio segment.

17. Apparatus for embedding a watermark into an image or audio segment, said apparatus comprising: means for maintaining a complex basis pattern; means for combining a predetermined number of instance of said complex basis patterns together to form a watermark, wherein the combination process entails a Factored Affine Transformation; and means for adding said watermark to said image or audio segments

18. Apparatus for embedding a watermark into a video sequence, said apparatus comprising: means for maintaining a complex basis pattern; means for combining a predetermined number of instances of said complex basis pattern together to form a watermark; and means for adding instances of said watermark to frames of said video sequence, wherein each instance is multiplied by a different complex constant before adding to a respective frame. 725004 -28-

19. Apparatus for embedding a watermark into a video sequence, said O apparatus comprising: means for maintaining a complex basis pattern; 00 00 5 means for combining a predetermined number of instances of said complex basis pattern together to form a watermark; and t means for adding instances of said watermark to frames of said video sequence, wherein each instance is affine transformed before adding to a respective frame.

20. Apparatus for detecting a watermark in a video sequence, wherein instances of a watermark are multiplied by a different complex constant before embedding the watermark to frames of said video sequence, said apparatus comprising: means for multiplying each of a predetermined number of frames of said video sequence with a predetermined sequence of complex constants to form a modified sequence of frames; means for averaging corresponding pixels of said modified sequence of frames to form a candidate image; and means for detecting said watermark in said candidate image.

21. Apparatus for removing a tiled watermark from an image, said apparatus comprising: means for determining a tile size of said tiled watermark; means for averaging corresponding pixels from a plurality of tiles to form a watermark tile; 725004 -29- O means for replicating and tiling said watermark tile together to form a tiled image; and means for subtracting a multiple of said tiled image from said image. 00 5 22. A computer readable medium, having a program recorded thereon, 00 where the program is configured to make a computer execute any one of the methods 1 to m 14.

23. A method substantially as described herein with reference to any one of the drawings. DATED this 20 th Day of June 2005 CANON KABUSHIKI KAISHA Patent Attorneys for the Applicant SPRUSON&FERGUSON 725004