US6678389B1 - Method and apparatus for embedding digital information in digital multimedia data - Google Patents

Method and apparatus for embedding digital information in digital multimedia data Download PDF

Info

Publication number
US6678389B1
US6678389B1 US09445142 US44514299A US6678389B1 US 6678389 B1 US6678389 B1 US 6678389B1 US 09445142 US09445142 US 09445142 US 44514299 A US44514299 A US 44514299A US 6678389 B1 US6678389 B1 US 6678389B1
Authority
US
Grant status
Grant
Patent type
Prior art keywords
digital
image
information
multimedia
method
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
US09445142
Inventor
Qibin Sun
Jiankang Wu
Huijie Robert Deng
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kent Ridge Digital Labs
Original Assignee
Kent Ridge Digital Labs
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Grant date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/0021Image watermarking
    • G06T1/005Robust watermarking, e.g. average attack or collusion attack resistant
    • G06T1/0064Geometric transfor invariant watermarking, e.g. affine transform invariant
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/0021Image watermarking
    • G06T1/0028Adaptive watermarking, e.g. Human Visual System [HVS]-based watermarking
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device
    • H04N1/32101Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
    • H04N1/32144Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
    • H04N1/32149Methods relating to embedding, encoding, decoding, detection or retrieval operations
    • H04N1/32154Transform domain methods
    • H04N1/32181Transform domain methods using Karhunen-Loeve transforms
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device
    • H04N1/32101Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
    • H04N1/32144Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
    • H04N1/32149Methods relating to embedding, encoding, decoding, detection or retrieval operations
    • H04N1/32154Transform domain methods
    • H04N1/32187Transform domain methods with selective or adaptive application of the additional information, e.g. in selected frequency coefficients
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device
    • H04N1/32101Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
    • H04N1/32144Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
    • H04N1/32149Methods relating to embedding, encoding, decoding, detection or retrieval operations
    • H04N1/32154Transform domain methods
    • H04N1/32187Transform domain methods with selective or adaptive application of the additional information, e.g. in selected frequency coefficients
    • H04N1/32192Transform domain methods with selective or adaptive application of the additional information, e.g. in selected frequency coefficients according to calculated or estimated visibility of the additional information in the image
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2389Multiplex stream processing, e.g. multiplex stream encrypting
    • H04N21/23892Multiplex stream processing, e.g. multiplex stream encrypting involving embedding information at multiplex stream level, e.g. embedding a watermark at packet level
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/835Generation of protective data, e.g. certificates
    • H04N21/8358Generation of protective data, e.g. certificates involving watermark
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2201/00General purpose image data processing
    • G06T2201/005Image watermarking
    • G06T2201/0052Embedding of the watermark in the frequency domain
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2201/00Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
    • H04N2201/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device
    • H04N2201/3201Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
    • H04N2201/3225Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document
    • H04N2201/3233Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document of authentication information, e.g. digital signature, watermark
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2201/00Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
    • H04N2201/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device
    • H04N2201/3201Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
    • H04N2201/3269Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of machine readable codes or marks, e.g. bar codes or glyphs
    • H04N2201/327Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of machine readable codes or marks, e.g. bar codes or glyphs which are undetectable to the naked eye, e.g. embedded codes

Abstract

A steganographic method is disclosed to embed an invisible watermark into an image. It can be used for copyright protection, content authentication or content annotation. The technique is mainly based on K-L transform. Firstly a block and cluster step 106 and cluster selection step 108 are performed to enhance the optimization of K-L transform 110 for a given image. Then a watermark is embedded 114 into the selected eigen-clusters. ECC (Error Correction Code) can be employed to reduce the embedded code error rate. The proposed method is characterized by robustness despite the degradation or modification on the watermarked content. Furthermore, the method can be extended to video, audio or other multimedia especially for multimedia databases in which the stored multimedia are categorized by their contents or classes.

Description

FIELD OF INVENTION

The present invention relates to a method and apparatus for embedding digital information in digital multimedia data. The invention also relates to a computer program product including a computer readable medium having recorded thereon a computer program for embedding digital information in digital multimedia data.

BACKGROUND OF INVENTION

Over recent decades there has been a rapid growth in computers and digital communications networks. With this growth, the illegal distribution, duplication and the manipulation of digital or digitised multimedia content have become more and more common place. Even though such activity may violate copyright laws, it is still widespread. There is thus a need to protect the rights of owners who own these original multimedia data.

Similarly, there has also been a proliferation of exchange of sensitive data over public communications networks. Recently, digital signatures have been used to verify and authenticate the integrity of such data and identity the sender of the data. However there exists a need for invisible digital signature for images, which adequately represents the content of the images and further can be used for verifying whether the signed images have been modified.

In addition, there has been an associated growth in multimedia databases. Usually the stored multimedia data needs to be annotated (captioning). If the annotation can be associated with the content permanently, it will be convenient and safe for some applications such as medical image database.

Digital watermarking techniques can be applied to the above-mentioned applications, namely, for copyright protection, multimedia content authentication and multimedia content annotation. Although visible watermarks have been previously used, there is much promise for invisible digital watermarking techniques due to its unobtrusive nature. Regardless of particular applications, the following are the basic and common requirements for invisible watermarks. The watermark should be unobtrusive and not annoying to bona-fide users. The watermark scheme should be robust to some reasonable distortions during storage and transmission (such as compression, noise addition, format conversion, and bit errors). The watermark scheme should also be robust to some artifacts caused by signal processing operations (such as noise reduction, filtering, sharpening, and colour & intensity variations) It should also be secure against typical attacks such as colluding pirates who combine multiple versions of the same multimedia content that are stamped with different watermarks. Erasing the watermark or replacing the watermark by another watermark should be technically difficult.

Although these requirements are essential to practical applications, so far no one method or system is capable of fully satisfying the above conditions due to its so many unsolved problems and challenges.

ASPECTS OF INVENTION

It is an object of the present invention to ameliorate one or more disadvantages of the prior art.

According to one aspect of the invention, there is provided a method of embedding digital information in digital multimedia data, wherein said method includes the following steps: generating one or more transforms from the digital multimedia data or a portion of the digital multimedia data, wherein said transforms are dependent on the digital multimedia data or the portion of the digital multimedia data; generating transformed digital multimedia data by applying the one or more transforms to the digital multimedia data or the portion of the digital multimedia data; embedding the digital information in the transformed digital multimedia data; and generating the digital multimedia data embedded with the digital information by applying one or more inverse transforms of the one or more transforms to the embedded transformed digital multimedia data.

According to still another aspect of the invention, there is provided a method of embedding digital information in digital multimedia data, wherein said method includes the following steps: generating one or more transforms from the digital multimedia data or a portion of the digital multimedia data, wherein said transforms are dependent on the digital multimedia data or the portion of the digital multimedia data; generating transformed digital multimedia data by applying the one or more transforms to the digital multimedia data or the portion of the digital multimedia data; embedding the digital information in the one or more transforms or their inverses; and generating the digital multimedia data embedded with the digital information by applying one or more embedded inverse transforms to the transformed digital multimedia data.

According to still another aspect of the invention, there is provided a method of embedding digital information in a digital image, wherein said method includes the following steps: partitioning the image into blocks of pixels; clustering the partitioned blocks into classes according to a homogeneity criteria; selecting one or more of the clusters in accordance with HVS analysis; generating an associated Karhunen-Loeve (K-L) transform for each selected cluster, wherein the image blocks of each selected cluster are used as a vector population for the purposes of generating the associated K-L transform; applying the associated K-L transform to the image blocks of the selected cluster to produce corresponding transformed image blocks; embedding, for each selected cluster, the digital information in the associated K-L transform; and applying, for each selected cluster, an inverse transform of the embedded associated K-L transform to the transformed image blocks of the selected cluster.

According to still another aspect of the invention, there is provided a method of embedding digital information in a digital image, wherein said method includes the following steps: partitioning the image into blocks of pixels; clustering the partitioned blocks into classes according to a homogeneity criteria; selecting one or more of the clusters in accordance with HVS analysis; generating an associated Karhunen-Loeve (K-L) transform for each selected cluster, wherein the image blocks of each selected cluster are used as a vector population for the purposes of generating the associated K-L transform; applying the associated K-L transform to the image blocks of the selected cluster to produce corresponding transformed image blocks; embedding, for each selected cluster, the digital information in the selected transformed image blocks; applying, for each selected cluster, an inverse transform of the associated K-L transform to the embedded transform image blocks of the selected cluster.

According to still another aspect of the invention, there is provided a method of embedding digital information in a digital image, wherein said method includes the following steps: partitioning the image into blocks of pixels; clustering the partitioned blocks into classes according to a homogeneity criteria; selecting one or more of the clusters in accordance with HVS analysis; generating an associated Karhunen-Loeve (K-L) transform for each selected cluster, wherein the image blocks of each selected cluster are used as a vector population for the purposes of generating the associated K-L transform; applying the associated K-L transform to the image blocks of the selected cluster to produce corresponding transformed image blocks; selecting, for each selected cluster, either the associated K-L transform or its corresponding transformed image blocks for embedding; embedding, for each selected cluster, the digital information in the selected K-L transform or the selected transformed image blocks; applying, for those selected clusters in which the digital information has been embedded in the associated K-L transform, an inverse transform of the embedded associated K-L transform to the transformed image blocks of the selected cluster; and applying, for those selected clusters in which the digital information has been embedded in the transformed image blocks, an inverse transform of the associated K-L transform to the embedded transform image blocks of the selected cluster.

According to still another aspect of the invention, there is provided a method of embedding digital information in a plurality of images, wherein the images are stored and classified into one or more classes and said method includes the following steps: generating one or more transforms from the images of said respective one or more classes, wherein said transforms are dependent on the images; generating transformed images by applying the one or more transforms to the images of said respective one or more classes; embedding the digital information in the transformed images; and generating the images data embedded with the digital information by applying one or more inverse transforms of the one or more transforms to the embedded transformed images.

According to still another aspect of the invention, there is provided a method of embedding digital information in a plurality of images, wherein the images are stored and classified into one or more classes and said method includes the following steps: generating one or more transforms from the images of said respective one or more classes, wherein said transforms are dependent on the images; generating transformed images by applying the one or more transforms to the images of said respective one or more classes; embedding the digital information in the one or more transforms or their inverses; and generating the images embedded with the digital information by applying one or more embedded inverse transforms to the transformed images.

According to still another aspect of the invention, there is provided a method of embedding digital information into video images including a plurality of frames, wherein said method includes the following steps: extracting one or more key frames from the video images; generating one or more transforms from the key frames or a part thereof, wherein said transforms are dependent on the key frames or the part thereof; generating transformed key frames or part thereof by applying the one or more transforms to the key frames or the part thereof; embedding the digital information in the transformed key frames or part thereof; and generating the key frames or part thereof embedded with the digital information by applying one or more inverse transforms of the one or more transforms to the embedded key frames or part thereof.

According to still another aspect of the invention, there is provided a method of embedding digital information into video images including a plurality of frames, wherein said method includes the following steps: extracting one or more key frames from the video images; generating one or more transforms from the key frames or a part thereof, wherein said transforms are dependent on the key frames or the part thereof; generating transformed key frames or part thereof by applying the one or more transforms to the key frames or the part thereof; embedding the digital information in the one or more transforms or their inverses; and generating the key frames or part thereof embedded with the digital information by applying one or more embedded inverse transforms to the transformed key frames or part thereof.

According to still another aspect of the invention, there is provided a method of registering an image, wherein the method includes the following steps: extracting key points from the image and their invariant attributes; extracting other global information from the image; aligning the extracted key points and their invariant attributes with the global information; and encrypting the aligned information to provide an image signature.

According to still another aspect of the invention, there is provided a method of modifying a first image in accordance with an image signature, wherein the image signature includes encrypted key points of a second image and their invariant attributes and other aligned global information of the second image and the method includes the steps of: decrypting the image signature to obtain the key points and their invariant attributes and other global information of the second image; extracting key points and their invariant attributes from the first image; extracting other global information from the first image; matching the key points and their invariant attributes and other global information of the first and second images; and normalising the first image based on said matched key points and their invariant attributes and other global information.

According to still another aspect of the invention, there is provided a method of extracting digital information from first digital multimedia data having embedded therein said digital information, wherein said method includes the following steps: deriving one or more embedded transforms from a reference file, wherein said embedded transforms are dependent on second digital multimedia data or a portion thereof; deriving transformed digital multimedia data from the reference file, wherein said transformed digital multimedia data is dependent on the second digital multimedia data or a portion thereof; generating transformed digital multimedia data by applying one or more of the derived embedded transforms to the first digital multimedia data or a portion thereof; and correlating the generated transformed digital multimedia with the derived transformed digital multimedia to check whether the digital information embedded in the first multimedia data is the same as that derived from the reference file.

According to still another aspect of the invention, there is provided a method of extracting digital information from first digital multimedia data having embedded therein said digital information, wherein said method includes the following steps: deriving one or more transforms from a reference file, wherein said transforms are dependent on second digital multimedia data or a portion thereof; deriving transformed digital multimedia data from the reference file, wherein said transformed digital multimedia data is dependent on the second digital multimedia data or a portion thereof; generating embedded transformed digital multimedia data by applying one or more of the derived embedded transforms to the first digital multimedia data or a portion thereof; and correlating the generated embedded transformed digital multimedia with the derived transformed digital multimedia so as to extract the digital information.

According to still another aspect of the invention, there is provided a method of embedding digital information in a digital image, wherein said method includes the following steps: partitioning the image into blocks of pixels; clustering the partitioned blocks into classes according to a homogeneity criteria; selecting one or more of the clusters in accordance with HVS analysis; averaging, for each selected cluster, the blocks of pixels; embedding, for each selected cluster, the digital information in the averaged blocks; and replacing, for each selected cluster, the blocks with the embedded averaged block for that cluster.

According to still another aspect of the invention, there is provided a method of extracting digital information from an image having embedded therein said digital information, wherein said method includes; partitioning the image into blocks of pixels; clustering the partitioned blocks into classes according to a homogeneity criteria; selecting one or more of the clusters in accordance with information obtained from a reference file; retrieving, for each selected cluster, an average block of pixels stored in the reference file; averaging, for each selected cluster, the blocks of pixels of the image; and correlating, for each selected pixel, the averaged blocks of the image with the retrieved blocks to determine whether a watermark exists.

According to still another aspect of the invention, there is provided an apparatus for implementing any one of the aforementioned methods.

According to still another aspect of the invention there is provided a computer program product including a computer readable medium having recorded thereon a computer program for implementing any one of the methods described above.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are described with reference to the drawings, in which:

FIG. 1 is a flow diagram of a method of embedding a watermark in a digital image in accordance with a preferred embodiment;

FIG. 2 shows a copyright protection system for protecting an owner's watermarked image in accordance with a preferred embodiment;

FIG. 3 is a flow diagram of a method of generating a reference file that can be used for both image registration and for image authentication in accordance with a preferred embodiment;

FIG. 4 is a flow diagram of a method of extracting a watermark from an image in accordance with a preferred embodiment.

FIG. 5 shows a flow diagram of a method of registering and recovering an image for use in the extraction method shown in FIG. 4;

FIG. 6 is a flow diagram of a method of embedding a watermark in a digital image according to a second preferred embodiment;

FIG. 7 there is shown a watermark extracting method for extracting a watermark embedded in accordance with the method of FIG. 6; and

FIG. 8 is a block diagram of a general-purpose computer for implementing the preferred methods of FIGS. 1 to 7.

DETAILED DESCRIPTION

The principles of the preferred method described herein have general applicability to a method of embedding a digital message within digital multimedia content. However, for ease of explanation, the steps of the preferred method are described with reference to a method of embedding a watermark within a still digital image. Though, it is not intended that the present invention be limited to the latter method. For example, the invention may have application to embedding an identification message within other multimedia content such as video, audio, print or film. The embedded identification message can also be used for copyright protection, content authentication and content annotation without departing from the scope and spirit of the invention.

First Preferred Embodiment(s)

FIG. 1 is a flow diagram of a method of embedding a watermark in a digital image according to a preferred embodiment. The method commences at step 100 and any necessary initialisation processes are undertaken. In the next step 102, an original digital image is input by a user for processing. The original digital image is a pixel-map representation which includes a numerical representation of the particular colour for each pixel location in a rectangular array. Any numerical representation of colour can be used and can be expressed as a sequence of one or more numbers. Also, other information may be input at this stage, such as header information. Such header information can include the size of the image, resolution etc.

The input original image is then preprocessed in step 104. The preprocessing may take the form of filtering, noise estimation and so on. After the preprocessing step 104, the method continues with a block and clustering step 106.

In the block and clustering step 106, the selected original image is divided into N non-overlapping blocks of size nxn. Preferably, these blocks are either 4×4 pixels or 8×8 pixels in size. In the event the original image is not a multiple of the image block size, a portion of the original image which is a multiple can be used by the method and the remaining portion disregarded. The block and cluster step 106 allocates these image blocks to clusters (sometimes known as classes) according to the details of the image block content and a homogeneity criteria. Thus the image blocks within each cluster will have similar amount of and type of content. Preferably, the clusters are classified according to their complexity or texture content. This classification is achieved in the following manner. Firstly, feature measures are extracted from each image block which are representative of the complexity of the content. Some preferred feature measures are grey scale, colour variation or texture measures. Afterwards, the clustering process classifies the image blocks according to their feature measures. The clustering process preferably utilises the k-means clustering method. Alternatively, the preferred method can use region growing, or split-merge clustering processes. In this way, the block and clustering step allocates the image blocks to a plurality of clusters. After the block and clustering step 106 is completed, the processing continues at step 108.

In step 108, a cluster selection process is undertaken. In this selection process, an analysis based on the Human Vision System (HVS) is applied to all clusters to determine whether and by how much the watermark can be embedded. Preferably, those clusters which satisfy the HVS requirements are selected for watermark embedding. The perceptual redundancies inherent in a still image are basically due to inconsistency in the sensitivity of the HVS to stimuli of varying levels of contrast and luminance changes in the spatial domain. Thus, by utilising the HVS model in the preferred method it is possible to embed more information while not overall affecting image quality. Or in other words, the embedded watermark information is invisible to human eyes. In the preferred method, the HVS model is used to select the clusters and the embedding strength. For example, the embedding strength is determined as a proportion of the contrast of the image content of the cluster. Preferably, the HVS model used is that described in the paper “A perceptually tuned subband image coder based on the measure of Just-Noticeable-Distortion profile” by C. H. Chou and Y. C. Li, IEEE Trans. On circuits and Systems for Video Technology, Vol. 5, No. 6, pp467-476, 1995 (herein incorporated by reference). In another embodiment, if there are a number of clusters which satisfy these HVS requirements, then one or more clusters can be selected by the user for embedding. Alternatively, different clusters can be embedded with different watermarks with the same seeds to increase the security and robustness of the watermarking. After the clusters have selected the preferred method proceeds to the next step 110.

In the next step 110, one or more Karhunen-Loeve transforms (KLT) are applied to the image blocks of one or more selected clusters. These KLT transforms are each derived from and applied to a corresponding selected cluster. It is noted that, up until today, the statistically optimum, signal-dependent transform, KLT is still believed to be impractical for image processing. This is due to the time-consuming computations needed for determining the data covariance matrix (related to eigen-vectors). However in the preferred method, this will not be a crucial problem any more. Firstly, the KLT adopted in the preferred method is the discrete KLT (also called the Principle Components Analysis) where there have been some fast implementations developed recently. Secondly, unlike image compression or image coding, for watermarking there is no need to transform all blocks or segmented regions in the images. The fast clustering processing associated with the cluster selection step 106 allows the reduction of the computational burden to an acceptable and practical level. Namely, the KLT transforms need not be applied over all the image, but only to the selected clusters.

In the step 110, a KLT transform is separately derived and applied for each cluster. Firstly, each non-overlapping block in the i th cluster (segment), i=1, Λ, N, is mapped to a d-dimensional vector Xij, where d=3n2, by sequentially scanning each location in the block and taking its corresponding tricomponent colour signals. (Note: for monochrome image, d=n2). Here N denotes the total number of clusters to be transformed. Thus each block in the same cluster can be implicitly modelled as separate multivariate Gaussian Markov random fields. The sets of vectors Xij created in this way are assumed to have jointly zero-mean Gaussion distributions. The d×d covariance matrix Ri of each such stationary zero-mean Gaussion source, i=1, Λ, N, is estimated as R i = 1 M i j = 1 M i X i j X i j T ( 1 )

Figure US06678389-20040113-M00001

Where Mi corresponds to the number of blocks in the ith cluster. The preferred method then calculates Eigenvalues of the covariance matrix Ri in accordance with

R i e ikik e ik  (2)

Where eik is the kth eigenvector (k=1, K, p)of the ith cluster and λik is its corresponding eigenvalue. The eigenvectors calculated from (2) are orthonormal, ie.,

e ik T e il=0  (3)

e ik T e ik=1  (4)

Subsequently, the eigenvectors are arranged such that λi1≧λi2≧Λ≧λid and the first Li eigenvectors, where Li≦d, can be used to form the transform matrix Ai=[ei1 ei2 Λ eiL i ]T, where d is the dimension of vector Xij. Preferably however, all the eigenvectors are used to form the transform matrix, viz Li=d. The reason for the latter is that the preferred method is primarily concerned with watermarking and not compression. Each vector Xij from the ith image cluster can then be processed by the linear transform:

Y ij =A i X ij j=1,K,M i  (5)

After step 110, the preferred method proceeds to the next step 112. In step 112, a user decides whether the watermark is to be embedded in either the eigenvector domain Ai or the coefficients of vectors Yij. Preferably, the user selects either the eigenvector domain Ai or the coefficients of vectors Yij for the entire image, viz for all clusters. However in another embodiment, the user is able to select the eigenvector domain Ai for one cluster and the coefficients of vectors Yij for another cluster. This will result in benefits in the security of the watermarking method. In a still further embodiment, the method automatically and exclusively embeds the watermark in the eigenvector domain Ai. Namely, the selection is automatic and no input from a user is required. Similarly, the method can automatically and exclusively embed the watermark in the coefficients of vectors Yij. In still further embodiment, the actual selection process is dependent on the requirements of the particular application. After step 112, the method continues processing at step 114.

In step 114, the method embeds the watermark in the selected domain, namely either the eigenvector domain Ai or the coefficients of vectors Yij. In the event the preferred method embeds the noise-like watermark into the coefficient domain, the following embedding process is performed. A property of the coefficients of the vectors Yij are that they are completely uncorrelated, as can be seen from eqn (6). E [ Y i j Y i j T ] = [ λ i1 0 Λ 0 0 λ i 2 Λ 0 M M M M 0 0 Λ λ iL 1 ] ( 6 )

Figure US06678389-20040113-M00002

where λi1≧λi2≧Λ≧λid and are the eigenvalues of the covariance matrix Ri. The preferred method embeds the watermark in the first Li coefficients of each vector Yij where

L i=arg max{λik k ik ≦D i  (7)

The parameter in (7) represents the allowed distortion within the cluster. Its value will be defined for each cluster separately and decided by the results of HVS analysis. Thus during this embedding process the vectors Yij are changed to {tilde over (Y)}ij.

In the event the preferred method embeds the noise-like watermark into the eigenvector domain, the watermark will be embedded into the first Li eigenvectors of Ai in accordance with eqn (7). Thus during this embedding process the transform Ai is changed to Ãi. As the eigenvectors of the transform matrix are orthogonal to each other, the generated watermark Wi should be orthogonalized with respect to the eigenvectors before embedding.

In another embodiment, the transform Ai may be first inverted and then embedded with the watermark.

All the parameters of the selected domain are embedded, by modulating the parameters by a noise-like watermark. This modulation is achieved in the following way

{tilde over (P)}=ƒ(P,PN(W))  (8)

Where ƒ is a function of watermarking modulation, {tilde over (P)} and P are parameters after and before watermarking, and PN(W) is generated noise-like watermark.

The content of image itself, the owner's private key of an encryption algorithm, embedding time or other information related to the owner or the buyer can be used to form the seeds of the embedded watermarks. Furthermore error correction encoding can be used to increase the robustness of the preferred method. The preferred method can use any one of a number of typical methods to embed a watermark (modulation). A simple example of such a embedding method is as follows

{tilde over (P)} i =P ii P i W i  (9)

After the completion of the watermark embedding step 114, the process continues at step 116. In step 116, the respective inverse KLT transforms are separately applied to their respective clusters. Where the watermark has been embedded in the eigenvector domain, the “embedded” inverse KLT transform is applied to the vectors. Thus the watermarked image blocks are transformed by

{tilde over (X)} ij i −1 Y ij j=1,K,M i  (10)

Where the watermark has been embedded in the coefficients of the vectors, the inverse KLT is applied to the embedded vectors. Thus the watermark image blocks are transformed by

{tilde over (X)} ij =A i −1 {tilde over (Y)} ij j=1,K,M i  (11)

These resultant image blocks form the basis of the watermarked image. To retrieve the watermark from the watermark image or to authenticate the watermark image a reference file is needed. This reference file may be the original image itself or data based on the original image. Such data can include the size of the image, the resolution, the clusters selected and their associated KLT transform, modulation technique and owner information. Consequently, after the original image is preprocessed in step 104, the preferred method stores 122 the original image or the derived data as the reference file. In step 122, the reference file is normally held in a secure location and maybe encrypted.

After completion of inverse transform KLT 116, the preferred method continues at step 118, where the watermarked image is output. The method terminates at step 118.

The use of the KLT transform as the watermark embedding domain has the following advantages. Firstly, the KLT transform is a kind of signal-dependant transform. That means the watermark to be embedded will follow any changes of image content. Even if the watermark is the same for all different images, the real form of the embedded watermark is different. Thus in this sense, the embedded digital watermark is dependent upon the image content and can be used to authenticate an image. Furthermore, whilst the type of transform used to generate the image may be known, the original watermark cannot be defeated by inverse transforming the watermarked image data to derive the original data, since this involves the use of a different inverse transform. That is, because the watermarked data is different from the original data. Likewise, attempts to fraudulently claim ownership on image data already watermarked by the preferred method will be defeated because a different optimal transform dependent upon the watermarked image data will be involved. Still further, the KLT transform presents an extra choice to select the embedding locations either in the eigen-vector domain (basis function of KLT transform) or in the coefficient domain of KLT transform. This means that watermarking based on KLT can be done in a much secure and flexible way than other methods. In addition, the KLT transform can provide a more effective energy-compact ability more than any other transform such as DCT, DST, FFT, etc. As there is significant redundant information contained in an image, applying a KLT transform to an image means that there is more space for embedding the watermark. Also, unlike most of common watermarking schemes which only embed the watermark into the luminance channel, the KLT transform can directly & jointly process colour images. It presents a flexible way to embed a watermark directly into all colour channels. In addition, the embedded messages will persist through image modifications such as scaling, rotating, printing-rescanning, colour & intensity variation.

Turning now to FIG. 2 there is shown a copyright protection system for protecting an owner's watermarked image in accordance with a preferred embodiment. Firstly, the owner of the image 200 sends his request 201 to a Copyright Management Centre (CMC) 202 for registration. The CMC 202 provides the owner of the image 200 with a watermark embedding tool 203 to embed his private watermark in his images 205. During this procedure, a reference file 206 is generated and sent to CMC 202. The reference file 206 is based on a content analysis 207 of the owner's image 205. After watermarking 204, the CMC 202 will activate an Internet Spider 208 to protect the owner's watermarked images 210. Of course, there are so many variant applications related to copyright protection, content authentication and content annotation. Preferably, the watermarking engine is that described with reference to FIG. 1, namely the watermarking engine is based on KLT transforms. This watermarking engine can be constructed as a kernel and integrated into various copyright protection systems either for public use or for private use.

Turning now to FIG. 3, there is shown a flow diagram of a method of generating a reference file that can be used for both image registration and for image authentication. This method of generating a reference file may be used in the method of watermarking an image as shown in FIG. 1. The method commences at step 300 and continues at steps 302 and 304 where an image is input and preprocessed. In the next step 306, the method summarises and extracts some key points of the preprocessed image. These key points must be invariant to some typical modifications such as affine distortions, colour or intensity changes etc. Alternatively, some variations of the key points can be allowed by adopting a global affine matching algorithm in the registration procedure. Then the invariant attributes are calculated 308. These attributes are associated with other relative information such as maximum and minimum colour values, clusters and the number of clusters, and KLT transform matrices etc. Finally this associated information is encrypted by some cryptography algorithms 310 to form the reference file 312. The method terminates at step 314. In this way, the watermarked image is registered with reference to the properties of the original image, which can be used in the method of extracting the watermark from the watermarked image. Alternatively, the reference file can be used for proving whether and by how much the watermarked image has been tampered with.

Turning now to FIG. 4, there is shown of a watermark extracting method in accordance with a preferred embodiment. The method commences at step 400, where any necessary parameters are initialised. In the next step 402, a watermarked image is input by a user for watermark extraction. Afterwards in step 404, the user then inputs a reference file 403 of the original image and/or associated parameters of the type as previously described. The watermarked image is then normalised 406 based on the parameters retrieved from the reference file 403. This normalised image is partitioned into blocks 106 and then clustered 108, in a similar manner as the watermarking procedure as described with reference to FIG. 1. In step 108, any parameters necessary for the selection of the clusters are retrieved from the reference file 403. After step 108, the method continues with step 408 where the relevant domain is selected from data from the reference file. In the case where the domain selected is the coefficient domain, the next step 410 applies the transform Ai to selected image blocks {tilde over (X)}ij of the test image to obtain the coefficients of vectors {tilde over (Y)}ij. The images blocks {tilde over (X)}ij of the test image are decided by the reference file or owner's private key. The transform Ai is also obtained from the reference file. Finally, a correlation 412 is undertaken between the last mentioned coefficients of vectors {tilde over (Y)}ij and the coefficients of vectors Yij obtained from the reference file and the watermark is extracted. The extracted watermark is then compared with the watermark obtained from the reference file. In the case where the domain selected is the transform domain, the step 410 applies the embedded transform Ãi to selected image blocks {tilde over (X)}ij of the test image to obtain the coefficients Yij. A correlation 412 is then undertaken between the last mentioned coefficients of vectors Yij and the coefficients of vectors Yij obtained from the reference file so as to check whether there exists a watermark. If the coefficients are the same then the watermark is the same as was originally embedded. The extracted watermark is then output in step 418 and the method then terminates.

Turning now to FIG. 5 there is shown a flow diagram of a method of registering and recovering an image for use in the extraction method shown in FIG. 4. As a watermarked image may undergo some modifications during it's lifetime, an operation like image registration is used to estimate the modification parameters of the watermarked image. The method commences at step 500 and continues at step 502 where a watermarked image is input. The watermarked image is then preprocessed 504 and summarised 506 to obtain some key features or points. Once these key points are optimally selected, their invariant feature attributes are computed. Meanwhile relative parameters are obtained from the reference file 508 by an information recovering procedure such as decryption. Such a reference file may be obtained by a method as described with reference to FIG. 3. In the next step, a rule-based global optimal search 510 is undertaken between the obtained invariant feature attributes of the watermarked image and reference file. For example, if some feature points within a neighbour are missed, then a chopping operation may be applied onto the watermarked image. The matching is mainly based on their invariant moments of the detected feature points. Based on the estimated parameters, the watermarked image is the normalised 512 to recover an image similar to the original image. The normalisation method can be that which is described in J. Flusser, “An adaptive method for image registration”, Pattern Recognition, Vol 25, No. 1, pp45-54, 1992. The method normalises the tested image by extracting the key content from the tested image, measuring the modified parameters of the tested image with reference to the reference file and then normalising the tested image with respect to these modified parameters.

The present invention is suitable for use in a number of different and widely varying applications, such as the copyright protection system as previously described. This system can provide an approach to track subsequent use of digital images and then present evidence for copyright protection, or alternatively provide a display warning users.

The present invention also can be used for content authentication. In this application, after summarising the image, the extracted key description about the image can be encrypted by the owner's private key. The buyer can then verify the authenticity of the image by using a public key. Based on the rule-based image registration method, the results of the authentication include the information about not only whether the given image is altered or not, how much it is altered, but also which kind of modification the said image undergoes. The application may also inform the owner of the images what kinds of attacks others have used on his or hers images. A digital signature may also be embedded into the image for verifying the authenticity of the content provider.

In addition, the present invention also can be used for content annotation. It is believed that the annotation done by watermarking will be helpful for multimedia database based on the current technologies.

The present invention is also particularly suitable for watermarking the images that has been stored in the multimedia database no matter what kind of applications it is used for (copyright protection, authentication and annotation). As in this case, the image will belong to some classes or categories. For example, a face database in the police station, thus a optimal basis matrix of KLT can be obtained based on the whole images from a class. The images are then embedded in a similar manner as described with reference to FIG. 1. In this way, it is very difficult for the users or the attackers to compute out the exact basis matrix. Without the exact basis matrix, it is unlikely to decode or to remove the embedded watermark.

Furthermore, the present invention can be used in relation to other media such as video and audio. For instance, in a video file, there is a strong relationship between consecutive frames. A technique called key frame extraction has been shown to be an effective approach to video indexing or summarising. By applying the KLT to hide the information directly into video file, the optimal basis matrix of the KLT can be computed out not only based the clustered blocks within each frame but also based on frames represented by the same key frame. Based on KLT, the data hiding can be implemented in a more secure and flexible way.

Second Preferred Embodiment(s)

FIG. 6 is a flow diagram of a method of embedding a watermark in a digital image according to a preferred embodiment. The method commences at step 100 and any necessary initialisation processes are undertaken. In the next step 102, an original digital image is input by a user for processing. The original digital image is a pixel-map representation which includes a numerical representation of the particular colour for each pixel location in a rectangular array. Any numerical representation of colour can be used and can be expressed as a sequence of one or more numbers. Also, other information may be input at this stage, such as header information. Such header information can include the size of the image, resolution etc. Steps 102 to 108 of this method are the same as those described with reference to steps 102 to 108 of FIG. 1 and will not be described further.

In step 602 of the method, the image blocks of each cluster i are averaged. These averaged blocks are each then embedded 604 with a watermark. An average (mean) block for Mi image blocks Bij (j=1,2, . . . M) that belong to cluster i is determined by B _ i = 1 M i j = 1 M i B i j ( 12 )

Figure US06678389-20040113-M00003

and a difference block is obtained by

X ij =B ij −{overscore (B)} i  (13)

The average block can be used to embed the watermark if it satisfies the HVS condition. If there are more than one suitable clusters then the user can select the cluster used for embedding. The averaging and embedding process is described in more detail below with reference to the following alternative embodiments.

In the first embodiment, the embedding is achieved by an orthogonalization process. Firstly, the average mean block {overscore (B)}i is aligned as a (n×n)×1 vector. This averaged mean block {overscore (B)}i is saved to a reference file for subsequent watermark extraction. The initial watermark Wi then is orthogonalized with respect to {overscore (B)}i to obtain the final watermark Wƒ in accordance with the following:

W ƒ =W i−(W i ·{circumflex over ({overscore (B)})} i){circumflex over ({overscore (B)})} i  (14)

where {circumflex over ({overscore (B)})}i is the unit vector along {overscore (B)}i.

Next, a smaller scaled version of W is added back to {overscore (B)}i to obtain a watermarked average block {tilde over ({overscore (B)})}i in accordance with the following:

{tilde over ({overscore (B)})} i ={overscore (B)} i +cW ƒ, where c is a small constant to control the strength of the watermark.  (15)

Finally in step 606, all the image blocks of the cluster are replaced with the watermarked average block {tilde over ({overscore (B)})}i to reconstruct a new image (watermarked image). The cluster criteria is chosen (108) so that the distortion caused by replacing all the image blocks within the cluster with the watermarked average block would be invisible to the human eye.

In the second embodiment, the embedding is achieved by a spread spectrum process. In this embodiment, the average mean block {overscore (B)}i is first aligned as a (n×n)×1 vector. The initial watermark Wi is shaped to the same n×n length and with gaussian distribution (zero mean value). A small scaled version of Wi is added back to {overscore (B)}i to obtain a watermarked average block {tilde over ({overscore (B)})}i in accordance with the following:

{tilde over ({overscore (B)})} i ={overscore (B)} i·(1+cW ƒ), where c is a small constant to control the strength of the watermark.  (16)

Finally in step 606, all the image blocks of the cluster are replaced with the watermarked average block {tilde over ({overscore (B)})}i to reconstruct a new image (watermarked image). The cluster criteria is chosen (108) so that the distortion caused by replacing all the image blocks within the cluster with the watermarked average block would be invisible to the human eye.

After the completion of step 606, the watermarked image is outpoured and the process terminates at step 610.

Turning now to FIG. 7, there is shown a watermark extracting method for extracting a watermark embedded in accordance with the method of FIG. 6. The method commences at step 700, where any necessary parameters are initialised. The test image is then input 102, preprocessed 104 and block and clustered 106 in a similar manner as that of the embedding method of FIG. 6. After completion of step 106, one or more clusters are selected for processing. The clusters selected are determined from information stored in the reference file 714. These clusters correspond to those selected in the embedding method. In the next step 706, the average mean block {overscore (B)}i is obtained from the reference file. In this step, the watermark Wi is also generated by the owner's key. The average mean block {overscore (B)}i (from the reference file) is then aligned as a (n×n)×1 vector. The watermark Wi is then orthogonalised with respect to {overscore (B)}i to obtain the original watermark in accordance with the following:

W o =W i−(W i ·{circumflex over ({overscore (B)})} i){circumflex over ({overscore (B)})} i where

{circumflex over ({overscore (B)})} i is the unit vector along {overscore (B)}i.  (17)

Similarly in the next step 708, a watermarked mean block {overscore (J)}i is then obtained from the test image. In the event, the watermark has been embedded by the orthogonal process, then watermarked mean block {overscore (J)}i is orthogonalised with respect to {overscore (B)}i to obtain another vector Xi. Finally in step 710, a correlation is undertaken between Wi and Xi to check whether the watermark exists. In the event, the watermark has been embedded by the spread spectrum process, then the DC component of {overscore (J)}i is removed. In this case a correlation is undertaken between Wi and {overscore (J)}i to check whether the watermark exists. After the correlation is completed the method terminates at step 712.

In a still further embodiment, the method of embedding and extraction of the second preferred embodiment may include a method of registering and recovering an image as shown in FIGS. 3 and 5. In this way, modifications that may subsequently occur to the embedded image are compensated by normalisation. In a still further embodiment, the first and second preferred embodiments may form part of the same watermarking engine. In the latter embodiment, a user's key is used to select the watermarking method to be used on a particular image. The user's key is an encrypted random key, which is divided into three parts. The first part is used to select the watermarking embedding space (mean block, the transform or vector coefficients), the second is used to select the watermark extraction method (eg. de-orthogonalization or correlation) and the third part is used as the seed to generate the initial watermark Wi. In addition, the user's key can be used to select the cluster from the one or more clusters that meet the HVS criteria.

Preferred Embodiment of Apparatus(s)

The methods of FIGS. 1, 3, 4, 5, 6 and 7 is preferably practiced using a conventional general-purpose computer, such as the one shown in FIG. 8 wherein the processes of FIGS. 1, 3, 4, 5, 6 and 7 may be implemented as software executing on the computer. In particular, the steps of methods of embedding and extracting a watermark are effected by instructions in the software that are carried out by the computer. The software may be divided into two separate parts; one part for carrying out the extraction and embedding methods; and another part to manage the user interface between the latter and the user. The software may be stored in a computer readable medium, including the storage devices described below, for example. The software is loaded into the computer from the computer readable medium, and then executed by the computer. A computer readable medium having such software or computer program recorded on it is a computer program product. The use of the computer program product in the computer preferably effects an advantageous apparatus for extracting and embedding a watermark in accordance with the embodiments of the invention.

The computer system 800 consists of the computer 802, a video display 816, and input devices 818, 820. In addition, the computer system 800 can have any of a number of other output devices including line printers, laser printers, plotters, and other reproduction devices connected to the computer 802. The computer system 800 can be connected to one or more other computers via a communication interface 808 b using an appropriate communication channel 830 such as a modem communications path, a computer network, or the like. The computer network may include a local area network (LAN), a wide area network (WAN), an Intranet, and/or the Internet

The computer 802 itself consists of a central processing unit(s) (simply referred to as a processor hereinafter) 804, a memory 806 which may include random access memory (RAM) and read-only memory (ROM), input/output (IO) interfaces 808 a, 808 b & 808 c, a video interface 810, and one or more storage devices generally represented by a block 812 in FIG. 8. The storage device(s) 812 can consist of one or more of the following: a floppy disc, a hard disc drive, a magneto-optical disc drive, CD-ROM, magnetic tape or any other of a number of non-volatile storage devices well known to those skilled in the art. Each of the components 804 to 812 is typically connected to one or more of the other devices via a bus 814 that in turn can consist of data, address, and control buses.

The video interface 810 is connected to the video display 816 and provides video signals from the computer 802 for display on the video display 816. User input to operate the computer 802 can be provided by one or more input devices 808 b. For example, an operator can use the keyboard 818 and/or a pointing device such as the mouse 820 to provide input to the computer 802.

The system 800 is simply provided for illustrative purposes and other configurations can be employed without departing from the scope and spirit of the invention. Exemplary computers on which the embodiment can be practiced include IBM-PC/ATs or compatibles, one of the Macintosh (TM) family of PCs, Sun Sparcstation (TM), or the like. The foregoing are merely exemplary of the types of computers with which the embodiments of the invention may be practiced. Typically, the processes of the embodiments, described hereinafter, are resident as software or a program recorded on a hard disk drive (generally depicted as block 812 in FIG. 8) as the computer readable medium, and read and controlled using the processor 804. Intermediate storage of the program and pixel data and any data fetched from the network may be accomplished using the semiconductor memory 806, possibly in concert with the hard disk drive 812.

In some instances, the program may be supplied to the user encoded on a CD-ROM or a floppy disk (both generally depicted by block 812 ), or alternatively could be read by the user from the network via a modem device connected to the computer, for example. Still further, the software can also be loaded into the computer system 800 from other computer readable medium including magnetic tape, a ROM or integrated circuit, a magneto-optical disk, a radio or infra-red transmission channel between the computer and another device, a computer readable card such as a PCMCIA card, and the Internet and Intranets including email transmissions and information recorded on websites and the like. The foregoing are merely exemplary of relevant computer readable mediums. Other computer readable mediums may be practiced without departing from the scope and spirit of the invention.

The foregoing only describes a small number of embodiments of the present invention, however, modifications and/or changes can be made thereto by a person skilled in the art without departing from the scope and spirit of the invention. For example, signal dependent transforms other than KLT transforms may used in the preferred methods. The present embodiments are, therefore, to be considered in all respects to be illustrative and not restrictive.

Claims (72)

What is claimed is:
1. A method of embedding digital information in digital multimedia data, wherein said method includes the following steps:
generating one or more transforms from the digital multimedia data or a portion of the digital multimedia data, wherein said transforms are dependent on the digital multimedia data or the portion of the digital multimedia data;
generating transformed digital multimedia data by applying the one or more transforms to the digital multimedia data or the portion of the digital multimedia data;
embedding the digital information in the transformed digital multimedia data; and
generating the digital multimedia data embedded with the digital information by applying one or more inverse transforms of the one or more transforms to the embedded transformed digital multimedia data;
wherein said digital multimedia data is an image and said one or more transforms are Karhunen-Loeve transforms and said digital information is embedded in the first Li coefficients of the transformed digital multimedia data, where the number Li is determined in accordance with HVS analysis.
2. A method as claimed in claim 1, wherein the method further comprises the following steps for selecting said portion of said digital multimedia data:
partitioning the image into blocks;
clustering the partitioned blocks into classes according to a homogeneity criteria; and
selecting one or more of the clusters as said portion of said digital multimedia data in accordance with HVS analysis.
3. A method as claimed in claim 2, wherein the selecting step is determined in accordance with HVS analysis and a user's selection.
4. A method as claimed in claim 1, wherein said digital information is a watermark.
5. A method as claimed in claim 1, wherein said method further includes the following step:
storing the digital multimedia data as a reference file together with associated information.
6. A method as claimed in claim 1, wherein said method further includes the following steps
extracting invariant representations of the image;
storing said extracted representations together with associated information in a reference file.
7. A method as claimed in claim 1, wherein said method further includes the following steps
extracting key features from said image; and
storing the key features together with associated information.
8. A method as claimed in claim 7, wherein said key features include: the positions of the key features, invariant attributes of the key features, maximum and minimum colour values of the image, the size of the image.
9. A method as claimed in claim 1, wherein said digital information is a copyright notice.
10. A method as claimed in claim 1, wherein said digital information is an annotation of said digital multimedia data.
11. A method as claimed in claim 1, wherein said digital information is authentication information verifying said digital multimedia data.
12. A method of embedding digital information in digital multimedia data, wherein said method includes the following steps:
generating one or more transforms from the digital multimedia data or a portion of the digital multimedia data, wherein said transforms are dependent on the digital multimedia data or the portion of the digital multimedia data;
generating transformed digital multimedia data by applying the one or more transforms to the digital multimedia data or the portion of the digital multimedia data;
embedding the digital information in the one or more transforms or their inverses; and
generating the digital multimedia data embedded with the digital information by applying one or more embedded inverse transforms to the transformed digital multimedia data.
13. A method as claimed in claim 12, wherein said digital multimedia data is an image.
14. A method as claimed in claim 12, wherein the method further comprises the following steps for selecting said portion of said digital multimedia data: partitioning the image into blocks;
clustering the partitioned blocks into classes according to a homogeneity criteria; and
selecting one or more of the clusters as said portion of said digital multimedia data in accordance with HVS analysis.
15. A method as claimed in claim 14, wherein the selecting step is determined in accordance with HVS analysis and a user's selection.
16. A method as claimed in claim 12, wherein said transform is a Karhunen-Loeve transform.
17. A method as claimed in claim 12, wherein said digital multimedia data is an image and said one or more transforms are Karhunen-Loeve transforms and said digital information is embedded in the first Lieigenvectors of the Karhunen-Loeve transforms, where the number Li is determined in accordance with HVS analysis.
18. A method as claimed in claim 12, wherein said digital information is a watermark.
19. A method as claimed in claim 12, wherein said method further includes the following step:
storing the digital multimedia data as a reference file together with associated information.
20. A method as claimed in claim 13, wherein said method further includes the following steps
extracting invariant representations of the image;
storing said extracted representations together with associated information in a reference file.
21. A method as claimed in claim 13, wherein said method further includes the following steps
extracting key features from said image; and
storing the key features together with associated information.
22. A method as claimed in claim 21, wherein said key features include: the positions of the key features, invariant attributes of the key features, colour values of the key features, maximum and minimum colour values of the image, the size of the image.
23. A method as claimed in claim 12, wherein said digital information is a copyright notice.
24. A method as claimed in claim 12, wherein said digital information is an annotation of said digital multimedia data.
25. A method as claimed in claim 12, wherein said digital information is authentication information verifying said digital multimedia data.
26. A method of embedding digital information in a digital image, wherein said method includes the following steps:
partitioning the image into blocks of pixels;
clustering the partitioned blocks into classes according to a homogeneity criteria;
selecting one or more of the clusters in accordance with HVS analysis;
generating an associated Karhunen-Loeve (K-L) transform for each selected cluster, wherein the image blocks of each selected cluster are used as a vector population for the purposes of generating the associated K-L transform;
applying the associated K-L transform to the image blocks of the selected cluster to produce corresponding transformed image blocks;
embedding, for each selected cluster, the digital information in the associated K-L transform; and
applying, for each selected cluster, an inverse transform of the embedded associated K-L transform to the transformed image blocks of the selected cluster.
27. A method as claimed in claim 26, wherein said digital information is embedded in the first Li eigenvectors of the Karhunen-Loeve transforms, where the number Li is determined in accordance with HVS analysis.
28. A method as claimed in claim 26, wherein said digital information is a noise-like watermark.
29. A method as claimed in claim 26, wherein the selecting step is determined in accordance with HVS analysis and a user's selection.
30. A method of embedding digital information in a digital image, wherein said method includes the following steps:
partitioning the image into blocks of pixels;
clustering the partitioned blocks into classes according to a homogeneity criteria;
selecting one or more of the clusters in accordance with HVS analysis;
generating an associated Karhunen-Loeve (K-L) transform for each selected cluster,
wherein the image blocks of each selected cluster are used as a vector population for the purposes of generating the associated K-L transform;
applying the associated K-L transform to the image blocks of the selected cluster to produce corresponding transformed image blocks;
embedding, for each selected cluster, the digital information in the selected transformed image blocks; and
applying, for each selected cluster, an inverse transform of the associated K-L transform to the embedded transform image blocks of the selected cluster.
31. A method as claimed in claim 30, wherein said digital information is embedded in the first Li coefficients of the transformed image blocks, where the number Li is determined in accordance with HVS analysis.
32. A method as claimed in claim 30, wherein said digital information is a noise-like watermark.
33. A method as claimed in claim 30, wherein the selecting step is determined in accordance with HVS analysis and a user's selection.
34. A method of embedding digital information in a digital image, wherein said method includes the following steps:
partitioning the image into blocks of pixels;
clustering the partitioned blocks into classes according to a homogeneity criteria;
selecting one or more of the clusters in accordance with HVS analysis;
generating an associated Karhunen-Loeve (K-L) transform for each selected cluster, wherein the image blocks of each selected cluster are used as a vector population for the purposes of generating the associated K-L transform;
applying the associated K-L transform to the image blocks of the selected cluster to produce corresponding transformed image blocks;
selecting, for each selected cluster, either the associated K-L transform or its corresponding transformed image blocks for embedding;
embedding, for each selected cluster, the digital information in the selected K-L transform or the selected transformed image blocks;
applying, for those selected clusters in which the digital information has been embedded in the associated K-L transform, an inverse transform of the embedded associated K-L transform to the transformed image blocks of the selected cluster; and
applying, for those selected clusters in which the digital information has been embedded in the transformed image blocks, an inverse transform of the associated K-L transform to the embedded transform image blocks of the selected cluster.
35. A method as claimed in claim 34, wherein said digital information is embedded in the first Li eigenvectors of the Karhunen-Loeve transforms, where the number Li is determined in accordance with HVS analysis.
36. A method as claimed in claim 34, wherein said digital information is embedded in the first Li coefficients of the transformed image blocks, where the number Li is determined in accordance with HVS analysis.
37. A method as claimed in claim 34, wherein said digital information is a noise-like watermark.
38. A method as claimed in claim 34, wherein the selecting step is determined in accordance with HVS analysis and a user's selection.
39. A method of embedding digital information in a plurality of images, wherein the images are stored and classified into one or more classes and said method includes the following steps:
generating one or more transforms from the images of said respective one or more classes, wherein said transforms are dependent on the images;
generating transformed images by applying the one or more transforms to the images of said respective one or more classes;
embedding the digital information in the transformed images; and
generating the images data embedded with the digital information by applying one or more inverse transforms of the one or more transforms to the embedded transformed images;
wherein said one or more transforms are Karhunen-Loeve transforms and said digital information is embedded in the first Li coefficients of the transformed digital multimedia data, where the number Li is determined in accordance with HVS analysis.
40. A method of embedding digital information in a plurality of images, wherein the images are stored and classified into one or more classes and said method includes the following steps:
generating one or more transforms from the images of said respective one or more classes, wherein said transforms are dependent on the images;
generating transformed images by applying the one or more transforms to the images of said respective one or more classes;
embedding the digital information in the one or more transforms or their inverses; and
generating the images embedded with the digital information by applying one or more embedded inverse transforms to the transformed images.
41. A method of embedding digital information into video images including a plurality of frames, wherein said method includes the following steps:
extracting one or more key frames from the video images;
generating one or more transforms from the key frames or a part thereof, wherein said transforms are dependent on the key frames or the part thereof;
generating transformed key frames or part thereof by applying the one or more transforms to the key frames or the part thereof;
embedding the digital information in the transformed key frames or part thereof; and
generating the key frames or part thereof embedded with the digital information by applying one or more inverse transforms of the one or more transforms to the embedded key frames or part thereof;
wherein said one or more transforms are Karhunen-Loeve transforms and said digital information is embedded in the first Li coefficients of the transformed digital multimedia data, where the number Li is determined in accordance with HVS analysis.
42. A method of embedding digital information into video images including a plurality of frames, wherein said method includes the following steps:
extracting one or more key frames from the video images;
generating one or more transforms from the key frames or a part thereof, wherein said transforms are dependent on the key frames or the part thereof;
generating transformed key frames or part thereof by applying the one or more transforms to the key frames or the part thereof;
embedding the digital information in the one or more transforms or their inverses; and
generating the key frames or part thereof embedded with the digital information by applying one or more embedded inverse transforms to the transformed key frames or part thereof.
43. A method of modifying a first image in accordance with an image signature, wherein the image signature includes encrypted key points of a second image and their invariant attributes and other aligned global information of the second image and the method includes the steps of:
decrypting the image signature to obtain the key points and their invariant attributes and other global information of the second image;
extracting key points and their invariant attributes from the first image; extracting other global information from the first image;
matching the key points and their invariant attributes and other global information of the first and second images; and
normalising the first image based on said matched key points and their invariant attributes and other global information.
44. An apparatus for embedding digital information in digital multimedia data, wherein said apparatus includes:
means for generating one or more transforms from the digital multimedia data or a portion of the digital multimedia data, wherein said transforms are dependent on the digital multimedia data or the portion of the digital multimedia data;
means for generating transformed digital multimedia data by applying the one or more transforms to the digital multimedia data or the portion of the digital multimedia data;
means for embedding the digital information in the transformed digital multimedia data; and
means for generating the digital multimedia data embedded with the digital information by applying one or more inverse transforms of the one or more transforms to the embedded transformed digital multimedia data
wherein said digital multimedia data is an image and said one or more transforms are Karhunen-Loeve transforms and said digital information is embedded in the first Li coefficients of the transformed digital multimedia data, where the number Li is determined in accordance with HVS analysis.
45. An apparatus for embedding digital information in digital multimedia data, wherein said apparatus includes:
means for generating one or more transforms from the digital multimedia data or a portion of the digital multimedia data, wherein said transforms are dependent on the digital multimedia data or the portion of the digital multimedia data;
means for generating transformed digital multimedia data by applying the one or more transforms to the digital multimedia data or the portion of the digital multimedia data;
means for embedding the digital information in the one or more transforms or their inverses; and
means for generating the digital multimedia data embedded with the digital information by applying one or more embedded inverse transforms to the transformed digital multimedia data.
46. An apparatus for embedding digital information in a digital image, wherein said apparatus includes:
means for partitioning the image into blocks of pixels;
means for clustering the partitioned blocks into classes according to a homogeneity criteria;
means for selecting one or more of the clusters in accordance with HVS analysis;
means for generating an associated Karhunen-Loeve (K-L) transform for each selected cluster, wherein the image blocks of each selected cluster are used as a vector population for the purposes of generating the associated K-L transform;
means for applying the associated K-L transform to the image blocks of the selected cluster to produce corresponding transformed image blocks.
47. An apparatus for embedding digital information in a digital image, wherein said apparatus includes:
means for partitioning the image into blocks of pixels;
means for clustering the partitioned blocks into classes according to a homogeneity criteria;
means for selecting one or more of the clusters in accordance with HVS analysis;
means for generating an associated Karhunen-Loeve (K-L) transform for each selected cluster, wherein the image blocks of each selected cluster are used as a vector population for the purposes of generating the associated K-L transform;
means for applying the associated K-L transform to the image blocks of the selected cluster to produce corresponding transformed image blocks;
means for embedding, for each selected cluster, the digital information in the selected transformed image blocks; and
means for applying, for each selected cluster, an inverse transform of the associated K-L transform to the embedded transform image blocks of the selected cluster.
48. An apparatus for embedding digital information in a digital image, wherein said apparatus includes:
means for partitioning the image into blocks of pixels;
means for clustering the partitioned blocks into classes according to a homogeneity criteria;
means for selecting one or more of the clusters in accordance with HVS analysis;
means for generating an associated Karhunen-Loeve (K-L) transform for each selected cluster, wherein the image blocks of each selected cluster are used as a vector population for the purposes of generating the associated K-L transform;
means for applying the associated K-L transform to the image blocks of the selected cluster to produce corresponding transformed image blocks;
means for selecting, for each selected cluster, either the associated K-L transform or its corresponding transformed image blocks for embedding;
means for embedding, for each selected cluster, the digital information in the selected K-L transform or the selected transformed image blocks;
means for applying, for those selected clusters in which the digital information has been embedded in the associated K-L transform, an inverse transform of the embedded associated K-L transform to the transformed image blocks of the selected cluster; and
means for applying, for those selected clusters in which the digital information has been embedded in the transformed image blocks, an inverse transform of the associated K-L transform to the embedded transform image blocks of the selected cluster.
49. An apparatus for embedding digital information in a plurality of images, wherein the images are stored and classified into one or more classes and said apparatus includes:
means for generating one or more transforms from the images of said respective one or more classes, wherein said transforms are dependent on the images;
means for generating transformed images by applying the one or more transforms to the images of said respective one or more classes;
means for embedding the digital information in the one or more transforms or their inverses; and
means for generating the images embedded with the digital information by applying one or more embedded inverse transforms to the transformed images.
50. An apparatus for embedding digital information into video images including a plurality of frames, wherein said apparatus includes:
means for extracting one or more key frames from the video images;
means for generating one or more transforms from the key frames or a part thereof, wherein said transforms are dependent on the key frames or the part thereof;
means for generating transformed key frames or part thereof by applying the one or more transforms to the key frames or the part thereof;
means for embedding the digital information in the transformed key frames or part thereof; and
means for generating the key frames or part thereof embedded with the digital information by applying one or more inverse transforms of the one or more transforms to the embedded key frames or part thereof;
wherein said one or more transforms are Karhunen-Loeve transforms and said digital information is embedded in the first Li coefficients of the transformed digital multimedia data, where the number Li is determined in accordance with HVS analysis.
51. An apparatus for embedding digital information into video images including a plurality of frames, wherein said apparatus includes:
means for extracting one or more key frames from the video images;
means for generating one or more transforms from the key frames or a part thereof, wherein said transforms are dependent on the key frames or the part thereof;
means for generating transformed key frames or part thereof by applying the one or more transforms to the key frames or the part thereof;
means for embedding the digital information in the one or more transforms or their inverses; and
means for generating the key frames or part thereof embedded with the digital information by applying one or more embedded inverse transforms to the transformed key frames or part thereof.
52. An apparatus for modifying a first image in accordance with an image signature, wherein the image signature includes encrypted key points of a second image and their invariant attributes and other aligned global information of the second image and the apparatus includes:
means for decrypting the image signature to obtain the key points and their invariant attributes and other global information of the second image;
means for extracting key points and their invariant attributes from the first image;
means for extracting other global information from the first image;
means for matching the key points and their invariant attributes and other global information of the first and second images; and
means for normalising the first image based on said matched key points and their invariant attributes and other global information.
53. A computer program product including a computer readable medium having recorded thereon a computer program for embedding digital information in digital multimedia data, wherein said computer program product includes:
means for generating one or more transforms from the digital multimedia data or a portion of the digital multimedia data, wherein said transforms are dependent on the digital multimedia data or the portion of the digital multimedia data;
means for generating transformed digital multimedia data by applying the one or more transforms to the digital multimedia data or the portion of the digital multimedia data;
means for embedding the digital information in the transformed digital multimedia data; and
means for generating the digital multimedia data embedded with the digital information by applying one or more inverse transforms of the one or more transforms to the embedded transformed digital multimedia data;
wherein said digital multimedia data is an image and said one or more transforms are Karhunen-Loeve transforms and said digital information is embedded in the first Li coefficients of the transformed digital multimedia data, where the number Li is determined in accordance with HVS analysis.
54. A computer program product including a computer readable medium having recorded thereon a computer program for embedding digital information in digital multimedia data, wherein said computer program product includes:
means for generating one or more transforms from the digital multimedia data or a portion of the digital multimedia data, wherein said transforms are dependent on the digital multimedia data or the portion of the digital multimedia data;
means for generating transformed digital multimedia data by applying the one or more transforms to the digital multimedia data or the portion of the digital multimedia data;
means for embedding the digital information in the one or more transforms or their inverses; and
means for generating the digital multimedia data embedded with the digital information by applying one or more embedded inverse transforms to the transformed digital multimedia data.
55. A computer program product including a computer readable medium having recorded thereon a computer program for embedding digital information in a digital image, wherein said computer program product includes:
means for partitioning the image into blocks of pixels;
means for clustering the partitioned blocks into classes according to a homogeneity criteria;
means for selecting one or more of the clusters in accordance with HVS analysis;
means for generating an associated Karhunen-Loeve (K-L) transform for each selected cluster, wherein the image blocks of each selected cluster are used as a vector population for the purposes of generating the associated K-L transform;
means for applying the associated K-L transform to the image blocks of the selected cluster to produce corresponding transformed image blocks;
means for embedding, for each selected cluster, the digital information in the associated K-L transform; and
means for applying, for each selected cluster, an inverse transform of the embedded associated K-L transform to the transformed image blocks of the selected cluster.
56. A computer program product including a computer readable medium having recorded thereon a computer program for embedding digital information in a digital image, wherein said computer program product includes:
means for partitioning the image into blocks of pixels;
means for clustering the partitioned blocks into classes according to a homogeneity criteria;
means for selecting one or more of the clusters in accordance with HVS analysis;
means for generating an associated Karhunen-Loeve (K-L) transform for each selected cluster, wherein the image blocks of each selected cluster are used as a vector population for the purposes of generating the associated K-L transform;
means for applying the associated K-L transform to the image blocks of the selected cluster to produce corresponding transformed image blocks;
means for embedding, for each selected cluster, the digital information in the selected transformed image blocks; and
means for applying, for each selected cluster, an inverse transform of the associated K-L transform to the embedded transform image blocks of the selected cluster.
57. A computer program product including a computer readable medium having recorded thereon a computer program for embedding digital information in a digital image, wherein said computer program product includes:
means for partitioning the image into blocks of pixels;
means for clustering the partitioned blocks into classes according to a homogeneity criteria;
means for selecting one or more of the clusters in accordance with HVS analysis;
means for generating an associated Karhunen-Loeve (K-L) transform for each selected cluster, wherein the image blocks of each selected cluster are used as a vector population for the purposes of generating the associated K-L transform;
means for applying the associated K-L transform to the image blocks of the selected cluster to produce corresponding transformed image blocks;
means for selecting, for each selected cluster, either the associated K-L transform or its corresponding transformed image blocks for embedding;
means for embedding, for each selected cluster, the digital information in the selected K-L transform or the selected transformed image blocks;
means for applying, for those selected clusters in which the digital information has been embedded in the associated K-L transform, an inverse transform of the embedded associated K-L transform to the transformed image blocks of the selected cluster; and
means for applying, for those selected clusters in which the digital information has been embedded in the transformed image blocks, an inverse transform of the associated K-L transform to the embedded transform image blocks of the selected cluster.
58. A computer program product including a computer readable medium having recorded thereon a computer program for embedding digital information in a plurality of images, wherein the images are stored and classified into one or more classes and said computer program product includes:
means for generating one or more transforms from the images of said respective one or more classes, wherein said transforms are dependent on the images;
means for generating transformed images by applying the one or more transforms to the images of said respective one or more classes;
means for embedding the digital information in the one or more transforms or their inverses; and
means for generating the images embedded with the digital information by applying one or more embedded inverse transforms to the transformed images.
59. A computer program product including a computer readable medium having recorded thereon a computer program for embedding digital information into video images including a plurality of frames, wherein said computer program product includes:
means for extracting one or more key frames from the video images;
means for generating one or more transforms from the key frames or a part thereof, wherein said transforms are dependent on the key frames or the part thereof;
means for generating transformed key frames or part thereof by applying the one or more transforms to the key frames or the part thereof;
means for embedding the digital information in the transformed key frames or part thereof; and
means for generating the key frames or part thereof embedded with the digital information by applying one or more inverse transforms of the one or more transforms to the embedded key frames or part thereof;
wherein said one or more transforms are Karhunen-Loeve transforms and said digital information is embedded in the first Li coefficients of the transformed digital multimedia data, where the number Li is determined in accordance with HVS analysis.
60. A computer program product including a computer readable medium having recorded thereon a computer program for embedding digital information into video images including a plurality of frames, wherein said computer program product includes:
means for extracting one or more key frames from the video images;
means for generating one or more transforms from the key frames or a part thereof, wherein said transforms are dependent on the key frames or the part thereof;
means for generating transformed key frames or part thereof by applying the one or more transforms to the key frames or the part thereof;
means for embedding the digital information in the one or more transforms or their inverses; and
means for generating the key frames or part thereof embedded with the digital information by applying one or more embedded inverse transforms to the transformed key frames or part thereof.
61. A computer program product including a computer readable medium having recorded thereon a computer program for modifying a first image in accordance with an image signature, wherein the image signature includes encrypted key points of a second image and their invariant attributes and other aligned global information of the second image and the computer program product includes:
means for decrypting the image signature to obtain the key points and their invariant attributes and other global information of the second image;
means for extracting key points and their invariant attributes from the first image;
means for extracting other global information from the first image;
means for matching the key points and their invariant attributes and other global information of the first and second images; and
means for normalizing the first image based on said matched key points and their invariant attributes and other global information.
62. A method as claimed in claim 34, further comprising registering the image including the following steps:
extracting key points from the image and their invariant attributes;
extracting other global information from the image;
aligning the extracted key points and their invariant attributes with the global information; and
encrypting the aligned information to provide an image signature.
63. An apparatus as claimed in claim 48, further comprising means for registering the image including:
means for extricating key points from the image and their invariant attributes;
means for extracting other global information from the image;
means for aligning the extracted key points and their invariant attributes with the global information;
means for encrypting the aligned information to provide an image signature.
64. A computer program product as claimed in claim 56, in which the computer program registers the image and includes:
means for extracting key points from the image and their invariant attributes;
means for extracting other global information from the image;
means for aligning the extracted key points and their invariant attributes with the global information;
means for encrypting the aligned information to provide an image signature.
65. A method as claimed in claim 39, further comprising modifying a first image in accordance with an image signature, wherein the image signature includes encrypted key points of a second image and their invariant attributes and other aligned global information of the second image, wherein said modifying the first image includes the steps of:
decrypting the image signature to obtain the key points and their invariant attributes and other global information of the second image;
extracting key points and their invariant attributes from the first image;
extracting other global information from the first image;
matching the key points and their invariant attributes and other global information of the first and second images; and
normalising the first image based on said matched key points and their invariant attributes and other global information.
66. An apparatus as claimed in claim 49, further comprising means for modifying a first image in accordance with an image signature, wherein the image signature includes encrypted key points of a second image and their invariant attributes and other aligned global information of the second image, said means for modifying including:
means for decrypting the image signature to obtain the key points and their invariant attributes and other global information of the second image;
means for extracting key points and their invariant attributes from the first image;
means for extracting other global information from the first image;
means for matching the key points and their invariant attributes and other global information from the first image; and
means for normalising the first image based on said matched key points and their invariant attributes and other global information.
67. A computer program product as claimed in claim 60, including:
means for modifying the image in accordance with the image signature; wherein the image signature includes encrypted key points of a second image and their invariant attributes and other aligned global information of the second image, said means for modifying including:
means for decrypting the image signature to obtain the key points and their invariant attributes and other global information of the second image;
means for extracting key points and their invariant attributes from the first image;
means for extracting other global information from the first image;
means for matching the key points and their invariant attributes and other global information of the first image based on said matched key points and their invariant attributes and other global information.
68. A method as claimed in claim 12, further comprising: extracting digital information from first digital multimedia data having embedded therein said digital information including the following steps:
deriving one or more embedded transforms from a reference file, wherein said embedded transforms are dependent on second digital multimedia data or a portion thereof;
deriving transformed digital multimedia data from the reference file, wherein said transformed digital multimedia data is dependent on the second digital multimedia data or a portion thereof;
generating transformed digital multimedia data by applying one or more of the derived embedded transforms to the first digital multimedia data or a portion thereof; and
correlating the generated transformed digital multimedia with the derived transformed digital multimedia to check whether the digital information embedded in the first multimedia data is the same as that derived from the reference file.
69. A method as claimed in claim 12, further comprising extracting digital information from first digital multimedia data having embedded therein said digital information, including the following steps:
deriving one or more transforms from a reference file, wherein said transforms are dependent on second digital multimedia data or a portion thereof;
deriving transformed digital multimedia data from the reference file, wherein said transformed digital multimedia data is dependent on the second digital multimedia data or a portion thereof;
generating embedded transformed digital multimedia data by applying one or more of the derived embedded transforms to the first digital multimedia data or a portion thereof; and
correlating the generated embedded transformed digital multimedia with the derived transformed digital multimedia so as to extract the digital information.
70. An apparatus as claimed in claim 49, further comprising means for extracting digital information from first digital multimedia data having embedded therein said digital information, including:
means for deriving one or more embedded transforms from a reference file, wherein said embedded transforms are dependent on second digital multimedia data or a portion thereof;
means for deriving transformed digital multimedia data from the reference file, wherein said transformed digital multimedia data is dependent on the second digital multimedia data or a portion thereof;
means for generating transformed digital multimedia data by applying one or more of the derived embedded transforms to the first digital multimedia data or a portion thereof; and
means for correlating the generated transformed digital multimedia with the derived transformed digital multimedia to check whether the digital information embedded in the first multimedia data is the same as that derived from the reference file.
71. A computer program product as claimed in claim 60, including means extracting digital information from first digital multimedia data having embedded therein said digital information, including:
means for deriving one or more embedded transforms from a reference file, wherein said embedded transforms are dependent on second digital multimedia data or a portion thereof;
means for deriving transformed digital multimedia data from the reference file, wherein said transformed digital multimedia data is dependent on the second digital multimedia data or a portion thereof;
means for generating transformed digital multimedia data by applying one or more of the derived embedded transforms to the first digital multimedia data or a portion thereof; and
means for correlating the generated transformed digital multimedia with the derived transformed digital multimedia to check whether the digital information embedded in the first multimedia data is the same as that derived from the reference file, having embedded therein said digital information, wherein said computer program product includes:
means for partitioning the image into blocks of pixels;
means for clustering the partitioned blocks into classes according to a homogeneity criteria;
means for selecting one or more of the clusters in accordance with information obtained from a reference file;
means for retrieving, for each selected cluster, an average block of pixels stored in the reference file;
means for averaging, for each selected cluster, the blocks of pixels of the image; and
means for correlating, for each selected pixel, the averaged blocks of the image with the retrieved blocks to determine whether a watermark exists.
72. A method of embedding digital information in a plurality of images, wherein the images are stored and classified into one or more classes and said method includes the following steps:
generating one or more transforms from the images of said respective one or more classes, wherein said transforms are dependent on the images; generating transformed images by applying the one or more transforms to the images of said respective one or more classes;
embedding the digital information in the transformed images;
generating the images data embedded with the digital information by applying one or more inverse transforms of the one or more transforms to the embedded transformed images; and
modifying a first image in accordance with an image signature, wherein the image signature includes encrypted key points of a second image and their invariant attributes and other aligned global information of the second image, wherein said modifying the first image includes the steps of:
decrypting the image signature to obtain the key points and their invariant attributes and other global information of the second image;
extracting key points and their invariant attributes from the first image; extracting other global information from the first image;
matching the key points and their invariant attributes and other global information of the first and second images; and
normalising the first image based on said matched key points and their invariant attributes and other global information.
US09445142 1998-12-29 1998-12-29 Method and apparatus for embedding digital information in digital multimedia data Expired - Fee Related US6678389B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/SG1998/000110 WO2000039954A1 (en) 1998-12-29 1998-12-29 Method and apparatus for embedding digital information in digital multimedia data

Publications (1)

Publication Number Publication Date
US6678389B1 true US6678389B1 (en) 2004-01-13

Family

ID=20429902

Family Applications (1)

Application Number Title Priority Date Filing Date
US09445142 Expired - Fee Related US6678389B1 (en) 1998-12-29 1998-12-29 Method and apparatus for embedding digital information in digital multimedia data

Country Status (3)

Country Link
US (1) US6678389B1 (en)
GB (1) GB2366112B (en)
WO (1) WO2000039954A1 (en)

Cited By (95)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010054150A1 (en) * 2000-03-18 2001-12-20 Levy Kenneth L. Watermark embedding functions in rendering description files
US20020038294A1 (en) * 2000-06-16 2002-03-28 Masakazu Matsugu Apparatus and method for detecting or recognizing pattern by employing a plurality of feature detecting elements
US20020094082A1 (en) * 2000-12-13 2002-07-18 Eastman Kodak Company Method and system for embedding message data in a digital image sequence
US20020114458A1 (en) * 2001-02-05 2002-08-22 Belenko Vyacheslav S. Copy protection method for digital media
US20020114459A1 (en) * 2001-02-21 2002-08-22 Belenko Vyacheslav S. Watermark-based copy management system and method for digital media copy protection
US20020118838A1 (en) * 2001-02-05 2002-08-29 Belenko Vyacheslav S. Copy protection method and system for digital media
US20020184502A1 (en) * 2001-03-28 2002-12-05 Sall M. A. Method of watermarking digital data
US20030021439A1 (en) * 2001-07-30 2003-01-30 Jeffrey Lubin Secure robust high-fidelity watermarking
US20030033530A1 (en) * 1996-05-16 2003-02-13 Sharma Ravi K. Variable message coding protocols for encoding auxiliary data in media signals
US20030147547A1 (en) * 2001-01-10 2003-08-07 Ching-Yung Lin Method and apparatus for watermarking images
US20030191941A1 (en) * 2002-04-05 2003-10-09 Koichi Terada Data transmitting apparatus, contents distribution apparatus, contents distribution system, digital contents distribution method and data storage apparatus
US20040009763A1 (en) * 2002-06-20 2004-01-15 Stone Chris L. Secure tracking system and method for video program content
US20040073916A1 (en) * 2002-10-15 2004-04-15 Verance Corporation Media monitoring, management and information system
US20040109583A1 (en) * 2002-12-09 2004-06-10 Condon John B. System and method for secret communication
US20040120546A1 (en) * 1998-12-02 2004-06-24 Hiroshi Yoshiura Method of judging digital watermark information
US20040181815A1 (en) * 2001-11-19 2004-09-16 Hull Jonathan J. Printer with radio or television program extraction and formating
US20040181747A1 (en) * 2001-11-19 2004-09-16 Hull Jonathan J. Multimedia print driver dialog interfaces
US20050005760A1 (en) * 2001-11-19 2005-01-13 Hull Jonathan J. Music processing printer
US20050010409A1 (en) * 2001-11-19 2005-01-13 Hull Jonathan J. Printable representations for time-based media
US20050008221A1 (en) * 2001-11-19 2005-01-13 Hull Jonathan J. Printing system with embedded audio/video content recognition and processing
US20050024682A1 (en) * 2000-11-30 2005-02-03 Hull Jonathan J. Printer with embedded retrieval and publishing interface
US20050034057A1 (en) * 2001-11-19 2005-02-10 Hull Jonathan J. Printer with audio/video localization
US20050036174A1 (en) * 2003-06-19 2005-02-17 Jeffrey Lubin Method and apparatus for providing a noise estimation for automatic selection of dither patterns in low frequency watermarks
US20050068570A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Printer user interface
US20050068572A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Printer with hardware and software interfaces for media devices
US20050068567A1 (en) * 2003-09-25 2005-03-31 Hull Jonathan J. Printer with audio or video receiver, recorder, and real-time content-based processing logic
US20050068573A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Networked printing system having embedded functionality for printing time-based media
US20050068569A1 (en) * 2003-09-25 2005-03-31 Hull Jonathan J. Printer with document-triggered processing
US20050071763A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Stand alone multimedia printer capable of sharing media processing tasks
US20050071520A1 (en) * 2003-09-25 2005-03-31 Hull Jonathan J. Printer with hardware and software interfaces for peripheral devices
US20050068571A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Stand alone multimedia printer with user interface for allocating processing
US20050068568A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. User interface for networked printer
US20050069362A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Printer having embedded functionality for printing time-based media
US20050071519A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Stand alone printer with hardware / software interfaces for sharing multimedia processing
US20050071746A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Networked printer with hardware and software interfaces for peripheral devices
US20050068581A1 (en) * 2003-09-25 2005-03-31 Hull Jonathan J. Printer with multimedia server
US6909784B1 (en) * 1999-06-25 2005-06-21 Victor Company Of Japan, Ltd. Recording apparatus and recording method of electronic watermark
US6922777B1 (en) * 1999-03-31 2005-07-26 Canon Kabushiki Kaisha Apparatus, computer medium, and method for embedding and extracting a first and second data into a watermark
US20050231739A1 (en) * 2004-03-30 2005-10-20 Dar-Shyang Lee Projector/printer for displaying or printing of documents
US20050262351A1 (en) * 2004-03-18 2005-11-24 Levy Kenneth L Watermark payload encryption for media including multiple watermarks
US20050271246A1 (en) * 2002-07-10 2005-12-08 Sharma Ravi K Watermark payload encryption methods and systems
US20060055982A1 (en) * 2003-01-14 2006-03-16 Lokhoff Gerardus C P Watermarking film prints
US20060075441A1 (en) * 2004-10-06 2006-04-06 Sony Corporation Method and system for a personal video recorder comprising multiple removable storage/tuner units
US20060074807A1 (en) * 2004-10-06 2006-04-06 Sony Corporation Method and system for content sharing and authentication between multiple devices
US7058570B1 (en) * 2000-02-10 2006-06-06 Matsushita Electric Industrial Co., Ltd. Computer-implemented method and apparatus for audio data hiding
US20060212705A1 (en) * 2005-03-17 2006-09-21 Interdigital Technology Corporation Wireless communication method and apparatus for generating, watermarking and securely transmitting content
US20060256388A1 (en) * 2003-09-25 2006-11-16 Berna Erol Semantic classification and enhancement processing of images for printing applications
US20070174059A1 (en) * 1996-05-16 2007-07-26 Rhoads Geoffrey B Methods, Systems, and Sub-Combinations Useful in Media Identification
US20070245400A1 (en) * 1998-11-06 2007-10-18 Seungyup Paek Video description system and method
US20080002854A1 (en) * 2003-10-08 2008-01-03 Verance Corporation Signal continuity assessment using embedded watermarks
US20080148405A1 (en) * 2006-12-18 2008-06-19 Carol Davids Method for securing streaming multimedia network transmissions
US20080181308A1 (en) * 2005-03-04 2008-07-31 Yong Wang System and method for motion estimation and mode decision for low-complexity h.264 decoder
US20080234069A1 (en) * 2007-03-23 2008-09-25 Acushnet Company Functionalized, Crosslinked, Rubber Nanoparticles for Use in Golf Ball Castable Thermoset Layers
US20080303942A1 (en) * 2001-12-06 2008-12-11 Shih-Fu Chang System and method for extracting text captions from video and generating video summaries
US20080304755A1 (en) * 2007-06-08 2008-12-11 Microsoft Corporation Face Annotation Framework With Partial Clustering And Interactive Labeling
US20090070877A1 (en) * 2006-12-18 2009-03-12 Carol Davids Method for securing streaming multimedia network transmissions
US20090316778A1 (en) * 2002-04-26 2009-12-24 Jae-Gon Kim Method And System For Optimal Video Transcoding Based On Utility Function Descriptors
US7644146B2 (en) 2004-06-02 2010-01-05 Hewlett-Packard Development Company, L.P. System and method for discovering communities in networks
US7697717B2 (en) 2001-03-28 2010-04-13 Lg Electronics Inc. Method of embedding watermark into digital image
US20100153739A1 (en) * 2008-12-15 2010-06-17 Guymon Jr John Lewis Securable independent electronic document
US7756288B2 (en) 2003-05-29 2010-07-13 Jeffrey Lubin Method and apparatus for analog insertion of low frequency watermarks
US7760904B2 (en) 2001-02-21 2010-07-20 Lg Electronics Inc. Proprietary watermark system for secure digital media and content distribution
US20100310134A1 (en) * 2009-06-08 2010-12-09 Microsoft Corporation Assisted face recognition tagging
US7877813B2 (en) 2001-05-11 2011-01-25 Lg Electronics Inc. Copy protection method and system for digital media
US20110025710A1 (en) * 2008-04-10 2011-02-03 The Trustees Of Columbia University In The City Of New York Systems and methods for image archeology
US20110145232A1 (en) * 2008-06-17 2011-06-16 The Trustees Of Columbia University In The City Of New York System and method for dynamically and interactively searching media data
US8005258B2 (en) 2005-04-26 2011-08-23 Verance Corporation Methods and apparatus for enhancing the robustness of watermark extraction from digital host content
US8259938B2 (en) 2008-06-24 2012-09-04 Verance Corporation Efficient and secure forensic marking in compressed
US8280103B2 (en) 2005-04-26 2012-10-02 Verance Corporation System reactions to the detection of embedded watermarks in a digital host content
US8340348B2 (en) 2005-04-26 2012-12-25 Verance Corporation Methods and apparatus for thwarting watermark detection circumvention
US20130070971A1 (en) * 2010-03-29 2013-03-21 Sony Corporation Data processing apparatus, data processing method, image processing apparatus and method, and program
US8451086B2 (en) 2000-02-16 2013-05-28 Verance Corporation Remote control signaling using audio watermarks
US8515773B2 (en) 2001-08-01 2013-08-20 Sony Corporation System and method for enabling distribution and brokering of content information
US8533481B2 (en) 2011-11-03 2013-09-10 Verance Corporation Extraction of embedded watermarks from a host content based on extrapolation techniques
US8549307B2 (en) 2005-07-01 2013-10-01 Verance Corporation Forensic marking using a common customization function
US8615104B2 (en) 2011-11-03 2013-12-24 Verance Corporation Watermark extraction based on tentative watermarks
US8671069B2 (en) 2008-12-22 2014-03-11 The Trustees Of Columbia University, In The City Of New York Rapid image annotation via brain state decoding and visual pattern mining
US8682026B2 (en) 2011-11-03 2014-03-25 Verance Corporation Efficient extraction of embedded watermarks in the presence of host content distortions
US8719893B2 (en) 1999-03-15 2014-05-06 Sony Corporation Secure module and a method for providing a dedicated on-site media service
US8726304B2 (en) 2012-09-13 2014-05-13 Verance Corporation Time varying evaluation of multimedia content
US8745404B2 (en) 1998-05-28 2014-06-03 Verance Corporation Pre-processed information embedding system
US8745403B2 (en) 2011-11-23 2014-06-03 Verance Corporation Enhanced content management based on watermark extraction records
US8781967B2 (en) 2005-07-07 2014-07-15 Verance Corporation Watermarking in an encrypted domain
US8838978B2 (en) 2010-09-16 2014-09-16 Verance Corporation Content access management using extracted watermark information
US8869222B2 (en) 2012-09-13 2014-10-21 Verance Corporation Second screen content
US8923548B2 (en) 2011-11-03 2014-12-30 Verance Corporation Extraction of embedded watermarks from a host content using a plurality of tentative watermarks
US9106964B2 (en) 2012-09-13 2015-08-11 Verance Corporation Enhanced content distribution using advertisements
US9208334B2 (en) 2013-10-25 2015-12-08 Verance Corporation Content management using multiple abstraction layers
US9251549B2 (en) 2013-07-23 2016-02-02 Verance Corporation Watermark extractor enhancements based on payload ranking
US9262794B2 (en) 2013-03-14 2016-02-16 Verance Corporation Transactional video marking system
US9323902B2 (en) 2011-12-13 2016-04-26 Verance Corporation Conditional access using embedded watermarks
US9330722B2 (en) 1997-05-16 2016-05-03 The Trustees Of Columbia University In The City Of New York Methods and architecture for indexing and editing compressed video over the world wide web
US9547753B2 (en) 2011-12-13 2017-01-17 Verance Corporation Coordinated watermarking
US9571606B2 (en) 2012-08-31 2017-02-14 Verance Corporation Social media viewing system
US9596521B2 (en) 2014-03-13 2017-03-14 Verance Corporation Interactive content acquisition using embedded codes

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7010144B1 (en) 1994-10-21 2006-03-07 Digimarc Corporation Associating data with images in imaging systems
EP1249002B1 (en) 2000-01-13 2011-03-16 Digimarc Corporation Authenticating metadata and embedding metadata in watermarks of media signals
US6834344B1 (en) * 1999-09-17 2004-12-21 International Business Machines Corporation Semi-fragile watermarks
US6834308B1 (en) 2000-02-17 2004-12-21 Audible Magic Corporation Method and apparatus for identifying media content presented on a media playing device
JP2002049631A (en) * 2000-08-01 2002-02-15 Sony Corp Information providing device, method and recording medium
US7562012B1 (en) 2000-11-03 2009-07-14 Audible Magic Corporation Method and apparatus for creating a unique audio signature
US6792130B1 (en) * 2000-12-13 2004-09-14 Eastman Kodak Company System and method for embedding a watermark signal that contains message data in a digital image
WO2002082271A1 (en) * 2001-04-05 2002-10-17 Audible Magic Corporation Copyright detection and protection system and method
US8972481B2 (en) 2001-07-20 2015-03-03 Audible Magic, Inc. Playlist generation method and apparatus
KR20040070267A (en) 2001-12-21 2004-08-06 코닌클리케 필립스 일렉트로닉스 엔.브이. Increasing integrity of watermarks using robust features
US7529659B2 (en) 2005-09-28 2009-05-05 Audible Magic Corporation Method and apparatus for identifying an unknown work
US8006314B2 (en) 2007-07-27 2011-08-23 Audible Magic Corporation System for identifying content of digital data
WO2010073236A1 (en) * 2008-12-22 2010-07-01 France Telecom A method of and apparatus for authenticating data content
US8199651B1 (en) 2009-03-16 2012-06-12 Audible Magic Corporation Method and system for modifying communication flows at a port level
US9081778B2 (en) 2012-09-25 2015-07-14 Audible Magic Corporation Using digital fingerprints to associate data with a work

Citations (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4910716A (en) 1989-01-31 1990-03-20 Amoco Corporation Suppression of coherent noise in seismic data
US5109451A (en) 1988-04-28 1992-04-28 Sharp Kabushiki Kaisha Orthogonal transform coding system for image data
US5212742A (en) 1991-05-24 1993-05-18 Apple Computer, Inc. Method and apparatus for encoding/decoding image data
US5530759A (en) 1995-02-01 1996-06-25 International Business Machines Corporation Color correct digital watermarking of images
US5613004A (en) 1995-06-07 1997-03-18 The Dice Company Steganographic method and device
EP0766468A2 (en) 1995-09-28 1997-04-02 Nec Corporation Method and system for inserting a spread spectrum watermark into multimedia data
US5636292A (en) 1995-05-08 1997-06-03 Digimarc Corporation Steganography methods employing embedded calibration data
US5646997A (en) 1994-12-14 1997-07-08 Barton; James M. Method and apparatus for embedding authentication information within digital data
US5657390A (en) 1995-08-25 1997-08-12 Netscape Communications Corporation Secure socket layer application program apparatus and method
US5659726A (en) 1995-02-23 1997-08-19 Sandford, Ii; Maxwell T. Data embedding
US5664018A (en) 1996-03-12 1997-09-02 Leighton; Frank Thomson Watermarking process resilient to collusion attacks
US5687191A (en) 1995-12-06 1997-11-11 Solana Technology Development Corporation Post-compression hidden data transport
US5689587A (en) 1996-02-09 1997-11-18 Massachusetts Institute Of Technology Method and apparatus for data hiding in images
US5710834A (en) 1995-05-08 1998-01-20 Digimarc Corporation Method and apparatus responsive to a code signal conveyed through a graphic image
US5721788A (en) 1992-07-31 1998-02-24 Corbis Corporation Method and system for digital image signatures
US5734752A (en) 1996-09-24 1998-03-31 Xerox Corporation Digital watermarking using stochastic screen patterns
US5748763A (en) 1993-11-18 1998-05-05 Digimarc Corporation Image steganography system featuring perceptually adaptive and globally scalable signal embedding
EP0859337A2 (en) 1997-02-14 1998-08-19 Nec Corporation Image data encoding system and image inputting apparatus
US5809139A (en) 1996-09-13 1998-09-15 Vivo Software, Inc. Watermarking method and apparatus for compressed digital video
EP0962886A2 (en) 1998-05-12 1999-12-08 Lucent Technologies Inc. Method and system for watermarking in the frequency domain
US6201879B1 (en) * 1996-02-09 2001-03-13 Massachusetts Institute Of Technology Method and apparatus for logo hiding in images
US6226387B1 (en) * 1996-08-30 2001-05-01 Regents Of The University Of Minnesota Method and apparatus for scene-based video watermarking
US6240121B1 (en) * 1997-07-09 2001-05-29 Matsushita Electric Industrial Co., Ltd. Apparatus and method for watermark data insertion and apparatus and method for watermark data detection
US6301368B1 (en) * 1999-01-29 2001-10-09 International Business Machines Corporation System and method for data hiding in compressed fingerprint images

Patent Citations (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5109451A (en) 1988-04-28 1992-04-28 Sharp Kabushiki Kaisha Orthogonal transform coding system for image data
US4910716A (en) 1989-01-31 1990-03-20 Amoco Corporation Suppression of coherent noise in seismic data
US5212742A (en) 1991-05-24 1993-05-18 Apple Computer, Inc. Method and apparatus for encoding/decoding image data
US5721788A (en) 1992-07-31 1998-02-24 Corbis Corporation Method and system for digital image signatures
US5748763A (en) 1993-11-18 1998-05-05 Digimarc Corporation Image steganography system featuring perceptually adaptive and globally scalable signal embedding
US5646997A (en) 1994-12-14 1997-07-08 Barton; James M. Method and apparatus for embedding authentication information within digital data
US5530759A (en) 1995-02-01 1996-06-25 International Business Machines Corporation Color correct digital watermarking of images
US5659726A (en) 1995-02-23 1997-08-19 Sandford, Ii; Maxwell T. Data embedding
US5636292A (en) 1995-05-08 1997-06-03 Digimarc Corporation Steganography methods employing embedded calibration data
US5710834A (en) 1995-05-08 1998-01-20 Digimarc Corporation Method and apparatus responsive to a code signal conveyed through a graphic image
US5636292C1 (en) 1995-05-08 2002-06-18 Digimarc Corp Steganography methods employing embedded calibration data
US5613004A (en) 1995-06-07 1997-03-18 The Dice Company Steganographic method and device
US5687236A (en) 1995-06-07 1997-11-11 The Dice Company Steganographic method and device
US5657390A (en) 1995-08-25 1997-08-12 Netscape Communications Corporation Secure socket layer application program apparatus and method
EP0766468A2 (en) 1995-09-28 1997-04-02 Nec Corporation Method and system for inserting a spread spectrum watermark into multimedia data
US5687191A (en) 1995-12-06 1997-11-11 Solana Technology Development Corporation Post-compression hidden data transport
US6201879B1 (en) * 1996-02-09 2001-03-13 Massachusetts Institute Of Technology Method and apparatus for logo hiding in images
US5689587A (en) 1996-02-09 1997-11-18 Massachusetts Institute Of Technology Method and apparatus for data hiding in images
US5664018A (en) 1996-03-12 1997-09-02 Leighton; Frank Thomson Watermarking process resilient to collusion attacks
US6226387B1 (en) * 1996-08-30 2001-05-01 Regents Of The University Of Minnesota Method and apparatus for scene-based video watermarking
US5809139A (en) 1996-09-13 1998-09-15 Vivo Software, Inc. Watermarking method and apparatus for compressed digital video
US5734752A (en) 1996-09-24 1998-03-31 Xerox Corporation Digital watermarking using stochastic screen patterns
EP0859337A2 (en) 1997-02-14 1998-08-19 Nec Corporation Image data encoding system and image inputting apparatus
US6240121B1 (en) * 1997-07-09 2001-05-29 Matsushita Electric Industrial Co., Ltd. Apparatus and method for watermark data insertion and apparatus and method for watermark data detection
EP0962886A2 (en) 1998-05-12 1999-12-08 Lucent Technologies Inc. Method and system for watermarking in the frequency domain
US6301368B1 (en) * 1999-01-29 2001-10-09 International Business Machines Corporation System and method for data hiding in compressed fingerprint images

Non-Patent Citations (9)

* Cited by examiner, † Cited by third party
Title
Carevic, D et al. "Region-Based Coding of Color Images Using Karhunen-Loeve Transform", Graphical Models and Image Processing, vol. 59, No. 1 pp. 27-38, (1997).
Chou, C.H. et al. "A perceptually tuned sub-band imgae coder based on the measure of Just-Noticeable-Distortion profile". IEEE Trans. on Circuits and Systems for Video Technology, vol. 5, No. 6, pp 467-476, (1995).
Craver, S et al. "Can Invisible Watermarks Resolve Rightful Ownerships?", Technical Report RC, 20509, SPIE, vol. 3022, pp. 310-321 (1996).
Flusser, J. "An adaptive method for image registration", Pattern Recognition, vol. 25, No. 1, pp 45-54, (1992).
Jain, Anil K. "Fundamentals of Digital Image Processing" Prentice Hall, pp. 163-175 (1986).
Petitcolas, Fabien A.P. et al. " Attacks on Copyright Marking systems". IEEE Workshop on Information Hiding, pp. 218-238, (1998).
Torres, L et al. "Video Coding". Kluwer Academic Publishers, pp. 411-417, (1996).
Turk, M et al. "Eigenfaces for Recognition" Journal of Cognitive Neuroscience, vol. 3, No. 1, pp 71-86, (1991).
Wolfram Szepanski, "A signal theoretic method for creating forgery proof documents for automatic verification", 1979 Carnahan on Crime Countermeasures, pp. 101-109.* *

Cited By (156)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030033530A1 (en) * 1996-05-16 2003-02-13 Sharma Ravi K. Variable message coding protocols for encoding auxiliary data in media signals
US7930546B2 (en) * 1996-05-16 2011-04-19 Digimarc Corporation Methods, systems, and sub-combinations useful in media identification
US20090060264A1 (en) * 1996-05-16 2009-03-05 Sharma Ravi K Variable Message Coding Protocols for Encoding Auxiliary Data in Media Signals
US7412072B2 (en) * 1996-05-16 2008-08-12 Digimarc Corporation Variable message coding protocols for encoding auxiliary data in media signals
US7778442B2 (en) 1996-05-16 2010-08-17 Digimarc Corporation Variable message coding protocols for encoding auxiliary data in media signals
US8094877B2 (en) 1996-05-16 2012-01-10 Digimarc Corporation Variable message coding protocols for encoding auxiliary data in media signals
US20070174059A1 (en) * 1996-05-16 2007-07-26 Rhoads Geoffrey B Methods, Systems, and Sub-Combinations Useful in Media Identification
US20110081041A1 (en) * 1996-05-16 2011-04-07 Sharma Ravi K Variable Message Coding Protocols For Encoding Auxiliary Data in Media Signals
US9330722B2 (en) 1997-05-16 2016-05-03 The Trustees Of Columbia University In The City Of New York Methods and architecture for indexing and editing compressed video over the world wide web
US8745404B2 (en) 1998-05-28 2014-06-03 Verance Corporation Pre-processed information embedding system
US9117270B2 (en) 1998-05-28 2015-08-25 Verance Corporation Pre-processed information embedding system
US20070245400A1 (en) * 1998-11-06 2007-10-18 Seungyup Paek Video description system and method
US8370869B2 (en) 1998-11-06 2013-02-05 The Trustees Of Columbia University In The City Of New York Video description system and method
US6959101B2 (en) * 1998-12-02 2005-10-25 Hitachi, Ltd. Method of judging digital watermark information
US20040120546A1 (en) * 1998-12-02 2004-06-24 Hiroshi Yoshiura Method of judging digital watermark information
US8719893B2 (en) 1999-03-15 2014-05-06 Sony Corporation Secure module and a method for providing a dedicated on-site media service
US20050172125A1 (en) * 1999-03-31 2005-08-04 Canon Kabushiki Kaisha Information processing system, information processing apparatus, and computer-readable recording medium
US6922777B1 (en) * 1999-03-31 2005-07-26 Canon Kabushiki Kaisha Apparatus, computer medium, and method for embedding and extracting a first and second data into a watermark
US7200231B2 (en) 1999-03-31 2007-04-03 Canon Kabushiki Kaisha Information processing system, information processing apparatus, and computer-readable recording medium
US6909784B1 (en) * 1999-06-25 2005-06-21 Victor Company Of Japan, Ltd. Recording apparatus and recording method of electronic watermark
US7058570B1 (en) * 2000-02-10 2006-06-06 Matsushita Electric Industrial Co., Ltd. Computer-implemented method and apparatus for audio data hiding
US8791789B2 (en) 2000-02-16 2014-07-29 Verance Corporation Remote control signaling using audio watermarks
US8451086B2 (en) 2000-02-16 2013-05-28 Verance Corporation Remote control signaling using audio watermarks
US9189955B2 (en) 2000-02-16 2015-11-17 Verance Corporation Remote control signaling using audio watermarks
US20010054150A1 (en) * 2000-03-18 2001-12-20 Levy Kenneth L. Watermark embedding functions in rendering description files
US7142691B2 (en) * 2000-03-18 2006-11-28 Digimarc Corporation Watermark embedding functions in rendering description files
US20020038294A1 (en) * 2000-06-16 2002-03-28 Masakazu Matsugu Apparatus and method for detecting or recognizing pattern by employing a plurality of feature detecting elements
US7054850B2 (en) * 2000-06-16 2006-05-30 Canon Kabushiki Kaisha Apparatus and method for detecting or recognizing pattern by employing a plurality of feature detecting elements
US7253919B2 (en) * 2000-11-30 2007-08-07 Ricoh Co., Ltd. Printer with embedded retrieval and publishing interface
US20050024682A1 (en) * 2000-11-30 2005-02-03 Hull Jonathan J. Printer with embedded retrieval and publishing interface
US20080037043A1 (en) * 2000-11-30 2008-02-14 Ricoh Co., Ltd. Printer With Embedded Retrieval and Publishing Interface
US20020094082A1 (en) * 2000-12-13 2002-07-18 Eastman Kodak Company Method and system for embedding message data in a digital image sequence
US6870931B2 (en) * 2000-12-13 2005-03-22 Eastman Kodak Company Method and system for embedding message data in a digital image sequence
US20030147547A1 (en) * 2001-01-10 2003-08-07 Ching-Yung Lin Method and apparatus for watermarking images
US6879703B2 (en) * 2001-01-10 2005-04-12 Trustees Of Columbia University Of The City Of New York Method and apparatus for watermarking images
US20020114458A1 (en) * 2001-02-05 2002-08-22 Belenko Vyacheslav S. Copy protection method for digital media
US7506376B2 (en) 2001-02-05 2009-03-17 Lg Electronics Inc. Copy protection method for digital media
US20020118838A1 (en) * 2001-02-05 2002-08-29 Belenko Vyacheslav S. Copy protection method and system for digital media
US7509682B2 (en) 2001-02-05 2009-03-24 Lg Electronics Inc. Copy protection method and system for digital media
US7760904B2 (en) 2001-02-21 2010-07-20 Lg Electronics Inc. Proprietary watermark system for secure digital media and content distribution
US20020114459A1 (en) * 2001-02-21 2002-08-22 Belenko Vyacheslav S. Watermark-based copy management system and method for digital media copy protection
US7434056B2 (en) 2001-02-21 2008-10-07 Lg Electronics Inc. Watermark-based copy management system and method for digital media copy protection
US7137005B2 (en) * 2001-03-28 2006-11-14 Lg Electronics Inc. Method of watermarking digital data
US20020184502A1 (en) * 2001-03-28 2002-12-05 Sall M. A. Method of watermarking digital data
US7697717B2 (en) 2001-03-28 2010-04-13 Lg Electronics Inc. Method of embedding watermark into digital image
US7877813B2 (en) 2001-05-11 2011-01-25 Lg Electronics Inc. Copy protection method and system for digital media
US7298865B2 (en) * 2001-07-30 2007-11-20 Sarnoff Corporation Secure robust high-fidelity watermarking
US20030021439A1 (en) * 2001-07-30 2003-01-30 Jeffrey Lubin Secure robust high-fidelity watermarking
US8515773B2 (en) 2001-08-01 2013-08-20 Sony Corporation System and method for enabling distribution and brokering of content information
US20050008221A1 (en) * 2001-11-19 2005-01-13 Hull Jonathan J. Printing system with embedded audio/video content recognition and processing
US20050034057A1 (en) * 2001-11-19 2005-02-10 Hull Jonathan J. Printer with audio/video localization
US20050010409A1 (en) * 2001-11-19 2005-01-13 Hull Jonathan J. Printable representations for time-based media
US20050005760A1 (en) * 2001-11-19 2005-01-13 Hull Jonathan J. Music processing printer
US7415670B2 (en) 2001-11-19 2008-08-19 Ricoh Co., Ltd. Printer with audio/video localization
US7747655B2 (en) 2001-11-19 2010-06-29 Ricoh Co. Ltd. Printable representations for time-based media
US7861169B2 (en) 2001-11-19 2010-12-28 Ricoh Co. Ltd. Multimedia print driver dialog interfaces
US20040181747A1 (en) * 2001-11-19 2004-09-16 Hull Jonathan J. Multimedia print driver dialog interfaces
US20040181815A1 (en) * 2001-11-19 2004-09-16 Hull Jonathan J. Printer with radio or television program extraction and formating
US7314994B2 (en) 2001-11-19 2008-01-01 Ricoh Company, Ltd. Music processing printer
US20080303942A1 (en) * 2001-12-06 2008-12-11 Shih-Fu Chang System and method for extracting text captions from video and generating video summaries
US8488682B2 (en) 2001-12-06 2013-07-16 The Trustees Of Columbia University In The City Of New York System and method for extracting text captions from video and generating video summaries
US7337321B2 (en) * 2002-04-05 2008-02-26 Hitachi, Ltd. Digital content protection using invisible and visible watermarks
US20030191941A1 (en) * 2002-04-05 2003-10-09 Koichi Terada Data transmitting apparatus, contents distribution apparatus, contents distribution system, digital contents distribution method and data storage apparatus
US20090316778A1 (en) * 2002-04-26 2009-12-24 Jae-Gon Kim Method And System For Optimal Video Transcoding Based On Utility Function Descriptors
US20040009763A1 (en) * 2002-06-20 2004-01-15 Stone Chris L. Secure tracking system and method for video program content
US8601504B2 (en) 2002-06-20 2013-12-03 Verance Corporation Secure tracking system and method for video program content
US20050271246A1 (en) * 2002-07-10 2005-12-08 Sharma Ravi K Watermark payload encryption methods and systems
US20040073916A1 (en) * 2002-10-15 2004-04-15 Verance Corporation Media monitoring, management and information system
US8806517B2 (en) 2002-10-15 2014-08-12 Verance Corporation Media monitoring, management and information system
US7788684B2 (en) 2002-10-15 2010-08-31 Verance Corporation Media monitoring, management and information system
US9648282B2 (en) 2002-10-15 2017-05-09 Verance Corporation Media monitoring, management and information system
US7177441B2 (en) * 2002-12-09 2007-02-13 International Business Machines Corporation System and method for secret communication
US20040109583A1 (en) * 2002-12-09 2004-06-10 Condon John B. System and method for secret communication
US20060055982A1 (en) * 2003-01-14 2006-03-16 Lokhoff Gerardus C P Watermarking film prints
US7756288B2 (en) 2003-05-29 2010-07-13 Jeffrey Lubin Method and apparatus for analog insertion of low frequency watermarks
US20050036174A1 (en) * 2003-06-19 2005-02-17 Jeffrey Lubin Method and apparatus for providing a noise estimation for automatic selection of dither patterns in low frequency watermarks
US20050068571A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Stand alone multimedia printer with user interface for allocating processing
US20090092322A1 (en) * 2003-09-25 2009-04-09 Berna Erol Semantic Classification and Enhancement Processing of Images for Printing Applications
US20050068572A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Printer with hardware and software interfaces for media devices
US20050068567A1 (en) * 2003-09-25 2005-03-31 Hull Jonathan J. Printer with audio or video receiver, recorder, and real-time content-based processing logic
US20050068573A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Networked printing system having embedded functionality for printing time-based media
US8373905B2 (en) 2003-09-25 2013-02-12 Ricoh Co., Ltd. Semantic classification and enhancement processing of images for printing applications
US20050068569A1 (en) * 2003-09-25 2005-03-31 Hull Jonathan J. Printer with document-triggered processing
US8077341B2 (en) 2003-09-25 2011-12-13 Ricoh Co., Ltd. Printer with audio or video receiver, recorder, and real-time content-based processing logic
US7511846B2 (en) 2003-09-25 2009-03-31 Ricoh Co., Ltd. Printer having embedded functionality for printing time-based media
US20060256388A1 (en) * 2003-09-25 2006-11-16 Berna Erol Semantic classification and enhancement processing of images for printing applications
US20050071763A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Stand alone multimedia printer capable of sharing media processing tasks
US20050068581A1 (en) * 2003-09-25 2005-03-31 Hull Jonathan J. Printer with multimedia server
US20050071520A1 (en) * 2003-09-25 2005-03-31 Hull Jonathan J. Printer with hardware and software interfaces for peripheral devices
US20050071746A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Networked printer with hardware and software interfaces for peripheral devices
US20050068570A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Printer user interface
US20050071519A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Stand alone printer with hardware / software interfaces for sharing multimedia processing
US20050069362A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. Printer having embedded functionality for printing time-based media
US20050068568A1 (en) * 2003-09-25 2005-03-31 Hart Peter E. User interface for networked printer
US7864352B2 (en) 2003-09-25 2011-01-04 Ricoh Co. Ltd. Printer with multimedia server
US20080002854A1 (en) * 2003-10-08 2008-01-03 Verance Corporation Signal continuity assessment using embedded watermarks
US9055239B2 (en) 2003-10-08 2015-06-09 Verance Corporation Signal continuity assessment using embedded watermarks
US8127137B2 (en) 2004-03-18 2012-02-28 Digimarc Corporation Watermark payload encryption for media including multiple watermarks
US20050262351A1 (en) * 2004-03-18 2005-11-24 Levy Kenneth L Watermark payload encryption for media including multiple watermarks
US8274666B2 (en) 2004-03-30 2012-09-25 Ricoh Co., Ltd. Projector/printer for displaying or printing of documents
US20050231739A1 (en) * 2004-03-30 2005-10-20 Dar-Shyang Lee Projector/printer for displaying or printing of documents
US7644146B2 (en) 2004-06-02 2010-01-05 Hewlett-Packard Development Company, L.P. System and method for discovering communities in networks
US20060075441A1 (en) * 2004-10-06 2006-04-06 Sony Corporation Method and system for a personal video recorder comprising multiple removable storage/tuner units
US20060074807A1 (en) * 2004-10-06 2006-04-06 Sony Corporation Method and system for content sharing and authentication between multiple devices
US8768844B2 (en) 2004-10-06 2014-07-01 Sony Corporation Method and system for content sharing and authentication between multiple devices
US20080181308A1 (en) * 2005-03-04 2008-07-31 Yong Wang System and method for motion estimation and mode decision for low-complexity h.264 decoder
US9060175B2 (en) 2005-03-04 2015-06-16 The Trustees Of Columbia University In The City Of New York System and method for motion estimation and mode decision for low-complexity H.264 decoder
US20060212705A1 (en) * 2005-03-17 2006-09-21 Interdigital Technology Corporation Wireless communication method and apparatus for generating, watermarking and securely transmitting content
US7599515B2 (en) * 2005-03-17 2009-10-06 Interdigital Technology Corporation Wireless communication method and apparatus for generating, watermarking and securely transmitting content
US8538066B2 (en) 2005-04-26 2013-09-17 Verance Corporation Asymmetric watermark embedding/extraction
US8811655B2 (en) 2005-04-26 2014-08-19 Verance Corporation Circumvention of watermark analysis in a host content
US8005258B2 (en) 2005-04-26 2011-08-23 Verance Corporation Methods and apparatus for enhancing the robustness of watermark extraction from digital host content
US8340348B2 (en) 2005-04-26 2012-12-25 Verance Corporation Methods and apparatus for thwarting watermark detection circumvention
US9153006B2 (en) 2005-04-26 2015-10-06 Verance Corporation Circumvention of watermark analysis in a host content
US8280103B2 (en) 2005-04-26 2012-10-02 Verance Corporation System reactions to the detection of embedded watermarks in a digital host content
US9009482B2 (en) 2005-07-01 2015-04-14 Verance Corporation Forensic marking using a common customization function
US8549307B2 (en) 2005-07-01 2013-10-01 Verance Corporation Forensic marking using a common customization function
US8781967B2 (en) 2005-07-07 2014-07-15 Verance Corporation Watermarking in an encrypted domain
US20080148405A1 (en) * 2006-12-18 2008-06-19 Carol Davids Method for securing streaming multimedia network transmissions
US8453241B2 (en) 2006-12-18 2013-05-28 Illinois Institute Of Technology Method for securing streaming multimedia network transmissions
US20090070877A1 (en) * 2006-12-18 2009-03-12 Carol Davids Method for securing streaming multimedia network transmissions
US20080234069A1 (en) * 2007-03-23 2008-09-25 Acushnet Company Functionalized, Crosslinked, Rubber Nanoparticles for Use in Golf Ball Castable Thermoset Layers
US20080304755A1 (en) * 2007-06-08 2008-12-11 Microsoft Corporation Face Annotation Framework With Partial Clustering And Interactive Labeling
US8014572B2 (en) 2007-06-08 2011-09-06 Microsoft Corporation Face annotation framework with partial clustering and interactive labeling
US20110025710A1 (en) * 2008-04-10 2011-02-03 The Trustees Of Columbia University In The City Of New York Systems and methods for image archeology
US8849058B2 (en) 2008-04-10 2014-09-30 The Trustees Of Columbia University In The City Of New York Systems and methods for image archaeology
US8364673B2 (en) 2008-06-17 2013-01-29 The Trustees Of Columbia University In The City Of New York System and method for dynamically and interactively searching media data
US20110145232A1 (en) * 2008-06-17 2011-06-16 The Trustees Of Columbia University In The City Of New York System and method for dynamically and interactively searching media data
US8259938B2 (en) 2008-06-24 2012-09-04 Verance Corporation Efficient and secure forensic marking in compressed
US8681978B2 (en) 2008-06-24 2014-03-25 Verance Corporation Efficient and secure forensic marking in compressed domain
US8346567B2 (en) 2008-06-24 2013-01-01 Verance Corporation Efficient and secure forensic marking in compressed domain
US20100153739A1 (en) * 2008-12-15 2010-06-17 Guymon Jr John Lewis Securable independent electronic document
US8671069B2 (en) 2008-12-22 2014-03-11 The Trustees Of Columbia University, In The City Of New York Rapid image annotation via brain state decoding and visual pattern mining
US9665824B2 (en) 2008-12-22 2017-05-30 The Trustees Of Columbia University In The City Of New York Rapid image annotation via brain state decoding and visual pattern mining
US20100310134A1 (en) * 2009-06-08 2010-12-09 Microsoft Corporation Assisted face recognition tagging
US8325999B2 (en) 2009-06-08 2012-12-04 Microsoft Corporation Assisted face recognition tagging
US9786052B2 (en) * 2010-03-29 2017-10-10 Sony Corporation Image processing apparatus and method for evaluating objects in an image
US20130070971A1 (en) * 2010-03-29 2013-03-21 Sony Corporation Data processing apparatus, data processing method, image processing apparatus and method, and program
US8838978B2 (en) 2010-09-16 2014-09-16 Verance Corporation Content access management using extracted watermark information
US8838977B2 (en) 2010-09-16 2014-09-16 Verance Corporation Watermark extraction and content screening in a networked environment
US9607131B2 (en) 2010-09-16 2017-03-28 Verance Corporation Secure and efficient content screening in a networked environment
US8923548B2 (en) 2011-11-03 2014-12-30 Verance Corporation Extraction of embedded watermarks from a host content using a plurality of tentative watermarks
US8682026B2 (en) 2011-11-03 2014-03-25 Verance Corporation Efficient extraction of embedded watermarks in the presence of host content distortions
US8615104B2 (en) 2011-11-03 2013-12-24 Verance Corporation Watermark extraction based on tentative watermarks
US8533481B2 (en) 2011-11-03 2013-09-10 Verance Corporation Extraction of embedded watermarks from a host content based on extrapolation techniques
US8745403B2 (en) 2011-11-23 2014-06-03 Verance Corporation Enhanced content management based on watermark extraction records
US9323902B2 (en) 2011-12-13 2016-04-26 Verance Corporation Conditional access using embedded watermarks
US9547753B2 (en) 2011-12-13 2017-01-17 Verance Corporation Coordinated watermarking
US9571606B2 (en) 2012-08-31 2017-02-14 Verance Corporation Social media viewing system
US8869222B2 (en) 2012-09-13 2014-10-21 Verance Corporation Second screen content
US8726304B2 (en) 2012-09-13 2014-05-13 Verance Corporation Time varying evaluation of multimedia content
US9106964B2 (en) 2012-09-13 2015-08-11 Verance Corporation Enhanced content distribution using advertisements
US9262794B2 (en) 2013-03-14 2016-02-16 Verance Corporation Transactional video marking system
US9251549B2 (en) 2013-07-23 2016-02-02 Verance Corporation Watermark extractor enhancements based on payload ranking
US9208334B2 (en) 2013-10-25 2015-12-08 Verance Corporation Content management using multiple abstraction layers
US9596521B2 (en) 2014-03-13 2017-03-14 Verance Corporation Interactive content acquisition using embedded codes

Also Published As

Publication number Publication date Type
GB2366112B (en) 2003-05-28 grant
GB2366112A (en) 2002-02-27 application
GB0115233D0 (en) 2001-08-15 grant
WO2000039954A1 (en) 2000-07-06 application

Similar Documents

Publication Publication Date Title
Nikolaidis et al. Robust image watermarking in the spatial domain
Barni et al. Improved wavelet-based watermarking through pixel-wise masking
Linnartz et al. Analysis of the sensitivity attack against electronic watermarks in images
Dittmann et al. Content-based digital signature for motion pictures authentication and content-fragile watermarking
Swanson et al. Multiresolution scene-based video watermarking using perceptual models
Miller et al. A review of watermarking principles and practices
Zhao et al. Forensic analysis of nonlinear collusion attacks for multimedia fingerprinting
US6625295B1 (en) Authentication of signals using watermarks
Jain et al. Hiding biometric data
Kundur et al. Digital watermarking for telltale tamper proofing and authentication
US5664018A (en) Watermarking process resilient to collusion attacks
US6430301B1 (en) Formation and analysis of signals with common and transaction watermarks
US20020172394A1 (en) Robust and stealthy video watermarking
Voyatzis et al. The use of watermarks in the protection of digital multimedia products
Singh et al. A survey of digital watermarking techniques, applications and attacks
US20010024510A1 (en) Digital content processing method
Zheng et al. A survey of RST invariant image watermarking algorithms
US20020009209A1 (en) Digital information embedding/extracting apparatus and method, and medium including a program for executing the method recorded thereon
Popescu et al. Exposing digital forgeries by detecting traces of resampling
US20020059520A1 (en) Informarion processing apparatus, method for controlling the same, and storage medium
US20040039914A1 (en) Layered security in digital watermarking
US6944313B1 (en) Method and device for inserting and decoding a watermark in digital data
Cox et al. A secure, robust watermark for multimedia
US6792129B1 (en) Digital watermarking in a perceptually uniform domain
US20060212704A1 (en) Forensic for fingerprint detection in multimedia

Legal Events

Date Code Title Description
AS Assignment

Owner name: KENT RIDGE DIGITAL LABS, SINGAPORE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:WU, JIANKANG;REEL/FRAME:010558/0273

Effective date: 19990506

Owner name: KENT RIDGE DIGITAL LABS, SINGAPORE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:DENG, HUIJIE ROBERT;REEL/FRAME:010530/0283

Effective date: 19990405

Owner name: KENT RIDGE DIGITAL LABS, SINGAPORE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SUN QIBIN;WU, JIANKANG;DENG, HUIJIE ROBERT;REEL/FRAME:010530/0276

Effective date: 19990503

FPAY Fee payment

Year of fee payment: 4

REMI Maintenance fee reminder mailed
LAPS Lapse for failure to pay maintenance fees
FP Expired due to failure to pay maintenance fee

Effective date: 20120113