Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
  • H04N19/51—Motion estimation or motion compensation
  • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T1/00—General purpose image data processing
  • G06T1/0021—Image watermarking
  • G06T1/0028—Adaptive watermarking, e.g. Human Visual System [HVS]-based watermarking
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N1/32101—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N1/32144—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
  • H04N1/32149—Methods relating to embedding, encoding, decoding, detection or retrieval operations
  • H04N1/32154—Transform domain methods
  • H04N1/32165—Transform domain methods using cosine transforms
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N1/32101—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N1/32144—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
  • H04N1/32149—Methods relating to embedding, encoding, decoding, detection or retrieval operations
  • H04N1/32154—Transform domain methods
  • H04N1/32187—Transform domain methods with selective or adaptive application of the additional information, e.g. in selected frequency coefficients
  • H04N1/32192—Transform domain methods with selective or adaptive application of the additional information, e.g. in selected frequency coefficients according to calculated or estimated visibility of the additional information in the image
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/46—Embedding additional information in the video signal during the compression process
  • H04N19/467—Embedding additional information in the video signal during the compression process characterised by the embedded information being invisible, e.g. watermarking
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2201/00—General purpose image data processing
  • G06T2201/005—Image watermarking
  • G06T2201/0052—Embedding of the watermark in the frequency domain
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2201/00—General purpose image data processing
  • G06T2201/005—Image watermarking
  • G06T2201/0053—Embedding of the watermark in the coding stream, possibly without decoding; Embedding of the watermark in the compressed domain
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20048—Transform domain processing
  • G06T2207/20052—Discrete cosine transform [DCT]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
  • H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N2201/3225—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document
  • H04N2201/3233—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document of authentication information, e.g. digital signature, watermark
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
  • H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N2201/3269—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of machine readable codes or marks, e.g. bar codes or glyphs
  • H04N2201/327—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of machine readable codes or marks, e.g. bar codes or glyphs which are undetectable to the naked eye, e.g. embedded codes

Definitions

• This invention relates to providing digital image data with a watermark, and, more particularly, where the image data are video data.
• a conventional watermark, on a paper document may consist of a translucent design which is visible when the document is held to the light. Or, more generally, a watermark may be viewed under certain lighting conditions or at certain viewing angles. Such watermarks, which are difficult to forge, can be included for the sake of authentication of documents such as bank notes, checks and stock certificates, for example.
• watermarks are being used to betoken certain proprietary rights such as a copyright, for example.
• the watermark is a visible or invisible pattern which is superposed on an image, and which is not readily removable without leaving evidence of tampering. Resistance to tampering is called "robustness" .
• a luminance level, ⁇ L is selected for the strength of the watermark, and the luminance of each individual pixel of the image is modified by ⁇ L and a nonlinear function. For increased security, the level ⁇ L is randomized over all the pixels in the image.
• DCT discrete cosine transformation
• Fig. 1 is an illustration for motion-compensated discrete cosine transformation (MC-DCT) .
• Fig. 2a is a watermark mask.
• Fig. 2b is an original image.
• Fig. 2c is a superposition of the original image and the watermark mask.
• Fig. 3 is a flow diagram of initial processing.
• Fig. 4 is a flow diagram of watermark superposition processing.
• Fig. 5 is a flow diagram of scaling for a region.
• a Mask Generation Module generates a DCT watermark mask based on the original video content.
• a Motion Compensation Module efficiently inserts the watermark in the DCT domain and outputs a valid video bitstream at specified bitrate. The following description applies specifically to image data in MPEG format.
• MPEG video consists of groups of pictures (GOP) as described in document ISO/IEC 13818 - 2 Committee Draft
• Each GOP starts with an intra coded "I-frame”, followed by a number of forward-referencing “P-frames” and bidirectionally-referencing “B- frames” .
• the image content changes from frame to frame.
• the watermark must be adapted to the video contents. For example, when an image is complicated or "busy", i.e., when it has many high-frequency components, the watermark should be stronger. For different regions in the same video frame, the watermark should be scaled regionally— thereby enhancing the security against tampering.
• a watermark mask image is first generated for each GOP, or for the first P-frame after a scene cut.
• the input watermark image is first converted to a gray scale image. Only the luminance channel of each image is modified. A transparent color (background color) is chosen. The luminance of all watermark pixels having the transparent color value is set to 0.
• the mask image is randomly shifted in both x- and y- direction.
• a DCT is applied to obtain the DCT mask of the watermark. The luminance of the mask will be scaled adaptively according to the input image content before adding to the input image.
• the following formulae have been proposed in the above-referenced report by G. W. Braudaway et al . :
• w nra ' nm • y w /38.667 • (y ⁇ ly 3 • ⁇ L for y nra /y w > 0.008856,
• m ' w nm - y w /903.3 • ⁇ L for y nm /y w ⁇ 0.008856 (1)
• w nra ' is the scaled watermark mask that will be added to the original image
• w nm is the non-transparent watermark pixel value at pixel (n,m)
• y w is the scene white
• y nm is the luminance value of the input image at image coordinates (n,m)
• ⁇ L is the scaling factor which controls the watermark strength.
• Equation (2) Equation (2)
• E [y 23 ] is a function of the mean and the variance of the pixel values .
• Equation (2) specifies a relationship between the moments of random variables w, w' and y. This relationship can be extended to the deterministic case to simplify Equation (2) , resulting in a linear approximation.
• Equation 3 and the mean ⁇ is used to approximate y in deciding which of the formulae to use in Equation 2.
• w- y ⁇ 0.1607 • w (/4 /( ⁇ . p 2 ) ⁇ L , o> 17.9319 ,
• Equation 4 approximates the nonlinear function according to Equation 2, by linear functions block by block.
• the scaled watermark strength depends on the mean and variance of the image block. For each image block, the higher the mean (i.e. the brighter), and the higher the variance (i.e. the more cluttered), the greater the required strength of the watermark for maintaining consistent visibility of the watermark.
• the DCT of Equation 4 can be used to obtain the DCT of the watermark mask, which can be inserted in the image in the DCT domain.
• the mean and variance of the input image can be derived from the DCT coefficients, ⁇ « (Y DC %) and
• a new watermark mask is calculated for each I -frame and P-frame, the latter in case of a scene cut.
• I- frames all DCT coefficients are readily accessible after minimal decoding of the MPEG sequence, i.e. inverse variable length coding, inverse run length coding and inverse quantization.
• P-frames since most blocks are in the scene cut, these DCT coefficient can be used immediately.
• non-intra coded blocks the average DC and AC energy obtained from intra coded blocks are substituted.
• the block-based ( ⁇ . ⁇ .-) pair can be replaced by the average ( ⁇ ,T3) over the whole image or over certain regions.
• ⁇ ,T3 the average
• the input image can be separated into many rectangular regions. As illustrated by Fig. 5, for each region an ( ⁇ ,T3) pair is calculated, and the mask is generated accordingly. Typically, the watermark is divided into top and bottom regions. This is suitable for most outdoor views with sky in the upper half of the frame and darker scenery in the lower half, as shown in Fig. 2a, for example. Each region will have a relatively visible watermark using different ( ,T3) pairs.
• a randomized location shift can be applied to the watermark image before applying the DCT. This makes removal of the watermark more difficult for attackers who are in possession of the original watermark image, e.g. when a known logo is used for watermark purposes.
• Sub-pixel randomized location shifting will make it very difficult for the attacker to remove the watermark without leaving some error residue.
• the following can be used for shifting. Two random numbers, for x- and y-direction, respectively, are generated and normalized to lie between -1.00 to 1.00.
• sub-pixel shifting is effected by bi-linear interpolation which involves only linear scaling and addition.
• a similar bi-linear operation can be used.
• the DCT blocks of the watermark are inserted into the DCT frames of the input video in one of three ways, as illustrated by Fig. 4, Section (ii) .
• the DCT of the scaled watermark is added directly:
• E'-. is the i,j-th resulting DCT block, E-- the original DCT block, and W'.. the scaled watermark DCT according to Equation 6.
• the watermark added in the anchor frame has to be removed when adding the current watermark.
• the resulting DCT error residue is :
• E' 13 E.- - MCDCT (W' p ,Vp.- )+ W'-_ (8)
• MCDCT is the motion compensation function in the DCT domain as described in the above-referenced paper by S.-F. Chang et al .
• W' F is the watermark DCT used in the forward anchor frame
• V Fl - is the forward motion vector, as shown in Fig . 1.
• E' 1D E ⁇ : - (MCDCT (W' F ,V P ⁇ . )+ MCDCT(W' B ,V B- .))/2 + W .. (9)
• V F and V B are forward and backward motion vector, respectively, as shown in Fig. 1.
• skipped blocks which are the 0-motion, 0-residue error blocks in B- and P-frames, no operations are necessary, as the watermark inserted in the anchor frame will be carried over.
• Motion vector selection setting the motion vector of a micro-block in P-frame to 0 when the error residue from using motion compensation of this motion vector is larger than without its use.
• Figs. 2a, 2b and 2c illustrate the use of the adaptive watermarking techniques.
• Fig. 2a shows the original watermark mask. While a binary version is shown here, the algorithm is capable of handling gray scale with any specified transparent color.
• Fig. 2b shows an original image.
• Fig. 2c shows the new watermarked image.
• the watermarking algorithm was tested on a HP J210 workstation, achieving a rate of 6 frames/second. Most of the computational effort went into the MC-DCT operations. If all possible MC-DCT blocks were precomputed, real time performance would be possible. This would require 12 megabytes of memory for 352x240 image size.
• preferred watermarks offer robustness in that they are not easily defeated or removed by tampering. For example, if a watermark is inserted in MPEG video by the method described above, it would be necessary to recover the watermark mask, estimate the embedding locations by extensive sub-pixel block matching, and then estimate the ( , " ⁇ ) factors for each watermark region. In experiments, there always remained noticeable traces in the tampered video, which can be used to reject false claims of ownership and to deter piracy.
• a watermark should not be binary, but should have texture which is similar to that of the scene on which it is placed. This can be accomplished by arbitrarily choosing an I -frame from the scene, decoding it by inverse DCT transform to obtain pixel values, and masking out the watermark from the decoded video frame .
• an inserted watermark may be defeated by applying video mosaicing, i.e. by assembling a large image from small portions of multiple image frames. The watermark then can be filtered out as outlier.
• video mosaicing i.e. by assembling a large image from small portions of multiple image frames.
• the watermark then can be filtered out as outlier.
• this technique will fail when there are actually moving objects in the foreground, as the watermark will be embedded in the moving foreground objects as well.
• a watermark can be used which appears static relative to over-all or background motion.
• the affine model can be described as
• the motion vectors in MPEG are usually generated by block matching: finding a block in the reference frame so that the mean square error is minimized. Although the motion vectors do not represent the true optical flow, it is still good in most cases to estimate the camera parameters in sequences that do not contain large dark or uniform regions.
• Dominant motion can be estimated using clustering as follows : For each B- or P-frame, obtain the forward motion vectors .

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Image Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)
• Editing Of Facsimile Originals (AREA)

Applications Claiming Priority (3)

Publications (1)

Family

ID=22049688

Family Applications (1)

Country Status (5)

Families Citing this family (6)

Family Cites Families (3)

• 1998
  • 1998-10-27 WO PCT/US1998/022790 patent/WO1999022480A1/en not_active Application Discontinuation
  • 1998-10-27 EP EP98956257A patent/EP1034635A1/de active Pending
  • 1998-10-27 JP JP2000518471A patent/JP2001522165A/ja active Pending
  • 1998-10-27 KR KR1020007004567A patent/KR20010031526A/ko not_active Application Discontinuation
  • 1998-10-27 CA CA002308402A patent/CA2308402A1/en not_active Abandoned

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events