JP2001522165A - Watermarking of digital image data - Google Patents

Abstract

  (57) [Summary] The digital watermark serves to represent the copyright owner of the digitized video. When transferring a video image to be transformed by a discrete cosine transform (DCT) with or without motion compensation, it is preferred to have a watermark after the transformation. To this end, a DCT watermark is generated for optimal visibility based on the original image data, and the generated watermark is superimposed on the transformed data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001] Technical Field The present invention is related to providing a watermark on a digital image, more particularly, the image data is related to the case of the video data.

[0002] customary watermark in the documents of the background paper of the invention, consists of a semi-transparent design visible when held up this document to light. That is, more generally, the watermark can be viewed under certain lighting conditions or at a certain viewing angle. Such watermarks that are difficult to counterfeit can be included for authentication of documents such as, for example, banknotes, checks, and registered stock certificates.

In digital video technology, watermarks have been used to indicate certain proprietary rights, such as copyright. Here, the watermark is a visible or invisible pattern that is superimposed on the image and cannot be easily separated without leaving evidence of tampering. The resistance to tampering is called "robustness."

[0004] One robust method of including visible watermarks in digitized images is described in Braudaway et al., "Protecting Publicly Available Images with Visible Image Watermarking," IBM Research, T.M. J. Watson Research Center, Technical Report 96A
0002248. A luminance level ΔL is selected for the strength of the watermark, and the luminance of each pixel of the image is changed by the ΔL and a non-linear function. To enhance security, the level ΔL is random for all pixels in the image.

SUMMARY OF THE INVENTION A discrete cosine transform (DCT) is used to compress a summary image with or without motion compensation.
It is advantageous to include a watermark after transmission when transmitting after conversion.
For this purpose, (i) a DCT watermark is generated with the highest visibility based on the original image data, and (ii) the generated watermark is superimposed on the transformed data.

DETAILED DESCRIPTION A mask generation module generates a DCT watermark mask based on original video content. The motion compensation module efficiently inserts the watermark into the DCT domain and outputs a valid video bitstream at a specified bit rate. The following description is particularly applicable to image data in the MPEG format.

[0007] MPEG video is described in the draft document ISO / IEC 13818-2 Committee (MPE
It consists of a group of pictures (GOP) as described in G-2). Each GO
P starts with an intra-coded "I frame" followed by a number of forward prediction "P frames" and bidirectional prediction "B frames".

For motion compensation, when inserting a watermark into the I-frame, the P and B
Also change the frame. For such corrections, motion compensation in the watermark in anchor or base frames must be removed when adding the watermark to the current frame. Regarding such removal, F. Chan
g) Others, such as those described in "Manipulation and Compositing of MC-DCT Compressed Video," IEEE Journal of Selected Areas in Communications, Special Issue on Intelligent Signal Processing, pages 1-11, January 1995. , The technique of motion compensation in the DCT domain can be used.

In a video sequence, the image includes a change from frame to frame. Therefore, in order to keep the watermark slightly visible throughout the video, the watermark must be adapted to the video content. For example, if the image is complex, ie "jumbled", i.e. if the image has many high frequency components, the watermark should be stronger. For different regions of the same video frame, the watermark should be scaled up or down depending on the region, thereby increasing security against tampering.

(I) Mask Generation Module In this module, a watermark mask image is generated first for each GOP or for the first P frame after a scene cut, as shown by section (i) in FIG. . This is based on the fact that video content tends to remain unchanged in a GOP, typically on the order of 15 frames or 0.5 seconds. However, if there is a scene cut in the GOP, the visual content changes significantly, using a new mask and adapting to this new visual content. In this way, the watermark is superimposed on the I frame or the first P frame after the scene cut.

To generate the mask, the input watermark image is first converted to a grayscale image, as shown by FIG. Only the luminance channel of each image is changed. Select a transparent color (background color). The brightness of all watermark pixels having this transparent color value is set to zero. Optionally, the mask image is randomly shifted in both x and y directions. Using DCT, a DCT mask for the watermark is obtained. The brightness of the mask is adaptively selected according to the input image content before being added to the input image.

In the pixel area, the following equation is expressed by G. W. Proposed in the report referenced above by Browaway et al.

(Equation 1) Where w _nm ′ is the selected watermark mask to be added to the original image, w _nm is the opaque watermark pixel value at pixel (n, m), y _w is the scene white, and y _nm is Let the luminance value at the image coordinates (n, m) of the input image be, and ΔL be the strength coefficient for controlling the watermark strength.

According to one embodiment of the present invention, a probabilistic approximation can be used for the strength in the DCT domain. Consider y _nm and w _nm as independent random variables,
To the luminance range used in MPEG, ie, [0,255] to [16
, 235] and y _w = 235, the predicted value of w ′ based on equation 1 is

(Equation 2) Becomes

Y is mean α, variance β²Is assumed to have a normal distribution of E [y ^2/3 ] Term,

(Equation 3) It can be expressed as. Thus, E [y ^2/3 ] is a function of the mean and variance of the pixel values.

Equation (2) specifies the relationship between the instants of random variables w, w ′ and y. This relationship can be extended to the deterministic case to simplify equation (2), resulting in a linear approximation.

For each 8 × 8 image block, approximate the α and β ² in Equation 3 using the mean and variance of the block, and use the average α to determine y in the determination of any of these equations. Approximate and used in equation 2.

(Equation 4) Here, k = 0,. . . , 63, let w _nm ′ be the k-th pixel of the i × j-th 8 × 8 block in the watermark image.

Equation 4 approximates the nonlinear function according to Equation 2 with a linear function for each block. The selected watermark strength depends on the mean and variance of the image block. For each image block, the larger the average (i.e., lighter) and the greater the variance (i.e., more cluttered), the strength required for the watermark to maintain consistent visibility of the watermark. Will be higher.

The DCT of Equation 4 can be used to obtain the watermark mask DCT, which can be inserted in the image in the DCT domain. The mean and variance of the input image can be obtained from DCT coefficients.

(Equation 5)

(Equation 6) Here, Y _DC and Y _AC are respectively referred to as DC− and AC−D of image block Y.
Let it be a CT coefficient.

A new watermark mask is calculated for each of the I-frame and, in the case of a scene cut, the P-frame. For I frames, all DCT coefficients are M
It is easily accessible after minimal decoding of the PEG sequence, i.e. inverse variable length encoding, inverse run length encoding and inverse quantization. For P frames, most of the blocks are in the scene cut, so these DCT coefficients can be used immediately. For non-intra coded blocks, remove the average DC and AC energy obtained from the intra coded blocks.

For further speedup, the block-based (α _ij , β _ij ) pair can be exchanged with the average (α, β) for the whole image or for a specific region. In the following, a multi-domain approach will be described.

The input image can be separated into many rectangular areas. As shown in FIG. 5, (α, β) is calculated for each region, and the mask is generated appropriately. Typically, the watermark is divided into upper and lower regions. This is, for example, FIG.
This is preferred for the majority of outdoor landscapes, where the upper half of the frame is empty and the lower half is darker, as shown in FIG. Each region has a relatively visible watermark using a different (α, β) pair.

To further enhance the security of the watermark, a random position shift can be applied to the watermark image before applying the DCT. this is,
For example, if a known logo is used for watermarking purposes, it makes it more difficult for an attacker possessing the original watermark image to remove the watermark. The random shift of sub-pixels makes it very difficult for an attacker to remove the watermark without leaving any errors.

The following can be used for shifting: Generate two random numbers for the x and y directions, respectively, and normalize to between -1.00 and 1.00. In the spatial domain, the sub-pixel shift is performed by bilinear interpolation including only linear scaling and addition. A similar bilinear operation can be used in the DCT domain.

Once the DCT block of the watermark is obtained, as shown in section (ii) d of FIG.
Inserted into the frame. The DCT of the scaled watermark is directly added to the intra-coded block of the I frame or the B or P frame.

(Equation 7) In this case, E ′ _{i, j} is the i, j-th DCT block obtained as a result, E _{i, j} is the original DCT block, and W ′ _{i, j} is scaled according to Equation 6. DCT.

For a block having a forward motion vector of a P frame, that is, a backward motion vector of a B frame, a watermark added to an anchor frame is
It must be removed when adding to the current watermark. The resulting DCT error is

(Equation 8) Becomes In this case, the MDCCT is the S.D. -F. The motion compensation coefficient is as described in the above-mentioned article by Chang et al. Let W ′ _F be the watermark DCT used in the forward anchor anchor frame as shown in FIG. 1, and let V _Fij be the forward motion vector as shown in FIG. For the bidirectional predicted block of the B-frame, the forward and backward motion compensation needs to be averaged and subtracted when adding the current watermark.

(Equation 9) In this case, each of the V _F and V _B, the forward and backward-motion vector as shown in Figure 1. For the skipped blocks that are 0-motion, 0-error blocks of B and P frames, no action is required because the watermark inserted in the anchor frame is carried over.

The control of the final bit rate has one or more of the following modes. 1. Quantization / dequantization of watermark DCT coefficients such that most high frequency coefficients are zero. The result is a coarse watermark using a small number of bits. 2. High frequency coefficient cutoff. The effect is similar to that of the low-pass filter processing of the pixel area. The result is a smoother watermark with more rounded edges. 3. Motion vector selection. If the error of the motion vector using the motion compensation is larger than when the motion vector is not used, the motion vector of the macroblock of the P frame is set to zero. When using motion vectors, the error is

(Equation 10) Becomes Otherwise, set V _Fij = 0.

[Equation 11] Here, the DCT anchor frame _{I F.} If | E ″ _ij | <| E ′ _ij |, set V _Fij = 0.

FIGS. 2 a, 2 b, and 2 c illustrate the use of adaptive watermarking techniques. FIG. 2a shows the original watermark mark. Here, while exhibiting the binary form, the algorithm can process grayscale with any particular transparent color. FIG. 2b shows the original image. FIG. 2c shows the image with the new watermark. The watermarked algorithm is based on the HPJ achieved at a rate of 6 frames per second.
Tested at 210 workstation. Most of the computational effort is MC-D
The CT operation is performed. If all possible MC-DCT blocks are pre-computed,
Real-time performance is possible. This requires 12 megabytes for a 352 × 240 image size.

According to aspects of the present invention, the preferred watermark provides robustness such that it is not easily invalidated or removed by a tamper. For example, if a watermark is inserted into the MPEG video by the above method, it will restore the watermark mask, evaluate the embedding position by extensive sub-image block alignment, and

[Outside 1] Evaluate the factors. In experiments, there are always marked marks on tamper videos, which reject the owner's false claims and prevent piracy.

For robustness, the watermark should not be binary, it should have a texture similar to the scene in which it is placed. This is done by arbitrarily selecting an I-frame from the scene, decoding it by an inverse DCT transform to obtain pixel values, and masking out the watermark from the decoded video frame. If there is a camera action such as panning or zooming in the video sequence, the inserted watermark is nullified by applying a video mosaic, ie, combining large images from small parts of multiple image frames. In this case, the watermark can be filtered out as out of range. However, this technique does not work if the moving object is present in the foreground, since the watermark is also embedded in the moving foreground. As a measure according to another embodiment of the present invention

Use a watermark that appears globally, that is to say stationary with respect to background motion. Such camera operations use a 2D affine model and transform and scale the watermark using the evaluated camera operations. The affine model is defined as follows. MPEG motion vectors are typically generated by finding the block that minimizes block matching, or mean square error. If the motion vector does not represent the exact optical flow, it is usually good to evaluate camera parameters in a sequence that does not have large dark or uniform areas.

If the distance between the object / background and the camera is long, it is usually sufficient to represent the global motion of the current frame using a 6-parameter affine transformation.

(Equation 12)In this case, let (x, y) be the coordinates of the macroblock in the current frame and [u, v]^T Is the motion vector associated with the macroblock, and [a₁a₂a₃a₄a₅a₆] ^T Is an affine transformation parameter. [u, v]^TIs represented by U,

[Outside 2] Is represented by X, and [a ₁ a ₂ a ₃ a ₄ a ₅ a ₆ ] ^T is represented by a.

Given a motion vector for each macroblock, find a global parameter using least squares (LS) evaluation, ie, the motion vector evaluated in (1) and M
Find the set of parameters δ that minimize the error between the actual motion vector obtained from the PEG stream.

(Equation 13) in this case,

[Outside 3] Is the evaluated motion vector. To solve δ, set the first derivative of S (δ) to zero,

[Equation 14] Get. in this case,

(Equation 15) And

All sums exceed all valid macroblocks for which motion vectors remain present after the nonlinear noise reduction process. After the first LS evaluation, motion vectors that are longer in distance from the evaluated motion vector are filtered out before the second LS evaluation. The evaluation process is repeated multiple times to improve accuracy. The dominant movement is evaluated using the following clustering. For each B or P frame, obtain a forward motion vector. Each motion vector is assigned to one of a plurality (for example, 4) of predefined classifications. The global affine parameters are assigned to the first class and zero is assigned to all other classes. Repeat the following items multiple times, for example 20 times, or until the difference is optimized. Assignment of each motion vector to the class that minimizes the Euclidean distance and recalculation of the two-dimensional affine parameters for each class using the motion vectors of its members.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a motion-compensated discrete cosine transform (MC-DCT).

2A is a diagram illustrating a watermark mark, FIG. 2B is a diagram illustrating an original image, and FIG. 2C is a diagram illustrating superimposition of an original image and a watermark mark.

FIG. 3 is a flowchart of an initial process.

FIG. 4 is a flowchart of a watermark superimposition process.

FIG. 5 is a flowchart for scaling an area.

[Procedure amendment]

[Submission date] May 22, 2000 (2000.5.22)

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0029

[Correction method] Change

[Correction contents]

For robustness, the watermark should not be binary, it should have a texture similar to the scene in which it is placed. This is done by arbitrarily selecting an I-frame from the scene, decoding it by an inverse DCT transform to obtain pixel values, and masking out the watermark from the decoded video frame. If there is a camera action such as panning or zooming in the video sequence, the inserted watermark is nullified by applying a video mosaic, ie, combining large images from small parts of multiple image frames. In this case, the watermark can be filtered out as out of range. However, this technique does not work if the moving object is present in the foreground, since the watermark is also embedded in the moving foreground. As a countermeasure according to another embodiment of the present invention, a watermark which appears stationary throughout the whole, that is, against background movement is used. Such a camera operation uses a 2D affine model and transforms and scales the watermark using the evaluated camera operation. The affine model is defined as follows.

[Procedure amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0030

[Correction method] Change

[Correction contents]

An MPEG motion vector is usually generated by finding a block that minimizes block matching, ie, the mean square error. If the motion vector does not represent the exact optical flow, it is usually good to evaluate camera parameters in a sequence that does not have large dark or uniform areas.

────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Shi-Fu Chan United States Southwest Mad Building Room 1312 Colombia University Department of Electrical Engineering (72) Inventor Jianhao Men United States of America 10027 New York West Onehand Red Nineteenth Street 435 Number 9 L F-term (Reference) 5B057 BA02 CA01 CA08 CA12 CA16 CB01 CB08 CB12 CB16 CE20 CG07 5C059 KK43 MA00 MA14 MA15 MA23 NN01 PP05 PP06 PP07 RB02 RC35 UA02 5C076 AA14 BA06 5J104 AA15 NA

Claims (5)

[Claims] 1. A method for including a watermark in a digital image, comprising: obtaining digital data of a transformed representation of the image; and determining a transformed representation of the watermark for optimal visibility of the watermark in the image. And overlaying the transformed representation of the watermark on the transformed representation of the image. 2. The method according to claim 1, wherein the transformed representation of the image is:
A method characterized by a compressed representation. 3. The method of claim 1, wherein the transformed representation of the image is:
A method characterized by being represented by a discrete cosine transform. 4. The method of claim 1, wherein the image is one of a sequence of video images. 5. The method according to claim 3, wherein said transformed representation comprises motion compensation.