KR20010031526A - Watermarking of digital image data - Google Patents

Abstract

The digital watermark can be used to indicate the copyright of the digitized video. If a video image is transformed and transmitted by a discrete cosine transform (DCT) for compression with or without motion compensation, it is advantageous to include a watermark after the conversion. To this end, a DCT watermark for optimal visibility is generated based on the original image data, and the generated watermark is superimposed on the converted data.

Description

Watermarking digital image data {WATERMARKING OF DIGITAL IMAGE DATA}

The watermark on a conventional paper deed consists of a translucent desige that can be visually identified when lighted onto the deed. Also, more typically, watermarks can be observed under certain light irradiation conditions or at certain viewing angles. For example, these watermarks are difficult to counterfeit for certification of instruments such as bank drafts, checks and stock securities.

In digital video technology, watermarks are used to indicate some ownership, for example copyright. In the present specification, a watermark is a visible pattern or invisible pattern that is superimposed on an image and cannot be easily removed without leaving a tapped trace. It is called "robustness" to leave no trace of the hand.

One robust method for including visible watermarks in digital images is by Braudaway et al., ″ Protecting Publically Available Images with a Visible Image, Technical Research 96A000248, by IBM Research Institute, TJ Watson Research Center. Watermark ″. The luminance level ΔL is selected for the strength of the watermark, and the luminance for each pixel of the image is changed by ΔL and a nonlinear function. For the sake of security, the level ΔL is random for all the pixels in the image.

<Summary of invention>

When images are transformed and sent by discrete cosine transform (DCT) for compression with or without motion compensation, it is advantageous to include a watermark after the conversion. To this end, (i) a DCT watermark is generated for optimal visibility based on the original image data and (ii) the generated watermark is superimposed on the converted data.

The present invention relates to providing digital image data having a watermark, in particular digital image data which is video data.

1 illustrates a motion compensated discrete cosine transform (MC-DCT).

2A is a diagram of a watermark mask.

2B is a view of the original image.

FIG. 2C illustrates the superposition of the original image and the watermark mask. FIG.

3 is a flow chart of initial processing.

4 is a flow chart of watermark superposition processing.

5 is a scaling flowchart for an area.

The mask generation module generates a DCT watermark mask based on the original video content. The motion compensation module effectively inserts a watermark into the DCT domain and outputs a valid video bitstream at the specified bit rate. The following will specifically describe image data in MPEG format.

MPEG video consists of groups of pictures (GOP) as disclosed in document ISO / IEC 13818-2 Committee Draft (MPEG-2). Each GOP starts with an intra coded ″ I-frame ″ followed by a number of forward lookup ″ P-frames ″ and bidirectional lookup ″ B-frames ″.

With motion compensation, when a watermark is inserted into an I-frame, the P- and B-frames in the GOP will also change. For this correction, when the watermark is added to the current frame, the motion compensation for the watermark in the anchor or base frame must be subtracted. For such subtraction, SF Chang et al., IEEE Journal of Selected Areas in Communications, a topic on intelligent signal processing, pp.1-11, January 1995, ″ Manipulation and Compositing of MC-DCT Compressed Video. of motion compensation techniques in the DCT domain, as shown in &quot; of MC-DCT Compressed Video &quot;

In a video sequence, the image content changes frame by frame. Therefore, in order to be able to sufficiently check the watermark throughout the video, the watermark should be applied to the video content. For example, when an image is complex or ″ busy ″, that is, when the image has many high frequency components, the watermark is stronger. For different regions within the same video frame, the watermark can be partially scaled, thereby improving security for contact.

(i) Mask Generation Module

In a module such as that shown in section (i) of FIG. 4, a watermark mask image is first generated for each first OPP frame after each GOP or scene cut. This is generally based on the fact that video content tends to be consistent within a GOP that is about 15 frames or 0.5 seconds long. However, when there are scene cuts in the GOP, the visual content will change significantly, and a new mask is used to accommodate the new visual content. Thus, the watermark mask is superimposed on the first P-frame after the I-frame or the scene cut.

In order to generate the mask as shown in FIG. 3, the input watermark image is first converted to a gray scale image. Only the luminance channel of each image is modified. Transparent color (background color) is selected. The luminance of all watermark pixels with a transparent color value is designated as zero. Optionally, the mask image moves randomly in the x- and y- directions. DCT is applied to obtain a DCT mask of the watermark. The luminance of the mask is scaled adaptively according to the input image content before being added to the input image.

In the pixel domain; W. In the above report by G. W. Braudaway et al, the following equations are proposed.

Where w _nm 'is the scaled watermark to be added to the original image, w _nm is the opaque watermark pixel value in pixels (n, m), y _w is the scene white, y _nm is the luminance value of the input image in the image axis (n, m), and ΔL is the scaling factor that controls the watermark intensity.

According to one aspect of the invention, stochastic approximation can be used for scaling in the DCT domain. If y _nm and w _nm are considered independent random variables, then y is normalized over the luminance range used in MPEG-ie from [0, 255] to [16, 235] and y _w = 235, based on Equation 1, the expected value of w 'is Equation 2.

If y is with a normal distribution (normal distribution) with a mean α and the deviation (variance) β ^2, in Equation (2), wherein E [y ^2/3] can be expressed as:

Thus, E [y ^2/3 ] is a function of the mean and the deviation of the pixel values.

Equation 2 defines the correlation between the moments of the random variables w, w ', and y. This correlation can be extended to the deterministic case in order to simplify Equation 2, resulting in a linear approximation.

For each 8 by 8 image block, the mean and the deviation of the block are used to approximate α and β ² in equation (3), where the average α determines the mean of the formulas used in equation (2). Is used to approximate y.

Here, for k = 0, ..., 63, w _ijk is the k th pixel of the i, j th 8 to 8 (8 by 8) block in the watermark image. w ' _ijk relates to the scaled watermark.

Equation 4 approximates the linear function according to Equation 2 using a linear function block by block. The scaled watermark intensity depends on the mean and the deviation of the image block. For each image block, the higher the average (i.e. the brighter) and the higher the deviation (i.e. the more confused), the intensity required for the watermark to maintain a consistent visibility of the watermark. Gradually increases.

The DCT of Equation 4 can be used to obtain the DCT of the watermark mask, which can be inserted into the image in the DCT domain. The mean and deviation of the input image can be derived from the DCT coefficients.

Where Y _DC and Y _AC are the DC- and AC-DCT coefficients of image block Y, respectively.

A new watermark mask is calculated for each I-frame and P-frame, for the latter in the case of a scene cut. For I-frames, all DCT constants are easy after minimal decoding of the MPEG sequence, i.e., inverse variable length coding, inverse run length coding and inverse quantization. Can be approached. For P-frames, since most blocks are in the scene cut, these DCT coefficients can be used immediately. In a non-intra coded block, the average DC and AC energy obtained from the intra coded block is substituted.

To further speed up, the block-based (α _ij , β _ij ) pairs are averaged over the entire image or a constant area. Is replaced by. Next, the multi-domain approach is described.

The input image can be separated into many orthogonal regions. As shown in Figure 5, in each region The pair is calculated and thus the mask is generated. In general, the watermark is divided into upper and lower regions. This is suitable for most exterior scenes, for example, with a sky in the upper half and a darker background in the lower half, as shown in FIG. 2A. Each area is different It will have a relatively visible watermark using pairs.

In order to further enhance the security of the watermark, a randomized location shift may be applied to the watermark image before applying the DCT. This makes it more difficult for attackers to remove the watermark, which retains the original watermark image-for example, when a known logo is used for the purpose of the watermark. The sub-pixel randomized location shift will make it very difficult for the attacker to remove the watermark without leaving any error residues.

The following may be used for shifting. For the x- and y-directions respectively, two random numbers occur and normalize to lie between -1.00 and 1.00. In the spatial domain, sub-pixel shifting is only affected by bi-linear interpolation involving linear scaling and addition. In the DCT domain, similar bilinear manipulations can be used.

(ii) motion compensation module

Once the DCT blocks of the watermark are obtained, they are inserted into the DCT frame of the input video in one of three ways as shown in section (ii) of FIG. For I-frame or intra coded blocks within a B- or P-frame, the DCT of the scaled watermark is added directly.

E ' _ij is the i-th, j-th result DCT block, E _ij is the first DCT block, W' _ij is the scaled watermark DCT according to equation (6).

For a block associated with a forward motion vector in a P-frame, or a block only associated with a backward motion vector in a B-frame, the watermark added in the anchor frame should be removed when adding the current watermark. The remainder of the resulting DCT error is:

Wherein MCDCT is S.-F. It is a motion compensation function in the DCT domain as described in the paper by Chang et al. W ' _F is a watermark DCT used in the forward anchor frame, and V _Fij is a forward motion vector as shown in FIG.

For bidirectional estimation blocks within a B-frame, both forward and backward motion compensation should be averaged and subtracted when adding the current watermark.

Where VF and VB are forward and backward motion vectors, respectively, as shown in FIG.

For skipped blocks, which are 0-motion, 0- remaining error blocks in B- and P-frames, no operation is necessary since the watermark inserted in the anchor frame will be carried over.

Control of the final bit rate may include one or more of the following features.

1. Quantize / dequantize the DCT coefficients of the watermark so that the highest frequency coefficients are zero.

2. Cut off the high frequency coefficients. The effect is similar to low pass filtering in the pixel domain. The result is a smoother wordmark with rounded edges.

3. Motion vector selection, setting the motion vector of the microblock in the P-frame to zero when the error residual from using the motion compensation of this motion vector is greater than without using it.

If a motion vector is used, the remaining errors are as follows.

Otherwise set V _Fij = 0.

Where I _F is the DCT of the anchor frame.

if , V _Fij = 0.

2A, 2B and 2C illustrate the use of an adaptive watermark technique. 2A illustrates the original watermarking technique. Although a binary version is shown here, the algorithm can handle gray scales with any particular transparent color. 2B shows the original image. 2C shows the new watermarked image.

The watermarking algorithm was tested on an HP J210 workstation, achieving a speed of 6 frames / second. Most of the computational effort was spent on MC-DCT computations. If all possible MC-DCT blocks were precomputed, real time execution would be possible. This would require 12 megabytes of memory for a 352 × 240 image size.

According to one aspect of the present invention, preferred watermarks provide rigidity in that the hand is not easily destroyed or removed by the dam. For example, if a watermark is inserted into MPEG video by the method described above, the watermark mask is recovered, the embedded position is estimated by extensive subpixel block matching, and then for each watermark region. You will need to estimate the arguments. In the experiments, there was always a visible trace of the touched video, which could be used to reject false claims and stop copyright infringement.

For robustness, the watermark should not be binary, but should have a structure similar to the structure of the scene in which it is placed. This can be accomplished by randomly selecting an I-frame from the scene and decoding it by inverse DCT transform to obtain a pixel value and masking out the watermark from the decoded video frame.

If there is camera motion in the video sequence, such as panning and zooming, the inserted watermark can be destroyed by applying video mosaicing, ie, assembling a large image from small portions of multiple image frames. have. The watermark can then be filtered out as an outliner. However, this technique will be ineffective because the watermark will be embedded in the moving foreground object when the object is actually moving in the foreground. As a countermeasure according to another embodiment of the present invention, a watermark that looks static for the whole or the background motion may be used. Such camera motion using a 2-D affine model and the estimated camera motion are used to transform and scale the watermark. The affine model can be described as follows.

Motion vectors in MPEG are generally generated by finding blocks in reference frames such that block matching, i.e., mean squared error, is minimized. Motion vectors do not represent true optical flow, but in most cases are good at estimating camera parameters in sequences that do not include large dark and uniform areas.

When the distance between the object / background and the camera is large, it is usually sufficient to use a 6 parameter affine transformation to describe the global motion of the current frame.

Where (x, y) is the coordinate of the macroblock in the current frame, Is the motion vector associated with that macro block, Is an affine transformation vector. Is written as U, Is written as X, silver It is written as.

Given a motion vector for each macroblock, obtain a global parameter using least squares (LS) estimation. That is, a parameter for minimizing an error between the motion vector estimated in Equation 1 and the actual motion vector obtained from the MPEG stream. Obtain the set of.

In the above formula Is the estimated motion vector. To solve for, S ( When the first derivative of) is zero, the following is obtained.

Where

All summation is done for all valid blocks whose motion vectors survive after the nonlinear noise reduction process. After the first LS estimation, motion vectors with a large distance from the estimated ones are filtered out before the second LS estimation. The estimation process is repeated several times to add accuracy.

The dominant motion can be estimated using clustering as follows.

For each B- or P-frame, a forward motion vector is obtained.

Each motion vector is assigned to one of a number of (eg 4) predefined classifications.

One global affine parameter estimation is performed.

Assign the global affine parameter to the first category and zero to all other classes.

Assigning each motion vector to a classification that minimizes the geometric distance and recalculating the 2-D affine parameters for each classification using its member motion vectors, for example 20 Repeat this time, or until the rest of the error has stabilized.

Claims (5)

A method of including a watermark in a digital image, Obtaining digital data of the transformed representation of the image; Determining a converted representation of the watermark for optimizing visibility of the watermark in the image; And Superimposing the converted representation of the watermark on the converted representation of the image Method comprising a. The method of claim 1, wherein the transformed representation of the image is a compressed representation. The method of claim 1, wherein the transformed representation of the image is a discrete cosine transformed representation. The method of claim 1, wherein the image is one of a series of video images. 4. The method of claim 3, wherein the transformed representation includes motion compensation.