JP4169161B2 - Video watermarking device - Google Patents

Description

  The present invention relates to a moving image digital watermarking device, and more particularly to a moving image digital watermarking device for clarifying the existence of copyright by embedding a watermark when distributing a moving image.

  Conventionally, attempts have been made to embed watermark data in a still image to clarify the copyright of the image. In this method, watermark data is embedded in a still image using Fourier transform, discrete cosine transform (DCT), wavelet transform, or the like. Embedding watermark data in a still image requires a large amount of calculation, but since there is only one screen for communicating a still image, there is no particular problem.

  In addition, with the spread of the Internet and the like, it is expected that opportunities to communicate or distribute moving images as digital images will increase. However, when trying to clarify the copyright of moving images, there are a large number of moving images. Since image frames exist, it is necessary to embed the same watermark in all frames or to embed the watermark in a specific frame. When applying the above-described method for embedding watermark data in a still image for embedding the watermark data, since it is necessary to embed the watermark data in a large number of frames in a moving image, the amount of calculation for embedding the watermark data There is a problem that it is difficult to realize because of enormous volume. In addition, when a watermark is embedded only in a specific frame, there is a problem that it is difficult to find a frame in which the watermark is embedded.

On the other hand, as a method for embedding a watermark in a moving image, a method for embedding a watermark by manipulating an MPEG moving vector has been proposed. According to this method, a watermark can be extracted without specifying one frame in an image frame, which is useful. As a patent related to the present invention, there is, for example, JP-A-10-178642.
Japanese Patent Laid-Open No. 10-178642

  However, the above-described method for manipulating MPEG motion vectors has a problem that watermark data can be easily removed by once decoding and then re-encoding the compressed data. In addition, since moving images are generally distributed in a format compressed by MPEG, it is required to determine the watermark in the MPEG compressed format, but in the past, sufficient consideration was given to this point. Was not.

  The object of the present invention is to eliminate the above-mentioned problems of the prior art, to determine watermark data in a compressed format such as MPEG, and to be large even when subjected to attacks such as thinning out frames from others. It is an object of the present invention to provide a moving image digital watermarking device having resistance. Another object of the present invention is to provide a moving image digital watermarking apparatus capable of detecting watermark data embedding at high speed.

In order to achieve the above object, the present invention provides means for extracting a block to be inspected for watermark data from a compressed block of image data, and a first embedded with watermark data for the extracted block to be inspected. The second DCT coefficient of the neighboring block corresponding to the DCT coefficient is determined so that the DCT coefficient value of the second DCT coefficient is within a predetermined range with respect to the DCT coefficient value of the first DCT coefficient. A means for removing the second DCT coefficient determined to have an abnormal value by the sieve, and an absolute value of the value of the first DCT coefficient has the abnormal value And a means for determining that the watermark data is embedded when the absolute value average value of the second DCT coefficient values from which the DCT coefficients have been removed differs by a predetermined threshold value or more.

According to the present gun invention, it is possible to previously removed second DCT coefficients having an abnormal value, it is possible to accurately detect the watermark data, it is possible to improve the accuracy of detection of the presence or absence of the watermark data .

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a watermark data embedding apparatus according to the present invention.

  In the figure, the block dividing unit 1 divides, for example, the image frame of FIG. 2A into 8 × 8 pixel blocks bij. The block extraction unit 2 extracts the macro block (16 × 16 pixels) mxy designated from the template 9. The DCT unit 3 performs discrete cosine transform (DCT) for each block mxiyi included in the macroblock mxy. The DCT coefficient extraction unit 4 extracts the DCT coefficient x of one block mx2y2 (see FIG. 2B) in the DCT-processed macroblock mxy according to the instruction of the template 9. Here, the template 9 provides information indicating a position where watermark data, which will be described later, is embedded. The block extracting unit 2 extracts a macroblock at the position where the watermark data is embedded, and the DCT coefficient extracting unit 4 The DCT coefficient x in which the watermark data is embedded is extracted.

  The watermark data embedding unit 5 embeds watermark data in the DCT coefficient x. For example, watermark data is embedded by changing the DCT coefficient x to a coefficient x ′ obtained by the following equation.

x ′ = sign (x) * (| x | + a)
Here, x is an original coefficient value, a is watermark data, that is, an embedded value, and sign (x) is a sign of x.

  That is, the watermark data embedding unit 5 functions to change the DCT coefficient x in the direction in which the absolute value increases while maintaining the + and − signs of the DCT coefficient x. For example, if the watermark data a = 5.0, if the DCT coefficient x is +15.7, x ′ = + 20.7, and if the DCT coefficient x is −15.7, x ′ = − 20. .7.

  The inverse DCT unit 6 performs inverse DCT on the block (macroblock) in which the watermark data is embedded. The block synthesizing unit 7 synthesizes the block subjected to the inverse DCT with the block not extracted by the block extracting unit 2, and generates image data in which watermark data is embedded. The MPEG encoding unit 8 performs MPEG encoding (quantization and Huffman encoding) on the image data in which the watermark data is embedded in order to create distributed moving image content.

  In one embodiment of the present invention, the block extraction unit 2 extracts macroblocks at a plurality of positions indicated by the template 9 from a moving image frame, and then the same as indicated by the template 9 by the DCT coefficient extraction unit 4 Alternatively, DCT coefficients at different positions are extracted from the macroblocks at the plurality of positions, and the watermark data embedding unit 5 embeds watermark data in the DCT coefficients.

  That is, as shown in FIG. 3, the block extraction unit 2 extracts, for example, four macroblocks M1 to M4 at a plurality of positions shown in FIG. For other frames F, for example, four macroblocks M5 to M8 at a plurality of positions shown in FIG.

  Next, the DCT coefficient extraction unit 4 extracts the DCT coefficient at the same position or the DCT coefficient at a different position indicated by the template 9 for the four macroblocks extracted by the block extraction unit 2, The watermark data embedding unit 5 embeds watermark data.

  If the macroblock extraction pattern in FIG. 3 (a) is named mark A and the macroblock extraction pattern in (b) is named mark B, then the image frame sequence as shown in FIG. , The frame with the mark A, the frame with the mark B, and the frame without the mark are classified. In the above description, the number of macroblocks extracted by the block extraction unit 2 is four. However, the present invention is not limited to this and may be two or more. Further, although the number of marks A and B is two, the number is not limited to this, and may be one or three or more. In the figure, ■ indicates a macroblock with embedding, and □ described later indicates a macroblock without embedding.

  Next, a second embodiment of the watermark data embedding will be described with reference to FIG. In this embodiment, for example, when moving images having the same content are distributed to the user X and the user Y, macroblocks at the same position in each moving image frame are set as candidates for embedding watermark data, and watermarks are further added to the macroblocks. A case where the distribution destination of the moving image is made clear by selecting whether to embed data or not is considered. For example, as shown in FIGS. 11 (a) and 11 (b), macroblocks X1 to X4 and Y1 to Y4 extracted as watermark data embedding candidates from moving images distributed to users X and Y are the same. For the user X, the macroblocks X1 and X3 are processed by embedding watermark data (= 1) and not embedded in X2 and X4 (= 0), and for the user Y, the macroblocks Y2 and Y4 are processed. There is a case where watermark data is embedded (= 1) and not embedded in Y1 and Y3 (= 0). The advantage of this process is that 1-bit information is expressed by whether or not watermark data is embedded. For example, a moving image in which binary data 1010 is embedded is distributed to the user X. It can be seen that the moving image in which the binary data is embedded is distributed to the user Y. For example, when a pirated version of the moving image is issued, the origin of the pirated version becomes clear from the binary data.

  In such a case, when the DCT coefficient of the moving image data distributed to the user Y is subtracted from the DCT coefficient of the moving image data distributed to the user X, as shown in FIG. The value of the position in which is embedded is 1 or -1, and the values of other positions are all 0. For this reason, the embedded position of the watermark data, that is, the secret key is exposed to the user.

  Therefore, in this embodiment, when providing a moving image with watermark data to users X, Y, etc., encoding noise included in the compressed image is changed by changing the encoding control parameter for each user and performing MPEG1 compression. The component is changed for each user. Then, even if the DCT coefficient of the moving image distributed to the users X and Y is subtracted, the DCT coefficient in which the watermark data is embedded is buried in the coding noise and is not easily detected. Note that even if the MPEG1 encoding control parameter is changed as described above, there is no problem in decoding, and no problem occurs in the quality of the decoded image.

  As a method for changing the encoding control parameter, for example, “Test Model 5” of a known encoding control technique (ISO / IEC JTC1 / SC29 / WG11 / N0400) can be used. In the Test Model 5, the excess bit exponent dj is calculated from the number of bits allocated so far for each macroblock and the number of bits actually generated, and the quantization step size Qj is determined by using the following equation. .

Qj = (dj × 31) / r
r = 2 × (fixed parameter)
Here, the encoding control can be changed by setting “2” used in the calculation formula of r to a different value as a parameter. For example, “2” is set to “1.8” for the user X, and “2.1” is set to “2.1” for the user Y. This change in encoding control can be performed by, for example, the MPEG encoding unit 8 in FIG. As Qj increases, the number of generated bits decreases and the noise on the image increases.

  Next, a watermark data detection apparatus that detects watermark data embedded in a moving image as described above will be described. FIG. 4 is a block diagram showing a schematic configuration of the detection apparatus.

  The video content 11 to be inspected is in the state of MPEG compressed data, and the inspection target macroblock extraction unit 12 extracts the inspection target macroblock. This extraction can be performed by an instruction from the template 13. The template 13 is preferably the same as the template 9 shown in FIG. That is, the template 13 outputs information on the marks A and B and information on which position of the DCT coefficient in each macroblock of the marks A and B is embedded with the watermark data x.

  According to the information from the template 13, the macroblock extracted by the inspection target macroblock extraction unit 12 is Huffman decoded by the Huffman decoding unit 14, and then dequantized by the inverse quantization unit 15. The inversely quantized DCT coefficient is sent to the watermark data detection unit 16 to detect whether or not the watermark data is embedded.

  Specifically, since the watermark data detection unit 16 is instructed from the template 13 by the watermark data embedding position information of the DCT coefficient, the watermark data detection unit 16 includes the four blocks mx1y1 to mx2y2 included in the macroblock. (See FIG. 5), four DCT coefficients x1, x2, x3, and x corresponding to the watermark data embedding position are extracted. These DCT coefficients x1, x2, x3, and x are generally equal because they are correlated with each other. Therefore, the watermark data detection unit 16 performs the following calculation.

(| X1 | + | x2 | + | x3 |) / 3 + s <| x | (1)
Here, x is an inspection target coefficient value, and s is a standard deviation of | x1 |, | x2 |, | x3 |, or a fixed value.

  If the inequality is satisfied, it is determined that the watermark data is embedded. If the inequality is not satisfied, it is determined that the watermark is not embedded. The majority decision determining unit 17 determines the mark of the video content frame. That is, the presence / absence of watermark data is determined from the inspection result for each of a plurality of macroblocks used for the mark A for a certain frame of video content. The same determination is performed for each of a plurality of macroblocks used for the mark B for the certain frame. Then, the mark having the largest number of macroblocks determined as having a watermark obtained as a result of this determination is determined as a mark embedded in the frame. When the mark can be determined in this way, it is determined that the video content is embedded with watermark data.

  As a result of the watermark data detection, as shown in FIG. 6, when the mark A, mark B and unmarked groups are detected, the distributor of the video content of the moving image can claim the copyright. become. In this case, even if the obstruction or attack of thinning out the frame is added by another person, the mark A, the mark B, and the group without the mark can be detected without fail, and the resistance to the obstruction or attack is great. Will be able to show. In addition, if the macro block position and / or DCT coefficient position in which the watermark data is embedded is changed for each distribution destination, for example, when a pirated version is released, it can be claimed that it owns the copyright and the origin of the pirated version Will be able to detect.

  Next, another method for detecting watermark data will be described with reference to FIG. In this method, for example, if an abnormal value exists in the three DCT coefficients x1, x2, and x3 corresponding to the watermark data embedding position in the above equation (1), the accuracy of the equation (1) decreases. When the three DCT coefficients x1, x2, and x3 are sieved and an abnormal value exists in the three DCT coefficients x1, x2, and x3, the abnormal value is removed, and the equation (1) or others The operation is performed.

  In order to remove this abnormal value, in this embodiment, DCT coefficients x1, x2, x3 and DCT coefficients x1 ', x2', x3 'existing in the same block are used. In the example of FIG. 7, the adjacent coefficient of the target DCT coefficient is used. Next, assuming that the DCT coefficient for which the watermark data embedding operation has been performed is x and the DCT coefficient of the adjacent pixel is x ′, the following determinations (a), (b), (c),. .

(a) If 0 ≦ x ′ <50, only xi satisfying 0 ≦ xi ′ <50 (where i = 1, 2, 3) is used for determination.
(b) If 50 ≦ x ′ <100, only xi satisfying 50 ≦ xi ′ <100 is used for determination.
(c) If 100 ≦ x ′ <150, only xi satisfying 100 ≦ xi ′ <150 is used for determination.
......

  Next, using xi that satisfies the above determination, it is determined whether or not the condition of the following equation (2) is satisfied.

x-Σxi / N> α (2)
(Where N is the number of xi remaining after sieving, α is a threshold, where α varies depending on the sieving class)

  If the condition of the above equation (2) is satisfied, it is determined that the watermark data embedding operation is performed on the DCT coefficient x. According to this determination method, since the abnormal value of the DCT coefficient xi can be removed in advance, the watermark data can be detected with high accuracy.

  Next, a method for detecting whether or not watermark data is embedded at high speed, which is still another method for detecting watermark data, will be described. The watermark data embedding according to the present invention employs a method of disturbing the similarity of local DCT coefficients of an image. For this reason, for example, in the inter-frame prediction at the time of encoding in MPEG1, since the prediction is lost, the DCT coefficient of the watermark data embedding position is always a non-zero value. Accordingly, it is possible to detect the presence or absence of embedding watermark data using MPEG1 P and B frames using interframe predictive coding. The simplest method is to individually check whether or not a non-zero value DCT coefficient exists at the embedded position of the watermark data provided from the templates 9 and 13, in other words, the position known from the secret key. Can be determined at high speed whether watermark data has been embedded, that is, whether or not the DCT coefficient has been manipulated.

  FIG. 12 is a flowchart for explaining a method for detecting whether or not marks A and B as shown in FIG. 3, for example, are embedded in an image frame using the watermark data detection method.

  In this embodiment, the P frame and the B frame are set as target frames for mark detection. In step S1, the number of marks, in other words, the number of patterns at the position of a macroblock in which watermark data is embedded in an image frame is set to i, and i = 0. In the example of FIG. 3, the number of marks is two. In step S2, it is determined whether or not it is the last mark. If this determination is negative, the process proceeds to step S3 and 1 is added to i. In step S4, the i = 1 mark indicated by the template, for example, mark A, is selected, and the values at predetermined positions of the DCT coefficients of the four blocks M1 to M4 included in the mark A, for example, (0, 3). Let n (i) be the number of DCT coefficient values other than 0 for the position. The (0, 3) position, which is the predetermined position, is indicated by the template, for example. In step S5, the absolute value sum of the four DCT (0, 3) coefficients indicated by the template is a (i). In step S6, x (i) = a (i) .times.n (i) is obtained.

  Then, it returns to step S2 again and enters the next cycle. In step S4, a mark of i = 2 shown in the template, for example, mark B is selected, and a value of a predetermined position of each DCT coefficient of the four blocks M5 to M8 included therein, for example, (0, 3) position. Let n (i) be the number of DCT coefficient values other than zero. In step S5, the sum of absolute values of the four DCT (0, 3) coefficients indicated by the template is a (i). In step S6, x (i) = a (i) × n (i) is set. Ask.

  Then, it returns to step S2 again and the said operation | movement is repeated. As a result of this repeated operation, if the determination in step S2 becomes affirmative, the process proceeds to step S7. In step S7, i which becomes the maximum x (i) in {x (i)} is set as a mark I. Now, assuming that the mark of the frame from which the embedded information is detected is the mark A, x (1)> x (2), and if the mark B is, x (2)> x (1 It is clear that

  In step S8, it is determined whether x (I) ≧ 20 is satisfied. If this determination is affirmative, the process proceeds to step S9, and it is determined that the mark I is embedded in the frame. On the other hand, if the determination in step S8 is negative, the process proceeds to step S10, and it is determined that no marks, for example, marks A and B are embedded in the frame. The numerical value 20 is an example, and is not limited to this numerical value.

  Next, another embodiment of the present invention will be described with reference to FIG. In this embodiment, a plurality of patterns of macro block positions for embedding watermark information in a frame are determined in advance and prepared as templates. For example, two patterns of group A and group B as shown in FIG. 8 are prepared. In the illustrated example, the group A has a pattern “1010” in order from the upper left pixel to the lower right pixel, and the group B has a pattern “0010”.

  In accordance with the binary representation of the embedded information, the watermark information embeder does not embed when 0, and embeds when 1. Note that □ indicates a macro block without embedding, and ■ indicates a macro block with embedding.

  The extractor of the watermark information detects the macroblock in which the watermark information is embedded and the macroblock in which the watermark information is not embedded from the image frame sequence by using the watermark data detection method described above, and the same as in the first embodiment. Then, group A and group B are detected. When the groups A and B are detected from the image frame sequence, the distributor of the video content of the moving image can claim the copyright.

  According to this embodiment, not only the watermark data embedding position but also the bit information represented by the presence or absence of watermark data embedding can be used as data for clarifying the copyright.

  Next, still another embodiment of the watermark data embedding will be described with reference to FIG. As will be apparent from the above description, the watermark data is detected using the fact that the DCT coefficients at the corresponding positions of the macroblock are correlated with each other. Therefore, in the first embodiment described above, for example, when a situation occurs in which there is no correlation between one or a plurality of DCT coefficients of the macroblocks A1 to A4 in FIG. 8A, for example, one of the macroblocks A1 to A4. If one or more happens to include the border of the image, such as the background and the contour of the object, the watermark data embedded in the macroblock cannot be detected stably, There is a possibility that the extracted “0” and “1” cannot be accurately determined.

  This embodiment has been made in view of the above-mentioned problems. First, the moving image frames are classified into a plurality of groups A, B,..., N, and a plurality of macro blocks in which watermark data is embedded for each group. For example, four pieces are extracted. The extraction positions of the plurality of macroblocks are determined by, for example, the template 9 and are different from each other. In the illustrated example, in group A, macroblocks A1, A2, A3, and A4 are extracted, and in group B, macroblocks B1, B2, B3, and B4 at positions different from these are extracted. Similarly, in the group N, macroblocks N1, N2, N3, and N4 at different positions are extracted. And information embedded DCT coefficient set S1 = {A1, B1,..., N1} with embedded watermark data, S2 = {A2, B2,..., N2}, S3 = {A3, B3,. Define S4 = {A4, B4,..., N4}.

  In this embodiment, the presence or absence of watermark data embedding in the extracted macroblock is comprehensively determined for each information embedding DCT coefficient set. For example, it is determined by majority decision for each information embedded DCT coefficient set. For example, whether or not watermark data is embedded in the information embedding DCT coefficient set S1 = {A1, B1,..., N1} is determined by the majority decision of A1 to N1. Therefore, even if it is detected that the macroblock A1 includes the boundary portion of the image, the DCT coefficient is not correlated, and the watermark data is not embedded, the other macroblocks B1 to N1 include the boundary portion of the image. Since the possibility is small, whether or not watermark data is embedded in these macroblocks is correctly detected. As a result, whether or not the watermark data is embedded as the whole S1 is correctly determined by the majority decision, and the above-mentioned problem can be avoided.

  Next, a method of operating the embodiment (FIG. 10) at high speed will be described with reference to the flowchart of FIG. Let the information-embedded DCT coefficient set be S (j). The number of the information-embedded DCT coefficient sets, that is, the number given to the macroblocks in each group in FIG. 10 is j, and j = 0 is set in step S11. In the case of FIG. 10, the number of information-embedded DCT coefficients is four. In step S12, it is determined whether or not it is the last information-embedded DCT coefficient set. If this determination is negative, the process proceeds to step S13, and 1 is added to j. In step S14, the number of DCT (0, 3) coefficients included in the set S (j) having an absolute value of 11 or more is defined as n (j). In step S15, the number of DCT (0, 3) coefficients included in the set S (j) is m (j). In step S16, it is determined whether or not n (j) / m (j) ≧ 50% is established. If this determination is affirmative, the process proceeds to step S17, and information on all elements of the set S (j) is obtained. It is determined that the bit is 1. On the other hand, when the result is negative, the process proceeds to step S18, and it is determined that the information bits of all elements of the set S (j) are zero.

  In the watermark data detection apparatus described with reference to FIG. 4, watermark data is detected from video content that is compressed data without decoding. However, the present invention is not limited to this, and is shown in FIG. As described above, watermark data may be detected using a coefficient obtained by decoding video content once and then DCT processing the decoded image.

  That is, the video content 11 is decoded by the MPEG decoding unit 21 (including Huffman decoding and inverse quantization), and the block in which the watermark data is embedded is extracted by the macroblock extraction unit 22 according to an instruction from the template 13. . The DCT unit 23 performs DCT processing on the extracted block. Since the information of the watermark data embedding position of the DCT coefficient is instructed from the template 13, the watermark data detection unit 16 extracts the DCT coefficient corresponding to the watermark data embedding position included in the macroblock, and performs the watermarking according to the procedure described above. Detect data. The majority decision determination unit 17 makes a majority decision as described above using the watermark data detection result by the watermark data detection unit 16, and determines a mark embedded in the frame.

It is a block diagram which shows the schematic structure of one Embodiment of the watermark data embedding apparatus with respect to the moving image of this invention. It is explanatory drawing of the embedding position of watermark data. It is explanatory drawing of the embedding position of the watermark data of one Embodiment of this invention. It is a block diagram which shows the schematic structure of one Embodiment of a watermark data detection apparatus. It is explanatory drawing of the detection method of watermark data. It is a conceptual diagram which shows an example of a watermark data detection result. It is explanatory drawing of the other detection method of watermark data. It is explanatory drawing of the embedding position of the watermark data of other embodiment of this invention. It is a block diagram which shows the structure of other embodiment of the watermark data detection apparatus. It is explanatory drawing of embodiment which determines the embedding of information using an information embedding DCT coefficient set. It is explanatory drawing of embodiment which prevents exposure of a secret key. 10 is a flowchart of an embodiment in which it is determined at high speed whether or not watermark data (mark) is embedded in a frame. 6 is a flowchart of an embodiment in which a method for determining information embedding using an information embedding DCT coefficient set is speeded up.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Block division part, 2 ... Block extraction part, 4 ... DCT coefficient extraction part, 5 ... Watermark data embedding part, 7 ... Block synthesis part, 8 ... MPEG encoding part, 9, 13 ... Template, 11 ... Video content, DESCRIPTION OF SYMBOLS 12 ... Inspection object macroblock extraction part, 14 ... Huffman decoding part, 15 ... Dequantization part, 16 ... Watermark data detection part, 17 ... Majority determination part, 21 ... MPEG decoding part, 22 ... Macroblock extraction part, 23 ... DCT part.

Claims (1)

Means for extracting an inspection target block of watermark data from the compressed block of image data;
The second DCT coefficient of the neighboring block corresponding to the first DCT coefficient in which the watermark data of the extracted block to be inspected is embedded is the DCT coefficient value of the neighborhood of the second DCT coefficient. It means for removing sieved by whether the value of the predetermined range, the second DCT coefficients is determined to have an abnormal value by the sieve relative to the neighboring DCT coefficient values of the DCT coefficients,
When the absolute value of the value of the first DCT coefficient is different from the absolute value average value of the second DCT coefficient value from which the DCT coefficient having the abnormal value is removed by a predetermined threshold or more, the watermark data is A moving image digital watermarking device comprising: means for determining that the image is embedded.

Images