JP2002191033A - Electronic watermark imbedding device, and electronic watermark imbedding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic watermark imbedding technology by which an electronic watermark is not missing even when compression is applied to image data in compliance with the MPEG 2 or the like while keeping deterioration in the image data within a permissible range. SOLUTION: An electronic watermark imbedding means, an MPEG 2 encoding means and an electronic watermark detecting means are operated in cooperation. Concretely the electronic watermark imbedding device is configured such that the electronic watermark imbedding means receives information with respect to electronic watermark data from MPEG 2 stream image data with electronic watermarks imbedded to them outputted from the MPEG 2 encoding means, especially information with respect to the detected quantity of the electronic watermark data and controls parameters such as the total amount of watermarks to be imbedded, imbedding strength and imbedding positions on the basis of the generated data quantity information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital watermark embedding apparatus for embedding digital watermark data in image data and a method for embedding a digital watermark, and particularly has a function of performing digital watermark embedding processing in combination with MPEG2 image format conversion processing. It relates to an image data processing device.

[0002]

2. Description of the Related Art Techniques for embedding other data in images, sounds, text data, and the like are also called data hiding, watermarks, digital watermarks, and the like, and include identification information and annotations related to these images, sounds, and text data. It has been studied so far as a technology for embedding.

Recently, data for protecting the copyright of such information is embedded in original data to be protected, and is also applied as a technique for preventing illegal operations such as creation of an illegal copy. It has become. These technologies are described in "Nikkei Electronics" (Nikkei Electronics, Inc., P149-P162, 1997.2.24).

If the data to be embedded is degraded by processing operations such as filtering or data compression on the original data, the function of preventing illegal operations will not work, so that even if these processings are performed, the data is effective. We are devising a method of embedding to compete.

[0005] In addition, various situations can be considered with respect to copy restrictions, such as original data that can be freely copied due to copyright, data that is prohibited to be copied, and data that can be copied only once. Therefore, it has been considered to embed corresponding data in accordance with these situations, and to control the copy by detecting this.

[0006]

Conventionally, image data has been processed by an image data compression method represented by MPEG2 or the like and then recorded on an optical disk or a hard disk for use. The reason is that (1) the original image data has a huge amount of information and it is not practical to handle it without compression, and (2) the image quality that the image data compression method represented by MPEG2 or the like can withstand practically That is, the data amount can be reduced to several tenths within the range of deterioration.

Here, specifically, DVD (Digital Vers
atile Disc) Compression (encoding) / expansion (decoding) of image data by MPEG2, which is practically used for Video etc.
It is assumed that the processing is applied to image data in which a digital watermark is embedded.

[0008] The digital watermark is used to protect the copyright of image data as described above. Therefore, a digital watermark must be detected even from image data that has undergone MPEG2 encoding processing for storing image data and decoding processing for reproduction. Therefore, the embedded digital watermark is not allowed to disappear by the normal compression / decompression processing by MPEG2. However, the image data compression processing of MPEG2 is an irreversible encoding method, and loss of information is generally unavoidable. In particular, MPEG2 uses a method to reduce the amount of data by reducing the accuracy of the data of the high-frequency component among the various spatial frequency components of the image by rounding, so if the digital watermark has high-frequency components, the digital watermark The accuracy of the data is reduced, and in the worst case it is lost. From the above, it is expected that the problem of lowering the reliability of digital watermark detection will occur due to the MPEG2 processing.

On the other hand, since embedding of a digital watermark is a process of modifying the original image data, deterioration of the image quality is inevitable. Therefore, there is a limit to increasing the embedding of digital watermarks in order to address the problem of the reduction in the reliability of digital watermark detection due to the above-described MPEG2 processing due to the problem of image quality deterioration. Therefore, in embedding the digital watermark, it is necessary to reliably detect the digital watermark while keeping the deterioration of the image quality within an allowable range.

In summary, it is a necessary condition when using a digital watermark to simultaneously solve the problem of the reduction in the reliability of the digital watermark detection by the MPEG2 processing and the problem of the image quality deterioration.
However, conventional digital watermark embedding processing and MPEG2
The processing is completely different processing, and the processing was not performed in consideration of the influence of both processing techniques. Therefore, it has actually been difficult to always satisfy the above-mentioned necessary conditions.

[0011] Therefore, developing and putting into practical use a digital watermarking technique that does not disappear even if image data compression processing such as MPEG2 is performed while keeping image deterioration within an allowable range protects the copyright of image data. This is a major challenge.

[0012]

In order to solve the above problems, the present invention combines a digital watermark embedding process, an MPEG2 encoding process, and a digital watermark detecting process, and detects a digital watermark from image data after the MPEG2 encoding process. By changing the content of the digital watermark embedding process from the digital watermark detection result, the problem of the reliability of digital watermark detection and the problem of image quality deterioration can be simultaneously solved.

First, a conventional digital watermark embedding process will be described. Digital watermark (watermark is Wate in English
Image data (hereinafter abbreviated as WM) which is not embedded (hereinafter abbreviated as WM) is input to a WM embedding device, where WM is embedded to become image data with WM. Is output. The obtained WM-attached image data is input to an MPEG2 encoding processor for the purpose of data compression for recording on a recording medium, where it is compressed and the WM-attached image data is converted into an MPEG2 stream. Therefore, WM embedding processing and MPEG2
The encoding processes were independent and independent of each other.

Therefore, in the present invention, WM embedding processing, MPE
The G2 encoding process and the newly added WM detection process are performed in conjunction with each other. That is, the original image data is input to the WM embedding device, where the WM is embedded and output as WM image data. W at this time
In the M embedding process, the WM embedding device performs information on the status of the WM detected by the WM detection device from the image data after the MPEG2 encoding process at that time, and particularly information on the detection amount of the WM data (hereinafter, the detected WM).
(Abbreviated as data amount information) from the WM detection device, and based on the detected WM data amount information, controls parameters such as the total amount of WM to be embedded, embedding strength, and embedding position.

[0015] Thereby, the WM embedding processing device can operate the MP.
When the loss of WM data occurs due to the influence of the EG2 encoding process, the phenomenon is detected through the detected WM data amount information, and the total amount of the WM data to be embedded, the embedding strength, the embedding position, and the like are changed to prevent the loss of the WM data. Function.

As a result, it is possible to prevent the embedded WM from being lost, and it is guaranteed that the embedded WM will not be lost. Therefore, it is not necessary to embed the WM more than necessary, and as a result, it is possible to minimize the deterioration of the image quality. It becomes possible.

[0017]

Embodiments of the present invention will be described below with reference to the drawings.

Here, for the sake of explanation, an example of MPEG2 encoding processing will be used as a means for compressing image information. However, this is not limited to MPEG2 encoding processing, and a method based on discrete cosine transformation such as MPEG1 or MPEG4 is used. The present invention can be applied to various image data compression methods using a method for reducing the data amount of high frequency components, such as a method based on other wavelet transform.

Also, an example is used in which the WM data to be embedded is copy control information for copyright protection, but this is of course not limited to copy control information.
Information indicating the type of the original image data or related information attached to the original data may be used.

FIG. 1 shows an embodiment of a configuration for realizing a digital watermark embedding apparatus, a format conversion apparatus having a digital watermark embedding function, and a digital watermark embedding method according to the present invention.

In FIG. 1, 1 is original image data, 2 is WM information to be embedded in the original image data, 3 is baseband WM-attached image data with a digital watermark embedded by a WM embedding means 10 (described later), and 5 is MPEG2. The WM-attached image data of the MPEG2 stream output from the encoding unit 20, and 6 is WM information detected by the WM detection unit 30.

Reference numeral 10 denotes WM embedding means, which is an image data analyzing means 11 for analyzing the properties of the input image data 1 and outputting image analysis data 17, and generating original WM data 15 from the input WM information 2. WM generation means 12,
The detected original WM data 15 corresponds to the detected original WM data 6,
WM embedding control means 14 for performing control using image analysis data 17 to generate embedded WM data 16 for actually embedding in original image data 1, input image data
The WM data adding means 13 adds the embedded WM data 16 to 1 and outputs the baseband WM-attached image data 3.

Reference numeral 20 denotes MPEG2 encoding means, which encodes the input baseband WM-attached image data 3 using an algorithm prescribed in MPEG2 and outputs MPEG2 stream WM-attached image data 5.

Numeral 30 denotes a WM detecting means, which receives an inputted MP.
MPEG2 decoding means 31 that decodes the EG2 stream image data 5 with WM using an algorithm specified by MPEG2 and outputs detected image data 34. Detects a digital watermark from the input detected image data 34 and detects WM.
An extraction / separation unit 32 that outputs data 35 and a WM data amount detection unit 33 that detects an embedded amount of WM data in the input detected WM data 35 and outputs detected WM information 6.

Next, an outline of WM embedding in this embodiment will be described.

The original image data 1 is digital signal image data, and there are various formats. Here, as an example, one frame of the image is composed of 720 pixels in the horizontal direction and 480 pixels in the vertical direction, and each frame expresses the image with a total of three types of data of a luminance component Y and color components Cr and Cb. , Each pixel of Cb data is 1
Description will be made using a format represented by 0 bits.

The WM information 2 is, for example, Co
There are types such as py Free (no restriction on the number of copies), Never Copy (copy prohibited), Copy Once (copy permitted only once), and any one of these is specified. W
The M generation means 12 generates original WM data 15 from the specified WM information 2.

Here, the WM data in the digital watermarking technique will be described.

The WM data embeds information in the original image data by adding a slight modification to the original image data and giving meaning to the content of the modification at this time. At the time of WM detection, only the modification added to the original image is separated and extracted, and the embedded information is extracted from the content of the modification.

In embedding the WM data in the original image data 1, the WM data can be embedded only in the portion where the image quality hardly deteriorates even when the WM is embedded, in order to minimize the image quality deterioration. Therefore, all the generated original WM data 15 cannot be embedded in the image data 1. Further, since the WM data is lost due to the MPEG2 encoding process, it is also difficult to detect all the embedded WM data. Therefore, in creating the original WM data 15, information necessary for restoring the WM information 2 is repeated at a plurality of locations in the screen so that the WM information 2 can be correctly restored even from some detected WM data. Embedded.

The image data analysis means 11 is provided for the purpose of minimizing the deterioration of the original image due to the WM embedding processing, in order to examine the position where the image deterioration is not conspicuous even when the WM is embedded and the level of the WM which can be embedded therein. Used. This is because, for example, when a small change in luminance or hue is given to the original image by WM embedding, the change is conspicuous in a portion where the original image has a uniform luminance distribution and little movement, but the luminance distribution of the original image is uniform. This is because the WM is embedded in the original image data that is in a state that is less noticeable as much as possible by focusing on the characteristic of human vision that the change is less noticeable in a portion where the movement is large.

Therefore, the image data analysis means 11 specifically includes the information amount of the image data, the individual pixel data values in the screen, the variance of the pixel data values, the magnitude of the spatial / temporal change of the pixel data values, Image data analysis is performed using various parameters such as the direction and magnitude of the motion vector, and the analyzed result is output as image analysis data 17.

The WM embedding control means 14 creates, based on the image analysis data 17, control data relating to a position on the screen which is hardly noticeable even when the WM is embedded and the strength of the WM to be embedded therein. For example, in a contour portion in which luminance rapidly changes in the original image, even if the WM data is embedded and changed in this portion, the WM data is buried in a change in the original image, so that the WM becomes less noticeable. Also, the larger the change in the brightness of the contour portion and the larger the change width, the larger the WM data can be embedded. Therefore, the control data created based on these rules is used as a weighting coefficient for the original image data 15 to embed the control data in the original WM data 15 to create the embedded WM data 16.

In the actual creation of the embedded WM data 16, for example, three sets of frame memories each having a capacity of one frame of the original image data are used, the first frame memory has control data, and the second frame memory has The original WM data and the third frame memory are configured to store the embedded WM data. In operation, when attention is paid to a specific pixel, after data is stored in both the first frame memory that stores control data of the pixel and the second frame memory that stores original WM data of the pixel, The data corresponding to the pixel in both frame memories is read, and the result of multiplying the two read data is stored in the area corresponding to the pixel in the third frame memory. By performing this process for all pixels of the frame, one frame of embedded WM
Data can be created.

Of course, the method of creating the embedded WM data 16 is not limited to the above-mentioned method. For example, instead of storing in the third frame memory, the required number of frame memories is stored in the first or second frame memory. It can be reduced.

The embedded WM data 16 created by the above method is converted by the WM data adding means 13 into the original image data.
The image data 3 with WM is generated by adding 1 to each corresponding pixel.

The WM embedding control means 14 has a WM
The detection WM information 6 is also input from the detection means 30. The reason will be described later.

Next, the MPEG2 encoding process will be described.

In the encoder of the MPEG2 encoder,
First, discrete cosine transform (Discrete C
After performing a DCT (sine transform) process, a quantization (rounding) process is performed. The data obtained as a result of the quantization process is quantized at the quantization scale value used at that time, and data with a change in the data value less than the quantization scale value loses the difference due to the rounding process. Since the data is converted into the same data, the information amount is compressed (reduced) here.

In general, the size of the WM data (ΔNWM) is set to a value sufficiently smaller than the original image data value (NPICTURE) in order to prevent image quality deterioration due to alteration of the original image data (NPICTURE>> ΔNWM). For this reason, the quantization scale value (ΔNQS) in the rounding process of the normal MPEG2 encoding process is often larger than the size of the WM data (ΔNWM), and as a result, most of the WM data is lost in the quantization process. Will do.

In the quantization process, the probability that WM data will remain is ΔNWM / ΔNQS, and the probability of disappearance is 1−ΔNWM
/ ΔNQS. As described above, since the WM data is repeatedly embedded in anticipation of the loss in the quantization process,
Normally, the WM data can be correctly read from the remaining WM data without being lost.
It is set so that information can be restored.

However, when an image having a larger amount of information than normal is continuous in the image data 1, the MPEG2 encoder increases the quantization scale value (ΔNQS) because the generated information amount needs to be lower than a predetermined bit rate. Try to reduce the amount of data output from the MPEG2 encoder. As a result, W
When the loss of M data increases and exceeds the WM detection limit, WM
The state where the information cannot be restored continues, causing a problem.

Therefore, in the present invention, to solve the above problem, the detected WM information 6 output from the WM detecting means 30 is also provided to the WM embedding control means 14 so that the state of the WM detected from the image data after MPEG2 encoding is obtained. Embedded WM data 16 can be controlled in accordance with. For example, when the WM amount detected from the detected WM information 6 decreases below a threshold required for WM detection, the WM embedding unit 10
Performs WM level correction control by changing the embedded WM data 16 so as to prevent the WM embedded in the image data after MPEG2 encoding from falling below a threshold necessary for detection.

Note that, in addition to the WM level correction control, an embedding position control for changing a position on the screen where WM embedding is performed according to the size of the WM data to be embedded may be used together.

Here, the embedding position control is for changing the WM embedding position on the screen according to the state of the WM data, and is a process for making the image quality deterioration due to the WM embedding less noticeable.

In general, when a change in luminance, hue, or the like is large in an original image, even if the screen data is further modified by WM embedding, it is hard to perceive a change in the original image. Conversely, when changes in luminance, hue, and the like in the original image are small, if the screen data is further modified by WM embedding, the change due to WM is likely to be perceived because the original image is uniform.

Therefore, for example, when the embedding amount is increased, the image quality is likely to be degraded due to the increase in the WM embedding amount.
The amount of embedding into a portion where there is little change in hue or the like is reduced. Conversely, the embedding position control may be used to increase the embedding amount in a contour portion having a change in luminance and hue. Further, the processing for reducing the amount of embedding in a portion having little change and the process for increasing the amount of embedding in a contour portion having change may be simultaneously performed by using both of them. In addition, since a flat portion having a small change and a contour portion having a large change can be identified by using a feature detection function widely performed in image processing technology,
Here, detailed description is omitted.

The embedding amount, embedding strength, and embedding position of a specific frequency component of the WM data may be controlled in accordance with the image information signal. For example, if it is found from the image information amount signal that a specific spatial frequency component is large, embedding a WM having a frequency close to this spatial frequency causes interference between the two to form an interference fringe, and the WM becomes more conspicuous, causing a problem. I do. Therefore, in such a case, as a countermeasure, (1) the original WM data has a spatial frequency close to the spatial frequency included in the original image data, and lowers the intensity of a specific spatial frequency component causing interference fringes.
(2) The spatial frequency spectrum of the original image data is examined in each of the divided regions of the screen, and the WM embedding intensity is reduced only in the region where interference fringes may occur. (3) The spatial frequency spectrum of the original image data is displayed on the screen. Is examined in each area divided into a plurality of areas, and in an area where an interference fringe may be generated, an area in which the intensity of a specific spatial frequency component causing the interference fringe is reduced from the original WM data is used. It is necessary to take measures.

Therefore, the WM data is separated into a plurality of bands on the spatial frequency axis in advance, and even if the WM data is superimposed on the main spatial frequency components of the original image data obtained from the image information amount signal, the occurrence of interference fringes, etc. WM embedding is performed by adding only the WM of the spatial frequency band component that does not have the problem described above, or the WM of the spatial frequency band component in which interference fringes may occur is smaller than the WM of other spatial frequency band components WM embedding is performed by multiplying and adding a coefficient.

The spectrum of the spatial frequency of the WM data is broad in general, and the energy is not concentrated on a specific spatial frequency. Therefore, even if only a specific spatial frequency component is reduced from the WM data, the WM detection ability is not adversely affected.

Although increasing the WM embedding amount usually tends to adversely affect the image quality, the above processing is performed in a situation where the embedded WM data 16 is easily lost in the MPEG2 encoding process. W under
The increase of the M embedding amount is due to the WM by the MPEG2 encoding process.
Since the amount of WM data embedded in the MPEG2 stream does not change much as a result of offsetting by the decrease in data,
The problem of image quality deterioration hardly occurs.

As described above, the digital watermark embedding means, the MPEG2 encoding means, and the WM detecting means are configured in cooperation with each other, so that the embedded WM can be prevented from being lost by the MPEG2 encoding, and the image quality degradation can be prevented. Can be implemented. The combination of the means for realizing each function described in the present embodiment is not limited to this, and another combination may be used.
Means for realizing each function may be realized by an electronic circuit or a combination of a microprocessor and software.

[0053]

According to the present invention, the digital watermark embedding can be controlled so that the digital watermark is hardly lost under the condition that the digital watermark is easily lost by the rounding processing of MPEG2. The problem of deterioration can be solved, and at the same time, it is not necessary to embed excessive WM, so that the problem of image quality deterioration can be solved.

Thus, it is possible to develop and put into practical use a digital watermarking technique which is less likely to be lost even if image data compression processing such as MPEG2 is performed, while keeping image deterioration within an allowable range. It is possible to construct a practical digital watermarking system for protecting rights.

[Brief description of the drawings]

FIG. 1 is a diagram showing an embodiment of a configuration for realizing a digital watermark embedding device, a format conversion device having a digital watermark embedding function, and a digital watermark embedding method according to the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Original image data, 2 ... WM information, 3 ... Image data with WM (baseband), 5 ... Image data with WM (MPEG2 stream), 6 ... Detected WM information, 10 ... WM embedding means, 11 ... Image data analysis Means 12 WM generating means 13 WM data adding means 14 WM embedding control means 15 Original WM data 16 Embedding WM data 17 Image analysis data 20 MPEG2 encoding means 30 WM detection Means: 31: MPEG2 decoding means, 32: WM data extraction / separation means, 33: WM data detection means, 34: detected image data, 35: detected WM data.

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Michihiro Tateishi 292 Yoshida-cho, Totsuka-ku, Yokohama-shi, Kanagawa Prefecture F-term in Hitachi Digital Media Development Division (Reference) 5B057 CA01 CA08 CA12 CA16 CB01 CB08 CB12 CB16 CB19 CC02 CE08 CE09 CG01 CG07 DA07 DA08 DA17 5C063 AC01 AC10 CA40 5C076 AA14 BA06

Claims (15)

[Claims] An electronic watermark embedding apparatus for performing processing for embedding an electronic watermark in image data, an electronic watermark data creating means for creating electronic watermark data to be embedded in the image data, and Digital watermark data embedding means for embedding in data, digital watermark information detecting means for detecting information relating to the digital watermark from image data in which the digital watermark is embedded, and outputting a digital watermark information signal; Controlling an amount of embedding digital watermark data in the image data according to the digital watermark information signal. 2. An electronic watermark embedding device for performing processing for embedding an electronic watermark in image data, an electronic watermark data creating means for creating electronic watermark data to be embedded in the image data, and Digital watermark data embedding means for embedding in data, digital watermark information detecting means for detecting information relating to the digital watermark from image data in which the digital watermark is embedded, and outputting a digital watermark information signal; Controlling the position of embedding digital watermark data in the image data in accordance with the digital watermark information signal. 3. An electronic watermark embedding apparatus for performing processing for embedding an electronic watermark in image data, an electronic watermark data creating means for creating electronic watermark data to be embedded in the image data, and Digital watermark data embedding means for embedding in data, digital watermark information detecting means for detecting information relating to the digital watermark from image data in which the digital watermark is embedded, and outputting a digital watermark information signal; Controlling the intensity of embedding digital watermark data in the image data in accordance with the digital watermark information signal. 4. The digital watermark embedding device according to claim 1, wherein said digital watermark embedding means controls an amount of embedding digital watermark data of a specific spatial frequency component in accordance with said digital watermark information signal. Digital watermark embedding device. 5. A digital watermark embedding apparatus according to claim 2, wherein said digital watermark embedding means controls a position at which digital watermark data of a specific spatial frequency component is embedded according to said digital watermark information signal. Digital watermark embedding device. 6. A digital watermark embedding device according to claim 3, wherein said digital watermark embedding means controls the intensity of embedding digital watermark data of a specific spatial frequency component according to said digital watermark information signal. Digital watermark embedding device. 7. The digital watermark embedding apparatus according to claim 1, wherein said digital watermark information detecting means is adapted to perform the ISO / IEC standard 13818-2 after embedding the digital watermark.
An electronic watermark embedding device, which detects and outputs information relating to an electronic watermark from image data that has undergone image data compression processing (hereinafter abbreviated as MPEG2). 8. The digital watermark embedding device according to claim 1, wherein said digital watermark embedding means is configured to embed the digital watermark signal when the digital watermark information amount indicated by said digital watermark information signal increases. A digital watermark embedding device, which controls so as to decrease within a predetermined range. 9. An electronic watermark embedding device according to claim 1, wherein said electronic watermark embedding means includes an amount of embedding an electronic watermark signal when the amount of electronic watermark information indicated by said electronic watermark information signal decreases. Is controlled so as to increase within a predetermined range. 10. The digital watermark embedding device according to claim 2, wherein the digital watermark embedding unit embeds the digital watermark signal when the digital watermark information amount indicated by the digital watermark information signal decreases. The digital watermark embedding device is characterized in that the digital watermark is changed to a position where the digital watermark is hardly lost. 11. A digital watermark embedding device according to claim 2, wherein said digital watermark embedding means embeds a digital watermark signal when an amount of digital watermark information indicated by said digital watermark information signal increases. The digital watermark embedding device is characterized in that the digital watermark is changed to a position where the digital watermark is hardly noticeable. 12. The digital watermark embedding device according to claim 3, wherein said digital watermark embedding means includes a digital watermark signal embedding strength when the digital watermark information amount indicated by said digital watermark information signal increases. A digital watermark embedding device, which controls so as to decrease within a predetermined range. 13. The digital watermark embedding device according to claim 3, wherein said digital watermark embedding means includes a digital watermark signal embedding strength when the digital watermark information amount indicated by said digital watermark information signal decreases. Is controlled so as to increase within a predetermined range. 14. A digital watermark embedding device according to claim 1, wherein said digital watermark embedding means converts the image data after the digital watermark embedding to an MP.
An electronic watermark embedding device, which operates in cooperation with a format conversion device for converting to an EG2 format. 15. A digital watermark embedding method for embedding a digital watermark in image data, wherein the digital watermark data is embedded in the image data, the digital watermark data is embedded in the image data, and the digital watermark is embedded. Detecting information related to a digital watermark from the detected image data, and controlling either the amount or the intensity or the embedding position of the digital watermark data in the image data according to the detected information related to the digital watermark. Digital watermark embedding method.