JP2000032406A - Electronic watermark insertion system to digital signal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic watermark system, where the detection rate is improved while suppressing deterioration in quality of a digital signal by taking deterioration in an image due to irreversible compression into account so as to insert an electronic watermark to the digital signal. SOLUTION: A linear conversion circuit 50 applies linear conversion to a digital signal to calculate an original signal conversion coefficient. A quantizer 51 quantizes the original signal conversion coefficient and provides the output of a quantized value and quantization information, and an inverse quantizer 53 applies inverse quantization to the quantized value to obtain an inversely quantized value. A unit visual sense/ audible sense distortion calculation circuit 55 calculates unit visual sense/audible sense distortion for each original signal conversion coefficient, a watermark signal generating circuit 54 calculates a watermark signal based on the original signal conversion coefficient, electronic watermark information, the inversely quantized value, the quantization information and the unit visual sense/audible sense distortion. In this case first the circuit 54 calculates an object signal to insert the electronic watermark, then calculates a watermark signal to correct the quantized value, an adder 5 adds the watermark signal to the quantized value, and a code converter 56 converts the result of sum into a code string.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the insertion of a digital watermark into a digital signal with lossy compression coding.
The present invention relates to a system for performing the same conversion as that used in encoding to insert a digital watermark.

[0002]

2. Description of the Related Art As this kind of conventional digital watermark insertion system for images, for example, reference 1 (January 1997, Encryption and Information Security Symposium (SCIS97) draft SCIS97-26A, "Digital image copyright protection"). The digital watermarking method in the frequency domain ”) is known. In this method, the original image is
The block is divided into blocks of a fixed size such as × 8, and each block is subjected to DCT (discrete cosine transform) to obtain a D
An electronic watermark is inserted in the CT coefficient. Specifically, D
Watermark information is inserted by rounding the CT coefficient D based on the digital watermark strength h. If the information to be inserted is 0, the DCT coefficient D

[0003] Round like

On the other hand, if the information to be inserted is 1,
DCT coefficient D

[0005] Round like

When detecting an embedded digital watermark, first, an image is divided into blocks of a fixed size such as 8.times.8, and DCT is performed on each block to obtain a DCT coefficient D '. Next, the quantization residual P when D ′ is quantized by the watermark strength h

[0007] To obtain embedded watermark information. At this time, If so, the embedded information is determined to be 0, If so, the embedded information is determined to be 1.

[0008] Insertion and detection of a digital watermark are performed in this manner. To which DCT coefficient a digital watermark is to be inserted is determined.
It is determined by a random number, and this random number is used as a key. Also,
The same watermark information is repeatedly inserted into the entire image to increase the durability. In this method, human visual characteristics are not taken into account at the time of insertion.

As a method using wavelet transform instead of DCT, for example, Reference 2 (February 1998,
CQ described in IEICE Technical Report Vol. 97, No. 565, pp. 37-42
97-76 "Digital watermarking method for digital image by multi-resolution analysis") is known. In this method, after an original image is decomposed into multiple resolutions by wavelet transform, a digital watermark is inserted into a low-frequency coefficient at a position where the power of a high-frequency coefficient becomes high. That is, the high frequency coefficient 2
A sum of squares is obtained, a low-frequency coefficient at which the sum is equal to or larger than a threshold is selected, and a digital watermark is inserted. The insertion method is described in the aforementioned document 1.
It is similar to the “digital watermarking scheme in the frequency domain for copyright protection of digital images”. If the information to be inserted is 0, the wavelet transform coefficient is set to be an even multiple of the watermark strength h. Round it in. On the other hand, if the information to be inserted is 1, the wavelet transform coefficient is rounded so as to be an odd multiple of the watermark strength h.

In each of the above-described conventional digital watermark insertion methods, a digital watermark is directly inserted into an original image, and irreversible compression is not taken into account when the digital watermark is inserted. That is, the insertion of the digital watermark and the lossy compression are completely independent processes. In the case where a digital watermark is inserted by these methods and then irreversible compression is performed, the following is performed.

FIG. 7 shows a conventional method of inserting a digital watermark,
On the other hand, it is a block diagram illustrating an example of the configuration of an electronic watermark insertion and encoder when irreversible encoding based on DCT is performed.

Referring to FIG. 7, a digital watermark inserter 10 is provided.
Inserts a digital watermark into the input image,
Output to The image encoder 11 includes the digital watermark inserter 1
The image input from 0 is encoded, a code string is generated and output.

Further, the digital watermark inserter 10 has a DCT
Circuit 1, digital watermark insertion circuit 2, and inverse DCT (Inve
The DCT circuit 1 performs DCT on an input image and outputs DCT coefficients to a digital watermark insertion circuit 2. The digital watermark insertion circuit 2 calculates a watermark signal based on the watermark information, puts the watermark signal on the DCT coefficient output from the DCT circuit 1, and generates an inverse DCT coefficient as the watermarked DCT coefficient.
Output to the T (Inverse DCT) circuit 3. Inverse DCT circuit 3
Is the watermarked D output from the digital watermark insertion circuit 2.
Inverse DCT is performed on the CT coefficient to output a watermark inserted image. Then, the watermark inserted image is output to the image encoder 11.
Is input to the DCT circuit 4.

The image encoder 11 includes a DCT circuit 4, a quantizer 5, and a code converter 6.
The DCT circuit 4 performs DCT on the digital watermark inserted image output from the inverse DCT circuit 3 and converts the DCT coefficient into a quantizer 5.
Output to The quantizer 5 quantizes the DCT coefficient output from the DCT circuit 4 and outputs a quantized value to the code converter 6. The code converter 6 performs variable length coding on the quantized value output from the quantizer 5 and outputs a code string.

Next, the operation of the circuit shown in FIG. 7 will be described. The input image is decomposed into 8 × 8 blocks, and D
In the CT circuit 1, DCT is performed on each block, and 8 × 8 DCT coefficients are output. The digital watermark insertion circuit 2 calculates a watermark signal from the input watermark information and adds it to the 8 × 8 DCT coefficient. This digital watermark insertion method includes, for example, a method described in the above-mentioned document 1 “Digital watermarking method in frequency domain for copyright protection of digital images”. Then, an 8 × 8 DCT coefficient into which the digital watermark has been inserted is obtained. The 8 × 8 DCT coefficient containing the watermark information is converted by the inverse DCT circuit 3 into an 8 × 8 DCT coefficient.
Inverse DCT is performed every eight blocks, and an image with a digital watermark inserted is obtained.

The image obtained in this way is input to the image encoder 11, and first, an 8 × 8 DC
T is performed and a DCT coefficient is output. This DCT coefficient is quantized in the quantizer 5. For example,
When performing JPEG encoding, the quantization width of each DCT coefficient is determined based on the quantization table and is quantized. Then, the obtained quantized value is one-dimensionally scanned by the code converter 6 by zigzag scanning or the like, and then converted into a code string by Huffman coding or arithmetic coding and output. When generating a code string, additional information such as a quantization parameter is multiplexed together.

FIG. 8 shows a conventional method of inserting a digital watermark,
FIG. 3 is a block diagram showing an example of the configuration of an electronic watermark insertion and encoder when irreversible encoding such as MPEG is performed. In FIG. 8, a digital watermark inserter 10 inserts a digital watermark into an input image and outputs the image to an image encoder 40. The digital watermark inserter 10 has the same configuration as the digital watermark inserter 10 shown in FIG. The image encoder 40 encodes an image input from the digital watermark inserter 10, generates a code string, and outputs the code string. The image encoder 40 includes a subtracter 21, a DCT 22, a quantizer 23, an inverse quantizer 25, an inverse DCT 26, an adder 27, a frame memory 28,
It comprises a code converter 24.

The watermark inserted image output from the inverse DCT circuit 3 of the digital watermark inserter 10 is input to the subtracter 21 of the image encoder 40. The subtracter 21 is used for the inverse DCT circuit 3
From the watermark inserted image output from the
The motion compensated prediction image output from 8 is subtracted, and a prediction error image as a difference signal is output to the DCT circuit 22.

The DCT circuit 22 performs DCT on the prediction error image signal output from the subtracter 21 and outputs a DCT coefficient to the quantizer 23. The quantizer 23 is a DCT circuit 2
2 is quantized, and the quantized value is output to the code converter 24 and the inverse quantizer 25. The code converter 24 performs variable length coding on the quantized value output from the quantizer 23 and outputs a code string. Inverse quantizer 25
Dequantizes the quantized value output from the quantizer 23 and outputs the dequantized value to the inverse DCT circuit 26. Inverse DC
The T circuit 26 performs an inverse DCT on the inversely quantized value output from the inverse quantizer 25, and outputs a locally decoded prediction error image to the adder 27. The adder 27 adds the local decoded prediction error image output from the inverse DCT circuit 26 to the frame memory 2.
8 and outputs the locally decoded image to the frame memory 28 by adding the motion compensated prediction image. The frame memory 28 performs motion compensation on the locally decoded image output from the adder 27, and outputs a motion-compensated predicted image to the subtractor 21 and the adder 27.

Next, the operation of the circuit shown in FIG. 8 will be described. In the digital watermark inserter 10, a digital watermark is inserted into an input image, and a digital watermark inserted image is generated and output. The operation of the digital watermark inserter 10 is the same as that shown in FIG.

The digital watermark inserted image obtained by the digital watermark inserter 10 is input to a moving picture encoder 40, and the subtractor 21 reduces the motion compensated predicted image output from the frame memory 28, thereby obtaining a prediction error image. Is found. The operation of the frame memory 28 will be described later. The DCT circuit 22 performs DCT on the obtained prediction error image for each 8 × 8 block, and outputs DCT coefficients. This DCT
The coefficients are quantized in a quantizer 23. For example, when performing MPEG coding, the quantization width of each DCT coefficient is determined and quantized based on a quantization matrix and a quantization scale value determined for each macroblock. Then, the obtained quantized value is one-dimensionally scanned by the code converter 24 by zigzag scan, alternate scan, or the like, and then converted into a code string by Huffman coding or arithmetic coding, and output. When generating a code string, additional information such as a quantization parameter is multiplexed together. Furthermore, the quantized value obtained by the quantizer 23 is inversely quantized by the inverse quantizer 25, and inverse DCT is performed for each 8 × 8 block by the inverse DCT circuit 26, thereby obtaining a local decoded prediction error image.

In the adder 27, the motion-compensated predicted image output from the frame memory 28 is added to the locally decoded error image to obtain a locally decoded image. This is input to the frame memory 28 and stored. In the frame memory 28, a locally decoded image stored in the past is stored, and motion compensation is performed using this as a reference image, and a motion-compensated predicted image is generated and output.

In the case of the above-mentioned document 2 “Digital watermarking method for digital image by multi-resolution analysis” using the wavelet transform instead of the DCT, the same can be said if the above-mentioned DCT part is changed to the wavelet transform.

[0024]

The above-mentioned conventional method has a problem that when lossy compression encoding is performed after inserting a digital watermark, the watermark detection accuracy is significantly deteriorated.

The reason for this is that when the digital watermark is inserted, the compression of the digital signal is not considered. Even after irreversible compression, it is necessary to increase the strength of the inserted digital watermark in order to be able to detect the digital watermark. However, when the intensity is increased, there is a problem that the quality of the digital signal is reduced.

Accordingly, the present invention has been made in view of the above-mentioned problems, and a main object of the present invention is to suppress deterioration of digital signal quality by inserting a digital watermark in consideration of deterioration due to lossy compression. Another object of the present invention is to provide a digital watermark system that improves the detection rate.

It is another object of the present invention to provide a digital watermark insertion system in which the visual / audible deterioration due to the watermark insertion is small by considering the visual / audible characteristics of a human.

[0028]

According to a first aspect of the present invention, which achieves the above object, a digital signal is encoded and a digital watermark is inserted at the same time, and a linear conversion is performed to project the digital signal onto frequency components. Means for outputting an original signal transform coefficient, quantizing the original signal transform coefficient, outputting a quantized value as a watermarkless quantized value, and outputting quantization information; and inverting the watermarkless quantized value. Means for quantizing and outputting a watermark-free decoded signal transform coefficient; and calculating and outputting a unit visual / auditory distortion corresponding to each of the original signal transform coefficients from the original signal transform coefficient based on a visual / auditory model. Means for calculating a target signal from the watermarkless decoded signal transform coefficient, the original signal transform coefficient, the quantization information, the unit visual / audible distortion, and the digital watermark information, Further, based on the unit visual / audible distortion, a quantization value correction for correcting the watermarkless quantization value so as to reduce the visual / audible distortion of the entire signal and to increase the detection rate of the digital watermark. Means for determining a signal from the target signal and outputting this as a watermark signal;
Adding the watermarkless quantized value and the watermark signal,
It is characterized by having means for outputting a watermarked quantized value, and means for generating and outputting a code string from the watermarked quantized value.

The second invention of the present application is a method of encoding a picture and inserting a digital watermark at the same time as performing a motion compensation on a locally decoded picture to generate and output a motion compensated prediction picture, and an input picture. Means for obtaining and outputting a prediction error image by subtracting the motion-compensated prediction image from, and means for performing linear transformation to project the prediction error image onto a spatial frequency component and obtaining and outputting a prediction error image conversion coefficient. Means for quantizing the prediction error image transform coefficient, outputting the quantized value as a watermarkless quantized value, and outputting quantization information, and inversely quantizing the watermarkless quantized value, Means for outputting a quantized value, means for performing the linear transformation on the motion-compensated predicted image, and calculating and outputting a predicted image transform coefficient, the watermarkless dequantized value and the predicted image transform coefficient A means for outputting a watermark-free decoded image transform coefficient, a means for performing the linear transformation on the input image, obtaining and outputting an original image transform coefficient, based on a visual model from the original image transform coefficient, Means for calculating and outputting a unit visual distortion corresponding to each of the original image transform coefficients, from the watermarkless decoded image transform coefficient, the original image transform coefficient, the quantization information, the unit visual distortion, and the digital watermark information. , A target signal is calculated, and further, based on the unit visual distortion, a quantized value correction signal for correcting the quantized value without watermark so as to visually reduce distortion as a whole image and increase the detection rate of a digital watermark. From the target signal and outputting the watermarked signal as a watermark signal, and adding the watermarkless quantized value and the watermark signal to output a watermarked quantized value. Means for generating and outputting a code string from the watermarked quantized value, means for dequantizing the watermarked quantized value, and means for outputting a watermarked dequantized value; and Means for performing an inverse transformation of the linear transformation, obtaining and outputting a local decoding prediction error image, and means for adding the motion compensation prediction image to the local decoding prediction error image and outputting a result as the local decoding image. It is characterized by having.

The third invention of the present application is a method of encoding a picture and inserting a digital watermark at the same time as performing a linear transformation for projecting an input picture onto a spatial frequency component, obtaining an original picture transform coefficient, and outputting it. Means, performing motion compensation on the locally decoded image, generating and outputting a motion-compensated predicted image, and means for performing the linear transformation on the motion-compensated predicted image, obtaining and outputting a predicted image conversion coefficient, Means for subtracting the predicted image conversion coefficient from the original image conversion coefficient and outputting the result as a predicted error image conversion coefficient, and quantizing the predicted error image conversion coefficient, and outputting a quantized value as a watermarkless quantized value Means for outputting quantization information, means for inversely quantizing the watermarked quantized value and outputting a watermarkless dequantized value, and the watermarkless inverse amount for the predicted image transform coefficient. Means for outputting a decoded image conversion coefficient without watermark, and means for calculating and outputting a unit visual distortion corresponding to each of the original image conversion coefficients based on a visual model from the original image conversion coefficients. A target signal is calculated from the watermark-free decoded image transform coefficient, the original image transform coefficient, the quantization information, the unit visual distortion, and the electronic watermark information, and further, based on the unit visual distortion, Means for determining, from the target signal, a quantization value correction signal that corrects the watermarkless quantization value so that the distortion is visually small as a whole and the detection rate of the digital watermark is increased, and output this as a watermark signal. Means for adding the watermarked quantized value and the watermark signal, and outputting the result as a watermarked quantized value; and means for generating and outputting a code string from the watermarked quantized value. Means for inversely quantizing the watermarked quantized value and outputting a watermarked inverse quantized value; performing inverse transformation of the linear transformation on the watermarked inversely quantized value to obtain and output a local decoding prediction error image Means, and means for adding the motion-compensated prediction image to the locally decoded prediction error image and outputting the locally decoded image.

[0031]

Embodiments of the present invention will be described. In a first preferred embodiment of the present invention, referring to FIG. 1, a linear conversion means (50) for performing a linear conversion for projecting a digital signal onto a frequency component and outputting an original signal conversion coefficient; Quantizing means (51) for quantizing the coefficient, outputting the quantized value as a watermarkless quantized value, and outputting quantization information, and inversely quantizing the watermarkless quantized value to convert a watermarkless decoded signal conversion coefficient. Inverse quantization means (53) for outputting a unit visual / auditory distortion corresponding to each of the original signal transform coefficients based on the visual / auditory model from the original signal transform coefficients and outputting the unit visual / auditory distortion Means (55), a watermarkless decoded signal transform coefficient from the inverse quantizer, an original signal transform coefficient from the linear transformer, and quantization information from the quantizer.
Unit visual / auditory distortion from the unit visual / auditory distortion calculating means,
A target signal is calculated from the given digital watermark information, and further, based on the unit visual / auditory distortion, the overall signal is visually / auditoryly small and the digital watermark detection rate is increased. A watermark signal generating means (5) for determining a quantized value correction signal for correcting a watermarkless quantized value from the target signal and outputting this as a watermark signal.
(4) an adding means (52) for adding the watermarkless quantized value from the quantizing means and the watermark signal from the watermark signal generating means and outputting the addition result as a watermarked quantized value;
And code conversion means (56) for generating and outputting a code string from the watermarked quantized value from the addition means.

In a preferred embodiment of the present invention, referring to FIG. 5, the watermark signal generating means is configured to determine a target signal from the original signal conversion coefficient, the unit visual / auditory distortion, and the digital watermark information. A target signal calculating means (200) for calculating and outputting a watermark temporary insertion signal by changing the value of the watermarkless decoded signal transform coefficient based on the quantization information and the target signal; A temporary watermark insertion means (201) for outputting a temporary watermark insertion signal, a temporary watermark insertion from the original signal conversion coefficient, the unit visual / auditory distortion, and a temporary watermark insertion signal output from the temporary watermark insertion means. A visual / auditory degradation calculating means (203) for calculating a value indicating the visual / audible degradation of the signal and outputting the value as the visual / audible degradation degree; the digital watermark information; A watermark insertion determining unit (204) for generating and outputting a watermark insertion control signal based on the watermark temporary insertion signal to be output and the visual / audible deterioration degree output from the visual / audible deterioration calculation unit; Generating and outputting a watermark signal from the watermarking information, the watermark-less decoded signal conversion coefficient, the watermark temporary insertion signal output from the watermark temporary insertion means, and the watermark insertion control signal output from the watermark insertion determination means. Watermark signal generating means (202)
And is provided.

In a preferred embodiment of the present invention, referring to FIG. 6, the watermark signal generating means converts a target signal from the original signal conversion coefficient, the unit visual / auditory distortion, and the digital watermark information. A target signal calculating means (200) for calculating and outputting; the quantization information; the digital watermark information; a target signal output from the target signal calculating means; and a visual / audible deterioration degree calculating means. A temporary watermark insertion means (231) for generating and outputting a watermark insertion signal by changing the value of the watermarkless decoded signal conversion coefficient based on the degree of visual / audible deterioration;
From the unit visual / audible distortion and the watermark temporary insertion signal output from the watermark temporary insertion means, the degree of visual / auditory deterioration of the watermark temporary insertion signal is calculated, and the visual / auditory output to the watermark temporary insertion means is calculated. Statistical deterioration degree calculating means (233), the quantization information, the watermarkless decoded signal transform coefficient,
A watermark signal generation unit (232) for generating and outputting a watermark signal from the watermark insertion signal output from the temporary watermark insertion unit.

Referring to FIG. 2, in a preferred embodiment of the present invention, a system for encoding a picture and inserting a digital watermark at the same time, performing motion compensation on a locally decoded picture, Means for generating and outputting a prediction error image by subtracting a motion-compensated prediction image from an input image to obtain and output a prediction error image; and a linear transformation for projecting the prediction error image onto a spatial frequency component. And a first linear transformation means (82) for calculating and outputting a prediction error image conversion coefficient, quantizing the prediction error image conversion coefficient, outputting the quantization value as a watermarkless quantization value, and Quantizing means for outputting information (83)
Inverse quantization means (74) for inversely quantizing the watermarkless quantized value and outputting the watermarkless inverse quantized value, linearly transforming the motion-compensated predicted image, and calculating and outputting a predicted image transform coefficient A second linear transformation unit (73), a first addition unit (75) that adds the watermark-less dequantized value and the predicted image transformation coefficient, and outputs a watermark-free decoded image transformation coefficient,
Third linear conversion means (71) for performing the linear conversion on the input image to obtain and output original image conversion coefficients, and a unit corresponding to each of the original image conversion coefficients based on the visual model from the original image conversion coefficients A unit visual distortion calculating means (72) for calculating and outputting visual distortion, a target signal is calculated from the watermarkless decoded image conversion coefficient, the original image conversion coefficient, the quantization information, the unit visual distortion, and the digital watermark information; Further, based on the unit visual distortion, visually the entire image is small distortion,
And a watermark signal generating means (77) for determining a quantized value correction signal for correcting the quantized value without watermark from the target signal so as to increase the detection rate of the digital watermark, and outputting this as a watermark signal; A second adding unit (76) for adding the quantized value and the watermark signal and outputting a watermarked quantized value, and generating a code string from the watermarked quantized value;
A code transforming means (84) for outputting the watermarked quantized value, inversely quantizing the watermarked quantized value, and outputting a watermarked inverse quantized value; and an inverse transform of the linear transformation to the watermarked inverse quantized value. , And an inverse linear conversion means (86) for obtaining and outputting a local decoded prediction error image, and a third adding means for adding the motion compensation predicted image to the local decoded prediction error image and outputting the addition result as a locally decoded image (87).

In a preferred embodiment of the present invention, referring to FIG. 3, a system for encoding an image and inserting a digital watermark at the same time, performs motion compensation on a locally decoded image, and Means for generating and outputting (88) a subtraction means for subtracting the motion compensated prediction image from the input image to obtain and output a prediction error image;
Linear transformation means (82) for performing a linear transformation for projecting the prediction error image onto a spatial frequency component, obtaining and outputting a prediction error image transformation coefficient, and quantizing the prediction error image transformation coefficient to watermark the quantized value; A quantizing means (83) for outputting the quantization information as well as outputting the quantized information, and an inverse quantizing means (74) for inversely quantizing the watermarkless quantized value and outputting a watermarkless decoded image transform coefficient. A unit visual distortion calculating means (72) for calculating and outputting a unit visual distortion corresponding to each of the prediction error image conversion coefficients based on the visual model from the prediction error image conversion coefficients, and the watermarkless decoded image conversion coefficient and the prediction A target signal is calculated from the error image conversion coefficient, the quantization information, the unit visual distortion, and the digital watermark information, and further, based on the unit visual distortion, a visual A watermark signal generating means for determining a quantization value correction signal for correcting the watermarkless quantization value from the target signal so as to reduce the distortion and increase the detection rate of the digital watermark, and to output this as a watermark signal ( 90)
Adding means (76) for adding the watermarked quantized value and the watermark signal, and outputting the addition result as a watermarked quantized value; and generating and outputting a code string from the watermarked quantized value. Code conversion means (84), inverse quantization means (85) for inversely quantizing the watermarked quantized value and outputting a watermarked inverse quantized value, and inverse of the linear transformation to the watermarked inverse quantized value; An inverse linear conversion means (86) for performing conversion, obtaining and outputting a local decoded prediction error image, and adding a motion compensation prediction image to the local decoded prediction error image, and outputting an addition result as the local decoded image. 2 adding means (87), and inserts a digital watermark only into a frame to be intra-coded.

Referring to FIG. 4, the present invention, in its preferred embodiment, is a system for encoding an image and simultaneously inserting a digital watermark, the system comprising a linear transformation that projects an input image onto a spatial frequency component. First linear conversion means (10 in FIG. 4) for calculating and outputting original image conversion coefficients.
1) means for performing motion compensation on the locally decoded image to generate and output a motion-compensated predicted image (88 in FIG. 4); and performing the linear transformation on the motion-compensated predicted image to obtain a predicted image transform coefficient. The second linear conversion means (10 in FIG. 4)
2) subtraction means (103 in FIG. 4) for subtracting the predicted image conversion coefficient from the original image conversion coefficient and outputting the subtraction result as a prediction error image conversion coefficient; and quantizing the prediction error image conversion coefficient; A quantizing means (83 in FIG. 4) for outputting the quantized value as a watermarkless quantized value and outputting quantization information, and inversely quantizing the watermarkless quantized value and outputting a watermarkless inverse quantized value And a first adding means (74 in FIG. 4) for adding the watermarked inverse quantized value to the predicted image transform coefficient and outputting a watermarkless decoded image transform coefficient. 4 75),
A unit visual distortion calculating means (72 in FIG. 4) for calculating and outputting a unit visual distortion corresponding to each of the original image conversion coefficients based on a visual model from the original image conversion coefficients; And the original image transform coefficient, the quantization information, the unit visual distortion and the digital watermark information,
Calculating a target signal, and further, based on the unit visual distortion, a quantized value correction signal for correcting the watermarkless quantized value so as to visually reduce distortion as a whole and increase the detection rate of a digital watermark. Is determined from the target signal, a watermark signal generating means (77 in FIG. 4) for outputting the target signal as a watermark signal, adding the watermark-free quantized value and the watermark signal, and quantizing the addition result with the watermarked quantization. A second adding means (76 in FIG. 4) for outputting a value as a value, a code converting means (84 in FIG. 4) for generating and outputting a code string from the watermarked quantized value, and an inverse quantum Inverse quantization means (85 in FIG. 4) for converting the watermarked inversely quantized value into a linearly decoded inversely quantized value, and obtaining and outputting a local decoding prediction error image. Inverse linear transformation means ( 4 of 86), it said adding the motion compensated prediction picture to the local decoded prediction error image, and a third adding means for outputting the local decoded image the addition result (87 in FIG. 4).

In the embodiment of the present invention, (a) linear conversion for projecting a digital signal to a frequency component is performed,
Means for outputting an original signal transform coefficient, (b) means for quantizing the original signal transform coefficient, outputting a quantized value as a watermarkless quantized value, and outputting quantization information, and (c).
Means for inversely quantizing the watermarkless quantized value and outputting a watermarkless decoded signal transform coefficient; (d) a unit corresponding to each of the original signal transform coefficients based on a visual / auditory model from the original signal transform coefficients Means for calculating and outputting visual / aural distortion, (e) the watermarkless decoded signal transform coefficient, the original signal transform coefficient, the quantization information, and the unit visual / audio distortion.
A target signal is calculated from the auditory distortion and the digital watermark information, and further, based on the unit visual / auditory distortion, the overall signal is visually / aurally less distorted, and the detection rate of the electronic watermark is increased. Means for determining, from the target signal, a quantized value correction signal for correcting the watermarkless quantized value, and outputting the same as a watermark signal; (f) adding the watermarkless quantized value and the watermark signal (A) means for outputting a watermarked quantized value; and (g) means for generating and outputting a code string from the watermarked quantized value.
Each of the means (a) to (g) may be realized by executing a program by a computer (CPU; also including a digital signal processor) constituting the digital watermark insertion system. In this case, the program for causing each of the means (a) to (g) to function is a hard disk,
It is configured to be stored in an OM or the like and read and executed by the CPU. The present invention also includes a recording medium storing these programs.

In the embodiment of the present invention, (a) means for performing motion compensation on a locally decoded image to generate and output a motion-compensated predicted image, and (b) subtracting the motion-compensated predicted image from an input image. Means for obtaining and outputting a prediction error image by
(C) means for performing a linear transformation on the prediction error image to project it onto a spatial frequency component, obtaining and outputting a prediction error image conversion coefficient, and (d) quantizing the prediction error image conversion coefficient to obtain a quantization value. Means for outputting quantization information without watermark and outputting quantization information; (e) means for inversely quantizing the quantized value without watermark to output a dequantized value without watermark; (f) motion compensation Means for performing the linear transformation on the predicted image, calculating and outputting a predicted image conversion coefficient,
(G) means for adding the watermark-less dequantized value and the predicted image transform coefficient to output a watermark-free decoded image transform coefficient, and (h) performing the linear transformation on the input image to obtain an original image transform coefficient. Means for obtaining and outputting; (i) means for calculating and outputting a unit visual distortion corresponding to each of the original image transform coefficients based on a visual model from the original image transform coefficients;
(J) calculating a target signal from the watermark-free decoded image transform coefficient, the original image transform coefficient, the quantization information, the unit visual distortion, and the digital watermark information, and further calculating an entire image based on the unit visual distortion; Means for determining, from the target signal, a quantization value correction signal for correcting the watermarkless quantization value so as to visually reduce distortion and increase the detection rate of a digital watermark, and output this as a watermark signal. ,
(K) means for adding the watermarkless quantized value and the watermark signal to output a watermarked quantized value; (l) means for generating and outputting a code string from the watermarked quantized value; Means for inversely quantizing the watermarked quantized value and outputting a watermarked inverse quantized value; (n) performing an inverse transform of the linear transformation on the watermarked inversely quantized value to obtain a local decoding prediction error image (A) adding the motion compensation prediction image to the local decoded prediction error image and outputting the addition result as the local decoded image; It may be realized by executing a program by a computer (CPU; also including a digital signal processor) constituting the digital watermark insertion system. In this case, a program for causing each of the means (a) to (o) to function is stored in a hard disk, a ROM, or the like, and is configured to be read and executed by the CPU. The present invention also includes a recording medium storing these programs.

In the embodiment of the present invention, (a) means for performing a linear transformation for projecting an input image on a spatial frequency component to obtain and output an original image transformation coefficient, and (b) moving a local decoded image Means for performing compensation to generate and output a motion-compensated predicted image; (c) means for performing the linear transformation on the motion-compensated predicted image to obtain and output a predicted image conversion coefficient; Means for subtracting the predicted image conversion coefficient from the coefficient and outputting the result as a predicted error image conversion coefficient; (e) quantizing the predicted error image conversion coefficient;
Means for outputting the quantized value as a watermarkless quantized value and outputting quantization information; (f) means for dequantizing the watermarkless quantized value and outputting a watermarkless dequantized value; (g) Means for adding the watermark-less dequantized value to the predicted image transform coefficient and outputting a watermark-free decoded image transform coefficient; (h) the original image transform coefficient based on a visual model from the original image transform coefficient; Means for calculating and outputting a unit visual distortion corresponding to (i) a target signal from the decoded image conversion coefficient without watermark, the original image conversion coefficient, the quantization information, the unit visual distortion, and the digital watermark information; Calculating, and further based on the unit visual distortion,
Means for determining, from the target signal, a quantization value correction signal for correcting the watermarkless quantization value so as to visually reduce distortion as a whole and to increase the detection rate of a digital watermark, and output this as a watermark signal. (J) means for adding the watermark-free quantized value and the watermark signal and outputting as a watermarked quantized value; (k) means for generating and outputting a code string from the watermarked quantized value; 1) means for inversely quantizing the watermarked quantized value and outputting a watermarked inverse quantized value; (m) performing inverse transformation of the linear transformation on the watermarked inversely quantized value to obtain a local decoded prediction error image. (N) adding the motion compensation prediction image to the local decoded prediction error image and outputting the local decoded image; and (a) to (n) the digital watermark. Insertion system It may be realized by executing a program; (including digital signal processor CPU) computer formed. In this case, a program for causing each of the above-described means (a) to (n) to function is stored in a hard disk, a ROM, or the like, and is stored in a CPU.
Is configured to execute reading. The present invention also includes a recording medium storing these programs.

[0040]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below in detail with reference to the drawings.

[First Embodiment] FIG. 1 is a block diagram showing the configuration of a first embodiment of the present invention. Referring to FIG. 1, in a first embodiment of the present invention, a linear conversion circuit 50 performs a linear conversion on an input digital signal, and uses the obtained conversion coefficient as a raw signal conversion coefficient with a quantizer 51 and a unit. Visual /
The signal is output to the auditory distortion calculation circuit 55 and the watermark signal generation circuit 54.

The quantizer 51 quantizes the original signal transform coefficient output from the linear transform circuit 50, and outputs the quantized value as a watermarkless quantized value to the inverse quantizer 53 and the adder 52. A watermark signal generation circuit 54
Output to

The inverse quantizer 53 inversely quantizes the watermarkless quantized value output from the quantizer 51, and outputs it to the watermark signal generating circuit 54 as a watermarkless decoded signal transform coefficient.

The unit visual / auditory distortion calculation circuit 55 calculates a unit visual / auditory distortion from the original signal conversion coefficient output from the linear conversion circuit 50 and outputs the calculated unit visual / auditory distortion to the watermark signal generating circuit 54.

The watermark signal generating circuit 54 includes an inverse quantizer 5
3, the original signal transform coefficient output from the linear transform circuit 50, the digital watermark information, the quantized information output from the quantizer 51, and the unit visual / auditory distortion. A watermark signal is calculated from the unit visual / aural distortion output from the calculation circuit 55, and the watermark signal is output to the adder 52.

The adder 52 adds the watermarkless quantized value output from the quantizer 51 and the watermark signal output from the watermark signal generating circuit 54, and outputs the addition result as a watermarked quantized value. And outputs it to the code converter 56.

The code converter 56 generates a code string from the watermarked quantized value output from the adder 52 and outputs it.

Next, the operation of the circuit shown in FIG. 1 will be described.

The input digital signal is supplied to a linear conversion circuit 50.
, Where a linear transformation is performed. Linear transformation refers to transformation for transforming a digital signal into frequency components.For example, DCT, orthogonal transformation such as discrete Hadamard transformation, Fourier transformation, Karhunen-Loeve transformation, and filter transformation such as wavelet transformation correspond to this. I do.
For example, when an input digital signal is an image signal and is subjected to JPEG encoding, an 8 × 8 DCT is used as a linear transformation. In the case of 8 × 8 DCT, an image signal as an input digital signal is divided into 8 × 8 blocks, and DCT is performed on each block.

On the other hand, when a filter transform such as a wavelet transform is used as the linear transform, a filter process is performed on the entire image. The transform coefficients obtained in this way are output as original signal transform coefficients.

The original signal transform coefficients are quantized by a quantizer 51. For example, when performing JPEG encoding, a quantization characteristic is determined based on a quantization table, and each transform coefficient is quantized. And the quantization value is
Output as a watermarkless quantized value. Also, the quantization information used in this quantization is output at the same time. The quantization information is, for example, a value of a quantization table in the case of JPEG. Alternatively, the quantization width itself may be used. Alternatively, it may be an index for uniquely defining the quantization characteristic.

The watermarkless quantized value is inversely quantized by the inverse quantizer 53, and the result is output as a watermarkless decoded signal transform coefficient. This watermark-less decoded signal conversion coefficient is
The conversion coefficient obtained by performing a linear conversion on the decoded signal obtained by the decoder when no digital watermark is inserted, matches the conversion coefficient except for a small error caused by the conversion into an integer or a difference in operation accuracy.

On the other hand, the conversion coefficients obtained by the linear conversion circuit 50 are also input to the unit visual / auditory distortion calculating circuit 55, and the unit visual / auditory distortion calculating circuit 55 outputs the unit visual / auditory corresponding to each conversion coefficient. The distortion is calculated.

Here, the unit visual / auditory distortion is an amount serving as a measure of visual deterioration of image quality when an input signal is an image, and represents the degree of visual deterioration based on this amount.

When the input signal is a voice or acoustic signal, the unit visual / auditory distortion is an amount which is a measure of the audible deterioration of the sound quality, and indicates the degree of the audible deterioration based on this amount. As this amount, for example, JND (Just Noticeab)
le Difference). JND is literally a limit value at which human eyes and ears can identify a difference, and a difference smaller than this value cannot be detected by human eyes or ears.

The unit visual / auditory distortion may be JND itself or a constant multiplied by JND. The unit visual / auditory distortion is calculated based on the visual / auditory model. Any visual / auditory model can be used as the visual / auditory model. For example, when the input signal is an image and the linear conversion circuit 50 performs a linear conversion, an 8 × 8 DCT
Is performed, see Reference 3 (SPIE, vol. 1913 (1993), pp.
202-216, “DCT quantization matrices visually
optimized for individual images "(Andrew B. Wa
JND is calculated using the model of tson)), and this can be used as the unit visual / auditory distortion.

Since the unit visual / auditory distortion differs for each conversion coefficient, it is calculated for each conversion coefficient. However, it is not necessary to calculate the conversion coefficient for the conversion coefficient not related to the insertion of the digital watermark. In this way, the unit visual / auditory distortion of each transform coefficient is obtained and output.

Then, the watermark signal generation circuit 54 generates and outputs a watermark signal using the watermarkless decoding transform coefficient, original signal transform coefficient, digital watermark information, quantization information, and unit visual / auditory distortion. Details of this operation will be described later. Further, in the adder 52, the watermark signal is added to the corresponding quantized value without watermark, and the result is output as a quantized value with watermark. The watermarked quantized value is converted into a code string in the code converter 56. For example, when the input digital signal is an image, it is one-dimensionally scanned by zigzag scanning or the like, converted to a code string by Huffman coding or arithmetic coding, and output. When generating a code string, additional information such as quantization information is multiplexed together.

Next, the details of the watermark signal generation circuit 54 in the first embodiment of the present invention will be described. FIG.
The watermark signal generation circuit 5 according to the first embodiment of the present invention
4 is a block diagram illustrating an example of the configuration of FIG. The configuration of the watermark signal generation circuit will be described with reference to FIG.

Referring to FIG. 5, the target signal calculating circuit 20
0, watermark temporary inserter 201, watermark signal generation circuit 20
2. It comprises an auditory / audible deterioration degree calculating circuit 203 and a watermark insertion determining circuit 204. The target signal calculation circuit 200 calculates a target signal from the original signal conversion coefficient, the unit visual / auditory distortion, and the digital watermark information, and outputs the target signal to the temporary watermark inserter 201. The watermark temporary inserter 201 changes the value of the watermark-less decoded signal conversion coefficient based on the quantization information and the target signal output from the target signal calculation circuit 200 to generate a watermark temporary insertion signal, Output to the auditory degradation calculation circuit 203, the watermark insertion determination circuit 204, and the watermark signal generation circuit 202.

The visual / auditory deterioration degree calculating circuit 203 includes an original signal conversion coefficient, a unit visual / auditory distortion, and a watermark temporary inserter 20.
A value indicating visual / audible deterioration of the watermark temporary insertion signal is calculated from the watermark temporary insertion signal output from No. 1 and output to the watermark insertion determination circuit 204 as the degree of visual / audible deterioration.

The watermark insertion determining circuit 204 determines the watermark based on the digital watermark information, the temporary watermark insertion signal output from the temporary watermark inserter 201, and the visual / audible deterioration degree output from the visual / audible deterioration degree calculation circuit 203. An insertion control signal is generated and output to the watermark signal generation circuit 202.

The watermark signal generation circuit 202 includes a quantized information, a decoded signal conversion coefficient without watermark, and a temporary watermark inserter 201.
And a watermark insertion control signal output from the watermark insertion determination circuit 204, and outputs a watermark signal.

Next, the detailed operation of the circuit shown in FIG. 5 will be described. The original signal conversion coefficient, the unit visual / audible distortion, and the digital watermark information are input to the target signal calculation circuit 200. The target signal calculation circuit 200 calculates a value to be obtained by the conversion coefficient after the watermark is inserted, from the original signal conversion coefficient and the digital watermark information to be inserted. Then, this value is output as a target signal. The target signal may be the conversion coefficient itself or a value obtained by performing some conversion on the conversion coefficient. How to find the target signal depends on
It is determined depending on the digital watermark insertion and detection method used.

As a digital watermark insertion method used for calculating a target signal, any method of inserting a digital watermark in a transform coefficient domain can be used. For example, in the linear conversion circuit 50 of FIG.
In the case of performing 8DCT, any 8 × TCT method including the conventional method such as the method described in the above-mentioned reference 1 “Digital watermarking method in frequency domain for copyright protection of digital images” is used.
An 8DCT-based watermark insertion method can be used. In calculating the target signal, it is also possible to calculate a target signal in which visual distortion is less noticeable in consideration of unit visual / auditory distortion.

The target signal is input to the temporary watermark inserter 201, where the temporary watermark insertion signal is calculated. The watermark temporary insertion signal is obtained by changing the watermark-free decoded signal conversion coefficient so as to approach the target signal. However,
When decoding by inverse quantization on the decoder side, taking into account that this value can only take a discrete value determined by the quantization information, when changing the value, the discrete value that can be taken when decoding by the decoder is considered. Vary between values. For example, when obtaining the watermarked decoded signal transform coefficient, if the quantization is performed by linear quantization, the value that can be taken as the inverse quantization value can only take the value of the equal quantization step interval, so the quantization step width Increase or decrease by an integral multiple of.

The temporary watermark insertion signal output from the temporary watermark insertion unit 201 is input to the visual / aural deterioration degree calculation circuit 203. In the visual / aural deterioration degree calculation circuit 203,
The difference between each watermark temporary insertion signal value and the corresponding original signal transform coefficient is calculated. And this is the corresponding unit vision /
The number of times the auditory distortion is calculated is calculated.
It is output as the degree of auditory deterioration.

The watermark temporary insertion signal output from the temporary watermark insertion unit 201 is also input to the watermark insertion determination circuit 204. Here, it is determined whether or not the watermark temporary insertion signal is valid by using the digital watermark information by a method described below.

The watermark temporary insertion signal does not always coincide with the target signal due to the influence of quantization, and a certain amount of shift occurs. First, the watermark insertion determination circuit 204 examines the effect of this shift on the detection of a digital watermark for each watermark temporary insertion signal value. This part depends on the method of detecting the digital watermark. As a method of checking the influence of quantization, it is sufficient to actually try to detect a digital watermark and check whether or not the temporarily inserted watermark signal contributes to the detection. Alternatively, without actually performing detection,
Various methods such as a method of examining the influence by an operation similar to this can be used.

If the deviation does not lead to the deterioration of the detection, the corresponding degree of visual / audible deterioration is examined. When the deterioration is small, it is determined that the watermark may be actually inserted, and a control signal (ON signal value) for permitting the insertion is output. As this determination method, for example, a method of using a threshold value and permitting insertion only when the absolute value of the degree of deterioration is equal to or less than the threshold value can be considered.

On the other hand, in the case where the insertion of the digital watermark does not make sense due to the above-mentioned deviation or the case where the detection is adversely affected, such as the cause of erroneous detection, the control signal (OFF signal Value).

The above determination is performed for each watermark temporary insertion signal value, and the control signal value corresponding to each watermark insertion signal value is output to the watermark generation circuit 202.

Further, the temporary watermark insertion signal output from the temporary watermark inserter is also input to the watermark signal generation circuit 202. The watermark signal generation circuit 202 determines whether to insert a watermark based on the watermark insertion control signal output from the watermark insertion determination circuit 204, and generates a watermark signal. For the watermark temporary insertion signal value for which the watermark insertion control signal value is ON, the difference between this and the watermarkless decoded signal conversion coefficient is obtained, and the difference is converted to a quantized value using quantization information. And outputs it as a watermark signal value. On the other hand, when the watermark insertion control signal is not ON, 0 is output as the watermark signal value. In this way, a watermark signal value is generated and output.

The watermark insertion determination circuit 2 shown in FIG.
At 04, the strength of the digital watermark to be inserted is determined, and if the strength does not satisfy the predetermined condition, the target signal calculating circuit 200 may be requested to recalculate the target signal. That is, when the watermark strength does not satisfy the predetermined condition, the target signal calculating circuit 200 is requested to calculate a target signal for embedding a watermark of higher strength. Upon receiving this request, the target signal calculation circuit 200 calculates the target signal again, and calculates the watermark signal again according to the above-described procedure. As described above, the strength of the digital watermark can be adaptively controlled by feeding back the determination result of the watermark insertion determination circuit.

The watermark signal generation circuit described above can be implemented in various forms depending on the digital watermark insertion method.

Next, an example in which the above-described configuration is applied to a digital watermarking system described in the above-mentioned reference 1 “Digital watermarking system in the frequency domain for copyright protection of digital images” to improve this insertion system will be described. State. When applied to this method, the digital signal input to the circuit shown in FIG. 1 is an image, and the linear conversion performed by the linear conversion circuit 50 is 8 × 8 DCT.

Of the circuits shown in FIG. 5, the operations of the visual / auditory deterioration degree calculating circuit 203 and the watermark signal generating circuit 202 do not particularly depend on the digital watermark insertion method and are as described above. The other circuits will be described.

The target signal calculation circuit 200 is configured as a circuit that calculates, as a target signal, a transform coefficient q after watermark insertion obtained by the above equation (1) or (2) according to the digital watermark information 0, 1 and outputs the target signal. You.

The temporary watermark insertion unit 201 is a circuit for calculating a watermark temporary insertion signal by changing the watermarkless decoded signal conversion coefficient according to the quantization information so as to approach the target signal. For example, a circuit that calculates a value obtained by quantizing each target signal value according to the quantization information as a watermark temporary insertion signal can be considered. The visual / aural deterioration degree calculation circuit 203 calculates and outputs the visual deterioration degree of each watermark temporary insertion signal as described above. The watermark insertion determination circuit 204 uses the digital watermark information to apply
The determination is attempted in accordance with 5, and a watermark temporary insertion signal value in which the actually inserted digital watermark information matches the determined digital watermark information is selected. Further, the degree of visual deterioration is checked for the selected watermark temporary insertion signal value, and if this value is below a certain value, the watermark insertion control signal value for that watermark temporary insertion signal value is turned ON. On the other hand, for other watermark temporary insertion signal values, the watermark insertion control signal value is turned off. This watermark insertion control signal value is output to the watermark signal generation circuit 202.

Further, the above embodiment is applied to the digital watermarking method described in the above-mentioned document 2 “Digital Watermarking Method for Digital Image by Multi-Resolution Analysis” by changing the above-mentioned DCT portion to wavelet transform. , An improved system can be similarly configured.

FIG. 6 shows a first embodiment of the present invention.
FIG. 11 is a block diagram illustrating an example of another configuration of the watermark signal generation circuit 54. The configuration of the watermark signal generation circuit will be described with reference to FIG.

The target signal calculation circuit 200 calculates a target signal from the original signal conversion coefficient, the unit visual / audible distortion, and the digital watermark information, and outputs the target signal to the temporary watermark inserter 231. The temporary watermark inserter 231 is based on the quantization information, the digital watermark information, the target signal output from the target signal calculation circuit 200, and the visual / audible deterioration degree output from the visual / audible deterioration degree calculation circuit 233. A watermark insertion signal is generated by changing the value of the watermark-less decoded signal conversion coefficient, and is output to the watermark signal generation circuit 232. Further, it outputs the watermark temporary insertion signal to the visual / aural deterioration degree calculation circuit 233.

The visual / aural deterioration degree calculating circuit 233 includes an original signal conversion coefficient, a unit visual / aural distortion, and the watermark temporary inserter 23.
Then, the degree of visual / audible deterioration of the watermark temporary insertion signal is calculated from the watermark temporary insertion signal output from No. 1 and output to the watermark temporary insertion unit 231.

The watermark signal generation circuit 232 includes quantization information, a decoded signal conversion coefficient without watermark, and a temporary watermark inserter 231.
A watermark signal is generated from the watermark insertion signal output from the and a watermark signal is output.

Next, the watermark signal generation circuit 2 shown in FIG.
35 will be described in detail.

The original signal conversion coefficient, the unit visual / auditory distortion, and the digital watermark information are input to the target signal calculation circuit 200.
The operation of the target signal calculation circuit 200 is the same as that shown in FIG.
Output to 31.

The temporary watermark inserter 231 calculates a watermark insertion signal. This is obtained by changing the watermark-less decoded signal conversion coefficient so as to approach the target signal. The temporary watermark inserter 231 calculates the watermark insertion signal in the same manner as when the temporary watermark insertion signal is calculated in the temporary watermark inserter 201 shown in FIG. Further, the temporary watermark insertion device 231 also calculates a temporary watermark insertion signal. This is a value indicating how much, if any, each value of the watermark insertion signal should be updated, that is, a value representing the next update value of the watermark insertion signal. Then, the watermark-less decoded signal conversion coefficient value is set to a value obtained by changing the watermark insertion signal by an extra number of quantization steps. In brief, the value may be changed by an extra amount corresponding to one quantization step. Or,
Digital watermark information may be used to determine how many quantization steps to change. Then, the watermark temporary insertion signal is output to the visual / aural deterioration degree calculation circuit 233.
On the other hand, for the watermark insertion signal, the watermark temporary inserter 23
The value is held in a storage device inside 1.

The operation of the visual / aural deterioration degree calculating circuit 233 is the same as that of the visual / aural deterioration degree calculating circuit 203 shown in FIG. Is output. In the temporary watermark insertion device 231,
From the degree of visual / audible deterioration, it is possible to know how much the degree of visual / audible deterioration will be increased if the current watermark insertion signal value is further changed by one quantization step (or several steps). In other words, with respect to the watermark temporary insertion signal value in which this value is reduced, it can be seen that even if the value is further changed by one quantization step (or several quantization steps), visual / audible deterioration is small.

Therefore, the temporary watermark insertion unit 231 preferentially changes the watermark insertion signal value from the one with the smallest degree of visual / audible deterioration. That is, the value stored in the storage device is updated. Then, the update of the watermark insertion signal value is stopped at the time of approaching the target signal, and the watermark insertion value finally obtained is replaced with the watermark signal generation circuit 232.
Output to

The operation of the watermark signal generation circuit 232 is described in FIG.
19, except that the input signal is changed from the temporary watermark insertion signal to the watermark insertion signal. Also, whether or not to insert a watermark is not controlled by the watermark insertion control signal. That is, the watermark signal generation circuit 202 shown in FIG. 5 performs the same operation as when the watermark insertion control signal is always ON, and outputs the watermark signal.

Next, a specific configuration of the watermark signal generating circuit 54 when applied to the digital watermarking method described in the above-mentioned document 1 “Digital watermarking method in frequency domain for copyright protection of digital images” will be described. This will be described with reference to FIGS.

When applied to this method, as described above, the digital signal input to the circuit shown in FIG. 1 is an image, and the linear conversion performed by the linear conversion circuit 50 is 8 × 8D
CT.

The target signal calculation circuit 200 calculates, as a target signal, the conversion coefficient q after the watermark insertion obtained by the above equation 1 or 2 according to the digital watermark information 0 or 1.
It is configured as an output circuit.

First, the temporary watermark inserter 231 sets the watermark insertion signal value to the same value as the input watermarkless decoded signal conversion coefficient value. Next, each watermark insertion signal value is changed by several quantization steps, and the corresponding watermark temporary insertion signal value is calculated. At this time, the direction of change, that is, whether to add or subtract the quantization step, is determined by whether the watermark insertion signal value is larger or smaller than the target signal value. If the watermark insertion signal value is large, the quantization step is reduced; otherwise, a quantization step is added. At this time, the number of quantization steps to be changed is determined based on whether or not it can be correctly detected in comparison with the digital watermark information. That is, when the quantization step is changed to one or two quantization steps, the minimum value at which digital watermark information can be correctly detected from the watermark temporary insertion signal is selected. In this way, the watermark temporary insertion value is determined.

Next, the obtained watermark temporary insertion signal value is input to the visual / aural deterioration degree calculation circuit 233, and the visual deterioration degree for each watermark temporary insertion signal value is calculated.
Output to the temporary watermark insertion unit 231. Watermark temporary inserter 2
In step 31, a signal having a smaller degree of visual deterioration is selected, and the watermark insertion signal value stored in the storage device is updated to a temporary watermark insertion signal value. At this time, the watermark temporary insertion signal value is also updated correspondingly. The method for determining the watermark temporary insertion signal value is as described above.

The above operation is repeated until the ratio of watermark information that can be correctly detected satisfies a certain condition. Alternatively, when the degree of visual deterioration for each watermark temporary insertion signal value becomes larger than a predetermined value, the update is ended there. Alternatively, the update may be terminated when the above-described repetition is performed a predetermined number of times.

Then, the watermark insertion signal finally obtained is output to the watermark signal generation circuit 232. The operation of the watermark signal generation circuit 232 is as described above, and the watermark signal is calculated and output.

Also, the above-mentioned embodiment is changed to the above-mentioned Reference 2 by changing the above-mentioned DCT portion to a wavelet transform.
A method for applying and improving the digital watermarking method shown in “Digital Watermarking Method for Digital Image by Multiresolution Analysis” can be similarly configured.

The first embodiment of the present invention described above prevents digital watermark data from deteriorating due to lossy compression, and
A digital watermark insertion method for a signal that suppresses visual / audible deterioration is provided.

[Embodiment 2] Next, a second embodiment of the present invention will be described in detail with reference to the drawings. FIG. 2 is a block diagram showing the configuration of the second embodiment of the present invention. Referring to FIG. 2, the subtracter 81 subtracts the motion compensated prediction image output from the frame memory 88 from the input image, and outputs the result to the linear conversion circuit 82 as a prediction error image.

The linear conversion circuit 82 performs a linear conversion on the prediction error image output from the subtracter 81, and outputs the obtained conversion coefficient to the quantizer 83 as a prediction error image conversion coefficient.

The quantizer 83 quantizes the predictive error image transform coefficient output from the linear transform circuit 82, and uses the quantized value as a watermarkless quantized value and an inverse quantizer 74 and an adder 76.
To the watermark signal generation circuit 77
Output to

The linear conversion circuit 71 performs a linear conversion on the input image and outputs the result to the watermark signal generation circuit 77 and the unit visual distortion calculation circuit 72 as original image conversion coefficients.

The unit visual distortion calculation circuit 72 calculates the unit visual distortion from the original image conversion coefficient output from the linear conversion circuit 71 and outputs the calculated unit visual distortion to the watermark signal generation circuit 77.

The linear conversion circuit 73 includes a frame memory 88
The DCT is performed on the motion-compensated predicted image output from, and the obtained transform coefficient is output to the adder 75 as a predicted image transform coefficient.

The inverse quantizer 74 inversely quantizes the watermarkless quantized value output from the quantizer 83, and outputs the result to the adder 75 as a watermarkless inverse quantized value.

The adder 75 adds the predicted image transform coefficient output from the linear transform circuit 73 and the watermarkless inverse quantized value output from the inverse quantizer 74, and outputs the result as the watermarkless decoded image transform coefficient. Is output to the watermark signal generation circuit 77.

The watermark signal generation circuit 77 outputs a watermark-free decoded image conversion coefficient output from the adder 75, an original image conversion coefficient output from the linear conversion circuit 71, digital watermark information, and an output from the quantizer 83. A watermark signal is calculated from the quantized information thus obtained and the unit visual distortion output from the unit visual distortion calculation circuit 72, and the watermark signal is output to the adder.

The adder 76 adds the watermark signal output from the watermark signal generating circuit 77 to the watermark-free quantized value output from the quantizer 83, and outputs the result as a watermarked quantized value to the code converter 84. Output to the inverse quantizer 85.

The code converter 84 performs variable length coding on the watermarked quantized value output from the adder 76 to generate and output a code string.

The inverse quantizer 85 inversely quantizes the watermarked quantized value output from the adder 76 and outputs the result to the inverse linear transform circuit 86 as a watermarked inverse quantized value.

The inverse linear transform circuit 86 performs an inverse linear transform on the watermarked inverse quantized value output from the inverse quantizer 85, and outputs the obtained transform coefficient to the adder 87 as a locally decoded prediction error image. .

The adder 87 adds the motion-compensated predicted image output from the frame memory 88 to the locally decoded error image output from the inverse linear conversion circuit 86, and outputs the locally decoded image to the frame memory 88.

The frame memory 88 performs motion compensation on the locally decoded image output from the adder 87, and outputs a motion-compensated predicted image to the subtractor 81, the adder 87, and the linear conversion circuit 73.

Next, the operation of the circuit shown in FIG. 2 will be described.

The input image is input to a subtracter 81, where the motion compensated prediction image output from the frame memory 88 is subtracted from the input image, and a prediction error image is obtained. The operation of the frame memory 88 will be described later. The obtained prediction error image is subjected to linear conversion in a linear conversion circuit 82, and the obtained conversion coefficients are output as prediction error image conversion coefficients. As in the circuit shown in FIG. 1, various conversions are possible as the linear conversion. In the case of international standard image coding methods such as MPEG, H.261, and H.263, 8 ×
8DCT is performed. The prediction error image conversion coefficient is input to the quantizer 83 and is quantized. For example, when performing MPEG coding, the quantization characteristics are determined based on a quantization matrix and a quantization scale value determined for each macroblock, and each transform coefficient is quantized. Then, the quantized value is output as a watermarkless quantized value. Also, the quantization information used in this quantization is output at the same time.
For example, in the case of MPEG, the quantization information is a value of a quantization matrix and a quantization scale value. Alternatively, the quantization width itself may be used. Alternatively, it may be an index for uniquely defining the quantization characteristic. The watermarkless quantized value is inversely quantized by the inverse quantizer 74, and the result is output as a watermarkless inversely quantized value.

The motion-compensated predicted image output from the frame memory 88 is input to the linear conversion circuit 73, and the motion-compensated predicted image is subjected to the same linear conversion as the linear conversion circuit 82. Then, the obtained transform coefficients are output as predicted image transform coefficients. The predicted image transform coefficient is input to the adder 75 and added to the watermarkless dequantized value output from the dequantizer 74. Then, the obtained result is output as a watermarkless decoded image conversion coefficient. Note that the watermark-free decoded image transform coefficient is obtained by excluding a transform coefficient obtained by performing DCT on a decoded image obtained by a decoder and a small error caused by integer conversion, operation accuracy, and the like when no digital watermark is inserted. Become equal.

On the other hand, the input image is also input to the linear conversion circuit 71, where the same linear conversion as the linear conversion circuit 82 is performed. Then, the obtained transform coefficients are output as original image transform coefficients. The original image conversion coefficients are input to the unit visual distortion calculation circuit 72. The operation of the unit visual distortion calculation circuit 72 is
This is the same as the unit visual / auditory distortion calculation circuit 55, and the unit visual distortion for each original image conversion coefficient is calculated and output.

The watermark signal generation circuit 77 generates and outputs a watermark signal using the watermarkless decoded image conversion coefficient, original image conversion coefficient, digital watermark information, quantization information, and unit visual distortion.

The operation of watermark signal generating circuit 77 is the same as that of watermark signal generating circuit 54 shown in FIG. However,
The parameters and the like used when generating the watermark signal may be different depending on the picture type or macroblock type indicating the image encoding mode, the motion compensation mode, or the like.

The watermark signal is added by the adder 76 to the watermarked quantized value output from the quantizer 83, and the result is output as a watermarked quantized value.
The watermarked quantized value is one-dimensionally scanned by a code converter 84 by zigzag scan, alternate scan, or the like, then converted into a code string by Huffman coding or arithmetic coding, and output. When a code string is generated, additional information such as a quantization parameter, which is essential for encoding and decoding an image, is also multiplexed.

Further, the watermarked quantized value output from the adder 76 is inversely quantized by the inverse quantizer 85, and further inversely transformed by the linear transformation circuit 82 by the inverse linear transformation circuit 86. The local decoding prediction error image is obtained. For example, when 8 × 8 DCT is performed by the linear transformation circuit 82, the inverse linear transformation circuit 86 performs 8 × 8 inverse DCT.
T is performed. In the adder 87, the motion-compensated predicted image output from the frame memory 88 is added to the locally decoded prediction error image, and a locally decoded image is obtained. This is input to the frame memory 88 and stored. In the frame memory 88, a locally decoded image that has been locally decoded in the past is stored, and motion compensation is performed using this as a reference image, and a motion compensated prediction image is generated and output.

[Embodiment 3] In the circuit of the second embodiment shown in FIG. 2, a digital watermark is inserted also into a frame to be subjected to inter-frame coding. Instead, it is also possible to insert a digital watermark only into the frame to be intra-coded.

FIG. 3 is a diagram showing the configuration of the third embodiment of the present invention, and is a block diagram showing the configuration of an encoder for inserting a digital watermark only into a frame to be internally encoded. FIG.
3, the linear conversion circuit 71, the linear conversion circuit 73, and the adder 75 in the circuit configuration shown in FIG.
And a watermark signal generation circuit 90 is provided instead of the watermark signal generation circuit 77. Then, the watermarkless inverse quantization output from the inverse quantizer 74 is output to the watermark signal generation circuit 90, and the prediction error image conversion coefficient output from the linear conversion circuit 82 is converted into the unit visual distortion calculation circuit 72 and the watermark signal. It is also output to the generation circuit 90. The other configuration is the same as the configuration shown in FIG.

When intra-frame encoding is performed, no motion-compensated predicted image is output from the frame memory 88.
Therefore, the output of the subtractor 81 is the same as that of the original image. Therefore, in this case, since the output of the linear conversion circuit 82 and the output of the linear conversion circuit 71 match in FIG. 2, one linear conversion circuit is sufficient. For this reason, in the circuit shown in FIG. 3, the prediction error image conversion coefficient output from the linear conversion circuit 82 is also output to the unit visual distortion calculation circuit 72 and the watermark signal generation circuit 90.

In the case of intra-frame coding, since the output of the adder 75 in FIG. 2 matches the inversely quantized value without watermark output from the inverse quantizer 74, the circuit shown in FIG. Except for the conversion circuit 73 and the adder 75, the watermarkless inversely quantized value output from the inverse quantizer 74 is directly input to the watermark signal generation circuit 90.

The watermark signal generation circuit 90 outputs a watermark signal when performing intra-frame encoding, and outputs 0 as a watermark signal when performing inter-frame encoding. The operation of the other circuits is the same as that of the circuit shown in FIG.

As described above, when an electronic watermark is inserted only into a frame to be subjected to intra-frame encoding, it is not necessary to consider a feedback loop, and the circuit is equivalent to the circuit shown in FIG. Therefore, the present invention can be implemented as described in the description of FIG.

The second and third embodiments of the present invention described above
A method of inserting a digital watermark into a moving image which prevents deterioration of digital watermark data due to lossy compression and suppresses visual deterioration.

[Embodiment 4] Next, a fourth embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 4 is a block diagram showing the configuration of the fourth embodiment of the present invention. Referring to FIG. 4, the linear conversion circuit 101 performs a linear conversion on an input image, and outputs the obtained conversion coefficient as an original image conversion coefficient to the subtractor 103, the watermark signal generation circuit 77, and the unit visual distortion calculation circuit 72. .

The linear conversion circuit 102 includes the frame memory 8
Then, a linear transformation is performed on the motion-compensated predicted image output from 8 and the obtained transform coefficient is output to the subtractor 103 and the adder 75 as a predicted image transform coefficient.

The subtractor 103 subtracts the predicted image conversion coefficient output from the linear conversion circuit 102 from the original image conversion coefficient output from the linear conversion circuit 101, and obtains the obtained conversion coefficient as a prediction error image conversion coefficient. Output to the converter 83.

The quantizer 83 quantizes the prediction error image transform coefficient output from the subtractor 103, outputs the quantized value as a watermarkless quantized value to the inverse quantizer 74 and the adder 76, and further The conversion information is output to the watermark signal generation circuit 77.

The connection relationship of the inverse quantizer 74 is the same as that of the circuit shown in FIG. The adder 75 is connected to the linear conversion circuit 10.
2 and the watermarked inverse quantization value output from the inverse quantizer 74, and outputs the result to the watermark signal generation circuit 77 as a watermarkless decoded image conversion coefficient.

The unit visual distortion calculation circuit 72 calculates the unit visual distortion from the original image conversion coefficient output from the linear conversion circuit 101 and outputs the calculated unit visual distortion to the watermark signal generation circuit 77.

The watermark signal generation circuit 77 outputs a watermark-free decoded image conversion coefficient output from the adder 75, an original image conversion coefficient output from the linear conversion circuit 101, digital watermark information, and an output from the quantizer 83. Quantization information,
A watermark signal is calculated from the unit visual distortion output from the unit visual distortion calculation circuit 72, and the watermark signal is output to the adder.

The connection relationship between the adder 76, the code converter 84, the inverse quantizer 85, the inverse linear conversion circuit 86, and the adder 87 is as follows.
This is the same as the circuit shown in FIG.

The frame memory 88 performs motion compensation on the locally decoded image output from the adder 87, and outputs a motion-compensated predicted image to the adder 87 and the linear conversion circuit 102.

Next, the operation of the circuit shown in FIG. 4 will be described.

The input image is input to a linear conversion circuit 101, where linear conversion is performed. The obtained conversion coefficient is
Output as original image conversion coefficients. On the other hand, the motion-compensated predicted image output from the frame memory 88 is input to the linear conversion circuit 102, and the same linear conversion as the linear conversion circuit 101 is performed on the motion-compensated predicted image. And
The obtained transform coefficients are output as predicted image transform coefficients.

The predicted image conversion coefficient is input to the subtractor 103, and is subtracted from the corresponding original image conversion coefficient to obtain a prediction error image conversion coefficient. If the linear conversion performed by the linear conversion circuit 101 is equivalent to the linear conversion performed by the linear conversion circuit 82 in FIG. 2, the prediction error image conversion coefficient output from the subtractor 103 is determined by the linearity of the conversion. , Coincides with the prediction error image conversion coefficient output from the linear conversion circuit 82 shown in FIG.

The obtained predictive error image transform coefficient is quantized by the quantizer 83, and a quantized value without watermark is output.

The quantized value without watermark is inversely quantized by the inverse quantizer 74, and the inversely quantized value without watermark is output. The operations of the quantizer 83 and the inverse quantizer 74 are the same as those shown in FIG.

The predicted image conversion coefficient is input to an adder 75, which adds the predicted image conversion coefficient to the inversely quantized value without watermark output from the inverse quantizer 74, and outputs a decoded image conversion coefficient without watermark. You. The operation of the adder 75 is also described in FIG.
Is the same as the circuit shown in FIG.

The original image conversion coefficient output from the linear conversion circuit 101 is input to the unit visual distortion calculation circuit 72. The operation of the unit visual distortion calculation circuit 72 is the same as that shown in FIG. 2, and the unit visual distortion for each transform coefficient is calculated and output.

Then, the watermark signal generation circuit 77 generates and outputs a watermark signal using the watermarkless decoded image conversion coefficient, original image conversion coefficient, digital watermark information, quantization information, and unit visual distortion. Watermark signal generation circuit 7
7 is the same as that shown in FIG.

The watermark signal is added by the adder 76 to the watermarked quantized value output from the quantizer 83, and the result is output as a watermarked quantized value.
The watermarked quantized value is converted into a code string by a code converter 84 and output. Adder 76, code converter 8
The operation of No. 4 is the same as that shown in FIG.

Further, the quantized value with watermark output from the adder 76 is inversely quantized by the inverse quantizer 85, and the inverse linear conversion circuit 86 performs the inverse conversion of the linear conversion performed by the linear conversion circuit 101. The local decoding prediction error image is obtained.

In the adder 87, the motion-compensated predicted image output from the frame memory 88 is added to the locally decoded prediction error image, and a locally decoded image is obtained.

The frame memory 88 performs motion compensation, and generates and outputs a motion-compensated predicted image. The operations of the inverse quantizer 85, the inverse linear transform circuit 86, the adder 87, and the frame memory 88 are the same as those shown in FIG.

The above-described fourth embodiment of the present invention provides a method for inserting a digital watermark into a moving image which prevents deterioration of digital watermark data due to lossy compression and suppresses visual deterioration. Further, the number of times of performing the linear conversion can be reduced as compared with the second embodiment of the present invention.

[0153]

As described above, according to the present invention, the following effects can be obtained.

A first effect of the present invention is that deterioration of digital watermark data due to lossy compression can be suppressed.

The reason is that, in the present invention, the digital watermark data is inserted so as to be suitable for the quantization at the time of the lossy compression by combining the lossy compression encoding circuit and the digital watermark inserter. . In the conventional digital watermarking method,
Since the digital watermark is inserted without considering irreversible compression, deterioration due to irreversible compression was unavoidable.However, in the present invention, the digital watermark is designed so as to mitigate the influence by considering the deterioration due to quantization. Because of the insertion, deterioration of digital watermark data can be prevented from being minimized.

A second effect of the present invention is that a digital watermark can be inserted while suppressing visual / audible deterioration.

The reason is that, in the present invention, the degree of visual / auditory deterioration is determined based on the visual / auditory model, and an electronic watermark is inserted so as to suppress the degree. In the conventional digital watermarking method, since a digital watermark is inserted without considering irreversible compression, even if the digital watermark inserted by quantization becomes undetectable, the digital watermark is inserted unnecessarily, Although unnecessary degradation has occurred, the present invention does not insert a digital watermark in such a wasteful case, so that visual / audible degradation can be suppressed to a small extent.

A third effect of the present invention is that it is not necessary to change the digital watermark detector even when the conventional digital watermark system is improved using the present invention. For this reason, even after the detector has already been manufactured and shipped, by improving the insertion method using the present invention,
The detection rate of the digital watermark can be improved while suppressing visual / audible deterioration.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an encoding and digital watermark inserting unit according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of an encoding and digital watermark inserting unit according to a second embodiment of the present invention.

FIG. 3 is a block diagram illustrating a configuration of an encoding and digital watermark inserting unit according to a third embodiment of the present invention.

FIG. 4 is a block diagram illustrating a configuration of an encoding and digital watermark inserting unit according to a fourth embodiment of the present invention.

FIG. 5 is a block diagram showing a configuration of a watermark signal generation circuit in the first, second, and fourth embodiments of the present invention.

FIG. 6 is a block diagram showing a configuration of a watermark signal generation circuit in the first, second, and fourth embodiments of the present invention.

FIG. 7 is a block diagram showing an example of the configuration of a conventional digital watermark inserter and encoder.

FIG. 8 is a block diagram illustrating an example of a configuration of a conventional digital watermark inserter and a moving image encoder.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 DCT circuit 2 Digital watermark insertion circuit 3 Inverse DCT circuit 4 DCT circuit 5 Quantizer 6 Code converter 10 Digital watermark inserter 11 Image encoder 21 Subtractor 22 DCT circuit 23 Quantizer 24 Code converter 25 Inverse quantum 26 Inverse DCT circuit 27 Adder 28 Frame memory 40 Video encoder 50 Linear transformation circuit 51 Inverse quantizer 52 Adder 53 Inverse quantizer 54 Watermark signal generation circuit 55 Unit visual / auditory distortion calculation circuit 46 Code Converter 71 linear conversion circuit 72 unit visual distortion calculation circuit 73 linear conversion circuit 74 inverse quantizer 75 adder 76 adder 77 watermark signal generation circuit 81 subtractor 82 linear conversion circuit 83 quantizer 84 code converter 85 inverse quantum Transformer 86 inverse linear transformation circuit 87 adder 88 frame memory 90 watermark signal generation circuit 101 line Conversion circuit 102 linear conversion circuit 103 subtractor 200 target signal calculation circuit 201 watermark temporary inserter 202 watermark signal generation circuit 203 visual degradation degree calculation circuit 204 watermark insertion determination circuit 205 watermark signal generation circuit 231 watermark temporary insertion device 232 watermark signal generation circuit 233 Visual deterioration degree calculation circuit 235 Watermark signal generation circuit

Claims (11)

[Claims] 1. A system for encoding a digital signal and inserting a digital watermark, comprising: means for performing a linear transformation for projecting the digital signal onto a frequency component to output an original signal transformation coefficient; Means for outputting a quantized value as a watermarked quantized value and outputting quantization information, and means for inversely quantizing the watermarkless quantized value and outputting a watermarkless decoded signal transform coefficient. Means for calculating and outputting a unit visual / auditory distortion corresponding to each of the original signal transform coefficients from the original signal transform coefficients based on a visual / auditory model, and the watermarkless decoded signal transform coefficient and the original signal transform. A target signal is calculated from the coefficient, the quantization information, the unit visual / auditory distortion, and the digital watermark information, and further, a target signal is calculated based on the unit visual / auditory distortion. A quantization value correction signal for correcting the watermarkless quantization value is determined from the target signal so that distortion as a whole is visually / audibly small and the detection rate of the digital watermark is increased. Watermark signal generating means for outputting as a signal, adding the watermarkless quantized value and the watermark signal,
A digital signal coded digital watermark insertion system, comprising: means for outputting a watermarked quantized value; and means for generating and outputting a code string from the watermarked quantized value. 2. The apparatus according to claim 2, wherein the watermark signal generating means includes: an original signal conversion coefficient; the unit visual / auditory distortion information;
A target signal calculating unit that calculates and outputs a target signal from the digital watermark information; and, based on the quantized information and the target signal, changes a value of the watermark-less decoded signal conversion coefficient to generate a watermark temporary. A temporary watermark insertion means for generating an insertion signal and outputting the temporary watermark insertion signal, the original signal conversion coefficient, the unit visual / auditory distortion information,
From the watermark temporary insertion signal output from the watermark temporary insertion means, a value indicating visual / audible deterioration of the watermark temporary insertion signal is calculated, and the visual / audible deterioration degree calculation is output as visual / audible deterioration degree information Means, the digital watermark information, the watermark temporary insertion signal output from the watermark temporary insertion means, and the visual / audible deterioration degree information output from the visual / audible deterioration degree calculating means, A watermark insertion determining unit that generates and outputs a control signal, the quantization information, the watermarkless decoded signal conversion coefficient, a watermark temporary insertion signal output from the watermark temporary inserting unit, and a watermark insertion determining unit. 2. The digital signal code according to claim 1, further comprising: a watermark signal generating unit configured to generate and output a watermark signal from the output watermark control signal. Of electronic watermark insertion system. 3. The watermark signal generating means includes: an original signal transform coefficient, the unit visual / auditory distortion information,
Target signal calculation means for calculating and outputting a target signal from the digital watermark information; the quantization information; the digital watermark information; a target signal output from the target signal calculation means; A temporary watermark insertion means for generating and outputting a temporary watermark insertion signal and a watermark insertion signal by changing the value of the watermarkless decoded signal conversion coefficient based on the visual / audible degradation degree information output from the degradation degree calculation means The original signal conversion coefficient, the unit visual / auditory distortion information,
A visual / audible degradation degree calculating means for calculating visual / audible deterioration degree information of the watermark temporary insertion signal from the watermark temporary insertion signal output from the watermark temporary insertion means and outputting the information to the watermark temporary insertion means; A watermark signal generation unit that generates and outputs a watermark signal from the quantization information, the watermark-less decoded signal conversion coefficient, and a watermark insertion signal output from the temporary watermark insertion unit, The digital signal encoding digital watermark insertion system according to claim 1. 4. A system for encoding an image and inserting a digital watermark, performing motion compensation on a locally decoded image to generate and output a motion compensated prediction image, and said motion compensation from an input image. Means for calculating and outputting a prediction error image by subtracting the prediction image; means for performing a linear transformation to project the prediction error image onto a spatial frequency component to obtain and output a prediction error image conversion coefficient; and Means for quantizing the image transform coefficient, outputting the quantized value as a watermarkless quantized value, and outputting quantization information; and dequantizing the watermarkless quantized value and outputting a watermarkless dequantized value. Means for performing the linear transformation on the motion-compensated predicted image, calculating and outputting a predicted image conversion coefficient, and adding the watermarkless inverse quantization value and the predicted image conversion coefficient, Means for outputting a watermarkless decoded image transform coefficient; means for performing the linear transformation on the input image to obtain and output an original image transform coefficient; and the original image transform coefficient based on a visual model from the original image transform coefficient. Means for calculating and outputting a corresponding unit visual distortion, and calculating a target signal from the watermarkless decoded image transform coefficient, the original image transform coefficient, the quantization information, the unit visual distortion, and the digital watermark information. Further, based on the unit visual distortion, the quantized value correction signal for correcting the watermark-less quantized value so that the distortion is visually small as a whole image and the detection rate of the digital watermark is increased. Determined from the signal, a watermark signal generating means for outputting this as a watermark signal, adding the watermarkless quantized value and the watermark signal,
Means for outputting a watermarked quantized value; means for generating and outputting a code string from the watermarked quantized value; means for dequantizing the watermarked quantized value; and means for outputting a watermarked dequantized value. Means for performing an inverse transformation of the linear transformation on the watermarked inverse quantization value to obtain and output a local decoding prediction error image; adding the motion compensation prediction image to the local decoding prediction error image; Means for outputting as the locally decoded image. 5. A system for encoding an image and inserting a digital watermark, comprising: performing motion compensation on a locally decoded image to generate and output a motion-compensated predicted image; Means for calculating and outputting a prediction error image by subtracting the prediction image; means for performing a linear transformation to project the prediction error image onto a spatial frequency component to obtain and output a prediction error image conversion coefficient; and Means for quantizing the image transform coefficient, outputting the quantized value as a watermarkless quantized value, and outputting quantization information; dequantizing the watermarkless quantized value and outputting a watermarkless decoded image transform coefficient Means for calculating and outputting a unit visual distortion corresponding to each of the prediction error image conversion coefficients based on a visual model from the prediction error image conversion coefficients, and A target signal is calculated from the uncoded decoded image conversion coefficient, the prediction error image conversion coefficient, the quantization information, the unit visual distortion, and the digital watermark information, and further, based on the unit visual distortion, visualizes the entire image. The distortion is small,
And a watermark signal generation means for determining a quantization value correction signal for correcting the watermarkless quantization value from the target signal so as to increase the detection rate of the digital watermark, and outputting this as a watermark signal; Adding the quantized value and the watermark signal,
Means for outputting the addition result as a watermarked quantized value; means for generating and outputting a code string from the watermarked quantized value; and means for inversely quantizing the watermarked quantized value. Outputting means, performing inverse transformation of the linear transformation on the watermarked inverse quantization value, obtaining and outputting a local decoding prediction error image, and adding the motion compensation prediction image to the local decoding prediction error image. Means for outputting the result of addition as the locally decoded image, wherein a digital watermark is inserted only into a frame to be intra-coded. 6. A system for encoding an image and inserting a digital watermark, comprising: a means for performing a linear transformation for projecting an input image onto a spatial frequency component to obtain and output an original image transformation coefficient; Means for performing motion compensation on the decoded image to generate and output a motion-compensated predicted image; means for performing the linear transformation on the motion-compensated predicted image to obtain and output a predicted image transform coefficient; Means for subtracting the predicted image conversion coefficient from the coefficients and outputting the result as a prediction error image conversion coefficient; andquantizing the prediction error image conversion coefficient, outputting the quantization value as a watermarkless quantization value, and performing quantization. Means for outputting information; means for inversely quantizing the watermarkless quantized value to output a watermarkless inverse quantized value; and applying the watermarkless inverse quantized value to the predicted image transform coefficient. Means for adding and outputting a watermark-free decoded image transform coefficient; means for calculating and outputting a unit visual distortion corresponding to each of the original image transform coefficients based on a visual model from the original image transform coefficients; Without the decoded image transform coefficient, the original image transform coefficient, the quantization information, the unit visual distortion, and the digital watermark information, a target signal is calculated, and further, based on the unit visual distortion, a visual A watermark signal generating means for determining a quantization value correction signal for correcting the quantization value without watermark so that the distortion is small and increasing the detection rate of the digital watermark from the target signal, and outputting this as a watermark signal, Adding the watermarkless quantized value and the watermark signal,
Means for outputting as a watermarked quantized value; means for generating and outputting a code string from the watermarked quantized value; means for dequantizing the watermarked quantized value; and means for outputting a watermarked dequantized value. Means for performing an inverse transformation of the linear transformation on the watermarked inverse quantized value to obtain and output a local decoding prediction error image; and adding the motion compensation prediction image to the local decoding prediction error image to perform the local decoding. Means for outputting an image. 7. The watermark signal generating means includes: the original signal conversion coefficient; the unit visual / auditory distortion information;
A target signal calculating unit that calculates and outputs a target signal from the digital watermark information; and, based on the quantized information and the target signal, changes a value of the watermark-less decoded signal conversion coefficient to generate a watermark temporary. A temporary watermark insertion means for generating an insertion signal and outputting the temporary watermark insertion signal, the original signal conversion coefficient, the unit visual / auditory distortion information,
From the watermark temporary insertion signal output from the watermark temporary insertion means, a value indicating visual / audible deterioration of the watermark temporary insertion signal is calculated, and the visual / audible deterioration degree calculation is output as visual / audible deterioration degree information Means, a watermark insertion from the digital watermark information, a watermark temporary insertion signal output from the watermark temporary insertion means, and visual / audible deterioration degree information output from the visual / audible deterioration degree calculation means. A watermark insertion determining unit that generates and outputs a control signal, the quantization information, the watermarkless decoded signal conversion coefficient, a watermark temporary insertion signal output from the watermark temporary inserting unit, and a watermark insertion determining unit. The image encoding apparatus according to claim 4, further comprising: a watermark signal generation unit configured to generate and output a watermark signal from the watermark insertion control signal to be output. Child watermark insertion system. 8. The watermark signal generating means, wherein: the original signal transform coefficient; the unit visual / auditory distortion information;
Target signal calculation means for calculating and outputting a target signal from the digital watermark information; the quantization information; the digital watermark information; a target signal output from the target signal calculation means; A temporary watermark insertion means for generating and outputting a temporary watermark insertion signal and a watermark insertion signal by changing the value of the watermarkless decoded signal conversion coefficient based on the visual / audible degradation degree information output from the degradation degree calculation means The original signal conversion coefficient, the unit visual / auditory distortion information,
A visual / audible degradation degree calculating means for calculating visual / audible deterioration degree information of the watermark temporary insertion signal from the watermark temporary insertion signal output from the watermark temporary insertion means and outputting the information to the watermark temporary insertion means; A watermark signal generation unit that generates and outputs a watermark signal from the quantization information, the watermark-less decoded signal conversion coefficient, and a watermark insertion signal output from the temporary watermark insertion unit, The image encoding digital watermark insertion system according to claim 4 or 6, wherein 9. A system for encoding a digital signal and inserting a digital watermark, comprising: (a) means for performing a linear transformation for projecting the digital signal onto frequency components and outputting original signal transformation coefficients; Means for quantizing the signal transform coefficient, outputting the quantized value as a watermarkless quantized value, and outputting quantization information; and (c) inversely quantizing the watermarkless quantized value to obtain a watermarkless decoded signal transform coefficient. (D) means for calculating and outputting a unit visual / auditory distortion corresponding to each of the original signal transform coefficients based on a visual / auditory model from the original signal transform coefficients, and (e) the watermark. A target signal is calculated from the no-decoded signal conversion coefficient, the original signal conversion coefficient, the quantization information, the unit visual / auditory distortion, and the digital watermark information. Based on the perceptual distortion, a quantized value correction signal for correcting the watermarkless quantized value from the target signal so as to reduce the visual / audible distortion as a whole signal and to increase the detection rate of the digital watermark. Means for determining and outputting this as a watermark signal; (f) means for adding the watermarkless quantized value and the watermark signal to output a watermarked quantized value; and (g) the watermarked quantized value. And a means for generating a code string from and outputting the program, and a program for causing each of the above means (a) to (g) to function on a computer constituting the system. 10. A system for encoding an image and inserting a digital watermark, comprising: (a) means for performing motion compensation on a locally decoded image to generate and output a motion-compensated predicted image; and (b) inputting. Means for obtaining and outputting a prediction error image by subtracting the motion compensated prediction image from the image, and (c) performing linear transformation for projecting the prediction error image onto a spatial frequency component to obtain and output a prediction error image conversion coefficient. (D) means for quantizing the prediction error image transform coefficient, outputting the quantized value as a watermarkless quantized value, and outputting quantization information, and (e) reversing the watermarkless quantized value. Means for quantizing and outputting a watermark-less dequantized value; (f) means for performing the linear transformation on the motion-compensated predicted image to obtain and output a predicted image conversion coefficient; and (g) the watermark-free dequantized value. When (H) means for adding the predicted image conversion coefficient and outputting a watermarkless decoded image conversion coefficient, (h) means for performing the linear conversion on the input image to obtain and output an original image conversion coefficient, and (i) means for outputting the original image conversion coefficient. Means for calculating and outputting a unit visual distortion corresponding to each of the original image transform coefficients from the image transform coefficients based on a visual model, and (j) the decoded image transform coefficient without watermark, the original image transform coefficient, and the quantization information. And a target signal is calculated from the unit visual distortion and the digital watermark information. Further, based on the unit visual distortion, the watermark is visually reduced as a whole image, and the detection rate of the digital watermark is increased. Means for determining a quantization value correction signal for correcting the no-quantization value from the target signal, and outputting this as a watermark signal; (k) adding the watermark-less quantization value and the watermark signal; Means for calculating and outputting a watermarked quantized value; (l) means for generating and outputting a code string from the watermarked quantized value; and (m) inversely quantizing the watermarked quantized value to obtain a watermarked inverse. Means for outputting a quantized value; (n) means for performing inverse transformation of the linear transformation on the watermarked inverse quantized value to obtain and output a local decoding prediction error image; and (o) the local decoding prediction error image. Means for adding the motion-compensated prediction image to and outputting a result of the addition as the locally decoded image. A program for causing each of the means (a) to (o) to function on a computer constituting the system is stored. recoding media. 11. A system for encoding an image and inserting a digital watermark, wherein (a) means for performing a linear transformation for projecting an input image onto a spatial frequency component, and obtaining and outputting an original image transformation coefficient. (B) means for generating and outputting a motion-compensated predicted image by performing motion compensation on the locally decoded image; and (c) performing the linear transformation on the motion-compensated predicted image to obtain and output a predicted image transform coefficient. (D) means for subtracting the predicted image conversion coefficient from the original image conversion coefficient and outputting the result as a predicted error image conversion coefficient; (e) quantizing the predicted error image conversion coefficient and watermarking the quantized value Means for outputting as the no quantization value and outputting quantization information; (f) means for dequantizing the watermarkless quantization value and outputting a watermarkless inverse quantization value; Means for adding the watermark-less dequantized value to a number and outputting a watermark-free decoded image transform coefficient, (h) based on a visual model from the original image transform coefficient,
Means for calculating and outputting a unit visual distortion corresponding to each of the original image transform coefficients; (i) the watermarkless decoded image transform coefficient, the original image transform coefficient, the quantization information, the unit visual distortion, and the digital watermark information From the above, a target signal is calculated, and further, based on the unit visual distortion, the watermark-free quantization value is corrected so that the distortion is visually reduced as a whole and the digital watermark detection rate is increased. Means for determining a value correction signal from the target signal and outputting it as a watermark signal; (j) means for adding the watermarkless quantized value and the watermark signal and outputting as a watermarked quantized value; k) means for generating and outputting a code string from the watermarked quantized value; (l) means for dequantizing the watermarked quantized value and outputting a watermarked dequantized value; Means for performing an inverse transformation of the linear transformation on the intricate inverse quantization value to obtain and output a local decoding prediction error image; and (n) adding the motion compensation prediction image to the local decoding prediction error image, Means for outputting a decoded image; and a recording medium storing a program for causing each of the above means (a) to (n) to function on a computer constituting the system.