WO1999067942A1

WO1999067942A1 - Signal processing apparatus and method, and signal decoding apparatus and method

Info

Publication number: WO1999067942A1
Application number: PCT/JP1999/003328
Authority: WO
Inventors: Nobuyoshi Miyahara; Yoichi Yagasaki; Kazuhisa Hosaka
Original assignee: Sony Corporation
Priority date: 1998-06-22
Filing date: 1999-06-22
Publication date: 1999-12-29

Abstract

A coded bit string and additive information are inputted into a watermark adder. The inputted coded bit string is simply decoded by a simple decoder so as to seek the position of the code for adding additive information f. The inputted code is decoded with reference to a coding table device for coding the differential value of the DC component of a DCT by the simple decoder. If the inputted additive information f is on, position information p on the position of the addition object is delivered to a watermark pattern checking controller from the additive information adder so as to add a watermark pattern to the code which is the object on a coded bit string.

Description

^Item: signal processing apparatus and method, and signal decoding apparatus and method TECHNICAL FIELD The present invention is an input signal, for example with respect to image data such as a specific still image or a moving image sequence, for example, with such as copyright information The present invention relates to a signal processing device and method for adding pertinent information and detecting and using the accompanying information during reproduction, and a signal decoding device and method for decoding an input signal to which the accompanying information is added. Scenery technology There is a technology that adds information associated with a specific image sequence (still image or moving image sequence) to the image data and detects and uses the accompanying information during playback. A typical example is the addition of copyright information.

If an unspecified user can use a specific image, the copyright holder must first copy the copyright information to the image in order to claim that right. It must be added in the evening. By adding copyright information, if copyright information that should render the image data undisplayable is detected in the processing procedure of the image playback device or playback method, the image data is displayed. It will be possible to take countermeasures such as not taking action. Attachment or detection of the above-mentioned copyright information is often used at present to prevent illegal copying of video tapes. Recently, there are many shops that rent video tapes, but if many users enjoy illegally copying video tapes borrowed from the store at low prices, those who hold copyrights of the video tapes and video tapes The damage to stores that rent rentals is enormous.

Since image data is recorded on a video tape in an analog manner, the image quality is degraded when copying is performed. Therefore, if copying is repeated several times, it is extremely difficult to maintain the original image quality.

On the other hand, in digital devices that record and reproduce image data digitally, etc., which have recently become widespread, the damage caused by unauthorized copying is even greater. This is because, in equipment that handles image data digitally, image quality does not deteriorate in principle due to copying. For this reason, prevention of unauthorized copying in devices that process digitally is much more important than in the case of analog.

There are two main methods for adding information accompanying image data to the image data.

The first method is to add the image data to the auxiliary part. For example, on a video tape, as shown in Fig. 1, auxiliary information of the image data is recorded at the upper part of the screen (auxiliary part) which is not substantially displayed on the display screen. By using a part of this area, it is possible to add additional information.

The second method is to add it to the main part of the image data (the part that is actually displayed). This is a particular feature, as shown in Figure 2. This is to add a fixed power mark (Water Mark) to the whole or part of the image to the extent that it cannot be visually perceived. As a specific example, there is a spread spectrum for adding or detecting information by using a key pattern generated by using a random number, an M sequence, or the like.

In the following, an example of adding or detecting accompanying information to the main part of the image data when using the watermark pattern is described. 3 to 6 show specific examples.

It is assumed that, for each pixel, one of two symbols, plus or minus, is used, as shown in FIG. In practice, it is desirable to take one of the two symbols at random in the War-Ma-Kupa evening, and the shape and size of that area may be arbitrary.

When adding the accompanying information, an area having the same size as the area of the watermark pattern is set on the image to be added. The set area and the watermark pattern are overlapped and illuminated, and the value a is added to the pixel corresponding to the plus symbol, and the value b is subtracted from the pixel corresponding to the minus symbol. Both a and b can be set to arbitrary values, but they should be kept constant throughout the war.

In the examples of Figs. 4 to 6, a = l and b = l are set. As shown in Fig. 4, when the pixel values of the area to be added are all 100, this padding is performed. The pixel values 101 and 99 are formed by the embedding operation.

When detecting the accompanying information, an area having the same size as the area of the short-time mark pattern is set on the image to be detected. This Is used as the evaluation value. When summing all the pixels, the set area and the watermark pattern are overlapped and illuminated, and the addition is performed for the pixel of the plus symbol and the subtraction is applied to the pixel of the minus symbol. In the example of FIG. 5, the pixel having the pixel value 101 is added, the pixel having the pixel value 99 is subtracted, and the calculation results are added. At this time, if the same pattern as the watermark pattern is not used when adding the accompanying information, the accompanying information cannot be correctly detected. By such a detection operation, for example, as shown in FIG. 5, the evaluation value when the additional information is added becomes (4n) ² (the same as the number of pixels included in the area), and FIG. As shown, the evaluation value is 0 when no additional information is added.

If the area of the war evening mark pattern is sufficiently large and the war evening mark pattern is sufficiently random, the evaluation value will always be almost 0 when the accompanying information is not added. Therefore, when the evaluation value exceeds a certain threshold value, it can be determined that additional information has been added. The above procedure makes it possible to add binary information (1 bit) indicating whether or not additional information has been added. If more information is to be added, 2 ^k (k bits) information can be added by processing the entire image into k regions and performing the above operations.

As the war evening mark pattern, for example, a pattern generated using an M sequence can be used. The M-sequence (longest code sequence) is a sequence of binary symbols 0 and 1, where the statistical distribution of 0 and 1 is constant, the code correlation is 1 at the origin, and in other cases it is inversely proportional to the code length It is. Of course, the warrior mark pattern may be generated by a method other than the M-sequence. When image data is recorded and reproduced digitally, the amount of information is very large if it is used as it is, so it is common to compress the data overnight. As a method for compressing image data, high efficiency such as JPEG (Joint Photographic Experts Group) (color still image coding method) or MPEG (Moving Picture Experts Group) (color moving image coding method) The encoding method has been internationally standardized and has been put to practical use. Taking the case where image data is compressed by this high-efficiency encoding as an example, a configuration example in which additional information is added and detected will be described below.

Figure 7 shows the configuration of the encoder. First, in the input image data, the accompanying information f is added as a warrior mark pattern in a warrior mark (pattern) adder 71. The image data to which the accompanying information f has been added is input to the encoder 72, and high-efficiency encoding is performed to generate an encoded bit sequence.

FIG. 8 shows a more specific configuration example of encoder 72. In this example, the encoder 72 has a frame memory 141, and the frame memory 141 stores the image data supplied from the watermark adder 71 in units of frames. It has been made like that. The motion vector detector 150 detects the motion vector V from the image data stored in the frame memory 141, and outputs the detection result to the motion compensator 144 and the variable length encoder 144. Output to In the motion vector detector 150, block matching processing is performed for each macroblock composed of 16 × 16 pixels, and the motion vector V is detected. In addition, in order to achieve higher accuracy, matching processing is also performed on a half-pixel basis.

The motion compensator 144 has a built-in frame memory, The pixel value at each position of the current frame to be predicted is predicted from the image that has already been encoded and is obtained by decoding it and stored in the built-in frame memory. The predicted value I '[i, j, t] of the pixel value I [i, j, t] at the position of the frame input at time t is represented by the motion vector ^,, and the cell corresponding to that position. Is determined using the following equation.

[i, j, t] = (I [i ,, j ,, t-T] + I [i, + l, j ,, tT] + I [i ,, j, + l, t-T] + I [i, + l, j, + l, t-T]) / 4

Here, i and j are represented by the following equations.

i '= int (i + vx (i ₅ j, t) T)

j, two int (j + vy (i, j, t) T)

Here, T represents the difference between the time when the currently predicted image I is input and the time when the image stored in the frame memory is input, and I [i, , J ,, tT], I [i, + 1, j ,, tT], I [i ,, j, + l, tT], I [i, + l, j '+ l, t-T] Represents the pixel value on the frame memory built in the motion compensator 144. Int (x) represents the largest integer value that does not exceed X.

The subtractor 1442 performs motion compensation based on the motion vector V supplied from the motion compensator 144 from the value of the pixel currently to be coded supplied from the frame memory 141. The calculated predicted value is subtracted and output to a DCT (Descrete Cosine Transform) unit 144. The DCT unit 144 is a (Descrete Cosine Transform) unit composed of the difference values input from the subtractor 144, and performs two-dimensional DCT processing on the block of 8 pixels x 8 pixels composed of the difference values. . The quantizer 1 4 5 has an appropriate value for the DCT coefficient c input from the DCT 1 Using the step size q, quantization processing is performed according to the following equation.

c, two int (c / Q)

The DCT coefficient c ′ quantized by the quantizer 144 is supplied to the variable length encoder 144 and the inverse quantizer 144. The variable-length coder 146 performs variable-length coding on the quantized DCT coefficient c ′ and the motion vector V supplied from the motion vector detector 150, and outputs a code bit string. You.

The inverse quantizer 147 performs inverse quantization as shown in the following equation using the step size Q at the same position as the step size used in the quantizer 144.

c "= c 'xQ

The data that has been inversely quantized by the inverse quantizer 147 is input to the IDCT unit 148 and subjected to inverse DCT processing to restore the pixel value difference value.

The difference value output from the IDCT unit 148 is added to the predicted value output from the motion compensator 143 by the adder 149, and becomes the original pixel value data. It is stored in the built-in frame memory.

Next, the operation will be described. The digitized image data is input to a warrior mark adder 71, and a warrior mark is added according to the accompanying information f.

The image data to which the night mark is added by the word mark adder 71 is supplied to the frame memory 141, and is stored in frame units. The motion vector detector 150 detects a motion vector V of the image data stored in the frame memory 144. The motion compensator 1 4 3 uses the reference frame stored in the built-in frame memory. The motion compensation is performed on the image data of the image data, and the predicted pixel data is generated and supplied to the subtractor 144. The subtracter 144 subtracts the predicted image data supplied from the motion compensator 144 from the image data supplied from the frame memory 144, and supplies the subtraction result to the DCT 144. . The DCT unit 144 converts the image data of the input difference value into DCT coefficients. The quantizer 145 quantizes the DCT coefficient supplied from the DCT unit 144 and outputs it to the variable length encoder 146. The variable-length encoder 146 converts the input quantized data into a variable-length code, and transmits it as a code bit string to a transmission path (not shown) or supplies it to a recording medium for recording.

The quantized data output from the quantizer 145 is inversely quantized by the inverse quantizer 147 and supplied to the IDCT 148. The 10〇 filter 148 performs IDCT processing on the input DCT coefficient, and outputs an image of the original difference value. The image data of this difference value is added to the predicted image data read out from the motion compensator 144 by the adder 149, and restored to the original image data, and restored to the original image data. This is stored in the built-in frame memory of the unit 144. Note that the variable-length encoder 146 also converts the motion vector V supplied from the motion vector detector 150 into a variable-length code and outputs it.

Normally, INTER coding for coding the difference from the predicted value as described above is performed. However, if the difference between the value of the pixel currently to be coded and the predicted value calculated by the motion compensator 144 is large, the intra-picture coding is performed to prevent the coding bit amount from increasing. (INTRA coding) may be performed. In other words, for each pixel value in the block, the DCT unit 14 4 and the pixel value is coded.

Figure 9 shows the configuration of the decoder. The input coded bit string is restored to image data in the decoder 81. Then, the auxiliary information f is detected by the war mark detector 82.

FIG. 10 shows a more detailed configuration example of the decoder 81. In this configuration example, the inverse variable-length encoder 161 of the decoder 81 performs inverse variable-length encoding (variable-length decoding) on the input code bit sequence and decodes it. The image data (DCT coefficients) are output to the inverse quantizer 162, and the decoded motion vector V is output to the motion compensator 1665. The inverse quantizer 16 2 inverse quantizes the input DCT coefficient and outputs the result to the IDCT device 16 3. The IDCT unit 163 performs IDCT processing on the input, inversely quantized DCT coefficients, restores the original difference value to the image data, and outputs the result to the adder 164. I have. The motion compensator 165 performs motion compensation on the image data stored in the built-in frame memory based on the motion vector V supplied from the inverse variable-length coder 161 to obtain a predicted image. Is generated and output to the adder 164. The adder 1664 is configured to add the difference value supplied from the I0〇cable 163 to the predicted image, restore the original frame image, and output the restored frame image.

The output of the adder 164 is supplied to and stored in a frame memory incorporated in the motion compensator 165, and is also supplied to the watermark detector 82. The watermark detector 82 detects and outputs the accompanying information f from the input image data, and outputs the original image data.

Next, the operation will be described. The inverse variable length encoder 1 6 1 The input code bit string is subjected to inverse variable length encoding processing, and the decoded DCT coefficients are output to the inverse quantizer 162. The inverse quantizer 16 2 inversely quantizes the input DCT coefficient and outputs the result to the IDCT unit 16 3. The IDCT unit 163 performs IDCT processing on the input DCT coefficients, and outputs the original difference image data.

The motion compensator 165 uses the motion vector V supplied from the inverse variable length encoder 161 for the already restored image data stored in the built-in frame memory based on the motion vector V. After performing motion compensation, a predicted image is generated and output to the adder 164. The adder 164 adds the image data of the difference value supplied from the IDCT unit 163 to the predicted image data, and restores the original image data. The original image data is supplied to and stored in the frame memory of the motion compensator 165 for generating the next predicted image.

Further, the image data output from the adder 164 is supplied to the watermark marker detector 82, and the watermark is detected.

In the case where the image data is not efficiently coded, if the configuration excluding the encoder 72 in FIG. 7 or the decoder 82 in FIG. 9 is used, additional information is added to the image data or detected. be able to.

The configurations of the Waryu mark adder 71 and the Waryu mark detector 82 are shown in Figs. 11 and 12, respectively.

FIG. 11 shows the configuration of the watermark adder 71. The input image data and accompanying information f are passed to the accompanying information adder 111. When the incidental information is on, a watermark is added to each pixel on the target image area. To this end, the incidental information adder 111 first sets the pixel position p to be added to the target image area. Mark pattern collation controller Hand over to 1 1 2 The pixel position p may be, for example, a one-dimensional position expression that indicates the position in the scanning order starting from the upper left position of the image. Based on the pixel position P, the warner mark pattern matching controller 112 refers to the symbol of the warner mark pattern stored in the warner mark pattern holding memory 113, and obtains the obtained symbol S. Pass to the additional information adder 1 1 1 Using the passed symbol S, the additional information adder 1 11 1 adds a warp mark pattern to the pixel to be added.

FIG. 13 shows a series of processing performed by the warrior mark adder 71. First, in step S 14 1, predetermined values are set to the additional levels a and b of the short mark. In step S142, an area of the same size as that of the warp mark pattern is set on the pixel to which the additional information is to be added, and each pixel in the area is compared with the warp mark pattern. I do. In step S144, the symbol of the war evening mark is determined, and if the symbol of the corresponding war evening mark is positive, a is added to the pixel in step S144. If the symbol of the war symbol corresponding to the pixel is negative, b is subtracted from the pixel in step S145. This process is repeated until it is determined in step S146 that all pixels in the target area have been processed.

FIG. 12 shows the configuration of the blue mark detector 82. The input image data is passed to the evaluation value calculator 132. In order to calculate an evaluation value from each pixel on the target image area, the evaluation value calculator 1332 passes the pixel position p of the evaluation target to the warp mark pattern collation controller 131. This pixel position p is, for example, It is a one-dimensional position expression that indicates the position in the scanning order starting from the position of. The watermark pattern matching controller 13 1 refers to the symbol of the watermark pattern stored in the memory pattern holding memory 13 3 based on the pixel position p, and evaluates the obtained symbol S. Pass to value calculator 1 3 2 The evaluation value calculator 13 2 calculates the evaluation value using the passed symbol S. The calculated evaluation value is subjected to threshold processing by the evaluation value comparator 134, and the accompanying information f is output. The input image data is output through the image converter 135 as it is or after being subjected to predetermined processing or processing.

FIG. 14 shows a series of processes performed by the war mark detector 22. First, in step S1661, the evaluation value sum is initialized and the threshold value th is set. In step S162, an area having the same size as that of the watermark pattern is set, and each pixel in the area is compared with the watermark pattern. If it is determined in step S166 that the symbol of the corresponding watermark symbol is positive, the pixel value is added to the evaluation value sum in step S166. If the symbol of the warp mark of that pixel is negative, the pixel value is subtracted from the evaluation value sum in step S165. This processing is repeated in step S166 until it is determined that the processing has been performed for all the pixels in the target area. Then, in step S166, the evaluation value sum is compared with the threshold value th. If sum> th, it is considered that additional information has been added, and in step S168, the additional information is added. Turn f on. If not, the accompanying information f is turned off in step S169.

Additional information To prevent unauthorized copying, for example, Used for In the case of the decoder of FIG. 9 as an example, the output image data and the attached information f are passed to an image display unit (not shown). The image display unit displays the image as it is when the accompanying information f is on, but does not display the image, for example, does not display the main area of the image when the accompanying information f is off. Perform processing or processing such as scrambling the image (displaying the received image data randomly). Alternatively, an image converter 135 shown in the watermark detector 82 of FIG. 12 may be provided to add or process such image data according to the accompanying information f. .

By the way, the above two methods of adding information accompanying the image data to the image data have the following problems.

Regarding the first method of adding the additional information to the auxiliary part of the image data, if the auxiliary part to which the additional information is added is ignored, it is difficult to prevent problems such as illegal copying in advance. For example, if a digitally recorded image is read into a commercially available personal computer, and only the main part is cut out and copied, ignoring the auxiliary part, the image quality will be the same as before copying. Will be the same. In this case, the meaning of adding the auxiliary information to the auxiliary part is completely lost. In the second method of adding the accompanying information to the main part of the image data, the attached information does not disappear and disappear, for example, by the copy procedure shown in the first method. However, when various kinds of signal processing such as noise reduction filtering are performed on image data, the additional information component added may be attenuated and may not be extracted.

In particular, when the original image data itself is compressed using high-efficiency coding such as JPEG or MPEG, the adverse effects due to the quantization process Often appears. The added auxiliary information components are amplified to such a degree that they can be visually perceived by these high-efficiency coding quantization processes, and the image quality is reduced or attenuated to such a degree that they cannot be extracted. The original meaning may be lost.

There is also a method of adding the additional information using a special area in the image so that the component of the additional information is not changed by the signal processing. However, since such an area is only a part of the entire image sequence, the area of the watermark pattern cannot be made sufficiently large. Therefore, the reputation becomes a large value other than 0 even when the accompanying information is not added, and when the threshold value exceeds a certain threshold, it is determined that the accompanying information is added. The use of statistical criteria makes it very difficult to detect incidental information.

Furthermore, if the area where the additional information is added is only a part of the entire image sequence, it is extremely difficult to add multiple pieces of information. For example, if the entire image is divided into k regions and the accompanying information is added, the area of the war evening mark pattern for each region is further narrowed according to the number of the regions, and the accompanying information is almost undetectable. It is possible. Note that these problems described above become particularly remarkable in a moving image sequence.

Also, depending on the technique described in the related art, it is necessary to uniformly change the value of the Water Mark to be added within the range. For example, when a watermark is added to the DC component (DC component) of the DCT, the luminance values of all the pixels in the 8x8 block on the image to be DCT are illuminated. For this reason, block noise may be visually perceived as an effect of image quality degradation due to the addition of a warrior mark. there were. New measures had to be taken to reduce the visual impact.

Also, for example, if the accompanying information is used for illegal copy prevention or generation management, for example, even if an illegal act such as not rewriting the added attached information is performed, the inconsistency of the accompanying information caused by that act There was no way to prevent it. DISCLOSURE OF THE INVENTION According to the present invention, it is possible to add accompanying information such as watermarks while suppressing signal deterioration such as image quality deterioration, and to configure a method for adding and detecting new accompanying information. Provided are a signal processing device and a signal processing method that can be used for prevention of illegal copying and generation management. Further, the present invention provides a signal decoding device and method capable of detecting accompanying information without significantly affecting original image data.

For this reason, a signal processing device according to the present invention is a signal processing device for embedding additional information in an input signal, wherein the position where the input signal is decoded and the additional information is added is determined by a variable length portion and a fixed length portion. Additional position detecting means for detecting the position within the combined range, and additional information adding means for adding the additional information to the position detected by the additional position detecting means are provided.

A signal processing method according to the present invention is a signal processing method for embedding incidental information in an input signal. Within the additional position detection step, and the above-described position detected in the additional position detection step. An additional information adding step of adding the additional information.

A signal processing device according to the present invention is a signal processing device that embeds incidental information in an input signal, wherein the additional position for decoding the input signal and detecting a position to add the incidental information within a range of a variable length portion. A detecting unit; and an additional information adding unit that receives the information on the position detected by the additional position detecting unit and the input signal, and adds the additional information to the position.

A signal processing method according to the present invention is a signal processing method for embedding incidental information in an input signal, wherein the additional position is obtained by decoding the input signal and detecting a position to add the incidental information within a range of a variable length portion. A detecting step; and an additional information adding step of receiving the information regarding the position detected in the additional position detecting step and the input signal, and adding the additional information to the position.

A signal decoding device according to the present invention is a signal decoding device for decoding an input signal in which incidental information is embedded. The signal decoding device decodes the input signal and determines a position where the additional information is added to a variable length portion and a fixed length portion. And an additional information extracting means for extracting the additional information based on the position information on the position detected by the additional position detecting means.

A signal decoding method according to the present invention is a signal decoding method for decoding a signal in which incidental information is embedded, wherein the input signal is decoded, and a position where the incidental information is added is a variable length portion and a fixed length portion. And an additional information extracting step of extracting the additional information based on the position information on the position detected in the additional position detection step. A signal decoding device according to the present invention is a signal decoding device for decoding an input signal in which incidental information is embedded. The signal decoding device decodes the input signal and detects a position where the incidental information is added within a range of a variable length portion. An additional position detecting means, and additional information extracting means for receiving the position information on the position detected by the additional position detecting means and the input signal and extracting the accompanying information.

A signal decoding method according to the present invention is a signal decoding method for decoding an input signal in which incidental information is embedded as a watermark, wherein the input signal is decoded, and a position where the incidental information is added is set to a variable length portion. An additional position detecting step of detecting the position within the range; and an additional information extracting step of receiving the input information and the position information relating to the position detected in the additional position detecting step and extracting the additional information.

The signal processing device according to the present invention is a signal processing device that embeds incidental information in an input signal, by performing a necessary modification on a signal before quantization to change a value after quantization. It has an additional information adding means for adding the additional information, and an encoding means for performing encoding including quantization on an output signal from the additional information adding means.

In the signal processing method according to the present invention, in a signal processing method for embedding accompanying information in an input signal, a signal necessary for changing a value after quantization is applied to a signal before quantization. It has an additional information adding step of adding the additional information, and an encoding step of performing encoding including quantization on an output signal from the additional information adding step.

A signal processing device according to the present invention is a signal processing device for embedding incidental information in an input signal, wherein the value within the range of a block consisting of a plurality of data of the input signal is corrected so that the representative value within the block range becomes regular. To have Thus, there is provided additional information adding means for adding the additional information. A signal processing method according to the present invention is a signal processing method for embedding incidental information in an input signal, wherein a value in a block range consisting of a plurality of data of the input signal is corrected to have regularity in a representative value in a block range. Accordingly, an additional information adding step of adding the additional information is provided. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram for explaining a recording position of auxiliary information on a video tape.

FIG. 2 is a diagram for explaining the war marks.

FIG. 3 is a diagram illustrating an example of a pattern of a short-lived mark.

FIG. 4 is a diagram illustrating an operation of adding a war mark. FIG. 5 is a diagram for explaining evaluation values when a war mark is added.

FIG. 6 is a diagram for explaining evaluation values when no war mark is added.

FIG. 7 is a block diagram showing a configuration of a conventional encoder. FIG. 8 is a block diagram showing a configuration of a conventional encoder inside an encoder.

FIG. 9 is a block diagram showing a configuration of a conventional decoder.

FIG. 10 is a block diagram showing a configuration of a decoder inside a conventional decoder.

FIG. 11 is a block diagram showing a configuration of a conventional watermark adder. FIG. 12 is a block diagram showing a configuration of a conventional blue mark detector.

FIG. 13 is a flowchart for explaining the operation of the watermark adder shown in FIG. 11 above.

FIG. 14 is a flowchart for explaining the operation of the watermark detector shown in FIG. 12 above.

FIG. 15 is a block diagram showing a configuration of an image signal processing device according to an embodiment of the image signal processing device and method of the present invention.

FIG. 16 is a detailed block diagram of the watermark adder constituting the image signal processing device.

FIG. 17 is a flowchart for explaining the operation of the watermark adder shown in FIG. 16 above.

FIG. 18 is a block diagram showing a detailed configuration of the watermark adder in the earlier application.

FIG. 19 is a flowchart for explaining the operation of the watermark adder in the earlier application shown in FIG.

FIG. 20 is a diagram for explaining a process of rewriting a fixed-length encoded portion.

FIG. 21 is a diagram for explaining the structure of G0P.

FIG. 22 is a diagram for explaining the structure of a macroblock. FIG. 23 is a diagram illustrating encoding of a DC component of a DCT coefficient.

FIG. 24 is a block diagram showing a configuration of an image signal decoding device according to an embodiment of the image signal decoding device and method according to the present invention.

FIG. 25 is a block diagram of the image signal decoding device shown in FIG. FIG. 4 is a block diagram showing a detailed configuration of the evening mark detector.

FIG. 26 is a flowchart for explaining the operation of the watermark detector shown in FIG. 25 above.

FIG. 27 is a block diagram showing a configuration of a modified example of the watermark adder shown in FIG. 16 described above.

FIG. 28 is a flow chart for explaining the operation of the modification shown in FIG. 27 above. FIG. 29 is a block diagram showing a configuration of a modification of the power mark detector shown in FIG. 25.

FIG. 30 is a flowchart for explaining the operation of the modification shown in FIG. 29 described above.

FIG. 31 is a block diagram showing a schematic configuration of the second embodiment according to the present invention.

FIG. 32 is a diagram for explaining an example in which the average value is increased by one by increasing some pixel values in the block by one.

FIG. 33 is a flowchart for explaining the operation of the second embodiment according to the present invention. FIG. 34 is a flowchart for explaining the operation of the second embodiment according to the present invention.

FIG. 35A is a block diagram showing a configuration of a watermark adder constituting a third embodiment according to the present invention.

FIG. 35B is a block diagram showing the configuration of the watermark detector constituting the third embodiment.

FIG. 36A and FIG. 36B are flow charts for explaining the additional information adding operation performed by the above-mentioned watermark adder. FIG. 37 is a flowchart for explaining the accompanying information detection operation of the third embodiment according to the present invention. BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, a first embodiment of a signal processing apparatus and method according to the present invention will be described with reference to the drawings. In the first embodiment, as shown in FIG. 15, an encoder 2 for performing encoding processing on an image and a watermark addition for adding a watermark to an encoded output of the encoder 2 are provided. This is an image signal processing device comprising the device 1.

The configuration and operation of the encoder 2 are the same as those of the encoder 72 shown in FIG. 8 described above, and the description is omitted here.

On the other hand, the configuration of the warrior mark adder 1 in FIG. 15 is different from that of the warrior mark adder 71 shown in FIG. As shown in the figure, a simple code decoder 11, ancillary information adder 12, a DC component difference value encoding table unit 13, a ゥ —even mark pattern matching controller 14, And a mark pattern holding memory 15.

The watermark bit adder 1 receives an encoded bit string and accompanying information. The input coded bit string is simply decoded by the simple code decoder 11 to find the position of the code to which the additional information is added. The simple code decoder 11 decodes the input code with reference to a coding table unit 13 for coding, for example, a difference value of a DC component of DCT (DC component).

Then, the encoded code obtained as a result of decoding by the simple code decoder 11 is used. The code position information b indicating the position of the target code when adding the additional information f on the packet sequence is passed to the additional information adder 12 together with the input encoded bit sequence. When the input accompanying information f is 0 n, the accompanying information adder 12 adds the position information p to be added to the target code on the coded bit string in order to add a watermark pattern. It is passed to evening pattern matching controller 14.

In high-efficiency coding such as MPEG, for example, coding is performed in units of macroblocks or blocks, so that the position information P of the code to be added generally indicates their spatial position . Specifically, for example, a one-dimensional position expression that indicates the position in the scanning order from the upper left position of the image as a starting point, and which coordinate in space also uses the upper left position of the image as the origin For example, a two-dimensional position expression that indicates whether the position is located is used.

The watermark mark pattern matching controller 14 refers to the symbol of the watermark mark pattern recorded in the memory mark holding memory 15 based on the position information p, and obtains the obtained symbol S as accompanying information. Hand over to adder 1 and 2. Using the passed symbol S, the additional information adder 12 adds a watermark pattern to the code to be added. Specifically, corrections are made, such as replacing the code to be added with a new code.

Here, the encoding table unit 13 for encoding the difference value of the DC component will be described in detail.

When encoding the differential value of the DC component, the DC component is represented by the size and the actual value (DC Differential) represented by the size. The former is a variable length code (VLC), the latter is a fixed length code (FLC: Fixed Length Code).

In the first embodiment, the accompanying information is added within a range in which the variable-length code portion and the fixed-length code portion are combined, so that the code length as a whole is not changed.

For example, in the case of a luminance signal, the size of the DC component (DC difference component) is defined as shown in Table 1 below.

【table 1】

DC component size of DCT for luminance 1? W J! ¾ v ^-J Review

τ η tifr + Knop,

Λ

丄 UU U 丄

υ 丄

101 3

110 4

1110 5

11110 6

111110 7

1111110 8

In addition, the color difference signal is specified as shown in Table 2 below [Table 2].

DC component size of DCT for color difference

Then t_i left ^ fe V ¹

^ ^ Live

η

fl 丄 1 丄

10 2

110 3

1110 4

11110 5

111110 6

1111110 7

11111110 8

For example, when the size of the DC component (DC difference component) is 3, the DC difference component is defined as shown in Table 3 below.

[Table 3]

DCT DC difference

Therefore, for example, in the case of a luminance signal, when the size of the DC component (DC difference component) is 3 and the actual DC difference component zz [0] is −6, the difference value is represented by 101001.

Based on the difference value encoding table of the DC component of the luminance signal in MPEG2, some code examples according to the actual value of the DC difference component zz [0] will be described below.

zz [0] sign

0 100 (VLC: 100, FLC: None)

+1 001 (VLC: 00, FLC: 1) -1 000 (VLC: 00, FLC: 0)

From the above, it can be seen that changing the value of zz [0] in the range of -1, 0, +1 does not change the code length. For example, if zz [0] = 0 is changed to 1, the VLC part changes from 3bit to 2bit, and the FLC part changes from Obit to lbit, but only 3bit becomes 3bit in total.

The configuration in the first embodiment is as shown in FIGS. 15 and 16 described above. The point that a watermark mark adder 1 is provided after the encoder 2 and the Characteristically, an encoding table 13 is provided inside the adder 1, and the target of correcting the encoded bit string is a fixed-length code portion and a variable-length code portion.

By the way, the applicant has already corrected the coded bit sequence in the specification and drawings of 1997 Patent Application No. 252819 (corresponding US Application No. 144845). The present invention clarifies the invention "image data overnight processing apparatus and method, transmission medium, and recording medium" in which the object is a fixed-length code portion.

In the earlier application, the word mark adder 1 shown in FIG. 16 is replaced with the word mark adder 3 shown in FIG. 18, and the encoding table 16 has a fixed length code table. Was required.

In the war mark adder 3, as shown in FIG. 19, first, the addition levels a and b of the short mark are set in step S1. Next, in step S2, the encoded bit sequence is read up to the position of the target code when the additional information is added on the encoded bit sequence. Thereafter, as shown by the dotted line in step S3, a correction may be made for the mismatch generated in the immediately preceding block. This will be described later. In step S4, matching with the watermark pattern is performed according to the block position of the target code on the coded bit string. If, for example, additional information is added to the code of the DC component of the DCT, and if the symbol of the watermark at the block position is a brass in step S5, the process proceeds to step S6 and the code is displayed. A value x 'is calculated by adding a to the DC component X to be calculated. If the symbol of the watermark at the block position is negative in step S5, the flow advances to step S7 to calculate a value x 'obtained by subtracting a from the DC component X represented by the sign. Then, in step S8B, the sign of the DC component X at that block position is corrected to the sign of x '. When making corrections, only the fixed-length code is changed, referring mainly to the table that encodes the difference value of the DC component of the DCT (DC component). Since only the fixed-length code part is changed, this correction process is merely a code replacement. Then, the determination in step S9 is repeated, and the process ends after performing these processes for all the codes in the target area.

Here, a specific example of the correction in step S3 will be described. In MPEG, the DC component of the DCT is subjected to differential coding (DPCM: Differential Pulse Code Modulation). To give a specific example of differential encoding, if there is a sequence of 3, 6, 12, 4, 7 then calculate the difference from the immediately preceding numerical value, for example, the sequence of 3, 3, 6, -8, 3 And encoding is performed. At this time, for example, if the third difference value is replaced with 4 and additional information is added, the values after the 4th will be 2 smaller than the original values unless correction is made. To correct the fourth and subsequent numerical values without replacing the third numerical value and adding the accompanying information, replace the fourth differential value with -6, which is larger by two. Thus, the supplement When performing the correction, the processing method may be determined according to the method of adding the accompanying information.

In the above description, an example has been described in which accompanying information is added in units of macro blocks or blocks, for example. More generally, as long as the necessary configurations are taken and the processing is performed, the watermark pattern may be added in any unit, not limited to macroblocks or blocks.

However, the actual coded bit sequence is generated according to various restrictions such as a coding rate. As an example, when the code to be added is replaced with a new code, the replacement often changes the word length of the code, causing various problems.

In order to prevent these problems, the watermark mark adder 3 shown in Fig. 18 examines the target code when adding the accompanying information on the coded bit string, Various measures are taken, such as adding additional information only to the long code part.

This will be described with reference to FIG. In MPEG, fixed-length encoding is performed using a DC component of DCT encoded in units of a block, a motion vector encoded in units of a macro block, and the like. In particular, when the code of the DC component of the DCT is selected as an object to be added, this value is fixed-length coded on a block-by-block basis in JPEG and the like, so that versatility is enhanced. FIG. 21 illustrates the principle of appending additional information in the MPEG system. As shown in the figure, G0P (Group Of Pictures) is composed of one I picture, a plurality of P pictures, and a plurality of B pictures. In the specific example of FIG. 21, one G0P is constituted by 15 pictures. DCT DC encoded in units of work When adding additional information to a component, configure G0P

I picture is selected from.

As shown in FIG. 22, the macro block of 16 × 16 pixels is composed of four blocks of 8 × 8 pixels in the case of the luminance signal (Y). The color difference signals Cb and Cr are represented by a block of 8x8 pixels for one macroprogram of 16x16 pixels. These pixel data are converted to DCT coefficients by DCT (Discrete Cosine Transform).

8 8 1) (1 ¹ coefficient of one Proc (06 [0] [0] - (06 [7] [7] is quantized with a predetermined quantization step, quantization levels QF [0] [0] to QF [7] Converted to [7].

Of the DCT coefficients, Coeff [0] [0] (scan [0]) in the upper left represents the DC component (DC component). From this DC component, the DC component of the block immediately before was used as the predicted value. A difference value is calculated, and the difference value is encoded. The remaining AC components (AC components) are coded by zigzag scanning within the block, after being sorted as scan [0] as a direct component, followed by scan [l] to scan [63]. You.

As shown in FIG. 23, of the DCT coefficients, the difference between the DC component and the immediately preceding DC component is calculated, and the difference value is encoded. In the case of a luminance signal, the order of the four blocks is as follows: upper left, upper right, lower left, lower right. Therefore, as the DC component of the upper left block, the difference from the DC component of the lower right block of the immediately preceding macroblock is encoded, and as the DC component of the upper right block, the difference from the DC component of the upper left block is coded. The DC component of the lower left block is coded as the difference between the DC component of the upper right block, and the DC component of the block of the lower right block is the difference between the DC component of the lower left block and the DC component of the lower left block. Encoded. For color difference signals, The difference between the immediately preceding corresponding color difference signal and the DC component of the block is encoded.

As described above, when encoding the difference value of the DC component, the DC component is represented by a size and an actual value represented by the size (DC Differential). The former is a variable length code (VLC: Variable Length Code), and the latter is a fixed length code (FLC: Fixed Length Code).

When the additional information is added, if the DC difference component is changed within a range where the size of the DC component (DC difference component) does not change, the additional information is added using only the fixed-length code portion. can do.

For example, consider a case where the size of the DC difference component of the luminance signal is 3, and the actual DC difference component zz [0] is -6. When the symbol of the warrior mark pattern is plus and the additional level a is 1, if the difference value is replaced from 101001 to 101010, the actual DC difference component becomes -5, which is one larger than -6. If the symbol of the warrior mark pattern is negative and the additional level b is 1, if the difference value is replaced from 101001 to 101000, the actual DC difference component becomes -7, which is one smaller than -6.

Similarly, if the size of the DC difference component of the luminance signal is 3 and the actual DC difference component zz [0] is -6, if the symbol of the warrior mark pattern is plus and the additional level a is 3, the fixed length Ancillary information cannot be added using only the code part. In this case, no additional information is added at this block position.

Alternatively, for example, up to the addition level 2, since the additional information can be added in a fixed-length code portion, the difference value may be replaced with 101001 to 101011. At this time, the fact that the actual DC difference component has become -4, which is larger than -6 by 2 is recorded in a register or the like and the next addition to the DC difference component is performed. Should be used by referring to the value.

When the size of the DC component is 0, that is, when the actual DC difference component zz [0] is 0, additional information cannot be added using only the fixed-length code portion, and the additional information is added at the block position. No information is added. Alternatively, additional information may be added by some processing. The actual deviation of the DC component due to the addition of the accompanying information should be recorded in a registry or the like, and the value should be used when the next addition to the DC difference component is performed. For example, if the actual value of the DC component is larger than the original value by 2 due to the addition of the preceding accompanying information, R = 2 is recorded in the register. When the next addition is performed, when the actual DC difference component zz [0] is 7, the size of the DC difference component of the luminance signal is 3. If the symbol of the watermark pattern is negative and the additional level b is 1, considering that the DC component is differentially coded, -Rb =-2-1 = -It is necessary to change the DC difference component by 3. Therefore, replace the difference value from 101111 to 101100, change the actual DC difference component from 7 to 4, and record that the actual value of the DC component has decreased by 1 by R = 2-3 = -Set to 1. The actual deviation of the DC component generated by adding the accompanying information may be recorded using a method other than the above.

In addition, when appending the additional information using the DC component (DC difference component), any method other than the above may be used.

In the above-mentioned prior application, the case where the additional information is added using the DC component (DC difference component) has been described. However, the Motion_residual in the code obtained by encoding the difference value of the motion vector (motion vector) is described. (FLC) can also be used. That is, in the MPEG system, as described in the conventional example of FIG. 8, the motion vector detector 150 detects the motion vectors of the P picture and the B picture, encodes them, and encodes them in the bit stream. It is intended to be transmitted together. This Motion Vector is represented by Motion-code as VLC as shown in Tables 4 and 5, and Motion_residual as FLC. Motion-code represents a rough value of the Motion Vector, and Motion-resi dual represents a correction value for representing a fine value. Also,: f_code indicates the accuracy (magnification) of Motion-code.

[Table 4]

Variable length codes for motion— code

Variable length code motion_code L r '' L s J L t J

0000 0011 001 -16

0000 0011 011 -15

0000 0011 101 -14

0000 0011 111-13

0000 0100 001 -12

0000 0100 011-11

0000 0100 11 10

0000 0101 01 -9

0000 0101 11-8

UUUU Ul l l -7

0000 1001 -6

0000 1011 -5

0000 111-4

0001 1 -3

0011 -2

011-1

[Table 5]

Variable length codes for motion— code

For example, if f-code is 1, then Motion-code represents a value with 0.5 precision. As a result, a sufficiently fine value is represented, and in this case, Motion_residual is not used.

If f-code is 2, Motion-code represents integer precision, and Motion-resi dual represents a value with 0.5 precision. That is, at this time, the Motion residual is Expressed as a 1-bit FLC indicating 0 or 0.5.

Further, if f-code is 3, Motion-code represents a value with a precision of a multiple of 2, and Motion-residual represents a 2-bit FLC representing 0, 0.5, 1.0 or 1.5. Become.

As in the case of DC Differential, when Motion-code is 0, Motion_residual does not exist.

As in the case of DC Differentia 1 described above, accompanying information can be added to such Motion-residual FLC. When detecting the accompanying information, for example, an evaluation value may be calculated by adding and subtracting a motion vector or a difference value thereof. The accompanying information may be detected using other methods.

Although the Motion residual exists in the P picture and the B picture, if the Motion residual of the B picture is used, the B picture is not used for the prediction of other pictures. It is possible to prevent other pictures from being affected by the insertion.

The additional information may be added using a code other than fixed length (variable code), but in this case, unnecessary bits are inserted into the coded bit sequence, It was necessary to take the necessary configuration, such as removing unnecessary bits above, before processing. These can be similarly applied to any encoding method or any decoding method.

For this reason, as shown above, the encoder table unit 13 is provided with a fixed-length code and a variable-length code in the power mark adder 1 shown in FIG.

To reiterate, the war symbol adder 1 shown in Figure 16 is The difference from the watermark adder 3 shown is the coding table unit 13 referred to by the simple code decoder 11. In the encoding table unit 13, not only fixed-length codes but also variable-length codes are indispensable. In the figure, an example is shown in which an encoding table of DC component difference values is used. FIG. 17 shows a series of processes performed by the watermark adder 1 in FIG. The coding table referred to in step S8A to correct the target code is different from the coding table referred to by the watermark adder 3 shown in FIG. In the earlier application, it was sufficient to prepare only a fixed-length code encoding table, but in the present embodiment, it is necessary to prepare a variable-length code table as well. In the figure, an example is shown in which an encoding table of the difference value of the DC component is used.

As described above, the image signal processing apparatus can add the accompanying information without re-encoding by the watermark adder 1, and has a code length within a range including the variable length code portion and the fixed length code portion. Additional information can be added without changing the information.

Next, embodiments of the signal decoding apparatus and method according to the present invention will be described. In this embodiment, as shown in FIG. 24, the input code bit string is first input to the Warner mark detector 22 to detect the Warner mark, and then the Warner mark detector This is an image signal decoding device in which the code bit string output from 22 is supplied to the decoder 21 and decoded.

The configuration and operation of the decoder 21 are the same as those of the decoder 81 shown in FIG. 10 described above, and description thereof is omitted here.

On the other hand, the watermark detector 22 shown in FIG. 24 has a different configuration from the watermark detector 82 shown in FIG. 12, and as shown in FIG. An encoding template 31; a simple code decoder 32; an evaluation value calculator 33; a Warmark pattern matching controller 34; a Warmark mark storage memory 35; An evaluation value comparator 36 and a code converter 37 are provided.

The encoded bit string is input to the watermark detector 22. The input coded bit string is simply decoded by the simple code decoder 32 in order to find the position of the code for detecting the accompanying information. The simplified code decoder 32 decodes the input code with reference to, for example, an encoding table 31 for coding the DC component (DC component) of the DCT. In the coding table unit 31, fixed-length code and variable-length code tables are indispensable. The position of the code in this case, which is decoded by the simple code decoder 32, refers to, for example, the position on the coded bit string or the spatial or frequency position when decoded. Various positions may be searched, including positions other than. The position of the code to be searched depends on the type of the code to be detected when detecting the accompanying information on the coded bit string, which is the same as described in the watermark adder 1. It is. In some cases, decoding may be performed on the entire or a part of the coded bit sequence to form a whole or a part of the reproduced image.

The code position information b indicating the position of the target code when detecting the accompanying information on the encoded bit string is passed to the evaluation value calculator 33 together with the input encoded bit string. In order to calculate the evaluation value from each target code on the encoded bit string, the evaluation value calculator 33 passes the code position p of the evaluation target to the watermark pattern matching controller 34. This code position P is the same as the one described in the Original or two-dimensional position representation is often used.

The key mark matching controller 34 refers to the key mark of the key mark pattern recorded in the key mark pattern holding memory 35 based on the code position p. The obtained symbol S is passed to the evaluation value calculator 33. The evaluation value calculator 33 calculates an evaluation value using the passed symbol S.

The calculated evaluation value is subjected to threshold processing in the evaluation value comparator 36, and the accompanying information f is output. Also, the input image data is output as it is. In order to obtain the same effect as that described in the section on preventing unauthorized copying, a code converter 37 shown by a dotted line is provided, and the input coded bit string is processed or processed and output. Sometimes. In the code converter 37, processing or processing such as not outputting the coded bit string or rearranging the coded bit string randomly is performed.

The configuration of this image signal decoding apparatus is as shown in FIGS. 24 and 25 above, and the point that a watermark detector 22 is provided in front of the decoder 21 and that the watermark detection is performed. It is characterized in that a coding table unit 31 is provided inside the unit 22 and the objects to be coded bit string are corrected to a fixed length code part and a variable length code part.

Fig. 26 shows a series of processes performed by the war mark detector 22. Here, in the encoding table referred to in order to correct the target code, in the conventional example, only the fixed length code is essentially required, but in this embodiment, the variable length code is used. Is also essentially required. In the figure, an example is shown in which an encoding table of DC component difference values is used.

However, the target code for detecting the accompanying information on the encoded bit string In order to read the encoded bit string to the position of the symbol, it is necessary to prepare an encoding table for all or at least the main part. The fixed-length code and variable-length code tables described above generally correspond to the coding table of the main part in many cases, and should be prepared in both the earlier application and this embodiment. There are many.

In the above example, an example is described in which an encoding table of DC component difference values is used, but any other encoding table may be used. An example of this is a coding table of a motion vector that is coded in macroblock units.

When comparing the evaluation value sum with the threshold th, any comparison method other than the comparison method described above may be used. For example, utilizing the fact that the bias component B of the evaluation value is constant, the comparison may be performed with a bias reliability coefficient c ((^ l) indicating how much the bias component is considered to be held. Alternatively, the auxiliary information may be detected by calculating a standard evaluation value using some method, and comparing the calculated value with the actual evaluation value.

Any symbol other than plus and minus may be used as the symbol of the war evening mark. Also, any two or more types of symbols may be used instead of the two types. For example, three types of symbols, plus, zero, and minus, are prepared, and a code whose symbol is zero when matched with a short-term mark pattern does not affect the evaluation value sum. (The value represented by the code is not added to or subtracted from the evaluation value sum.) Each symbol may have any meaning.

The range in which the watermark pattern is added on the coded bit string is arbitrary. In addition, matching with the added warrior mark pattern is checked. As long as the range is determined, the range for obtaining the evaluation value at the time of detection may be arbitrary. In addition, it is acceptable to add or detect a short-time mark pattern using a wider range over time or space. For example, in a moving image sequence, a temporal reference may be used, and not only the temporal position of the current frame but also past and future frames may be used.

For example, in the case of a still image having a very large image size, one image is divided into a plurality of image regions in a certain unit and handled, and the spatial standard is used to code the current target image region. On the other hand, for example, the codes of the image areas located before and after in the scanning order may be used.

As described above, the image signal decoding apparatus can add the accompanying information without performing the encoding again by using the watermark detector 21 and code the variable-length code portion and the fixed-length code portion together. Accompanying information can be detected without changing the length.

Next, a modified example of the image signal processing device will be described with reference to FIGS. 27 and 28. FIG. In this modified example, a word mark adder 4 is used instead of the word mark adder 2 shown in FIG.

In an image signal processing device using the watermark adder 2, an example in which the difference value of the DC component is encoded, and the code length does not change within the range of the fixed-length code portion and the variable-length code portion Was mentioned. However, it is also possible that the code length does not change only within the range of the variable length code portion without using the fixed length code portion. This modification will be described in this modification.

The 8x8 DCT coefficients Coeff [0] [0] to Coeff [7] [7] of one process are quantized at predetermined quantization steps, and the quantization levels QF [0] [0] to QF [7 ] [7]. In the MPEG system, this DC component (DC component) Although the fixed-length code is used as described above, the AC component (AC component) is generally a variable-length code. The AC component (AC component) is coded by zigzag scanning in the block, after being arranged as scan [0] as a DC component, followed by scan [l] to scan [63].

Consider the case where a war mark is added to this AC component. In this case, the evening mark pattern can be assigned, for example, in the coding scanning order.

A specific example of the coding operation order will be described with reference to FIG. When a macroblock is coded, its AC component is first coded for the AC component scan [l] to scan [63] for the block at the upper left 0 of the luminance Y. Subsequently, the AC components scan [l] to scan [63] are encoded for one upper right block of the luminance Y, and a few blocks are encoded in the same manner. To add a warm-up mark only to the luminance signal, the lower right three blocks are coded, and then the next (generally right) macroblock is moved to the upper left of luminance Y. Encoded for zero blocks. The following is the same.

When adding not only to the luminance signal but also to the color difference signal, two scanning orders can be considered. One is a scanning order method in which the blocks of the lower right 3 of the luminance Y are encoded in the above example, and then the blocks of the color difference blocks 4 and 5 are encoded. The other is a scanning order method in which the chrominance signals Cb and Cr are encoded after encoding the luminance Y in the screen.

The above examples are merely examples, and any other scanning order may be used. For example, in MPEG2, there are two types of scanning order of DCT coefficients in a block: the above-mentioned ordinary zigzag scanning and the above-mentioned ordinary scanning which is adapted by in-line scanning of an image. Rice cake The scanning order may include not only the AC component but also the DC component. It is not necessary to use all AC components. For example, some scan [x] (0 <x 63) to be used in each block may be determined in advance, or some characteristic coefficients may be selected.

When encoding each DCT coefficient (mainly the AC component), the number (run) of coefficient 0 preceding (following) in the scanning order and its coefficient value (level) are grouped by a variable length code. Encoded. Tables 6 to 11 show the encoding tables for DCT coefficients in MPEG2.

[Table 6]

[Table 7] VLC (RUN) (Level) of DCT coefficient of first and second run resole

0010 0110S 0 5

0010 0001S 0 6

0010 0101S 1 3

0010 OIOOS 3 2

0010 OlllS 10 1

0010 OOllS 11 1

0010 OOIOS 12 1

0010 oooos 13 1

0010 0010 10S 0 7

0000 0011 oos 1 4

0000 0010 US 2 3

0000 0011 US 4 2

0000 0010 OIS 5 2

0000 0011 10S 14 1

0000 0011 OIS 15 1

0000 0010 oos 16 1 [Table 8] VLC of first DCT coefficient and DCT coefficient of next run level (RUN) (Lve l)

0000 0001 1101S 0 8

0000 0001 1000S 0 9

0000 0001 0011S 0 10

0000 0001 oooos 0 11

0000 0001 ions 1 5

0000 0001 OIOOS 2 4

0000 0001 1100S 3 3

0000 0001 0010S 4 3

0000 0001 1110S 6 2

0000 0001 0101S 7 2

0000 0001 0001S 8 2

0000 0001 m is 17 1

0000 0001 1010S 18 1

0000 0001 1001S 19 1

0000 0001 Oll lS 20 1

0000 0001 0110S 21 1

0000 0000 1101 OS 0 12

0000 0000 1100 IS 0 13

0000 0000 1100 OS 0 14

0000 0000 1011 IS 1 15 [Table 9] VLC (RUN) (Level) of first DCT coefficient and next run level DCT coefficient

0000 0000 1011 OS 1 6

0000 0000 1010 IS 1 7

0000 0000 1010 OS 2 5

0000 0000 1001 IS 3 4

0000 0000 1001 OS 5 3

0000 0000 1000 IS 9 2

0000 0000 1000 OS 10 2

0000 0000 1111 IS 22 1

0000 0000 1111 OS 23 1

0000 0000 1110 IS 24 1

0000 0000 1110 OS 25 1

0000 0000 1101 IS 26 1

0000 0000 0111 US 0 16

0000 0000 0111 10S 0 17

0000 0000 0111 OIS 0 18

0000 0000 0111 oos 0 19

0000 0000 0110 US 0 20

0000 0000 0110 10S 0 21

0000 0000 0110 OIS 0 22

0000 0000 0110 oos 0 23 [Table 10] VLC (RUN) (Leve l) of first DCT coefficient and next run level DCT coefficient

0000 0000 0101 US 0 24

0000 0000 0101 10S 0 25

0000 0000 0101 OIS 0 26

0000 0000 0101 oos 0 27

0000 0000 0100 U S 0 28

0000 0000 0100 10S 0 29

0000 0000 0100 OIS 0 30

0000 0000 0100 oos 0 31

0000 0000 0011 ooos 0 32

0000 0000 0010 111S 0 33

0000 0000 0010 110S 0 34

0000 0000 0010 101S 0 35

0000 0000 0010 100S 0 36

0000 0000 0010 011S 0 37

0000 0000 0010 OIOS 0 38

0000 0000 0010 OOIS 0 39

0000 0000 0010 ooos 0 40

0000 0000 0011 111S 1 8

0000 0000 0011 110S 1 9

0000 0000 0011 101S 1 10 [Table 11] First and second run-level DCT coefficients VLC (RUN) (Leve l)

0000 0000 0011 100S 1 11

0000 0000 0011 011S 1 12

0000 0000 0011 010S 1 13

0000 0000 0011 001 S 1 14

0000 0000 0001 OOl lS 1 15

0000 0000 0001 OOIOS 1 16

0000 0000 0001 0001S 1 17

0000 0000 0001 oooos 1 18

0000 0000 0001 OIOOS 6 3

0000 0000 0001 1010S 11 z

0000 0000 0001 1001S 12 2

0000 0000 0001 1000S 13 2

0000 0000 0001 0111S 14 2

0000 0000 0001 0110S 15 2

0000 0000 0001 0101S 16

0000 0000 0001 mis 27

0000 0000 0001 1110S 28

0000 0000 0001 1101S 29

0000 0000 0001 1100S 30

0000 0000 0001 ions 31 Basically, encoding is performed using the encoding tables in Tables 6 to 11 described above. However, when performing intra-image encoding (INTRA), another encoding table may be used. If the combination of run and level cannot be represented by the tables prepared in Table 6 to Table 11, the run and level are each determined using the run and level coding table following the escape code. And are fixed-length coded.

In the following, description will be made using the encoding tables in Tables 6 to 11 above.

As an example of encoding the AC component, consider the following case. On the right side, examples of Waryu mark patterns added to each coefficient are also listed. The watermark pattern was not added when the coefficient was 0, and the added amount of il was used.

• AC component 'Water Mark pattern

scan [l] = 12 +, 13

scan [2] = 6-, 5

scan [3] = 0 +

scan [4] =-7 +, -6 In this example, the codes when no short mark pattern is added are as follows.

[Run] [Level] [Sign]

0 12 14bits: 0000 0000 1101 00

0 6 9bits: 0010 0001 0

1 -7 14bits: 0000 0000 1010 11 Also, when the war evening mark pattern is added, the result is as follows.

[Run] [Level] [Sign]

0 13 14bits: 0000 0000 1100 10

0 5 9bits: 0010 0110 0

1 -6 14bits: 0000 0000 1011 01 As described above, it is possible to prevent the code length from changing only within the range of the variable length code portion.

When such a process is actually performed, it is general to check in advance what the code length will be after adding the watermark pattern, and to add the code length when the code length does not change.

In order to increase the degree of freedom at the time of addition, the code may be modified so that the code length does not change within a wider range of an arbitrary block. For example, consider the following example.

AC component 'Water Mark'

scan [l] = 3 +, 4

scan [2] = 6-, 5

scan [3] = 0 +, 1

scan [4] d -8 +, -7 In this example, the code when no war is added is as follows. From scan [l] to scan [4], 31bits.

[Run] [Level] [Sign]

0 3 6bits: 0010 10

0 6 9bits: 0010 0001 0 1 -8 16bits: 0000 0000 0011 1111 When the mark is added, it is as follows. This is also 31 bits from scan [1] to scan [4], and the code length is unchanged compared to before the addition.

[Run] [Level] [Sign]

0 4 8bits: 0000 1100

0 5 9bits: 0010 0110 0

0 1 3bits: 110

0 -7 l lbits: 0000 0010 101 When such a process is actually performed, how much the code length for the coefficient processed so far has changed by adding a warrior mark pattern It is better to remember. It is common to determine whether the next code can be corrected by referring to the stored code length fluctuation.

Of course, by changing not only the level but also the run as in the above example, the coefficient that was 0 before the addition of the war evening mark may be changed to a value other than 0, and conversely, the coefficient may be changed to 0 before the addition. Coefficients other than the above may be set to 0.

For any coefficient, what should originally be at level V when a watermark is added may be at level V + m (m is arbitrary) so as not to change the code length in the block.

Of course, m is preferably a small positive value such as +1 or +2 when the warrior mark pattern is +.

In the above example, the range where the code length does not change was within an arbitrary block. However, this range may be anything. However, in general, in the MPEG system, it is better to perform the above processing in the range of macroblocks, slices, pictures, G0P, and so on.

As shown in FIG. 27, the watermark signal adder 4 included in this image signal processing apparatus is configured by using only the configuration of the encoding table unit 17 as shown in FIGS. 15 and 16 above. Different from the configuration of 1. The encoding table unit 17 is an encoding table that is essential for the simple code decoder 11, and is characterized in that the target for correcting the encoding bit string is only the variable-length code part.

That is, in the encoding table unit 17, a table of variable length codes is indispensable. In the figure, an example using a coding table of DCT coefficients (generally a variable length code including an AC component) is described.

FIG. 28 shows a series of processes performed by the warp mark adder 4. The coding table referred to in step S8C to correct the target code is different from the coding table referred to by the watermark adder 2 shown in FIG.

As described above, in the image signal processing apparatus according to the modified example, with the watermark adder 4, the additional information can be added without re-encoding, and the additional information can be added without changing the code length in the variable-length code portion. Information can be added.

Next, a modified example of the image signal decoding device will be described with reference to FIGS. An image signal clothing device according to this modification is a watermark detector 24 as shown in FIG. 29, and a DCT coefficient encoding table device 39 is characteristic.

That is, the encoder table unit 31 of the DCT coefficient of the watermark detector 22 shown in FIG. 25 described above stores the variable length code portion and the fixed length code portion. While it was possible to detect the accompanying information whose code length did not change within the combined range, it was possible to detect the accompanying information whose code length did not change only in the variable-length code part.

The encoding table unit 39 must have a table of variable length codes. In the figure, an example is shown in which an encoding table of DCT coefficients (generally a variable length code including an AC component) is used.

FIG. 30 shows a series of processes performed by the watermark detector 24. A variable length code is required as an encoding table to be referred to in step S12C to correct the target code.

As described above, in the modified example of the image signal decoding device, the watermark information detector 24 can add the additional information without re-encoding, and can change the additional information without changing the code length in the variable-length code portion. Can be detected.

However, in order to read the coded bit string up to the position of the target code for detecting the accompanying information on the coded bit string, all or at least the main part of the coding table must be read. It is necessary to prepare. In the above example, an example is described in which a coding table of DCT coefficients (including AC components, generally a variable length code) is used, but any other coding table may be used. An example of this is a coding table of a motion vector that is coded in macroblock units.

When comparing the evaluation value sum with the threshold th, any comparison method other than the comparison method described above may be used. For example, by utilizing the fact that the bias component B of the evaluation value is constant, the comparison may be performed together with the bias reliability coefficient c (O ^ c) indicating how much the bias component is considered to be held. In addition, a standard evaluation value is calculated using some method, and the value is compared with an actual evaluation value to detect accompanying information. Is also good.

Further, any symbol other than plus and minus may be used as the symbol of the war evening mark. It is also acceptable to use any three or more symbols instead of two. For example, three types of symbols, plus, zero, and minus, are prepared, and a code whose symbol is zero when matching with a warrior mark pattern does not affect the evaluation value sum ( Any meaning may be given to each symbol, such as adding or subtracting the value represented by the code to the evaluation value sum. The range in which the watermark pattern is added on the coded bit string is arbitrary. Also, the range for obtaining the evaluation value at the time of detection may be arbitrarily set as long as the matching with the added watermark pattern is taken. Further, the watermark pattern may be added or detected using a wider range over time or space. For example, in a moving image sequence, a temporal reference may be used, and not only the temporal position of the current frame but also past and future frames may be used.

Next, a second embodiment of the signal processing device and method according to the present invention will be described. The second embodiment is an image signal processing apparatus that estimates a rounding process performed by quantization when adding additional information, thereby minimizing a change in a value to be corrected at the time of addition. Hereinafter, description will be made with reference to FIGS.

In the second embodiment, additional information such as a war mark is attached. If the value to be added is quantized, minimize the change in the value to be corrected when adding the accompanying information by devising the addition method so that the rounding process performed in quantization can be used properly. Can be. That is, according to the second embodiment, in the signal processing for embedding the accompanying information in the input signal, the correction necessary for changing the value after quantization with respect to the signal before quantization is performed. By doing so, the above-mentioned additional information is added, and the signal including the additional information is subjected to encoding including quantization.

Here, in the additional information, the minimum value required to change the quantization value in the encoding is modified for the signal before encoding according to the additional information. It is mentioned. In this case, it is preferable to estimate a rounding error at the time of quantization, and to correct at least a part of the signal before encoding according to the estimated rounding error and the accompanying information.

FIG. 31 shows a main part of an image signal processing apparatus according to a second embodiment of the present invention, in particular, an image signal processing apparatus for embedding and encoding a Water Mark as incidental information in an image signal. FIG. 2 is a block diagram showing a schematic configuration.

In FIG. 31, an input signal such as an image signal is sent to a rounding error estimator 8 of a warner mark adding device 1 which is an additional information adding means, and an output from the rounding error estimator 8 corrects a pixel value. It is sent to the additional information adder 9 as a means. The output from the additional information adder 9 of the watermark adder 1 is sent to the encoder 2 and encoded. As the encoder 2, for example, one having a configuration such as the encoder 72 in FIG. 8 described above, which performs DCT encoding and then quantization, can be used. It is not limited to this.

A rounding error estimator 8 in the watermark adder 1 estimates a rounding error at the time of quantization, and sends rounding error information err to an additional information adder 9. The accompanying information adder 9 converts the pixel value and the like of the input signal data such as the image signal in accordance with the accompanying information f and the power mark pattern from the watermark mark holding memory 10. The evening was corrected and sent to encoder 2.

In the following, a description will be given of an example in which a watermark is added to the DC component (DC component) of the DCT.

In the MPEG system, the DC component of the DCT (DC component) is quantized by encoding. In the case where a short mark is added to this DC component with an addition amount of, for example, ± 1, conventionally, processing has been performed such that the DC component is corrected by ± 1. This is equivalent to making the luminance value ± 1 for all the pixels in the 8 × 8 block on the image subjected to DCT. However, since quantization is performed at the time of encoding, only the minimum necessary pixels corresponding to the rounding error method at the time of quantization are corrected, and the DC component is marked with an additional amount of ± 1 for the DC component. The same effect as that added can be obtained.

This example is shown in FIG.

The upper part of Fig. 32 shows an example of an 8x8 block before DCT. The DC component of the DCT is equivalent to the average value of the block. Although the average values of the three blocks shown in Fig. 32 are different, when quantized by rounding the first decimal place to the nearest whole number, the average value of all blocks is 43. Adding +1 to the DC component is equivalent to adding +1 to the average of the block obtained by rounding off the first decimal place of the average. If the average value before quantization becomes 43.5, the DC component should be +1. Therefore, the number of pixels n is calculated by increasing some of the pixels in the proxies by 1 so that the average value becomes 43.5 or more.

In the left block with an average value of 43.000, the average value only needs to be increased by 0.5, so 32 pixels out of 64 pixels of 8x8 should be incremented by one. In the central block with an average value of 43.436, 5 pixels should be added to +1. In the block with an average value of 43.756 on the right, 48 pixels should be +1. In this way, it is possible to add +1 to the DC component without adding +1 to all 64 pixels in the 8x8 block. It should be noted that which pixel in the block is incremented by +1 may be determined by any reference.

By performing such a contrivance, the number of pixels to be corrected can be reduced, and for example, an effect such that the degree of deterioration of image quality due to addition can be reduced can be obtained.

In the above example, a case was described in which every pixel is incremented by 1. However, any additional amount may be set, such as providing a pixel having another additional amount such as +2. For example, in the central block of FIG. 32, instead of incrementing 5 pixels, 1 pixel may be incremented by 1 and 1 pixel may be incremented by 4. Alternatively, two pixels may be set to −2, and three pixels to +3. The pixel to be corrected in the block may be set at random, or the position of a pixel containing many high-frequency components that are difficult to detect with human eyes may be detected and added intensively there. .

Of course, other methods may be used. The range for estimating rounding during quantization usually depends on the range of the quantization target. However, other ranges may be estimated as units. In the above example, the estimation is made in the range of 8x8 procks, but as an example, a range of this integral multiple (such as 16x16) may be used as a unit. The rounding method at the time of quantization may be any method other than rounding. In the above description, the DC component (DC component) of the DCT Although the example of adding a tag has been described, it is not necessary to be limited to this. — — Any value can be used as long as the value to be added to the evening mark is quantized.

FIG. 33 and FIG. 34 show a series of processes when the additional information is added in the second embodiment of the signal processing device and the signal processing method. The configuration of the second embodiment differs from the conventional technology in that the value of the watermark addition target is estimated by the rounding process at the time of quantization, and the rounding error information err which is the estimation result is obtained. The feature is that is reflected when a watermark is added.

FIG. 31 above, which shows an example of the configuration according to the present embodiment, mainly shows the configuration of the watermark mark adder 1, and what happens to the value to be added with the watermark mark due to rounding processing during quantization. In order to check the rounding error, the rounding error estimator 8 estimates rounding error information err at the time of quantization. When the additional information f is on, the additional information is added by the additional information adder 9 using the Warm evening mark pattern recorded in the Warm evening mark pattern holding memory 10 and the rounding error information err. . When the accompanying information f is off, the accompanying information adder 9 outputs the input image data as it is.

Next, a series of processes performed by the watermark adder 1 will be described with reference to FIGS. First, in steps S 101 of FIG. 33, the additional levels a and b of the war mark are set. In the next step S102, the rounding error information err in the block is estimated on the image to which the additional information is to be added, and the number n of pixels for correcting the luminance value is obtained. After that, the process from 1 to 2 (① to ②) is repeated while changing the pixel position in the block. From 1 in the circle 2 (処理 to ②), in step S103 of FIG. 34, in the image to which the additional information is to be added, the target block (DCT block) including pixel X is marked with a watermark. Performs pattern matching.

In step S 104, it is determined whether the symbol of the power mark of the block is plus or minus, and if the symbol of the war mark is positive, the process proceeds to step S 105, and Add a to the pixel value X of the pixel X in the block. When the symbol of the war mark of the block is negative, the process proceeds to step S106, and b is subtracted from the pixel value X of the pixel X in the block.

These processes are repeated for all blocks in the target area. Next, in step S107 of FIG. 33, it is determined whether or not the processing has been performed for all the blocks in the target area. If NO, the process returns to step S102, and if YES, the process ends. are doing. Here, the processing for setting the values of a and b in step S101 may be arranged immediately before the pattern matching step S103 in FIG.

Any symbol other than plus and minus may be used as the war symbol. In addition, any two or more symbols may be used instead of two. For example, three kinds of symbols, plus, zero, and minus, are prepared, and the sign whose symbol is zero when matching with the warrior mark pattern does not affect the evaluation value sum. (The value represented by the sign is not added to or subtracted from the evaluation value sum.) Each symbol may have any meaning.

Adds a watermark pattern to an image or encoded bit string. The range may be arbitrary. In addition, the range for obtaining the evaluation value at the time of detection may be arbitrarily set as long as the matching with the added watermark pattern is achieved.

Further, the watermark pattern may be added or detected using a wider range over time or space. For example, in a moving image sequence, a temporal reference is used, and not only the temporal position of the current frame but also past and future frames may be used. For example, in the case of a still image having a very large image size, the image of the profile is divided into a plurality of image regions in a certain unit and handled. On the other hand, for example, the codes of the image areas located before and after in the scanning order may be used.

Next, a third embodiment of the signal processing device and method according to the present invention will be described. In the above-described second embodiment, additional information such as a watermark is added by using a rounding process performed by quantization. By applying this technique, the values within a certain range, such as a DCT block, can be modified appropriately to make the representative values in that range regular.

That is, in the third embodiment, when embedding accompanying information in an input signal, a value within a block range consisting of a plurality of data of the input signal is corrected so that a representative value within the block range has regularity. Thus, the additional information is added.

At the time of adding the accompanying information, modification of the value required to make the representative value within the range of a block consisting of a plurality of data of the input signal into a value in accordance with a predetermined rule is performed in the block. At least some of the data. In addition, correction of the value required to make the decimal part of the average value within the range of the block composed of a plurality of data of the input signal constant for all the blocks in the input signal is performed in the block. At least a part of the data is added to add the accompanying information. Furthermore, it is possible to detect the incidental information by judging the regularity of the representative values in the block range from the signal to which the incidental information is added by giving the regularity to the representative value in the block range. Be mentioned

The above-mentioned regularity is, for example, a regularity in which a decimal part of the average value of pixel data in the DCT block is fixed. By utilizing the regularity of the representative values, it is possible to provide a technique for adding and detecting new accompanying information.

In the third embodiment, an image signal is assumed as an input signal, and a watermark is added to the DC component of the DCT, for example, for each 8 × 8 pixel block, which is a DCT block for DCT coding. explain. However, the case where quantization is not performed will be mainly described.

When the method of the second embodiment is applied, if an appropriate pixel in a certain block is corrected, the decimal component of the average value can be set to an arbitrary value. For example, in the example of FIG. 32 described above, it may be considered that the number of pixels to be corrected is determined so that the decimal component of the average value in the block is set to 0.5. The arbitrarily configurable fractional value of the average value alone can have the meaning of the new accompanying information as follows.

For example, by modifying the above pixel, all blocks in the image

Consider the case where the decimal component of the average value of (total be) is 0.5. In the normal case where the above pixel correction is not performed, this fractional component varies. And the probability P no (0 <= Pno <= 1) that this fractional component becomes 0.5 in any block is small. Therefore, considering the probability P−0.5 that the decimal component of the average value is 0.5 in all blocks, P−0.5 should be the power of Pno to be.

P-one 0,5 = Pno-one be

For example, Pno: 1/2, be = 100. 2 to the 10th power can be approximated as 1000 2 10 "3.

P— 0.5 = 2 -100) = (2 (-10) factory 10 ≠ (10 (-3) factory 10 = 1 (Γ (-30)

The probability of one thirtieth power of 30 is an astronomically low probability, and this is not usually the case. Therefore, by modifying the pixel in block units and setting the fractional component of the average value of all blocks in the image to a certain value, it is possible to add additional information. At the time of detection of the accompanying information, if the decimal component of the average value of all the programs becomes a certain value, it is determined that the accompanying information is added.

When correcting the value of a pixel in the block, it is not necessary to increase the value of any pixel by +1. Any additional amount may be set, such as providing a pixel having another additional amount such as +2. For example, in the central block in FIG. 32, in order to set the value of the decimal component to 0.5, instead of incrementing 5 pixels, 1 pixel may be incremented by 1 and 1 pixel may be incremented by 4. Alternatively, two pixels may be set to −2, and three pixels to +3.

The pixels to be corrected in the block may be set at random, or the positions of pixels that contain many high-frequency components that are difficult to detect with human eyes may be detected and added intensively there. good. Of course, other methods may be used. A certain range is set for the decimal component at the time of detection. May be. For example, if the fractional component at the time of addition is 0.4, the additional information is added when the fractional component err-now at the time of detection is 0.3 <err—now <0.5 in all blocks in the image. It may be considered that it is done.

A plurality of different decimal component values may have the meaning of a plurality of accompanying information corresponding thereto. For example, when the value of the decimal component is 0.1, it is considered that the information A is added, and when the value of the decimal component is 0.8, the information B is considered to be added. You may do it. The unit for adding the accompanying information may be any area or range other than the above.

In the above example, the decimal component is calculated within the 8x8 block and the whole image is determined.However, the decimal component is calculated within the 2x5 block and the determination is performed using arbitrary 36 blocks of the image. You may do it. The representative value of each area may be any value. In the above example, the case where the average value in the block is the representative value has been described.However, any value other than the average value, such as the maximum value, the minimum value, the intermediate value, the value of a specific position in the block, etc. Is also good. In the above example, the additional information is detected and added to the pixels in the image, but other values may be used. Further, a value which originally appears only as an integer value may be reduced to a decimal number, and the additional information may be added and detected using the decimal value. For example, in a coded bit string generated by the MPEG system, the motion vector obtained with one-pixel accuracy may be halved, and the value may be added and the accompanying information may be detected.

A watermark pattern may be prepared and used when adding and detecting accompanying information. For example, a one-night mark pattern having two symbols, 3 and 4, is prepared, and when added, the decimal component is set to 0.3 when the symbol is 3, and the decimal component is set to 0.4 when the symbol is 4. . At detection In this case, when a decimal component corresponding to the pattern is detected, it may be considered that additional information is added. Of course, any other warrior mark pattern may be used.

In the above description, the case where the value to which the watermark is added is not quantized is mainly described, but a method of using the accompanying information in consideration of the quantization may be used. For example, if the additional information is added by setting the decimal component of the average value of the 8x8 block to 0.5 as described above, and the image is encoded by the MPEG method, the average value will be an integer due to the quantization of the DC component of the DCT. Therefore, the decimal component on the decoded image becomes 0, and the additional information added cannot be detected.

Here's an example that makes good use of this. When the added image itself is detected as the first route, the accompanying information is detected. As a second route, if the added image is encoded into an encoded bit sequence and then returned to the image by decoding, the accompanying information is not detected. That is, the accompanying information can be used so that it can be determined at the time of detection whether or not the image to be detected has been encoded once. A specific example of the first path is, for example, dubbing using a television signal, and a specific example of the second path is, for example, a television broadcast using an encoded bit sequence encoded by the MPEG system. Of course, the method of using the accompanying information may be any other method.

An example of the configuration according to the third embodiment is shown in FIGS. 35A and 35B.

FIGS. 36 and 37 show a series of processes when adding or detecting accompanying information in the third embodiment.

The difference between the configuration of the third embodiment and the conventional technology is that The point is that the process for adding or detecting a report is fundamentally different, and that it is not necessary to use a power mark. FIG. 35A shows the configuration of the watermark adder 1.ォ In order to check what happens to the value of the mark to be added due to the rounding process at the time of quantization, the rounding error estimator 8 estimates rounding error information e rr at the time of quantization (where err is the same as a decimal component). When the accompanying information f is on, the accompanying information is added by the accompanying information adder 9 using the rounding error information err. When the additional information f is off, the additional information adder 9 outputs the input image data as it is.

Fig. 36 shows a series of processes performed by the Warmer Mark Adder 1. That is, in the first step S121 shown in FIG. 36A, on the image to which the additional information is to be added, the rounding error information err in the block is estimated, and the number n of pixels for correcting the luminance value is determined. Ask. After that, the process from 1 to 2 (① to の) is repeated while changing the pixel position in the block. In the processing from 1 to 2 (① to の) in the circle, the value X of the pixel X in the block is changed in step S122 of FIG. 36B. These processes are repeated for all blocks in the target area.

FIG. 35B shows the configuration of the watermark detector 22. For the input image data, an evaluation value is calculated by the evaluation value calculator 33. The calculated evaluation value is subjected to threshold processing in the evaluation value comparator 36, and the accompanying information f is output. Also, the input image data is output as it is. It is to be noted that the image converter 37 shown by the dotted line is placed, and the input image data may be processed or processed and output. This is the same as in the conventional example. FIG. 37 shows a series of processing performed by the war mark detector 22. First, in step S131, the evaluation value sum is initialized and the threshold value th is set, and the decimal component reference value err-normal is set. This err-normal is the fractional component of the average value in the block, set by changing the pixel when adding. In the next step S132, the fractional component err in the block is estimated on the image from which the accompanying information is to be detected.

In the next step S133, when this err is equal to err-normal, 1 is added to the evaluation value sum. This process is repeated for all the blocks in the target area (step S134). Then, in step S135, the evaluation value sum is compared with the threshold th, and if sum> th, the additional information f is set to on assuming that the additional information is added. Otherwise, turn off the accompanying information f.

In this example, the evaluation value sum is used for the determination as the number of counts where err is equal to err-normal, but it is also possible to use the evaluation value to determine the accompanying information by other methods.

It should be noted that the present invention is not limited to the above-described embodiments. For example, it is needless to say that the present invention is not limited to DCT coding, and various transform coding such as wavelet coding can be adopted. INDUSTRIAL APPLICABILITY According to the present invention, additional information can be added or detected without re-encoding or decoding, and a variable-length code portion and a fixed-length code portion can be added or detected within a range. Fixed length, just like the sign part These processes can be performed without being limited to the range of the code portion.

Further, according to the present invention, in signal processing for embedding incidental information in an input signal, the signal before quantization is subjected to a necessary modification to change the value after quantization, so that Since information is added and the signal obtained by adding the accompanying information is subjected to encoding including quantization, the addition method is devised so that the rounding process performed by quantization can be used effectively. By doing so, the change of the value to be corrected at the time of addition can be minimized.

Further, according to the present invention, when embedding accompanying information in an input signal, a value within a block range composed of a plurality of data of the input signal is corrected to give a regularity to a representative value within a block range. As a result, the additional information described above is added, so that by appropriately modifying a value in a certain range, the representative value in the range can be given regularity. By utilizing the regularity of this representative value, it is possible to construct a method for adding and detecting new accompanying information.

Claims

The scope of the claims

1. A signal processing device for embedding incidental information in an input signal, wherein the input signal is decoded, and additional position detecting means for detecting a position to add the incidental information within a range including a variable length portion and a fixed length portion; And additional information adding means for adding the additional information to the position detected by the additional position detecting means.

A signal processing device characterized by the above-mentioned.

2. The input signal is an image signal, and the additional position detecting means is within a range including a variable length portion and a fixed length portion of a DC component of an orthogonal transform coefficient when the orthogonal transform process is performed on the input image signal. To detect the above additional position

The signal processing device according to claim 1, wherein:

3. In a signal processing method for embedding incidental information in an input signal, an additional position detecting step of decoding the input signal and detecting a position to add the incidental information within a range including a variable length portion and a fixed length portion; An additional information adding step of adding the additional information to the position detected in the additional position detecting step.

A signal processing method characterized by the above-mentioned.

4. In a signal processing device for embedding incidental information in an input signal, additional position detecting means for decoding the input signal and detecting a position to add the incidental information within a range of a variable length part;

An additional information adding means for receiving the information on the position detected by the additional position detecting means and the input signal and adding the additional information to the position is provided. A signal processing device characterized by the above-mentioned.

5. The input signal is an image signal, and the additional position detecting means is an orthogonal transform coefficient A when the input signal is subjected to orthogonal transform processing. Detecting the additional position within the variable length part of the component

5. The signal processing device according to claim 4, wherein:

6. In a signal processing method for embedding incidental information in an input signal, an additional position detecting step of decoding the input signal and detecting a position where the additional information is added within a range of a variable length portion;

An additional information adding step of receiving the information on the position detected in the additional position detecting step and the input signal and adding the additional information to the position.

A signal processing method characterized by the above-mentioned.

7. In a signal decoding device that decodes an input signal in which additional information is embedded,

An additional position detecting means for decoding the input signal and detecting a position where the additional information is added within a range including the variable length portion and the fixed length portion;

A signal decoding device, comprising: additional information extracting means for extracting the additional information based on position information on the position detected by the additional position detecting means.

8. The input signal is an image signal, and the additional position detecting means sets the input signal within a range including the variable length portion and the fixed length portion of the DC component of the orthogonal transform coefficient when the orthogonal transform process is performed on the input signal. Detecting the above additional position

8. The signal decoding device according to claim 7, wherein:

9. In a signal decoding method for decoding an input signal in which ancillary information is embedded,

An additional position detection step of decoding the input signal and detecting a position where the additional information is added within a range including the variable length portion and the fixed length portion;

A signal extracting method for extracting the accompanying information based on position information on the position detected in the additional position detecting step.

10. In a signal decoding device that decodes an input signal in which additional information is embedded,

Additional position detecting means for decoding the input signal and detecting a position where the accompanying information is added within a range of a variable length portion;

An auxiliary information extracting means for receiving the position information relating to the position detected by the additional position detecting means and the input signal and extracting the auxiliary information;

A signal decoding device characterized by the above-mentioned.

1 1. The input signal is an image signal, and the additional position detection means performs the additional position detection within a variable length portion of the AC component of the orthogonal transform coefficient when the orthogonal signal is subjected to the orthogonal transform processing. Detecting

10. The signal decoding device according to claim 10, wherein:

1 2. In a signal decoding method for decoding an input signal in which additional information is embedded,

An additional position detecting step of decoding the input signal and detecting a position where the additional information is added within a range of the variable length portion;

The position information related to the position detected in the additional position detection step and the And an additional information extracting step of receiving the input signal and extracting the above additional information.

A signal decoding method characterized by the above-mentioned.

13 3. In the signal processing device that embeds the accompanying information in the input signal, the signal before quantization is modified as necessary to change the value after quantization, so that the additional information is added. Adding means, and coding means for performing coding including quantization on an output signal from the additional information adding means.

A signal processing device characterized by the above-mentioned.

14. The additional information adding means corrects the minimum value required to change the quantization value in the encoding means with respect to the signal before encoding in accordance with the additional information. Applying

The signal processing device according to claim 13, wherein:

15. The additional information adding means includes: an estimating means for estimating a rounding error at the time of quantization in the encoding means; and a signal before the encoding according to the rounding error estimated by the estimating means and the additional information. Have a corrective means to correct at least part of it

The signal processing device according to claim 13, wherein:

1 6. The above input signal is an image signal,

The encoding means includes orthogonal transformation means for performing orthogonal transformation in units of blocks consisting of a plurality of pixels of an image signal, and quantization means for quantizing orthogonal transformation output data,

The additional information adding means modifies at least the minimum value required to change the quantized value of the DC component of the orthogonal transform output data, according to the additional information, the block before the orthogonal transform. Few in Apply to at least some pixels

The signal processing device according to claim 13, wherein:

1 7. The above input signal is an image signal,

The encoding means has a DCT means for performing discrete cosine transform (DCT) processing in units of a block composed of a plurality of pixels of an image signal, and a quantizing means for quantizing DCT coefficient data,

The additional information adding means includes an estimating means for estimating a rounding error in quantizing a DC component of the DCT coefficient data, and the DCT processing according to the rounding error estimated by the estimating means and the additional information. Have correction means to correct at least some of the pixel values in the previous block

The signal processing device according to claim 13, wherein:

1 8. In the signal processing method that embeds the additional information in the input signal, the additional information is added to the signal before quantization by making necessary corrections to change the value after quantization. An adding step, and an encoding step of performing encoding including quantization on the output signal from the additional information adding step.

A signal processing method characterized by the above-mentioned.

19. The additional information adding step includes modifying the minimum value required to change the quantization value in the encoding step with respect to the signal before encoding according to the additional information. Applying

19. The signal processing method according to claim 18, wherein:

20. The additional information adding step estimates a rounding error at the time of quantization in the encoding step, and corrects at least a part of the signal before encoding according to the estimated rounding error and the additional information. To do 19. The signal processing method according to claim 18, wherein:

2 1. The above input signal is an image signal,

In the encoding step, the orthogonal transform is performed in units of blocks each including a plurality of pixels of the image signal, and the orthogonal transform output data is quantized.

In the additional information adding step, the correction of at least the minimum value required to change the quantized value of the DC component of the orthogonal transform output data is performed according to the additional information. Applied to at least some pixels within

19. The signal processing method according to claim 18, wherein:

2 2. The above input signal is an image signal,

In the encoding step, discrete cosine transform (DCT) processing is performed in units of blocks each including a plurality of pixels of the image signal, and the DCT coefficient data is quantized.

The additional information adding step estimates a rounding error in quantizing the DC component of the DCT coefficient data, and at least a part of the block before the DCT processing according to the estimated rounding error and the additional information. Modifying the pixel value of

19. The signal processing method according to claim 18, wherein:

23. In the signal processing device that embeds the accompanying information in the input signal, the value within the block range consisting of a plurality of data of the input signal is corrected so that the representative value within the block range has regularity. Additional information adding means for adding additional information

A signal processing device comprising:

24. The above-mentioned additional information adding means converts the representative value within the range of a block consisting of a plurality of data of the input signal into a value according to a predetermined rule. Modify the required values to at least some of the data in the block to add the accompanying information

24. The signal processing device according to claim 23, wherein:

25. The additional information adding means outputs a value required for converting a decimal part of an average value within a block consisting of a plurality of data of the input signal into a constant value according to the additional information. 24. The signal processing device according to claim 23, wherein the additional information is added by performing a correction on at least a part of the data in the block.

26. The above-mentioned additional information adding means is necessary to make the decimal part of the average value within the range of the block consisting of a plurality of data of the input signal constant for all the blocks in the input signal. The additional information described above by modifying at least part of the data in the block

24. The signal processing device according to claim 23, wherein:

27. The above-mentioned additional information adding means corrects the value required to make the fractional part of the average value within the range of the block for orthogonal transformation a constant value according to the above-mentioned additional information in the block. At least part of the data to add the accompanying information

24. The signal processing device according to claim 23, wherein:

2 8. Ancillary information for detecting the ancillary information by determining the regularity of the representative value in the block range from the signal to which the ancillary information is added by giving the regularity to the representative value in the block range Have additional detection means

24. The signal processing device according to claim 23, wherein:

2 9. In a signal processing method for embedding accompanying information in an input signal, Having an additional information adding step of adding the above additional information by modifying the value in the block range consisting of a plurality of data of the input signal to make the representative values in the block range regular.

A signal processing method characterized by the above-mentioned.

30. In the additional information adding step, the modification of the value required to make the representative value within the range of the block composed of a plurality of data of the input signal into a value in accordance with a predetermined rule is performed in the block. Attach the above accompanying information by applying to at least some data

29. The signal processing method according to claim 29, wherein:

31. The additional information adding step is required to make the decimal part of the average value within the range of the block composed of a plurality of data of the input signal a constant value for all the blocks in the input signal. By adding at least some of the values in the block above

29. The signal processing method according to claim 29, wherein:

3 2. Ancillary information that detects the ancillary information by determining the regularity of the representative value in the block range from the signal to which the ancillary information is added by giving the regularity to the representative value in the block range. Have an additional detection step

29. The signal processing method according to claim 29, wherein: