JP4580898B2 - Digital watermark embedding device - Google Patents

Description

  The present invention relates to a digital watermark embedding apparatus effective for preventing unauthorized duplication of a digital moving image signal provided via a recording medium, for example.

  With the widespread use of apparatuses for recording and reproducing digital image data such as digital VTRs or DVDs (Digital Versatile Discs), a large number of digital moving images that can be reproduced by these apparatuses have been provided. In addition, various digital moving images are distributed through digital television broadcasting via the Internet, broadcasting satellites, communication satellites, etc., and users can use high-quality digital moving images.

  A digital moving image can be easily made a high-quality copy at the digital signal level, and there is a possibility that the copy is unlimited without any copy inhibition or copy control. Therefore, in order to prevent illegal copying (copying) of digital moving images or to control the number of generations of copying by authorized users, information for copying control is added to the digital moving images, and this additional information is used. A method of preventing illegal duplication and limiting duplication is considered.

  As a technique for superimposing other additional information on a digital moving image in this way, digital watermarking is known. Digital watermarks are used for digital data such as audio, music, video, still images, etc., content copyright owner and user identification information, copyright owner rights information, content usage conditions, and usage By embedding secret information necessary at times, or information such as the above-mentioned duplication control information (these are called watermark information) so that it is not easy to perceive, and later detecting watermark information from the content as necessary It is a technology for copyright protection including use control and copy control, and promotion of secondary use.

As one method of digital watermarking, a method using spread spectrum technology is known. In this method, watermark information is embedded in a digital moving image by the following procedure.
[Step E1] The image signal is multiplied by a PN (Pseudorandom Noise) sequence to perform spread spectrum.
[Step E2] The image signal after the spread spectrum is subjected to frequency conversion (for example, DCT conversion).
[Step E3] The watermark information is embedded by changing the value of a specific frequency component.
[Step E4] Inverse frequency conversion (for example, IDCT conversion) is performed.
[Step E5] Perform spectrum despreading (multiply the same PN sequence as in step E1).

On the other hand, the watermark information is detected from the digital moving image in which the watermark information is embedded in the following procedure.
[Step D1] The image signal is multiplied by a PN (Pseudorandom Noise) sequence (the same PN sequence as in Step E1) to perform spectrum spreading.
[Step D2] The image signal after the spread spectrum is subjected to frequency conversion (for example, DCT conversion).
[Step D3] Focusing on the value of a specific frequency component, the embedded watermark information is extracted.

  When digital watermarking is applied to prevent unauthorized use, the watermark information is not lost or altered by various operations or intentional attacks that are normally assumed to be applied to digital works. It is necessary to have unique properties (robustness).

  One of the most effective methods for improving robustness is to increase the embedding strength of the digital watermark to prevent the loss of information. However, when the embedding strength of the digital watermark is increased, the digital watermark signal is perceived as noise, which degrades the image quality. On the other hand, if the digital watermark signal itself is visible, there is a risk that the secrecy of the digital watermark method will be weakened and attacks will be made easier.

  In other words, there is a trade-off between lack of perception of digital watermark signals (referred to as transparency) and robustness, and improving robustness without reducing transparency is the most important issue in embedding digital watermarks. It becomes one of.

Therefore, an object of the present invention is to provide a digital watermark embedding device capable of improving robustness while maintaining transparency.
A more specific object is to make it possible to optimally control the embedding strength of a digital watermark signal.
Another object of the present invention is to embed a digital watermark signal in consideration of compression coding of an image signal.
Another object of the present invention is to embed a digital watermark signal in consideration of telecine conversion and inverse telecine conversion.
Another object of the present invention is to realize accurate re-embedding of a digital watermark signal.
Another object of the present invention is to perform re-embedding that can stably detect a re-embedded digital watermark signal.
Another object of the present invention is to embed a digital watermark signal in consideration of frame rate conversion.

  (1) Embedding a digital watermark signal in an input image signal to generate an output image signal When embedding a digital watermark, the digital watermark signal is extracted from the output image signal and the signal strength of the extracted digital watermark signal is detected. The embedding strength of the digital watermark signal with respect to the input image signal is controlled according to the signal strength.

  By embedding the signal strength of the embedded watermark signal in this way and dynamically controlling the embedded strength of the watermark signal, stable watermark embedding can be performed regardless of the pattern or nature of the image signal. Therefore, it is possible to easily realize embedding with higher robustness that guarantees desired detection performance at the time of digital watermark detection.

  (2) At the time of embedding a digital watermark in which an electronic watermark signal is embedded in an input image signal to generate an output image signal, the degree of image quality deterioration of the output image signal with respect to the input image signal is detected, and the input image is determined according to the degree of image quality deterioration The embedding strength of the digital watermark signal with respect to the signal is controlled.

  In this way, the embedding strength is dynamically controlled by the feedback of the image quality degradation level resulting from the embedding of the digital watermark signal, and embedding with the image quality degradation constant or the maximum value of the image quality degradation level is suppressed to a predetermined value or less. Etc. can be easily controlled. This makes it easy to suppress the image quality deterioration and increase the embedding strength as much as possible, and to perform digital watermark embedding with less image quality deterioration and high robustness.

  (3) When embedding a digital watermark in which an electronic watermark signal is embedded in an input image signal to generate an output image signal, an activity is detected from at least one of the input image signal and the digital watermark signal, and the input image signal is determined according to the activity The embedding strength of the digital watermark signal with respect to is controlled.

  In this way, the embedding strength is adaptively controlled by feedforward control according to the activity that is one of the properties of the image signal and the digital watermark signal, so that the watermark embedding strength can be maximized while suppressing image quality degradation. This makes it possible to embed a more robust digital watermark.

  (4) Arbitrary two of control of embedding strength by the above-described first to third methods, that is, feedback of signal strength of digital watermark signal, feedback of image quality degradation level by embedding of digital watermark signal, and feed forward of activity It is also possible to control the embedding strength of the digital watermark signal with the above combination. Thus, by organically combining the control of embedding strength by a plurality of methods, it becomes possible to realize digital watermark embedding with little degradation in image quality and guaranteeing stable high detection performance.

  (5) A repeat field is detected from an input image signal at the time of embedding an electronic watermark to generate an output image signal by embedding an electronic watermark signal in a field unit with respect to an input image signal generated by telecine conversion by insertion of a repeat field. Based on the detection result, among the input image signals, the same digital watermark signal as that embedded in the field two fields before (the in-phase field of the immediately preceding frame) is embedded in the repeat field.

  In MPEG2 moving image encoding used in DVDs or the like, video signals that have been subjected to telecine conversion are often encoded by inverse telecine conversion. In telecine conversion, a film material having a frame frequency of 24 Hz is periodically repeated to insert a field to convert it into a 60 Hz field signal, which is displayed on a TV or recorded by a recording / playback device such as a VTR.

  In a DVD or the like, inverse telecine conversion is automatically performed on a video signal subjected to telecine conversion in order to increase compression efficiency, compression encoding is performed as a 24 Hz frame image, and telecine conversion is displayed again during reproduction. Inverse telecine conversion is usually realized by automatically detecting a repeat field. However, embedding different digital watermark signals for each field before inverse telecine conversion leads to a decrease in repeat field detection rate and accurate inverse telecine conversion. It will prevent the conversion. As a result, the coding efficiency of compression coding is lowered, which causes image quality degradation.

  Here, as described above, even when embedding a digital watermark, a repeat feed of telecine conversion is detected, and in the repeat field, the same digital watermark signal as that of the in-phase field one frame before is embedded, thereby compressing and encoding. Thus, it is possible to embed a digital watermark signal without degrading the detection performance of inverse telecine conversion.

  (6) Inverse telecine conversion is performed on the input image signal, a digital watermark signal is generated for each image signal after the inverse telecine conversion, and a digital watermark signal is framed with respect to the image signal after the inverse telecine conversion. Embed in units.

  In this way, for example, moving picture coding such as MPEG2 and digital watermark embedding are linked, a digital watermark signal is embedded in a frame subjected to inverse telecine conversion for moving picture coding, and then encoded. Thus, it is possible to embed a digital watermark signal in an image signal while completely eliminating the influence on the inverse telecine conversion by the digital watermark embedding.

  (7) Estimating the geometric deformation received by the input image signal from the input image signal in which the first digital watermark signal is embedded, generating a parameter related to the estimated geometric deformation, and performing geometric deformation according to the parameter A second digital watermark signal is generated, and the second digital watermark signal is re-embedded in the input image signal to generate an output image signal.

  As a form of use of the digital watermark, not only the pre-embedded digital watermark information is detected but also the digital watermark information is rewritten (remarked), and the weakened digital watermark signal is reinforced by rewriting. The former is used for generation management by digital watermarking. For example, when a signal indicating that copying is possible once is written by digital watermark, it is necessary to change the information of the digital watermark signal to “copy not possible” in conjunction with the execution of one copy. In the latter case, the digital watermark is reinforced by overwriting the same information on the digital watermark signal that has been attenuated along with geometrical deformation such as enlargement / reduction of the video signal, and image quality degradation due to encoding / decoding or recording / playback. In some cases, the signal strength is restored to normal.

  In either case of remarking or overwriting, if the image signal itself has undergone some geometrical deformation after the initial digital watermark embedding, normal re-embedding becomes difficult. However, as described above, re-embedding such as remarking or overwriting is difficult. In this case, for example, pre-embedded digital watermark information is detected, and parameters related to geometric deformation such as enlargement / reduction and phase of time or space are detected, and re-embedding is performed according to the detected geometric deformation parameters. By generating a digital watermark signal, re-embedding can be performed more accurately.

  (8) Upon re-embedding such a digital watermark signal, the first digital watermark signal is extracted from the input image signal and the signal strength of the first digital watermark signal is detected, while the second digital watermark signal is detected from the output image signal. The digital watermark signal is extracted to detect the signal strength of the second digital watermark signal, and the second electronic signal corresponding to the input image signal is detected in accordance with the magnitude relationship between the detected signal strengths of the first and second watermark signals. The re-embedding strength of the watermark signal may be controlled.

  When re-embedding such as remarking of the watermark signal, if only specific information is changed, after re-embedding, if the balance of the signal strength of the watermark signal between the changed part and the part not changed is extremely disturbed, In some cases, the detection performance of the digital watermark after remarking is significantly reduced. In particular, in an image signal that has undergone image quality degradation such as geometric deformation or compression coding, the strength of a digital watermark signal embedded in advance also decreases, making it difficult to uniquely determine the re-embedding embedding strength.

  Here, if the re-embedding strength is dynamically controlled based on the balance between the detection strength of the digital watermark signal before re-embedding and the detection strength of the digital watermark signal after re-embedding as described above, It is possible to realize re-embedding that is well balanced with the digital watermark signal, and it is possible to guarantee stable detection performance even for partial re-embedding.

  (9) Further, when re-embedding the digital watermark signal as described above, the first digital watermark signal is detected from the input image signal at a predetermined interval, and predetermined from the time when the first digital watermark signal is detected in the re-embedding. The second digital watermark signal may be re-embedded in the input image signal for more than the time.

  When re-embedding such as remarking of a digital watermark signal, the digital watermark information to be remarked may be determined according to the detection result of the digital watermark information embedded in advance. In this case, for example, if the detection of digital watermark information embedded in advance due to some image deformation is unstable, the re-embedding itself is also unstable, and it is difficult to stably detect the digital watermark signal after re-embedding. It becomes.

  On the other hand, when re-embedding a digital watermark signal as described above, for example, by performing re-embedding processing continuously for a predetermined time or more necessary for digital watermark detection, the digital watermark can be stably generated from the re-embedded image signal. A signal can be detected.

  (10) The compressed image signal is decoded, the frame rate of the decoded image signal is converted and output, and the digital watermark signal is detected from the decoded image signal before the frame rate conversion.

  (11) The compressed image signal is decoded, the frame rate of the decoded image signal is converted and output, and the digital watermark signal is detected from the decoded image signal before the frame rate conversion.

  (12) The compressed image signal in which the first digital watermark signal is embedded is decoded, the frame rate of the decoded image signal is converted and output, and the decoded image before the frame rate conversion is output. The second digital watermark signal is re-embedded in the signal.

  In a system in which an image signal is transmitted or stored in a state of being subjected to digital moving image compression such as MPEG2, detection, embedding, or re-embedding of a digital watermark signal may be performed at the time of decoding the compressed image signal. Many. The compressed image signal may be output after being converted into various frame rates in accordance with the display device. If digital watermark detection, embedding, or re-embedding is performed in accordance with an arbitrary frame rate, the processing cost related to these digital watermarks increases, and it is affected by jitter in the time direction. It may also cause a decrease in detection performance.

  By applying digital watermark detection, embedding, or re-embedding processing to the image frame immediately after the compressed image signal is decoded, that is, before undergoing frame rate conversion, the above-described cost increase and performance are achieved. It is possible to suppress the decrease.

  (13) Decode the compressed image signal and output the decoded image signal and information on the display timing of the image frame or field, and according to the display timing information for the decoded image signal Embed digital watermark signal.

  (14) Decode the compressed image signal, output information about the display timing of the decoded image signal and image frame or field, and use the digital watermark according to the display timing information from the decoded image signal Detect the signal.

  Decodes the compressed image signal in which the first digital watermark signal is embedded, outputs the decoded image signal and information on the display timing of the image frame or field, and outputs the decoded image signal to the decoded image signal The second digital watermark signal is re-embedded in accordance with the display timing information.

  When a digital watermark has a time-direction state transition or phase characteristic, it may cause a decrease in the detection performance of the digital watermark if affected by frame rate conversion or trick reproduction. On the other hand, in the image signal data compressed by MPEG2 or the like, information on the display time for each frame or field is also encoded. Therefore, if the information about the display time that compression coding has as described above is used as time information in the detection, embedding, and re-embedding of the digital watermark, it is possible to appropriately control the phase characteristics in the time direction of the digital watermark. In addition, the detection performance of the digital watermark can be improved.

  According to the present invention, optimal embedding strength control for feedback or feedforward and embedding in consideration of compression encoding of an image signal are performed, thereby realizing a highly robust digital watermark embedding while suppressing image quality deterioration. It becomes possible.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram showing the configuration of the digital watermark embedding apparatus according to the first embodiment of the present invention. The digital watermark embedding apparatus according to the present embodiment includes a digital watermark signal generation unit 10, a digital watermark embedding unit 11, a digital watermark detection unit 12, and a digital watermark strength control unit 13.

  The input image signal 14 is input to the digital watermark signal generation unit 10 and the digital watermark embedding unit 11. A digital watermark signal 15 is generated by a known technique based on the input image signal 14 by the digital watermark signal generation unit 10, and the digital watermark signal 15 is embedded in the input image signal 14 by the digital watermark embedding unit 11. An output image signal 16 is generated.

  The output image signal 16 is input to the digital watermark detection unit 12. The digital watermark detection unit 12 extracts a digital watermark signal from the output image signal 16, obtains the signal strength of the extracted digital watermark signal, and supplies signal strength information 17 to the digital watermark strength control unit 13. The digital watermark strength control unit 13 determines the embedding strength of the digital watermark signal according to the signal strength information 17 so that the signal strength of the digital watermark signal extracted by the digital watermark detection unit 12 is constant, for example. An embedding strength control signal 18 is supplied to the unit 11. In accordance with the embedding strength control signal 18, the embedding strength of the digital watermark signal 15 in the digital watermark embedding unit 11 is controlled to be, for example, a desired constant strength.

  In this way, the strength of the digital watermark signal embedded in the output image signal 16 is fed back to the digital watermark embedding unit 11 as the embedding strength, and the embedding strength of the digital watermark signal 15 is controlled to embed the digital watermark signal 15. It becomes possible to carry out stably at a constant strength.

(Second Embodiment)
FIG. 2 shows a configuration of a digital watermark embedding apparatus according to the second embodiment of the present invention. This embodiment is different from the first embodiment in that the digital watermark detection unit 12 in FIG. 1 is replaced with an image quality deterioration degree detection unit 19. The image quality degradation level detection unit 19 inputs the image signal before and after embedding the digital watermark signal 15, that is, the input image signal 14 and the output image signal 16, and determines the image quality degradation level of the output image signal 16 with respect to the input image signal 14. Then, the image quality deterioration degree information 20 is supplied to the digital watermark strength control unit 13. The degree of image quality degradation is obtained, for example, as a sum of absolute differences of pixel values (luminance values) between the input image signal 14 and the output image signal 16 or an average square error.

  The digital watermark strength control unit 13 determines the embedding strength of the digital watermark signal according to the image quality degradation level information 20 so that the image quality degradation level is, for example, a predetermined threshold value or less, and embeds the digital watermark signal in the digital watermark embedding unit 11 based thereon. An intensity control signal 18 is provided. By controlling the embedding strength of the digital watermark signal 15 in the digital watermark embedding unit 11 in accordance with the embedding strength control signal 18, it is possible to prevent perception of image quality degradation caused by embedding the digital watermark signal 15.

(Third embodiment)
FIG. 3 shows the configuration of a digital watermark embedding apparatus according to the third embodiment of the present invention. In the first and second embodiments, the embedding strength of the digital watermark signal is controlled by feedback, whereas in this embodiment, the embedding strength of the digital watermark signal is controlled by feedforward control. Specifically, from the input image signal 14 and the digital watermark signal 15 generated by the digital watermark signal generation unit 10, the activity detection unit 21 detects each activity, that is, the complexity, for example, the input image signal 14 and the digital watermark Activity information 22 indicating the activity ratio of the signal 15 is supplied to the digital watermark strength control unit 13.

  The digital watermark strength control unit 13 determines the embedding strength of the digital watermark signal according to the activity information 22 so that the ratio of the activity of the input image signal 14 and the digital watermark signal 15 becomes a predetermined constant value, for example. An embedding strength control signal 18 is supplied to the digital watermark embedding unit 11. By controlling the embedding strength of the digital watermark signal 15 in the digital watermark embedding unit 11 according to the embedding strength control signal 18, the embedding strength of the digital watermark signal is maximized while suppressing image quality deterioration caused by embedding the digital watermark signal 15. Can be increased.

(Fourth embodiment)
FIG. 4 is a block diagram showing a configuration of a digital watermark embedding apparatus according to the fourth embodiment of the present invention. In this embodiment, the feedback control based on the signal strength of the digital watermark signal included in the output image signal 16 described in the first embodiment and the feedforward control based on the activity information described in the third embodiment are combined. Thus, the embedding strength of the digital watermark signal is controlled.

  That is, the digital watermark strength control unit 13 can maintain a predetermined embedding strength while suppressing image quality degradation by using the signal strength information 17 output from the digital watermark detection unit 12 and the activity information 22 output from the activity detection unit 21. Thus, for example, the embedding strength control signal 18 is generated as the product of the embedding strength adjustment amount based on the signal strength information 17 and the embedding strength adjustment amount based on the activity information 22. By controlling the embedding strength of the digital watermark signal 15 in the digital watermark embedding unit 11 in accordance with the embedding strength control signal 18, it is possible to stably embed the digital watermark signal 15 while suppressing deterioration in image quality due to the embedding of the digital watermark signal 15. Strength can be increased. Therefore, even when the output image signal 16 after the digital watermark signal is embedded is deteriorated or deformed, the digital watermark can be detected more robustly.

(Fifth embodiment)
FIG. 5 shows the configuration of a digital watermark embedding apparatus according to the fifth embodiment of the present invention. In the present embodiment, the feedback control based on the image quality degradation degree of the output image signal 16 with respect to the input image signal 14 described in the second embodiment is combined with the feedforward control based on the activity information described in the third embodiment. Thus, the embedding strength of the digital watermark signal is controlled.

  In other words, the digital watermark strength control unit 13 uses the image quality degradation level information 20 output from the image quality degradation level detection unit 19 and the activity information 22 output from the activity detection unit 21 to obtain a predetermined embedding strength while suppressing image quality degradation. For example, the embedding strength control signal 18 is generated as a product of the embedding strength adjustment amount based on the image quality degradation level information 20 and the embedding strength adjustment amount based on the activity information 22. With such a configuration, the embedding strength of the digital watermark signal 15 in the digital watermark embedding unit 11 is controlled according to the embedding strength control signal 18, thereby suppressing image quality deterioration caused by embedding the digital watermark signal 15. The embedding strength of the digital watermark signal 15 can be increased stably.

(Sixth embodiment)
FIG. 6 is a block diagram showing a configuration of a digital watermark embedding apparatus according to the sixth embodiment of the present invention. In the present embodiment, when the input image signal 14 is an image signal generated by telecine conversion, the digital watermark signal is embedded in consideration of telecine conversion.

  In order to describe this embodiment, telecine conversion and inverse telecine conversion will be described first with reference to FIG. The frame rate (frame frequency) of the video recorded on the movie film is generally 24 Hz, and the process of converting this film video into a video signal (TV signal) with a frame rate of 30 Hz (field frequency 60 Hz) is telecine conversion. Call it.

  At the time of telecine conversion, as shown in FIG. 7, each frame 30 to 35 of the film image is divided into two fields of a top field and a bottom field by interlace. Next, a video image composed of the fields 40a to 40c, 41a to 41b,... Is generated by inserting a field (repeat field) for redisplaying the bottom field once every two frames of the film image.

  By this operation, for example, the frame 30 in the film image is converted into three fields 40a, 40b, and 40c. Of these fields, the fields 40a and 40b are fields based on interlace, and the field 40c is the same repeat field as the field 40a. The next frame 31 in the field video is converted into two fields 41a and 41b based on interlace. Thus, by converting each frame of the film image alternately into 3 fields and 2 fields, frame rate conversion from 24 Hz to 30 Hz, that is, telecine conversion is performed.

  Conversely, an operation for returning a video signal with a frame rate of 30 Hz obtained by telecine conversion to a film image signal with a frame rate of 24 Hz is called inverse telecine conversion. Inverse telecine conversion is performed by deleting a repeat field and restoring one frame from two interlaced fields.

In MPEG2 moving image compression used in DVDs or the like, when compressing a video signal obtained by telecine conversion, in order to increase encoding efficiency, after performing inverse telecine conversion to return the frame rate to the original 24 Hz, the compression code It is common to perform the conversion. In inverse telecine conversion, a repeat field is usually automatically detected by calculating the correlation or difference with the in-phase field one frame before each video signal field. For example, as shown in Equation (1), y _i (v, h) indicates the luminance signal of the pixel (v, h) of the i-th field, and the absolute value of the luminance signal of the field two fields before The difference sum J _i is calculated. If J _i is smaller than a predetermined threshold, the field i is detected as a repeat field.

  In the present embodiment, as shown in FIG. 6, the repeat field detector 23 has a repeat field detection from the input image signal 14 obtained by telecine conversion in the same manner as in the inverse telecine conversion described above. The detection signal 24 is supplied to the digital watermark signal generation unit 10. The digital watermark signal generating unit 10 recognizes whether the input image signal 10 is a normal field or a repeat field based on the detection signal 24, and the repeat field is the same as the digital watermark signal embedded in the field two fields before. Generate a signal. The digital watermark signal 15 generated in this way is embedded in the input image signal 10 by the digital watermark embedding unit 11.

  If a different digital watermark signal is embedded for each field, it becomes difficult to detect a repeat field in inverse telecine conversion at the time of compression coding, and the coding efficiency is lowered. Alternatively, if a repeat field is detected and the repeat field is deleted during compression encoding, there is a problem that a digital watermark signal is lost.

  On the other hand, if the same digital watermark signal as that two fields before is embedded in the repeat field as in this embodiment, the signals in the repeat field and the field two fields before match even after the digital watermark signal 15 is embedded. To do. Therefore, it becomes easy to detect a repeat field in inverse telecine conversion at the time of compression encoding, and the influence on inverse telecine conversion, such as a decrease in encoding efficiency, can be eliminated. Even if the repeat field is deleted during inverse telecine conversion, the digital watermark signal embedded in the repeat field is the same as the previous two fields, so that it is possible to prevent the digital watermark signal from being lost.

(Seventh embodiment)
FIG. 8 is a block diagram showing a configuration of a digital watermark embedding apparatus according to the seventh embodiment of the present invention. This embodiment is another example of a technique for embedding a digital watermark signal in consideration of telecine conversion when the input image signal 14 is an image signal generated by telecine conversion, as in the sixth embodiment. is there.

  In the present embodiment, the input image signal 14 is subjected to inverse telecine conversion by the inverse telecine conversion unit 25 and then input to the digital watermark signal generation unit 10 and the digital watermark embedding unit 11. In this way, the digital watermark signal 15 is embedded after the inverse telecine conversion in the input image signal 14 generated by the telecine conversion. Thereby, it is possible to eliminate the influence on the automatic inverse telecine conversion accompanying the embedding of the digital watermark signal 15 and to prevent the digital watermark signal from being lost due to the inverse telecine conversion.

(Eighth embodiment)
Next, an eighth embodiment of the present invention will be described with reference to FIGS. FIG. 9 is a diagram illustrating the configuration of the digital watermark embedding device according to the present embodiment, and is a device that re-embeds the digital watermark signal in the image signal in which the digital watermark signal is embedded.
Such re-embedding of the watermark signal reinforces the watermark signal that has been weakened due to repeated compression coding and recording / playback, so that the already-embedded watermark signal can be overwritten again or copy generation management can be performed. Therefore, it is used when the digital watermark signal is rewritten according to the number of times of copying.

  In the present embodiment, the digital watermark detection unit 50 detects the digital watermark and the geometric deformation from the input image signal 53, and the digital watermark / geometric deformation information 54 is sent to the digital watermark signal generation unit 51 based on these detection results. Supply. The digital watermark signal generation unit 51 generates a digital watermark signal 55 based on the input image signal 53 and the digital watermark / geometric deformation information 54 detected by the digital watermark detection unit 50. The digital watermark signal 55 generated in this manner is superimposed on the input image signal 53 by the digital watermark re-embedding unit 52, thereby re-embedding the digital watermark signal and generating the output image signal 56 with the digital watermark re-embedded. To do.

  In general, an image signal in which a digital watermark signal is embedded may be subjected to some geometric deformation. Specifically, there are spatial geometric deformations such as enlargement / reduction or parallel movement of the screen, and temporal geometric deformations such as fast-forwarding, slow reproduction, or frame rate conversion.

  FIG. 10 shows an example of spatial geometric deformation. A frame 60 is an image in which a digital watermark signal is embedded, and the digital watermark signal is embedded in a central wavy line portion. With respect to the frame 60, a frame 61 shows a reduced image, a frame 62 shows an enlarged image, and a frame 63 shows an image translated in the upper left.

  FIG. 11 shows an example in which a digital watermark signal three-dimensionally embedded in the space-time of an image signal is subjected to deformation in the time direction. The plurality of frames 64 are images in which digital watermark signals are embedded three-dimensionally in the shaded portions. The frame 65 is an example of an image in which the phase of the digital watermark signal is shifted in the time direction as the time axis of the frame 64 is shifted. The frame 66 has the digital watermark signal expanded in the time direction by slow playback or frame rate conversion. An example of an image, a frame 67, is an example of an image subjected to time-axis compression in contrast to the frame 66.

  When the image is subjected to spatial or temporal geometrical deformation as shown in FIG. 10 or FIG. 11, it becomes difficult to detect a digital watermark signal that is generally embedded or to re-embed a digital watermark signal. However, the detection performance of the digital watermark signal can be improved by estimating the geometric deformation parameter when detecting or re-embedding the digital watermark signal for the image signal subjected to these geometric deformations. It is also possible to accurately re-embed the digital watermark signal. In the present embodiment, such a geometric deformation parameter is detected by the digital watermark detection unit 50 shown in FIG.

  FIG. 12 shows an example of the internal configuration of the digital watermark detection unit 50 in FIG. With the configuration of FIG. 12, it is possible to simultaneously detect a digital watermark and estimate a geometric deformation. First, based on the image signal 53 in which the digital watermark signal is embedded in advance, the digital watermark signal 72 is generated by the digital watermark signal generation unit 71 in the same manner as when the digital watermark signal is embedded.

  Next, a plurality of geometric deformation units 73 perform geometric deformation processing of a plurality of different parameters such as enlargement / reduction or parallel movement on the digital watermark signal 72, and a plurality of digital watermark signals 75a subjected to the geometric deformation processing. Are generated, and each geometric deformation parameter 75b is output.

 The plurality of digital watermark signals 75a and the input image signal 53 after the geometric deformation processing are input to the matching unit 74, and a correlation operation between each digital watermark signal 75a and the input image signal 53 is performed, and each digital watermark signal 75a is performed. A plurality of correlation coefficients 76 corresponding to are generated.

  Each correlation coefficient 76 and the geometric deformation parameter 75b are influential to the geometric deformation estimation unit 77. The geometric deformation estimation unit 77 compares the magnitudes of the respective correlation coefficients 76, and a plurality of electrons after the geometric deformation processing is performed. Among the watermark signals 75a, a geometric deformation parameter that gives the maximum correlation coefficient is selected from 75b, and digital watermark information is detected. The geometric deformation estimation unit 77 outputs the selected geometric deformation parameter information 78 and the detected digital watermark information 79 as the digital watermark / geometric deformation information 54 described with reference to FIG.

  FIG. 13 shows an example of the internal configuration of the digital watermark signal generation unit 51 and the digital watermark re-embedding unit 52 in FIG. These digital watermark signal generation unit 51 and digital watermark re-embedding unit 52 are arranged in the subsequent stage of the digital watermark detection unit 50 shown in FIG.

  The re-embedding digital watermark signal generation unit 80 generates a re-embedding digital watermark signal 81 based on the input image signal 53 and the digital watermark information 79 detected by the digital watermark detection unit 50 of FIG. When the re-embedding of the watermark signal is intended to reinforce the watermark signal already embedded in the input image signal 53, the re-embedding watermark signal generation unit 80 is the same as the detected watermark information 79. A digital watermark signal is generated. When the re-embedding of the digital watermark signal is for the purpose of copy generation management or the like, the re-embedding digital watermark signal generation unit 80 generates a digital watermark signal indicating that the copy generation has progressed.

  The re-embedded digital watermark signal 81 is input to the geometric deformation unit 82, and is subjected to geometric deformation based on the geometric deformation parameter 78 detected by the digital watermark detection unit 50 shown in FIG. The re-embedded digital watermark signal after geometric deformation is re-embedded in the input image signal 53 by the digital watermark re-embedding unit 83.

  Finally, according to the digital watermark information 79 detected by the digital watermark detection unit 50, either an image signal in which the digital watermark signal is re-embedded or an image signal that has not been re-embedded is switched by the selector 84 and output. . For example, if a digital watermark signal for copy generation management is embedded in the selector 84, an image signal in which the digital watermark signal with updated generation information is re-embedded is output, and a digital watermark signal that does not require copy generation management is embedded. If so, the input image signal 53 not re-embedded with the digital watermark signal is output as it is.

(Ninth embodiment)
FIG. 14 is a block diagram showing a configuration of a digital watermark re-embedding apparatus according to the ninth embodiment of the present invention. An input image signal in which the digital watermark signal described in the eighth embodiment is already embedded is shown. A re-embedding intensity control function is provided in a digital watermark re-embedding apparatus that re-embeds a digital watermark signal.

  As with the digital watermark detection unit 50 described in the eighth embodiment, the digital watermark detection unit 91 detects digital watermark information and estimates geometric deformation parameters for the input image signal 90 in which the digital watermark signal has already been embedded. And digital watermark / geometric deformation information 93a is output. The digital watermark detection unit 91 further outputs signal strength information 93 b indicating the strength of the detected digital watermark signal and supplies the signal strength information 93 b to the digital watermark strength control unit 95.

  The re-embedding digital watermark signal generation unit 92 generates the re-embedding digital watermark signal 94 based on the digital watermark / geometric deformation information 93a from the digital watermark detection unit 91 and the input image signal 90, as in the eighth embodiment. . The digital watermark re-embedding unit 97 generates the output image signal 100 by superimposing the re-embedded digital watermark signal 94 on the input image signal 90.

  The output image signal 100 in which the digital watermark signal is re-embedded is input to the digital watermark detection unit 99. The digital watermark detection unit 99 supplies the signal strength information 98 to the digital watermark strength control unit 95 by extracting the digital watermark signal from the output image signal 100 and detecting the signal strength as in the first embodiment. .

  Based on the signal strength information 93 b and 98, the digital watermark strength control unit 95 determines the signal strength of the digital watermark signal in the input image signal 90 and the signal strength of the digital watermark signal in the output image signal 100 after re-embedding the digital watermark signal. Then, the embedding strength at the time of re-embedding the digital watermark signal is determined, and a re-embedding strength control signal 96 is supplied to the digital watermark re-embedding unit 97 based on the embedding strength. Thereby, the digital watermark re-embedding unit 97 dynamically controls the re-embedding strength of the digital watermark signal.

For example, when the re-embedding of the digital watermark signal is intended to reinforce the strength by overwriting, the re-embedding strength is controlled so that the detection strength of the digital watermark detection unit 99 is higher than the detection strength of the digital watermark detection unit 91. . When the digital watermark re-embedding is intended for copy generation management, the re-embedding strength is controlled so that the detection strength of the digital watermark detection unit 99 is substantially equal to the detection strength of the digital watermark detection unit 91.
Therefore, according to the present embodiment, the strength of re-embedding of the digital watermark can be stably controlled, and the detection performance of the digital watermark signal after re-embedding can be guaranteed.

(Tenth embodiment)
FIG. 15 is a block diagram showing a configuration of a digital watermark embedding apparatus having a function of controlling the re-embedding strength of a digital watermark signal according to the tenth embodiment of the present invention. In this embodiment, a re-embedding counter 101 and a selector 102 are added to the configuration of the ninth embodiment shown in FIG. 14, and the other components are the same as those in FIG.

  In this embodiment, the digital watermark detection unit 91 has a function of detecting the presence or absence of a digital watermark signal and determining whether it is necessary to re-embed the digital watermark. The digital watermark signal is detected and re-embedding is necessary. Is determined, the re-embedding counter 101 is reset by the signal 93c, that is, a predetermined initial value is set in the counter 101. The re-embedding counter 101 is a subtraction counter, and continues to count down until the value becomes zero. The selector 102 selects the image signal 100 after re-embedding the digital watermark signal from the digital watermark re-embedding unit 97 during the period when the value of the re-embedding counter 101 is a non-zero value, and the value of the re-embedding counter 101 becomes zero. Then, the input image signal 90 is selected and output as the output image signal 103, respectively.

Next, the procedure of the digital watermark re-embedding process in the present embodiment will be described using the flowchart shown in FIG.
The digital watermark detection unit 91 generates a detection event at a constant period (step S1), decodes the information of the digital watermark signal at the time when the detection event occurs, and determines whether or not to re-embed (remark). If it is determined that remarking should be performed (step S2), the re-embedding counter 101 is reset and an initial value is set (step S3).

  Next, it is checked whether or not the value of the re-embedding counter 101 is positive (step S4). If it is positive, the digital watermark signal is re-embedded (step S5), and the re-embedding counter 101 is counted down (step S4). S6). Even when a detection event has not occurred (No in step S1) and when a digital watermark signal to be remarked is not detected (No in step S2), the re-embedding counter is determined in step S4. As long as the value is positive, the re-embedding of the digital watermark (step S5) continues.

  FIG. 17 is a timing chart showing the processing shown in FIG. 16 in the present embodiment. In the input image signal 90, a digital watermark signal is embedded in the period 104 of FIG. The digital watermark detection unit 91 generates a detection event at a predetermined period while sliding the detection window 105 having a certain time width. The end time of each detection window (106 in the figure for the detection window 105) is the occurrence time of the detection event. Among these detection events, the one indicated by the arrow in the figure (for example, 107) is a digital watermark to be remarked. Is detected. The hatched portions in the re-embedding pattern A and the re-embedding pattern B in FIG. 17 indicate periods during which the re-embedding process is executed.

  In the re-embedding pattern A, when a digital watermark to be remarked is detected for each detection event, the digital watermark is re-embedded only for one detection event period. When the re-embedding pattern A is used, if the digital watermark signal detection rate in the digital watermark detection unit 91 is not 100%, on / off of the re-embedding process of the digital watermark signal occurs intermittently. Therefore, it is difficult to stably detect the digital watermark signal after the image signal 103 output from the digital watermark re-embedding apparatus is received via the transmission path or the recording medium.

  On the other hand, in the re-embedding pattern B, an initial value is set in the re-embedding counter 101 every time a digital watermark signal to be remarked is detected, and re-embedding is continued as long as the value of the re-embedding counter 101 is positive. Regardless of the detection rate of the digital watermark signal in the digital watermark detection unit 91, the digital watermark re-embedding unit 97 can stably re-embed the digital watermark signal 94, and is output from the digital watermark re-embedding device. When the image signal 103 is received and the digital watermark signal is detected, the digital watermark that has been stably re-embedded can be detected.

  In FIG. 17, a time variation pattern 108 of the value of the re-embedding counter 101 is also shown. As is apparent from this pattern 108, the initial value at the time of resetting the re-embedding counter 101 is preferably the same as or longer than the length of the detection window 105, thereby detecting the re-embedded digital watermark signal more. It becomes possible to carry out stably.

(Eleventh embodiment)
FIG. 18 is a diagram showing a configuration of a digital watermark embedding apparatus according to the eleventh embodiment of the present invention. In the present embodiment, a digital watermark signal generation unit 113 and a digital watermark embedding unit 115 are sequentially arranged after the MPEG decoder 111 that decodes the image data 110 compressed by a moving image compression method such as MPEG. A frame rate conversion unit 117 is arranged following the digital watermark embedding unit 115.

  In other words, the digital watermark signal 114 generated by the digital watermark signal generation unit 113 is embedded in the image signal 112 decoded by the MPEG decoder 111 by the digital watermark embedding unit 115. A frame rate conversion unit 117 converts the frame rate of the image signal 116 in which the digital watermark signal 114 is embedded, and generates an output image signal 118. The frame rate conversion unit 117 performs telecine conversion or frame rate conversion according to the characteristics of the display device.

  According to the present embodiment, by embedding a digital watermark signal in the image signal 112 immediately after decoding before frame rate conversion, the output frame rate (the frame rate of the output image signal 118) is determined. Therefore, it is possible to easily embed a common digital watermark signal. Therefore, it is not necessary to perform different digital watermark embedding for each output frame rate, and the digital watermark signal embedding process can be realized at low cost.

(Twelfth embodiment)
FIG. 19 is a diagram showing a configuration of a digital watermark detection apparatus according to the twelfth embodiment of the present invention. In this embodiment, a digital watermark detection unit 119 and a frame rate conversion unit 117 are arranged in parallel at a subsequent stage of the MPEG decoder 111 that decodes the image data 110 compressed by a moving image compression method such as MPEG. That is, the digital watermark signal is detected at the same time as the image signal 112 decoded by the MPEG decoder 111, and at the same time, the frame rate conversion is performed to generate the output image signal 118.

  According to the present embodiment, as in the eleventh embodiment, the digital watermark signal is detected for the image signal 112 immediately after the decoding before the frame rate conversion, so that the output frame rate (output image signal) is detected. 118), a common digital watermark signal can be easily detected, and there is no need to perform different digital watermark detection for each output frame rate. Detection can be performed.

(13th Embodiment)
FIG. 20 is a diagram showing a configuration of a digital watermark re-embedding apparatus according to the thirteenth embodiment of the present invention. In the present embodiment, a digital watermark detection unit 120, a digital watermark signal generation unit 122, and a digital watermark re-embedding unit 124 are provided after the MPEG decoder 111 that decodes the image data 110 compressed by a moving image compression method such as MPEG. Are arranged sequentially. A frame rate conversion unit 117 is arranged following the digital watermark re-embedding unit 124.

  That is, a digital watermark signal is detected by the digital watermark detection unit 120 for the image signal 112 decoded by the MPEG decoder 111, and the re-embedded electronic signal is detected according to the digital watermark detection result 121 and the decoded image signal 112. The watermark signal generation unit 122 generates a re-embedding digital watermark signal 123, and the frame rate conversion unit 117 performs frame rate conversion on the image signal 125 in which the digital watermark re-embedding unit 124 re-embeds the digital watermark signal. The output image signal 118 is generated.

  According to the present embodiment, the output frame rate (the frame rate of the output image signal 118) is detected by performing digital watermark detection and re-embedding on the image signal 112 immediately after decoding before frame rate conversion. Regardless, the digital watermark detection and re-embedding processes can be shared. Therefore, it is not necessary to perform different digital watermark detection and re-embedding for each output frame rate, and stable digital watermark detection and re-embedding can be performed while suppressing processing costs.

(Fourteenth embodiment)
FIG. 21 is a diagram showing a configuration of a digital watermark embedding apparatus according to another embodiment of the present invention. In the present embodiment, a digital watermark signal generation unit 113 and a digital watermark embedding unit 115 are sequentially arranged after the MPEG decoder 111 that decodes the image data 110 compressed by a moving image compression method such as MPEG.

  The MPEG decoder 111 outputs a decoded image signal 112 and time stamp information 212 regarding the display time of each decoded frame or field. The digital image signal 112 is generated by the digital watermark signal generation unit 113 and the digital watermark signal is embedded by the digital watermark embedding unit 115 according to the time stamp information 212 related to the display time. Is generated.

(Fifteenth embodiment)
FIG. 22 is a block diagram showing a configuration of a digital watermark detection apparatus according to another embodiment of the present invention. In the present embodiment, a digital watermark detection unit 119 is arranged after the MPEG decoder 111 that decodes the image data 110 compressed by a moving image compression method such as MPEG.

  Similar to the fourteenth embodiment, the MPEG decoder 111 outputs a decoded image signal 112 and time stamp information 212 relating to the display time of each decoded frame or field. In the present embodiment, the digital watermark signal is detected by the digital watermark detection unit 119 according to the time stamp information 212 related to the display time for the decoded image signal 112, and at the same time, the decoded image signal 112 is Output.

(Sixteenth embodiment)
FIG. 23 is a diagram showing a configuration of a digital watermark embedding apparatus that re-embeds a digital watermark signal according to the sixteenth embodiment of the present invention. In the present embodiment, a digital watermark detection unit 120, a digital watermark signal generation unit 122, and a digital watermark re-embedding unit 124 are provided after the MPEG decoder 111 that decodes the image data 110 compressed by a moving image compression method such as MPEG. Are arranged sequentially. Similar to the fourteenth and fifteenth embodiments, the image signal 112 decoded from the MPEG decoder 111 and the time stamp information 212 relating to the display time of each decoded frame or field are output.

  In the present embodiment, detection of a digital watermark signal by the digital watermark detection unit 120 and re-embedding digital watermark signal by the digital watermark signal generation unit 122 are performed on the decoded image signal 112 in accordance with the time stamp information 212 regarding the display time. And re-embedding the digital watermark signal by the digital watermark re-embedding unit 124, and output the image signal 125 in which the digital watermark signal is re-embedded.

A moving image signal compressed by MPEG or the like may not have a constant encoding frame rate due to frame decimation, inverse telecine conversion, or the like, and a frame or field encoding / decoding order and display order may be reordered. May be different. This situation is shown in FIGS.
FIG. 24 shows an example in which the frame rate is variable by frame thinning. In the encoded frames 130, 131, and 132, time stamps related to the display times are uniquely determined at the time of decoding as information for determining the display time of each frame.
FIG. 25 shows an example of reordering, in which encoded frames 140, 141, 142, and 143 are arranged in the order of encoding or decoding, and each time stamp is uniquely determined as information that determines the display time. .

  When digital watermark embedding, detection, or re-embedding is processed as a spatio-temporal three-dimensional pattern for a moving image signal, it is correctly embedded and detected if it is affected by frame decimation or reordering caused by encoding. Or, it becomes difficult to perform re-embedding. However, in a moving picture coding system such as MPEG, a time stamp relating to a display time is uniquely determined at the time of decoding for each coded frame or field. Therefore, by receiving the time stamp information 212 from the MPEG decoder 111 and embedding, detecting, or re-embedding the digital watermark as in this embodiment, even if each process includes a spatio-temporal process, It is possible to embed, detect or re-embed a digital watermark.

1 is a block diagram showing a configuration of a digital watermark embedding apparatus according to a first embodiment of the present invention. The block diagram which shows the structure of the digital watermark embedding apparatus based on the 2nd Embodiment of this invention. The block diagram which shows the structure of the digital watermark embedding apparatus based on the 3rd Embodiment of this invention. The block diagram which shows the structure of the digital watermark embedding apparatus based on the 4th Embodiment of this invention. The block diagram which shows the structure of the digital watermark embedding apparatus based on the 5th Embodiment of this invention. The block diagram which shows the structure of the digital watermark embedding apparatus based on the 6th Embodiment of this invention. Diagram explaining telecine conversion and inverse telecine conversion The block diagram which shows the structure of the digital watermark embedding apparatus based on the 7th Embodiment of this invention. The block diagram which shows the structure of the digital watermark embedding apparatus which re-embeds a digital watermark signal in consideration of the geometric deformation which concerns on the 8th Embodiment of this invention. The figure which shows the example of the geometric deformation of the spatial direction of an image signal The figure which shows the example of the geometric deformation of the time direction of an image signal FIG. 10 is a block diagram showing a configuration example of a digital watermark detection unit in the eighth embodiment of the present invention. The block diagram which shows the structural example of the digital watermark signal generation part and digital watermark re-embedding part in the 8th Embodiment of this invention. The block diagram which shows the structure of the digital watermark embedding apparatus which re-embeds the digital watermark signal which has the feedback intensity | strength control part which concerns on the 9th Embodiment of this invention. FIG. 10 is a block diagram showing a configuration of a digital watermark embedding apparatus that re-embeds a digital watermark signal according to a tenth embodiment of the present invention. The flowchart which shows the procedure of the digital watermark re-embedding control in the 10th Embodiment of this invention. Timing chart showing the procedure of digital watermark re-embedding control in the tenth embodiment of the present invention The block diagram which shows the structure of the digital watermark embedding apparatus based on the 11th Embodiment of this invention. The block diagram which shows the structure of the digital watermark detection apparatus which concerns on the 12th Embodiment of this invention. The block diagram which shows the structure of the digital watermark re-embedding apparatus based on the 13th Embodiment of this invention. Block diagram showing the configuration of a digital watermark embedding apparatus according to a fourteenth embodiment of the present invention A block diagram showing a configuration of a digital watermark detection apparatus according to a fifteenth embodiment of the present invention. The block diagram which shows the structure of the digital watermark re-embedding apparatus based on the 16th Embodiment of this invention. The figure which shows the example of the time stamp of a decoding image The figure which shows the example of the time stamp of a decoding image

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Digital watermark signal generation part 11 ... Digital watermark embedding part 12 ... Digital watermark detection part 13 ... Digital watermark strength control part 19 ... Image quality degradation degree detection part 21 ... Activity detection part 23 ... Repeat field detection part 25 ... Inverse telecine conversion part DESCRIPTION OF SYMBOLS 50 ... Digital watermark detection part 51 ... Digital watermark signal generation part 52 ... Digital watermark re-embedding part 71 ... Digital watermark generation part 91 ... Digital watermark detection part 92 ... Re-embedding digital watermark signal generation part 95 ... Digital watermark intensity control part 97 ... Digital watermark re-embedding unit 99 ... Digital watermark detection unit 101 ... Re-embedding counter 102 ... Selector 111 ... MPEG decoder 113 ... Digital watermark signal generation unit 115 ... Digital watermark embedding unit 117 ... Frame rate conversion unit 119 ... Digital watermark detection unit 120 ... Digital watermark detector 122 ... Digital watermark Number generation unit 124 ... Digital watermark re-embedding unit

Claims (1)

In an electronic watermark embedding apparatus for generating an output image signal by embedding an electronic watermark signal in a field unit with respect to an input image signal generated by telecine conversion by insertion of a repeat field,
Detecting means for detecting the repeat field from the input image signal;
An electronic watermark comprising an embedding means for embedding the same digital watermark signal as the digital watermark signal embedded in the field two fields before the repeat field of the input image signal based on the detection result by the detection means Implantation device.

Images