JP2010258645A - Method and device for embedding digital watermark - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and a device for embedding a digital watermark, for embedding change information that changes sequentially and regularly in content such as video and sound as a digital watermark. <P>SOLUTION: The method for embedding a digital watermark is used for a device for embedding a digital watermark which includes: a video input-output board 111 for capturing video, receiving the captured video as an image frame and outputting an image frame obtained by embedding the digital watermark in the input image frame; and a memory for storing an digital watermark pattern as a watermark pattern. The method for embedding the digital watermark generates the watermark pattern in accordance with the change information that changes sequentially and regularly, stores the generated watermark pattern into the memory, retrieves the watermark pattern by using the change information corresponding to the input image frame from the video input-output board as a key, and outputs the image frame obtained by embedding the retrieved watermark pattern in the input image frame as the digital watermark to the video input-output board 111. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a digital watermark technique for embedding information in digital content, and to a digital watermark embedding method and apparatus.

  There is a digital watermark technology as one of the technologies capable of protecting the copyright and verifying the identity of content that can be used illegally. The digital watermark technique is a technique for embedding and detecting arbitrary information such as identification information in the content so that the content is not apparent by changing the content slightly. Since the embedded information can be detected even after processing such as format conversion, it can be applied to content copyright claims, distribution channel tracking, authenticity certification, and the like. For example, Patent Document 1 and Non-Patent Document 1 disclose a method for detecting a change in the reproduction time of a video using a digital watermark. In this way, in order to use the digital watermark for authenticity certification, it is possible to detect the presence or absence of tampering by embedding authenticity information such as a time code in the content, and detecting this to confirm consistency. Become.

  However, there are moving images, still images, sounds, and the like as contents targeted for the digital watermark technology. Since these data have a large data size, the digital watermark embedding process requires a lot of processing time.

  For example, there are Patent Document 2, Patent Document 3, and Patent Document 4 as techniques related to increasing the watermark embedding speed.

  Patent Document 2 states that “the process of embedding digital watermark information in content data is divided into two processes, a preliminary process and a main process. In the preliminary process, an embedding possibility analysis process is performed on content, and an allowable change level is set. The embedding strength map to be shown is created and retained, and in this process, the amount of change determined according to the allowable change level of the feature value at the relevant part indicated in the embedding strength map and the bit value of the bit is changed. The means for embedding the digital watermark information in the content "(paragraphs 0015 to 0018) is provided.

  Patent Document 3 states that “a step of generating digital watermark embedding information, a content element having a predetermined length is extracted from the content every predetermined time length, and m types (m is 1 or more) for each content element. (N) is embedded to generate m types of content elements, and based on the embedded information of the digital watermark, the content and the content element embedded with the digital watermark are combined to generate one content. Steps "(paragraph 0023) are provided. Furthermore, Patent Document 2 provides means for “generating in advance m types of digital watermark patterns that do not depend on content prior to digital watermark embedding” (paragraph 0051).

Patent Document 4 states, “MPEG is a process for detecting a motion vector and a fluctuation amount necessary for optimizing a change portion and a change intensity in consideration of a property as a moving image included in content information to be embedded with watermark information. The technology for reducing the load of a series of processing by sharing with the compression processing (paragraph 0010) is provided.

JP 2003-259314 A JP 2002-027225 A JP 2007-207051 A JP 2004-048219 A

Takaaki Yamada, Yoshiyasu Takahashi, Yasuhiro Fujii, Ryu Ebisawa, Hiroshi Yoshiura and Isao Echizen: "Evaluation of Integrity Verification System for Video Content Using Digital Watermarking", in book chapter of George A. Tsihrintzis, Maria Virvou, Robert J. et al. Howlett, Lakhmi C. et al. Jain (Eds.) New Directions in Intelligent Interactive Multimedia, Studies in Computational Intelligence Vol. 142, Springer, pp. 363-372 (2008)

  Patent Documents 2 and 3 are effective methods for obtaining n different watermark embedded contents by repeatedly embedding n types of identification information in one content. Patent Document 4 is an effective method for obtaining one watermark embedded content by embedding one type of identification information in one content. However, there is no mention of a corresponding method in the case of obtaining one watermark embedded content by embedding n types of identification information in one content.

  In the conventional digital watermark embedding speed-up method, assuming that the information to be embedded is fixed, a part of the digital watermark embedding process is executed in advance as a preprocess, and only the calculation result of the preprocess is obtained in the subsequent processes. By using it, the speed of the entire system is increased. Examples of the pre-processing include embeddability analysis processing in Patent Literature 2, watermark pattern generation processing in Patent Literature 3, and motion vector and variation detection processing in Patent Literature 4. However, if the information to be embedded in the video stream is constant, the strength of the digital watermark is weak, so it is desirable to change the information to be embedded. If the information to be embedded as a digital watermark is changed, the premise of these prior arts is not satisfied, so that the digital watermark embedding process cannot be accelerated.

  The present invention provides a digital watermark embedding method and apparatus that embeds change information that changes sequentially and regularly in content such as video and sound as a digital watermark.

  An aspect of the present invention includes a video input / output board that captures a video, inputs it as an image frame, outputs an image frame in which a digital watermark is embedded in the input image frame, and a memory that stores the digital watermark pattern as a watermark pattern. A digital watermark embedding apparatus and a digital watermark embedding method provided for generating a watermark pattern corresponding to change information that changes sequentially and regularly, storing the generated watermark pattern in a memory, Using the change information corresponding to the input image frame from the output board as a key, a watermark pattern is searched, and an image frame in which the searched watermark pattern is embedded as an electronic watermark in the input image frame is output to the video input / output board.

  In another aspect of the present invention, change information that changes sequentially and regularly is time information, and a key for searching for a watermark pattern includes current time information for processing an input image frame input from a video input / output board. Including.

  In still another aspect of the present invention, the change information is position information.

  In another aspect of the present invention, the memory for storing the watermark pattern is an on-memory DB.

  According to still another aspect of the present invention, a video input / output board that captures a video, inputs it as an image frame, and outputs an image frame in which a digital watermark is embedded in the input image frame, and stores the digital watermark pattern as a watermark pattern The first CPU includes a buffer memory and first and second CPUs. The first CPU generates a watermark pattern corresponding to change information that changes sequentially and regularly, and the generated watermark pattern is stored in the buffer memory. The second CPU inputs the watermark pattern from the buffer memory in response to the input image frame input from the video input / output board, and embeds the input watermark pattern in the input image frame as a digital watermark. Output the frame to the video input / output board.

  A further example of the content of the present invention is video content, and its specific mode is as follows.

[System preparation]
Even when the information to be embedded changes, if the manner of change is regular and predictable, such as a time code, it can be calculated in advance. Therefore, by predicting all time codes (for example, for 100 years) in advance and calculating the corresponding watermark pattern in advance, the calculated watermark pattern can be safely stored in the database (DB). deep. This is called a watermark pattern DB. The watermark pattern DB is generated in advance or securely distributed at the time of system installation.

[Pre-processing of digital watermark embedding]
When the digital watermark embedding system is activated, the watermark pattern DB is safely read into the memory.

[Post-processing of digital watermark embedding]
When a video starts to be input according to a user instruction, the time code embedding process is performed as a sequential process for each video frame. Here, at the time of embedding, a DB search is performed using a time code value corresponding to the current time as a key, and a watermark pattern corresponding to this is obtained. The time code embedding process is performed as a process for superimposing the input image and the watermark pattern.

[System operation]
Video with embedded watermark is output to a recorder or distribution facility. The integrity of continuous time codes is verified by detecting embedded information from recorded video or video that has been format-converted on another medium. That is, it is determined whether or not the video has been tampered with.

  According to this aspect, the time code embedding process can be executed at high speed.

  Furthermore, since the time code can be embedded in the video input by real-time processing, the video recording system and the video distribution system can be easily configured by connecting the video output to a recorder or the like.

  Further, in addition to the time code, change information such as GPS information in the in-vehicle camera can be sequentially embedded in the video.

  In addition to video, change information can be sequentially embedded in video even for audio data in conference recording.

  According to the present invention, change information that changes sequentially and regularly, such as a time code, is embedded as a digital watermark in content such as video and sound, so that the digital watermark embedding process is accelerated.

It is a block diagram of the digital watermark embedding apparatus in the first embodiment. It is a figure explaining the outline | summary of a digital watermark embedding and a detection process, and the concept of a watermark pattern. It is an operation example of a system including a digital watermark embedding device. It is explanatory drawing which shows the outline | summary of tampering detection. It is explanatory drawing which shows the outline | summary of embedding of the time code as a digital watermark. It is explanatory drawing about the synchronization of embedding information and a video recorder. It is a PAD of a superimposition processing unit that embeds time code information as a digital watermark. This is a PAD of N frame embedding processing. This is a PAD for alteration detection processing of video content. It is a PAD of a detection result diagnosis process that identifies the type of tampering action. It is explanatory drawing of the field structure of watermark pattern DB. It is PAD of the process which produces | generates watermark pattern DB. It is a block diagram of the electronic watermark embedding apparatus in a 2nd Example. It is a block diagram of the digital watermark embedding apparatus in a 3rd Example. It is a timing chart in the digital watermark embedding in the third embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiment, and the repetitive description thereof will be omitted.

  The apparatuses according to the present embodiment are connected via a network such as the Internet (not shown) and can transmit and receive data to and from each other. Each device may exchange data using a portable medium. The processing in each device described below is realized by executing a program for controlling input / output and image processing on a general information processing device such as a personal computer having a memory, a CPU, a disk, and an input / output device. Each processing unit to be executed executes. The program may be stored in advance in the storage device of each device, or may be introduced into the storage device from another device via an input / output device and a medium that can be used by each device when necessary. Good. The medium refers to, for example, a storage medium that can be attached to and detached from the input / output device, or a communication medium (that is, a network such as the Internet or a carrier wave or a digital signal that propagates through the network). In addition, the digital watermark embedding device, the recorder, the playback device, and the digital watermark detection device described below may be configured on the same information processing device.

  Embodiments 1 to 3 will describe a digital watermark embedding detection system that uses a moving image (video) as content and embeds and detects a time code.

  Note that the content targeted in these embodiments is not limited to moving images, but may be other types of data such as sound.

  FIG. 1 shows a configuration diagram of a digital watermark embedding apparatus for realizing digital watermark embedding in the first embodiment.

  The camera 101, the PC 102, and the recorder 103 are connected by a video signal line, and the time information source 104 is connected to the network. Instead of the camera 101, a playback device, a recording / playback device, a broadcast receiver, a video distribution facility, or the like that sends out a video signal may be used. Instead of the recorder 103, a video player, video monitor, broadcast transmitter, video distribution facility, or the like may be used. The time information source 104 may be connected to a higher-level certificate authority or the like to transmit and receive highly reliable time information. The validity and authenticity of the time may be verified using the time stamp certificate sent together with the transmitted time information. The time information source 104 may include a GPS function and a GPS antenna, and time information included in the GPS reception wave may be used. The timer 108 in the PC 102 may be used instead of the time information source 104, or the timer 103 in the PC 102 may be adjusted with reference to the time information source 104.

  The PC 102 includes a video input / output board 111 and a timer 108, and further includes (1) a watermark pattern construction processing unit 114, (2) a digital watermark embedding processing unit 112, and (3) a disk management processing unit mounted on a memory. Three processing units 113 are provided. The disk management processing unit 113 may be entirely or partially mounted on the disk device. Furthermore, a removable portable medium 105 that securely stores the key 106 and the certificate 107 in the tamper-resistant area is provided.

  The video input / output board 111 continuously captures the video signal input from the camera 101 as the input video 109, and sends the captured video frame 123 to the digital watermark embedding processing unit 112. Further, the video input / output board 111 receives the video frame 124 processed by the digital watermark embedding processing unit 112 and outputs it to the recorder 103 as the output video 110. The input video 109 and the output video 110 are synchronized, and an electronic watermark is embedded in the input video 109 input from the camera 101, and then output to the recorder 103. Sound is processed in the same way with the image.

  (1) The watermark pattern construction processing unit 114 further includes three processing units: a key setting process 115, a time setting process 116, and a watermark pattern generation process 116. The key setting processing unit 115 reads the key 106 and the certificate 107 stored in the portable medium 105, the portable medium 105 is a regular medium, the user is a regular user, and the digital watermark embedding device Mutual authentication that the core PC 102 is a legitimate device, that is, product authentication (medium authentication), user authentication (user authentication), and device authentication (PC 102 authentication) is executed. Each certificate may be prepared, or a single certificate may be used for batch authentication. For verification of the certificate, an external verification device may be inquired via the network. The time setting processing unit 116 sets the time using the time information from the time information source 104 or the timer 108. The watermark pattern generation processing unit 116 generates a watermark pattern corresponding to time code information corresponding to current time information or time information designated by the user. Time code information including time information (time code) will be described later. The generated watermark pattern may be encrypted using the key 106 read by the key setting processing unit 115. The watermark pattern encryption key may be encrypted using the key 106 on the portable medium and stored in the DB, or the key 106 may be used directly. The encryption method is based on common key encryption, replacement, ID-based encryption, or the like. The watermark pattern generated by the watermark pattern generation processing unit 116 or the encrypted watermark pattern is output to the disk management processing unit 113.

  (2) The disk management processing unit 113 receives a watermark pattern generated according to the current time or a future time designated by the user, or an encrypted watermark pattern, and registers it in the watermark pattern DB 117. The watermark pattern DB 117 stores a watermark pattern corresponding to the time code information. The access control unit 118 accesses the watermark pattern DB 117 through the verification process 119 for the certificate 107 read by the key setting processing unit 115.

  (3) The digital watermark embedding processing unit 112 embeds a digital watermark in the input video frame 123 and outputs the embedded image as a video frame 124. The digital watermark embedding processing unit 112 includes an initialization processing unit 120 and a post-processing unit 126 for each video frame. The initialization processing unit 120 performs device authentication, product authentication, and user authentication using the key 106 and the certificate 107 on the portable medium 105 read by the key setting processing unit 115, and the watermark pattern DB 117 is stored on the disk device. In some cases, the access control unit 118 is used to read the watermark pattern DB 121 (on-memory DB) on the memory that can be accessed at high speed. In accessing the watermark pattern DB 121, the certificate 107 is verified and access control is performed. The post-processing unit 127 for each video frame further includes a watermark pattern call processing unit 125 and a superimposition processing unit 126. In post-processing 127 for each video frame, first, the video frame 123 is read from the video input / output board 111. Next, the watermark pattern call processing unit 125 generates time code information corresponding to the current time, searches the watermark pattern DB 121 using the time code information as a key, and obtains a corresponding watermark pattern. The superimposing process 126 superimposes the searched watermark pattern on the video frame 123 and outputs it to the video input / output board 111 as a video frame 124 in which a digital watermark is embedded. The post-processing unit 127 for each video frame executes these series of processes for each video frame in real time.

  It has been described that the disk management processing unit 113 is provided and the generated watermark pattern is stored in the watermark pattern DB 117 and then read into the watermark pattern DB 121 on the memory. May be stored).

  FIG. 2 is a diagram for explaining the outline of digital watermark embedding and detection processing and the concept of a watermark pattern. The moving image content is composed of a plurality of video frames (hereinafter referred to as frames). In order to embed a digital watermark including time code information 200 in one or a plurality of frame sets, a watermark pattern 201 corresponding to the time code information 200 is generated, and the generated watermark pattern 201 is superimposed on the frame of the input image. To embed the watermark pattern in the frame. The digital watermark embedded content 202 is obtained by continuously embedding digital watermarks in all frames. The inspection target content 204 is obtained by adding image processing 203 such as format conversion and compression encoding to the digital watermark embedded content 202. A watermark pattern 205 is detected from the inspection target content 204, and time code information 206 corresponding to the detected watermark pattern 205 is obtained.

As the time code, for example, a relative value as an elapsed time from an absolute time such as midnight on January 1, 2000 is used. The time code information includes a segment position, segment number, time code, content ID, user ID, camera number, device number, recorder number, and the like. The video is divided into a plurality of segments in units of a predetermined time (for example, 1 minute). The segment position included in the time code information indicates marks such as a start position and an end position for authenticating the video. The segment number is a serial number in the segment that is the subject of video authenticity, and is incremented in order of time. If the video has a frame rate of 30 fps, the resolution of the time code may be 1/30 second. . For example, in order to represent a time code for 1000 years with a resolution of 1/30 seconds, 40 bits are sufficient (30 frames × 60 seconds × 60 minutes × 24 hours × 365 days × 1000 years = 946,080). , 000,000 <2 ^40). Furthermore, lossless compression coding is performed to efficiently represent with a small number of bits. As the time code information including other IDs in the time code data amount, a data amount obtained by adding 20 to 30 bits is used.

  FIG. 3 shows an operation example of the system including the digital watermark embedding apparatus 301. A digital watermark is embedded in the video input from the camera 101 by the digital watermark embedding device 301. The digital watermark embedding device 301 is technically equivalent to a device in which the portable medium 105 is mounted on the PC 102 in FIG. 1 and connected to the time information source 104. Further, a video with a digital watermark is recorded by the recorder 103 and recorded on the video recording medium 304. When any alteration of the video recording medium 304 is suspected, the video recording medium 304 is reproduced by the playback device 302, the presence / absence of the alteration is verified by the digital watermark detection device 303, and the verification result is output. The digital watermark detection apparatus 303 detects tampering as will be described later with reference to FIGS. 9 and 10.

  FIG. 4 is an explanatory diagram showing an outline of tampering detection. Video alteration includes deletion, insertion, replacement, and copying. Insertion or replacement is detected based on the undetected state of the digital watermark including the time code embedded in the video. Further, even if the time code is detected, the consistency of the time code can be verified by collating with the reproduction time, so deletion or copying is detected. In this way, tampering added to the video is detected.

FIG. 5 is an explanatory diagram showing an outline of embedding a time code as a digital watermark. In the digital watermark, the same time code t is embedded in a frame set of d seconds (N frames). It may be every frame. In FIG. 5, the time code t ₁ at time 0 is embedded in N frames from time 0, and the time code t ₂ at time d is embedded in N frames from time d.

  FIG. 6 is an explanatory diagram regarding the synchronization of the embedded information and the recorder. As described above, appropriate time code information is embedded as a digital watermark in a plurality of frames constituting a video as time elapses. When the camera 101, the digital watermark embedding device 301, and the recorder 103 are connected, even if the power is turned on to these devices at the same time, proper synchronization cannot be achieved due to the difference in the startup time for each device.

  FIG. 6 shows a frame flow of an output video of the digital watermark embedding apparatus 301 by taking as an example a case where time code information corresponding to one time code is embedded as a digital watermark in one frame for easy understanding. One time code may be embedded in a plurality of frames. In the digital watermark embedding apparatus 301, the embedded information (time code information) at the head of the output video is Open as the segment position and the date and time. Thereafter, video segments are arranged every predetermined time. For each video segment, the segment position starts with Start, and segment numbers TC1, TC2,..., TCm (here, time code is used) as serial numbers in the segment are assigned as segment positions, and the last The end of one segment is indicated by End at the segment position. Subsequently, the next video segment is started. When the digital watermark embedding device 301 is stopped, the last embedding information of the output video is the segment position Close and the date and time.

  FIG. 6 illustrates a case where the start of the recorder 103 is delayed from the digital watermark embedding device 301. The time code as the segment number embedded in the first frame at the start of recording is TC3. At this time, it is impossible to distinguish whether the video recorded before time TC3 is normal or deleted as a recorded video. As will be described later, tampering is detected in units of segments that include both the segment positions Start and End. Tampering of the lead-out and lead-in areas and the frames outside them that do not include one of Start and End cannot be detected. Therefore, this problem is solved by mutual communication between the digital watermark embedding apparatus 301 and the recorder 103. The recorder 103 informs the digital watermark embedding device 301 of the time when recording can be started. The digital watermark embedding apparatus 301 changes the segment number at the time of TC3 at which recording can be started to the segment position Open, and outputs it. Specifically, an electronic watermark pattern corresponding to time code information whose segment position is Open and date / time is time code TC3 is selected and embedded in the video at the time point TC3. Thereby, a lead-in area does not occur. The same processing is performed for the lead-out area at the end of recording.

  A lead-in area and a lead-out area also exist between the camera 101 and the digital watermark embedding apparatus 301. The camera 101 and the digital watermark embedding apparatus 301 may communicate with each other so as to appropriately correspond to the lead-in area and the lead-out area.

  FIG. 7 is a PAD (Problem Analysis Diagram) of the superimposition processing unit 126 that embeds time code information as a digital watermark. The embedding of time code information as a digital watermark means that a watermark pattern corresponding to the time code information is searched from the watermark pattern DB 121 and the searched watermark pattern is superimposed on a corresponding video frame as a digital watermark. In step 701, the system is initialized. In step 702, the captured video frame is stored in a buffer. When the camera power is turned off, the end of the video signal may suddenly come without prior notice, and even after the video signal ends, it is possible to properly verify the authenticity by processing the image in the buffer. .

  In step 703, the video start time information is set as time code information. The current time is acquired as the video start time information, and Open is set as the segment position. Specifically, the current time is the time code, and Open is the time code information with the segment position.

  In step 704, a new video frame is taken into the buffer, the watermark pattern DB 121 is searched using the time code information set in step 703 as a key, and the searched watermark pattern is superimposed on the video frame stored in the buffer as a digital watermark. And output. This process is repeated N times. Details of the N frame embedding process in step 704 will be described later with reference to FIG. When executing the N frame embedding process, the value of N and the time code information set in step 703 are passed as parameters.

  In step 705, the processing from step 706 to step 712 is repeated until the input video is interrupted and there are no captured video frames.

  In step 706, the input video is divided into segments at predetermined time intervals, and the processing in steps 707 to 712 is repeated for each segment.

  In step 707, an i (i = 0,..., M + 1) -th embedded window indicating a certain segment is set. Here, it is assumed that one segment has N video frames.

  An embedding window is a unit of embedding operation set for a frame when the same information is repeatedly embedded in one or a plurality of continuous frames using a digital watermark. One digital watermark embedding information is set for each embedding window.

  In this embodiment, one segment and one embedded window are technically equivalent. An embedding window is set for a unit of N consecutive frames constituting the embedding window. The size of the embedded window at this time is N. The embedding operation for each frame within one segment is performed N times.

  In step 708, when i = 0 (first embedded window), in step 709, Start is set as the segment position of the embedded window. In step 708, when i = m + 1 (last embedded window), in step 710, End is set as the segment position of the embedded window. In both cases of Step 709 and Step 710, the current time time is acquired. Specifically, Step 709 and Step 710 use time code information with the current time as the time code and Start or End as the segment position. In step 708, otherwise, in step 711, i × N is set as the segment number of the embedded window. Specifically, it is time code information with i × N as a segment number.

  In step 712, while sequentially capturing N frames of video frames 123, time code information is embedded in the video frames stored in the buffer and output as video frames 124 embedded with time code information. Specifically, in step 712, the watermark pattern DB 121 is searched using the time code information set in any of steps 709 to 711 as a key, and the searched watermark pattern is converted into a video frame stored in the buffer as a digital watermark. Superimpose and output. This N frame embedding process is repeated N times. The parameters for the N frame embedding process are as described above. However, if an interrupt process occurs in response to a user instruction or the end of the input video 109, the process is interrupted and the process proceeds to step 713.

  In step 713, video end time information is set as time code information. The current time is acquired as the video end time information, and Close is set as the segment position. Specifically, the current time is the time code, and the close is the time code information with the segment position.

  In step 714, the time code information is embedded in the video frames stored in the buffer for N frames, and the output video frame is written. Specifically, the watermark pattern DB 121 is searched using the set time code information as a key, and the searched watermark pattern is superimposed on a video frame stored in a buffer as a digital watermark and output. This N frame embedding process is repeated N times. However, capturing of the input video frame is stopped. If necessary, communication for synchronization with the camera and the recorder may be performed.

  FIG. 8 is a PAD of N frame embedding processing which is a central processing portion of the time code embedding method. In step 801, authentication processing for DB access is performed based on the key 106 and certificate 107 read by the key setting processing unit 115.

  In step 802, the watermark pattern DB 121 is searched using the time code information set and passed as a parameter as a key to obtain a watermark pattern or an encrypted watermark pattern. In step 803, if the watermark pattern is encrypted, it is decrypted using the key 106.

  In step 804, N frames are repeatedly processed according to the value of N passed as a parameter. A variable k for confirming the number of repetitions is set, and k is sequentially incremented from 1 to N every time the process is completed.

  In step 805, the presence / absence of an input video is checked. If the input video has ended, an end flag is set in step 806, and the current process is exited. In step 805, if the video signal continues, in step 807, the captured video frame is stored in the buffer.

  In step 808, the (i × N + k) th image frame is read from the buffer. In step 809, the consistency between the watermark pattern obtained by searching the watermark pattern DB 121 and the image frame is confirmed. For example, when both sizes are different, a process of repeatedly writing a watermark pattern is performed. In step 810, the image frame read from the buffer is analyzed to determine the embedding strength. For this analysis, for example, the embeddability analysis process of Patent Document 2 may be used.

  In step 811, a watermark embedded image is generated by superimposing the watermark pattern on the image frame.

  In step 812, the watermark embedded image is stored in the output video frame buffer. If N frames are processed according to step 804, the process ends.

  In this embodiment, in step 702, the video frame is sufficiently buffered. In steps 807 and 808, a queue operation, that is, a first-in first-out operation is performed on the buffer, and the subsequent processing is performed on the obtained frame data. For this reason, even after the input video is interrupted, a state in which some frames remain in the buffer is maintained. If the buffer size is set to a large value (minimum value 2N), at the time of execution of step 714, there are N or more frames in the buffer that date back from the time when the video was interrupted.

  In order to complete the process in step 707 in units of N frames, when the end flag set in step 806 is set, the process repeats the process in step 707 and proceeds to the process in step 713.

  FIG. 9 is a PAD for video content alteration detection processing using time code information embedded as a digital watermark.

In step 901, variables on the memory used in this processing are initialized.
TCcur = TCpre = nD = 0
AttackType [0] = “No Attack”, i = 0
In step 902, the digital watermark of the beginning video frame is detected, and it is confirmed that the segment position of the time code information is Open.

  In step 903, while the input of the video frame continues, the repetitive processing is controlled. However, the video is composed of a plurality of segments separated by a predetermined length.

  In step 904, the segment start condition is verified. That is, digital watermark detection is performed, and it is confirmed that the segment position of the time code information is Start.

  In step 905, the verification process inside the segment is controlled repeatedly. However, one segment does not have a one-to-one correspondence with the detection window.

  A detection window is a unit of operation for performing digital watermark detection using one or a plurality of continuous frames. If the detection is successful, one piece of digital watermark embedding information is detected for each detection window.

  In the present embodiment, when the size of the embedded window is N, the size of the detection window is N / 2.

  After image processing such as compression encoding, the embedded window and the detection window may be out of synchronization. For this reason, in the video data used at the time of detection, where the head frame of a certain segment (or embedded window) is located may not be explicitly communicated to the detector. If the inspector obtains the information and the embedded window and the detection window are completely synchronized, two detection windows are included for one segment. Even when synchronization cannot be achieved, at least one detection window is included for one segment.

  In step 906, the i-th detection window is set, and the iterative process is controlled for (N / 2) video frames within the detection window range.

  If there is no video frame input in step 907, an end flag is set in step 908 to exit the process. If there is an input of a video frame in step 907, a subsequent image is captured in step 909. In step 910, the captured image is superimposed.

  In step 911, a digital watermark is detected from the superimposed data of (N / 2) video frames in the i-th detection window.

  In step 912, the presence or absence of video alteration is diagnosed using the detection result. This diagnosis will be described later with reference to FIG.

  In step 913, the end condition of the segment is verified. That is, a digital watermark is detected, and it is confirmed that the segment position of the time code information is End.

  In step 914, the digital watermark of the video frame at the video end point is detected, and it is confirmed that the segment position of the time code information is Close. If the recorder 103 is stopped before the digital watermark embedding device 301, there is a case where the end or the close cannot be determined. In this case, the above-described way of thinking about the lead-out time may be applied.

  FIG. 10 is a PAD of a detection result diagnosis process that identifies the type of tampering action in the process of detecting tampering of video. This is processing for the i-th detection window in FIG.

  In step 1001, it is determined whether digital watermark can be detected. If not detected, the process proceeds to step 1002. If it is detected in step 1001, the process proceeds to step 1006.

  In step 1002, the value nD of the digital watermark continuous undetected number is incremented.

  In step 1003, ΔTC = TCcur−TCpre is calculated. If ΔTC is zero or 1, and nD is 1, the process proceeds to step 1004. In step 1004, the attack content AttackType [i] for the current detection window is set to “No Attack”. If it is determined in step 1003 that the above condition is not satisfied, the process proceeds to step 1005. In step 1005, the attack content AttackType [i] for the current detection window is set to “Unconfirmed”. That is, the decision is put on hold.

  In step 1006, TCcur is stored in TCpre, and the segment number of the time code detected as a digital watermark is stored in TCcur. Step 1006 is processing subsequent to the determination in step 1001 that the digital watermark has been detected.

  If it is determined in step 1007 that the previous detection window determination result AttackType [i−1] is “Unconfirmed”, the process proceeds to step 1008. Otherwise, go to step 1009.

  In step 1008, based on the detection result of the i-th detection window, the part of the diagnosis result AttackType [1] to AttackType [i-1] that has been provisionally determined as “Unconfirmed” in the previous detection window is deleted. Therefore, it is determined that the insertion has been tampered with and replaced with “Attached”.

  In step 1009, ΔTC = TCcur−TCpre is calculated. Further, when ΔTC is zero or 1, and nD is 0 or 1, the attack content AttackType [i] for the current detection window is set to “No Attack”. Alternatively, when this condition is not satisfied, AttackType [i] is set to “Attached”.

  In step 1010, the value nD of the digital watermark continuous undetected number is returned to zero.

  FIG. 11 is an explanatory diagram of the field structure of the watermark pattern DB 117.

  The watermark pattern DB 117 includes three tables, and includes a segment number table 1101, a segment position table 1102, and a date / time information table 1103. The segment number table 1101 registers and searches the watermark pattern data 1105 using the segment number 1104 as a key. The segment position table 1102 registers and searches the watermark pattern data 1107 using the segment position 1106 as a key. The date / time information table 1103 registers and searches the watermark pattern data 1111 using the date / time information (date 1108, time 1109, frame 1110) as a key. The time code information is provided with these three keys. Furthermore, DB management may be performed by combining camera IDs, user IDs, device IDs, and the like. Instead of dividing the table, the integrated watermark pattern data may be held using the integrated time code information as a key.

  The watermark pattern is obtained by uniquely losslessly encoding key information. For example, a “digital watermark pattern” described in Patent Document 2 may be used.

  FIG. 12 is a PAD of processing for generating the watermark pattern DB 117.

  In step 1201, authentication using the key 106 and the certificate 107 is performed. This authentication includes device authentication, product authentication, and user authentication.

  In step 1202, control is repeatedly performed on time code information set up in advance by the user until the time when the input of the video frame is assumed to be completed.

  In step 1203, the time code information is encoded. In step 1204, the encoded information is converted into a watermark pattern basic form as a two-dimensional shape. The basic form of a watermark pattern is usually smaller than the width x height of a video frame. If it is large, it may be divided and stored in a plurality of frames.

  In step 1205, the repetition processing is controlled for the entire video frame of the input video.

  In step 1206, the basic shape of the watermark pattern is repeatedly arranged and converted to the size of the width × height of the image. Even if it does not convert here, you may convert when superimposing later.

  In step 1207, the watermark pattern is encrypted. It does not have to be encrypted. Since digital watermarking is a technology that ensures security by concealing algorithms, encryption can eliminate the risk of exposing watermark pattern information to the eyes of an attacker.

  In step 1208, the watermark pattern or the encrypted watermark pattern is registered in the watermark pattern DB 117 using the time code information as a key. Returning to step 1202, if the generation of the watermark pattern corresponding to the predetermined time code information has been completed, the processing is terminated. Otherwise, the above process is repeated.

  In step 1202, the time code information can be easily predicted as a future time without being preset by the user. In the case of GPS information as information to be embedded, it is position information. The current GPS positioning system is about 10 m, and it is possible to comprehensively calculate all positions on the earth in advance. Even latitude and longitude representations with 10 m accuracy can be represented with about 54 bits. This amount of information is not a problem as the amount of information embedded as a digital watermark.

  A second embodiment will be described. FIG. 13 is a configuration diagram of a digital watermark embedding device that realizes digital watermark embedding in the second embodiment.

  As in the first embodiment shown in FIG. 1, the camera 101, the PC 102, and the recorder 103 are connected by a video signal line and connected to the time information source 104 via a network. The main difference from the first embodiment is that in addition to the key 1307 and the certificate 1309, processing equivalent to the watermark pattern construction processing 114 and the disk management processing 113 (time setting processing 1302, watermark pattern generation processing 1303, timer 1304, watermark pattern DB 1305, access control 1306, verification 1308) are mounted on a portable medium 1301 including an arithmetic device such as an IC card, and access control for watermark pattern data is more safely implemented on the PC 102. It is that. Unlike the timer 108 of the PC 102, the timer 1304 is mounted on the portable medium 1301, and thus is not easily attacked by time tampering or the like. Furthermore, a key 1311 and a certificate 1310 are also mounted on the PC 102 side, and system control by an unauthorized portable medium can be rejected by the authentication unit 1312. The portable medium 1301 is detachable from the PC 102, but may be fixed to a memory or a disk in the PC 102.

  A third embodiment will be described. FIG. 14 is a configuration diagram of a digital watermark embedding apparatus that realizes digital watermark embedding in the third embodiment.

  The main difference from the first embodiment is that a parallel processing technique is introduced via the pattern buffer (I) 1407 and the pattern buffer (II) 1408. As parallel processing, process A is executed on CPU_A 1401 and process B is executed on CPU_B 1402.

  In the process A, the following three processes (1), (2) and (3) are performed, and the overall control A 1406 manages the whole. (1) By the embedded information setting process 1403, change information such as a time code is set. (2) A watermark pattern corresponding to the embedded information is created by a watermark pattern generation process 1404. (3) The generated watermark pattern is stored in either the pattern buffer (I) 1407 or the pattern buffer (II) 1408 by the pattern buffer selection storage process 1405.

  In the process B, the following three processes (1), (2), and (3) are performed, and the overall control B 1409 manages the whole. (1) By the frame reading process 1409, the input video frame 123 is stored in the input buffer. (2) The embedding process 1410 superimposes either the pattern buffer (I) 1407 or the pattern buffer (II) 1408 and the image data of the input video frame 123 to generate an output video frame 124. (3) The output video frame 124 is stored in the output buffer by the frame writing process 1411.

  FIG. 15 is a timing chart for digital watermark embedding in the third embodiment.

  Process A on CPU_A 1401 creates a watermark pattern and stores it in one of pattern buffer (I) 1407 and pattern buffer (II) 1408. The process B on the CPU_B 1402 superimposes the pattern buffer data and the input video frame while synchronizing with the video input / output.

  In FIG. 15, the watermark pattern on the pattern buffer (I) 1407 is applied to the first frame and the second frame, and another watermark pattern generated between them is stored in the pattern buffer (II) 1408. Stored. In the next embedding process, the pattern buffer (II) 1408 is read and superimposed on the third frame.

Such parallel processing is suitable when the processing time of the watermark pattern generation processing 1404 is long and the process A and process B cannot be synchronized with the video when executed as a single process.
Each processing time is expressed as a variable as follows.
Processing time P (ms) of watermark pattern generation processing 1404,
Processing time of embedding process 1410 (superimposition process) Q (ms)
Processing time R (ms) of frame reading processing 1409
Processing time of frame writing process 1411 S (ms)
If the video frame rate is 30 fps, then R + Q + S ≦ 30 (ms) is a necessary condition for real-time processing. FIG. 15 shows an example in which there are two pattern buffers. However, if the processing time of the watermark pattern generation processing 1404 is longer, the number of pattern buffers (T) increases and more memory is stored. An area is required. The required number of pattern buffers has a relationship of T ≧ 1 + Quotent (P, (Q + S + R)). Quotient (a, b) indicates a quotient when a is divided by b.

  If one video frame is calculated, the number of pattern buffers required for real-time processing can be calculated. In the overall control B 1409, when synchronization with the video input / output cannot be achieved, a synchronization error signal is returned from the image input / output board. The overall control B 1409 and the overall control A 1406 measure the processing time PQRS, and increase the number of pattern buffers by obtaining the required pattern buffer number T. By this action, it is possible to synchronize with the real-time processing of video input / output. The pattern buffer is a buffer memory for storing a watermark pattern. When there are two (two sides), the pattern buffer operates as a replacement buffer. When there are three or more, it operates as a ring buffer.

  In the above embodiment, the time information (time code) is used as the change information that is sequentially changed and embedded in the digital watermark. For the change information that changes sequentially, the watermark pattern in which the change information is embedded can be changed, and a watermark pattern can be prepared in advance by using the regularity of the change. Therefore, the change information that changes sequentially and regularly is not limited to time information, but may be information that has regularity in change. For example, even if the position information is touched easily, the change may be given regularity. In the extreme, even if the numerical information is in ascending order or descending order, there should be regularity such that the difference between the numerical information is constant. In the case of using time information as the change information, it has been described that the input video frame and the current time are associated with each other. However, the change information associated with the input video frame only needs to be associated at a certain time. For example, 0 may be associated as change information with the first video frame at the start of video input, and numerical values in ascending order may be sequentially associated with the input video frames.

  Although the embodiments of the present invention have been specifically described above, the present invention is not limited to the above-described embodiments, and it goes without saying that various modifications can be made without departing from the scope of the invention.

101: Camera, 102: PC, 103: Recorder, 104: Time information source, 105: Portable medium, 106: Key, 107: Certificate, 108: Timer, 111: Video input / output board, 112: Digital watermark embedding Processing unit 113: Disk management processing unit 114: Watermark pattern construction processing unit 117: Watermark pattern DB 121: Watermark pattern DB 301: Digital watermark embedding device 303: Digital watermark detection device

Claims (10)

A digital input / output board that captures video, inputs as an image frame, outputs a video frame in which the digital watermark is embedded in the input image frame, and a memory that stores the digital watermark pattern as a watermark pattern An electronic watermark embedding method in an embedding device, comprising:
In response to change information that changes sequentially and regularly, the watermark pattern is generated, and the generated watermark pattern is stored in the memory,
The watermark pattern is searched using the change information corresponding to the input image frame from the video input / output board as a key,
An electronic watermark embedding method, wherein an image frame in which the searched watermark pattern is embedded as an electronic watermark in the input image frame is output to the video input / output board. 2. The digital watermark embedding method according to claim 1, wherein the change information that changes sequentially and regularly is time information, and a key for searching for the watermark pattern is the input input from the video input / output board. An electronic watermark embedding method comprising current time information for processing an image frame. 2. The digital watermark embedding method according to claim 1, wherein the change information is position information. 2. The digital watermark embedding method according to claim 1, wherein the memory for storing the watermark pattern is an on-memory DB. A video input / output board that captures video, inputs as an image frame, and outputs an image frame in which a digital watermark is embedded in the input image frame; a buffer memory that stores the digital watermark pattern as a watermark pattern; A digital watermark embedding method in a digital watermark embedding apparatus comprising a second CPU,
The first CPU generates the watermark pattern in response to change information that changes sequentially and regularly, stores the generated watermark pattern in the buffer memory,
The second CPU inputs the watermark pattern from the buffer memory in response to the input image frame input from the video input / output board,
An electronic watermark embedding method comprising: outputting an image frame in which the input watermark pattern is embedded in the input image frame as a digital watermark to the video input / output board. A video input / output board that captures video, inputs it as an image frame, and outputs an image frame in which a digital watermark is embedded in the input image frame;
A memory for storing the digital watermark pattern as a watermark pattern;
In response to change information that changes sequentially and regularly, a watermark pattern construction processing unit that generates the watermark pattern and stores the generated watermark pattern in the memory;
Using the change information corresponding to the input image frame from the video input / output board as a key, a watermark pattern calling processing unit for searching for the watermark pattern, and embedding the searched watermark pattern in the input image frame as a digital watermark, An electronic watermark embedding apparatus, comprising: a superimposition processing unit that outputs an image frame in which the searched watermark pattern is embedded as an electronic watermark to the video input / output board. 4. The digital watermark embedding apparatus according to claim 3, wherein the sequentially and regularly changing information is time information, and a key for searching for the watermark pattern is the input input from the video input / output board. An electronic watermark embedding apparatus comprising current time information for processing an image frame. 2. The digital watermark embedding apparatus according to claim 1, wherein the change information is position information. 2. The digital watermark embedding apparatus according to claim 1, wherein the memory for storing the watermark pattern is an on-memory DB. A video input / output board that captures video, inputs it as an image frame, and outputs an image frame in which a digital watermark is embedded in the input image frame;
A buffer memory for storing the digital watermark pattern as a watermark pattern;
The input image frame from the video input / output board and a first CPU that generates the watermark pattern corresponding to the change information that changes sequentially, and stores the generated watermark pattern in the buffer memory A second CPU that inputs the watermark pattern from the buffer memory and outputs an image frame in which the input watermark pattern is embedded in the input image frame as a digital watermark to the video input / output board. A digital watermark embedding apparatus comprising:

Images