WO2015028098A1 - Method and system for embedding information and authenticating a h.264 video using a digital watermark - Google Patents

Method and system for embedding information and authenticating a h.264 video using a digital watermark Download PDF

Info

Publication number
WO2015028098A1
WO2015028098A1 PCT/EP2013/068067 EP2013068067W WO2015028098A1 WO 2015028098 A1 WO2015028098 A1 WO 2015028098A1 EP 2013068067 W EP2013068067 W EP 2013068067W WO 2015028098 A1 WO2015028098 A1 WO 2015028098A1
Authority
WO
WIPO (PCT)
Prior art keywords
embedding
watermark
video
frames
macroblock
Prior art date
Application number
PCT/EP2013/068067
Other languages
French (fr)
Inventor
Luis PEREZ FREIRE
Gabriel DOMINGUEZ CONDE
David VAZQUEZ PADIN
Lukasz Zbigniew DZIANACH
Original Assignee
Centum Research & Technology S.L.U.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Centum Research & Technology S.L.U. filed Critical Centum Research & Technology S.L.U.
Priority to PCT/EP2013/068067 priority Critical patent/WO2015028098A1/en
Publication of WO2015028098A1 publication Critical patent/WO2015028098A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46Embedding additional information in the video signal during the compression process
    • H04N19/467Embedding additional information in the video signal during the compression process characterised by the embedded information being invisible, e.g. watermarking
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/0021Image watermarking
    • G06T1/0028Adaptive watermarking, e.g. Human Visual System [HVS]-based watermarking
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
    • H04N1/32101Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
    • H04N1/32144Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
    • H04N1/32101Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
    • H04N1/32144Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
    • H04N1/32149Methods relating to embedding, encoding, decoding, detection or retrieval operations
    • H04N1/32154Transform domain methods
    • H04N1/32165Transform domain methods using cosine transforms
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
    • H04N1/32101Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
    • H04N1/32144Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
    • H04N1/32149Methods relating to embedding, encoding, decoding, detection or retrieval operations
    • H04N1/32154Transform domain methods
    • H04N1/32187Transform domain methods with selective or adaptive application of the additional information, e.g. in selected frequency coefficients
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/18Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a set of transform coefficients
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/48Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2201/00General purpose image data processing
    • G06T2201/005Image watermarking
    • G06T2201/0053Embedding of the watermark in the coding stream, possibly without decoding; Embedding of the watermark in the compressed domain
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2201/00General purpose image data processing
    • G06T2201/005Image watermarking
    • G06T2201/0061Embedding of the watermark in each block of the image, e.g. segmented watermarking

Definitions

  • the present invention relates to the field of digital information protection, and more in particular to a scheme for embedding in a digital video a digital watermark containing information about the authenticity of the video and other metadata, and a scheme for retrieving such information from the video.
  • the digital video standard H.264 has become the most prevalent video compression standard, due to its versatility and compression capabilities, with application in many different fields like high compression video for mobile phone networks, high-definition television (HDTV) and Blu-Ray. Moreover, due to cost reductions, the number and variety of electronic devices with H.264 encoding capabilities is steadily increasing, ranging from mobile phones to modern video surveillance cameras, and even toys.
  • watermarking-based authentication techniques a low-power signal (watermark) is embedded into the digital content (the video, in this case) to be protected.
  • the authenticity of the received signal is determined by verifying the presence of the correct watermark with an appropriately designed watermark detector.
  • Modern video codecs do not treat the video as a sequence of independent pictures (frames), but they exploit the correlation between frames to provide a more efficient compression.
  • the H.264 standard in particular, obtains huge compression rates without degrading video quality at the expense of a computationally complex encoding process.
  • H.264 divides each video frame in one or more slices, and each slice in a collection of non-overlapping blocks of size 16x 16 pixels, called macroblocks, which are in turn divided in blocks of size 4x4.
  • the H.264 standard exploits both the spatial correlation between the adjacent blocks of each video frame, and the temporal correlation between blocks corresponding to contiguous frames. For every macroblock, a prediction based on a previously coded macroblock or set of macroblocks is computed.
  • Such prediction is called intra prediction when the spatial correlation is exploited, and inter prediction in the case of the temporal correlation.
  • two types of slices are defined in the H.264 standard: l-slices which contain only intra predicted macroblocks (I- macroblocks) and P-slices which contain intra and/or inter predicted macroblocks (P- macroblocks).
  • the slices are arranged together in sequences, forming Groups of Pictures (GOPs) where the first slice is an l-slice and the remaining ones (if any) are P- slices.
  • GOPs Groups of Pictures
  • Intra prediction consists in specifying how a given macroblock can be estimated from previously coded pixels that belong to neighboring macroblocks in the same slice. Thirteen different prediction modes are defined in the H.264 standard. On the other hand, inter prediction consists in specifying how to compute the pixels in a given macroblock using other macroblocks belonging to previously coded slices. In H.264, inter prediction is coded by means of Motion Vectors (MV). The result of computing the difference between a given block and its intra or inter prediction is called residue.
  • MV Motion Vectors
  • each macroblock is represented by three syntax elements: the transformed and quantized residue, the MVd, and the intra prediction mode. For those macroblocks belonging to P-slices whose residue and MVd is null, only the SKIP label is transmitted.
  • the final step of the H.264 encoding consists in entropy-coding the syntax elements.
  • Entropy coding is a lossless coding that provides a more compact representation of the final H.264 bitstream, which can be either a Context Adaptive Variable Length Coder (CAVLC) or a Context-based Adaptive Binary Arithmetic Coder (CABAC).
  • CAVLC Context Adaptive Variable Length Coder
  • CABAC Context-based Adaptive Binary Arithmetic Coder
  • watermarking-based video authentication faces a number of additional constraints related to the application scenarios and the kind of devices that must carry out the embedding of the watermarks in the video content. These constraints can be summarized as follows:
  • the authentication process must be provably secure, by means of cryptographic keys or any other mechanism which guarantees that authentication watermarks cannot be forged.
  • the amount of information (i.e. the number of bits) that can be embedded by means of the watermark must be high enough so as to guarantee, at least, room for embedding the authentication information.
  • the authentication method must be able to authenticate video at the frame level, i.e. it must be able to authenticate each frame independently of the others, or at least at a group of frames. Additionally, it is desirable that the authentication method is able to deal with different profiles of H.264, where the term profile, considered in the standard, means a particular combination of capabilities and configuration parameters of the H.264 codec. For instance, it is desirable that the authentication method is able to cope with I and P- slices, CABAC and CAVLC entropy-coded bitstreams, etc.
  • the methods for embedding watermarks in digital videos encoded with H.264 can be divided in two groups: those that perform embedding inside the H.264 encoder, and those that perform embedding in the bitstream.
  • the methods in the first group integrate the watermark embedding process in the H.264 encoding workflow by adding one or more elements to the H.264 encoder that modify one or more syntax elements as they are being encoded.
  • the main drawback of these methods is that they require modification of the internal architecture of H.264 encoders, which is hardly applicable in practical systems, as explained above.
  • One example of such methods is disclosed in the article by Profrock et al.
  • the authors present a method designed to work on a real-time scenario by embedding a watermark on a pseudorandomly selected coefficient of each macroblock of l-slices after entropically decoding a H.264 video.
  • the main drawbacks of this method are the lack of solution to deal with the spatial error drift generated by the modification of the reference blocks used in the intra prediction, and the impossibility of using this method in inter predicted frames, where inter prediction is a key feature of the H.264 standard (and of all video compression standards, in general).
  • the proposed idea is based on a property of the 4x4 Integer Cosine Transform that makes possible to maintain unaltered the border of the inverse-transform block in the spatial domain, by changing accordingly pairs of coefficients. Using this property and knowing the prediction modes of the blocks which are adjacent to the one used for embedding, the corresponding pair of coefficients is selected to embed the information without generating any error in the following predictions.
  • the main limitation of this method is that the knowledge of the prediction modes of future macroblocks is necessary to avoid the spatial distortion. Therefore, the proposed algorithm is not suitable for real-time, low-latency applications.
  • the watermark decoder identifies the watermarked skipped blocks, decodes the watermark determining the authenticity of the received video and restablishes the skipped blocks that were watermarked (thus, this technique is a reversible H.264 watermarking method).
  • this method is not suitable for real-time applications due to the necessary analysis of the whole H.264 video and there is not guarantee of having enough room for a watermark of a given length, as the available payload depends on the statistics of the specific video.
  • the present invention overcomes the problems previously identified for watermarking- based authentication of H.264 video.
  • the invention relates to a method for embedding, in a H.264 video, a digital watermark that contains information about the authenticity of the video and other metadata.
  • the invention is highly efficient, secure, and with imperceptible effect in the visual quality of the watermarked video.
  • the watermarked video is fully compliant with the H.264 standard.
  • a system for embedding a digital watermark in a H.264 video comprises a watermarking and H.264 video stream generation module with a watermark embedding module for embedding a watermark in non-entropy-coded H.264 syntax elements of the H.264 video, obtaining watermarked non-entropy-coded H.264 syntax elements.
  • the watermark embedding module comprises processing means configured for processing each frame / of the non-entropy-coded H.264 syntax elements to:
  • the candidate macroblock is a 4x4 intra coded macroblock;
  • the selected block of the candidate macroblock to convey the watermark belongs to a predefined set of candidate blocks and the intra prediction modes of its successive adjacent blocks do not take as input the value of the selected block;
  • the predefined set of candidate blocks in the first level conditions comprises the nine blocks contained within the first three rows and the first three columns of the candidate macroblock.
  • the set of conditions may also include second level conditions to be checked when the first level conditions are not met, said second level conditions being met when:
  • the candidate macroblock is a 4x4 intra coded macroblock
  • the selected block of the candidate macroblock to convey the watermark belongs to a set of candidate blocks which comprises the four blocks contained within the first two rows and the first two columns of the candidate macroblock, and the intra prediction modes are such that a modification in the selected block does not propagate any spatial error to any of the seven blocks located in the fourth row or in fourth column of the macroblock.
  • the set of conditions also includes forcing embedding conditions to be checked, said forcing embedding conditions being met when the total number of bits to embed in the corresponding frame j is equal to the remaining macroblocks of the frame j which have not yet being analyzed for candidate macroblock purposes.
  • the set of conditions preferably also includes checking whether the candidate macroblock is an inter predicted macroblock, and in that case selecting at least one block of said macroblock and at least one coefficient of said block.
  • the set of conditions may also include checking whether the candidate macroblock is a 16x16 intra predicted macroblock, and in that case selecting at least one block of said macroblock and at least one coefficient of said block.
  • the processing means are configured to pseudorandomly determine whether a given macroblock is a candidate macroblock for conveying the watermark by computing a pseudorandomly obtained outcome of a Bernoulli random variable using a parameter P previously updated using a pseudorandom number generator initialized with a seed that is function of the index of the current frame / and a secret key K.
  • the watermarking and H.264 video stream generation module may further comprise: an entropy coder unit for entropically coding the watermarked non-entropy- coded H.264 syntax elements, obtaining entropically coded watermarked H.264 syntax elements, and
  • a H.264 video stream generation unit for producing a watermarked H.264 video.
  • system further comprises an embedding information unit configured to obtain a sequence of bits as the watermark to be embedded.
  • the embedding information unit is preferably configured to obtain, for each frame j of the H.264 video stream, an information binary vector as a function of metadata information, an index of the current frame, data obtained from the watermarking and H.264 video stream generation module, and a security vector used to validate a set of frames.
  • the embedding information unit may also comprise: a metadata information coding unit for obtaining a metadata information vector for each frame y;
  • a security unit for generating the security vector as a function of a secret key K, by using the metadata information vectors of a set of frames comprising at least one frame, their corresponding indices, and data obtained from the watermarking and H.264 video stream generation module,
  • the embedding information unit for obtaining the information binary vector for each frame j.
  • the embedding information unit is preferably configured to divide the security vector, obtained for each set of frames, into N vectors, where N denotes the number of frames which compose each set of frames,
  • the information binary vector corresponding to the frame j of the set of frames i+ 1 comprises:
  • the security vector (digest, ,/ ), where / is the corresponding index for the /-th chunk of the digestj Oi the previous set of frames / ' .
  • the data obtained from the watermarking and H.264 video stream generation module preferably comprises syntax elements of the H.264 video stream.
  • the metadata information preferably comprises at least one of the following data:
  • the embedding information unit is preferably configured to generate an information binary vector b used as watermark, such that the value of the coefficient with index / ' selected to convey a bit c- is modified by the watermark embedding module, allowing to retrieve the bit information b- at the decoder by processing the value of the watermarked coefficient c i as h
  • a system for generating a digital watermark for embedding in a H.264 video comprises an embedding information unit configured to obtain, for each frame y of the non-entropy-coded H.264 syntax elements of the H.264 video, an information binary vector as the watermark to be embedded, said vector obtained as a function of metadata information, an index of the current frame, syntax elements of the previous frames of the H.264 video stream, and a security vector used to validate a set of frames.
  • the embedding information unit preferably comprises:
  • a metadata information coding unit for obtaining a metadata information vector for each frame y;
  • a security unit for generating the security vector as a function of a secret key K, by using the metadata information vectors of a set of frames comprising at least one frame, their corresponding indices, and syntax elements of the previous frames of the H.264 video stream,
  • the embedding information unit for obtaining the information binary vector for each frame j.
  • the embedding information unit is preferably configured to divide the security vector, obtained for each set of frames, into N vectors, where N denotes the number of frames which compose each set of frames,
  • the information binary vector corresponding to the frame j of the set of frames i+ 1 comprises: the metadata information vector for said frame /,
  • the security vector (digest, ,/ ), where / is the corresponding index for the current frame / of the security vector (digestj) of the previous set of frames / (SF).
  • the metadata information may comprise at least one of the following data:
  • a system for decoding a digital watermark in a H.264 video comprising a watermark and H.264 video stream decoding module (500) with a watermark decoder module (510) for decoding, from each frame j of the watermarked non-entropy-coded H.264 syntax elements (H.264_R W _ SE ) of the H.264 video (H.264_R W VIDEO), an information binary vector b ⁇ which contains a metadata information vector m 1 , the index of the current frame / and a security vector ⁇ r_diges , where / is the corresponding index for the /-th chunk of the r digestj of the previous set of frames / ' (SF,), used to validate said previous set of frames (SF,).
  • a system for authenticating a H.264 video using a digital watermark comprising:
  • a watermark and H.264 video stream decoding module with a watermark decoder module for decoding, from each frame j of the watermarked non-entropy- coded H.264 syntax elements of the H.264 video, an information binary vector which contains a metadata information vector, the index of the current frame j and a security vector (r_digestji), where / is the corresponding index for the /-th chunk of the r digestj of the previous set of frames / ' , used to validate said previous set of frames;
  • a decoding information unit configured to:
  • c_digesti a calculated security vector as a function of a secret key K and using the metadata information vectors m 1 of a set of frames (SF,) comprising at least one frame, their corresponding indices (INDEX), and data (INF0 50 o) obtained from the watermark and H.264 video stream decoding module (500);
  • the watermark decoder module is preferably configured to extract the information binary vector using a secret key K and the index of the current frame.
  • a method for embedding a digital watermark in a H.264 video comprises: embedding a watermark in non-entropy-coded H.264 syntax elements of the H.264 video, obtaining watermarked non-entropy-coded H.264 syntax elements;
  • the candidate macroblock is a 4x4 intra coded macroblock (14x4); the selected block of the candidate macroblock to convey the watermark belongs to a predefined set of candidate blocks and the intra prediction modes of its successive adjacent blocks do not take as input the value of the selected block;
  • the predefined set of candidate blocks in the first level conditions comprises the nine blocks contained within the first three rows and the first three columns of the candidate macroblock
  • the set of conditions also may include second level conditions to be checked when the first level conditions are not met, said second level conditions being met when: the candidate macroblock is a 4x4 intra coded macroblock;
  • the selected block of the candidate macroblock to convey the watermark belongs to a set of candidate blocks which comprises the four blocks contained within the first two rows and the first two columns of the candidate macroblock, and the intra prediction modes are such that a modification in the selected block does not propagate any spatial error to any of the seven blocks located in the fourth row or in fourth column of the macroblock.
  • the set of conditions preferably also includes forcing embedding conditions to be checked, said forcing embedding conditions being met when the total number of bits to embed in the corresponding frame j is equal to the remaining macroblocks of the frame j which have not yet being analyzed for candidate macroblock purposes.
  • the set of conditions may also include checking whether the candidate macroblock is an inter predicted macroblock, and in that case selecting at least one block of said macroblock and at least one coefficient of said block.
  • the set of conditions may also include checking whether the candidate macroblock is a 16x16 intra predicted macroblock, and in that case selecting at least one block of said macroblock and at least one coefficient of said block.
  • the method may further comprise pseudorandomly determining whether a given macroblock is a candidate macroblock for conveying the watermark by computing a pseudorandomly obtained outcome of a Bernoulli random variable using a parameter P previously updated using a pseudorandom number generator initialized with a seed that is function of the index of the current frame / and a secret key K.
  • the method preferably comprises obtaining a sequence of bits as the watermark to be embedded.
  • the method may further comprise obtaining, for each frame j of the H.264 video stream, an information binary vector as a function of metadata information, an index of the current frame, data previously obtained, and a security vector used to validate a set of frames.
  • the method preferably comprises dividing the security vector, obtained for each set of frames, into N vectors, where N denotes the number of frames which compose each set of frames,
  • the information binary vector corresponding to the frame j of the set of frames i+ 1 comprises:
  • the security vector (digest, ,/ ), where / is the corresponding index for the /-th chunk of the digestj Oi the previous set of frames / ' .
  • the data previously obtained preferably comprises syntax elements of the H.264 video stream.
  • the metadata information may comprise at least one of the following data:
  • a method for generating a digital watermark for embedding in a H.264 video comprises obtaining, for each frame / of the non-entropy-coded H.264 syntax elements of the H.264 video, an information binary vector as the watermark to be embedded, said vector obtained as a function of metadata information, an index of the current frame, syntax elements of the previous frames of the H.264 video stream, and a security vector used to validate a set of frames (SF,).
  • the method may further comprise:
  • the method may also comprise dividing the security vector, obtained for each set of frames, into N vectors, where N denotes the number of frames which compose each set of frames, and wherein the information binary vector corresponding to the frame of the set of frames i+ 1 comprises:
  • the metadata information may comprise at least one of the following data:
  • a method for decoding a digital watermark in a H.264 video comprises decoding, from each frame / of the watermarked non-entropy-coded H.264 syntax elements (H.264_R W _ SE ) of the H.264 video (H.264_R W VIDEO), an information binary vector b ⁇ which contains a metadata information vector m 1 , the index of the current frame / and a security vector (r_digestji), where / is the corresponding index for the /-th chunk of the r digestj Oi the previous set of frames / (SF,), used to validate said previous set of frames (SF,).
  • a method for authenticating a H.264 video using a digital watermark comprises:
  • c_digesti a calculated security vector as a function of a secret key K and using the metadata information vectors of a set of frames comprising at least one frame, their corresponding indices, and data previously obtained;
  • the method may also comprise extracting the information binary vector using a secret key K and the index of the current frame.
  • a smart mechanism based on checking three conditions, for embedding the watermark in the macroblocks, guaranteeing a given payload per frame and a low embedding distortion.
  • the present invention is suitable to authentication of streamed video, and to other scenarios with strict real-time requirements.
  • the present invention does not require high memory or high computing power. Thus, it is amenable to integration in cheap electronic devices.
  • the embedding of the watermark with the present invention can be performed either during or after the encoding process, requiring a small amount of changes in the encoding workflow.
  • the present invention comprises mechanisms for adapting the security level of the authentication watermark to the application scenario of interest. It can provide cryptographic security, compliant with generally accepted security standards for electronic devices, if necessary.
  • FIG. 1 depicts a schematic block diagram of the elements of the system according to the present invention.
  • FIG. 2 shows the internal block diagram of the watermarking and H .264 video stream generation module.
  • FIG. 3 shows the internal scheme of the embedding information unit.
  • FIG. 4 depicts a basic diagram of the processing of a set of four frames.
  • FIG. 5 represents the flow chart of the watermark embedding module.
  • FIG. 6 shows an example of the neighbouring partitions that are considered for the computation of the prediction of a Motion Vector (MV).
  • MV Motion Vector
  • FIG. 7 shows the flow chart of the conditions that must be checked to watermark a candidate macroblock.
  • FIG. 8 illustrates a 1 6x 1 6 macroblock partitioned in sixteen 4x4 blocks.
  • FIG. 9 illustrates the ordering of the coefficients in a 4x4 block.
  • FIG. 1 0 illustrates the surrounding pixels used for the computation of the intra prediction of a 4x4 block.
  • FIG. 1 1 illustrates the surrounding pixels used for the computation of the intra prediction of a 1 6x 1 6 block.
  • FIG. 1 2 shows the internal scheme of the watermarking and H .264 video stream decoding module.
  • FIG. 1 3 shows the internal scheme of the decoding information unit.
  • FIG. 14 represents the flow chart of the watermark decoding module.
  • FIG. 15 depicts a schematic block diagram of the elements representing a possible implementation of the described invention integrated in a electronic device.
  • FIG. 16 illustrates a possible implementation of the described extraction method in a computational system for the purpose of exposing forgeries.
  • FIG. 1 depicts a schematic block diagram of the elements of the watermark embedding system and of the watermark extracting system according to the present invention.
  • non-entropy-coded H.264 syntax elements are subjected to techniques of watermarking a digital content in order to convey information, obtaining watermarked non-entropy-coded H.264 syntax elements, which are subsequently coded obtaining a watermarked H.264 video. Later, the conveyed information can be extracted from the watermarked H.264 video, being the authenticity and the sequentiality of the watermarked H.264 video and the conveyed information verified.
  • a system and method for watermarking H.264 videos there is provided a system and method for watermarking H.264 videos.
  • a watermark extracting system and method for authenticating H.264 videos comprises a watermarking and H.264 video stream generation module 100 and an embedding information unit 300, whereas the watermark extracting system comprises a watermark and H.264 video stream decoding module 500 and a decoding information unit 700.
  • H.264 SE non-entropy-coded H.264 syntax elements
  • the watermarked H.264 video can be saved as a file in a certain storage medium (hard disk, memory card, etc.) or transmitted through a digital network 800 (Internet, Local Area Network, etc.).
  • the information is coded as an information binary vector b that is embedded by the watermark embedding method according to the present invention.
  • the information binary vector b is computed in the embedding information unit 300 as a function of metadata information, an index of the current frame, data (INFO 100 ) obtained from the watermarking and H.264 video stream generation module 100, and a security vector (e.g., digital signature, etc.) used to validate the information binary vector b .
  • the embedding information unit 300 reads and process data and sends the information binary vector b to the watermarking and H.264 video stream generation module 100.
  • the watermarked H.264 video (H.264 w VIDEO) is retrieved from the digital network 800, received watermarked H.264 video (H.264_R W VIDEO), by a watermark and H.264 video stream decoding module 500 that implements the watermark decoding method according to the present invention.
  • the watermark and H.264 video stream decoding module 500 can be configured to generate an output H.264 video stream showing the results obtained after decoding and processing the watermark.
  • a decoding information unit 700 reads the information binary vector b (the symbol ⁇ denotes "estimate of"; thus, b denotes the estimated information binary vector b obtained in the decoding process, which may differ from the information binary vector b generated by the embedding information unit 300 in the coding process) from the watermark and H.264 video stream decoding module 500, which generates output information, and determines the authenticity (AUTHENTICITY) of the output information, and decides on the integrity of the watermarked H.264 video.
  • the minimum information generated by the watermark and H.264 video stream decoding module 500 as output is the decision on the authenticity of the analyzed video.
  • this module 500 can also output the metadata extracted from the analyzed video (e.g, GPS coordinates, embedding time, etc.) and it also can be configured to generate a video indicating the decision on its authenticity and the extracted information. For example, if the system detects that the received video was temporally modified (i.e., changing the order of the frames) the system can show a visual signal in the output video.
  • the metadata extracted from the analyzed video e.g. GPS coordinates, embedding time, etc.
  • this module 500 can also output the metadata extracted from the analyzed video (e.g, GPS coordinates, embedding time, etc.) and it also can be configured to generate a video indicating the decision on its authenticity and the extracted information. For example, if the system detects that the received video was temporally modified (i.e., changing the order of the frames) the system can show a visual signal in the output video.
  • FIG. 2 shows the internal scheme of the watermarking and H.264 video stream generation module 100, which comprises a watermark embedding module 110, an entropy coder unit 270, and a H.264 video stream generation unit 290.
  • H.264 S E non-entropy-coded H.264 syntax elements
  • the watermark embedding module 1 10 embeds the watermark using the information binary vector b obtained from the embedding information unit 300.
  • the watermarked non- entropy-coded H.264 syntax elements (H.264 w _SE) are entropically coded in the entropy coder unit 270.
  • the resulting entropically coded watermarked H.264 syntax elements (H.264 cw SE) pass to the H.264 video stream generation unit 290 that produces a watermarked H.264 video (H.264 w VIDEO), which is H.264 compliant.
  • the watermark embedding module 1 10 embeds the watermark (i.e. the information binary vector b ) into the H.264 video sequence using a secret key (K) and the index of the current frame (INDEX), in order to make the watermark embedding secure. Without using said secret key and the correct index, the generation of a valid watermarked H.264 video is a highly complex problem, from the computational point of view, thus making the system secure.
  • the embedding information unit 300 can read different kind of information (INF0 10 o) from the watermarking and H.264 video stream generation module 100: the output of the information of the watermark embedding module 1 1 0, the output of the entropy coder unit 270, and the output of the H.264 video stream generation unit 290.
  • FIG. 3 shows the internal scheme of the embedding information unit 300, which comprises a metadata information coding unit 330, a register of the index of the current frame 31 0, a security unit 350, and an embedding information control unit 370.
  • Metadata information (METADATA) to be embedded is received from the watermarking and H.264 video stream generation module 100.
  • a metadata information vector m e.g., GPS position coordinates, an identifier of the embedding device, etc.
  • This metadata information vector m can be updated for each frame or can be fixed during the H.264 video processing.
  • the information that the embedding information control unit 370 passes to the watermarking and H.264 video stream generation module 100 is previously computed, using a set of frames as a basic unit to be validated.
  • the security unit 350 is fed with the metadata information vector to be embedded (which is the output of the metadata information coding unit 330), the index of the current frame (INDEX) from the register of the index of the current frame 310, and information (INFOi 00 ) from the watermarking and H.264 video stream generation module 100, to generate as a function of the secret key K a digest whereby the authenticity of the information can be verified.
  • the digest generated by the security unit 350 is stored together with the metadata information, and the index of the current frame in the embedding information control unit 370 as an information binary vector.
  • the information (INFO 100 ) read from the watermarking and H.264 video stream generation module 100 is selected according to a semantic criterion, i.e. it is relevant from the visual point of view.
  • the syntax elements of the H.264 bitstream are selected: motion vectors, intra prediction modes and residues of the macroblocks. As it is schematically depicted in FIG.
  • the security unit 350 needs to use the information of the previous watermarked set of frames, i.e., it needs to access the output of the modules 1 10, 270 and 290.
  • the signal INFO 1 00 represents the output of all those modules.
  • the obtained digest ( digest j) is divided into N digests ⁇ digest, ⁇ , digest ⁇ , digest /,/ v ⁇ , where N denotes the number of frames which compose each set of frames (four frames, in the example of FIG. 4).
  • the information binary vector corresponding to the y-th frame of the /+1 -th set of frames (SF i+ i) is constructed by arranging in a vector b j the following data: digest, , , (where / is the corresponding index for the current j- frame of the digests of the previous set of frames, /-th set of frames SF,), the metadata m 1 of the current frame y, and the index of the current frame, i.e., j.
  • the embedding information control unit 370 resets the security unit 350 to an initial state after processing each set of N frames.
  • FIG. 5 represents the flow chart of the watermark embedding module 1 10, wherein each frame of the non-entropy-coded H.264 syntax elements (H.264 S E) is processed.
  • the first step to embed the watermark (information binary vector b 1 ) into the current frame j is to pseudorandomly permute as a function of a secret key K Q the information binary vector b 1 obtained from the embedding information unit 300 in order to uniformly spread the information over the current frame / (step 1 1 1 ) , although this first step is optional.
  • a parameter P between 0 and 1 is updated in the step 1 13, which is used to pseudorandomly determine, as a function of the secret key K, whether a given macroblock is used for embedding information.
  • the value of the number of remaining coefficients to be embedded is used to initialize R_COEFFS_TO_EMBED (step 1 15), then the value of the total number of bits to embed is used to initialize N_TOTAL_BITS_TO_EMBED (step 1 17) .
  • the value R CO E F FS_TO_E M B E D is used to measure the number of coefficients that can be used to embed the watermark in the worst case scenario. In our application scenario, this value is obtained by assuming that the remaining macroblocks are all 16x16 intra-coded and only one coefficient per macroblock can be used to convey the watermark. Accordingly, we initialize R_COEFFS_TO_EMBED as (width in pixels x height in pixels)/(16 x 16).
  • variable FORCING is initialized to zero (step 1 19).
  • step 121 the comparison between R_COEFFS_TO_EMBED and N TOT AL B I TS_TO_E M B E D is computed; if they have the same value, FORCING is set to one (step 123) .
  • the decision of a current macroblock as a candidate macroblock to convey watermark information is done in step 125 by computing an OR function between a pseudorandomly obtained outcome of a Bernoulli random variable using the parameter P updated in step 1 13 and FORCING.
  • step 125 the current macroblock is selected as a candidate macroblock (MB) to be watermarked (step 127), a sequence of bits (in this particular embodiment one bit per macroblock is used; however, note that the present algorithm can be configured to use more bits per macroblock) of the information binary vector b j may be embedded in the candidate macroblock and
  • N_TOTAL_BITS_TO_EMBED is updated by subtracting from its previous value the number of embedded bits (step 129).
  • R_COEFFS_TO_EMBED is updated by subtracting one to its value (step 131 ).
  • the flow of the algorithm goes to step 135 if the value of N_TOTAL_BITS_TO_EMBED is zero (step 133); otherwise, the algorithm goes to step 119.
  • the algorithm After embedding the watermark (i.e. the information binary vector b 1 ) into the current j- th frame, the algorithm checks in step 135 whether there are SKIP macroblocks that were watermarked in the current frame. If the answer is positive, the embedding algorithm modifies the necessary syntax elements of the set of non-entropy-coded H.264 syntax elements in step 137, in such a way that the skipped macroblocks are converted into new P-macroblocks composed of a single partition, the associated residue macroblock (where the watermark has been embedded), and a corresponding Motion Vector difference (MVd). Said MVd is obtained by computing the difference between the actual motion vector (MV) and a prediction of this, called Motion Vector prediction (MVp). Note that for the skipped macroblock no data is coded other than an indication that the macroblock is to be decoded as "skipped", so the associated MV is directly the MVp.
  • MVp Motion Vector prediction
  • FIG. 6 shows an example of the neighbouring partitions (i.e., A, B and C) that are considered for the calculation of the MVp either for a skipped macroblock or a P- macroblock.
  • the calculation of the MVp differs for both types of macroblock when some of the neighbouring partitions are not available. For instance, if the partition A is not available, then the MVp (and thus the MV) associated to the skipped macroblock would be inmediately equal to zero independtly of the partitions B and C, while for the P-macroblock the MVp would be calculated in a different way, taking into consideration the other partitions.
  • the motion vector difference of the new P-macroblock can be used as a bias to compensate possible misalignments between the two motion vectors, in order to obtain the same resulting MV after the conversion. Otherwise, the different reconstruction of the macroblock in the pixel domain, would generate visible distortions that would propagate along the image. For this reason, the MVd associated to each new P-macroblock must be accordingly updated in step 139. Due to the modifications of the non-entropy-coded H.264 syntax elements of the input, the coded block pattern (CBP) is updated by taking into account the introduced modifications (step 141 ), in order to keep compliance with the H.264 standard.
  • CBP coded block pattern
  • step 143 the values on the statistics of probability of watermarking a macroblock are updated with the information obtained in step 127: the number of candidate macroblocks and their type, and the number of the watermarked candidate macroblocks and their type.
  • FIG. 7 shows the flow chart of the conditions that must be checked to watermark a candidate macroblock in step 127.
  • the candidate macroblock to convey the watermark is selected for watermark embedding.
  • This candidate macroblock is analyzed in order to decide whether it meets the imperceptibility requirements to be watermarked; in addition, the selection of this candidate can be forced in order to guarantee that the watermark fits onto the current frame.
  • step 201 it is checked whether the prediction of the candidate macroblock is intra (I MB); if the answer is positive, then the algorithm goes to step 205.
  • the candidate macroblock is an inter predicted macroblock, and a 4x4 block of the current P-macroblock, BLOCK P, is pseudorandomly selected using as a coefficient one that introduces low embedding distortion COEF P (e.g., CF13 or CF14 following the zigzag order, where the index of each coefficient following this order is shown in FIG. 9) wherein the spatial error drift is unnoticeable.
  • COEF P low embedding distortion
  • the value of BLOCKS is set to BLOCK_P and the value of COEFS is set to COEF_P in step 203.
  • the selected indices of the coefficients are a parameter previously defined in the system that introduces low embedding distortion; for instance, if the coefficients follow a zigzag order, the last indices introduces low embedding distortion.
  • step 205 the type of intra prediction carried out in the candidate macroblock is checked. If the candidate macroblock is a 4x4 intra coded macroblock (14x4) then the algorithm goes to step 209; otherwise, the candidate macroblock is a 1 6x 1 6 intra coded macroblock and a 4x4 block to convey the watermark BLOCK M 6 is pseudorandomly selected without considering the bottom row and right column of the 1 6x 1 6 macroblock (hence, avoiding 4x4 blocks whose indexes are 6, 8, 14, 1 1 , 12, 15, 1 6 in FIG.
  • BLOCKS is set to BLOCK_i16 and the value of COEFS is set to COEF_i16 in step 207.
  • step 209 a search of blocks of the candidate macroblock verifying first level conditions is carried out. If there are blocks verifying the first level conditions, the BLOCKS is set to the selected blocks to convey the watermark, BLOCK I st, and COEFS is set to the selected coefficients, COEF I st, in step 211 .
  • the search of blocks to convey the watermark stops if the number of elements of BLOCKS reaches N_TOTAL_BITS_TO_EMBED (i.e., there are not remaining bits of b to be embedded). Otherwise, the candidate macroblock is analyzed in order to find blocks that verify second level conditions in step 213, this step 213 being optional.
  • BLOCKS is set to BLOCK_2nd, indicating that these blocks have been selected to convey the watermark, and COEFS is set to the selected coefficients, COEF_2nd, in step 215.
  • the search of blocks to convey the information stops when the number of elements of BLOCKS is N_TOTAL_BITS_TO_EMBED. If the current macroblock does not have blocks verifying the first level conditions and the second level conditions and FORCING is one (checked in step 217), then the BLOCKS is set to the selected block to convey the watermark, BLOCK FORCING, and COEFS is set to the selected coefficients, COEF FORCING, in step 219.
  • step 221 the selected coefficients (COEFS) of the selected blocks (BLOCKS) of the candidate macroblock are watermarked.
  • COEFS selected coefficients
  • BLOCKS selected blocks
  • the candidate macroblock is a 4x4 intra coded macroblock (14x4) a search of 4x4 blocks of the candidate macroblock verifying first level conditions is carried out in step 209.
  • the content of each 4x4 block is reconstructed by calculating the corresponding spatial prediction (following any of the 9 possible intra prediction modes for 4x4 intra prediction) and adding to said spatial prediction the decoded version of the residue. Therefore, by modifying the residue of a 4x4 block (i.e., when the watermark is embedded), the reconstructed content of the current block will differ from the original one. If this difference affects the border pixels of the block, then the spatial prediction of the successive adjacent blocks will be altered and a distortion will be originated and propagated along the whole decoded picture.
  • FIG. 10 shows the surrounding pixels (i.e., the set of pixels represented as ⁇ A,...,M ) used for the calculation of the intra prediction of a 4x4 block 362 (its contour marked with thick lines).
  • FIG. 11 shows the surrounding pixels (i.e., the vectors H and V) used for the calculation of the intra prediction of a 1 6x 1 6 block 364.
  • the first level conditions are defined for watermarking a candidate block.
  • the first level conditions that must satisfy a 4x4 block 362 to be watermarked are: 1 ) the 4x4 block must belong to a set of candidate blocks; and 2) the intra prediction performed on any of the successive adjacent 4x4 blocks cannot take as input the reconstructed pixel values of the current block under analysis.
  • the present invention assumes that the intra prediction modes performed on the successive adjacent blocks to the bordering 4x4 blocks (blocks number 6,8,1 1 ,12,14,15,16) of the current 16x16 macroblock 364 are unknown.
  • the set of candidate 4x4 blocks is defined as ⁇ 1 ,2,3,4,5,7,9,10,13 ⁇ , following the labels in the 16x16 macroblock 364 shown in FIG. 8, which depicts the 1 6x 1 6 macroblock 364 partitioned in sixteen 4x4 blocks (blocks numbered from 1 to 1 6).
  • This feature of the present invention allows to perform the processing of the macroblocks in a serial manner.
  • the evaluation of the first level conditions is carried out for each 4x4 block 362 in the candidate macroblock 364 to be watermarked. If a 4x4 block 362 is compliant with the first level conditions then it is appended to the blocks to watermark (BLOCKS). As stated above, this search of blocks to be watermarked stops when there are no more bits of b left to be embedded in the current frame. Otherwise, if none of the candidates fulfill the conditions, then the current macroblock is signaled as one that does not verify the first level conditions, so that second level conditions are verified in step 213.
  • BLOCKS blocks to watermark
  • the intra prediction modes of the adjacent blocks 2, 3, and 4 are checked to verify whether the neighboring pixel values of block 1 are used as input to compute the corresponding predictions.
  • step 213 Following an analogous approach to that for the first level conditions, second level conditions are verified in step 213.
  • the conditions for watermarking a 4x4 block are relaxed allowing some spatial drift that will not spread out of the bounds of the candidate macroblock 364.
  • the second level conditions that must be satisfied by a 4x4 block 362 to be watermarked are: 1 ) the 4x4 block 362 must belong to the set of candidate blocks ⁇ 1 ,2,3,4 ⁇ labeled according to FIG. 8; and 2) the intra prediction of the contiguous blocks to the previous group of successive adjacent 4x4 blocks (blocks ⁇ 6,8, 1 1 , 1 2, 14, 1 5, 1 6 ⁇ in FIG. 8) cannot propagate the originated spatial error, in order to keep the spatial error inside the bounds of the candidate macroblock 364.
  • the intra prediction of the successive adjacent 4x4 blocks (blocks ⁇ 5,7,9, 1 0, 1 3 ⁇ in FIG. 8) can be carried out using any intra prediction mode (thus originating spatial errors).
  • the evaluation of the second level conditions is carried out for each 4x4 block in a candidate macroblock to be watermarked.
  • the chain of intra prediction references is calculated, and each block in the set that has a chain that does not originate spatial errors outside the current macroblock 364 is selected as the one used to embed the watermark.
  • the number of maximum blocks of the candidate macroblock to be watermarked is limited by N_TOTAL_BITS_TO_EMBED.
  • a possible block to watermark is block number 1 (according to the diagram of FIG. 8)
  • the block 1 is still a candidate, the same procedure is followed with the adjacent blocks 3 and 4.
  • a forcing embedding mode is used as a mean to guarantee that there is room enough to embed the whole sequence of bits in an l-slice.
  • the forcing embedding mode is used when none of the 4x4 blocks 362 in a candidate macroblock 364 to be watermarked satisfy the first and second level conditions (decision taken in step 217 in FIG. 7).
  • ICT Integer Cosine Transform
  • the aforementioned ICT property provides a way to watermark a 4x4 block such that the corresponding distortion generated on the spatial domain is located in the two central vertical columns of the block or, in the same manner, in the two central horizontal rows, or even in the 4 central components (CF5, CF8, CF9, CF12) of the block.
  • This goal is achieved by arranging some of the transform coefficients of the 4x4 block into pairs such that by modifying one member of these pairs, the distortion originated can be compensated by changing the other coefficient of the pair in the opposite way.
  • the pair of coefficients (CF10, CF15) is the one used for embedding the watermark in any of the following 4x4 blocks 362 in the sub-set ⁇ 6,8,14 ⁇ .
  • the corresponding pair of coefficients for the subset of blocks ⁇ 1 1 ,12,15 ⁇ is (CF7, CF14) and for the block 16, two pairs of coefficients must be adjusted, i.e., (CF4, CF12) and (CF1 , CF6).
  • an informed embedding technique is used to embed a watermark into the non-entropy-coded H.264 syntax elements.
  • a technique based on uniform scalar quantizers is used in step 221 of FIG. 7.
  • the embedding information unit 300 generates the information binary vector b that is watermarked in the watermark embedding module 110.
  • the value of the coefficient with index / selected to convey a bit c ⁇ is modified allowing to retrieve the bit information bj at the decoder just by processing the value of the watermarked coefficient c i as where 2 ⁇ stands for a uniform scalar quantizer with step size Delta, and where in the last expression cf ⁇ is an outcome of a binary random variable whose probability mass distribution is which, in order to minimize the introduced embedding distortion, is pseudorandomly obtained using a pseudorandom number generator initialized with a seed that is function of the index of the current frame and the secret key K.
  • the value of the current processed embedding coefficient with index / is updated with
  • FIG. 12 shows the internal scheme of the watermark and H.264 video stream decoding module 500, which consists of an entropy decoder 570, and a watermark decoder module 510.
  • the watermarked H.264 video (H.264_R W VIDEO, which can be obtained, for instance, from the digital network 800) is entropically decoded by the entropy decoder 570, which transfers the watermarked non-entropy-coded H.264 syntax elements (H.264_R W SE) to the watermark decoder module 510.
  • the information binary vector b decoded by the watermark decoder module 510 using the secret key K and the extracted index of the current frame (INDEX) is transferred to the decoding information unit 700, where the metadata m is extracted, the authenticity of the metadata is checked and the temporal authentication is verified.
  • the decoding information unit 700 can read different kind of information (INFO 500 ) from the watermark and H.264 video stream decoding module 500: it can access to the watermarked H.264 video (H.264_R W VIDEO), and the watermarked non-entropy-coded H.264 syntax elements (H.264_R W SE).
  • INFO 500 watermarked H.264 video
  • H.264_R W VIDEO watermarked non-entropy-coded H.264 syntax elements
  • FIG. 13 shows the internal scheme of the decoding information unit 700, which consists of a metadata information decoding unit 710, a security unit 730, and a decoding information control unit 750.
  • the metadata information decoding unit 710 takes as an input the information binary vector b from the watermark decoder and
  • H.264 video stream decoding module 500 extracts from it the metadata information m and the index of the frame (INDEX), which are then fed along with the information INFO 500 from the watermark and H.264 video stream decoding module 500 to the security unit 730 that generates as a function of the secret key K a calculated security vector or calculated digest (c_digesti), whereby the integrity of this information can be verified.
  • the calculated digest ⁇ c_digest t ) generated by the security unit 730 is transferred together with the extracted metadata information m , and the index of the current frame (INDEX) to the decoding information control unit 750.
  • the decoding information control unit 750 resets (signal RESET) the security unit 730 to an initial state after processing each set of frames.
  • the decoding information control unit 750 arranges the extracted N digests obtaining a extracted digest of the previous set of frames, i.e., received digest r digest, obtained from the information binary vector b . Then, the decoding information control unit 750 compares the received digest r digest, with the calculated digest c digest, computed from the /-th previous set of frames in the security unit 730. If they match, then the decoding information control unit 750 indicates that the extracted metadata, indices, and the content of the (/)-th set of frames are authentic; otherwise, it points out that they are false.
  • the decoding information control unit 750 decides whether the order of the frames of the watermarked H.264 video was altered.
  • the decision on the authenticity (AUTHENTICITY) and the corresponding extracted metadata are the outputs of the decoding information control unit 750 and also of the decoding information unit 700.
  • FIG. 14 represents the flow chart of the watermark decoder module 510 shown in FIG. 12, wherein each frame of watermarked non-entropy-coded H.264 syntax elements (H.264_R W SE) is processed.
  • the first step to extract the watermark from the current frame is the update of a parameter P taking values between 0 and 1 in step 513, which is used to pseudorandomly determine, as a function of the secret key K Q , whether a given macroblock conveys watermark or not.
  • the value of the number of remaining coefficients to read is used to initialize R_COEFFS_TO_READ (step 515), then the value of the total number of bits to be read is used to initialize N_TOTAL_BITS_TO_READ (step 517).
  • FORCING is initialized to zero (step 519).
  • step 521 the comparison between R CO E F FS_TO_R E A D and N_TOTAL_BITS_TO_READ is computed; if they have the same value, then FORCING is set to one (step 523).
  • the decision of selecting a current macroblock as candidate to extract the watermark is carried out in step 525 by computing an OR function between a pseudorandomly obtained outcome of a Bernoulli random variable using the parameter P updated in step 513 and FORCING.
  • step 525 the current macroblock is selected as a candidate macroblock to extract the watermark (step 527) and N_TOTAL_BITS_TO_READ is updated by subtracting from its previous value the number of extracted bits, if any (step 529). R_COEFFS_TO_READ is updated by subtracting one to its value in step 531.
  • the flow of the algorithm goes to step 511 if the value of N_TOTAL_BITS_TO_READ is zero (step 533); otherwise, the algorithm goes to step 519.
  • the last step which is optional but it must be carried out if this step has been done in the embedding process, is the pseudorandom permutation as a function of the secret key K e oi the extracted N_TOTAL_BITS_TO_READ in step 511 to obtain the extracted information binary vector b 1 , which is passed to the decoding information unit 700.
  • step 527 of FIG. 14 the current macroblock is selected as a candidate macroblock to extract the watermark.
  • the flow chart of the conditions that must be checked to select the candidate blocks and coefficients of the current macroblock is depicted in FIG. 7, already explained for the embedding method.
  • the number of selected blocks BLOCKS cannot exceed N_TOTAL_BITS_TO_READ.
  • the watermark extraction algorithm of the watermark decoding method is carried out in step 221 in FIG. 7, wherein the selected coefficients (COEFS) of the selected blocks (BLOCKS) of the candidate macroblock are decoded to extract the watermark.
  • This watermark extraction algorithm is similar to its counterpart watermark embedding algorithm.
  • the extracted information binary vector b J is sent to the decoding information unit 700, which processes the information, extracting the information and verifying the spatial and temporal authenticity of the analyzed watermarked H .264 video.
  • a parameter P is computed.
  • the objective of selecting the candidate macroblock to be watermarked using a Bernoulli random variable is to uniformly distribute the watermark into each frame; thus in order to obtain this, an estimation of the ratio between the number of coefficients to embed and the total number of available coefficients to convey the watermark is necessary; therefore, in the preferred embodiment of the current invention, this ratio is obtained using the obtained information in the watermark embedding and watermark decoding.
  • Two probabilities are calculated: one probability for l-slices p ⁇ and another P-slices pp. The value of these two probabilities is calculated as totaLnuiriber_bits-to embed
  • PP i 3 ⁇ 4 where in the previous expressions, denotes the number of 4x4 intra predicted macroblocks, denotes the number of watermarked 4x4 intra predicted macroblocks, denotes the number of candidate 4x4 intra predicted macroblocks, ⁇ /1 6 ⁇ 1 6 denotes the number of 1 6x 1 6 intra predicted macroblocks, n
  • FIG. 15 shows a scheme of elements representing a possible implementation of the described embedding method that can be embodied, for instance, in a digital camera 1500.
  • the digital camera includes a lens 1501 that focalizes an image on an image sensor 1502, an H.264 encoder 1503 that uses the flow of images captured by the image sensor 1502 to produce a video stream which is compliant with the H.264 standard.
  • a control circuit 1504 is in charge of the acquisition of the flow of images, the H.264 video stream generation and, optionally, the communication with a digital network 1506. Moreover, the control circuit 1504 has the capacity to execute operations stored in a memory 1505, where the necessary instructions for the implementation of the insertion method proposed in this invention are saved.
  • FIG. 16 illustrates a possible implementation of the described extraction method in a computational system 1600 for the purpose of exposing forgeries.
  • the computational system comprises a processor 1601 and a memory 1602.
  • the processor 1601 is connected to a digital network 1603 to obtain H.264 bitstreams from camera devices 1604 connected to the digital network 1603.
  • the processor 1601 performs the extraction method by executing the operations stored in the memory 1602. Therefore, the described processes to perform the extraction of the hidden information and detect manipulations can be implemented in these computational systems.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Method and system for embedding information and authenticating a H.264 video using a digital watermark. It comprises a watermarking and H.264 video stream generation module (100) for embedding a watermark in non-entropy-coded H.264 syntax elements (H.264SE), processing each frame j to pseudorandomly determine whether a given macroblock is a candidate macroblock for conveying the watermark and check whether a determined set of conditions to watermark a selected coefficient (COEFS) of a selected block (BLOCKS) of the candidate macroblock are met, which are met when the candidate macroblock is a 4×4 intra coded macroblock (I4x4) and the selected block of the candidate macroblock to convey the watermark belongs to a predefined set of candidate blocks and the intra prediction modes of its successive adjacent blocks do not take as input the value of the selected block. The module (100) then embed the watermark in the selected coefficient (COEFS) of the selected block (BLOCKS) meeting the set of conditions.

Description

Method and system for embedding information and authenticating a H.264 video using a digital watermark
Field of the Invention
The present invention relates to the field of digital information protection, and more in particular to a scheme for embedding in a digital video a digital watermark containing information about the authenticity of the video and other metadata, and a scheme for retrieving such information from the video. Background of the Invention
The digital video standard H.264 has become the most prevalent video compression standard, due to its versatility and compression capabilities, with application in many different fields like high compression video for mobile phone networks, high-definition television (HDTV) and Blu-Ray. Moreover, due to cost reductions, the number and variety of electronic devices with H.264 encoding capabilities is steadily increasing, ranging from mobile phones to modern video surveillance cameras, and even toys.
From a security standpoint, some of the applications that involve video compression are critical. This is the case of video surveillance, either performed by traditional video surveillance systems or by mobile systems (e.g. unmanned aerial vehicles). Nowadays, there are many powerful multimedia editing tools for digital video available, which allow any unskilled person to easily modify digital videos, thus arising serious concerns about the trustworthiness of those videos. To regain confidence on digital video, several authentication tools have been proposed. The first proposals were based on cryptographic techniques, more in particular using cryptographic hashes and digital signatures. The problem with cryptographic approaches for video authentication is that the authentication information is not self- contained in the video, but it must be attached in separate files or interleaved with the video content, usually by means of ad-hoc proprietary formats which give rise to management and compatibility problems. In order to solve these problems, digital watermarking techniques for video authentication were proposed. In watermarking- based authentication techniques, a low-power signal (watermark) is embedded into the digital content (the video, in this case) to be protected. The authenticity of the received signal is determined by verifying the presence of the correct watermark with an appropriately designed watermark detector.
The main advantage of watermarking-based video authentication is that, once applied to a video signal, the latter transparently conveys the authentication information during its whole lifecycle, keeping its format unaltered and without degrading the visual quality. Nonetheless, the application of video watermarking is usually not straightforward. In fact, successfully embedding a watermarking in a video without degrading its visual quality becomes more complex as the complexity of the video coding standard increases.
Modern video codecs do not treat the video as a sequence of independent pictures (frames), but they exploit the correlation between frames to provide a more efficient compression. The H.264 standard, in particular, obtains huge compression rates without degrading video quality at the expense of a computationally complex encoding process. H.264 divides each video frame in one or more slices, and each slice in a collection of non-overlapping blocks of size 16x 16 pixels, called macroblocks, which are in turn divided in blocks of size 4x4. The H.264 standard exploits both the spatial correlation between the adjacent blocks of each video frame, and the temporal correlation between blocks corresponding to contiguous frames. For every macroblock, a prediction based on a previously coded macroblock or set of macroblocks is computed. Such prediction is called intra prediction when the spatial correlation is exploited, and inter prediction in the case of the temporal correlation. According to the two types of available predictions, two types of slices are defined in the H.264 standard: l-slices which contain only intra predicted macroblocks (I- macroblocks) and P-slices which contain intra and/or inter predicted macroblocks (P- macroblocks). The slices are arranged together in sequences, forming Groups of Pictures (GOPs) where the first slice is an l-slice and the remaining ones (if any) are P- slices.
Intra prediction consists in specifying how a given macroblock can be estimated from previously coded pixels that belong to neighboring macroblocks in the same slice. Thirteen different prediction modes are defined in the H.264 standard. On the other hand, inter prediction consists in specifying how to compute the pixels in a given macroblock using other macroblocks belonging to previously coded slices. In H.264, inter prediction is coded by means of Motion Vectors (MV). The result of computing the difference between a given block and its intra or inter prediction is called residue. In order to provide a compact representation of each macroblock, the residue is transformed using the Integer Cosine Transform (ICT) and quantized, whereas for the MVs only the difference between the MV and its prediction computed from its neighboring MVs, called Motion Vector difference (MVd) is kept. Thus, finally, each macroblock is represented by three syntax elements: the transformed and quantized residue, the MVd, and the intra prediction mode. For those macroblocks belonging to P-slices whose residue and MVd is null, only the SKIP label is transmitted. The final step of the H.264 encoding consists in entropy-coding the syntax elements. Entropy coding is a lossless coding that provides a more compact representation of the final H.264 bitstream, which can be either a Context Adaptive Variable Length Coder (CAVLC) or a Context-based Adaptive Binary Arithmetic Coder (CABAC).
Besides the complexity imposed by the H.264 standard, watermarking-based video authentication faces a number of additional constraints related to the application scenarios and the kind of devices that must carry out the embedding of the watermarks in the video content. These constraints can be summarized as follows:
· Low latency. In scenarios with real-time requirements, such as videosurveillance and remote video streaming, the temporal delay introduced by watermark embedding must be as low as possible. In streaming environments (e.g. remote videosurveillance), each slice must be processed macroblock by macroblock in order to minimize such delay.
· Low computational resources. Usually the devices (e.g. videosurveillance cameras) that embed the watermark in the video has to perform a number of other tasks such as video encoding, network communication, etc. In order to limit the energy consumption and the cost of those devices, they are buit with hardware with low computational resources. Thus, the complexity of the watermark embedding process must be as low as possible, both in terms of memory and processing.
• Simple integration in the H.264 encoding workflow. Devices with H.264 encoding capabilities are usually built from standard hardware modules (SoC, FPGA, etc) that perform video encoding and other related video processing operations. Such modules cannot be modified at will for adding watermark embedding capabilities. In addition, there are other constraints derived from the watermarking process itself:
• Imperceptibility. The addition of a watermark to a H.264 encoded video must not degrade its visual quality.
• Security. The authentication process must be provably secure, by means of cryptographic keys or any other mechanism which guarantees that authentication watermarks cannot be forged.
• High payload. The amount of information (i.e. the number of bits) that can be embedded by means of the watermark must be high enough so as to guarantee, at least, room for embedding the authentication information.
• Granularity. The authentication method must be able to authenticate video at the frame level, i.e. it must be able to authenticate each frame independently of the others, or at least at a group of frames. Additionally, it is desirable that the authentication method is able to deal with different profiles of H.264, where the term profile, considered in the standard, means a particular combination of capabilities and configuration parameters of the H.264 codec. For instance, it is desirable that the authentication method is able to cope with I and P- slices, CABAC and CAVLC entropy-coded bitstreams, etc.
The methods for embedding watermarks in digital videos encoded with H.264 can be divided in two groups: those that perform embedding inside the H.264 encoder, and those that perform embedding in the bitstream. The methods in the first group integrate the watermark embedding process in the H.264 encoding workflow by adding one or more elements to the H.264 encoder that modify one or more syntax elements as they are being encoded. The main drawback of these methods is that they require modification of the internal architecture of H.264 encoders, which is hardly applicable in practical systems, as explained above. One example of such methods is disclosed in the article by Profrock et al. "H.264/AVC video authentication using skipped macroblocks for an erasable watermark", in SPIE Visual Communications and Image Processing, volume 5960, 2005. Further examples, among others, of this kind of methods are given by the article by Ramaswamy and Rao "Video authentication for H.264/AVC using digital signature standard and secure hash algorithm", in Proceedings of the ACM 2006 International Workshop on Network and Operating Systems Support for Digital Audio and Video, NOSSDAVO6, pages 21 :1 -21 :6, NY, USA; the article by Zhang and Ho "Efficient video authentication for H.264/AVC", in First IEEE International Conference on Innovative Computing, Information and Control, ICICIC'06, volume 3, pages 46-49, September 2006; and the article by Noorkami and Mersereau "Digital video watermarking in P-Frames with controlled video bit-rate increase", in IEEE Transactions on Information Forensics and Security, 3(3):441 -455, September 2008.
The methods that embed the watermark in the bistream do not present the drawback of requiring the modification of the internal architecture of the H.264 encoder, as they work with H.264 video that has already been encoded. Some of these methods embed the watermark directly in the final H.264 bitstream, whereas others perform embedding in the non-entropy-coded syntax elements. One of the most referenced works dealing H.264 watermarking under real-time constraints is Noorkami, and Mersereau, "Compressed-domain video watermarking for H.264," IEEE International Conference on Image Processing, 2005, vol.2, pp. II- 890-3, 1 1 -14 Sept. 2005. The authors present a method designed to work on a real-time scenario by embedding a watermark on a pseudorandomly selected coefficient of each macroblock of l-slices after entropically decoding a H.264 video. The main drawbacks of this method are the lack of solution to deal with the spatial error drift generated by the modification of the reference blocks used in the intra prediction, and the impossibility of using this method in inter predicted frames, where inter prediction is a key feature of the H.264 standard (and of all video compression standards, in general).
An example of a technique for watermarking l-slices of a H.264 compressed bit stream without causing perceptual visual distortions can be found in the article by X. Ma, Z. Li, H. Tu, and B. Zhang: "A Data Hiding Algorithm for H.264/AVC Video Streams Without Intra-Frame Distortion Drift," published in IEEE Transactions on Circuits and Systems for Video Technology, Vol. 20, No. 10, October 2010. This document presents a method for embedding information on the quantized coefficients of intra predicted macroblocks without introducing errors in the prediction of the subsequent adjacent blocks. The proposed idea is based on a property of the 4x4 Integer Cosine Transform that makes possible to maintain unaltered the border of the inverse-transform block in the spatial domain, by changing accordingly pairs of coefficients. Using this property and knowing the prediction modes of the blocks which are adjacent to the one used for embedding, the corresponding pair of coefficients is selected to embed the information without generating any error in the following predictions. The main limitation of this method is that the knowledge of the prediction modes of future macroblocks is necessary to avoid the spatial distortion. Therefore, the proposed algorithm is not suitable for real-time, low-latency applications.
In the article by Profrock et al. entitled "H.264/AVC video authentication using skipped macroblocks for an erasable watermark", Proc. SPIE Visual Commun. Image Process., vol. 5960, pp.1480-2005, a new watermarking-based authentication method for H.264 video is presented. The watermark is embedded onto the AC coefficients of certain skipped macroblocks, which are selected by analyzing the inter prediction references to each skipped macroblock of a frame of the whole H.264 video stream, and selecting those which have fewer references. The payload of the watermark is an encrypted hash of the digital content and a certificate with a public key. The watermark decoder identifies the watermarked skipped blocks, decodes the watermark determining the authenticity of the received video and restablishes the skipped blocks that were watermarked (thus, this technique is a reversible H.264 watermarking method). However, this method is not suitable for real-time applications due to the necessary analysis of the whole H.264 video and there is not guarantee of having enough room for a watermark of a given length, as the available payload depends on the statistics of the specific video.
Related to the previous work by Profrock et al., He et al. disclose in US Patent App. No. 12/737783 a method to determine the visual impact, due to inter and intra prediction, of a modification in a H.264 video. By employing this technique, a watermark embedder can select the least noticeable modification, from a range of tested modifications, to embed the watermark. Since this technique requires to measure the embedding distortion on the spatial domain and also needs to track embedding modifications through several P frames, this invention is not suitable for applications with low latency and low computational requirements.
The generation of watermark signals for embedding with low latency in digital contents, such as audio or video, is disclosed in patent document US2010001 1217-A1 . The disclosed method can be used for embedding watermarks that protect multimedia contents which must be streamed in real time to the consumers. The proposed method works in a real time with low latency, but it is not possible to check neither the integrity nor the authenticity of the watermarked contents. Besides, due to the fact that the embedded watermark is robust to several signal processing operations, it is not possible to recognize or localize spatial or temporal manipulations. Zou and Bloom disclose in patent document US 20100169349-A1 a method for watermarking H.264 bitstreams which are entropy-coded with CAVLC. Most of the computational complexity takes place during an analysis stage wherein an embedding look-up table is built. Such table identifies fragments of data in the video stream that can be replaced in order to embed the watermark, and provides as well the values for the different replacement fragments to encode the watermark. The replacement fragments must be appropriately designed so that the resulting bitstream is compliant with the H.264/AVC standard and does not introduce significant distortion to the decoded video. The computation of the replacement fragments in the proposed method is performed by modifying the intra prediction modes of certain macroblocks. The main drawback of this method is that the watermark embedding process requires a previous stage for analyzing the video stream and building the look-up table, which is highly costly from the computational point of view and must be performed offline, thus it does not meet the constraints. From the previous analysis of the prior art, it can be concluded that the existing methods for watermarking-based authentication of H.264 video present important shortcomings. In other words, there is a need for a watermarking-based authentication method which can simultaneously satisfy the requirements of low complexity, low latency, low embedding distortion and high security.
Description of the Invention
The present invention overcomes the problems previously identified for watermarking- based authentication of H.264 video. The invention relates to a method for embedding, in a H.264 video, a digital watermark that contains information about the authenticity of the video and other metadata. The invention is highly efficient, secure, and with imperceptible effect in the visual quality of the watermarked video. The watermarked video is fully compliant with the H.264 standard.
In accordance with one aspect of the present invention there is provided a system for embedding a digital watermark in a H.264 video. The system comprises a watermarking and H.264 video stream generation module with a watermark embedding module for embedding a watermark in non-entropy-coded H.264 syntax elements of the H.264 video, obtaining watermarked non-entropy-coded H.264 syntax elements. The watermark embedding module comprises processing means configured for processing each frame / of the non-entropy-coded H.264 syntax elements to:
- pseudorandomly determine, for each macroblock and until there are no remaining bits of the watermark to be embedded in the corresponding frame y, whether a given macroblock is a candidate macroblock for conveying the watermark, and in that case check whether a determined set of conditions to watermark at least one selected coefficient of at least one selected block of the candidate macroblock are met, said set of conditions including at least first level conditions which are met when:
the candidate macroblock is a 4x4 intra coded macroblock; the selected block of the candidate macroblock to convey the watermark belongs to a predefined set of candidate blocks and the intra prediction modes of its successive adjacent blocks do not take as input the value of the selected block;
- embed the watermark in the at least one selected coefficient of the selected block meeting the set of conditions. In a preferred embodiment the predefined set of candidate blocks in the first level conditions comprises the nine blocks contained within the first three rows and the first three columns of the candidate macroblock.
The set of conditions may also include second level conditions to be checked when the first level conditions are not met, said second level conditions being met when:
the candidate macroblock is a 4x4 intra coded macroblock;
the selected block of the candidate macroblock to convey the watermark belongs to a set of candidate blocks which comprises the four blocks contained within the first two rows and the first two columns of the candidate macroblock, and the intra prediction modes are such that a modification in the selected block does not propagate any spatial error to any of the seven blocks located in the fourth row or in fourth column of the macroblock.
In yet a preferred embodiment the set of conditions also includes forcing embedding conditions to be checked, said forcing embedding conditions being met when the total number of bits to embed in the corresponding frame j is equal to the remaining macroblocks of the frame j which have not yet being analyzed for candidate macroblock purposes. The set of conditions preferably also includes checking whether the candidate macroblock is an inter predicted macroblock, and in that case selecting at least one block of said macroblock and at least one coefficient of said block.
The set of conditions may also include checking whether the candidate macroblock is a 16x16 intra predicted macroblock, and in that case selecting at least one block of said macroblock and at least one coefficient of said block.
In a preferred embodiment the processing means are configured to pseudorandomly determine whether a given macroblock is a candidate macroblock for conveying the watermark by computing a pseudorandomly obtained outcome of a Bernoulli random variable using a parameter P previously updated using a pseudorandom number generator initialized with a seed that is function of the index of the current frame / and a secret key K. The watermarking and H.264 video stream generation module may further comprise: an entropy coder unit for entropically coding the watermarked non-entropy- coded H.264 syntax elements, obtaining entropically coded watermarked H.264 syntax elements, and
a H.264 video stream generation unit for producing a watermarked H.264 video.
In a preferred embodiment the system further comprises an embedding information unit configured to obtain a sequence of bits as the watermark to be embedded.
The embedding information unit is preferably configured to obtain, for each frame j of the H.264 video stream, an information binary vector as a function of metadata information, an index of the current frame, data obtained from the watermarking and H.264 video stream generation module, and a security vector used to validate a set of frames. The embedding information unit may also comprise: a metadata information coding unit for obtaining a metadata information vector for each frame y;
a security unit for generating the security vector as a function of a secret key K, by using the metadata information vectors of a set of frames comprising at least one frame, their corresponding indices, and data obtained from the watermarking and H.264 video stream generation module,
an embedding information control unit for obtaining the information binary vector for each frame j. The embedding information unit is preferably configured to divide the security vector, obtained for each set of frames, into N vectors, where N denotes the number of frames which compose each set of frames,
and wherein the information binary vector corresponding to the frame j of the set of frames i+ 1 comprises:
the metadata information vector for said frame y,
the corresponding index (y), and
the security vector (digest,,/), where / is the corresponding index for the /-th chunk of the digestj Oi the previous set of frames /'.
The data obtained from the watermarking and H.264 video stream generation module preferably comprises syntax elements of the H.264 video stream.
The metadata information preferably comprises at least one of the following data:
GPS position coordinates;
an identifier of the embedding device;
embedding time.
The embedding information unit is preferably configured to generate an information binary vector b used as watermark, such that the value of the coefficient with index /' selected to convey a bit c- is modified by the watermark embedding module, allowing to retrieve the bit information b- at the decoder by processing the value of the watermarked coefficient ci as h
c-i = ( + di QA Ci
2 ) ) where qA stands for a uniform scalar quantizer with step size Delta, and where dj \s an outcome of a binary random variable whose probability mass distribution is if k = 1
fo(k) { Ϊ if k =—1 which is pseudorandomly obtained using a pseudorandom number generator initialized with a seed that is function of the index of the current frame and a secret key K.
In accordance with a further aspect of the present invention there is provided a system for generating a digital watermark for embedding in a H.264 video. The system comprises an embedding information unit configured to obtain, for each frame y of the non-entropy-coded H.264 syntax elements of the H.264 video, an information binary vector as the watermark to be embedded, said vector obtained as a function of metadata information, an index of the current frame, syntax elements of the previous frames of the H.264 video stream, and a security vector used to validate a set of frames. The embedding information unit preferably comprises:
a metadata information coding unit for obtaining a metadata information vector for each frame y;
a security unit for generating the security vector as a function of a secret key K, by using the metadata information vectors of a set of frames comprising at least one frame, their corresponding indices, and syntax elements of the previous frames of the H.264 video stream,
an embedding information control unit for obtaining the information binary vector for each frame j. The embedding information unit is preferably configured to divide the security vector, obtained for each set of frames, into N vectors, where N denotes the number of frames which compose each set of frames,
and wherein the information binary vector corresponding to the frame j of the set of frames i+ 1 comprises: the metadata information vector for said frame /,
the corresponding index (j) , and
the security vector (digest,,/), where / is the corresponding index for the current frame / of the security vector (digestj) of the previous set of frames / (SF).
The metadata information may comprise at least one of the following data:
GPS position coordinates;
an identifier of the embedding device;
embedding time.
In accordance with yet a further aspect of the present invention there is provided a system for decoding a digital watermark in a H.264 video, comprising a watermark and H.264 video stream decoding module (500) with a watermark decoder module (510) for decoding, from each frame j of the watermarked non-entropy-coded H.264 syntax elements (H.264_RW_SE) of the H.264 video (H.264_RW VIDEO), an information binary vector b} which contains a metadata information vector m1 , the index of the current frame / and a security vector {r_diges , where / is the corresponding index for the /-th chunk of the r digestj of the previous set of frames /' (SF,), used to validate said previous set of frames (SF,).
In accordance with yet a further aspect of the present invention there is provided a system for authenticating a H.264 video using a digital watermark. The system comprises:
a watermark and H.264 video stream decoding module with a watermark decoder module for decoding, from each frame j of the watermarked non-entropy- coded H.264 syntax elements of the H.264 video, an information binary vector which contains a metadata information vector, the index of the current frame j and a security vector (r_digestji), where / is the corresponding index for the /-th chunk of the r digestj of the previous set of frames /', used to validate said previous set of frames;
a decoding information unit configured to:
- obtain, for each set of frames (SF,), a calculated security vector (c_digesti) as a function of a secret key K and using the metadata information vectors m1 of a set of frames (SF,) comprising at least one frame, their corresponding indices (INDEX), and data (INF050o) obtained from the watermark and H.264 video stream decoding module (500);
- compare the security vector {r_digestj) with the calculated security vector (c_digesti) so that if they match the corresponding set of frames is considered as authentic.
The watermark decoder module is preferably configured to extract the information binary vector using a secret key K and the index of the current frame.
In accordance with another further aspect of the present invention there is provided a method for embedding a digital watermark in a H.264 video. The method comprises: embedding a watermark in non-entropy-coded H.264 syntax elements of the H.264 video, obtaining watermarked non-entropy-coded H.264 syntax elements;
processing each frame j of the non-entropy-coded H.264 syntax elements (H.264SE) to:
- pseudorandomly determine, for each macroblock and until there are no remaining bits of the watermark to be embedded in the corresponding frame y, whether a given macroblock is a candidate macroblock for conveying the watermark, and in that case check whether a determined set of conditions to watermark at least one selected coefficient (COEFS) of at least one selected block (BLOCKS) of the candidate macroblock are met, said set of conditions including at least first level conditions which are met when:
the candidate macroblock is a 4x4 intra coded macroblock (14x4); the selected block of the candidate macroblock to convey the watermark belongs to a predefined set of candidate blocks and the intra prediction modes of its successive adjacent blocks do not take as input the value of the selected block;
- embed the watermark in the at least one selected coefficient (COEFS) of the selected block (BLOCKS) meeting the set of conditions. In a preferred embodiment the predefined set of candidate blocks in the first level conditions comprises the nine blocks contained within the first three rows and the first three columns of the candidate macroblock
The set of conditions also may include second level conditions to be checked when the first level conditions are not met, said second level conditions being met when: the candidate macroblock is a 4x4 intra coded macroblock;
the selected block of the candidate macroblock to convey the watermark belongs to a set of candidate blocks which comprises the four blocks contained within the first two rows and the first two columns of the candidate macroblock, and the intra prediction modes are such that a modification in the selected block does not propagate any spatial error to any of the seven blocks located in the fourth row or in fourth column of the macroblock.
The set of conditions preferably also includes forcing embedding conditions to be checked, said forcing embedding conditions being met when the total number of bits to embed in the corresponding frame j is equal to the remaining macroblocks of the frame j which have not yet being analyzed for candidate macroblock purposes.
The set of conditions may also include checking whether the candidate macroblock is an inter predicted macroblock, and in that case selecting at least one block of said macroblock and at least one coefficient of said block.
The set of conditions may also include checking whether the candidate macroblock is a 16x16 intra predicted macroblock, and in that case selecting at least one block of said macroblock and at least one coefficient of said block.
The method may further comprise pseudorandomly determining whether a given macroblock is a candidate macroblock for conveying the watermark by computing a pseudorandomly obtained outcome of a Bernoulli random variable using a parameter P previously updated using a pseudorandom number generator initialized with a seed that is function of the index of the current frame / and a secret key K.
In a preferred embodiment the method further comprises:
entropically coding the watermarked non-entropy-coded H.264 syntax elements, obtaining entropically coded watermarked H.264 syntax elements, and
producing a watermarked H.264 video.
The method preferably comprises obtaining a sequence of bits as the watermark to be embedded. The method may further comprise obtaining, for each frame j of the H.264 video stream, an information binary vector as a function of metadata information, an index of the current frame, data previously obtained, and a security vector used to validate a set of frames.
In a preferred embodiment the method further comprises:
obtaining a metadata information vector for each frame y;
generating the security vector as a function of a secret key K, by using the metadata information vectors of a set of frames comprising at least one frame, their corresponding indices, and data previously obtained,
obtaining the information binary vector for each frame j.
The method preferably comprises dividing the security vector, obtained for each set of frames, into N vectors, where N denotes the number of frames which compose each set of frames,
and wherein the information binary vector corresponding to the frame j of the set of frames i+ 1 comprises:
the metadata information vector for said frame y,
the corresponding index (y), and
the security vector (digest,,/), where / is the corresponding index for the /-th chunk of the digestj Oi the previous set of frames /'.
The data previously obtained preferably comprises syntax elements of the H.264 video stream.
The metadata information may comprise at least one of the following data:
GPS position coordinates;
an identifier of the embedding device;
embedding time.
The information binary vector b used as watermark is preferably such that the value of the coefficient with index / selected to convey a bit c^ is modified allowing to retrieve the bit information at the decoder by processing the value of the watermarked coefficient as
Figure imgf000017_0001
where stands for a uniform scalar quantizer with step size Delta, and where dj \s an outcome of a binary random variable whose probability mass distribution is fD(k) = I . ,k =
J K ' \ if k = -1 which is pseudorandomly obtained using a pseudorandom number generator initialized with a seed that is function of the index of the current frame and a secret key K.
In accordance with a further aspect of the present invention there is provided a method for generating a digital watermark for embedding in a H.264 video. The method comprises obtaining, for each frame / of the non-entropy-coded H.264 syntax elements of the H.264 video, an information binary vector as the watermark to be embedded, said vector obtained as a function of metadata information, an index of the current frame, syntax elements of the previous frames of the H.264 video stream, and a security vector used to validate a set of frames (SF,). The method may further comprise:
obtaining a metadata information vector for each frame y;
generating the security vector as a function of a secret key K, by using the metadata information vectors of a set of frames comprising at least one frame, their corresponding indices, and syntax elements of the previous frames of the H.264 video stream,
obtaining the information binary vector for each frame y.
The method may also comprise dividing the security vector, obtained for each set of frames, into N vectors, where N denotes the number of frames which compose each set of frames, and wherein the information binary vector corresponding to the frame of the set of frames i+ 1 comprises:
the metadata information vector for said frame y,
the corresponding index (y), and
the security vector (digest,,), where / is the corresponding index for the current frame / of the security vector (digest-) of the previous set of frames /'.
The metadata information may comprise at least one of the following data:
GPS position coordinates;
an identifier of the embedding device;
embedding time.
In accordance with a further aspect of the present invention, there is provided a method for decoding a digital watermark in a H.264 video. It comprises decoding, from each frame / of the watermarked non-entropy-coded H.264 syntax elements (H.264_RW_SE) of the H.264 video (H.264_RW VIDEO), an information binary vector b} which contains a metadata information vector m1 , the index of the current frame / and a security vector (r_digestji), where / is the corresponding index for the /-th chunk of the r digestj Oi the previous set of frames / (SF,), used to validate said previous set of frames (SF,).
In accordance with yet a further aspect of the present invention there is provided a method for authenticating a H.264 video using a digital watermark. The method comprises:
decoding, from each frame / of the watermarked non-entropy-coded H.264 syntax elements of the H.264 video, an information binary vector which contains a metadata information vector, the index of the current frame / and a security vector {r digestj j, where / is the corresponding index for the /-th chunk of the r digestj Oi the previous set of frames /', used to validate said previous set of frames;
obtaining, for each set of frames, a calculated security vector (c_digesti) as a function of a secret key K and using the metadata information vectors of a set of frames comprising at least one frame, their corresponding indices, and data previously obtained;
comparing the security vector (r_digest-) with the calculated security vector
(c_digesti) so that if they match the corresponding set of frames is considered as authentic.
The method may also comprise extracting the information binary vector using a secret key K and the index of the current frame. The main technical novelties of the present invention are the following:
• A mechanism for embedding the watermark in a serial manner, macroblock by macroblock, minimizing the time and computational resources required for embedding, while keeping a low embedding distortion.
• A smart mechanism, based on checking three conditions, for embedding the watermark in the macroblocks, guaranteeing a given payload per frame and a low embedding distortion.
• An adaptive randomization mechanism for selecting the watermark embedding locations in each frame, that guarantees a low perceptual impact and a given payload per frame.
The main advantages of the present invention are the following:
• Low latency. The embedding of the watermark is performed with minimal delay.
Thus, the present invention is suitable to authentication of streamed video, and to other scenarios with strict real-time requirements.
• Low computational demand. The present invention does not require high memory or high computing power. Thus, it is amenable to integration in cheap electronic devices.
• Easily integrable in the H.264 encoding workflow. The embedding of the watermark with the present invention can be performed either during or after the encoding process, requiring a small amount of changes in the encoding workflow.
• Highly secure. The present invention comprises mechanisms for adapting the security level of the authentication watermark to the application scenario of interest. It can provide cryptographic security, compliant with generally accepted security standards for electronic devices, if necessary.
• Imperceptible embedding distortion. The videos watermarked with the present invention are indistinguishable from non-watermarked videos, even for high compression factors.
Brief Description of the Drawings
A series of drawings which aid in better understanding the invention and which are expressly related with an embodiment of said invention, presented as a non-limiting example thereof, are very briefly described below. FIG. 1 depicts a schematic block diagram of the elements of the system according to the present invention.
FIG. 2 shows the internal block diagram of the watermarking and H .264 video stream generation module.
FIG. 3 shows the internal scheme of the embedding information unit.
FIG. 4 depicts a basic diagram of the processing of a set of four frames.
FIG. 5 represents the flow chart of the watermark embedding module.
FIG. 6 shows an example of the neighbouring partitions that are considered for the computation of the prediction of a Motion Vector (MV).
FIG. 7 shows the flow chart of the conditions that must be checked to watermark a candidate macroblock.
FIG. 8 illustrates a 1 6x 1 6 macroblock partitioned in sixteen 4x4 blocks.
FIG. 9 illustrates the ordering of the coefficients in a 4x4 block.
FIG. 1 0 illustrates the surrounding pixels used for the computation of the intra prediction of a 4x4 block.
FIG. 1 1 illustrates the surrounding pixels used for the computation of the intra prediction of a 1 6x 1 6 block.
FIG. 1 2 shows the internal scheme of the watermarking and H .264 video stream decoding module.
FIG. 1 3 shows the internal scheme of the decoding information unit.
FIG. 14 represents the flow chart of the watermark decoding module. FIG. 15 depicts a schematic block diagram of the elements representing a possible implementation of the described invention integrated in a electronic device.
FIG. 16 illustrates a possible implementation of the described extraction method in a computational system for the purpose of exposing forgeries.
Description of a Preferred Embodiment of the Invention
FIG. 1 depicts a schematic block diagram of the elements of the watermark embedding system and of the watermark extracting system according to the present invention. In these elements, non-entropy-coded H.264 syntax elements are subjected to techniques of watermarking a digital content in order to convey information, obtaining watermarked non-entropy-coded H.264 syntax elements, which are subsequently coded obtaining a watermarked H.264 video. Later, the conveyed information can be extracted from the watermarked H.264 video, being the authenticity and the sequentiality of the watermarked H.264 video and the conveyed information verified.
In accordance with one aspect of the present invention, there is provided a system and method for watermarking H.264 videos. In accordance with a further aspect of the present invention, there is provided a watermark extracting system and method for authenticating H.264 videos. The watermark embedding system comprises a watermarking and H.264 video stream generation module 100 and an embedding information unit 300, whereas the watermark extracting system comprises a watermark and H.264 video stream decoding module 500 and a decoding information unit 700. After a digital image acquisition system obtains the non-entropy-coded H.264 syntax elements (H.264SE), the latter is processed by the watermarking and H.264 video stream generation module 100, which implements the watermark embedding method disclosed in the present invention. Then, an entropy encoding and H.264 video stream generation is also performed obtaining the watermarked H.264 video (H.264w VIDEO) - The watermarked H.264 video can be saved as a file in a certain storage medium (hard disk, memory card, etc.) or transmitted through a digital network 800 (Internet, Local Area Network, etc.).
The information is coded as an information binary vector b that is embedded by the watermark embedding method according to the present invention. The information binary vector b is computed in the embedding information unit 300 as a function of metadata information, an index of the current frame, data (INFO100) obtained from the watermarking and H.264 video stream generation module 100, and a security vector (e.g., digital signature, etc.) used to validate the information binary vector b . From a functional point of view, the embedding information unit 300 reads and process data and sends the information binary vector b to the watermarking and H.264 video stream generation module 100.
The watermarked H.264 video (H.264w VIDEO) is retrieved from the digital network 800, received watermarked H.264 video (H.264_RW VIDEO), by a watermark and H.264 video stream decoding module 500 that implements the watermark decoding method according to the present invention. In addition, the watermark and H.264 video stream decoding module 500 can be configured to generate an output H.264 video stream showing the results obtained after decoding and processing the watermark. A decoding information unit 700 reads the information binary vector b (the symbol Λ denotes "estimate of"; thus, b denotes the estimated information binary vector b obtained in the decoding process, which may differ from the information binary vector b generated by the embedding information unit 300 in the coding process) from the watermark and H.264 video stream decoding module 500, which generates output information, and determines the authenticity (AUTHENTICITY) of the output information, and decides on the integrity of the watermarked H.264 video. The minimum information generated by the watermark and H.264 video stream decoding module 500 as output is the decision on the authenticity of the analyzed video. Optionally, this module 500 can also output the metadata extracted from the analyzed video (e.g, GPS coordinates, embedding time, etc.) and it also can be configured to generate a video indicating the decision on its authenticity and the extracted information. For example, if the system detects that the received video was temporally modified (i.e., changing the order of the frames) the system can show a visual signal in the output video.
A first aspect of the invention relates to a watermark embedding system and method. FIG. 2 shows the internal scheme of the watermarking and H.264 video stream generation module 100, which comprises a watermark embedding module 110, an entropy coder unit 270, and a H.264 video stream generation unit 290. When non-entropy-coded H.264 syntax elements (H.264SE) are introduced in the watermarking and H.264 video stream generation module 100, the watermark embedding module 1 10 embeds the watermark using the information binary vector b obtained from the embedding information unit 300. Then, the watermarked non- entropy-coded H.264 syntax elements (H.264w_SE) are entropically coded in the entropy coder unit 270. Finally, the resulting entropically coded watermarked H.264 syntax elements (H.264cw SE) pass to the H.264 video stream generation unit 290 that produces a watermarked H.264 video (H.264w VIDEO), which is H.264 compliant. The watermark embedding module 1 10 embeds the watermark (i.e. the information binary vector b ) into the H.264 video sequence using a secret key (K) and the index of the current frame (INDEX), in order to make the watermark embedding secure. Without using said secret key and the correct index, the generation of a valid watermarked H.264 video is a highly complex problem, from the computational point of view, thus making the system secure.
The embedding information unit 300 can read different kind of information (INF010o) from the watermarking and H.264 video stream generation module 100: the output of the information of the watermark embedding module 1 1 0, the output of the entropy coder unit 270, and the output of the H.264 video stream generation unit 290.
FIG. 3 shows the internal scheme of the embedding information unit 300, which comprises a metadata information coding unit 330, a register of the index of the current frame 31 0, a security unit 350, and an embedding information control unit 370.
Metadata information (METADATA) to be embedded is received from the watermarking and H.264 video stream generation module 100. A metadata information vector m (e.g., GPS position coordinates, an identifier of the embedding device, etc.) to be embedded in the H.264 video is obtained in the metadata information coding unit 330. This metadata information vector m can be updated for each frame or can be fixed during the H.264 video processing.
Given the real-time and computational resources constraints, the information that the embedding information control unit 370 passes to the watermarking and H.264 video stream generation module 100 is previously computed, using a set of frames as a basic unit to be validated. For each frame in the set of frames of the H.264 video to be watermarked, the security unit 350 is fed with the metadata information vector to be embedded (which is the output of the metadata information coding unit 330), the index of the current frame (INDEX) from the register of the index of the current frame 310, and information (INFOi00) from the watermarking and H.264 video stream generation module 100, to generate as a function of the secret key K a digest whereby the authenticity of the information can be verified. For each frame in the set of frames, the digest generated by the security unit 350 is stored together with the metadata information, and the index of the current frame in the embedding information control unit 370 as an information binary vector. It is worth pointing out that the information (INFO100) read from the watermarking and H.264 video stream generation module 100 is selected according to a semantic criterion, i.e. it is relevant from the visual point of view. Hence, the syntax elements of the H.264 bitstream are selected: motion vectors, intra prediction modes and residues of the macroblocks. As it is schematically depicted in FIG. 4, which depicts a basic diagram of the processing of a set of four frames, the security unit 350 needs to use the information of the previous watermarked set of frames, i.e., it needs to access the output of the modules 1 10, 270 and 290. The signal INFO100 represents the output of all those modules. As it is shown in FIG. 4, after processing the /-th set of frames (SF,) by the security unit 350, the obtained digest ( digest j) is divided into N digests {digest,^ , digest^, digest/,/v}, where N denotes the number of frames which compose each set of frames (four frames, in the example of FIG. 4). The information binary vector corresponding to the y-th frame of the /+1 -th set of frames (SFi+i) is constructed by arranging in a vector bj the following data: digest,,, (where / is the corresponding index for the current j- frame of the digests of the previous set of frames, /-th set of frames SF,), the metadata m 1 of the current frame y, and the index of the current frame, i.e., j. In addition, the embedding information control unit 370 resets the security unit 350 to an initial state after processing each set of N frames.
In FIG. 4, a basic diagram of the generation of the information binary vector b +I , with l={4,5,6,7}, for sets of frames with N=4, the digest (digest,) of the /-th set of frames SF, is divided into 4 digests (digest^ ; digest,^; digest,^; digest^) and is embedded into the (/+1 )-th set of frames SFi+i (frame #j+4, frame #j+5, frame #j+6, frame #j+7) along with the corresponding metadata information (metadataj+4; metadataj+5; metadataj+6; metadataj+7) and the indices of those frames (j+4; j+5; j+6; j+7). Each information binary vector b1 is embedded by the watermark embedding module 1 10 in the corresponding frame j. Notice that this disclosed framework also includes the case where the set of frames is comprised of only one frame, i.e., Λ/=1 .
FIG. 5 represents the flow chart of the watermark embedding module 1 10, wherein each frame of the non-entropy-coded H.264 syntax elements (H.264SE) is processed. The first step to embed the watermark (information binary vector b1 ) into the current frame j is to pseudorandomly permute as a function of a secret key KQ the information binary vector b1 obtained from the embedding information unit 300 in order to uniformly spread the information over the current frame / (step 1 1 1 ) , although this first step is optional. A parameter P between 0 and 1 is updated in the step 1 13, which is used to pseudorandomly determine, as a function of the secret key K, whether a given macroblock is used for embedding information. The value of the number of remaining coefficients to be embedded is used to initialize R_COEFFS_TO_EMBED (step 1 15), then the value of the total number of bits to embed is used to initialize N_TOTAL_BITS_TO_EMBED (step 1 17) . The value R CO E F FS_TO_E M B E D is used to measure the number of coefficients that can be used to embed the watermark in the worst case scenario. In our application scenario, this value is obtained by assuming that the remaining macroblocks are all 16x16 intra-coded and only one coefficient per macroblock can be used to convey the watermark. Accordingly, we initialize R_COEFFS_TO_EMBED as (width in pixels x height in pixels)/(16 x 16).
Then, variable FORCING is initialized to zero (step 1 19). In step 121 , the comparison between R_COEFFS_TO_EMBED and N TOT AL B I TS_TO_E M B E D is computed; if they have the same value, FORCING is set to one (step 123) . The decision of a current macroblock as a candidate macroblock to convey watermark information is done in step 125 by computing an OR function between a pseudorandomly obtained outcome of a Bernoulli random variable using the parameter P updated in step 1 13 and FORCING. If the result of step 125 is one, the current macroblock is selected as a candidate macroblock (MB) to be watermarked (step 127), a sequence of bits (in this particular embodiment one bit per macroblock is used; however, note that the present algorithm can be configured to use more bits per macroblock) of the information binary vector bj may be embedded in the candidate macroblock and
N_TOTAL_BITS_TO_EMBED is updated by subtracting from its previous value the number of embedded bits (step 129). R_COEFFS_TO_EMBED is updated by subtracting one to its value (step 131 ). The flow of the algorithm goes to step 135 if the value of N_TOTAL_BITS_TO_EMBED is zero (step 133); otherwise, the algorithm goes to step 119.
After embedding the watermark (i.e. the information binary vector b1 ) into the current j- th frame, the algorithm checks in step 135 whether there are SKIP macroblocks that were watermarked in the current frame. If the answer is positive, the embedding algorithm modifies the necessary syntax elements of the set of non-entropy-coded H.264 syntax elements in step 137, in such a way that the skipped macroblocks are converted into new P-macroblocks composed of a single partition, the associated residue macroblock (where the watermark has been embedded), and a corresponding Motion Vector difference (MVd). Said MVd is obtained by computing the difference between the actual motion vector (MV) and a prediction of this, called Motion Vector prediction (MVp). Note that for the skipped macroblock no data is coded other than an indication that the macroblock is to be decoded as "skipped", so the associated MV is directly the MVp.
FIG. 6 shows an example of the neighbouring partitions (i.e., A, B and C) that are considered for the calculation of the MVp either for a skipped macroblock or a P- macroblock. The calculation of the MVp differs for both types of macroblock when some of the neighbouring partitions are not available. For instance, if the partition A is not available, then the MVp (and thus the MV) associated to the skipped macroblock would be inmediately equal to zero independtly of the partitions B and C, while for the P-macroblock the MVp would be calculated in a different way, taking into consideration the other partitions.
Given that the calculation process of the motion vector prediction MVp for skipped macroblocks can differ from the one used for P-macroblocks, the motion vector difference of the new P-macroblock can be used as a bias to compensate possible misalignments between the two motion vectors, in order to obtain the same resulting MV after the conversion. Otherwise, the different reconstruction of the macroblock in the pixel domain, would generate visible distortions that would propagate along the image. For this reason, the MVd associated to each new P-macroblock must be accordingly updated in step 139. Due to the modifications of the non-entropy-coded H.264 syntax elements of the input, the coded block pattern (CBP) is updated by taking into account the introduced modifications (step 141 ), in order to keep compliance with the H.264 standard.
In step 143, the values on the statistics of probability of watermarking a macroblock are updated with the information obtained in step 127: the number of candidate macroblocks and their type, and the number of the watermarked candidate macroblocks and their type.
FIG. 7 shows the flow chart of the conditions that must be checked to watermark a candidate macroblock in step 127. As it was explained before, in step 127 (FIG. 5) the candidate macroblock to convey the watermark is selected for watermark embedding. This candidate macroblock is analyzed in order to decide whether it meets the imperceptibility requirements to be watermarked; in addition, the selection of this candidate can be forced in order to guarantee that the watermark fits onto the current frame.
To explain the selection of the candidate macroblock, reference will be made to FIG. 8, which illustrates a 1 6x 1 6 macroblock partitioned in sixteen 4x4 blocks, and FIG. 9, which illustrates the ordering of the coefficients in a 4x4 block. The selection of a candidate macroblock to convey the watermark will depend on the prediction type of the macroblock. In step 201 , it is checked whether the prediction of the candidate macroblock is intra (I MB); if the answer is positive, then the algorithm goes to step 205. Otherwise, the candidate macroblock is an inter predicted macroblock, and a 4x4 block of the current P-macroblock, BLOCK P, is pseudorandomly selected using as a coefficient one that introduces low embedding distortion COEF P (e.g., CF13 or CF14 following the zigzag order, where the index of each coefficient following this order is shown in FIG. 9) wherein the spatial error drift is unnoticeable. The value of BLOCKS is set to BLOCK_P and the value of COEFS is set to COEF_P in step 203. The selected indices of the coefficients are a parameter previously defined in the system that introduces low embedding distortion; for instance, if the coefficients follow a zigzag order, the last indices introduces low embedding distortion.
In step 205, the type of intra prediction carried out in the candidate macroblock is checked. If the candidate macroblock is a 4x4 intra coded macroblock (14x4) then the algorithm goes to step 209; otherwise, the candidate macroblock is a 1 6x 1 6 intra coded macroblock and a 4x4 block to convey the watermark BLOCK M 6 is pseudorandomly selected without considering the bottom row and right column of the 1 6x 1 6 macroblock (hence, avoiding 4x4 blocks whose indexes are 6, 8, 14, 1 1 , 12, 15, 1 6 in FIG. 8) in order to avoid the drift of the spatial error, and a coefficient that introduces low embedding distortion COEF M 6 (e.g., 13 or 14 following the zigzag order, where the index of each coefficient following this order is shown in FIG. 9) are selected. The value of BLOCKS is set to BLOCK_i16 and the value of COEFS is set to COEF_i16 in step 207.
In step 209, a search of blocks of the candidate macroblock verifying first level conditions is carried out. If there are blocks verifying the first level conditions, the BLOCKS is set to the selected blocks to convey the watermark, BLOCK I st, and COEFS is set to the selected coefficients, COEF I st, in step 211 . The search of blocks to convey the watermark stops if the number of elements of BLOCKS reaches N_TOTAL_BITS_TO_EMBED (i.e., there are not remaining bits of b to be embedded). Otherwise, the candidate macroblock is analyzed in order to find blocks that verify second level conditions in step 213, this step 213 being optional. If there exist blocks verifying those second level conditions, then BLOCKS is set to BLOCK_2nd, indicating that these blocks have been selected to convey the watermark, and COEFS is set to the selected coefficients, COEF_2nd, in step 215. Again, the search of blocks to convey the information stops when the number of elements of BLOCKS is N_TOTAL_BITS_TO_EMBED. If the current macroblock does not have blocks verifying the first level conditions and the second level conditions and FORCING is one (checked in step 217), then the BLOCKS is set to the selected block to convey the watermark, BLOCK FORCING, and COEFS is set to the selected coefficients, COEF FORCING, in step 219. Otherwise, the algorithm ends without watermarking the candidate macroblock. Finally, in step 221 , the selected coefficients (COEFS) of the selected blocks (BLOCKS) of the candidate macroblock are watermarked. There are many embedding methods (such as Spread-Spectrum or Scalar Costa Scheme) that a person skilled in the art may use to embed a watermark. An informed embedding method is explained below.
As indicated before, if the candidate macroblock is a 4x4 intra coded macroblock (14x4) a search of 4x4 blocks of the candidate macroblock verifying first level conditions is carried out in step 209. For the macroblocks whose intra prediction is made in a 4x4 block-wise manner, the content of each 4x4 block is reconstructed by calculating the corresponding spatial prediction (following any of the 9 possible intra prediction modes for 4x4 intra prediction) and adding to said spatial prediction the decoded version of the residue. Therefore, by modifying the residue of a 4x4 block (i.e., when the watermark is embedded), the reconstructed content of the current block will differ from the original one. If this difference affects the border pixels of the block, then the spatial prediction of the successive adjacent blocks will be altered and a distortion will be originated and propagated along the whole decoded picture.
FIG. 10 shows the surrounding pixels (i.e., the set of pixels represented as {A,...,M ) used for the calculation of the intra prediction of a 4x4 block 362 (its contour marked with thick lines). FIG. 11 shows the surrounding pixels (i.e., the vectors H and V) used for the calculation of the intra prediction of a 1 6x 1 6 block 364.
In order to avoid the creation and propagation of this spatial error, the first level conditions are defined for watermarking a candidate block. Given a candidate macroblock 364, the first level conditions that must satisfy a 4x4 block 362 to be watermarked are: 1 ) the 4x4 block must belong to a set of candidate blocks; and 2) the intra prediction performed on any of the successive adjacent 4x4 blocks cannot take as input the reconstructed pixel values of the current block under analysis. The present invention assumes that the intra prediction modes performed on the successive adjacent blocks to the bordering 4x4 blocks (blocks number 6,8,1 1 ,12,14,15,16) of the current 16x16 macroblock 364 are unknown. Therefore the set of candidate 4x4 blocks is defined as {1 ,2,3,4,5,7,9,10,13}, following the labels in the 16x16 macroblock 364 shown in FIG. 8, which depicts the 1 6x 1 6 macroblock 364 partitioned in sixteen 4x4 blocks (blocks numbered from 1 to 1 6). This feature of the present invention allows to perform the processing of the macroblocks in a serial manner.
The evaluation of the first level conditions is carried out for each 4x4 block 362 in the candidate macroblock 364 to be watermarked. If a 4x4 block 362 is compliant with the first level conditions then it is appended to the blocks to watermark (BLOCKS). As stated above, this search of blocks to be watermarked stops when there are no more bits of b left to be embedded in the current frame. Otherwise, if none of the candidates fulfill the conditions, then the current macroblock is signaled as one that does not verify the first level conditions, so that second level conditions are verified in step 213.
For example, given the 4x4 block labeled as 1 in FIG. 8, the intra prediction modes of the adjacent blocks 2, 3, and 4 are checked to verify whether the neighboring pixel values of block 1 are used as input to compute the corresponding predictions. Another example: given the block 1 0, then the adjacent blocks 1 1 , 1 2, 1 3, and 1 5 are checked.
Following an analogous approach to that for the first level conditions, second level conditions are verified in step 213. The conditions for watermarking a 4x4 block are relaxed allowing some spatial drift that will not spread out of the bounds of the candidate macroblock 364.
Given a candidate macroblock 364, the second level conditions that must be satisfied by a 4x4 block 362 to be watermarked are: 1 ) the 4x4 block 362 must belong to the set of candidate blocks { 1 ,2,3,4} labeled according to FIG. 8; and 2) the intra prediction of the contiguous blocks to the previous group of successive adjacent 4x4 blocks (blocks {6,8, 1 1 , 1 2, 14, 1 5, 1 6} in FIG. 8) cannot propagate the originated spatial error, in order to keep the spatial error inside the bounds of the candidate macroblock 364. Note that the intra prediction of the successive adjacent 4x4 blocks (blocks {5,7,9, 1 0, 1 3} in FIG. 8) can be carried out using any intra prediction mode (thus originating spatial errors).
The evaluation of the second level conditions is carried out for each 4x4 block in a candidate macroblock to be watermarked. For each 4x4 block in the set { 1 ,2,3,4}, the chain of intra prediction references is calculated, and each block in the set that has a chain that does not originate spatial errors outside the current macroblock 364 is selected as the one used to embed the watermark. The number of maximum blocks of the candidate macroblock to be watermarked is limited by N_TOTAL_BITS_TO_EMBED.
For example, assuming that a possible block to watermark is block number 1 (according to the diagram of FIG. 8), it is checked whether the block number 2 needs (for the computation of its intra prediction) samples from the block 1 . If this is verified, then none of the neighboring blocks of 2 (i.e., blocks number 3, 4, 5 and 7) can use samples from 2 to compute their intra prediction, in order to maintain the block 1 as a candidate to be watermarked; otherwise, the block 1 is dismissed to embed the watermark. Assuming that the block 1 is still a candidate, the same procedure is followed with the adjacent blocks 3 and 4. If the neighboring blocks of 3 and 4 do not propagate any spatial error, then the block 1 is finally used to embed the watermark; otherwise, one of the remaining blocks in the set {1 ,2,3,4} is analyzed. A forcing embedding mode is used as a mean to guarantee that there is room enough to embed the whole sequence of bits in an l-slice. The forcing embedding mode is used when none of the 4x4 blocks 362 in a candidate macroblock 364 to be watermarked satisfy the first and second level conditions (decision taken in step 217 in FIG. 7). In this situation, the Integer Cosine Transform (ICT) property introduced by Ma et al., which was previously explained, can be used to watermark the set of 4x4 blocks 362 labeled as {6,8,1 1 ,12,14,15,16} in FIG. 8.
The aforementioned ICT property provides a way to watermark a 4x4 block such that the corresponding distortion generated on the spatial domain is located in the two central vertical columns of the block or, in the same manner, in the two central horizontal rows, or even in the 4 central components (CF5, CF8, CF9, CF12) of the block. This goal is achieved by arranging some of the transform coefficients of the 4x4 block into pairs such that by modifying one member of these pairs, the distortion originated can be compensated by changing the other coefficient of the pair in the opposite way.
According to the diagram of the coefficients of the 4x4 block 362 shown in FIG. 9, the pair of coefficients (CF10, CF15) is the one used for embedding the watermark in any of the following 4x4 blocks 362 in the sub-set {6,8,14}. The corresponding pair of coefficients for the subset of blocks {1 1 ,12,15} is (CF7, CF14) and for the block 16, two pairs of coefficients must be adjusted, i.e., (CF4, CF12) and (CF1 , CF6).
As an example, considering the pair of coefficients (CF10, CF15), if by the embedding technique one is added to the coefficient CF10, one has to be subtracted from the value of the coefficient CF15. The accumulation of the spatial error in the aforementioned vertical, horizontal or central region of the 4x4 block due to the forcing embedding produces a noticeable distortion. That is the main reason why this mode is only considered when it is necessary to guarantee that a given number of bits can be embedded in each frame.
As explained before, an informed embedding technique is used to embed a watermark into the non-entropy-coded H.264 syntax elements. In a preferred embodiment a technique based on uniform scalar quantizers is used in step 221 of FIG. 7.
The embedding information unit 300 generates the information binary vector b that is watermarked in the watermark embedding module 110. The value of the coefficient with index / selected to convey a bit c^ is modified allowing to retrieve the bit information bj at the decoder just by processing the value of the watermarked coefficient ci as
Figure imgf000032_0001
where 2Δ stands for a uniform scalar quantizer with step size Delta, and where in the last expression cf^ is an outcome of a binary random variable whose probability mass distribution is
Figure imgf000032_0002
which, in order to minimize the introduced embedding distortion, is pseudorandomly obtained using a pseudorandom number generator initialized with a seed that is function of the index of the current frame and the secret key K. The value of the current processed embedding coefficient with index / is updated with
Ά
In accordance with a further aspect of the present invention there is provided a watermark extracting system and method for authenticating H.264 videos. FIG. 12 shows the internal scheme of the watermark and H.264 video stream decoding module 500, which consists of an entropy decoder 570, and a watermark decoder module 510.
The watermarked H.264 video (H.264_RW VIDEO, which can be obtained, for instance, from the digital network 800) is entropically decoded by the entropy decoder 570, which transfers the watermarked non-entropy-coded H.264 syntax elements (H.264_RW SE) to the watermark decoder module 510. The information binary vector b decoded by the watermark decoder module 510 using the secret key K and the extracted index of the current frame (INDEX) is transferred to the decoding information unit 700, where the metadata m is extracted, the authenticity of the metadata is checked and the temporal authentication is verified. From a structural standpoint, the decoding information unit 700 can read different kind of information (INFO500) from the watermark and H.264 video stream decoding module 500: it can access to the watermarked H.264 video (H.264_RW VIDEO), and the watermarked non-entropy-coded H.264 syntax elements (H.264_RW SE).
FIG. 13 shows the internal scheme of the decoding information unit 700, which consists of a metadata information decoding unit 710, a security unit 730, and a decoding information control unit 750.
For each frame of the /-th set of frames (SF,), the metadata information decoding unit 710 takes as an input the information binary vector b from the watermark decoder and
H.264 video stream decoding module 500 and extracts from it the metadata information m and the index of the frame (INDEX), which are then fed along with the information INFO500 from the watermark and H.264 video stream decoding module 500 to the security unit 730 that generates as a function of the secret key K a calculated security vector or calculated digest (c_digesti), whereby the integrity of this information can be verified. The calculated digest {c_digestt ) generated by the security unit 730 is transferred together with the extracted metadata information m , and the index of the current frame (INDEX) to the decoding information control unit 750. In addition, the decoding information control unit 750 resets (signal RESET) the security unit 730 to an initial state after processing each set of frames. After the last frame of the (/+1 )-th set of frames (SFi+1) is watermark decoded, the decoding information control unit 750 arranges the extracted N digests obtaining a extracted digest of the previous set of frames, i.e., received digest r digest, obtained from the information binary vector b . Then, the decoding information control unit 750 compares the received digest r digest, with the calculated digest c digest, computed from the /-th previous set of frames in the security unit 730. If they match, then the decoding information control unit 750 indicates that the extracted metadata, indices, and the content of the (/)-th set of frames are authentic; otherwise, it points out that they are false. Moreover, from the verified extracted indices, the decoding information control unit 750 decides whether the order of the frames of the watermarked H.264 video was altered. The decision on the authenticity (AUTHENTICITY) and the corresponding extracted metadata are the outputs of the decoding information control unit 750 and also of the decoding information unit 700.
FIG. 14 represents the flow chart of the watermark decoder module 510 shown in FIG. 12, wherein each frame of watermarked non-entropy-coded H.264 syntax elements (H.264_RW SE) is processed.
The first step to extract the watermark from the current frame is the update of a parameter P taking values between 0 and 1 in step 513, which is used to pseudorandomly determine, as a function of the secret key KQ, whether a given macroblock conveys watermark or not. The value of the number of remaining coefficients to read is used to initialize R_COEFFS_TO_READ (step 515), then the value of the total number of bits to be read is used to initialize N_TOTAL_BITS_TO_READ (step 517).
Then, FORCING is initialized to zero (step 519). In step 521 , the comparison between R CO E F FS_TO_R E A D and N_TOTAL_BITS_TO_READ is computed; if they have the same value, then FORCING is set to one (step 523). The decision of selecting a current macroblock as candidate to extract the watermark is carried out in step 525 by computing an OR function between a pseudorandomly obtained outcome of a Bernoulli random variable using the parameter P updated in step 513 and FORCING. If the result of step 525 is one, the current macroblock is selected as a candidate macroblock to extract the watermark (step 527) and N_TOTAL_BITS_TO_READ is updated by subtracting from its previous value the number of extracted bits, if any (step 529). R_COEFFS_TO_READ is updated by subtracting one to its value in step 531. The flow of the algorithm goes to step 511 if the value of N_TOTAL_BITS_TO_READ is zero (step 533); otherwise, the algorithm goes to step 519.
The last step, which is optional but it must be carried out if this step has been done in the embedding process, is the pseudorandom permutation as a function of the secret key Ke oi the extracted N_TOTAL_BITS_TO_READ in step 511 to obtain the extracted information binary vector b1 , which is passed to the decoding information unit 700.
In step 527 of FIG. 14 the current macroblock is selected as a candidate macroblock to extract the watermark. The flow chart of the conditions that must be checked to select the candidate blocks and coefficients of the current macroblock is depicted in FIG. 7, already explained for the embedding method. Similarly to the forcing embedding method already detailed, there is a forcing decoding mode, checked in step 217, when FORCING=1 . In addition, as in the embedding method, given a candidate macroblock the number of selected blocks BLOCKS cannot exceed N_TOTAL_BITS_TO_READ. The watermark extraction algorithm of the watermark decoding method is carried out in step 221 in FIG. 7, wherein the selected coefficients (COEFS) of the selected blocks (BLOCKS) of the candidate macroblock are decoded to extract the watermark. This watermark extraction algorithm is similar to its counterpart watermark embedding algorithm.
Once a coefficient C of a block of a macroblock is selected to extract the y-th bit of the information binary vector bj , the distance to a centroid corresponding to the embedding of zero i/ is calculated as
Figure imgf000035_0001
and the y-th bit is calculated as
0 if vj < Δ/4
1 otherwise. The extracted information binary vector bJ is sent to the decoding information unit 700, which processes the information, extracting the information and verifying the spatial and temporal authenticity of the analyzed watermarked H .264 video. Back to FIG. 5 and FIG. 14, in step 113 (for the watermark embedding module 110) and step 511 (for the watermark decoder module 510), respectively, a parameter P is computed. The objective of selecting the candidate macroblock to be watermarked using a Bernoulli random variable (step 125) is to uniformly distribute the watermark into each frame; thus in order to obtain this, an estimation of the ratio between the number of coefficients to embed and the total number of available coefficients to convey the watermark is necessary; therefore, in the preferred embodiment of the current invention, this ratio is obtained using the obtained information in the watermark embedding and watermark decoding. Two probabilities are calculated: one probability for l-slices p^ and another P-slices pp. The value of these two probabilities is calculated as totaLnuiriber_bits-to embed
Pi
«14x4 X „ , „ + "116x 16 X —
totaLnuniber.bits _to embed
PP = i ¾ where in the previous expressions, denotes the number of 4x4 intra predicted macroblocks, denotes the number of watermarked 4x4 intra predicted macroblocks, denotes the number of candidate 4x4 intra predicted macroblocks, π/1 6χ 1 6 denotes the number of 1 6x 1 6 intra predicted macroblocks, n|/|/1 gx 1 g denotes the number of watermarked 1 6x 1 6 intra predicted macroblocks, nc1 gx 1 g denotes the number of candidate 1 6x 1 6 intra predicted macroblocks, rip denotes the number of inter predicted macroblocks, nwp denotes the number of watermarked inter predicted macroblocks, and nQp denotes the number of candidate P macroblocks to watermark.
The used values for the parameter P are obtained from the previous frame used, i.e., previous I slice for ^, and previous P slice for pp, where P takes the corresponding value according to the type of current slice (l-slice or P-slice) to be watermarked. FIG. 15 shows a scheme of elements representing a possible implementation of the described embedding method that can be embodied, for instance, in a digital camera 1500. The digital camera includes a lens 1501 that focalizes an image on an image sensor 1502, an H.264 encoder 1503 that uses the flow of images captured by the image sensor 1502 to produce a video stream which is compliant with the H.264 standard. A control circuit 1504 is in charge of the acquisition of the flow of images, the H.264 video stream generation and, optionally, the communication with a digital network 1506. Moreover, the control circuit 1504 has the capacity to execute operations stored in a memory 1505, where the necessary instructions for the implementation of the insertion method proposed in this invention are saved.
FIG. 16 illustrates a possible implementation of the described extraction method in a computational system 1600 for the purpose of exposing forgeries. The computational system comprises a processor 1601 and a memory 1602. The processor 1601 is connected to a digital network 1603 to obtain H.264 bitstreams from camera devices 1604 connected to the digital network 1603. The processor 1601 performs the extraction method by executing the operations stored in the memory 1602. Therefore, the described processes to perform the extraction of the hidden information and detect manipulations can be implemented in these computational systems.
Note that the flow charts shown in the previous figures do not use any special syntax nor any special programming language. Moreover, they represent the information necessary so that a person skilled in the art can manufacture integrated circuits or generate the software that executes the necessary processes. For example, every function represented by a block or a flow chart can be implemented by a set of software instructions, by a DSP digital signal processor, by a FPGA configurable digital circuit, by a specific ASIC application circuit or any combination thereof. Different embodiments of the present invention have been detailed. It is evident for a person skilled in the art that there exist variants in the state of the art applicable to the practical embodiments presented.

Claims

1 . System for embedding a digital watermark in a H.264 video, comprising a watermarking and H.264 video stream generation module (100) with a watermark embedding module (1 10) for embedding a watermark in non-entropy-coded H.264 syntax elements (H.264SE) of the H.264 video, obtaining watermarked non-entropy- coded H.264 syntax elements (H.264W_SE);
characterized in that the watermark embedding module (1 10) comprises processing means configured for processing each frame / of the non-entropy-coded H.264 syntax elements (H.264SE) to:
- pseudorandomly determine, for each macroblock and until there are no remaining bits of the watermark to be embedded in the corresponding frame y, whether a given macroblock is a candidate macroblock for conveying the watermark, and in that case check whether a determined set of conditions to watermark at least one selected coefficient (COEFS) of at least one selected block (BLOCKS) of the candidate macroblock are met, said set of conditions including at least first level conditions which are met when:
the candidate macroblock is a 4x4 intra coded macroblock (14x4);
the selected block of the candidate macroblock to convey the watermark belongs to a predefined set of candidate blocks and the intra prediction modes of its successive adjacent blocks do not take as input the value of the selected block;
- embed the watermark in the at least one selected coefficient (COEFS) of the selected block (BLOCKS) meeting the set of conditions.
2. System for embedding a digital watermark in a H.264 video according to claim 1 , wherein the predefined set of candidate blocks in the first level conditions comprises the nine blocks contained within the first three rows and the first three columns of the candidate macroblock.
3. System for embedding a digital watermark in a H.264 video according to any of previous claims, wherein the set of conditions also includes second level conditions to be checked when the first level conditions are not met, said second level conditions being met when:
the candidate macroblock is a 4x4 intra coded macroblock (14x4); the selected block of the candidate macroblock to convey the watermark belongs to a set of candidate blocks which comprises the four blocks contained within the first two rows and the first two columns of the candidate macroblock, and the intra prediction modes are such that a modification in the selected block does not propagate any spatial error to any of the seven blocks located in the fourth row or in fourth column of the macroblock.
4. System for embedding a digital watermark in a H.264 video according to any of previous claims, wherein the set of conditions also includes forcing embedding conditions to be checked, said forcing embedding conditions being met when the total number of bits to embed (N_TOTAL_BITS_TO_EMBED) in the corresponding frame j is equal to the remaining macroblocks of the frame j which have not yet being analyzed for candidate macroblock purposes.
5. System for embedding a digital watermark in a H.264 video according to any of previous claims, wherein the set of conditions includes checking whether the candidate macroblock is an inter predicted macroblock, and in that case selecting at least one block (BLOCK P) of said macroblock and at least one coefficient (COEF P) of said block (BLOCK_P).
6. System for embedding a digital watermark in a H.264 video according to any of previous claims, wherein the set of conditions includes checking whether the candidate macroblock is a 16x16 intra predicted macroblock, and in that case selecting at least one block (BLOCK M 6) of said macroblock and at least one coefficient (COEF P) of said block (BLOCK_i16).
7. System for embedding a digital watermark in a H.264 video according to any of previous claims, wherein the processing means are configured to pseudorandomly determine whether a given macroblock is a candidate macroblock for conveying the watermark by computing a pseudorandomly obtained outcome of a Bernoulli random variable using a parameter P previously updated using a pseudorandom number generator initialized with a seed that is function of the index of the current frame / and a secret key K.
8. System for embedding a digital watermark in a H.264 video according to any of previous claims, wherein the watermarking and H.264 video stream generation module (100) further comprises:
an entropy coder unit (270) for entropically coding the watermarked non- entropy-coded H.264 syntax elements (H.264w_SE), obtaining entropically coded watermarked H.264 syntax elements (H.264cw_SE), and
a H.264 video stream generation unit (290) for producing a watermarked H.264 video (H.264w VIDEO)-
9. System for embedding a digital watermark in a H.264 video according to any of previous claims, further comprising an embedding information unit (300) configured to obtain a sequence of bits as the watermark to be embedded.
10. System for embedding a digital watermark in a H.264 video according to claim 9, wherein the embedding information unit (300) is configured to obtain, for each frame y of the H.264 video stream, an information binary vector bj as a function of metadata information (METADATA), an index of the current frame (INDEX), data (INFO100) obtained from the watermarking and H.264 video stream generation module (100), and a security vector (digestj) used to validate a set of frames (SF,).
1 1 . System for embedding a digital watermark in a H.264 video according to claim 10, wherein the embedding information unit (300) comprises:
a metadata information coding unit (330) for obtaining a metadata information vector m1 for each frame y;
a security unit (350) for generating the security vector (digestj) as a function of a secret key K, by using the metadata information vectors m1 of a set of frames (SF,) comprising at least one frame, their corresponding indices (INDEX), and data (INFOi00) obtained from the watermarking and H.264 video stream generation module (100), an embedding information control unit (370) for obtaining the information binary vector b 1 for each frame y.
12. System for embedding a digital watermark in a H.264 video according to claim 1 1 , wherein the embedding information unit (300) is configured to divide the security vector (digestj), obtained for each set of frames (SF,), into N vectors {digest/ 1 , digest^, digest ,/ν}, where N denotes the number of frames which compose each set of frames and wherein the information binary vector bJ corresponding to the frame of the set of frames i+ 1 (SFi+1) comprises:
the metadata information vector m1 for said frame y,
the corresponding index (y), and
the security vector (digest,,), where / is the corresponding index for the /-th chunk of the digest■ of the previous set of frames /' (SF,).
13. System for embedding a digital watermark in a H.264 video according to any of claims 10 to 12, wherein the data (INFOi00) obtained from the watermarking and H.264 video stream generation module (100) comprises syntax elements of the H.264 video stream.
14. System for embedding a digital watermark in a H.264 video according to any of claims 10 to 13, wherein the metadata information (METADATA) comprises at least one of the following data:
GPS position coordinates;
an identifier of the embedding device;
embedding time.
15. System for embedding a digital watermark in a H.264 video according to any of claims 9 to 14, wherein the embedding information unit (300) is configured to generate an information binary vector b used as watermark, such that the value of the coefficient with index / selected to convey a bit Cj is modified by the watermark embedding module (1 10), allowing to retrieve the bit information at the decoder by processing the value of the watermarked coefficient c, as
Figure imgf000041_0001
where 2Δ stands for a uniform scalar quantizer with step size Delta, and where dj \s an outcome of a binary random variable whose probability mass distribution is
Figure imgf000042_0001
which is pseudorandomly obtained using a pseudorandom number generator initialized with a seed that is function of the index of the current frame and a secret key K.
16. System for generating a digital watermark for embedding in a H.264 video, characterized in that it comprises an embedding information unit (300) configured to obtain, for each frame of the non-entropy-coded H.264 syntax elements (H.264SE) of the H.264 video, an information binary vector b 1 as the watermark to be embedded, said vector obtained as a function of metadata information (METADATA), an index of the current frame (INDEX), syntax elements of the previous frames of the H.264 video stream, and a security vector (digestj) used to validate a set of frames (SF,).
17. System for generating a digital watermark for embedding in a H.264 video according to claim 16, wherein the embedding information unit (300) comprises:
a metadata information coding unit (330) for obtaining a metadata information vector m 1 for each frame y;
a security unit (350) for generating the security vector (digestj) as a function of a secret key K, by using the metadata information vectors m 1 of a set of frames (SF,) comprising at least one frame, their corresponding indices (INDEX), and syntax elements of the previous frames of the H.264 video stream,
an embedding information control unit (370) for obtaining the information binary vector b j for each frame j.
18. System for generating a digital watermark for embedding in a H.264 video according to claim 17, wherein the embedding information unit (300) is configured to divide the security vector (digestj), obtained for each set of frames (SF,), into N vectors
{digest/1 , digest^, digest/,w}, where N denotes the number of frames which compose each set of frames (SF,),
and wherein the information binary vector b 1 corresponding to the frame j of the set of frames i+ 1 (SFi+1) comprises:
the metadata information vector m 1 for said frame j, the corresponding index (J), and
the security vector (digest,,/), where / is the corresponding index for the current frame / of the security vector (digestj) of the previous set of frames / (SF,).
19. System for generating a digital watermark for embedding in a H.264 video according to any of claims 16 to 18, wherein the metadata information (METADATA) comprises at least one of the following data:
GPS position coordinates;
an identifier of the embedding device;
embedding time.
20. System for decoding a digital watermark in a H.264 video, characterized in that it comprises a watermark and H.264 video stream decoding module (500) with a watermark decoder module (510) for decoding, from each frame / of the watermarked non-entropy-coded H.264 syntax elements (H.264_RW SE) of the H.264 video
(H.264_RW VIDEO), an information binary vector b} which contains a metadata information vector m 1 , the index of the current frame / and a security vector (r_digestij), where / is the corresponding index for the /-th chunk of the r digestj Oi the previous set of frames / (SF,), used to validate said previous set of frames (SF,).
21 . System for authenticating a H.264 video using a digital watermark, characterized in that it comprises a watermark and H.264 video stream decoding module (500) with a watermark decoder module (510) for decoding, from each frame / of the watermarked non-entropy-coded H.264 syntax elements (H.264_RW SE) of the H.264 video (H.264_RW VIDEO), an information binary vector b} which contains a metadata information vector m 1 , the index of the current frame / and a security vector (r_digestij), where / is the corresponding index for the /-th chunk of the r digestj Oi the previous set of frames / (SF,), used to validate said previous set of frames (SF,);
and in that the system further comprises a decoding information unit (700) configured to:
obtain, for each set of frames (SF,), a calculated security vector (c_digesti) as a function of a secret key K and using the metadata information vectors m 1 of a set of frames (SF,) comprising at least one frame, their corresponding indices (INDEX), and data (INFO500) obtained from the watermark and H.264 video stream decoding module (500);
compare the security vector (r_ digestj) with the calculated security vector
(c_digesti) so that if they match the corresponding set of frames (SF,) is considered as authentic.
22. System for authenticating a H.264 video using a digital watermark according to claim 21 , wherein the watermark decoder module (510) is configured to extract the information binary vector b using a secret key K and the index of the current frame (INDEX).
23. Method for embedding a digital watermark in a H.264 video, comprising embedding a watermark in non-entropy-coded H.264 syntax elements (H.264SE) of the H.264 video, obtaining watermarked non-entropy-coded H.264 syntax elements (H.264W_SE); characterized in that it further comprises processing each frame / of the non-entropy- coded H.264 syntax elements (H.264SE) to:
- pseudorandomly determine, for each macroblock and until there are no remaining bits of the watermark to be embedded in the corresponding frame y, whether a given macroblock is a candidate macroblock for conveying the watermark, and in that case check whether a determined set of conditions to watermark at least one selected coefficient (COEFS) of at least one selected block (BLOCKS) of the candidate macroblock are met, said set of conditions including at least first level conditions which are met when:
the candidate macroblock is a 4x4 intra coded macroblock (14x4);
the selected block of the candidate macroblock to convey the watermark belongs to a predefined set of candidate blocks and the intra prediction modes of its successive adjacent blocks do not take as input the value of the selected block;
- embed the watermark in the at least one selected coefficient (COEFS) of the selected block (BLOCKS) meeting the set of conditions.
24. Method for embedding a digital watermark in a H.264 video according to claim 23, wherein the predefined set of candidate blocks in the first level conditions comprises the nine blocks contained within the first three rows and the first three columns of the candidate macroblock
25. Method for embedding a digital watermark in a H.264 video according to any of claims 23 to 24, wherein the set of conditions also includes second level conditions to be checked when the first level conditions are not met, said second level conditions being met when:
the candidate macroblock is a 4x4 intra coded macroblock (14x4);
the selected block of the candidate macroblock to convey the watermark belongs to a set of candidate blocks which comprises the four blocks contained within the first two rows and the first two columns of the candidate macroblock, and the intra prediction modes are such that a modification in the selected block does not propagate any spatial error to any of the seven blocks located in the fourth row or in fourth column of the macroblock.
26. Method for embedding a digital watermark in a H.264 video according to any of claims 23 to 25, wherein the set of conditions also includes forcing embedding conditions to be checked, said forcing embedding conditions being met when the total number of bits to embed (N_TOTAL_BITS_TO_EMBED) in the corresponding frame j is equal to the remaining macroblocks of the frame j which have not yet being analyzed for candidate macroblock purposes.
27. Method for embedding a digital watermark in a H.264 video according to any of claims 23 to 26, wherein the set of conditions includes checking whether the candidate macroblock is an inter predicted macroblock, and in that case selecting at least one block (BLOCK P) of said macroblock and at least one coefficient (COEF P) of said block (BLOCK_P).
28. Method for embedding a digital watermark in a H.264 video according to any of claims 23 to 27, wherein the set of conditions includes checking whether the candidate macroblock is a 16x16 intra predicted macroblock, and in that case selecting at least one block (BLOCK M 6) of said macroblock and at least one coefficient (COEF P) of said block (BLOCK_i16).
29. Method for embedding a digital watermark in a H.264 video according to any of claims 23 to 28, further comprising pseudorandomly determining whether a given macroblock is a candidate macroblock for conveying the watermark by computing a pseudorandomly obtained outcome of a Bernoulli random variable using a parameter P previously updated using a pseudorandom number generator initialized with a seed that is function of the index of the current frame / and a secret key K.
30. Method for embedding a digital watermark in a H.264 video according to any of claims 23 to 29, further comprising:
entropically coding the watermarked non-entropy-coded H.264 syntax elements (H.264w_SE), obtaining entropically coded watermarked H.264 syntax elements
Figure imgf000046_0001
producing a watermarked H.264 video (H.264w VIDEO) -
31 . Method for embedding a digital watermark in a H.264 video according to any of claims 23 to 30, further comprising obtaining a sequence of bits as the watermark to be embedded.
32. Method for embedding a digital watermark in a H.264 video according to claim 31 , further comprising obtaining, for each frame / of the H.264 video stream, an information binary vector b1 as a function of metadata information (METADATA), an index of the current frame (INDEX), data (INFOi00) previously obtained, and a security vector (digest j) used to validate the information binary vector b 1 .
33. Method for embedding a digital watermark in a H.264 video according to claim 32, further comprising:
obtaining a metadata information vector m1 for each frame /;
generating the security vector (digestj) as a function of a secret key K, by using the metadata information vectors m 1 of a set of frames (SF,) comprising at least one frame, their corresponding indices (INDEX), and data (INFOi00) previously obtained, obtaining the information binary vector b 1 for each frame /.
34. Method for embedding a digital watermark in a H.264 video according to claim 33, further comprising dividing the security vector (digestj), obtained for each set of frames
(SF,), into N vectors {digest/ 1 , digest^, digest,^}, where N denotes the number of frames which compose each set of frames (SF,), and wherein the information binary vector bJ corresponding to the frame of the set of frames i+ 1 (SFi+i) comprises:
the metadata information vector m1 for said frame j,
the corresponding index (j), and
the security vector (digest,,/), where / is the corresponding index for the /-th chunk of the digestj Oi the previous set of frames / (SF,).
35. Method for embedding a digital watermark in a H.264 video according to any of claims 32 to 34, wherein the data (INFOi00) previously obtained comprises syntax elements of the H.264 video stream.
36. Method for embedding a digital watermark in a H.264 video according to any of claims 32 to 35, wherein the metadata information (METADATA) comprises at least one of the following data:
GPS position coordinates;
an identifier of the embedding device;
embedding time.
37. Method for embedding a digital watermark in a H.264 video according to any of claims 32 to 36, wherein the information binary vector b used as watermark is such that the value of the coefficient with index /' selected to convey a bit Cj is modified allowing to retrieve the bit information at the decoder by processing the value of the watermarked coefficient ci as
Figure imgf000047_0001
where 2Δ stands for a uniform scalar quantizer with step size Delta, and where dj \s an outcome of a binary random variable whose probability mass distribution is which is pseudorandomly obtained using a pseudorandom number generator initialized with a seed that is function of the index of the current frame and a secret key K.
38. Method for generating a digital watermark for embedding in a H.264 video, characterized in that it comprises obtaining, for each frame j of the non-entropy-coded
H.264 syntax elements (H.264SE) of the H.264 video, an information binary vector b1 as the watermark to be embedded, said vector obtained as a function of metadata information (METADATA), an index of the current frame (INDEX), syntax elements of the previous frames of the H.264 video stream, and a security vector (digest j) used to validate a set of frames (SF,).
39. Method for generating a digital watermark for embedding in a H.264 video according to claim 38, further comprising:
obtaining a metadata information vector m1 for each frame y;
generating the security vector (digestj) as a function of a secret key K, by using the metadata information vectors m 1 of a set of frames (SF,) comprising at least one frame, their corresponding indices (INDEX), and syntax elements of the previous frames of the H.264 video stream,
obtaining the information binary vector b j for each frame j.
40. Method for generating a digital watermark for embedding in a H.264 video according to claim 39, further comprising dividing the security vector (digestj), obtained for each set of frames (SF,), into N vectors {digest,^ , digest,^, digest,^}, where N denotes the number of frames which compose each set of frames (SF,),
and wherein the information binary vector b 1 corresponding to the frame j of the set of frames i+ 1 (SFi+1) comprises:
the metadata information vector m 1 for said frame y,
the corresponding index (j), and
the security vector (digest, ,), where / is the corresponding index for the current frame j of the security vector (digestj) of the previous set of frames /' (SF,).
41 . Method for generating a digital watermark for embedding in a H.264 video according to any of claims 38 to 40, wherein the metadata information (METADATA) comprises at least one of the following data:
GPS position coordinates;
an identifier of the embedding device;
embedding time.
42. Method for decoding a digital watermark in a H.264 video, characterized in that it comprises decoding, from each frame / of the watermarked non-entropy-coded H.264 syntax elements (H.264_RW_SE) of the H.264 video (H.264_RW VIDEO), an information binary vector b1 which contains a metadata information vector m1 , the index of the current frame / and a security vector {r diges , where / is the corresponding index for the /-th chunk of the r digestj Oi the previous set of frames /' (SF,), used to validate said previous set of frames (SF,).
43. Method for authenticating a H.264 video using a digital watermark, characterized in that it comprises:
decoding, from each frame / of the watermarked non-entropy-coded H.264 syntax elements (H.264w SE) of the H.264 video, an information binary vector b} which contains a metadata information vector m1 , the index of the current frame / and a security vector {r digest, ), where / is the corresponding index for the /-th chunk of the r digestj Oi the previous set of frames /' (SF,), used to validate said previous set of frames (SF,);
obtaining, for each set of frames (SF,), a calculated security vector (c_digesti) as a function of a secret key K and using the metadata information vectors m1 of a set of frames (SF,) comprising at least one frame, their corresponding indices (INDEX), and data (INFO500) previously obtained;
comparing the security vector (r_digestj) with the calculated security vector
{c digestjj so that if they match the corresponding set of frames (SF,) is considered as authentic.
44. Method for authenticating a H.264 video using a digital watermark according to claim 43, further comprising extracting the information binary vector b using a secret key K and the index of the current frame (INDEX).
PCT/EP2013/068067 2013-09-02 2013-09-02 Method and system for embedding information and authenticating a h.264 video using a digital watermark WO2015028098A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/EP2013/068067 WO2015028098A1 (en) 2013-09-02 2013-09-02 Method and system for embedding information and authenticating a h.264 video using a digital watermark

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2013/068067 WO2015028098A1 (en) 2013-09-02 2013-09-02 Method and system for embedding information and authenticating a h.264 video using a digital watermark

Publications (1)

Publication Number Publication Date
WO2015028098A1 true WO2015028098A1 (en) 2015-03-05

Family

ID=49085035

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2013/068067 WO2015028098A1 (en) 2013-09-02 2013-09-02 Method and system for embedding information and authenticating a h.264 video using a digital watermark

Country Status (1)

Country Link
WO (1) WO2015028098A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3151189A1 (en) * 2015-10-02 2017-04-05 ContentArmor Method for determining a modifiable block
WO2018177522A1 (en) * 2017-03-29 2018-10-04 Fundació Per A La Universitat Oberta De Catalunya Video watermarking
CN110072109A (en) * 2019-04-25 2019-07-30 四川大学 Macro-block level minimizes the H.264/AVC Video Information Hiding Algorithms of distortion
CN110248193A (en) * 2019-07-12 2019-09-17 四川大学 A kind of reversible information hidden method based on improvement difference expansion
US10645419B2 (en) 2016-05-10 2020-05-05 Nxp Usa, Inc. System encoder and decoder for verification of image sequence

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040120544A1 (en) * 2002-09-10 2004-06-24 Canon Kabushiki Kaisha Digital-watermark embedding method and device and storage medium for storing the method
WO2013112744A2 (en) * 2012-01-24 2013-08-01 Futurewei Technologies, Inc. Compressed domain watermarking with reduced error propagation

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040120544A1 (en) * 2002-09-10 2004-06-24 Canon Kabushiki Kaisha Digital-watermark embedding method and device and storage medium for storing the method
WO2013112744A2 (en) * 2012-01-24 2013-08-01 Futurewei Technologies, Inc. Compressed domain watermarking with reduced error propagation

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
NOORKAMI M ET AL: "Digital Video Watermarking in P-Frames With Controlled Video Bit-Rate Increase", IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, IEEE, PISCATAWAY, NJ, US, vol. 3, no. 3, 1 September 2008 (2008-09-01), pages 441 - 455, XP011328855, ISSN: 1556-6013, DOI: 10.1109/TIFS.2008.923825 *
SAMPATH K BAVIPATI ET AL: "Secure Compressed Domain Watermarking for H.264 Video", INFORMATION TECHNOLOGY: NEW GENERATIONS (ITNG), 2010 SEVENTH INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 12 April 2010 (2010-04-12), pages 387 - 391, XP031702625, ISBN: 978-1-4244-6270-4 *
YUNXIA LIU ET AL: "A novel data hiding scheme for H.264/AVC video streams without intra-frame distortion drift", COMMUNICATION TECHNOLOGY (ICCT), 2012 IEEE 14TH INTERNATIONAL CONFERENCE ON, IEEE, 9 November 2012 (2012-11-09), pages 824 - 828, XP032390372, ISBN: 978-1-4673-2100-6, DOI: 10.1109/ICCT.2012.6511318 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3151189A1 (en) * 2015-10-02 2017-04-05 ContentArmor Method for determining a modifiable block
US10645419B2 (en) 2016-05-10 2020-05-05 Nxp Usa, Inc. System encoder and decoder for verification of image sequence
WO2018177522A1 (en) * 2017-03-29 2018-10-04 Fundació Per A La Universitat Oberta De Catalunya Video watermarking
US11494866B2 (en) 2017-03-29 2022-11-08 Fundació Per A La Universitat Oberta De Catalunya Video watermarking
CN110072109A (en) * 2019-04-25 2019-07-30 四川大学 Macro-block level minimizes the H.264/AVC Video Information Hiding Algorithms of distortion
CN110072109B (en) * 2019-04-25 2021-04-09 四川大学 H.264/AVC video information hiding method for macroblock-level minimized distortion
CN110248193A (en) * 2019-07-12 2019-09-17 四川大学 A kind of reversible information hidden method based on improvement difference expansion

Similar Documents

Publication Publication Date Title
Fallahpour et al. Tampering detection in compressed digital video using watermarking
Dalal et al. A survey on information hiding using video steganography
CN110708557B (en) Video decoding method and device in decoder, and computer device
Patel et al. Study and investigation of video steganography over uncompressed and compressed domain: a comprehensive review
WO2015028098A1 (en) Method and system for embedding information and authenticating a h.264 video using a digital watermark
Farfoura et al. Low complexity semi-fragile watermarking scheme for H. 264/AVC authentication
Joshi et al. Real time implementation of digital watermarking algorithm for image and video application
Horng et al. A low cost fragile watermarking scheme in H. 264/AVC compressed domain
Tew et al. Multi-layer authentication scheme for HEVC video based on embedded statistics
Lin et al. Recovering sign bits of DCT coefficients in digital images as an optimization problem
JP5080304B2 (en) Display method of image data with confidential data inserted
Zhang et al. A video watermarking algorithm of H. 264/AVC for content authentication
JP4684701B2 (en) Scramble processing device, descramble processing device, scramble processing method, and descramble processing method
Su et al. A privacy protection scheme in H. 264/AVC by data hiding
JP4165752B2 (en) Secret data insertion method and secret data detection method for image data
Lo et al. Generic attacks on content-based video stream authentication
Ait Sadi et al. Content fragile watermarking for H. 264/AVC video authentication
Saadi et al. Combined fragile watermark and digital signature for H. 264/AVC video authentication
Wei et al. Trustworthy authentication on scalable surveillance video with background model support
WO2013112744A2 (en) Compressed domain watermarking with reduced error propagation
JP4268377B2 (en) Digital watermark embedding device, its detection device, and digital watermark embedding program
Uddin et al. Deep learning-based counter anti-forensic of GAN-based attack in HEVC compressed domain using coding pattern analysis
Zhang et al. Object coding based video authentication for privacy protection in immersive communication
Chaumont Ensuring security of H. 264 videos by using watermarking
Dalbouchi et al. Software/Hardware implementations of a video watermarking scheme based on motion vectors

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13756153

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13756153

Country of ref document: EP

Kind code of ref document: A1