CN114677257A

CN114677257A - Watermark embedding, extracting and comparing method and system based on 3D video

Info

Publication number: CN114677257A
Application number: CN202210313703.4A
Authority: CN
Inventors: 刘熙尧; 王佳钰; 陈志鸿; 张健; 贺建飙; 张伟; 方辉
Original assignee: Central South University
Current assignee: Central South University
Priority date: 2022-03-28
Filing date: 2022-03-28
Publication date: 2022-06-28

Abstract

The invention relates to a watermark embedding, extracting and comparing method and system based on a 3D video, relating to the technical field of information security. Respectively blocking each two-dimensional video frame to be embedded into the three-dimensional video to obtain a plurality of sub-blocks; carrying out multi-level DT-CWT decomposition on any component in the subblock color space to obtain a subband of the subblock under each level of decomposition; modifying the amplitude matrix of the sub-band according to the watermark to be embedded to obtain a modified sub-band; carrying out inverse DT-CWT operation on the modified sub-band to obtain an embedded watermark sub-block corresponding to the sub-block; splicing all embedded watermark sub-blocks corresponding to the two-dimensional video frame to obtain a watermark embedded two-dimensional video frame corresponding to the two-dimensional video frame; and embedding all watermarks corresponding to the three-dimensional video to be embedded into the two-dimensional video frame to obtain a watermark embedded video and a zero watermark of the three-dimensional video to be embedded. The invention can further improve the accuracy of copyright protection on the premise of ensuring the high robustness of the watermark.

Description

Watermark embedding, extracting and comparing method and system based on 3D video

Technical Field

The invention relates to the technical field of information security, in particular to a watermark embedding, extracting and comparing method and system based on a 3D video.

Background

With the aging of three-dimensional display technology, the threshold for spreading and manufacturing 3D products is gradually reduced, and the three-dimensional display technology is widely applied in many fields, such as movies, games, remote education, medical treatment, and the like, and common consumers can enjoy 3D technology at home. As an information carrier, on one hand, 3D video can bear richer information amount than 2D video and can provide more novel visual experience for audiences, and on the other hand, the manufacturing cost of 3D video is higher, and illegal copying and illegal use of 3D video will bring more serious loss than 2D video to copyright owners. The DIBR (Depth-image-based-Rendering) method is a low-cost technique for generating a 3D video, and performs pixel horizontal displacement and hole filling on a series of 2D video frames and corresponding Depth images to obtain virtual left and right stereo images, thereby making a 3D video frame. The DIBR method can generate new forms of media content and can be used as a tool for evaluating the robustness of watermarks. However, since the 3D video generated by the DIBR method comes from two carriers, i.e. the original two-dimensional frame and the depth map, it also brings difficulty to copyright protection of the 3D video. And the frame synthesized after the DIBR operation has a horizontal pixel shift relative to the original two-dimensional frame, which causes a certain distortion to the image content, which means that the 3D video protection scheme should be DIBR-invariant, and the content source to be protected is more complex than the conventional two-dimensional video, thus having a certain challenge. And the copyright protection research method for two-dimensional multimedia technology products starts earlier and becomes mature day by day, so the research on copyright protection of 3D videos has more practical significance and value, and the copyright protection problem of the 3D videos also becomes a hotspot problem in the digital media copyright protection field in recent years.

In order to effectively protect the copyright of video, the protection methods can be divided into two categories:

watermark embedding: by utilizing the redundancy of the carrier, information such as text, logo, image and the like is embedded in the carrier needing to be protected, and if watermarks can be extracted from the carrier subsequently, copyright can be proved. The general method includes a space domain and a frequency domain, and in the space domain, the classical method is a Least Significant Bit (LSB) method and a histogram method, for example, the histogram method is to construct a histogram for a two-dimensional video frame, and then select a pixel range with a proper size according to the histogram to embed a watermark; in the frequency domain, the two-dimensional video frame is decomposed by applying methods such as discrete Fourier transform, discrete wavelet transform, dual-tree complex wavelet transform and the like, and the coefficient of the low-frequency sub-band is slightly changed by combining the watermark through a reversible quantization method to realize embedding.

And (3) zero watermark characteristic construction: the zero watermark scheme does not directly embed the watermark into the three-dimensional video content, but generates the relation between the three-dimensional video characteristics and the watermark, thereby ensuring that the three-dimensional video has no direct information embedding, and further not causing any influence on the content of the three-dimensional video. A common method is to convert a two-dimensional video frame into a frequency domain, and format wavelet coefficients therein for binarization; or calculating the moment results of the left and right stereo images based on various moments, such as Zernike moments, radial harmonic Fourier moments, Chebyshev octave moments and the like, extracting the characteristics of the two frames of stereo images and constructing a hash value for the characteristics.

However, the above prior art has the following disadvantages:

(1) although the existing digital watermarking scheme based on two-dimensional frames has robustness and differentiability which can meet the requirements, the digital watermarking scheme based on two-dimensional frames embeds information in video frames, which inevitably destroys the association between pixels, causes irreversible distortion to the video content, and is not allowed for the field with high precision requirement.

(2) The zero watermark algorithm based on non-direct information embedding can generate a mapping relation between watermarks representing video characteristics and copyrighted information, so that 3D video copyright protection is not required to be performed by directly embedding watermarks in video frames, and compared with the traditional watermark algorithm, the zero watermark protection scheme has no influence on the quality of video contents, is superior to the watermark embedding scheme in terms of video quality and watermark robustness, but can not realize better distinguishability under the condition of ensuring enough robustness, namely the robustness and distinguishability performance are still difficult to meet the requirements.

Disclosure of Invention

The invention aims to provide a 3D video-based watermark embedding, extracting and comparing method and system, which can further improve the accuracy of copyright protection on the premise of ensuring the high robustness of a watermark.

In order to achieve the purpose, the invention provides the following scheme:

a watermark embedding method based on 3D video comprises the following steps:

respectively blocking each two-dimensional video frame to be embedded into the three-dimensional video to obtain a plurality of sub-blocks;

for any subblock in any two-dimensional video frame, carrying out multi-level DT-CWT decomposition on any component in the subblock color space to obtain a sub-band of the subblock under each level of decomposition;

modifying the amplitude matrix of the sub-band obtained by decomposing the sub-blocks in a set series range according to the watermark to be embedded to obtain a modified sub-band;

carrying out inverse DT-CWT operation on the modified sub-band to obtain an embedded watermark sub-block corresponding to the sub-block;

splicing all the embedded watermark sub-blocks corresponding to the two-dimensional video frame to obtain a watermark embedded two-dimensional video frame corresponding to the two-dimensional video frame;

and embedding all watermarks corresponding to the three-dimensional video to be embedded into the two-dimensional video frame to obtain a watermark embedded video and a zero watermark of the three-dimensional video to be embedded.

Optionally, embedding the two-dimensional video frame with all watermarks corresponding to the three-dimensional video to be embedded to obtain a zero watermark of the three-dimensional video to be embedded specifically includes:

respectively blocking each watermark embedding two-dimensional video frame corresponding to the three-dimensional video to be embedded to obtain a plurality of watermark sub-blocks;

for any watermark sub-block, embedding any watermark into a two-dimensional video frame, and respectively performing multi-stage DT-CWT decomposition on all components in the color space of the watermark sub-block to obtain sub-bands of the watermark sub-block under each stage of decomposition;

SVD operation is carried out on the amplitude matrix of the sub-band of the watermark sub-block obtained under the set level decomposition to obtain a singular value corresponding to the watermark sub-block;

generating a zero watermark of the watermark embedded into the two-dimensional video frame according to singular values corresponding to all the watermark sub-blocks in the watermark embedded into the two-dimensional video frame;

and generating the zero watermark of the three-dimensional video to be embedded according to the zero watermarks of all the watermarks embedded into the two-dimensional video frames corresponding to the three-dimensional video to be embedded.

Optionally, the modifying, according to the watermark to be embedded, the amplitude matrix of the sub-band obtained by decomposing the sub-blocks in the set number range to obtain the modified sub-band specifically includes:

calculating an amplitude matrix and a phase matrix of a sub-band obtained by decomposing the sub-blocks in a set series range;

quantizing the amplitude matrix of the sub-band obtained by decomposing the sub-block in the set level range according to the watermark to be embedded to obtain a quantized sub-band amplitude matrix corresponding to the sub-block;

and obtaining a modified sub-band according to the phase matrix and the quantized sub-band amplitude matrix corresponding to the sub-block.

A watermark extraction method based on 3D video comprises the following steps:

respectively blocking each two-dimensional video frame in a three-dimensional video to be extracted to obtain a plurality of extracted sub-blocks;

for any extracted subblock in any two-dimensional video frame, performing multi-stage DT-CWT decomposition on any component in the color space of the extracted subblock to obtain a sub-band of the extracted subblock under each stage of decomposition;

determining the watermark embedded in the three-dimensional video to be extracted according to the amplitude matrix of the sub-band obtained by decomposing each extracted sub-block in the three-dimensional video to be extracted in a set progression range;

and generating a zero watermark of the three-dimensional video to be extracted according to the three-dimensional video to be extracted.

Optionally, the determining, according to a sub-band amplitude matrix obtained by decomposing each extracted sub-block in the three-dimensional video to be extracted in a set progression range, a watermark embedded in the three-dimensional video to be extracted specifically includes:

for any extracted subblock, quantizing the amplitude matrix of the subband obtained by decomposing the extracted subblock in a set level range to obtain a quantized subband amplitude matrix corresponding to the extracted subblock;

calculating the watermark embedded into the extracted sub-block according to the quantized sub-band amplitude matrix corresponding to the extracted sub-block;

and determining the watermarks embedded in the three-dimensional video to be extracted according to the watermarks embedded in all extracted sub-blocks in the three-dimensional video to be extracted.

A watermark comparison method based on 3D video comprises the following steps:

processing the three-dimensional video to be protected according to the 3D video-based watermark embedding method to obtain an embedded watermark and a zero watermark;

processing the three-dimensional video to be authenticated according to the 3D video-based watermark extraction method to obtain an extracted embedded watermark and an extracted zero watermark;

and determining whether the three-dimensional video to be identified infringes the copyright of the three-dimensional video to be embedded according to the embedded watermark, the extracted embedded watermark, the zero watermark and the extracted zero watermark.

Optionally, the determining, according to the embedded watermark, the extracted embedded watermark, the zero watermark, and the extracted zero watermark, whether the three-dimensional video to be authenticated violates the copyright of the three-dimensional video to be embedded specifically includes:

carrying out XOR operation on the embedded watermark and the extracted embedded watermark to obtain a watermark error rate;

carrying out XOR operation on the zero watermark and the extracted zero watermark to obtain a zero watermark error rate;

judging whether the watermark error rate is smaller than a first set threshold value and whether the zero watermark error rate is smaller than a second set threshold value to obtain a first judgment result;

if the first judgment result is yes, the three-dimensional video to be identified infringes the copyright of the three-dimensional video to be protected;

and if the first judgment result is negative, the three-dimensional video to be identified does not infringe the copyright of the three-dimensional video to be protected.

A 3D video based watermark embedding system, comprising:

the first blocking module is used for respectively blocking each two-dimensional video frame to be embedded into the three-dimensional video to obtain a plurality of sub-blocks;

the first decomposition module is used for carrying out multi-stage DT-CWT decomposition on any component in the subblock color space for any subblock in any two-dimensional video frame to obtain a sub-band of the subblock under each stage of decomposition;

the sub-band modification module is used for modifying the amplitude matrix of the sub-band obtained by decomposing the sub-blocks in the range of the set number of levels according to the watermark to be embedded to obtain the modified sub-band;

a sub-block watermark embedding module, configured to perform inverse DT-CWT on the modified sub-band to obtain an embedded watermark sub-block corresponding to the sub-block;

the video frame watermark embedding module is used for splicing all the embedded watermark sub-blocks corresponding to the two-dimensional video frame to obtain a watermark embedded two-dimensional video frame corresponding to the two-dimensional video frame;

and the watermark determining module is used for embedding all watermarks corresponding to the three-dimensional video to be embedded into the two-dimensional video frame to obtain a watermark embedded video and a zero watermark of the three-dimensional video to be embedded.

A 3D video based watermark extraction system, comprising:

the second blocking module is used for respectively blocking each two-dimensional video frame in the three-dimensional video to be extracted to obtain a plurality of extraction sub-blocks;

the second decomposition module is used for carrying out multi-stage DT-CWT decomposition on any component in the extracted subblock color space for any extracted subblock in any two-dimensional video frame to obtain a sub-band of the extracted subblock under each stage of decomposition;

the embedded watermark extraction module is used for determining the watermark embedded in the three-dimensional video to be extracted according to a sub-band amplitude matrix obtained by decomposing each extracted sub-block in the three-dimensional video to be extracted in a set progression range;

and the zero watermark extraction module is used for generating a zero watermark of the three-dimensional video to be extracted according to the three-dimensional video to be extracted.

A 3D video based watermark comparison system, comprising:

the watermark embedding module is used for processing the three-dimensional video to be protected according to the 3D video-based watermark embedding method to obtain an embedded watermark and a zero watermark;

the watermark extraction module is used for processing the three-dimensional video to be identified according to the 3D video-based watermark extraction method to obtain an extracted embedded watermark and an extracted zero watermark;

and the copyright judging module is used for determining whether the three-dimensional video to be identified infringes the copyright of the three-dimensional video to be embedded according to the embedded watermark, the extracted embedded watermark, the zero watermark and the extracted zero watermark.

According to the specific embodiment provided by the invention, the invention discloses the following technical effects: the method comprises the steps of respectively blocking each two-dimensional video frame to be embedded into a three-dimensional video to obtain a plurality of sub-blocks; carrying out multi-stage DT-CWT decomposition on any component in the subblock color space to obtain a subband of the subblock under each stage of decomposition; modifying the amplitude matrix of the sub-band obtained by decomposing the sub-blocks in the set level range according to the watermark to be embedded to obtain a modified amplitude matrix of the sub-band; obtaining a modified sub-band according to the modified amplitude matrix of the sub-band; carrying out inverse DT-CWT operation on the modified sub-band to obtain an embedded watermark sub-block corresponding to the sub-block; splicing all embedded watermark sub-blocks corresponding to the two-dimensional video frame to obtain a watermark embedded two-dimensional video frame corresponding to the two-dimensional video frame; the method comprises the steps of embedding all watermarks corresponding to a three-dimensional video to be embedded into a two-dimensional video frame to obtain a watermark embedded video and a zero watermark of the three-dimensional video to be embedded, and combining the embedded watermark and the zero watermark, so that the copyright protection precision can be further improved on the premise of ensuring the high robustness of the watermark.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.

Fig. 1 is a diagram of the results of a discrete cosine transform based watermark embedding method;

FIG. 2 is a first exemplary diagram;

FIG. 3 is a second illustration;

FIG. 4 is a comparison of video frames at different baseline distances;

FIG. 5 is an example of the orientation of a DT-CWT decomposed sub-band;

fig. 6 is a flowchart of a watermark embedding process provided by an embodiment of the present invention;

fig. 7 is a flowchart of a watermark comparison process provided by an embodiment of the present invention;

fig. 8 is a diagram of a first frame of a video to be protected according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

The watermark is embedded in the video frame, the trade-off between the watermark invisibility and the watermark robustness exists, the watermark invisibility and the watermark robustness are mutually restricted, and the optimal watermark cannot be achieved at the same time. Since the lower the watermark presence, the more likely it is to be corrupted, the less robust it is likely to be. If robustness to various attacks is required, watermark embedding strength needs to be increased, and video frame content is damaged. Therefore, research has been focused on how to satisfy the requirement of watermark invisibility while improving watermark robustness. If the robustness of the watermark embedding scheme is not enough, even if the watermark embedding scheme is well invisible, all watermarks of a copyright side cannot be extracted during copyright identification, and the copyright protection effect cannot be achieved; if the video frame is robust to common operations, the embedded watermark destroys the appearance or generates a certain degree of distortion, so that the applicable range of the video frame is greatly reduced.

Fig. 1 shows the result of a discrete cosine transform-based watermark embedding method: with the change of the watermark embedding strength and the change of the watermark robustness and invisibility, measuring the robustness and invisibility results by adopting a BIT ERROR rate (BIT ERROR RATIO) and a peak signal-TO-NOISE RATIO (PEAK SIGNAL-TO-NOISE RATIO, PSNR), wherein the lower the BER is, the better the watermark integrity is, and the robustness of the watermark method is strong; as the PSNR value is higher, the image distortion is lower, and the invisibility of the watermark is better, as shown in fig. 1, as the embedding strength is increased, the left image has more obvious block boundaries, i.e. the image quality is reduced, which affects the appearance. However, the right image has no obvious distortion but cannot play a role in protection, and a complete watermark cannot be extracted after simple signal attack.

The zero watermark characteristic is constructed for the video frame, although the video content is not damaged, the identifiability is not high enough, and the problems of misjudgment and missed judgment during the copyright identification can occur. Taking a method based on ternary moments as an example, constructing a zero watermark for fig. 2 to obtain an [ 01110011110100011101 ] feature sequence, constructing a zero watermark for fig. 3 to obtain an [ 00110011010000011101 ] feature sequence, and generating 1600-bit features for both fig. 2 and fig. 3 respectively by using the method, wherein the top 20-bit features are selected for display. Comparing two zero watermark features, the similarity reaches 89.8%, but from the viewpoint of human eye perception, the two different videos can be distinguished. Therefore, even if a certain zero watermark method has good robustness to various signal attacks and geometric attacks, once whether the video belongs to all videos of a copyright side cannot be distinguished, certain practical value is lost. Robustness and differentiability are contradictory properties, similar to the watermarking scheme, and the two need to be balanced for different use environments, so that the two cannot be optimized simultaneously.

In recent years, much research has been carried out to pay attention to the special properties of the DIBR technology, and a corresponding copyright protection scheme is proposed for its features, but the robustness to a higher baseline distance is not enough, the baseline distance is a parameter for synthesizing a 2D video frame and a depth map into a left view and a right view, fig. 4 shows the stereo degree of the video frame under the influence of different baseline distances, fig. 4(a) is a 2D video frame, fig. 4(b) is a left view of a synthesized 3D video frame with a baseline distance of 3%, fig. 4(c) is a left view of a synthesized 3D video frame with a baseline distance of 5%, fig. 4(D) is a left view of a synthesized 3D video frame with a baseline distance of 7% (fig. 4 all take the left view as an example), as shown in fig. 4, the longer baseline is, the accuracy of the synthesized 3D video can be correspondingly improved within a certain range, and the stereoscopic impression is better. However, the influence of lens distortion is increased, and the more distorted image content is seen by naked eyes, the more difficult binocular matching of features is. Therefore, it is necessary to improve robustness of the DIBR method within a reasonable baseline distance, i.e. in 3D video synthesized based on the DIBR method, watermark extraction or zero watermark feature construction is not affected by a slightly longer baseline distance.

Based on this, an embodiment of the present invention provides a watermark embedding method based on a 3D video, including:

and respectively blocking each two-dimensional video frame to be embedded into the three-dimensional video to obtain a plurality of sub-blocks. In this embodiment, the three-dimensional video to be embedded is based on a DIBR 3D video.

And for any subblock in any two-dimensional video frame, carrying out multi-stage DT-CWT decomposition on any component in the subblock color space to obtain a sub-band of the subblock under each stage of decomposition. Within the range of causing acceptable content distortion to the 3D video, a Dual-tree complex wavelet transform (DT-CWT) method is used for embedding a watermark with certain strength into a video frame. The DT-CWT adopts an independent double-discrete wavelet tree structure to carry out multi-scale decomposition, and can obtain sub-band information in the directions of +/-15 degrees and +/-75 degrees besides the horizontal direction and the diagonal direction, so that the DT-CWT has direction selectivity, limited redundancy and approximate translation invariance, compared with the traditional wavelet transformation, the details of an image can be described more finely and abundantly, and the obtained description sub-band cannot be changed and distorted easily along with the change of the content of the image, so that the DT-CWT method is selected as a technical means of a watermark embedding scheme, and the DT-CWT can better meet the requirements under the requirements of robustness, invisibility and safety required by a digital watermark technology. The DT-CWT generates 6 symmetrical sub-bands H with different directions (+ -15 °, + -45 ° and + -75 °) after each first-order decomposition_3,drAnd dr is 1,2,3,4,5 and 6, so that when image pixels move left and right and move up and down, even turn at a certain angle, the DT-CWT processed video frame is not very sensitive to the changes and has better robustness. As shown in FIG. 5, the 6 subbands of the DT-CWT can be divided into three pairs according to the symmetry of the direction: (H)_3,1,H_3,6)，(H_3,2,H_3,5) And (H)_3,3,H_3,4). Wherein the first pair of coefficients has mainly profile information for horizontal edges, the second pair has mainly profile information for diagonal edges, and the third pair has mainly profile information for vertical edges. Watermark is embedded in the low frequency of DT-CWT, which can effectively cope with signal attack, such as smoothing operations of Gaussian filtering, median filtering, mean filtering and the like, JPEG compression used in daily life, and noise attack such as salt-pepper noise, Gaussian noise and the like; and subsequent watermark embedding quantization methodMultiple statistical differences in (b) enhance robustness to geometric attacks such as scaling, edge clipping; the watermark embedding method of the DT-CWT ensures the robustness of DIBR transformation due to the approximate translation invariance.

And modifying the amplitude matrix of the sub-band obtained by decomposing the sub-blocks in the set level range according to the watermark to be embedded to obtain the modified sub-band.

And carrying out inverse DT-CWT operation on the modified sub-band to obtain an embedded watermark sub-block corresponding to the sub-block.

And splicing all the embedded watermark sub-blocks corresponding to the two-dimensional video frame to obtain a watermark embedded two-dimensional video frame corresponding to the two-dimensional video frame.

In practical applications, the performing multi-level DT-CWT decomposition on any component in the sub-block color space to obtain a sub-band of the sub-block under each level of decomposition specifically includes:

and converting the sub-blocks from the RGB color space to the YCbCr color space to obtain converted sub-blocks.

And carrying out multi-stage DT-CWT decomposition on the Y component of the conversion subblock to obtain a sub-band of the conversion subblock under each stage of decomposition, wherein three-stage decomposition can be carried out in practical application.

In practical application, the modifying the amplitude matrix of the sub-band obtained by decomposing the sub-blocks in a set series range according to the watermark to be embedded to obtain the modified sub-band specifically includes:

and calculating the amplitude matrix and the phase matrix of the sub-band obtained by decomposing the sub-blocks in the set series range.

And quantizing the amplitude matrix of the sub-band obtained by decomposing the sub-block in the set level range according to the watermark to be embedded to obtain a quantized sub-band amplitude matrix corresponding to the sub-block.

In practical application, the calculating of the amplitude matrix and the phase matrix of the sub-band obtained by decomposing the sub-blocks in the set number range specifically includes: according to the formula

A matrix of amplitude values is calculated and,

according to the formula

Calculating a phase matrix, wherein the amplitude matrix is obtained by solving an absolute value of the subband coefficients, the phase matrix is obtained by solving an angle of the subband coefficients, wherein,

a magnitude matrix representing the dr sub-band at the lv level decomposition,

representing the phase matrix of the dr subband at the lv level decomposition, W and H representing the width and height of the video frame; m and N are the width and height of the sub-blocks,

the values representing the last row and the last column of the subband coefficient matrix are decremented by one because the length and width count for this matrix starts from 0. The length of the coefficient matrix is then

Width is

The absolute value sign outside the brackets is the absolute value found, and the amplitude is obtained.

Letter in parentheses means H_lv,drThe same, the symbol is the meaning of obtaining the phase value. Specifically, if three DT-CWT decompositions are performed in total, lv may be made equal to2, 3; dr 1,2, 5,6, the principle of DIBR systems is to move each pixel horizontally, so vertical edges lose more information than horizontal edges. Thus, instead of operating on all sub-bands, a careful selection is made of the direction, the first and second pairs of coefficients are selected for quantization for watermark embedding, i.e. (H)_3,1,H_3,6) And (H)_3,2,H_3,5) And excessive vertical edge information is avoided, so that the robustness of DIBR conversion is ensured.

In practical application, the obtaining of the quantized subband amplitude matrix corresponding to the subblock by quantizing the amplitude matrix of the subband obtained by decomposing the subblock in a set level range according to the watermark to be embedded specifically includes: according to the formula

Carrying out quantization, wherein

Indicating the ith row in the quantized subband magnitude matrix for the 1 st subband of the sub-block at the lv level decomposition,

row i in the modified amplitude matrix representing the distance between the asymmetric subbands of the 1 st subband at the decomposition of the lv level,

the ith row in the quantized subband magnitude matrix representing the 6 th subband under the lv level decomposition of the subblock,

indicating the ith row in the magnitude matrix of the 1 st sub-band at the decomposition of the lv level,

indicating the ith row in the magnitude matrix of the 6 th sub-band at the lv level decomposition of the sub-block,

row i in the modified amplitude matrix representing the distance between the asymmetric subbands of the 6 th subband of the subblock at the decomposition of the lv level, w represents the watermark value,

row i and column k in the modified amplitude matrix representing the distance between the asymmetric sub-bands of the dr sub-band at the decomposition of the lv level,

the ith row and the kth column of the magnitude matrix representing the dr th sub-band of the sub-block at the lv level decomposition,

indicating the ith row in the quantized subband magnitude matrix for the 2 nd subband under the decomposition of the lv level for the subblock,

row i in the modified amplitude matrix representing the distance between the asymmetric subbands of the 2 nd subband at the decomposition of the lv level,

indicating the ith row in the quantized subband magnitude matrix for the 5 th subband of the sub-block at the lv level decomposition,

representing the ith row in the magnitude matrix of the 2 nd sub-band at the decomposition of the lv level,

indicating the ith row in the magnitude matrix of the 5 th sub-band at the decomposition of the lv level,

an ith row in the modified amplitude matrix representing the distance between the asymmetric sub-bands of the sub-block 5 under the decomposition of the lv level, wherein if the DT-CWT decomposition is performed three times in total, lv can be 2, 3; dr is 1,2, 5, 6.

In practical application, the step of splicing all the embedded watermark sub-blocks corresponding to the two-dimensional video frame to obtain a watermark embedded two-dimensional video frame corresponding to the two-dimensional video frame specifically includes:

and splicing all the watermark embedding sub-blocks corresponding to the two-dimensional video frame to obtain a complete two-dimensional video frame.

And converting the complete two-dimensional video frame from a YCbCr color space to an RGB color space to obtain a watermark embedding two-dimensional video frame corresponding to the two-dimensional video frame.

In practical application, embedding a two-dimensional video frame with all watermarks corresponding to the three-dimensional video to be embedded to obtain a zero watermark of the three-dimensional video to be embedded specifically includes:

and respectively blocking each watermark embedding two-dimensional video frame corresponding to the three-dimensional video to be embedded to obtain a plurality of watermark sub-blocks.

And for any watermark sub-block in any watermark embedding two-dimensional video frame, respectively carrying out multi-stage DT-CWT decomposition on all components in the color space of the watermark sub-block to obtain sub-bands of the watermark sub-block under each stage of decomposition.

And carrying out SVD operation on the amplitude matrix of the sub-band of the watermark sub-block obtained under the set level decomposition to obtain a singular value corresponding to the watermark sub-block.

And generating the zero watermark of the watermark embedded two-dimensional video frame according to the singular values corresponding to all the watermark sub-blocks in the watermark embedded two-dimensional video frame.

And generating the zero watermark of the three-dimensional video to be embedded according to the zero watermarks of all the watermarks embedded into the two-dimensional video frames corresponding to the three-dimensional video to be embedded. The DT-CWT method is selected to process the video frame after the watermark is embedded, and Singular Value Decomposition (SVD) is added to jointly construct the feature mapping of the zero watermark, so that the robustness of DIBR transformation, signal attack and geometric attack can be further improved. The DT-CWT method has been described above and will not be described here. The decomposition of SVD is as follows:

X＝USV^T

for matrix X ∈ R^n×nThe decomposed U and V are both unitary matrixes, namely UU^T＝E，VV^T＝E，U^TAnd V^TU and V, respectively, and E is an identity matrix. The values on the diagonal of the S matrix are singular values, which is equivalent to extracting the characteristics of a matrix and representing the complete matrix by a small amount of data. The singular values are arranged in the order from big to small, the speed of reduction is very fast, and the singular values far smaller than 1 are arranged at the back, so that the information of the whole matrix is often represented by only the first few or more than ten singular values. Therefore, the SVD can reduce data, extract the most important information in the data and achieve the effect of compression. And the method has good stability, displacement invariance and certain rotation invariance, and is widely applied to image processing. The magnitude relation characteristic of the singular value and the characteristic extracted by the DT-CWT are combined together to construct a robust and distinguishable zero watermark characteristic.

The invention is lossy in the step of watermark embedding, but because of the double verification and characteristic complementation of the watermark and the zero watermark characteristic, the watermark scheme does not need to sacrifice the precision of the image content for high robustness, because the copyright retrieval is carried out by the zero watermark scheme; the zero watermark scheme does not need to sacrifice robustness against signal attacks, geometric attacks and DIBR operations for high distinguishability, since authentication of copyrights is performed by the watermark scheme. Compared with the traditional single watermark embedding or zero watermark constructing method, the detection precision of the invention is improved to a certain extent, and the effectiveness of copyright protection is further improved on the premise of ensuring the precision of video content.

In practical application, the multi-stage DT-CWT decomposition is performed on all components in the color space of the watermark sub-block, so as to obtain sub-bands of the watermark sub-block under each stage of decomposition, which specifically includes:

and respectively carrying out multi-level DT-CWT decomposition on R, G and B channels of the watermark sub-block to obtain a sub-band of the watermark sub-block under each level of decomposition, and specifically carrying out three-level DT-CWT decomposition.

In practical application, the SVD operation is performed on the amplitude matrix of the sub-band of the watermark sub-block obtained under the set level decomposition to obtain the singular value corresponding to the watermark sub-block, and the method specifically includes:

when three-level DT-CWT decomposition is carried out, amplitude matrixes of 1 st, 2 nd, 5 th and 6 th sub-bands obtained by the watermark sub-blocks under the third-level decomposition are calculated.

And determining the amplitude matrix of the 1 st, 2 nd, 5 th and 6 th sub-bands obtained by the watermark sub-blocks under the third-level decomposition as a matrix P.

And carrying out SVD operation on the P to obtain a singular value corresponding to the watermark sub-block.

In practical application, the generating of the zero watermark of the watermark-embedded two-dimensional video frame according to the singular values corresponding to all the watermark sub-blocks in the watermark-embedded two-dimensional video frame specifically includes:

and enabling the watermark to be embedded into the median of singular values corresponding to all watermark sub-blocks in the two-dimensional video frame as a threshold value.

And changing singular values which are larger than a threshold value from singular values which correspond to all watermark sub-blocks in the watermark embedding two-dimensional video frame into 1, and changing the singular values which are smaller than or equal to the threshold value into 0, and finally obtaining the zero watermark of the watermark embedding two-dimensional video frame.

The embodiment of the invention combines the embedded watermark and the zero watermark, and can further improve the accuracy of copyright protection on the premise of ensuring the high robustness of the watermark.

The embodiment of the invention also provides a watermark extraction method based on a 3D video aiming at the watermark embedding method, which comprises the following steps:

and respectively blocking each two-dimensional video frame in the three-dimensional video to be extracted to obtain a plurality of extracted sub-blocks. In this embodiment, the three-dimensional video to be extracted is based on a DIBR 3D video.

And for any extracted subblock in any two-dimensional video frame, performing multi-stage DT-CWT decomposition on any component in the extracted subblock color space to obtain a sub-band of the extracted subblock under each stage of decomposition.

And determining the watermark embedded in the three-dimensional video to be extracted according to the amplitude matrix of the sub-band obtained by decomposing each extracted sub-block in the three-dimensional video to be extracted in a set progression range.

In practical application, the steps of performing multi-stage DT-CWT decomposition on any component in the extracted subblock color space to obtain a sub-band of the extracted subblock under each stage of decomposition are the same as the steps of performing multi-stage DT-CWT decomposition on any component in the subblock color space to obtain a sub-band of the subblock under each stage of decomposition.

In practical application, the determining the watermark embedded in the three-dimensional video to be extracted according to the amplitude matrix of the sub-band obtained by decomposing each extracted sub-block in the three-dimensional video to be extracted in a set progression range specifically includes:

for any extracted subblock, quantizing a subband amplitude matrix obtained by decomposing the extracted subblock in a set level range to obtain a quantized subband amplitude matrix corresponding to the extracted subblock, specifically, if three times of DT-CWT decomposition are performed in total, making lv 2, 3; dr is 1,2, 5,6, the quantized subband magnitude matrix is computed.

And calculating the watermark embedded in the extracted sub-block according to the quantized sub-band amplitude matrix corresponding to the extracted sub-block, specifically, when the value of the ith row of the 1 st sub-band amplitude matrix is smaller than the value of the ith row of the 6 th sub-band amplitude matrix, adding 1 to the number of 0 watermarks embedded in the extracted sub-block, and otherwise, adding 1 to the number of 1 watermarks. And when the value of the ith row of the 2 nd sub-band amplitude matrix is smaller than the value of the ith row of the 5 th sub-band amplitude matrix, adding 1 to the number of 0 watermarks embedded in the sub-blocks, otherwise, adding 1 to the number of 1 watermarks, and obtaining the watermarks embedded in the extracted sub-blocks after all the values in the amplitude matrix are judged.

And determining the watermarks embedded in the three-dimensional video to be extracted according to the watermarks embedded in all the extracted sub-blocks in the three-dimensional video to be extracted.

The embodiment of the invention also provides a watermark comparison method based on the 3D video, which comprises the following steps:

and processing the three-dimensional video to be protected according to the 3D video-based watermark embedding method to obtain the embedded watermark and the zero watermark.

And processing the three-dimensional video to be authenticated according to the 3D video-based watermark extraction method to obtain the extracted embedded watermark and the extracted zero watermark.

And determining whether the three-dimensional video to be authenticated infringes the copyright of the three-dimensional video to be embedded according to the embedded watermark, the extracted embedded watermark, the zero watermark and the extracted zero watermark.

In practical application, the determining whether the three-dimensional video to be authenticated infringes the copyright of the three-dimensional video to be embedded according to the embedded watermark, the extracted embedded watermark, the zero watermark and the extracted zero watermark specifically includes:

and carrying out exclusive OR operation on the embedded watermark and the extracted embedded watermark to obtain a watermark error rate B1.

And carrying out exclusive OR operation on the zero watermark and the extracted zero watermark to obtain a zero watermark error rate B2.

And judging whether the watermark error rate is smaller than a first set threshold value T1 or not and whether the zero watermark error rate is smaller than a second set threshold value T2 or not to obtain a first judgment result.

And if the first judgment result is yes, the three-dimensional video to be identified infringes the copyright of the three-dimensional video to be protected.

Aiming at the embedding method, the embodiment of the invention also provides a watermark embedding system based on the 3D video, which comprises the following steps:

the first blocking module is used for respectively blocking each two-dimensional video frame to be embedded into the three-dimensional video to obtain a plurality of sub-blocks.

And the first decomposition module is used for carrying out multi-stage DT-CWT decomposition on any component in the subblock color space for any subblock in any two-dimensional video frame to obtain a sub-band of the subblock under each stage of decomposition.

And the sub-band modification module is used for modifying the amplitude matrix of the sub-band obtained by decomposing the sub-blocks in the range of the set number of levels according to the watermark to be embedded to obtain the modified sub-band.

And the sub-block watermark embedding module is used for carrying out inverse DT-CWT operation on the modified sub-band to obtain an embedded watermark sub-block corresponding to the sub-block.

And the video frame watermark embedding module is used for splicing all the watermark embedding sub-blocks corresponding to the two-dimensional video frame to obtain a watermark embedding two-dimensional video frame corresponding to the two-dimensional video frame.

The embodiment of the present invention further provides a watermark extraction system based on a 3D video, including:

and the second blocking module is used for respectively blocking each two-dimensional video frame in the three-dimensional video to be extracted to obtain a plurality of extraction sub-blocks.

And the second decomposition module is used for carrying out multi-stage DT-CWT decomposition on any component in the extracted subblock color space for any extracted subblock in any two-dimensional video frame to obtain a sub-band of the extracted subblock under each stage of decomposition.

And the embedded watermark extraction module is used for determining the watermark embedded in the three-dimensional video to be extracted according to the amplitude matrix of the sub-band obtained by decomposing each extracted sub-block in the three-dimensional video to be extracted in a set progression range.

Aiming at the comparison method, the embodiment of the invention also provides a watermark comparison system based on the 3D video, which comprises the following steps:

and the watermark embedding module is used for processing the three-dimensional video to be protected according to the 3D video-based watermark embedding method to obtain an embedded watermark and a zero watermark.

And the watermark extraction module is used for processing the three-dimensional video to be authenticated according to the 3D video-based watermark extraction method to obtain the extracted embedded watermark and the extracted zero watermark.

The invention also provides a more specific implementation mode of the whole process of watermark embedding, extracting and comparing:

for the watermark embedding process, firstly, watermarks are embedded in all 2D video frames in a DIBR 3D-based video, as the subsequent authentication requirement, the whole algorithm process uses the zero watermark characteristic for retrieval, and the video frames are protected by two methods, so that the difficulty of contradiction balance between robustness and invisibility in the watermark embedding algorithm characteristic is reduced, the visual impression of human eyes is not damaged for achieving high-strength robustness, the number of the watermark bits embedded in each frame of image is not required to be too long, and the subsequent authentication can be achieved, and the specific process is shown in FIG. 6.

Step 1, inputting a 2D video frame to be protected, and blocking the 2D video frame to obtain (I)_W/M)×(I_H/N) sub-blocks B_s(s＝1,2,…(I_W/M)×(I_H/N)), M and N are sub-blocks B_sOne sub-block embeds a one-bit watermark.

And 2, vectorizing the DT-CWT coefficients. Sub-block B_sAnd transferring the RGB color format to a YCbCr color space, selecting a Y component to carry out three-level DT-CWT decomposition, selecting a second level sub-band and a third level sub-band to carry out subsequent quantization operation, and pairing the directions 1 and 6 with the directions 2 and 5: (H)_lv，1，H_lv，6) And (H)_lv，2，H_lv，5) Wherein lv is 2, 3.

And 3, embedding the watermark video frame. Determining amplitude matrix and phase matrix of sub-band, modifying the amplitude matrix according to row, modifying (H) when the bit of embedded watermark is 0_lv，1，H_lv，6) H in (1)_lv，1，(H_lv，2，H_lv，5) H in (1)_lv，2The other two remain unchanged; when embedding 1, then modify H separately_lv，6And H_lv，5. Wherein the amplitude matrix and the phase matrix are calculated as follows:

wherein lv represents the decomposition to the next stage, and since three-stage decomposition is performed and only coefficients of the next two stages are selected, lv is 2 and 3; dr denotes the respective subband: 1,2,3,4,5, 6; w and H represent the width and height of the video frame; m and N are the width and height of the sub-blocks.

The amplitude matrix is obtained by solving the absolute value of the subband coefficients, and the phase matrix is obtained by solving the angle of the subband coefficients. The watermark is embedded by modifying the magnitude matrix, and the phase matrix is used to recover the sub-band coefficients after embedding the watermark.

For the modification of the amplitude matrix, each row of the amplitude matrix is regarded as a whole, and quantization operation is performed in row units, that is, the following operations are performed on all columns of a certain row of the amplitude matrix, and the quantization calculation formula is as follows:

wherein the content of the first and second substances,

is calculated byThe step is to pull the difference between the amplitude matrix of each row of the 1 st and 6 th sub-bands and the 2 nd and 5 th sub-bands, because the amplitude matrix is not too large, the division in the formula is rounding, thus 256 is used as divisor, and rounding is performed twice, at this time

And is generally calculated to be a very small value or even 0.

Taking the sub-bands 1 and 6 as examples, i represents all column operations on the ith row, and when the embedded watermark is 0, i.e. w equals 0, the quantized amplitude matrix of the sub-band 1 under the decomposition of the lv level is assigned

(at this time

Typically a small number or even 0) while the 6 th subband magnitude matrix remains unchanged, thereby further augmenting the difference between 1 and 6, i.e., the magnitude matrices in row i of the 1 st subband magnitude matrix are all 0 or small values while the magnitude matrices in row i of the 6 th subband magnitude matrix remain unchanged. When the embedded watermark is 1, i.e. w equals 1, the ith row of the amplitude matrix of the 1 st sub-band is not modified, but the ith row of the amplitude matrix of the 6 th sub-band is modified, the amplitude matrix of which is previously calculated

Similarly, the 2 nd and 5 th sub-bands are also modified according to the 0 and 1 bit of the watermark. Thus, for the same block, each row of the magnitude matrix embeds the same watermark bit sequence.

And 4, after the amplitude matrix of each row of each sub-band is quantized, restoring the amplitude matrix to the original sub-band format according to the modified amplitude matrix and the previously calculated phase matrix to obtain the modified sub-band, then executing inverse DT-CWT operation to obtain blocks with embedded watermarks, recombining all the blocks into a complete 2D video frame, and converting the complete 2D video frame from a YCbCr domain to an RGB domain to finish the watermark embedding operation on the whole video frame.

Next, a zero watermark construction method is introduced, which specifically includes the following steps:

step 1, DT-CWT + SVD. The video sequence after embedding the watermark is subjected to scaling preprocessing, namely, a video frame is converted into a fixed length and width, the size is set according to the actual situation, and the video frame is generally set into a square with the same length and width so as to be more robust to rotation attack and scaling attack. And then, the frame sequence in the 2D video is subjected to normalization and downsampling, every n frames are combined into a group, and the average number is obtained, so that the redundancy of time information can be effectively reduced, and time domain information is fully utilized.

Step 2, each frame of the merged video sequence is divided into (I) blocks_W/M)×(I_H/N) sub-blocks B_s(s ═ 1, 2.. mxn), M and N are B_sLength of the side.

Step 3, pair sub-block B_sThe R, G, B three channels are respectively subjected to three-level DT-CWT decomposition, a subband coefficient in the third-level horizontal direction is selected, and the 1 and 6 directions and the 2 and 5 directions are paired: p₁＝(H_3，1，H_3，6) And P₂＝(H_3，2，H_3，5) And calculating the amplitude value according to the following formula:

is an amplitude matrix, where dr is 1,2, 5,6,

indicating the dr th sub-block at level 3 decompositionWith a matrix of

Go to the first

The coefficient values of the columns.

Step 4, obtaining P₁And P₂After the amplitude matrix is obtained, the amplitude matrix is put into the same matrix, the detailed structure of P is shown below, SVD operation is carried out on the P, the first singular value is saved, and the intermediate eigenvector f is generated_m。

And 5, zero watermark characteristic. Taking the first singular value of all the sub-blocks, namely the median of the intermediate characteristic vectors, as a threshold value t, the structure which is larger than the threshold value is 1, and the structure which is smaller than or equal to the threshold value is 0, carrying out binarization, and finally connecting in series to form a zero watermark bit sequence f to obtain a series of bit sequences of 0 or 1, registering the bit sequences in a database for storage, and as the requirement of retrieval in the future copyright identification, f can represent a certain 2D video. The binarization process is as follows:

f_m(c) denotes the c-th value of the previously generated intermediate feature vector, f (c) denotes the c-th value of the finally constructed zero-watermark sequence.

And 6, a third party digital watermark information base. Finally, storing the video, the watermark and the zero watermark characteristic of the embedded watermark in a database, constructing the zero watermark characteristic when a video V which is possibly infringed is found on the network, then retrieving the zero watermark in the database according to the zero watermark characteristic, and if the video zero watermark similar to a certain content is matched in the database, further authenticating: and (4) extracting the watermark from the V, and judging the infringement if the extracted watermark is very similar to the watermark stored in the database, namely the calculated ber is less than a set threshold value.

For watermark extraction and comparison, the steps of video frame blocking, color space conversion, amplitude matrix calculation and the like in the previous stage are basically the same as those in the watermark embedding stage, so that only the extraction of the watermark is explained in detail:

step 1, partitioning the 2D video frame to obtain extracted sub-blocks.

And 2, transferring the extracted sub-block to a YCbCr color space, selecting a Y component to perform three-level DT-CWT decomposition, performing three-level DT-CWT decomposition, and selecting sub-band coefficients in the second and third levels of horizontal directions of the extracted sub-block. The method is divided into three pairs: (H'_lv，1，H_lv，6)，(H′_lv，2，H_lv，5) And (H'_lv，3，H_lv，4) The first two pairs are selected for the watermark embedding operation, and the last pair is discarded because it represents the information of the vertical edge of the video frame content, which is very vulnerable to distortion and distortion under DIBR operation.

Step 3, p (H'_lv，1，H′_lv，6) And (H'_lv，2，H′_lv，5) Calculating the amplitude matrix of the expressed coefficients, quantizing the amplitude matrix, and calculating according to rows, wherein the calculation process is the same as that of the watermark embedding process, so that the amplitude matrix of the quantized sub-band of the extracted sub-block is obtained: extracting the ith row in the quantized subband magnitude matrix of the 1 st subband of the subblock at the lv level decomposition

Extracting the ith row in the quantized subband magnitude matrix of the 6 th subband of the subblock at the lv level decomposition

Extracting the ith row in the quantized subband magnitude matrix of the 2 nd subband of the subblock at the lv level decomposition

And extracting the ith row in the quantized subband magnitude matrix of the 5 th subband of the subblock at the lvf level decomposition

Passing systemThe difference is counted to determine whether to extract 0 or 1, again taking the amplitude matrix of the 1 st and 6 th sub-bands as an example:

if it is

Is less than

The count representing the 0 watermark is increased 1 time if

Is greater than

The number of 1 watermark is increased by 1 time, and after all the rows are executed, 0 or 1 is determined to be extracted according to the statistical size comparison result, so as to obtain the final watermark extraction sequence W'.

And 4, step 4: and performing exclusive or operation on the W' and the original watermark sequence W to obtain a final error rate:

if ber is smaller than the set experience threshold, copyright authentication is finished, and the watermark is judged according to the final statistical total number in the extraction scheme, so that the watermark extraction error is reduced. The method does not use a single watermark embedding or zero watermark constructing method, but combines the two methods, fully utilizes the advantages of the two methods, complementarily reduces the disadvantages, enlarges the application range and improves the copyright identification precision. The step 4 specifically comprises the following steps: firstly, extracting zero watermark features from a 3D video to be identified, then, carrying out feature retrieval in a registered copyright information database, if the error rate is smaller than a T2 threshold value, namely, the similarity is very high, further extracting an embedded watermark, and carrying out infringement judgment by combining the results of the two. The copyright authentication process is as shown in fig. 7, firstly inputting a 3D video frame to be detected, then carrying out preprocessing such as scaling and down sampling, then respectively carrying out DT-CWT conversion processing on the preprocessed video frame, extracting a watermark, calculating a watermark error rate B1 and a zero watermark characteristic constructed by DT-CWT + SVD, calculating a zero watermark characteristic error rate B2, judging whether B1 is less than T1 and whether B2 is more than T2, proving infringement, and if not, proving non-infringement. Where T1 and T2 are empirically set thresholds.

The embodiment has the following technical effects:

1. the invention relates to a watermark embedding and zero watermark mixing copyright protection method based on DIBR (depth image based rendering) synthesized 3D video, and the existing watermark scheme and zero watermark scheme are not sufficient for considering the characteristics of the DIBR synthesized 3D video. The frame synthesized after DIBR operation has horizontal movement of pixels relative to the original two-dimensional frame, so the DIBR operation can be regarded as an attack for changing the content of the video frame, which means that the protection scheme of the 3D video is not changed by DIBR.

2. The invention utilizes the watermark scheme to carry out authentication, and the zero watermark scheme carries out retrieval, and takes the advantages of the two schemes into account, thereby eliminating the contradiction of the two methods to a certain extent and improving the copyright identification precision.

The embodiment of the invention provides a copyright protection method based on the combination of watermark embedding and zero watermark construction of a DIBR 3D video, which comprises the steps of embedding a watermark into a 2D video frame, extracting zero watermark characteristics from the video frame embedded with the watermark, and registering the obtained characteristics in a database, wherein the watermark embedding method and the zero watermark construction method have robustness to DIBR operation, namely, the synthesized 3D video can still extract the zero watermark characteristics similar to the watermark and the construction, the whole process has invariance to the 3D video synthesized based on DIBR, namely, after the 2D video frame embedded with the watermark and a depth map are synthesized into a left view and a right view through a DIBR technology, the watermark can still be extracted from virtual left and right views, and the constructed zero watermark characteristics are not influenced by the DIBR operation, so that the aim of copyright protection of the DIBR 3D video is achieved. When a video which is possible to infringe is found in a network subsequently, firstly, zero watermark features are constructed, the extracted features and the feature sequences stored in the database are searched, if the similar zero watermark features can be matched, the watermark extraction operation is further executed on the video, and when the extracted watermark is similar to the original watermark, the video infringement can be finally judged.

The invention provides a specific embodiment applying the watermark comparison method, which comprises the following steps:

the difference between a video and an image is that there are hundreds of frames in a video, if each frame is embedded with a watermark, the time consumption is long and the memory is too much, but if the number of embedded frames is too small, the attacks such as frame loss and frame insertion cannot be handled, so in order to balance the processing efficiency and the copyright protection, the watermark embedding operation is performed every 10 frames, and the operation of each frame is the same, taking a certain video to be protected as an example, as shown in fig. 8, the size is 320 × 240, and since the subsequent 100 frames are similar to the first frame, only the first frame is shown.

The specific protection steps are as follows:

1. and (3) performing watermark embedding operation every 10 video frames, wherein the block size is 40 × 40, and watermark bits can be embedded by 48 bits: (320/40) × (240/40) ═ 8 × 6. The watermark to be embedded is 010101011100010111000101110001010000111011010000.

2. And (3) down-sampling the video embedded with the watermark, normalizing the down-sampled video to 320 × 320, averaging every 4 video frames to obtain 25 video frames, similarly setting 40-sized blocks for each video frame, constructing a zero-watermark feature according to the method, and connecting the zero-watermark feature in series to obtain a 25 × 8 × 8-1600-dimensional feature.

3. The watermarked 2D video frames, the watermark and the zero watermark feature of each video are registered in a database.

4. When a video which is possible to infringe is found in a network, firstly, zero watermark characteristics are constructed, matching and searching are carried out on the video and the characteristic sequences stored in the database, if matched contents exist in the database, the operation is further carried out, the watermark is extracted, the ber of the original watermark is calculated, and when the ber of the original watermark is smaller than a set threshold value, infringement can be judged, and copyright protection is realized.

The above examples describe the use of this method in a specific example, and experiments have also demonstrated the feasibility of this method. The method combines the watermark embedding method and the zero watermark construction method, realizes the balance of robustness, watermark invisibility and distinguishability, meets the requirement of copyright protection, and improves the precision of the copyright protection.

In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.

The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.

Claims

1. A watermark embedding method based on 3D video is characterized by comprising the following steps:

for any subblock in any two-dimensional video frame, performing multi-level DT-CWT decomposition on any component in the subblock color space to obtain a sub-band of the subblock under each level of decomposition;

modifying the amplitude matrix of the sub-band obtained by decomposing the sub-blocks in the set level range according to the watermark to be embedded to obtain the modified sub-band;

2. The 3D video-based watermark embedding method according to claim 1, wherein embedding all watermarks corresponding to the three-dimensional video to be embedded into the two-dimensional video frame to obtain a zero watermark of the three-dimensional video to be embedded specifically comprises:

performing SVD operation on the amplitude matrix of the sub-band of the watermark sub-block obtained under the set level decomposition to obtain a singular value corresponding to the watermark sub-block;

3. The method according to claim 1, wherein the modifying, according to the watermark to be embedded, the amplitude matrix of the sub-band obtained by decomposing the sub-blocks in the set number range to obtain the modified sub-band specifically comprises:

and obtaining the modified sub-band according to the phase matrix and the quantized sub-band amplitude matrix corresponding to the sub-block.

4. A watermark extraction method based on 3D video is characterized by comprising the following steps:

5. The method as claimed in claim 4, wherein the determining the watermark embedded in the three-dimensional video to be extracted according to a subband amplitude matrix obtained by decomposing each extracted sub-block in the three-dimensional video to be extracted in a set progression range specifically includes:

for any extracted subblock, quantizing an amplitude matrix of a subband obtained by decomposing the extracted subblock in a set progression range to obtain a quantized subband amplitude matrix corresponding to the extracted subblock;

6. A watermark comparison method based on 3D video is characterized by comprising the following steps:

the watermark embedding method based on 3D video according to any one of claims 1-3, wherein the embedded watermark and the zero watermark are obtained by processing the three-dimensional video to be protected;

the 3D video-based watermark extraction method according to any one of claims 4 to 5, wherein the three-dimensional video to be authenticated is processed to obtain an extracted embedded watermark and an extracted zero watermark;

7. The method as claimed in claim 6, wherein the determining whether the three-dimensional video to be authenticated violates the copyright of the three-dimensional video to be embedded according to the embedded watermark, the extracted embedded watermark, the zero watermark, and the extracted zero watermark includes:

8. A 3D video-based watermark embedding system, comprising:

and the watermark determining module is used for embedding the two-dimensional video frames according to all watermarks corresponding to the three-dimensional video to be embedded to obtain a watermark embedded video and a zero watermark of the three-dimensional video to be embedded.

9. A 3D video-based watermark extraction system, comprising:

the second blocking module is used for respectively blocking each two-dimensional video frame in the three-dimensional video to be extracted to obtain a plurality of extracted sub-blocks;

10. A 3D video-based watermark comparison system, comprising:

the watermark embedding module is used for processing the three-dimensional video to be protected according to the watermark embedding method based on the 3D video in any one of claims 1 to 3 to obtain an embedded watermark and a zero watermark;

the watermark extraction module is used for processing the three-dimensional video to be authenticated according to the 3D video-based watermark extraction method of any one of claims 4 to 5 to obtain an extracted embedded watermark and an extracted zero watermark;