CN116645260B - Digital watermark attack method based on conditional diffusion model - Google Patents
Digital watermark attack method based on conditional diffusion model Download PDFInfo
- Publication number
- CN116645260B CN116645260B CN202310926431.XA CN202310926431A CN116645260B CN 116645260 B CN116645260 B CN 116645260B CN 202310926431 A CN202310926431 A CN 202310926431A CN 116645260 B CN116645260 B CN 116645260B
- Authority
- CN
- China
- Prior art keywords
- image
- noise
- diffusion
- conditional
- watermark
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000009792 diffusion process Methods 0.000 title claims abstract description 100
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000012549 training Methods 0.000 claims abstract description 29
- 238000012545 processing Methods 0.000 claims abstract description 16
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 4
- 238000005070 sampling Methods 0.000 claims description 49
- 230000000873 masking effect Effects 0.000 claims description 18
- 230000002441 reversible effect Effects 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000000295 complement effect Effects 0.000 claims description 3
- 238000003786 synthesis reaction Methods 0.000 claims description 3
- 230000009191 jumping Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 5
- 238000005516 engineering process Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000005457 optimization Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000007792 addition Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0021—Image watermarking
- G06T1/005—Robust watermarking, e.g. average attack or collusion attack resistant
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Editing Of Facsimile Originals (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the technical field of image processing, and discloses a digital watermark attack method based on a conditional diffusion model, which comprises the following steps: step 1, establishing a diffusion process and a conditional denoising device; step 2, training a conditional denoising device according to a diffusion process to obtain a conditional diffusion model suitable for a digital watermark attack task; step 3, inputting an original image, and synthesizing a condition control image by using a condition control module; and 4, inputting a noise prediction module for iterative denoising to obtain the watermark-free image. The invention solves the problems of poor watermark attack effect, low image quality and the like caused by the fact that the traditional digital watermark attack method is limited by unknown watermark embedding method and strong watermark robustness, and improves the removal rate of watermark information in the image and the image quality after attack.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a digital watermark attack method based on a conditional diffusion model.
Background
With the development of the internet, the information security problem is increasingly highlighted, and the image information hiding technology represented by the digital watermarking technology can hide and extract secret information on the premise of not changing the content of a carrier medium, can easily bypass conventional security detection, is used for spreading secret information and malicious content by lawbreakers, and threatens the security of common users and networks. In order to interrupt secret communication by using the digital watermarking technology, the digital watermarking attack technology damages the hidden secret watermarking information in the image by modifying the image, so that a receiver cannot detect the existence of the watermarking information or cannot extract the hidden watermarking information correctly, thereby preventing secret communication.
Most of the existing digital watermarking methods use image processing technologies such as additive noise, image filtering, lossy compression and the like to destroy the hidden information of the image, and obtain better effects in the face of some early watermarking methods, but with the development of deep learning technology, the robustness of the digital watermarking technology is obviously improved, and normal extraction of the hidden information can be ensured after conventional image distortion processing, which means that malicious users can carry out secret communication by bypassing network security detection more easily.
In recent years, some scholars propose watermark attack methods based on self-Encoder (AE) and deep learning technologies such as generating countermeasure networks (Generative adversarial networks, GAN), however, the existing digital watermark attack methods still have some disadvantages, for example, CN107343119a is a digital image steganography data erasure method, and the method described in the patent cannot erase secret information hidden in a robust watermark; CN115358909a is a hidden digital watermark attack method and system based on SAD network, the method uses hole convolution to make watermark attack, the attack effect facing depth robust watermark is not known, and the image quality is greatly lost; CN115272039a is a GAN-based watermark attack method, a GAN-based watermark attack system, and a digital watermark embedding method, where the GAN needs to be retrained for different watermark methods, and has specificity, and cannot be applied to secret communication performed by using an unknown watermark method in an actual scene.
Disclosure of Invention
Aiming at the defects existing in the prior art, the invention provides the digital watermark attack method based on the conditional diffusion model, which considers the security problem of hidden communication by a malicious user by using a robust watermark technology under an actual network scene, solves the problems of poor watermark attack effect, low image quality and the like caused by the fact that the conventional digital watermark attack method is not known by a watermark embedding method and has strong watermark robustness, and improves the removal rate of watermark information in an image and the image quality after attack. Specifically, it is: the invention establishes a diffusion process and trains a condition denoising device, wherein the condition denoising device comprises a condition control module and a noise prediction module, the condition control module is responsible for destroying watermark information, and the condition control module outputs a condition control image by utilizing masking attack and an image patching network to erase the watermark information of the image; the noise prediction module is responsible for predicting noise added in a forward diffusion process, and taking a condition control image and random Gaussian noise as inputs to perform gradual denoising, so that a watermark-free image with high visual quality is finally obtained. The method trains the conditional denoising device by using the diffusion process to obtain a trained conditional diffusion model, and the image to be processed is input into the conditional diffusion model to obtain the image after watermark removal. The invention solves the problems of low watermark removal rate and low image quality of the prior digital watermark attack method, and can remove the watermark embedded in the image in the face of a watermark algorithm with higher robustness.
In order to solve the technical problems, the invention adopts the following technical scheme:
a digital watermark attack method based on a conditional diffusion model comprises the following steps:
step 1, establishing a diffusion process and a conditional denoising device;
the diffusion process comprises a forward diffusion process from left to right and a reverse diffusion process from right to left, wherein the forward diffusion process performs noise adding, and the reverse diffusion process performs noise removing; the condition denoising device comprises a condition control module and a noise prediction module, wherein the condition control module is used for destroying watermark information, erasing the watermark information of an image by utilizing a masking attack and image patching network and outputting a condition control image; the noise prediction module is used for optimizing image quality and comprises an up-sampling network and a down-sampling network, wherein the up-sampling network comprises a plurality of up-sampling residual modules, each up-sampling residual module comprises a plurality of residual networks and an up-sampling layer, the down-sampling network comprises a plurality of down-sampling residual modules, each down-sampling residual module comprises a plurality of residual networks and a down-sampling layer, and the up-sampling network and the down-sampling network are connected in a jumping manner, namely, the output of the down-sampling layer is used as the input of the corresponding up-sampling layer; the noise prediction module predicts the noise added in the forward diffusion process, and takes the random Gaussian noise in the condition control image and the noise adding process as input to carry out gradual denoising, so as to obtain a watermark-free image;
step 2, training the conditional denoising device according to the diffusion process:
inputting the training image into a conditional denoising device according to a diffusion process, and training the conditional denoising device according to the diffusion process to obtain a conditional diffusion model suitable for a digital watermark attack task;
step 3, inputting an original image, and synthesizing a condition control image by using a condition control module;
and 4, inputting a noise prediction module for iterative denoising to obtain the watermark-free image.
Further, in step 1, the diffusion process includes two processes, namely a left-to-right forward diffusion process, performing T-step noise adding, and normalizing the original imageConversion to random Gaussian noise->The method comprises the steps of carrying out a first treatment on the surface of the And a right-to-left reverse diffusion process, wherein the conditional denoising device is used for predicting the noise added previously and denoising gradually, and the specific steps of denoising in the forward diffusion process are as follows:
step 1.1, setting the diffusion step number T and Gaussian noise parameters；
Step 1.2, inputting an original image I and performing standardization processing to obtainCalculating diffusion coefficient according to current diffusion step sequence number t>；
Step 1.3, calculating the noise-added output of the t-th diffusion step，/>WhereinRepresenting noise->The standard gaussian distribution with mean zero matrix and covariance unity matrix E is represented.
Further, in step 2, the training steps of the conditional denoising device are as follows:
step 2.1, inputting an original image datasetWherein k is the sequence number of the image, +.>Representing the kth original image, selecting a training image from the data set for standardization processing during training to obtain +.>；
Step 2.2,The condition control image is obtained after the processing of the condition control module>The specific flow is as follows:
step 2.2.1 forPerforming random masking attacks, i.e.
；
Wherein the method comprises the steps ofRepresentation pair->Mask when performing a random masking attack, +.>The expression probability is +.>Bernoulli distribution, ->Representing a term-wise multiplication operation;
step 2.2.2, willThe input image patching network obtains a condition control image +.>；
Step 2.3, inputting the diffusion step number T, fromA random sampling diffusion step t is carried out, and embedding transformation is carried out to obtain time embedding +.>；
Step 2.4, willAnd->Input noise prediction module for outputting prediction noise +.>；
Step 2.5, calculatingLoss ofGradient descent and network weight update are carried out;
and 2.6, repeating the steps 2.1-2.5 until the loss function converges, and completing the training of the conditional denoising device.
Further, in step 3, unlike the training phase, the condition control uses the improved masking attack in the reasoning phase, and the generation process of the condition control image is as follows:
step 3.1, inputting an original image I and performing standardization processing to obtain；
Step 3.2, pairWatermark information attack is carried out to obtain->And->：
；
Wherein the method comprises the steps ofRepresentation pair->Masking when watermark information attack is carried out; b (1, 0.5) represents a Bernoulli distribution with a probability of 0.5,/L>Representing a term-wise multiplication operation;
step 3.3, willAnd->In the input image inpainting network, output +.>And->；
Step 3.4, performing mask complementary synthesis to obtain a condition control image：
。
In step 4, the condition control image obtained in step 3 is input into a noise prediction module for iterative denoising, so as to obtain a watermark-free imageThe method comprises the following specific steps:
step 4.1, initializing t=t,，/>is random Gaussian noise, when 0 is less than T is less than T>The predicted value is output for the t+1st reverse diffusion step;
step 4.2, performing embedded transformation on t to obtain；
Step 4.3, inputting a condition control imageTime embedding->And the predicted value of the last reverse diffusion step +.>Obtaining the predicted noise->；
Step 4.4, calculating the current time step predicted image：
；
Wherein the method comprises the steps of，/>Is the Gaussian noise parameter corresponding to step t, < ->Variance parameter indicating the diffusion step at step t, < ->When t > 1, < > is given>Z is a random standard Gaussian distribution, +.>A diffusion coefficient representing the t-th diffusion step;
step 4.5, repeating the steps 4.2-4.4 when the value of 0 is less than T until the value is output。
Compared with the prior art, the invention has the advantages that:
(1) The invention can destroy watermark information embedded by most watermark algorithms, including robust watermark based on deep learning. The watermark attack mode used in the invention is based on masking attack design, can effectively remove hidden information embedded by a depth robust watermark method, and simultaneously ensures that the image after watermark attack can still keep high visual quality in an image optimization stage, comprehensively replaces original image information, ensures that watermark information cannot be recovered in image quality optimization, and greatly improves the success rate of watermark attack. The existing watermark attack methods have certain reversibility, and even if watermark information is destroyed during watermark attack, partial information can be recovered during subsequent image optimization, so that watermark attack failure is caused.
(2) The image processed by the method has higher fidelity and is equivalent to the visual quality of the original image. The image optimization method used in the invention is designed based on a conditional diffusion model, and meanwhile, the condition control image provides the whole image information, so that the restoration of the detail structure of the image is ensured. The existing watermark attack method often causes larger and irreversible damage to the image quality in an attack mode used during watermark attack, thereby causing distortion, detail deletion and blurring of the image.
(3) The watermark attack method provided by the invention does not need to retrain the specific watermark algorithm. Because the watermark attack is completely replaced, the image trained by the invention does not need to be processed by using a watermark algorithm, and the method is applicable to the attack of an unknown watermark algorithm. Some existing watermark attack algorithms need to be trained by using a plurality of different watermark algorithms as training sets, and the specific watermark algorithm needs to be retrained in an actual scene, and an unknown watermark algorithm cannot be attacked. The digital watermark attack method provided by the invention has a good removal effect on information embedded by both a steganography method and a watermarking method.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a digital watermark attack method based on a conditional diffusion model according to the present invention;
FIG. 2 is a schematic illustration of a diffusion process established in accordance with the present invention;
FIG. 3 is a schematic diagram of a conditional denoiser according to the present invention;
FIG. 4 is a schematic diagram of the generation of a condition control image during the training phase of the present invention;
FIG. 5 is a schematic diagram of the generation of a condition controlled image at the reasoning stage of the present invention;
FIG. 6 is a schematic diagram of a training process of a conditional diffusion model according to the present invention;
FIG. 7 is a schematic diagram of a use flow of the present invention.
Detailed Description
The invention will be further described with reference to the accompanying drawings and specific examples.
Aiming at the security problem caused by hidden communication of lawless persons in a network space by utilizing a watermark technology, the embodiment provides a digital watermark attack method based on a conditional diffusion model, as shown in fig. 1, which comprises the following steps:
step 1, establishing a diffusion process and a conditional denoising device;
the diffusion process includes two processes, a forward diffusion process from left to right and a reverse diffusion process from right to left, the forward diffusion process performing noise addition, the reverse diffusion process performing noise removal.
As shown in fig. 2, the forward diffusion process from left to right performs T-step noise addition to normalize the original image and output the normalized resultConversion to random Gaussian noise->The method comprises the steps of carrying out a first treatment on the surface of the A right-to-left reverse diffusion process, in which forward diffusion is performed, using a conditional denoiser to predict previously added noise and progressively denoiseThe specific steps of the process for adding noise are as follows:
step 1.1, setting the diffusion step number T and Gaussian noise parameters；
Step 1.2, inputting an original image I and performing standardization processing to obtainCalculating diffusion coefficient according to current diffusion step sequence number t>；
Step 1.3, calculating the noise-added output of the t-th diffusion step，/>WhereinRepresenting noise->The standard Gaussian distribution with the mean value of zero matrix and the covariance of unit matrix E is represented, and the noise adding output of any diffusion step can be calculated according to the formula only by the input.
As shown in fig. 3, the conditional denoising device comprises a conditional control module and a noise prediction module, wherein the conditional control module is used for destroying watermark information, erasing the watermark information of an image by using a masking attack and an image patching network, and outputting a conditional control image; the noise prediction module is used for optimizing image quality and comprises an up-sampling network and a down-sampling network, wherein the up-sampling network consists of 4 up-sampling residual modules, each up-sampling residual module comprises 2 residual networks and an up-sampling layer, the down-sampling network consists of 4 down-sampling residual modules, and each down-sampling residual module comprises 2 residual networks and a down-sampling layer. The up-sampling layer is a bicubic linear interpolation layer with a scale factor of 2 and a convolution layer with a step size of 1, while the down-sampling layer is a convolution layer with a step size of 2. The up-sampling network and the down-sampling network are connected in a skip manner, i.e. the output of the down-sampling layer serves as the input of the corresponding up-sampling layer. The noise prediction module predicts the noise added in the forward diffusion process, and takes the random Gaussian noise in the condition control image and the noise adding process as input to carry out gradual denoising, so as to obtain the watermark-free image with high visual quality. It should be noted that, the number of residual modules of the up-sampling network and the down-sampling network, the number of steps of the convolution layer, and other parameters are all examples.
Step 2, training the conditional denoising device according to the diffusion process:
and inputting the training image into a conditional denoising device according to a diffusion process, and training the conditional denoising device according to the diffusion process to obtain a conditional diffusion model suitable for a digital watermark attack task.
As shown in fig. 6, the training steps of the conditional denoiser are as follows:
step 2.1, inputting an original image datasetWherein k is the sequence number of the image, +.>Representing the kth original image, selecting a training image from the data set for standardization processing during training to obtain +.>；
Step 2.2,The condition control image is obtained after the processing of the condition control module>The specific flow is as follows in connection with fig. 4:
step 2.2.1 forFollow-upMechanical masking attacks, i.e.
；
Wherein the method comprises the steps ofRepresentation pair->Mask when performing a random masking attack, +.>The expression probability is +.>Bernoulli distribution, ->Representing a term-wise multiplication operation;
step 2.2.2, willThe input image patching network obtains a condition control image +.>；
Step 2.3, inputting the diffusion step number T, fromA random sampling diffusion step t is carried out, and embedding transformation is carried out to obtain time embedding +.>；
Step 2.4, willAnd->Input noise prediction module for outputting prediction noise +.>；
Step 2.5, calculating lossesGradient descent and network weight update are carried out;
and 2.6, repeating the steps 2.1-2.5 until the loss function converges, and completing the training of the conditional denoising device.
And step 3, inputting an original image, and synthesizing a condition control image by using a condition control module.
The original image is input to a condition control module, and the composite condition control image is covered with a mask. Unlike the training phase, the condition control uses a modified masking attack in the reasoning phase, and the condition control image is generated as shown in fig. 5:
step 3.1, inputting an original image I and performing standardization processing to obtain；
Step 3.2, pairThe watermark information attack shown in fig. 5 is performed to obtain +.>And->：
；
Wherein the method comprises the steps ofRepresentation pair->Masking when watermark information attack is carried out; b (1, 0.5) represents a Bernoulli distribution with a probability of 0.5,/L>Representing a term-wise multiplication operation;
step 3.3, willAnd->In the input image inpainting network, output +.>And->；
Step 3.4, performing mask complementary synthesis to obtain a condition control image：
。
And 4, inputting a noise prediction module for iterative denoising to obtain the watermark-free image.
Inputting the condition control image obtained in the step 3 into a noise prediction module for iterative denoising to obtain a watermark-free imageAs shown in fig. 7, the specific steps are as follows:
step 4.1, initializing t=t,，/>is random Gaussian noise, when 0 is less than T is less than T>The predicted value is output for the t+1st reverse diffusion step;
step (a)4.2, performing embedded transformation on t to obtain；
Step 4.3, inputting a condition control imageTime embedding->And the predicted value of the last reverse diffusion step +.>Obtaining the predicted noise->；
Step 4.4, calculating the current time step predicted image：
；
Wherein the method comprises the steps of，/>Is the Gaussian noise parameter corresponding to step t, < ->Variance parameter indicating the diffusion step at step t, < ->When t > 1, < > is given>Z is a random standard Gaussian distribution, +.>A diffusion coefficient representing the t-th diffusion step;
step 4.5, repeating the steps 4.2-4.4 when the value of 0 is less than T until the value is output。
In summary, the present invention addresses the problem of digital watermark attack, and the existing method cannot simultaneously meet the following requirements: 1. the method has higher watermark information removal rate, and ensures that the embedded watermark information cannot be recovered after being attacked. 2. The method has enough fidelity, and the processed image maintains the image quality similar to the original image. 3. The method has enough universality and can attack most watermarking methods without retraining.
In order to meet the three-point requirements, the invention provides a watermark attack method based on a conditional diffusion model from two aspects. Firstly, in order to ensure the removal rate of watermark information, the invention uses a condition control module based on masking attack to replace original image information, so as to ensure that the watermark information cannot be recovered after being attacked. Then, in order to ensure that the image quality of the processed image is consistent with that of the original image, the invention uses a conditional diffusion model to optimize the image quality, and the optimized image maintains the detail structure of the original image. The data set used in training of the invention does not need to be processed by a watermark algorithm, and can effectively attack an unknown watermark algorithm.
It should be understood that the above description is not intended to limit the invention to the particular embodiments disclosed, but to limit the invention to the particular embodiments disclosed, and that various changes, modifications, additions and substitutions can be made by those skilled in the art without departing from the spirit and scope of the invention.
Claims (2)
1. The digital watermark attack method based on the conditional diffusion model is characterized by comprising the following steps:
step 1, establishing a diffusion process and a conditional denoising device;
the diffusion process comprises a forward diffusion process from left to right and a reverse diffusion process from right to left, wherein the forward diffusion process performs noise adding, and the reverse diffusion process performs noise removing; the condition denoising device comprises a condition control module and a noise prediction module, wherein the condition control module is used for destroying watermark information, erasing the watermark information of an image by utilizing a masking attack and image patching network and outputting a condition control image; the noise prediction module is used for optimizing image quality and comprises an up-sampling network and a down-sampling network, wherein the up-sampling network comprises a plurality of up-sampling residual modules, each up-sampling residual module comprises a plurality of residual networks and an up-sampling layer, the down-sampling network comprises a plurality of down-sampling residual modules, each down-sampling residual module comprises a plurality of residual networks and a down-sampling layer, and the up-sampling network and the down-sampling network are connected in a jumping manner, namely, the output of the down-sampling layer is used as the input of the corresponding up-sampling layer; the noise prediction module predicts the noise added in the forward diffusion process, and takes the random Gaussian noise in the condition control image and the noise adding process as input to carry out gradual denoising, so as to obtain a watermark-free image;
in the step 1, the diffusion process comprises two processes, namely a forward diffusion process from left to right, a noise adding process with the total diffusion step number of T is executed, and the result I after the original image is normalized 0 Conversion to random Gaussian noise I T The method comprises the steps of carrying out a first treatment on the surface of the And a right-to-left reverse diffusion process, wherein the conditional denoising device is used for predicting the noise added previously and denoising gradually, and the specific steps of denoising in the forward diffusion process are as follows:
step 1.1, setting a total diffusion step number T and a Gaussian noise parameter beta 0 ，β 1 ，…，β T ；
Step 1.2, inputting an original image I and performing standardization processing to obtain the I 0 Calculating diffusion coefficient according to current diffusion step number t
Step 1.3, calculating the noise adding output corresponding to the current diffusion step number tepsilon-N (0, E), wherein epsilon represents noise, and N (0, E) represents standard Gaussian distribution with mean value of zero matrix and covariance of identity matrix E;
step 2, training the conditional denoising device according to the diffusion process:
inputting the training image into a conditional denoising device according to a diffusion process, and training the conditional denoising device according to the diffusion process to obtain a conditional diffusion model suitable for a digital watermark attack task;
in step 2, the training steps of the conditional denoiser are as follows:
step 2.1, inputting an original image datasetWhere k is the sequence number of the image, I k Representing the kth original image, selecting a training image from the data set for standardization processing during training to obtain I 0 ；
Step 2.2, I 0 The condition control image I is obtained after the processing of the condition control module cond The specific flow is as follows:
step 2.2.1 for I 0 Performing random masking attacks, i.e.
I mask ＝I 0 *mask 1 ，mask 1 ～B(1，p)；
Wherein mask is used for 1 Representation pair I 0 Mask when random masking attack is performed, B (1, p) represents bernoulli distribution with probability p, and x represents multiplication operation item by item;
step 2.2.2, step I mask The input image patching network obtains a condition control image I cond ；
Step 2.3, inputting a total diffusion step number T, randomly sampling the diffusion step number T from {1,2, & gt, T } and performing embedding transformation to obtain a time embedded T e ；
Step 2.4, I cond And t e Input noise prediction module for outputting prediction noise
Step 2.5, calculating lossesGradient descent and network weight update are carried out;
step 2.6, repeating the steps 2.1-2.5 until the loss function converges, and finishing the training of the conditional denoising device;
step 3, inputting an original image, and synthesizing a condition control image by using a condition control module;
in step 3, unlike the training phase, the condition control uses an improved masking attack in the reasoning phase, and the condition control image is generated as follows:
step 3.1, inputting an original image I and performing standardization processing to obtain the I 0 ；
Step 3.2, pair I 0 Watermark information attack is carried out to obtain I m1 And I m2 ：
I m1 ＝I 0 *mask 2 ，I m2 ＝I 0 *(1-mask 2 )，mask 2 ～B(1，0.5)；
Wherein mask 2 Representation pair I 0 Masking when watermark information attack is carried out; b (1, 0.5) represents a bernoulli distribution with a probability of 0.5, which represents a multiplication by item operation;
step 3.3, I m1 And I m2 Input image inpainting network, output I c1 And I c2 ；
Step 3.4, performing mask complementary synthesis to obtain a condition control image I c ：
I c ＝I c1 *(1-mask 2 )+I c2 *mask 2 ；
And 4, inputting a noise prediction module for iterative denoising to obtain the watermark-free image.
2. The digital watermark attack method based on conditional diffusion model according to claim 1, wherein in step 4, the conditional control image obtained in step 3 is input with noise pre-emphasisThe testing module carries out iterative denoising to obtain a watermark-free imageThe method comprises the following specific steps:
step 4.1, initializing t=t,I T for random Gaussian noise, when 0 < T < T +.>The predicted value is output for the t+1st reverse diffusion step;
step 4.2, performing embedded transformation on t to obtain t e ；
Step 4.3, inputting a condition control image I c Time-embedded t e And the predicted value of the last reverse diffusion stepObtain prediction noise->
Step 4.4, calculating the current time step predicted image
Wherein alpha is t ＝1-β t ，β t Is the gaussian noise parameter corresponding to step t,representing the variance parameter corresponding to step t, +.>When t > 1, ">z is a random standard Gaussian distribution, +.>Representing the diffusion coefficient of the t-th step;
step 4.5, repeating the steps 4.2-4.4 until the output is achieved when T is more than 0 and less than T
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310926431.XA CN116645260B (en) | 2023-07-27 | 2023-07-27 | Digital watermark attack method based on conditional diffusion model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310926431.XA CN116645260B (en) | 2023-07-27 | 2023-07-27 | Digital watermark attack method based on conditional diffusion model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116645260A CN116645260A (en) | 2023-08-25 |
| CN116645260B true CN116645260B (en) | 2024-02-02 |
Family
ID=87643737
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310926431.XA Active CN116645260B (en) | 2023-07-27 | 2023-07-27 | Digital watermark attack method based on conditional diffusion model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116645260B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117376484A (en) * | 2023-12-05 | 2024-01-09 | 北京邮电大学 | Electronic license anti-counterfeiting oriented generation type steganography method |
| CN117911230A (en) * | 2024-03-19 | 2024-04-19 | 清华大学 | Image invisible watermark embedding detection processing method and device based on diffusion model |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113177882A (en) * | 2021-04-29 | 2021-07-27 | 浙江大学 | Single-frame image super-resolution processing method based on diffusion model |
| CN113935882A (en) * | 2021-09-27 | 2022-01-14 | 暨南大学 | Watermark removing method, device, equipment and medium based on convolution characteristic fusion |
| CN114359009A (en) * | 2021-12-28 | 2022-04-15 | 宁波大学科学技术学院 | Watermark embedding method, watermark embedding network construction method and system of robust image based on visual perception and storage medium |
| CN114549273A (en) * | 2022-02-28 | 2022-05-27 | 中山大学 | Self-adaptive robust watermark embedding method and system based on deep neural network |
| CN114820398A (en) * | 2022-07-01 | 2022-07-29 | 北京汉仪创新科技股份有限公司 | Image font replacing method, system, equipment and medium based on diffusion model |
| CN115660931A (en) * | 2022-11-01 | 2023-01-31 | 南京信息工程大学 | Robust watermarking method based on Transformer and denoising diffusion model |
| CN115908187A (en) * | 2022-12-07 | 2023-04-04 | 北京航空航天大学 | Image characteristic analysis and generation method based on rapid denoising diffusion probability model |
| CN116304701A (en) * | 2023-03-13 | 2023-06-23 | 西安电子科技大学 | HRRP sample generation method based on conditional denoising diffusion probability model |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20230103638A1 (en) * | 2021-10-06 | 2023-04-06 | Google Llc | Image-to-Image Mapping by Iterative De-Noising |
-
2023
- 2023-07-27 CN CN202310926431.XA patent/CN116645260B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113177882A (en) * | 2021-04-29 | 2021-07-27 | 浙江大学 | Single-frame image super-resolution processing method based on diffusion model |
| CN113935882A (en) * | 2021-09-27 | 2022-01-14 | 暨南大学 | Watermark removing method, device, equipment and medium based on convolution characteristic fusion |
| CN114359009A (en) * | 2021-12-28 | 2022-04-15 | 宁波大学科学技术学院 | Watermark embedding method, watermark embedding network construction method and system of robust image based on visual perception and storage medium |
| CN114549273A (en) * | 2022-02-28 | 2022-05-27 | 中山大学 | Self-adaptive robust watermark embedding method and system based on deep neural network |
| CN114820398A (en) * | 2022-07-01 | 2022-07-29 | 北京汉仪创新科技股份有限公司 | Image font replacing method, system, equipment and medium based on diffusion model |
| CN115660931A (en) * | 2022-11-01 | 2023-01-31 | 南京信息工程大学 | Robust watermarking method based on Transformer and denoising diffusion model |
| CN115908187A (en) * | 2022-12-07 | 2023-04-04 | 北京航空航天大学 | Image characteristic analysis and generation method based on rapid denoising diffusion probability model |
| CN116304701A (en) * | 2023-03-13 | 2023-06-23 | 西安电子科技大学 | HRRP sample generation method based on conditional denoising diffusion probability model |
Non-Patent Citations (3)
| Title |
|---|
| DENOISING DIFFUSION PROBABILISTIC MODELS AS A DEFENSE AGAINST ADVERSARIAL ATTACKS;Lars Ankile et al;《arXiv》;全文 * |
| Denoising Diffusion Semantic Segmentation with Mask Prior Modeling;Zeqiang Lai et al;《arxIV》;全文 * |
| 结合扩散模型图像编辑的图文检索后门攻击;杨舜 等;《计算机科学与探索》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116645260A (en) | 2023-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116645260B (en) | Digital watermark attack method based on conditional diffusion model | |
| Elharrouss et al. | An image steganography approach based on k-least significant bits (k-LSB) | |
| Qin et al. | An inpainting-assisted reversible steganographic scheme using a histogram shifting mechanism | |
| Peng et al. | Adaptive reversible data hiding scheme based on integer transform | |
| Kumar et al. | Enhanced pairwise IPVO-based reversible data hiding scheme using rhombus context | |
| CN110968845B (en) | Detection method for LSB steganography based on convolutional neural network generation | |
| Zheng et al. | A new reversible watermarking scheme using the content-adaptive block size for prediction | |
| CN112634120A (en) | Image reversible watermarking method based on CNN prediction | |
| CN115345768A (en) | Robust watermark attack method and system based on neural network | |
| Zhu et al. | Destroying robust steganography in online social networks | |
| CN115908095A (en) | Hierarchical attention feature fusion-based robust image watermarking method and system | |
| CN115358909A (en) | Hidden digital watermark attack method and system based on SAD network | |
| CN116091288A (en) | Diffusion model-based image steganography method | |
| Maity et al. | Genetic algorithms for optimality of data hiding in digital images | |
| CN116112685A (en) | Image steganography method based on diffusion probability model | |
| Zhu et al. | Image sanitization in online social networks: A general framework for breaking robust information hiding | |
| Wahed et al. | A simplified parabolic interpolation based reversible data hiding scheme | |
| Zhou et al. | Reversible data hiding algorithm with high imperceptibility based on histogram shifting | |
| Vashishtha et al. | Least significant bit matching steganalysis based on feature analysis | |
| CN113766084B (en) | Reversible information hiding method and system for enhancing image smoothness | |
| Maity et al. | Intelligent modified difference expansion for reversible watermarking | |
| Maity et al. | Reversible image watermarking using modified difference expansion | |
| Rebahi et al. | Image Watermarking Technique Using Convolutional Autoencoder | |
| CN112561773B (en) | Deep disturbance-based countersteganography method | |
| CN116630124A (en) | High-robustness watermark embedding method based on AI model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |