CN114359011A - Neural network watermark embedding method and device, electronic equipment and storage medium - Google Patents
Neural network watermark embedding method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114359011A CN114359011A CN202210016799.8A CN202210016799A CN114359011A CN 114359011 A CN114359011 A CN 114359011A CN 202210016799 A CN202210016799 A CN 202210016799A CN 114359011 A CN114359011 A CN 114359011A
- Authority
- CN
- China
- Prior art keywords
- neural network
- sequence
- label
- picture
- watermark
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention relates to a neural network watermark embedding method, a neural network watermark embedding device, electronic equipment and a storage medium. The invention discloses a neural network watermark embedding method, which comprises the following steps: obtaining a key K, wherein the key K corresponds to a unique timestamp T; randomly selecting n pictures in an original training set, wherein n is larger than 1, and the original training set comprises a plurality of pictures and a label corresponding to each picture; scrambling and encrypting the pictures through a chaotic sequence generated by a secret key K to obtain n pictures of a trigger set, and putting the n pictures and corresponding labels into the trigger set; and putting the original training set and the trigger set into a neural network to be embedded with the watermark for training to obtain the neural network embedded with the watermark. According to the neural network watermark embedding method, the trigger set is generated on the premise that the image characteristics are kept unchanged, the original label is given to the trigger set, and the ownership of the model is protected on the premise that the decision boundary of the original classification task is not influenced.
Description
Technical Field
The present invention relates to the field of information hiding, and in particular, to a neural network watermark embedding method, apparatus, electronic device, and storage medium.
Background
As deep learning models become more widely used, many large companies begin to train deep learning models using the large amount of proprietary data and computing resources they own, and then deploy and commercialize them for distribution to other users as a way to profit. However, this may cause a copyright dispute problem, and thus a secure and reliable copyright protection method is required to protect the interests of the owner.
In recent years, researchers have proposed a watermarking method suitable for deep learning model copyright protection by taking the idea of the original digital watermarking method as a reference. Watermarking methods based on deep learning copyright protection can be roughly divided into two types: one is called white-box watermark, which realizes the embedding of watermark by modifying the weight in the model, and the principle of the method is similar to that of the original digital watermark; another method, called black-box watermarking, does not require access to the internal parameters of the network, and it implements the embedding of watermark information by using a specific trigger set as the input of the neural network, outputting a specified label, and finally verifying the ownership of the network by comparing the correctness of the specified label and the output label.
For white-box watermarking, Uchida et al make a first attempt by taking the idea of digital watermarking as a reference, and realize the embedding of the watermark by modifying some weight parameters in the network, and the original performance of the network is not affected after the watermark information is embedded. Wang and Kerschbaum et al found that the scheme of Uchida modified the statistical distribution of the model, which would result in an attacker being able to not only detect the watermark, but even extract its embedded length and delete it by a cover attack. To improve the stability and security of the embedded watermark, Wang and Wu et al, based on the method of Uchida, sift out some weights that have relatively small impact on the performance of the embedding model, and use a separate neural network to generate a matrix for embedding watermark information. Wang and Kerschbaum et al propose a white-box watermarking method based on a generation countermeasure network, which makes it impossible to distinguish the parameter distribution of a neural network with an embedded watermark from that without a watermark, and therefore is difficult to detect. Due to the limitation of white-box watermarks, watermark embedding needs to access and extract internal parameters and structures of the model, and some scholars propose a method for adding watermarks in a neural network model in a black-box mode for the purpose of referencing the thought of backdoor attack so that details such as model parameters and the like can not be extracted during watermark verification.
For black box watermarking, Adi et al propose for the first time to embed abstract images as backdoor watermarks into neural networks, thereby ensuring ownership of the model without access and without internal parameters and structures of the model; on the basis of Adi et al, Zhang et al propose a black box method based on three different ways of text, noise and irrelevant pictures as a trigger set; in order to resist watermark attacks such as parameter trimming and adversarial trimming coverage, Merrer et al propose a trigger set construction method based on decision boundaries, which not only protects the model but also improves the performance of the model, but also has the problem of easily causing false positives; because the correlation between the trigger set constructed by the prior method and the original training set is not large, the watermark is easy to erase by retraining and fine tuning, Li et al propose a self-defined filter, modify on the original training set, the embedding process can only be carried out in the initial training of the model, which also enables the method to resist the attack method of deleting the watermark by fine tuning or incremental training, but the method has the problems that the model needs to be trained from the beginning for embedding the watermark and the size of the watermark is limited; in the black box method, the watermark is embedded by setting a trigger set with a specified label, however, the behaviors can affect the decision boundary of an original model on an original task, and Zhong et al propose a new black box watermark frame, and realize the watermark embedding in the model by adding a new label in the classification task of the model, however, the method has the hidden danger that the watermark is easily detected, so an invisible watermark embedding/verifying protocol is needed to resist the fraud attack.
Disclosure of Invention
Based on this, the present invention provides a neural network watermark embedding method, apparatus, electronic device and storage medium, which generates a trigger set and gives an original tag to the trigger set on the premise of keeping image characteristics unchanged, so that the ownership of the model is protected on the premise of not affecting the decision boundary of the original classification task, and the intellectual property rights of the owner are maintained while the performance precision of the model is almost the same as that of the original model.
In a first aspect, the present invention provides a neural network watermark embedding method, including the following steps:
obtaining a key K, wherein the key K corresponds to a unique timestamp T;
randomly select n pictures (P) in the original training set1、P2、…、Pn) Wherein n is greater than 1, wherein the original training set comprises a plurality of pictures and a label corresponding to each picture;
chaotic sequence pair picture (P) generated by secret key K1、P2、…、Pn) Scrambling and encrypting are carried out to obtain n pictures (Pe) of the trigger set1、Pe2、…、Pen) The n pictures (Pe)1、Pe2、…、Pen) And its corresponding label (L)1、L2、…、Ln) Putting the trigger set;
and putting the original training set and the trigger set into a neural network to be embedded with the watermark for training, so that the neural network to be embedded with the watermark learns the mapping between the trigger set and the corresponding label while learning the original classification task, and obtaining the neural network embedded with the watermark.
Further, when verifying ownership of the neural network, the method further comprises:
acquiring the n pictures (P)1、P2、…、Pn) And the label (L) corresponding thereto1、L2、…、Ln) And the key K;
chaotic sequence pair picture (P) generated by secret key K1、P2、…、Pn) Scrambling and encrypting are carried out to obtain a trigger set picture (Pe)1、Pe2、…、Pen)；
Inputting the trigger set picture into a neural network to be verified to obtain a label (R) output by the neural network1、R2、…、Rn)；
By comparing the labels (R) output by the neural network1、R2、…、Rn) And said original label (L)1、L2、…、Ln) And obtaining the verification result of the neural network.
Further, the chaotic sequence generated by the key K is used for picture (P)1、P2、…、Pn) Scrambling and encrypting are carried out to obtain n pictures (Pe) of the trigger set1、Pe2、…、Pen) The method comprises the following steps:
for each picture PiA 1 is to PiDivided into multiple subblocks of size M B(1,1)，…，B(w/M,h/M)}; wherein w and h represent the width and height of the image, respectively; m is a preset numerical value;
dividing the first 4 bits of the secret key K by 10000 to obtain K1And the last 4 bits are divided by 10000 to give k2Each as x0Inputting the following logistic chaotic mapping formula to generate a pseudo-random sequence Q1，Q2：
xn+1＝λ*xn*(1-xn)
Q1＝{x0,x1,…,xnIn which x0＝k1
Q2＝{x0,x1,…,xnIn which x0＝k2
Wherein, λ ∈ [0, 4 ]]，xn∈[0，1](ii) a n is the number of iterations;
will random sequence Q1，Q2Sorting to generate an integer scrambling sequence O1，O2；
Using the sequence O1Scrambling each partition using the sequence O2Scrambling the pixel value in each block to obtain a trigger set picture Pei。
Further, λ ═ 4.
Further, a random sequence Q1，Q2Sorting to generate an integer scrambling sequence O1，O2The method comprises the following steps:
obtaining an integer scrambling sequence O by using the following formula1，O2：
O1＝{r1,r2,…,rn+1}
Wherein r isiIs xiAt Q1Rank values in the sequence; r isiIs a positive integer;
O2＝{r1,r2,…,rn+1}
wherein r isiIs xiAt Q2Rank values in the sequence; r isiIs a positive integer.
Further, the sequence O is used1Scrambling each partition using the sequence O2Scrambling the pixel value in each block to obtain a trigger set picture PeiThe method comprises the following steps:
for each picture PiFor all subblocks { B(1,1)，…，B(w/M,h/M)Sequencing to obtain the number {1, 2, …, n +1} of each subblock; wherein n is the sequence O1N in (1) are equal;
according to the sequence O1＝{r1,r2,…,rn+1The ordering of each sub-block is filled in the number of the sub-block in the sequence O1Position in (2), rearranging the order of each sub-block;
sequencing all pixel points in each subblock to obtain the serial number {1, 2, …, n +1} of each pixel point; wherein n is the sequence O2N in (1) are equal;
according to the sequence O2＝{r1,r2,…,rn+1Sequencing, namely filling the serial number of each pixel point into the pixel point in a sequence O2And (5) rearranging the sequence of each pixel point.
Further, by comparing the labels (R) output by the neural network1、R2、…、Rn) And said original label (L)1、L2、…、Ln) Obtaining a verification result of the neural network, including:
calculating a label (R) of the neural network output using the following formula1、R2、…、Rn) And said original label (L)1、L2、…、Ln) The consistency of (2):
V(t＝{f(Pk)，f(Pek)}nk＝1，τ)＝{True，False}
wherein V () is a consistency function for verifying the output label and the real label, f () is a label output after the network model processes the picture, PkAnd PekRespectively representing an original training set and a trigger set; k is the number of rounds of the current verification; t represents f (P)k) And f (Pe)k) Outputting the ratio of the number and the total number of the same results in the label; τ is a preset verification threshold;
and when t is calculated to be more than or equal to tau, the ownership of the verification result of the neural network is verified.
In a second aspect, the present invention further provides a neural network watermark embedding apparatus, including:
the key acquisition module is used for acquiring a key K, and the key K corresponds to a unique timestamp T;
an original picture selection module for randomly selecting n pictures (P) in an original training set1、P2、…、Pn) Wherein n is greater than 1, wherein the original training set comprises a plurality of pictures and a label corresponding to each picture;
a trigger set acquisition module for aligning the picture (P) by the chaos sequence generated by the secret key K1、P2、…、Pn) Scrambling and encrypting are carried out to obtain n pictures (Pe) of the trigger set1、Pe2、…、Pen) The n pictures (Pe)1、Pe2、…、Pen) And its corresponding label (L)1、L2、…、Ln) Putting the trigger set;
and the watermark embedding module is used for putting the original training set and the trigger set into a neural network to be embedded with the watermark for training, so that the neural network to be embedded with the watermark learns the mapping between the trigger set and the corresponding label while learning the original classification task, and the neural network embedded with the watermark is obtained.
In a third aspect, the present invention provides an electronic device, including:
at least one memory and at least one processor;
the memory for storing one or more programs;
when executed by the at least one processor, cause the at least one processor to implement the steps of a neural network watermark embedding method according to any one of the first aspect of the present invention.
In a fourth aspect, the present invention also provides a computer-readable storage medium, characterized in that:
the computer readable storage medium stores a computer program which, when executed by a processor, implements the steps of a neural network watermark embedding method according to any one of the first aspect of the present invention.
According to the neural network watermark embedding method, the neural network watermark embedding device, the electronic equipment and the storage medium, the watermark trigger set is generated by endowing the original label to the image after the image is scrambled through a scrambling mode for keeping the image characteristics, and the classification task of the trigger set is well learned by a model on the premise that the decision boundary of the original classification task is not distorted. Aiming at the black box counterfeiting attack in the neural network copyright protection, a unique watermark protocol mode is provided, namely when an attacker obtains the neural network of an owner, the attacker wants to forge a watermark belonging to the attacker through the same watermark construction mode, and when the attacker applies a secret key to a third-party copyright authentication company for counterfeiting the watermark, the application time is recorded, so that even if the counterfeiting watermark is successfully embedded, the authenticity of the watermark can be verified by comparing the application time, and the watermark embedded through the mode can resist the black box counterfeiting attack.
For a better understanding and practice, the invention is described in detail below with reference to the accompanying drawings.
Drawings
Fig. 1 is a schematic flow chart of a neural network watermark embedding method provided by the present invention;
FIG. 2 is a schematic diagram of a watermark embedding and watermark verification process in one embodiment of the present invention;
FIG. 3 is a flow diagram of a trigger set construction process in one embodiment of the invention;
fig. 4 is a schematic structural diagram of a neural network watermark embedding apparatus provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
It should be understood that the embodiments described are only some embodiments of the present application, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without any creative effort belong to the protection scope of the embodiments in the present application.
The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the embodiments of the present application. As used in the examples of this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the application, as detailed in the appended claims. In the description of the present application, it is to be understood that the terms "first," "second," "third," and the like are used solely to distinguish one from another and are not necessarily used to describe a particular order or sequence, nor are they to be construed as indicating or implying relative importance. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art as appropriate.
Further, in the description of the present application, "a plurality" means two or more unless otherwise specified. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
Aiming at the problems in the background art, the embodiment of the application provides a neural network watermark embedding method, and for the intellectual property of a neural network to be protected, a novel watermark protocol based on logistic chaotic mapping is provided.
As shown in fig. 1, the method comprises the steps of:
s01: and acquiring a key K, wherein the key K corresponds to a unique timestamp T.
In a particular embodiment, the key K is a string of at least 8-bit random numbers.
The time stamp is data generated by using a digital signature technology, and a signed object comprises original file information, signature parameters, signature time and other information. The time stamp system is used for generating and managing time stamps, and the time stamps are generated by digitally signing signature objects so as to prove that original files exist before the signature time.
Preferably, the time stamp T contains time information for applying a key to the third party copyright authenticating company.
S02: in the original placeRandomly selecting n pictures (P) in training set1、P2、…、Pn) And n is greater than 1, wherein the original training set comprises a plurality of pictures and a label corresponding to each picture.
In a specific embodiment, the selected n original pictures are saved by the copyright owner of the neural network, and are used for generating the trigger set picture when the watermark is embedded and the ownership of the neural network is verified subsequently.
S03: chaotic sequence pair picture (P) generated by secret key K1、P2、…、Pn) Scrambling and encrypting are carried out to obtain n pictures (Pe) of the trigger set1、Pe2、…、Pen) The n pictures (Pe)1、Pe2、…、Pen) And its corresponding label (L)1、L2、…、Ln) Put into the trigger set.
S04: and putting the original training set and the trigger set into a neural network to be embedded with the watermark for training, so that the neural network to be embedded with the watermark learns the mapping between the trigger set and the corresponding label while learning the original classification task, and obtaining the neural network embedded with the watermark.
According to the neural network watermark embedding method provided by the invention, the trigger set is generated on the premise of keeping the image characteristics unchanged, and the original label is given to the trigger set, so that the ownership of the model is protected on the premise of not influencing the decision boundary of the original classification task, and the intellectual property rights of owners are maintained while the performance precision of the model is almost the same as that of the original model.
Based on the neural network embedded with the watermark obtained in the way, as shown in fig. 2, in the watermark verification stage, a picture to be processed (a trigger set picture before generation) is submitted to a third party authority authentication center, the authority center processes the picture according to a corresponding secret key stored in a database, and the picture is authenticated under the condition that the authority center is not disclosed, so that an attacker is prevented from attacking more pertinently, and finally, when the hit rate of the network tag reaches a threshold value, the ownership verification is successful, otherwise, the verification fails.
In a preferred embodiment, when verifying ownership of the neural network, the method further comprises:
s05: acquiring the n pictures (P)1、P2、…、Pn) And the label (L) corresponding thereto1、L2、…、Ln) And the key K.
In a specific embodiment, when copyright disputes, the owner can extract n pictures (P) only known by himself from the original training set1、P2、…、Pn) And its original label (L)1、L2、…、Ln) And the authentication is handed to the third party authority center for authentication.
S06: chaotic sequence pair picture (P) generated by secret key K1、P2、…、Pn) Scrambling and encrypting are carried out to obtain a trigger set picture (Pe)1、Pe2、…、Pen)。
In a particular embodiment, the third party authority centre incorporates the picture (P) by means of a key K retained in a database1、P2、…、Pn) Get the trigger set (Pe)1、Pe2、…、Pen)。
S07: inputting the trigger set picture into a neural network to be verified to obtain a label (R) output by the neural network1、R2、…、Rn)。
S08: by comparing the labels (R) output by the neural network1、R2、…、Rn) And said original label (L)1、L2、…、Ln) And obtaining the verification result of the neural network.
In a preferred embodiment, the label (R) of the neural network output is calculated using the following formula1、R2、…、Rn) And said original label (L)1、L2、…、Ln) The consistency of (2):
V(t＝{f(Pk)，f(Pek)}nk＝1，τ)＝{True，False}
wherein V () is the agreement of the verification output tag with the genuine tagThe linear function, f () is the label, P, output after the network model processes the picturekAnd PekRespectively representing an original training set and a trigger set; k is the number of rounds of the current verification; t represents f (P)k) And f (Pe)k) Outputting the ratio of the number and the total number of the same results in the label; τ is a preset verification threshold.
In a specific embodiment, the threshold may be set to 0.9, which is nine times the random hit rate (0.1) in a normal ten-class task, which is sufficient to prove ownership of the network. Therefore, to prove its ownership of the network model, its accuracy needs to be greater than or equal to 90%.
And when t is calculated to be more than or equal to tau, the ownership of the verification result of the neural network is verified.
Aiming at the black box counterfeiting attack in the neural network copyright protection, due to the uniqueness of the watermark protocol mode, namely when an attacker obtains the neural network of the owner, the attacker wants to forge a watermark belonging to the attacker through the same watermark construction mode, when the attacker applies a secret key to forge the watermark to a third-party copyright authentication company, the application time is recorded, so that even if the forged watermark is successfully embedded, the authenticity of the watermark can be verified by comparing the application time, and the watermark embedded through the mode can resist the black box counterfeiting attack.
In a preferred embodiment, the chaotic sequence generated by the key K is used to align pictures (P) as shown in FIG. 31、P2、…、Pn) Scrambling and encrypting are carried out to obtain n pictures (Pe) of the trigger set1、Pe2、…、Pen) The method comprises the following substeps:
s031: for each picture PiA 1 is to PiDivided into multiple subblocks of size M B(1,1)，…，B(w/M,h/M)}; wherein w and h represent the width and height of the image, respectively; m is a preset numerical value.
S032: dividing the first 4 bits of the secret key K by 10000 to obtain K1And the last 4 bits are divided by 10000 to give k2Each as x0Inputting the following logistic chaotic mapping formula to generate pseudo-randomSequence Q1，Q2：
xn+1＝λ*xn*(1-xn)
Q1＝{x0,x1,…,xnIn which x0＝k1
Q2＝{x0,x1,…,xnIn which x0＝k2
Wherein, λ ∈ [0, 4 ]]，xn∈[0，1](ii) a And n is the iteration number.
Preferably, λ ═ 4.
In one illustrative example, the key K is 21936339, and the sequence Q is1Input x of0＝k10.2193, sequence Q2Input x of0＝k20.6399. After 3 iterations, the sequence Q is obtained1Sequence Q, {0.2193, 0.6848, 0.8634, 0.4718}, sequence Q2＝{0.6399，0.9217，0.2886，0.8213}。
S033: will random sequence Q1，Q2Sorting to generate an integer scrambling sequence O1，O2。
Preferably, the integer scrambling sequence O is obtained by using the following formula1，O2：
O1＝{r1,r2,…,rn+1}
Wherein r isiIs xiAt Q1Rank values in the sequence; r isiIs a positive integer;
O2＝{r1,r2,…,rn+1}
wherein r isiIs xiAt Q2Rank values in the sequence; r isiIs a positive integer.
In an illustrative example, for the sequence Q described above1Sequence Q, {0.2193, 0.6848, 0.8634, 0.4718}, sequence Q2O can be calculated as {0.6399, 0.9217, 0.2886, 0.8213}1＝{1，3，4，2}，O2＝{2，4，1，3}。
S034: order of utilizationColumn O1Scrambling each partition using the sequence O2Scrambling the pixel value in each block to obtain a trigger set picture Pei。
In a specific embodiment, the sequence O is utilized1The sub-step of scrambling each partition comprises:
s0341: for each picture PiFor all subblocks { B(1,1)，…，B(w/M,h/M)Sequencing to obtain the number {1, 2, …, n +1} of each subblock; wherein n is the sequence O1N in (1) are equal;
s0342: according to the sequence O1＝{r1,r2,…,rn+1The ordering of each sub-block is filled in the number of the sub-block in the sequence O1In the order of each sub-block.
And after the sub-blocks are rearranged, obtaining the picture with the scrambled sub-blocks.
In a specific embodiment, the sequence O is utilized2The step of scrambling the pixel values within each tile comprises:
s0343: sequencing all pixel points in each subblock to obtain the serial number {1, 2, …, n +1} of each pixel point; wherein n is the sequence O2N in (a) are equal.
S0344: according to the sequence O2＝{r1,r2,…,rn+1Sequencing, namely filling the serial number of each pixel point into the pixel point in a sequence O2And (5) rearranging the sequence of each pixel point.
And after the pixel value sequence is rearranged, obtaining a scrambled picture, namely a trigger set picture.
Most of the current black box watermarking methods add noise, signatures and other backdoor attacks to an original image, then give the original image an error label or give an abstract image a custom label to construct a trigger set, and when the trigger sets are placed in network training, decision boundaries of an original model are influenced. The scrambling mode for reserving the image features is used, the original label is given to the image after the image is scrambled to generate the watermark trigger set, and the classification task of the trigger set is well learned by the model on the premise of ensuring that the decision boundary of the original classification task is not distorted.
An embodiment of the present application further provides a neural network watermark embedding apparatus, as shown in fig. 4, the neural network watermark embedding apparatus 400 includes:
a key obtaining module 401, configured to obtain a key K, where the key K corresponds to a unique timestamp T;
an original picture selection module 402 for randomly selecting n pictures (P) in an original training set1、P2、…、Pn) Wherein n is greater than 1, wherein the original training set comprises a plurality of pictures and a label corresponding to each picture;
a trigger set obtaining module 403 for comparing the picture (P) with the chaotic sequence generated by the secret key K1、P2、…、Pn) Scrambling and encrypting are carried out to obtain n pictures (Pe) of the trigger set1、Pe2、…、Pen) The n pictures (Pe)1、Pe2、…、Pen) And its corresponding label (L)1、L2、…、Ln) Putting the trigger set;
a watermark embedding module 404, configured to put the original training set and the trigger set into a neural network to be embedded with a watermark for training, so that the neural network to be embedded with the watermark learns the mapping between the trigger set and the corresponding tag while learning the original classification task, thereby obtaining the neural network embedded with the watermark.
Preferably, when verifying ownership of the neural network, the apparatus further comprises:
a raw data acquisition module for acquiring said n pictures (P)1、P2、…、Pn) And the label (L) corresponding thereto1、L2、…、Ln) And the key K;
a trigger set generation module for generating a picture (P) by the chaos sequence generated by the key K1、P2、…、Pn) Scrambling and encrypting are carried out to obtain a trigger setPicture (Pe)1、Pe2、…、Pen)；
A label obtaining module for inputting the trigger set picture into the neural network to be verified to obtain a label (R) output by the neural network1、R2、…、Rn)；
A verification module for comparing the labels (R) output by the neural network1、R2、…、Rn) And said original label (L)1、L2、…、Ln) And obtaining the verification result of the neural network.
Preferably, the trigger set acquiring module includes:
an image dividing unit for dividing each picture PiA 1 is to PiDivided into multiple subblocks of size M B(1,1)，…，B(w/M,h/M)}; wherein w and h represent the width and height of the image, respectively; m is a preset numerical value;
a pseudo-random sequence generation unit for dividing the first 4 bits of the secret key K by 10000 to obtain K1And the last 4 bits are divided by 10000 to give k2Each as x0Inputting the following logistic chaotic mapping formula to generate a pseudo-random sequence Q1，Q2：
xn+1＝λ*xn*(1-xn)
Q1＝{x0,x1,…,xnIn which x0＝k1
Q2＝{x0,x1,…,xnIn which x0＝k2
Wherein, λ ∈ [0, 4 ]]，xn∈[0，1](ii) a n is the number of iterations;
a scrambling sequence generation unit for generating a random sequence Q1，Q2Sorting to generate an integer scrambling sequence O1，O2；
A scrambling unit for using the sequence O1Scrambling each partition using the sequence O2Scrambling the pixel value in each block to obtain a trigger set picture Pei。
Preferably, λ ═ 4.
Preferably, the scrambling sequence generation unit obtains the integer scrambling sequence O by using the following formula1，O2：
O1＝{r1,r2,…,rn+1}
Wherein r isiIs xiAt Q1Rank values in the sequence; r isiIs a positive integer;
O2＝{r1,r2,…,rn+1}
wherein r isiIs xiAt Q2Rank values in the sequence; r isiIs a positive integer.
Preferably, the scrambling unit includes:
a sub-block ordering element for each picture PiFor all subblocks { B(1,1)，…，B(w/M,h/M)Sequencing to obtain the number {1, 2, …, n +1} of each subblock; wherein n is the sequence O1N in (1) are equal;
sub-block scrambling elements for ordering according to the sequence O1＝{r1,r2,…,rn+1The ordering of each sub-block is filled in the number of the sub-block in the sequence O1Position in (2), rearranging the order of each sub-block;
the pixel value sequencing element is used for sequencing all pixel points of each subblock to obtain the serial number {1, 2, …, n +1} of each pixel point; wherein n is the sequence O2N in (1) are equal;
a pixel value scrambling element for scrambling according to a sequence O2＝{r1,r2,…,rn+1Sequencing, namely filling the serial number of each pixel point into the pixel point in a sequence O2And (5) rearranging the sequence of each pixel point.
Preferably, the verification module includes:
a consistency calculation unit for calculating a label (R) of the neural network output using the following formula1、R2、…、Rn) And said original label (L)1、L2、…、Ln) The consistency of (2):
V(t＝{f(Pk)，f(Pek)}nk＝1，τ)＝{True，False}
wherein V () is a consistency function for verifying the output label and the real label, f () is a label output after the network model processes the picture, PkAnd PekRespectively representing an original training set and a trigger set; k is the number of rounds of the current verification; t represents f (P)k) And f (Pe)k) Outputting the ratio of the number and the total number of the same results in the label; τ is a preset verification threshold;
and when t is calculated to be more than or equal to tau, the ownership of the verification result of the neural network is verified.
An embodiment of the present application further provides an electronic device, including:
at least one memory and at least one processor;
the memory for storing one or more programs;
when executed by the at least one processor, cause the at least one processor to implement the steps of a neural network watermark embedding method as described above.
Embodiments of the present application also provide a computer-readable storage medium,
the computer readable storage medium stores a computer program which, when executed by a processor, implements the steps of a neural network watermark embedding method as described above.
In a specific embodiment, the neural network watermark embedding method provided by the invention is characterized in that a third-party authority center is introduced, a scrambling key used by the third-party authority center is issued, the authority center only needs to store the key without storing a whole trigger set, so that the pressure of storing a large amount of data is greatly reduced, the verification process is independently carried out by the third-party authority center, the ownership of an owner is verified on the premise of not disclosing the watermark, and the more purposeful attack of an attacker is avoided; when a copyright owner applies for an encryption key, the third party authority center generates a time stamp to record the application time, so that the algorithm can resist counterfeiting attacks and is stronger in robustness.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.
Claims (10)
1. A neural network watermark embedding method is characterized by comprising the following steps:
obtaining a key K, wherein the key K corresponds to a unique timestamp T;
randomly select n pictures (p) in the original training set1、P2、…、Pn) Wherein n is greater than 1, wherein the original training set comprises a plurality of pictures and a label corresponding to each picture;
chaotic sequence pair picture (P) generated by secret key K1、p2、…、Pn) Scrambling and encrypting are carried out to obtain n pictures (Pe) of the trigger set1、Pe2、…、Pen) The n pictures (Pe)1、Pe2、…、Pen) And its corresponding label (L)1、L2、…、Ln) Putting the trigger set;
and putting the original training set and the trigger set into a neural network to be embedded with the watermark for training, so that the neural network to be embedded with the watermark learns the mapping between the trigger set and the corresponding label while learning the original classification task, and obtaining the neural network embedded with the watermark.
2. The neural network watermark embedding method according to claim 1, wherein when verifying ownership of the neural network, the method further comprises:
acquiring the n pictures (P)1、P2、…、Pn) And the label (L) corresponding thereto1、L2、…、Ln) And the key K;
chaotic sequence pair picture (P) generated by secret key K1、P2、…、Pn) Scrambling and encrypting are carried out to obtain a trigger set picture (Pe)1、Pe2、…、Pen)；
Inputting the trigger set picture into a neural network to be verified to obtain a label (R) output by the neural network1、R2、…、Rn)；
By comparing the labels (R) output by the neural network1、R2、…、Rn) And said original label (L)1、L2、…、Ln) And obtaining the verification result of the neural network.
3. The neural network watermark embedding method of claim 1, wherein the chaotic sequence generated by the secret key K is used for picture (P)1、P2、…、Pn) Scrambling and encrypting are carried out to obtain n pictures (Pe) of the trigger set1、Pe2、…、Pen) The method comprises the following steps:
for each picture PiA 1 is to PiDivided into multiple subblocks of size M B(1,1)，…，B(w/M,h/M)}; wherein w and h represent the width and height of the image, respectively; m is a preset numerical value;
dividing the first 4 bits of the secret key K by 10000 to obtain K1And the last 4 bits are divided by 10000 to give k2Each as x0Inputting the following logistic chaotic mapping formula to generate a pseudo-random sequence Q1，Q2：
xn+1＝λ*xn*(1-xn)
Q1＝{x0,x1,…,xnIn which x0＝k1
Q2＝{x0,x1,…,xnIn which x0＝k2
Wherein, λ ∈ [0, 4 ]]，xn∈[0，1](ii) a n is the number of iterations;
will random sequence Q1，Q2Sorting to generate an integer scrambling sequence O1，O2；
Using the sequence O1Scrambling each partition using the sequence O2Scrambling the pixel value in each block to obtain a trigger set picture Pei。
4. The neural network watermark embedding method of claim 3, wherein:
λ＝4。
5. the neural network watermark embedding method of claim 3, wherein the random sequence Q is1，Q2Sorting to generate an integer scrambling sequence O1，O2The method comprises the following steps:
obtaining an integer scrambling sequence O by using the following formula1，O2：
O1＝{r1,r2,…,rn+1}
Wherein r isiIs xiAt Q1Rank values in the sequence; r isiIs a positive integer;
O2＝{r1,r2,…,rn+1}
wherein r isiIs xiAt Q2Rank values in the sequence; r isiIs a positive integer.
6. The neural network watermark embedding method of claim 5, wherein the sequence O is used1Scrambling each partition using the sequence O2Scrambling the pixel value in each block to obtain a trigger set picture PeiThe method comprises the following steps:
for each graphSheet PiFor all subblocks { B(1,1)，…，B(w/M,h/M)Sequencing to obtain the number {1, 2, …, n +1} of each subblock; wherein n is the sequence O1N in (1) are equal;
according to the sequence O1＝{r1,r2,…,rn+1The ordering of each sub-block is filled in the number of the sub-block in the sequence O1Position in (2), rearranging the order of each sub-block;
sequencing all pixel points in each subblock to obtain the serial number {1, 2, …, n +1} of each pixel point; wherein n is the sequence O2N in (1) are equal;
according to the sequence O2＝{r1,r2,…,rn+1Sequencing, namely filling the serial number of each pixel point into the pixel point in a sequence O2And (5) rearranging the sequence of each pixel point.
7. A neural network watermark embedding method according to claim 2, wherein the comparison of the labels (R) output by the neural network is performed1、R2、…、Rn) And said original label (L)1、L2、…、Ln) Obtaining a verification result of the neural network, including:
calculating a label (R) of the neural network output using the following formula1、R2、…、Rn) And said original label (L)1、L2、…、Ln) The consistency of (2):
V(t＝{f(Pk)，f(Pek)}nk＝1，τ)＝{True，False}
wherein V () is a consistency function for verifying the output label and the real label, f () is a label output after the network model processes the picture, PkAnd PekRespectively representing an original training set and a trigger set; k is the number of rounds of the current verification; t represents f (P)k) And f (Pe)k) Outputting the ratio of the number and the total number of the same results in the label; τ is a preset verification threshold;
and when t is calculated to be more than or equal to tau, the ownership of the verification result of the neural network is verified.
8. A neural network watermark embedding apparatus, comprising:
the key acquisition module is used for acquiring a key K, and the key K corresponds to a unique timestamp T;
an original picture selection module for randomly selecting n pictures (P) in an original training set1、P2、…、Pn) Wherein n is greater than 1;
the key acquisition module is used for acquiring a key K, and the key K corresponds to a unique timestamp T;
an original picture selection module for randomly selecting n pictures (P) in an original training set1、P2、…、Pn) Wherein n is greater than 1, wherein the original training set comprises a plurality of pictures and a label corresponding to each picture;
a trigger set acquisition module for aligning the picture (P) by the chaos sequence generated by the secret key K1、P2、…、Pn) Scrambling and encrypting are carried out to obtain n pictures (Pe) of the trigger set1、Pe2、…、Pen) The n pictures (Pe)1、Pe2、…、Pen) And its corresponding label (L)1、L2、…、Ln) Putting the trigger set;
and the watermark embedding module is used for putting the original training set and the trigger set into a neural network to be embedded with the watermark for training, so that the neural network to be embedded with the watermark learns the mapping between the trigger set and the corresponding label while learning the original classification task, and the neural network embedded with the watermark is obtained.
9. An electronic device, comprising:
at least one memory and at least one processor;
the memory for storing one or more programs;
when executed by the at least one processor, cause the at least one processor to perform the steps of a neural network watermark embedding method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium characterized by:
the computer readable storage medium stores a computer program which when executed by a processor implements the steps of a neural network watermark embedding method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210016799.8A CN114359011A (en) | 2022-01-07 | 2022-01-07 | Neural network watermark embedding method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210016799.8A CN114359011A (en) | 2022-01-07 | 2022-01-07 | Neural network watermark embedding method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114359011A true CN114359011A (en) | 2022-04-15 |
Family
ID=81107466
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210016799.8A Pending CN114359011A (en) | 2022-01-07 | 2022-01-07 | Neural network watermark embedding method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114359011A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114862650A (en) * | 2022-06-30 | 2022-08-05 | 南京信息工程大学 | Neural network watermark embedding method and verification method |
| CN115984082A (en) * | 2023-03-21 | 2023-04-18 | 杭州虎符网络有限公司 | Dark watermark adding and extracting method, device, storage medium and terminal |
| CN116881871A (en) * | 2023-09-06 | 2023-10-13 | 腾讯科技（深圳）有限公司 | Model watermark embedding method, device, computer equipment and storage medium |
-
2022
- 2022-01-07 CN CN202210016799.8A patent/CN114359011A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114862650A (en) * | 2022-06-30 | 2022-08-05 | 南京信息工程大学 | Neural network watermark embedding method and verification method |
| CN114862650B (en) * | 2022-06-30 | 2022-09-23 | 南京信息工程大学 | Neural network watermark embedding method and verification method |
| CN115984082A (en) * | 2023-03-21 | 2023-04-18 | 杭州虎符网络有限公司 | Dark watermark adding and extracting method, device, storage medium and terminal |
| CN116881871A (en) * | 2023-09-06 | 2023-10-13 | 腾讯科技（深圳）有限公司 | Model watermark embedding method, device, computer equipment and storage medium |
| CN116881871B (en) * | 2023-09-06 | 2023-11-24 | 腾讯科技（深圳）有限公司 | Model watermark embedding method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114359011A (en) | Neural network watermark embedding method and device, electronic equipment and storage medium | |
| CN108304858B (en) | Generation method, verification method and system of confrontation sample recognition model | |
| Katiyar et al. | Online voting system powered by biometric security using steganography | |
| CN110868300B (en) | Block chain evidence-storing method and system | |
| CN110390623B (en) | Secret sharing-based image media security authentication recovery method | |
| CN107846530A (en) | Digital watermarking algorithm | |
| CN113065149A (en) | Data copyright protection method based on block chain and collusion attack resistant fingerprint code | |
| MaungMaung et al. | A protection method of trained CNN model with a secret key from unauthorized access | |
| CN112132733B (en) | Chaos-based automatic labeling algorithm for black box watermark trigger set of intelligent model | |
| Chang et al. | A block-based watermarking scheme for image tamper detection and self-recovery | |
| CN1517855A (en) | Image digital watermark method | |
| CN101034985A (en) | Method and system for the anti-counterfeit of the mobile phone with the dynamic code | |
| Zhu et al. | Fragile neural network watermarking with trigger image set | |
| CN114998922B (en) | Electronic contract generating method based on format template | |
| CN114998080B (en) | Face tamper-proof watermark generation method, tamper detection method and attribute detection method | |
| CN112613001A (en) | Method for realizing integrity authentication of convolutional neural network through reversible watermark | |
| Li et al. | Leveraging Multi-task Learning for Umambiguous and Flexible Deep Neural Network Watermarking. | |
| CN117395474B (en) | Locally stored tamper-resistant video evidence obtaining and storing method and system | |
| Sun et al. | Protecting the intellectual properties of deep neural networks with an additional class and steganographic images | |
| Das et al. | Cryptanalysis of optimal differential energy watermarking (DEW) and a modified robust scheme | |
| CN116527278A (en) | Block chain hidden communication method based on generation type hidden network and image double hidden | |
| Xu et al. | A novel method for identifying the deep neural network model with the serial number | |
| CN115689852A (en) | Digital watermarking method and related device | |
| WO2023041212A1 (en) | Method of verification for machine learning models | |
| Chakraborty et al. | Dynamarks: Defending against deep learning model extraction using dynamic watermarking |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |