CN114493972A - Confrontation type network copyright generation protection method - Google Patents
Confrontation type network copyright generation protection method Download PDFInfo
- Publication number
- CN114493972A CN114493972A CN202210110918.6A CN202210110918A CN114493972A CN 114493972 A CN114493972 A CN 114493972A CN 202210110918 A CN202210110918 A CN 202210110918A CN 114493972 A CN114493972 A CN 114493972A
- Authority
- CN
- China
- Prior art keywords
- watermark
- label
- model
- image
- generator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000012549 training Methods 0.000 claims abstract description 43
- 230000003042 antagnostic effect Effects 0.000 claims abstract description 9
- 238000012795 verification Methods 0.000 claims abstract description 7
- 238000013441 quality evaluation Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000004806 packaging method and process Methods 0.000 claims description 2
- 238000011160 research Methods 0.000 abstract description 6
- 238000003062 neural network model Methods 0.000 description 19
- 238000013473 artificial intelligence Methods 0.000 description 9
- 238000013528 artificial neural network Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 235000000332 black box Nutrition 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 244000062793 Sorghum vulgare Species 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 235000019713 millet Nutrition 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000009966 trimming Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0021—Image watermarking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/10—Protecting distributed programs or content, e.g. vending or licensing of copyrighted material ; Digital rights management [DRM]
- G06F21/106—Enforcing content protection by specific content processing
- G06F21/1063—Personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/10—Protecting distributed programs or content, e.g. vending or licensing of copyrighted material ; Digital rights management [DRM]
- G06F21/16—Program or content traceability, e.g. by watermarking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2201/00—General purpose image data processing
- G06T2201/005—Image watermarking
- G06T2201/0065—Extraction of an embedded watermark; Reliable detection
Abstract
The invention discloses a method for protecting a confrontation type generation network copyright. The method comprises the following steps: step 1, collecting N pictures containing specified objects as watermark images; step 2, obtaining a watermark label with a specific length by a specified key character string through a key generating function, and taking the watermark label as the watermark label of the watermark image collected in the step 1; combining the watermark image and the watermark label as a trigger set; step 3, putting the trigger set obtained in the step 2 and the training task image into an antagonistic generation network for training to obtain a network model with a watermark; and 4, the network model owner holds the watermark label, when the copyright of the network model is disputed, the model owner inputs the specific watermark label into the generator, and the generator returns the corresponding watermark image to be compared with the provided training watermark image to finish the copyright verification. The invention can protect the copyright of the model under the condition of not changing the structure of the network and not influencing the performance of the model, and enriches the research on the watermark of the black box model.
Description
Technical Field
The invention belongs to the technical field of artificial intelligence safety, and particularly relates to a confrontation type generation network copyright protection method.
Background
Recently, neural networks have enjoyed breakthrough success in the field of artificial intelligence and have received attention from many researchers. With the popularization of artificial intelligence, some scientific and technological companies, such as google, millet and the like, have introduced artificial intelligence online services by means of advanced neural network models, provide intelligent products and high-quality services for consumers, and the excellent intelligent services bring great convenience to the lives of people. Along with the development of artificial intelligence in various fields of social economy, how to ensure the health of the artificial intelligence and safely popularize and use becomes a problem of great concern. Notably, as a key component in a product or service, a commercial neural network model is not easily trained and built. As such models require a significant training data set and expensive computational resources to invest. Meanwhile, the neural network model with commercial value is easy to be stolen by attackers, and the intellectual property of the owner of the model is easy to be infringed. Therefore, how to protect intellectual property of the neural network model has become an urgent problem to be solved.
Digital watermarking is used for embedding digital signals in data needing protection at the earliest, and meanwhile usability of original data is not affected. With the continuous popularization of the neural network model, the watermark concept is expanded to the field of artificial intelligence models, and watermarks are embedded into the model to be protected and used for protecting the neural network model. When the owner of the model finds that the model is suspected to be stolen, copyright disputes can be dealt with by activating the watermark hidden in the model, and the rights and interests of the owner are guaranteed. Embedded efficient model watermarks need to meet several basic requirements, 1) fidelity: the diversity and sharpness of the model-generated image are not reduced by embedding the watermark, 2) robustness: model watermarks do not get deleted when subjected to watermark attacks, such as trimming of trim and parameters, 3) security: attacker will not detect the watermark, 4) integrity: the extracted watermark has a small false alarm rate. Model watermarks meeting the above requirements can only be put into use.
At present, in the aspect of neural network watermarking, a plurality of problems need to be solved, such as: 1) at present, only a few watermarking algorithms are aimed at a generative neural network model, and the robustness of the watermarking algorithms needs to be improved; 2) some neural network watermarking algorithms are easy to detect by attackers and need to be improved in the aspect of concealment; 3) in the model watermark algorithm, it is desirable that not only the copyright of the model can be proved, but also the watermark can carry a certain amount of information, so as to improve the capacity of the model watermark algorithm. Therefore, the research on the neural network model watermarking algorithm needs to be further and deeply researched with pertinence, so that the further development of the artificial intelligence safety research can be realized, the health and safety of the neural network can be promoted to be put into practical use, the illegal criminal behaviors which are profitable by using a stealing model can be struck, and the method has deeper practical significance and profound influence. ,
in the current research, most of the model watermarking algorithm research focuses on a neural network model with classification and prediction functions, and the model watermarking algorithm is difficult to be applied to a resist generation type neural network model in different input and output sample forms. However, the commercial value of the antagonistic generative model also needs to be protected.
Current neural network watermarking techniques are mainly classified into white-box watermarking and black-box watermarking. The white-box watermark is embedded into the watermark through model parameters, and a parameter regularizer is used for embedding the watermark into the model parameters during the training of the neural network. Although many white-box watermarking methods can effectively embed watermarks and resist watermark attacks, the model owner is required to access the structure and weights of the target model in the verification process, which greatly limits the application of the method in practice. In order to solve the problem, a black box watermarking method is provided, a back door technology is used for ownership protection of the model, an abstract sample is used as a back door set to be put into a model training set through a model training or fine tuning means, the original performance of the model is guaranteed, and ownership authentication can be carried out. The black box watermark can embed the watermark into the backdoor set of the protected network and detect the watermark through the output of the predictive model, and the model watermark can be verified by remotely accessing the model. However, most of the proposed black-box watermarking algorithms are directed to classifying neural network models, and cannot be applied to generating neural network models in an antagonistic manner due to different sample forms of input and output.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a conditional countermeasure type generation network copyright protection method, which is characterized in that a training task generated by a watermark image is additionally added in a model according to the characteristic that neural network parameters have redundancy, thereby achieving the purpose of protecting the property right of the model and avoiding an attacker stealing the model to earn inappropriate benefits.
The countermeasure generation network copyright protection method based on the condition specifically comprises the following steps:
the specified object is a picture of the same object in different scenes; for example, if the designated object is a person, pictures of the same person in different scenes are needed.
Step 2, obtaining a watermark label with a specific length by a specified key character string through a key generating function, and taking the watermark label as the watermark label of the watermark image collected in the step 1; the watermark image and the watermark label are combined as a trigger set.
The key generation function includes brotli compression algorithm compression.
And 3, putting the trigger set obtained in the step 2 and the training task image into an antagonistic generation network for training to obtain a network model with a watermark, wherein the specific steps are as follows:
step 3.1, packaging the trigger set and the training task images into tfrecrds files to be used as training sets;
the training task image is a confrontation type generation network original data set and is used for training the confrontation type generation network.
Step 3.2, initializing parameters of a generator and a discriminator network;
3.3, the generator obtains a generated image according to the random vector z and the label, and the generated image, the training task image and the label are used as the input of the discriminator; the labels comprise watermark labels of a trigger set and original labels carried by training task images.
And the discriminator returns the quality evaluation result of the generated image according to the calculation as the feedback of the generator parameter adjustment. The generator and the arbiter train the model according to the countermeasure network equation (1).
Where x is true data belonging to PrDistribution, PrRepresenting images from a training set, z being that the noisy data belongs to PgDistribution, PgIs a random vector with standard normal distribution, E (eta.) represents mathematical expectation, G (z) is a generated image obtained by a generator, and D (x) represents the quality evaluation result predicted by the discriminator. Alpha denotes an influencing parameter of the watermark, LwmIndicating a loss of the watermark.
And 3.4, repeating the step 3.3, and obtaining the generator and the discriminator of the trained watermarked image after n rounds. The generator is deployed in the cloud and provides services through the API.
And 4, the model owner holds the watermark label. When the network model copyright is disputed, the model owner inputs a specific watermark label into the generator, and the generator returns a corresponding watermark image to be compared with the provided training watermark image to finish the copyright verification.
1. The invention has the following beneficial effects:
a watermark for protecting a countermeasure type generation model is provided, which can protect the copyright of the model under the condition of not changing the structure of the network and not influencing the performance of the model, and enriches the research of the black box model watermark.
The watermark is embedded with a certain amount of watermark information in a label embedding mode, so that the ownership of the model can be remotely verified and the source of a divulger can be tracked.
The watermark embedding method is based on the neural network model characteristics of model parameter redundancy and training loss to find local minimum values, trains additional watermarks in the model, can be applied to various antagonistic generation models, and particularly has good performance in ProGAN and StyleGAN2 and strong algorithm adaptability.
Drawings
Fig. 1 is a flowchart of a model watermark embedding and verifying method according to the present invention.
Detailed Description
The invention is further illustrated by the following figures and examples.
The invention discloses a method for protecting a confrontation type generation network copyright. According to the method, the black box model watermark is embedded into the antagonistic generation network, and because the loss function adjustment is to find a local minimum of loss in the training process of the neural network model, the model can find a new local minimum of loss after the watermark is embedded, and the performance of the model can not be influenced. Based on the characteristic, the preset trigger set watermark and the model task training data are put into the neural network model together for training, and the verification watermark is embedded while the performance of the model is not influenced. Wherein, the discriminator can learn the ability of discriminating the watermark and feed back to the generator, and the generator can generate the watermark image according to the specific label. The method can embed the watermark in the neural network model so as to achieve the purpose of protecting the copyright. Compared with the prior art, the method can protect the copyright of a new type of neural network model, namely the generative confrontation network model, and has important effects on verifying whether the neural network model is stolen and protecting the commercial value of the neural network model technology.
As shown in fig. 1, a method for countervailing network copyright protection specifically includes the following steps:
In the invention, 100 slim character images are collected, data enhancement is carried out by methods of rotation, left-right turning, noise adding, copying and the like, and 3000 color watermark images with the size of 32 multiplied by 32 are finally obtained.
And 2, setting a short character string text as a key character string and representing the key character string in a binary form. And obtaining the watermark label with a specific length by the appointed key character string through a key generation function. All watermark images are provided with uniform watermark labels, and the watermark images and the watermark labels form a trigger set.
Further, the key generation function includes MD 5.
Further, the specific key string may also be compressed into a watermark label with a specific length containing only {0,1} as the watermark label of the watermark image collected in step 1, by using a brotli compression algorithm derived from google and combining the variant LZ77 algorithm, Huffman coding and second-order text modeling.
And 3, putting the trigger set obtained in the step 2 and the training task image into an antagonistic generation network for training to obtain a network model with a watermark. The invention uses ProGAN model to carry out experiment, which comprises the following steps:
and 3.1, in the training process, the model not only needs to learn the original training task, but also needs to learn how to generate the watermark image. Due to the large amount of data, a more efficient way of handling data I/O is important. In the experiment, a tensierflow frame is used for training a model, and tfrecrds files of binary streams are used for storing files, so that the rapid reading and storage of data are facilitated. The trigger set and training task images are packaged into tfrecrds files for use as the training set.
Step 3.2, initializing a generator parameter theta and a discriminator parameter omega;
and 3.3, inputting the sample data with standard normal distribution into a generator to generate an image, wherein the random vector z is sample data with standard normal distribution. The network structure of the generator is progressive, and the generated image is obtained according to the random vector z and the real label. The task of the discriminator is to discriminate the quality of the generated image.
And in the training process, the generated image, the training task image and the label are used as input of a discriminator, and the discriminator returns quality evaluation of the generated image according to calculation and is used as feedback of generator parameter adjustment. The generator and the arbiter adjust the parameters according to the countermeasure network formula (1)
Where x is true data belonging to PrDistribution, PrRepresenting images from a training set, z beingNoisy data belonging to PgDistribution, PgIs a random vector with standard normal distribution, E (eta.) represents mathematical expectation, G (z) is a generated image obtained by a generator, and D (x) represents the quality evaluation result predicted by the discriminator. Alpha denotes an influencing parameter of the watermark, LwmIndicating a loss of the watermark.
The parameters of the generator and the arbiter are constantly adjusted during each round. The quality of the generated image is expected to be good enough by the generator, so that the output image has good quality and can also accord with the corresponding label, and in the parameter adjustment process of the generator, the discriminator needs to have good evaluation on the generated image, so that the distance difference between the discriminator and the original image is minimized. The model loss calculation here uses the Wasserstein distance calculation method proposed in WGAN plus the ACGAN loss including the category judgment. The discriminator requires the ability to discriminate and classify the generated counterfeit images, and the discriminator is equivalent to standing on the opposite side of the generator during the training process, and the task of the discriminator is to minimize the loss of the part of the discrimination real data and maximize the loss of the discrimination generated pictures.
And 3.4, setting an epoch, repeating the step 3.3, wherein the epoch set by the user in the example is 10000, and adjusting the learning rate by using an Adam optimizer to accelerate the convergence rate. And obtaining a generator with a watermark and a discriminator after the training is finished. And judging the performance of the generated model by using a Frechet inclusion Distance evaluation index, wherein the index can represent the Distance of an inclusion characteristic vector between a real image and a generated image in the same domain, the quality and diversity of the generated image can be evaluated, and the lower the index is, the better the performance is.
When the artificial intelligence service is provided by the actual application, the generation model can be deployed at the cloud end to provide the service through the API. The model owner holds the watermark label, and the model user can obtain corresponding service by calling the API.
And 4, the model owner holds the watermark label. When the network model copyright is disputed, the model owner inputs a specific watermark label into the generator, and the generator returns a corresponding watermark image to be compared with the provided training watermark image to finish the copyright verification. In practice, the similarity between the output watermark image and the verification image provided in advance is evaluated according to the image quality evaluation index. The similarity and the quality of the image watermark are objectively evaluated by combining the structural similarity index, the peak signal-to-noise ratio and the cosine similarity.
Claims (3)
1. A method for protecting network copyright in an antagonistic mode is characterized by comprising the following steps:
step 1, collecting N pictures containing specified objects as watermark images;
step 2, obtaining a watermark label with a specific length by a specified key character string through a key generating function, and taking the watermark label as the watermark label of the watermark image collected in the step 1; combining the watermark image and the watermark label as a trigger set;
step 3, putting the trigger set obtained in the step 2 and the training task image into an antagonistic generation network for training to obtain a network model with a watermark;
and 4, the network model owner holds the watermark label, when the copyright of the network model is disputed, the model owner inputs the specific watermark label into the generator, and the generator returns the corresponding watermark image to be compared with the provided training watermark image to finish the copyright verification.
2. The method for opportunistically generating copyright protection on a network according to claim 1, wherein step 3 is implemented as follows:
step 3.1, packaging the trigger set and the training task images into tfrecrds files to be used as training sets;
step 3.2, initializing parameters of a generator and a discriminator network;
3.3, the generator obtains a generated image according to the random vector z and the label, and the generated image, the training task image and the label are used as the input of the discriminator; the discriminator returns the quality evaluation result of the generated image according to the calculation as the feedback of the generator parameter adjustment; the generator and the discriminator train the model according to the countermeasure network formula (1);
where x is true data belonging to PrDistribution, PrRepresenting images from a training set, z being that the noisy data belongs to PgDistribution, PgThe random vector is in standard normal distribution, E (.) represents mathematical expectation, G (z) is a generated image obtained by a generator, and D (x) represents a quality evaluation result predicted by a discriminator; alpha denotes an influencing parameter of the watermark, LwmRepresents a loss of the watermark;
step 3.4, repeating the step 3.3, and obtaining a generator and a discriminator of the trained watermarked image after n rounds; the generator is deployed in the cloud and provides services through the API.
3. A method for protecting copyright of countermeasure generation network as claimed in claim 2, wherein said label of step 3.3 includes watermark label of trigger set and original label of training task image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210110918.6A CN114493972A (en) | 2022-01-29 | 2022-01-29 | Confrontation type network copyright generation protection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210110918.6A CN114493972A (en) | 2022-01-29 | 2022-01-29 | Confrontation type network copyright generation protection method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114493972A true CN114493972A (en) | 2022-05-13 |
Family
ID=81478617
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210110918.6A Pending CN114493972A (en) | 2022-01-29 | 2022-01-29 | Confrontation type network copyright generation protection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114493972A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115879072A (en) * | 2023-03-03 | 2023-03-31 | 南京信息工程大学 | Copyright protection method, device and medium for deep fake fingerprint detection model |
-
2022
- 2022-01-29 CN CN202210110918.6A patent/CN114493972A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115879072A (en) * | 2023-03-03 | 2023-03-31 | 南京信息工程大学 | Copyright protection method, device and medium for deep fake fingerprint detection model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107240061B (en) | Watermark embedding and extracting method and device based on dynamic BP neural network | |
| Meng et al. | A survey of image information hiding algorithms based on deep learning | |
| Zhou et al. | On security enhancement of steganography via generative adversarial image | |
| Yuan et al. | Multiscale fragile watermarking based on the Gaussian mixture model | |
| Loukhaoukha | Image watermarking algorithm based on multiobjective ant colony optimization and singular value decomposition in wavelet domain | |
| Tang et al. | Detection of GAN-synthesized image based on discrete wavelet transform | |
| CN114493972A (en) | Confrontation type network copyright generation protection method | |
| CN114881838B (en) | Bidirectional face data protection method, system and equipment for deep forgery | |
| Wang et al. | Alteration-free and model-agnostic origin attribution of generated images | |
| Chen et al. | A novel blind watermarking scheme based on neural networks for image | |
| Ito et al. | Access control of semantic segmentation models using encrypted feature maps | |
| Hadmi et al. | A novel approach for robust perceptual image hashing | |
| Dhivyaa et al. | Video Matting, Watermarking and Forensics | |
| Chen et al. | When deep learning meets watermarking: A survey of application, attacks and defenses | |
| Nie et al. | Deep Model Intellectual Property Protection with Compression-Resistant Model Watermarking | |
| Trung et al. | Secure eeg-based user authentication system integrated with robust watermarking | |
| Liu et al. | BiFPro: A Bidirectional Facial-data Protection Framework against DeepFake | |
| Pal et al. | Secured Digital Watermarking Using Neural Networks | |
| Wu et al. | A survey on neural network-based image data hiding for secure communication | |
| Kallas et al. | Mixer: Dnn watermarking using image mixup | |
| Mahmood et al. | Copyright protection and content integrity for digital video based on the watermarking techniques | |
| El-Den et al. | ‘Watermarking models and artificial intelligence | |
| Saini et al. | Watermarked Hashing As a Video Content Authentication Technique | |
| Haldar et al. | Secured Information Communication Exploiting Fuzzy Weight Strategy | |
| Luo et al. | Halftone image steganalysis by reconstructing grayscale image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |