CN113987429A - Copyright verification method of neural network model based on watermark embedding - Google Patents
Copyright verification method of neural network model based on watermark embedding Download PDFInfo
- Publication number
- CN113987429A CN113987429A CN202111293312.2A CN202111293312A CN113987429A CN 113987429 A CN113987429 A CN 113987429A CN 202111293312 A CN202111293312 A CN 202111293312A CN 113987429 A CN113987429 A CN 113987429A
- Authority
- CN
- China
- Prior art keywords
- neural network
- watermark
- network model
- trigger set
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003062 neural network model Methods 0.000 title claims abstract description 121
- 238000012795 verification Methods 0.000 title claims abstract description 91
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 21
- 238000012545 processing Methods 0.000 claims abstract description 17
- 238000012549 training Methods 0.000 claims abstract description 14
- 238000004422 calculation algorithm Methods 0.000 claims description 21
- 230000006870 function Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 8
- 230000004044 response Effects 0.000 claims description 2
- 239000011159 matrix material Substances 0.000 description 10
- 238000013528 artificial neural network Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 230000000694 effects Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 241000282326 Felis catus Species 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 238000013475 authorization Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/10—Protecting distributed programs or content, e.g. vending or licensing of copyrighted material ; Digital rights management [DRM]
- G06F21/16—Program or content traceability, e.g. by watermarking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0021—Image watermarking
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Technology Law (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Editing Of Facsimile Originals (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the field of digital watermark embedding, in particular to a copyright verification method, a device, equipment and a storage medium of a neural network model based on watermark embedding, which comprises the following steps: acquiring a watermark image data set; extracting a plurality of watermark images in the watermark image data set as an original trigger set, and performing singular value decomposition processing on the original trigger set to obtain a trigger set sample; extracting classification label data corresponding to watermark images in an original trigger set, and enabling the classification label data to correspond to trigger set samples one by one to obtain a trigger set; inputting the watermark image data set and the trigger set into a neural network model to be embedded with the watermark, and training the neural network model for a plurality of times to obtain a target neural network model; and responding to a model copyright verification instruction of a user, inputting the trigger set into the target neural network model, acquiring a classification result, and acquiring a verification result output by the target neural network model according to the watermark image in the trigger set, the corresponding classification label data and the classification result.
Description
Technical Field
The invention relates to the field of digital watermark embedding, in particular to a copyright verification method, a copyright verification device, copyright verification equipment and a storage medium based on a neural network model with watermark embedding.
Background
With the rapid development of the deep learning field, the application of the neural network model among enterprises is more and more extensive, but since an attacker gains benefits to others through illegal authorization by tampering with the neural network model of others, the copyright of the model creator is violated, and huge economic loss is brought to the model creator, so that the protection of the copyright of the neural network model is necessary.
With the wide application of the digital watermarking technology, the neural network model can be protected by embedding the watermark in the neural network model, and the current neural network watermark embedding mode can influence the functions of the neural network model, so that the problems of reduced model task classification precision, insufficient watermark universality or reduced robustness are caused.
Disclosure of Invention
Based on this, the present invention aims to provide a copyright verification method, device, equipment and storage medium for a neural network model based on watermark embedding, wherein a trigger set is constructed by performing singular value decomposition on a watermark image data set, and a watermark embedding frame is constructed according to the watermark image data set and the trigger set, so that the watermark embedding efficiency is improved, meanwhile, the influence of the watermark on the function of the neural network model is reduced, the embedding effect is more stable, the watermark is not easy to erase or forge, and the universality and robustness of the watermark are improved.
In a first aspect, an embodiment of the present application provides a copyright verification method based on a neural network model with embedded watermarks, including the following steps:
acquiring a watermark image data set, wherein the watermark image data set comprises a plurality of watermark images, and each watermark image has corresponding classification label data;
extracting a plurality of watermark images in the watermark image data set as an original trigger set, and performing singular value decomposition processing on the original trigger set to obtain a trigger set sample;
extracting classification label data corresponding to the watermark images in the original trigger set, and enabling the classification label data to correspond to the trigger set samples one by one to obtain a trigger set;
inputting the watermark image data set and the trigger set into a neural network model to be embedded with a watermark, and training the neural network model for a plurality of times to obtain a target neural network model;
responding to a model copyright verification instruction of a user, wherein the model copyright verification instruction comprises the trigger set, inputting the trigger set into the target neural network model, obtaining a classification result, and obtaining a verification result output by the target neural network model according to the watermark image in the trigger set, the corresponding classification label data and the classification result.
In a second aspect, an embodiment of the present application provides an apparatus for a copyright verification method based on a neural network model with watermark embedding, including:
the system comprises an acquisition module, a classification module and a classification module, wherein the acquisition module is used for acquiring a watermark image data set, the watermark image data set comprises a plurality of watermark images, and each watermark image has corresponding classification label data;
the first extraction module is used for extracting a plurality of watermark images in the watermark image data set as an original trigger set, and performing singular value decomposition processing on the original trigger set to obtain a trigger set sample;
the second extraction module is used for extracting classification label data corresponding to the watermark images in the original trigger set, and enabling the classification label data to correspond to the trigger set samples one by one to obtain the trigger set;
the training-embedding module is used for inputting the watermark image data set and the trigger set into a neural network model to be embedded with the watermark, and training the neural network model for a plurality of times to obtain a target neural network model;
and the verification module is used for responding to a model copyright verification instruction of a user, wherein the model copyright verification instruction comprises the trigger set, inputting the trigger set into the target neural network model, acquiring a classification result, and acquiring the verification result output by the target neural network model according to the watermark image in the trigger set, the corresponding classification label data and the classification result.
In a third aspect, an embodiment of the present application provides an apparatus, including: a processor, a memory, and a computer program stored on the memory and executable on the processor; the computer program when executed by the processor implements the steps of the method for copyright verification based on a neural network model for watermark embedding according to the first aspect.
In a fourth aspect, the present application provides a storage medium storing a computer program, which when executed by a processor implements the steps of the copyright verification method based on the neural network model with embedded watermarks according to the first aspect.
In the embodiment of the application, the singular value decomposition is carried out on the watermark image data set to construct the trigger set, and the watermark embedding efficiency is improved according to the watermark embedding frame formed by the watermark image data set and the trigger set, meanwhile, the influence of the watermark on the function of the neural network model is reduced, the embedding effect is more stable, the watermark is not easy to erase or forge, and the universality and the robustness of the watermark are improved.
For a better understanding and practice, the invention is described in detail below with reference to the accompanying drawings.
Drawings
Fig. 1 is a schematic flowchart of a copyright verification method based on a neural network model with watermark embedding according to an embodiment of the present application;
fig. 2 is a schematic flowchart of acquiring a trigger set according to an embodiment of the present application;
FIG. 3 is a schematic flow chart illustrating a process for obtaining a target neural network model according to an embodiment of the present application;
fig. 4 is a schematic flowchart of S4 in the copyright verification method based on the neural network model with watermark embedding according to an embodiment of the present application;
FIG. 5 is a schematic flow chart illustrating model copyright validation according to an embodiment of the present application;
fig. 6 is a schematic flowchart of S5 in the method for copyright verification based on a neural network model with watermark embedding according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a copyright verification apparatus based on a neural network model for embedding watermarks according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an apparatus according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if/if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
Referring to fig. 1, fig. 1 is a schematic flowchart of a copyright verification method based on a neural network model with watermark embedding according to an embodiment of the present application, where the method includes the following steps:
s1: and acquiring a watermark image data set, wherein the watermark image data set comprises a plurality of watermark images, and each watermark image has corresponding classification label data.
The main execution body of the copyright verification method based on the watermark-embedded neural network model is copyright verification equipment (hereinafter referred to as verification equipment for short) of the neural network model, and in an optional embodiment, the verification equipment can be one computer equipment, a server or a server cluster formed by combining a plurality of computer equipment.
The watermark image data set comprises a plurality of watermark images, each watermark image comprises a plurality of watermark training images and a watermark verification image, each watermark image is an image with main body outline information, each watermark image is provided with classification label data corresponding to the main body outline information, but the labels of the watermark images cannot be accurately classified by human eyes and a pre-trained neural network model. For example, the main outline information of the watermark image is a cat, and the classification label data thereof is "cat".
In this embodiment, the verification device obtains a watermark image data set input by a user, and is used to embed a watermark in the neural network model.
Referring to fig. 2, fig. 2 is a schematic flowchart illustrating a process of acquiring a trigger set according to an embodiment of the present application.
S2: and extracting a plurality of watermark images in the watermark image data set as an original trigger set, and performing singular value decomposition processing on the original trigger set to obtain a trigger set sample.
Singular values are concepts in a matrix and are generally found by the singular value decomposition theorem. Let X be an m × n order matrix, q ═ min (m, n), and the arithmetic square root of the q non-negative eigenvalues of a × a is called the singular value of a. Singular value decomposition is an important matrix decomposition method in linear algebra and matrix theory, and is suitable for the fields of signal processing, statistics and the like.
Generally, singular values correspond to main information in a matrix, so that the singular value decomposition is used for data analysis to extract the main information of the matrix.
The singular value decomposition algorithm is as follows:
A＝U×S×VT
in the formula, A is singular value decomposition sample data of the watermark image; s is a matrix with a diagonal as a singular value element and elements at other positions of the matrix are all 0; u is about ATA is a matrix constructed by the characteristic vectors; v is with respect to AATIs constructed from the feature vectors of (a).
In this embodiment, the verification device extracts a plurality of watermark images from the watermark image data set to serve as an original trigger set, performs singular value decomposition on the watermark images in the original trigger set according to a preset singular value decomposition algorithm to obtain singular value decomposition sample data, and constructs a trigger set sample a 'according to the singular value decomposition sample data, where a' is U × S1×VT+U×S2×VT+……+U×Sn×VT，{Si|i∈n}。
Wherein, SiAnd | i belongs to n } is a matrix with singular value numerical values on the diagonal line in sequence and other elements of 0, and the maximum value of n is the pixel number of the width of the sample A in the watermark image.
In an optional embodiment, the verification device selects the first 2 trigger set samples obtained after the singular value decomposition algorithm processing, which are optimal, so that the main body contour information of the image is retained, and the pre-trained and clean neural network model cannot identify the correct classification label. The number of watermark images of the trigger set sample is generally controlled to be within 200, since it is ensured that the training of the trigger set does not interfere with the original classification task.
S3: and extracting classification label data corresponding to the watermark images in the original trigger set, and enabling the classification label data to correspond to the trigger set samples one by one to obtain the trigger set.
In this embodiment, the verification apparatus extracts the classification tag data corresponding to the watermark image in the original trigger set, and associates the classification tag data with the trigger set samples one by one to obtain the trigger set.
Referring to fig. 3, fig. 3 is a schematic flowchart illustrating a process for obtaining a target neural network model according to an embodiment of the present application.
S4: and inputting the watermark image data set and the trigger set into a neural network model to be embedded with the watermark, and training the neural network model for a plurality of times to obtain a target neural network model.
The neural network model to be embedded with the watermark is a convolutional neural network structure, and the convolutional neural network comprises a ResNet series neural network structure, a VGG series neural network structure, a LeNet series neural network structure, an AlexNet series neural network structure and the like.
In an optional embodiment, when the neural network model to be embedded with the watermark is a ResNet series or VGG series neural network structure, the verification device usually adopts a fine tuning training model mode, that is, the neural network model to be embedded with the watermark is trained by modifying a small amount of weight parameters of the ResNet series or VGG series neural network structure, so as to ensure that the embedding effect of the watermark does not easily influence the structure and function of the neural network model.
In another optional embodiment, when the neural network model to be embedded with the watermark is a LeNet series or AlexNet series neural network structure, the verification device usually trains the model from scratch, that is, by modifying all weight parameters of the LeNet series or AlexNet series neural network structure, the neural network model to be embedded with the watermark is trained, so as to ensure that the embedding effect of the watermark is more stable.
In an optional embodiment, the verification device mixes the watermark image data set and the trigger set in a ratio of 16:1, trains the neural network model, and obtains the trained neural network model.
Referring to fig. 4, fig. 4 is a schematic flowchart of S4 in the method for verifying copyright based on the neural network model with watermark embedding according to an embodiment of the present application, including steps S401 to S403:
s401: first loss data associated with the watermark image data set is obtained according to the watermark image data set and a first loss calculation algorithm.
The first loss calculation algorithm is typically a Cross Entropy loss Function (Cross Entropy Error Function), as follows:
in the formula, Loss1The first loss data is obtained; i is the number of classification task categories in the watermark image data set; y isiAn indicator variable representing a category i; p is a radical ofiRepresenting the predicted probability that the predicted label belongs to category i.
In this embodiment, the verification device inputs the watermark image data set into the neural network model, obtains a prediction result of the neural network model for the watermark image data set, obtains a first prediction probability according to the prediction result and classification label data corresponding to a watermark image in the watermark image data set, and obtains first loss data related to the watermark image data set according to the first prediction probability and the first loss calculation algorithm.
S402: and acquiring second loss data related to the trigger set according to the trigger set and a second loss calculation algorithm.
The second loss calculation algorithm is:
in the formula, Loss1The second loss data; m is the number of the classification task categories in the trigger set; y isiAn indicator variable representing a category i; p is a radical ofiA prediction probability representing that the predicted label belongs to category i; and k is a watermark embedding weight parameter and is used for preventing the deviation of a model decision boundary.
In this embodiment, the verification device inputs the trigger set into the neural network model, obtains a prediction result of the neural network model for the trigger set, obtains a second prediction probability according to the prediction result and classification label data corresponding to watermark images in the trigger set, and obtains second loss data related to the trigger set according to the second prediction probability and the second loss calculation algorithm.
S403: and superposing the first loss data and the second loss data to obtain total loss data of the neural network model, and training the neural network model for a plurality of times according to the loss data until the neural network model is converged to obtain a target neural network model.
In this embodiment, the verification device superimposes the first loss data and the second loss data to obtain total loss data of the neural network model, where the total loss data is:
Lossall＝Loss1+Loss2
and inputting the obtained total loss data into an optimizer corresponding to the neural network model, and training the neural network model for a plurality of times until the neural network model is converged to obtain a target neural network model.
Referring to fig. 5, fig. 5 is a schematic flowchart illustrating a model copyright verification according to an embodiment of the present application.
S5: responding to a model copyright verification instruction of a user, wherein the model copyright verification instruction comprises the trigger set, inputting the trigger set into the target neural network model, obtaining a classification result, and obtaining a verification result output by the target neural network model according to the watermark image in the trigger set, the corresponding classification label data and the classification result.
In this embodiment, a verification device obtains a model copyright verification instruction sent by a user, responds, obtains a trigger set in the model copyright verification instruction, inputs the trigger set into the target neural network model, obtains a classification result output by the target neural network model according to a watermark image in the trigger set and the target neural network model, and obtains a verification result output by the target neural network model according to the watermark image in the trigger set, a corresponding classification label, and the classification result.
Referring to fig. 6, fig. 6 is a schematic flowchart of S5 in the method for verifying copyright based on a neural network model with watermark embedding according to an embodiment of the present application, including steps S501 to S502:
s501: and acquiring error data according to the classification result, the classification label data corresponding to the watermark images in the trigger set and a classification error algorithm.
The classification error algorithm is as follows:
wherein E is the output result of the classification error algorithm; k is the total number of watermark images in the trigger set, Mp (x)i) Is the classification result; z is a radical ofiClassifying label data corresponding to the watermark images in the trigger set; Δ is a binary function.
In this embodiment, the verification device outputs the classification result and the classification label data corresponding to the watermark images in the trigger setEntering the classification error algorithm for comparison to obtain a plurality of output results E, wherein the output results comprise 1 and 0, and when Mp (x)i)≠ziIf yes, the output result is 1, otherwise, the output result is 0, and the proportion of the output result 1 to all the output results is obtained as the error data according to the output result E.
S502: according to the error data and a preset error threshold, when the error data is smaller than the error threshold, obtaining a successful verification result; and when the error data is larger than the error threshold, obtaining a verification failure result.
S6: responding to a model copyright verification instruction of a user, wherein the model copyright verification instruction comprises the trigger set, inputting the trigger set into the target neural network model, obtaining a classification result, and obtaining a verification result output by the target neural network model according to classification label data corresponding to watermark images in the trigger set and the classification result.
In this embodiment, a verification device obtains a model copyright verification instruction sent by a user, inputs a trigger set in the model copyright verification instruction into a target neural network model, obtains a classification result output by the target neural network model, compares classification tag data corresponding to watermark images in the trigger set with the classification result, if the classification tag data is the same as the classification result, the verification is successful, and if the classification tag data is not the same as the classification result, the verification fails, and obtains a verification result output by the target neural network model.
In the embodiment, the verification device is preset with an error threshold, when the verification device acquires the error data, the error data is compared with the error threshold, and when the error data is smaller than the error threshold, a successful verification result is acquired; and when the error data is larger than the error threshold, obtaining a verification failure result.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a copyright verification apparatus based on a neural network model for watermark embedding according to an embodiment of the present application, where the apparatus may implement all or a part of the copyright verification apparatus based on the neural network model for watermark embedding through software, hardware, or a combination of the two, and the apparatus 7 includes:
an obtaining module 71, configured to obtain a watermark image data set, where the watermark image data set includes a plurality of watermark images, and each watermark image has corresponding classification label data;
a first extraction module 72, configured to extract a plurality of watermark images in the watermark image data set as an original trigger set, and perform singular value decomposition processing on the original trigger set to obtain a trigger set sample;
a second extraction module 73, configured to extract classification tag data corresponding to the watermark image in the original trigger set, and make the classification tag data and the trigger set samples correspond to each other one by one to obtain a trigger set;
a training-embedding module 74, configured to input the watermark image data set and the trigger set into a neural network model to be embedded with a watermark, and train the neural network model for a plurality of times to obtain a target neural network model;
a verification module 75, configured to respond to a model copyright verification instruction of a user, where the model copyright verification instruction includes the trigger set, input the trigger set into the target neural network model, obtain a classification result, and obtain a verification result output by the target neural network model according to the watermark image in the trigger set, the corresponding classification tag data, and the classification result.
In the embodiment of the application, a watermark image data set is obtained through an obtaining module, wherein the watermark image data set comprises a plurality of watermark images, and each watermark image has corresponding classification label data; extracting a plurality of watermark images in the watermark image data set as an original trigger set through a first extraction module, and performing singular value decomposition processing on the original trigger set to obtain a trigger set sample; inputting the watermark image data set and the trigger set into a neural network model to be embedded with a watermark through a second extraction module, and training the neural network model for a plurality of times to obtain a target neural network model; responding to a model copyright verification instruction of a user through a verification module, wherein the model copyright verification instruction comprises the trigger set, inputting the trigger set into the target neural network model to obtain a classification result, and obtaining the verification result output by the target neural network model according to the watermark image in the trigger set, the corresponding classification label data and the classification result. According to the method and the device, the watermark frame consisting of the watermark image data set and the trigger set is constructed, the watermark embedding efficiency is improved, the influence of the watermark on the neural network model is reduced, the influence of the watermark on the function of the neural network model is reduced, the embedding effect is more stable, the watermark is not easy to erase or forge, and the universality and the robustness of the watermark are improved.
Referring to fig. 8, fig. 8 is a schematic structural diagram of an apparatus according to an embodiment of the present application, where the apparatus 5 includes: a processor 81, a memory 82, and a computer program 83 stored on the memory 82 and operable on the processor 81; the computer device may store a plurality of instructions, where the instructions are suitable for being loaded by the processor 81 and executing the method steps in the embodiments shown in fig. 1 to fig. 5, and a specific execution process may refer to specific descriptions of the embodiments shown in fig. 1, fig. 4, and fig. 6, which is not described herein again.
The Memory 82 may include a Random Access Memory (RAM) 82, and may also include a Read-Only Memory (Read-Only Memory) 82. Optionally, the memory 82 includes a non-transitory computer-readable medium. The memory 82 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 82 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions, etc.), instructions for implementing the various method embodiments described above, and the like; the storage data area may store data and the like referred to in the above respective method embodiments. The memory 82 may optionally be at least one memory device located remotely from the processor 81.
An embodiment of the present application further provides a storage medium, where the storage medium may store a plurality of instructions, where the instructions are suitable for being loaded by a processor and executed by the method steps in the embodiments shown in fig. 1, fig. 4, and fig. 6, and a specific execution process may refer to specific descriptions of the embodiments shown in fig. 1, fig. 4, and fig. 6, which is not described herein again.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc.
The present invention is not limited to the above-described embodiments, and various modifications and variations of the present invention are intended to be included within the scope of the claims and the equivalent technology of the present invention if they do not depart from the spirit and scope of the present invention.
Claims (6)
1. A copyright verification method based on a watermark embedded neural network model is characterized by comprising the following steps:
acquiring a watermark image data set, wherein the watermark image data set comprises a plurality of watermark images, and each watermark image has corresponding classification label data;
extracting a plurality of watermark images in the watermark image data set as an original trigger set, and performing singular value decomposition processing on the original trigger set to obtain a trigger set sample;
extracting classification label data corresponding to the watermark images in the original trigger set, and enabling the classification label data to correspond to the trigger set samples one by one to obtain a trigger set;
inputting the watermark image data set and the trigger set into a neural network model to be embedded with a watermark, and training the neural network model for a plurality of times to obtain a target neural network model;
responding to a model copyright verification instruction of a user, wherein the model copyright verification instruction comprises the trigger set, inputting the trigger set into the target neural network model, obtaining a classification result, and obtaining a verification result output by the target neural network model according to the watermark image in the trigger set, the corresponding classification label data and the classification result.
2. The watermark embedding and verifying method of the neural network model as claimed in claim 1, wherein the step of inputting the watermark image data set and the trigger set into the neural network model to be embedded with the watermark, training the neural network model for several times to obtain the target neural network model comprises the steps of:
obtaining first loss data related to the watermark image data set according to the watermark image data set and a first loss calculation algorithm, wherein the first loss calculation algorithm is as follows:
in the formula, Loss1The first loss data is obtained; i is the number of classification task categories in the watermark image data set; y isiAn indicator variable representing a category i; p is a radical ofiA prediction probability representing that the predicted label belongs to category i;
acquiring second loss data related to the trigger set according to the trigger set and a second loss calculation algorithm, wherein the second loss calculation algorithm is as follows:
in the formula, Loss1The second loss data; m is the number of the classification task categories in the trigger set; y isiAn indicator variable representing a category i; p is a radical ofiA prediction probability representing that the predicted label belongs to category i; k is a watermark embedding weight parameter;
and superposing the first loss data and the second loss data to obtain total loss data of the neural network model, and training the neural network model for a plurality of times according to the loss data until the neural network model is converged to obtain a target neural network model.
3. The watermark embedding and verification method of the neural network model according to claim 1, wherein the model copyright verification instruction in response to the user comprises the trigger set, the trigger set is input into the target neural network model to obtain the classification result, and the verification result output by the target neural network model is obtained according to the watermark image in the trigger set, the corresponding classification tag data and the classification result, comprising the steps of:
obtaining error data according to the classification result, the classification label data corresponding to the watermark images in the trigger set and a classification error algorithm, wherein the classification error algorithm is as follows:
wherein E is the output result of the classification error algorithm; k is the total number of watermark images in the trigger set, Mp (x)i) Is the classification result; z is a radical ofiClassifying label data corresponding to the watermark images in the trigger set; delta is a binary function;
according to the error data and a preset error threshold, when the error data is smaller than the error threshold, obtaining a successful verification result; and when the error data is larger than the error threshold, obtaining a verification failure result.
4. A copyright verification apparatus based on a neural network model for watermark embedding, comprising:
the system comprises an acquisition module, a classification module and a classification module, wherein the acquisition module is used for acquiring a watermark image data set, the watermark image data set comprises a plurality of watermark images, and each watermark image has corresponding classification label data;
the first extraction module is used for extracting a plurality of watermark images in the watermark image data set as an original trigger set, and performing singular value decomposition processing on the original trigger set to obtain a trigger set sample;
the second extraction module is used for extracting classification label data corresponding to the watermark images in the original trigger set, and enabling the classification label data to correspond to the trigger set samples one by one to obtain the trigger set;
the training-embedding module is used for inputting the watermark image data set and the trigger set into a neural network model to be embedded with the watermark, and training the neural network model for a plurality of times to obtain a target neural network model;
and the verification module is used for responding to a model copyright verification instruction of a user, wherein the model copyright verification instruction comprises the trigger set, inputting the trigger set into the target neural network model, acquiring a classification result, and acquiring the verification result output by the target neural network model according to the watermark image in the trigger set, the corresponding classification label data and the classification result.
5. An apparatus, comprising: a processor, a memory, and a computer program stored on the memory and executable on the processor; the computer program when executed by the processor implements the steps of a copyright verification apparatus based on a neural network model of watermark embedding as claimed in any one of claims 1 to 3.
6. A storage medium, characterized by: the storage medium stores a computer program which, when executed by a processor, implements the steps of the watermark embedding-based neural network model-based copyright verification apparatus according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111293312.2A CN113987429A (en) | 2021-11-03 | 2021-11-03 | Copyright verification method of neural network model based on watermark embedding |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111293312.2A CN113987429A (en) | 2021-11-03 | 2021-11-03 | Copyright verification method of neural network model based on watermark embedding |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113987429A true CN113987429A (en) | 2022-01-28 |
Family
ID=79746124
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111293312.2A Pending CN113987429A (en) | 2021-11-03 | 2021-11-03 | Copyright verification method of neural network model based on watermark embedding |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113987429A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114862650A (en) * | 2022-06-30 | 2022-08-05 | 南京信息工程大学 | Neural network watermark embedding method and verification method |
| CN115879072A (en) * | 2023-03-03 | 2023-03-31 | 南京信息工程大学 | Copyright protection method, device and medium for deep fake fingerprint detection model |
| CN117473469A (en) * | 2023-12-28 | 2024-01-30 | 广东佛山联创工程研究生院 | Model watermark embedding method and device, electronic equipment and storage medium |
-
2021
- 2021-11-03 CN CN202111293312.2A patent/CN113987429A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114862650A (en) * | 2022-06-30 | 2022-08-05 | 南京信息工程大学 | Neural network watermark embedding method and verification method |
| CN114862650B (en) * | 2022-06-30 | 2022-09-23 | 南京信息工程大学 | Neural network watermark embedding method and verification method |
| CN115879072A (en) * | 2023-03-03 | 2023-03-31 | 南京信息工程大学 | Copyright protection method, device and medium for deep fake fingerprint detection model |
| CN117473469A (en) * | 2023-12-28 | 2024-01-30 | 广东佛山联创工程研究生院 | Model watermark embedding method and device, electronic equipment and storage medium |
| CN117473469B (en) * | 2023-12-28 | 2024-05-10 | 广东佛山联创工程研究生院 | Model watermark embedding method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110020620B (en) | Face recognition method, device and equipment under large posture | |
| US9349076B1 (en) | Template-based target object detection in an image | |
| CN113987429A (en) | Copyright verification method of neural network model based on watermark embedding | |
| CN108427927B (en) | Object re-recognition method and apparatus, electronic device, program, and storage medium | |
| CN113159147B (en) | Image recognition method and device based on neural network and electronic equipment | |
| CN107958230B (en) | Facial expression recognition method and device | |
| CN113283446B (en) | Method and device for identifying object in image, electronic equipment and storage medium | |
| CN109190470B (en) | Pedestrian re-identification method and device | |
| CN112164002B (en) | Training method and device of face correction model, electronic equipment and storage medium | |
| CN110807314A (en) | Text emotion analysis model training method, device and equipment and readable storage medium | |
| CN114155244B (en) | Defect detection method, device, equipment and storage medium | |
| US11436436B2 (en) | Data augmentation system, data augmentation method, and information storage medium | |
| CN113761259A (en) | Image processing method and device and computer equipment | |
| CN112288831A (en) | Scene image generation method and device based on generation countermeasure network | |
| WO2024060684A1 (en) | Model training method, image processing method, device, and storage medium | |
| WO2021159802A1 (en) | Graphical captcha recognition method, apparatus, computer device, and storage medium | |
| CN111291695B (en) | Training method and recognition method for recognition model of personnel illegal behaviors and computer equipment | |
| CN114049568A (en) | Object shape change detection method, device, equipment and medium based on image comparison | |
| CN116229066A (en) | Portrait segmentation model training method and related device | |
| CN112966547A (en) | Neural network-based gas field abnormal behavior recognition early warning method, system, terminal and storage medium | |
| CN114387524B (en) | Image identification method and system for small sample learning based on multilevel second-order representation | |
| CN111222558A (en) | Image processing method and storage medium | |
| CN113627394B (en) | Face extraction method and device, electronic equipment and readable storage medium | |
| US20220092448A1 (en) | Method and system for providing annotation information for target data through hint-based machine learning model | |
| CN114881103A (en) | Countermeasure sample detection method and device based on universal disturbance sticker |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |