CN112613001A - Method for realizing integrity authentication of convolutional neural network through reversible watermark - Google Patents

Method for realizing integrity authentication of convolutional neural network through reversible watermark Download PDF

Info

Publication number
CN112613001A
CN112613001A CN202011517502.3A CN202011517502A CN112613001A CN 112613001 A CN112613001 A CN 112613001A CN 202011517502 A CN202011517502 A CN 202011517502A CN 112613001 A CN112613001 A CN 112613001A
Authority
CN
China
Prior art keywords
watermark
sequence
neural network
convolutional neural
reversible
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011517502.3A
Other languages
Chinese (zh)
Inventor
俞能海
张卫明
管玺权
周航
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology of China USTC
Original Assignee
University of Science and Technology of China USTC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology of China USTC filed Critical University of Science and Technology of China USTC
Priority to CN202011517502.3A priority Critical patent/CN112613001A/en
Publication of CN112613001A publication Critical patent/CN112613001A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/10—Protecting distributed programs or content, e.g. vending or licensing of copyrighted material ; Digital rights management [DRM]
    • G06F21/16—Program or content traceability, e.g. by watermarking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Technology Law (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Editing Of Facsimile Originals (AREA)

Abstract

The invention discloses a method for realizing integrity authentication of a convolutional neural network through reversible watermarking, which can embed watermarking information into a model through a reversible watermarking technology, and realizes the integrity authentication of the model by combining the theory of traditional multimedia reversible information hiding, so that a model receiver can actively judge the integrity and reliability of the model, and potential dangers such as introduction of a model backdoor and the like under the unknowing condition are avoided.

Description

Method for realizing integrity authentication of convolutional neural network through reversible watermark
Technical Field
The invention relates to the technical field of neural network integrity authentication, in particular to a method for realizing convolutional neural network integrity authentication through reversible watermarks.
Background
Deep Convolutional Neural Networks (CNNs) have achieved significant achievements in the field of computer vision, such as image classification, automatic driving, etc. However, as the performance of the models increases, the structures of the models become more complex and training becomes more difficult. To address these problems, many scholars publish their pre-trained models for others to download and study. However, these pre-trained models are vulnerable to illegal tampering by attackers during propagation, such as backdoor attacks, replay attacks, and the like. These attacks can leave fatal holes in the model, reduce the accuracy of the model and threaten the safety of the model. Therefore, it is an important content of model security research to ensure that the model is not illegally tampered and to perform integrity certification on the model.
There are two main methods for protecting the security of the model: passive defense and active verification. Passive defense focuses on detection and erasure of tampering, but this approach easily leads to missed detection and false alarms; active verification is achieved by manually embedding some meaningful information (e.g., a model watermark). Depending on whether the internal details of the model are known to the public, model watermarks can be roughly divided into two categories: white-box watermarks and black-box watermarks. In the white-box watermarking method, watermark information is directly embedded into weight and deviation inside a model; the black box watermarking method is to change the decision boundary of the model by a certain method and embed the watermark into the model only having the access right of an Application Programming Interface (API). However, these watermarking techniques are all irreversible. During the embedding process, irreversible watermarking can only minimize the impact on the original model performance, but such watermarking can still permanently modify the internal parameters and destroy the integrity of the model.
Disclosure of Invention
The invention aims to provide a method for realizing integrity authentication of a convolutional neural network through reversible watermarks.
The purpose of the invention is realized by the following technical scheme:
a method for implementing convolutional neural network integrity authentication through reversible watermarking, comprising:
reversible watermark embedding stage of convolutional neural network model: the operation object is an original convolution neural network model single layer or multiple layers, each layer is independently carried out, and the operation process of each layer comprises the following steps: acquiring a feature map generated by an ith layer of an original convolutional neural network model for an input image, and selecting parameters of a plurality of channels as a carrier sequence for embedding watermarks according to an entropy value calculation result of the feature map; after preprocessing the carrier sequence, reversibly embedding watermark information into the carrier sequence by adopting an image reversible hiding method to obtain a secret-carrying sequence, updating the parameter of the ith layer of the convolutional neural network model by utilizing the secret-carrying sequence, and embedding the encrypted parameter information which ensures that the watermark can be reversibly extracted into the related parameter of the ith layer in a replacement mode; finally, obtaining an embedded watermark model; the watermark information comprises characteristic information of an original convolutional neural network model and parameter information for ensuring reversible extraction of the watermark;
and (3) integrity authentication stage of the convolutional neural network model: an application acquires an embedded watermark model, extracts watermark information in a mode opposite to a reversible watermark embedding stage, and reconstructs an original convolutional neural network model; and extracting the characteristic information of the reconstructed original convolutional neural network model, and comparing the characteristic information with the extracted watermark information so as to judge whether the embedded watermark model is tampered.
According to the technical scheme provided by the invention, the watermark information can be embedded into the model through the reversible watermark technology, and the integrity authentication of the model is realized by combining the theory of reversible information hiding of the traditional multimedia, so that a model receiver can actively judge the integrity and reliability of the model, and potential dangers such as introduction of a back door of the model due to illegal tampering under the unknown condition are avoided.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a schematic diagram of a method for implementing integrity authentication of a convolutional neural network by using reversible watermarking according to an embodiment of the present invention;
fig. 2 is a flowchart of reversible watermark embedding and extraction provided by an embodiment of the present invention;
fig. 3 is a schematic diagram of a process of embedding a watermark in a histogram shift according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a process of embedding a watermark in a histogram shift according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
Aiming at the problem that the existing convolutional neural network model watermark permanently modifies the internal parameters of the model and can not realize integrity authentication, the invention provides a reversible watermarking method aiming at the convolutional neural network model, and the integrity authentication of the model is realized by combining the theory of traditional multimedia reversible information hiding. The method enables a model receiver to judge the integrity and reliability of the model actively, and avoids potential dangers such as introduction of a model backdoor under an unknown condition. Experiments prove that the influence of the reversible watermark embedding in the invention on the performance of the model can be ignored, and the method has higher concealment; after the watermark is extracted, the model can be completely reconstructed and is completely consistent with the original model parameters. In the aspect of integrity authentication, the authentication method provided by the invention can also well realize integrity authentication, and the data security of the model is protected.
As shown in fig. 1, a schematic diagram of a method for implementing integrity authentication of a convolutional neural network by using a reversible watermark according to an embodiment of the present invention mainly includes the following two stages:
first, reversible watermark embedding stage of the convolution neural network model.
The operation object of the reversible watermark embedding stage is a single layer or multiple layers of an original convolutional neural network model, each layer is independently carried out, and the operation process of each layer mainly comprises the following steps: acquiring a feature map generated by an ith layer of an original convolutional neural network model for an input image, and selecting parameters of a plurality of channels as a carrier sequence for embedding watermarks according to an entropy value calculation result of the feature map; after the carrier sequence is preprocessed, watermark information is reversibly embedded into the carrier sequence (parameters ensuring that the watermark can be reversibly extracted are embedded into the carrier together) by adopting an image reversible hiding method to obtain a secret-carrying sequence; updating the parameter of the ith layer of the convolutional neural network model by using the secret-carrying sequence, and embedding the encrypted parameter which ensures reversible extraction of the watermark into the relevant parameter of the ith layer in a replacement mode; and finally obtaining the embedded watermark model.
As shown in the upper part of fig. 2, a preferred embodiment of this stage is as follows:
1. and acquiring a characteristic diagram generated by the ith layer of the original convolutional neural network model for the input image, and selecting parameters of a plurality of channels as a carrier sequence for embedding the watermark according to the entropy calculation result of the characteristic diagram.
1) And selecting mu images from the data set as the input of an original convolutional neural network model, and acquiring the feature tensor T output by the ith layer.
For each image, a tensor with size d × h × w can be obtained, where d is the number of output channels, and h and w are the length and width of the output feature map, respectively. For μ images, a tensor T of size μ × d × h × w can be obtained.
2) In order to better characterize the importance of the channel, the average pooling operation is performed on several dimensions of the feature tensor T to obtain the feature matrix F.
For example, the 3 rd and 4 th dimensions of the feature tensor T may be selected to perform an average pooling operation, so as to obtain a tensor with a size of μ × d × 1 × 1, and discard the dimension with a size of 1, so as to obtain the feature matrix F with a size of μ × d.
3) Rearranging the characteristic matrix F according to columns to obtain a sequence FF1,F2,…,FdFor each column }Vector FlThe intervals are equally divided, and then the probability p of falling into the interval r is calculatedrFor calculating an entropy value Hl：
Figure BDA0002848467580000041
Wherein, FlE.g. F, each column vector FlAll the sizes of (1) are Mux 1; d is the number of channels, prIs the probability of falling in the r interval.
4) For the resulting entropy sequence H ═ H1,H2,…,HdArranging according to the order from small to large to obtain an entropy value ascending sequence
Figure BDA0002848467580000045
Wherein J ═ { J ═ J1,j2,…,jdIs an arrangement of {1,2, …, d }; selecting the first N channels corresponding to the ascending sequence of entropy values as alternative parameters of the carrier sequence structure, and sequentially arranging according to the ascending sequence to obtain:
Figure BDA0002848467580000042
wherein N is less than d; k represents the convolution kernel of the corresponding index channel, and the size is K x (K × c), wherein c represents the number of convolution kernels of the channel;
for convenience of description of the subsequent steps, W isiThe rewrite is:
Figure BDA0002848467580000043
wherein, a is k × N, b is k × c; w is the convolution kernel parameter.
2. Pre-treating the vector sequence to obtain vector sequence WiThe signed floating point number in the watermark embedding vector is processed to generate an unsigned integer vector sequence so as to meet the requirement of embedding the watermark into the vector.
1) Vector sequence WiMiddle alpha row and beta columnHas an element value of wαβThen its structure is represented as:
Figure BDA0002848467580000044
wherein, ± represents wαβThere are positive or negative two cases, p and q respectively represent wαβThe number of 0 digits and non-zero digits from the decile digit, p is more than or equal to 0, and q is more than 0;
2) selecting wαβAdjacent two non-zero digits are used as an integer sequence; in the selection strategy of digit, in order to make the embedded information quantity of the constructed sequence larger, n is selected in turn1n2,n2n3,…,nq-1nqConstructing a carrier, calculating the sequence entropy of the carrier, and selecting the adjacent digit n corresponding to the carrier with the minimum entropy valuecnc+1Then, add a positive integer V to adjust the value range to fall within [0,255]In (1), the corresponding w of the new structure is recordedαβHas an element value of
Figure BDA0002848467580000051
The vector sequence after pre-data processing is then expressed as:
Figure BDA0002848467580000052
3. watermark information can be reversibly embedded into a carrier sequence by adopting an image reversible hiding method.
1) The pre-treated vector sequence
Figure BDA0002848467580000053
As image form, statistics
Figure BDA0002848467580000054
Is given as a histogram of
Figure BDA0002848467580000055
2) Writing applianceSquare drawing
Figure BDA0002848467580000056
The peak value of (a) corresponds to an element value of
Figure BDA0002848467580000057
The value of the element (corresponding abscissa) corresponding to the bottom (generally 0) of the histogram is
Figure BDA0002848467580000058
Without loss of generality, order
Figure BDA0002848467580000059
3) As shown in fig. 2 and 3, watermark information is embedded into a carrier sequence according to a histogram shifting method
Figure BDA00028484675800000510
To obtain a new element value
Figure BDA00028484675800000511
Figure BDA00028484675800000512
Wherein alpha beta is a row and column number,
Figure BDA00028484675800000513
the element value before embedding watermark information.
The embedded watermark information is a bit string (e.g., 011010010 … …), where 1 bit information (denoted as b) represents 1 bit of the bit string, i.e., + b in the equation, and the equation +1 represents histogram shift (the process is shown in the embedding example).
4) Vector sequence to be constructed
Figure BDA00028484675800000514
Embedding watermark information according to the above mode to obtainTo the secret carrier sequence
Figure BDA00028484675800000515
In the embodiment of the invention, the watermark information mainly comprises two parts of information: the first part is parameter information for ensuring reversible extraction of the watermark, and the second part is characteristic information of an original convolutional neural network model; the first part is used as the head information and combined with the second part to form the final watermark information which is embedded into the carrier sequence to obtain the carrier sequence
Figure BDA00028484675800000516
The parameter information for ensuring the reversible extraction of the watermark is used as the additional information to ensure the smooth proceeding of the watermark extraction process
The related parameter information mainly includes: determining the ascending index sequence J ═ { J ═ J of the corresponding channel according to the entropy calculation result of the characteristic diagram1,j2,…,jdN, the number of selected channels, c, the embedded bit of the selected watermark information, V, which is a positive integer added during the preprocessing of the carrier sequence, and the element value corresponding to the peak value in the histogram of the preprocessed carrier sequence
Figure BDA00028484675800000517
Value of element corresponding to valley value
Figure BDA0002848467580000061
4. And updating the model parameters.
Obtaining secret-carrying sequence by the above method
Figure BDA0002848467580000062
Will carry the secret sequence
Figure BDA0002848467580000063
The elements in (1) are put back into the model according to the taking-out sequence, and parameters are updated.
Then, after the aforementioned extra information (i.e. the parameter information that ensures reversible extraction of the watermark) is encrypted, the extra information is embedded into the parameter sequence of the i-th layer by using an LSB (least significant bit replacement) method (without special filtering), and the embedded position is the convolution kernel parameter. Here, the least significant bit to be replaced needs to be used as the header information of the embedded watermark (step 3) in advance to ensure the lossless whole process.
The above introduces a single-layer operation process, and if multiple original convolutional neural network models are needed to perform reversible watermark embedding in multiple layers, each layer independently executes the above operations, and updates parameters of the corresponding layer, thereby finally obtaining an embedded watermark model.
And II, carrying out integrity authentication on the convolutional neural network model.
The integrity authentication phase mainly comprises: an application acquires an embedded watermark model, extracts watermark information in a mode which is reversible to a reversible watermark embedding stage, and reconstructs an original convolutional neural network model; and extracting the characteristic information of the reconstructed original convolutional neural network model, and comparing the characteristic information with the extracted watermark information so as to judge whether the embedded watermark model is tampered.
The preferred embodiment of this stage is as follows:
1. and uploading the embedded watermark model to a cloud server by the model holder for downloading and using by others.
2. And a model application user downloads the model from the cloud server, extracts watermark information in a mode reversible to the reversible watermark embedding stage, and reconstructs the original convolutional neural network model.
The reversible watermark extraction process involved in this step is reciprocal to the watermark embedding process described above, and as shown in the lower part of fig. 2, mainly includes:
1) extracting necessary information for watermark extraction from the ith layer of the embedded watermark model; i.e. the ascending index sequence J, the number of channels N, the embedding bit c, the value range adjustment parameter V, and the information related to the carrier histogram mentioned above
Figure BDA0002848467580000064
And
Figure BDA0002848467580000065
2) based on necessary information in watermark extraction, restoring secret-carrying sequence
Figure BDA0002848467580000066
3) Statistical secret-carrying sequence
Figure BDA0002848467580000067
Histogram of (1)
Figure BDA0002848467580000068
Extracting watermark information by the following formula:
Figure BDA0002848467580000069
thereby recovering the vector sequence
Figure BDA0002848467580000071
Figure BDA0002848467580000072
Wherein the content of the first and second substances,
Figure BDA0002848467580000073
each being a secret carrying sequence
Figure BDA0002848467580000074
Vector sequences
Figure BDA0002848467580000075
The value of the element in (1), alpha beta is a row number and a column number;
4) restoring the extracted watermark information header to the least significant bits modified by the embedding of the additional information, based on the information necessary for watermark extraction and the carrier sequence
Figure BDA0002848467580000076
And (5) restoring the parameters of the ith layer, and reconstructing an original convolutional neural network model.
3. And extracting the characteristic information of the reconstructed original convolutional neural network model, and comparing the characteristic information with the extracted watermark information (head information is removed) to judge whether the model is tampered.
Two main cases are distinguished: case 1: if the model is illegally tampered by an attacker, the model is relatively unsuccessful (i.e. the two are different), and the model is determined to be an unsafe model (as shown in the upper right part of fig. 1). Case 2: if the model is not tampered with, the comparison is successful (i.e. the two are the same), and the model is determined to be a safe model and can be used safely (as shown in the lower right part of fig. 1).
If the reversible watermark embedding is carried out on the multiple layers of the original convolutional neural network model, each layer needs to be judged through the above mode, and the judgment results of all the layers are the condition 2, the model is determined to be a safety model.
In order to illustrate the effectiveness of the invention, the implementation of the reversible watermark and the integrity authentication are respectively verified through experiments.
Reversible watermark implementation of convolutional neural network model
According to the method provided by the invention, 5 models of the main flow are selected: AlexNet, DenseNet121, MobileNet, VGG19 and ResNet152 are verified on a data set ImageNet, and the single-layer watermark embedding effect, the multi-layer watermark embedding effect and the error of the extraction and reconstruction of the model watermark are respectively verified.
1) Model single-layer watermark embedding effect.
According to a reversible watermarking algorithm of the model, 5 types of model inverse first layer convolution layer embedded watermarking information are respectively selected, and classification accuracy of the embedded watermarking model and a clean model is compared, so that the influence of the embedded watermarking model on the performance of the model can be ignored after the watermarking is embedded, and the specific experimental result is as follows:
Figure BDA0002848467580000077
Figure BDA0002848467580000081
TABLE 1 model Single layer watermark embedding Performance (Classification accuracy%, watermark length bits)
As can be seen from the data in table 1, the model embedding reversible watermark has little influence on the performance of itself, and can be ignored. Meanwhile, as can be seen from comparison of the sizes of the embedded watermarks in the first four rows and the second two rows of the table, the length of the embedded watermark has little influence on the performance of the model. This demonstrates that the present invention is effective.
2) And (5) modeling multi-layer watermark embedding effect.
In the part, three layers of convolution layers with the last number being 5 models are respectively selected to embed watermark information, and the classification accuracy of the embedded watermark model and the classification accuracy of the clean model are compared, so that the influence of the embedded watermark model on the performance of the model can be ignored, and the specific experimental results are as follows:
Figure BDA0002848467580000082
TABLE 2 model Multi-layer watermark embedding Performance (% Classification accuracy)
From the experimental results in table 2, it can be seen that the number of layers of the model embedded with the reversible watermark has little influence on the performance of the model itself, and can be ignored.
3) And (5) model reconstruction error.
In this section, model reconstruction errors after the watermarks are extracted from the 5 models are respectively displayed, and the specific experimental results are as follows:
type of model AlexNet DenseNet MobileNet VGG19 ResNet152
Reconstruction error
0 0 0 0 0
TABLE 3 model reconstruction error
From the experimental results in table 3, it can be seen that the watermark is extracted and reconstructed from the model embedded with the watermark, and the obtained model is completely the same as the original model, which directly proves the reversibility of the invention.
Integrity authentication of convolutional neural network model by reversible watermark
In the part, ResNet152 is selected for model integrity authentication, the characteristic information of the model is encrypted by using a Hash algorithm of SHA-256 and is used as watermark information, 10 times of experiments are repeated, wherein in the experiments numbered 1,2, 4, 6 and 8, model parameters are slightly modified, and in the other experiments, the model parameters are not modified. The experimental results are as follows: for the experiment with modified parameters, the integrity authentication method can judge success and prompt the illegal tampering of the model; for the experiment without parameter modification, the integrity authentication method can also prompt that the model integrity authentication is passed.
Through the above description of the embodiments, it is clear to those skilled in the art that the above embodiments can be implemented by software, and can also be implemented by software plus a necessary general hardware platform. With this understanding, the technical solutions of the embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.), and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods according to the embodiments of the present invention.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (7)

1. A method for realizing integrity authentication of a convolutional neural network through reversible watermarking, which is characterized by comprising the following steps:
reversible watermark embedding stage of convolutional neural network model: the operation object is an original convolution neural network model single layer or multiple layers, each layer is independently carried out, and the operation process of each layer comprises the following steps: acquiring a feature map generated by an ith layer of an original convolutional neural network model for an input image, and selecting parameters of a plurality of channels as a carrier sequence for embedding watermarks according to an entropy value calculation result of the feature map; after preprocessing the carrier sequence, reversibly embedding watermark information into the carrier sequence by adopting an image reversible hiding method to obtain a secret-carrying sequence, updating the parameter of the ith layer of the convolutional neural network model by utilizing the secret-carrying sequence, and embedding the encrypted parameter information which ensures that the watermark can be reversibly extracted into the related parameter of the ith layer in a replacement mode; finally, obtaining an embedded watermark model; the watermark information comprises characteristic information of an original convolutional neural network model and parameter information for ensuring reversible extraction of the watermark;
and (3) integrity authentication stage of the convolutional neural network model: an application acquires an embedded watermark model, extracts watermark information in a mode opposite to a reversible watermark embedding stage, and reconstructs an original convolutional neural network model; and extracting the characteristic information of the reconstructed original convolutional neural network model, and comparing the characteristic information with the extracted watermark information so as to judge whether the embedded watermark model is tampered.
2. The method according to claim 1, wherein the obtaining a feature map generated by an i-th layer of an original convolutional neural network model for an input image, and selecting parameters of a plurality of channels as a carrier sequence for embedding a watermark according to an entropy calculation result of the feature map comprises:
selecting mu images from the data set as the input of an original convolutional neural network model, and acquiring a feature tensor T output by the ith layer;
carrying out average pooling operation on a plurality of dimensions of the feature tensor T to obtain a feature matrix F;
rearranging the characteristic matrix F according to columns to obtain a sequence F ═ { F }1，F2，…，FdFor each column vector FlThe intervals are equally divided, and then the probability p of falling into the interval r is calculatedrFor calculating an entropy value Hl：
Figure FDA0002848467570000011
Wherein, FlE is F, d is the number of channels, prIs the probability of falling within the r interval;
for the resulting entropy sequence H ═ H1，H2，…，HdArranging the entropy values in the order from small to large to obtain an entropy value ascending sequence H ═ Hj1，Hj2，…，HjdWhere J ═ J1，j2，…，jdIs an arrangement of {1,2, …, d }; selecting the first N channels in the ascending sequence of entropy as the preparation of carrier sequence structureSelecting parameters, and sequentially arranging according to an ascending order to obtain:
Figure FDA0002848467570000021
wherein N is less than d; k represents the convolution kernel of the corresponding index channel, and the size is K x (K × c), wherein c represents the number of convolution kernels of the channel;
w is to beiThe rewrite is:
Figure FDA0002848467570000022
wherein, a is k × N, b is k × c; w is the convolution kernel parameter.
3. The method for realizing integrity authentication of convolutional neural networks through reversible watermarking as claimed in claim 1 or 2, wherein the preprocessing the carrier sequence comprises: vector sequence WiThe signed floating point number in the sequence is processed to generate an unsigned integer carrier sequence, and the steps are as follows:
vector sequence WiThe value of the element in the middle alpha row and beta column is wαβThen its structure is represented as:
Figure FDA0002848467570000023
wherein, ± represents wαβThere are positive or negative two cases, p and q respectively represent wαβThe number of 0 digits and non-zero digits from the decile digit, p is more than or equal to 0, and q is more than 0;
selecting wαβAdjacent two non-zero digits are used as an integer sequence; sequentially selecting n in the selection strategy of digit1n2，n2n3，...，nq-1nqConstructing a carrier, calculating the sequence entropy of the carrier, and selecting the adjacent digit n corresponding to the carrier with the minimum entropy valuecnc+1Then, add a positive integer V to adjust the value range to fall within [0,255]In (1), the corresponding w of the new structure is recordedαβHas an element value of
Figure FDA0002848467570000024
The vector sequence after pre-data processing is then expressed as:
Figure FDA0002848467570000025
4. the method for realizing integrity authentication of convolutional neural network through reversible watermark according to claim 1, wherein the reversible embedding of watermark information into carrier sequence by using image reversible hiding method comprises:
the pre-treated vector sequence
Figure FDA0002848467570000026
As image form, statistics
Figure FDA0002848467570000027
Is given as a histogram of
Figure FDA0002848467570000028
Recording histogram
Figure FDA0002848467570000029
The peak value of (a) corresponds to an element value of
Figure FDA00028484675700000210
The value of the element corresponding to the bottom of the histogram is
Figure FDA00028484675700000214
Watermarking according to histogram translation methodInformation embedding in carrier sequences
Figure FDA00028484675700000212
To obtain a new element value
Figure FDA00028484675700000213
Figure FDA0002848467570000031
Wherein alpha beta is a row and column number,
Figure FDA0002848467570000032
the element value before embedding watermark information.
5. The method for realizing integrity authentication of convolutional neural network through reversible watermark according to claim 1 or 4, wherein the step of ensuring that the parameter information reversibly extracted by the watermark is used as header analysis of the watermark information comprises:
the method comprises the steps of determining an ascending index sequence of a corresponding channel according to an entropy calculation result of a characteristic diagram, the number of selected channels, the embedding position of selected watermark information, a positive integer added when a carrier sequence is preprocessed, and an element value corresponding to a peak value and an element value corresponding to a valley value in a preprocessed carrier sequence histogram.
6. The method for realizing integrity authentication of convolutional neural network through reversible watermark according to claim 1, wherein the embedding the encrypted parameter information ensuring reversible extraction of the watermark into the relevant parameters of the i-th layer in an alternative manner includes:
and embedding the encrypted parameter information which ensures reversible extraction of the watermark into the convolution kernel parameter of the ith layer by using a least significant bit replacement method.
7. The method of claim 1, wherein extracting watermark information in a manner opposite to the reversible watermark embedding stage and reconstructing the original convolutional neural network model comprises:
extracting parameter information for ensuring reversible extraction of the watermark from the ith layer of the embedded watermark model;
based on relevant parameter information, recovering secret-carrying sequence
Figure FDA0002848467570000033
Statistical secret-carrying sequence
Figure FDA0002848467570000034
Histogram of (1)
Figure FDA0002848467570000035
Extracting watermark information by the following formula:
Figure FDA0002848467570000036
thereby recovering the vector sequence
Figure FDA0002848467570000037
Figure FDA0002848467570000038
Wherein the content of the first and second substances,
Figure FDA0002848467570000039
each being a secret carrying sequence
Figure FDA00028484675700000310
Vector sequences
Figure FDA00028484675700000311
The value of the element in (1), alpha beta is a row number and a column number;
based on related parameter information and carrier sequence
Figure FDA00028484675700000312
And (5) restoring the parameters of the ith layer, and reconstructing an original convolutional neural network model.
CN202011517502.3A 2020-12-21 2020-12-21 Method for realizing integrity authentication of convolutional neural network through reversible watermark Pending CN112613001A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011517502.3A CN112613001A (en) 2020-12-21 2020-12-21 Method for realizing integrity authentication of convolutional neural network through reversible watermark

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011517502.3A CN112613001A (en) 2020-12-21 2020-12-21 Method for realizing integrity authentication of convolutional neural network through reversible watermark

Publications (1)

Publication Number Publication Date
CN112613001A true CN112613001A (en) 2021-04-06

Family

ID=75243850

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011517502.3A Pending CN112613001A (en) 2020-12-21 2020-12-21 Method for realizing integrity authentication of convolutional neural network through reversible watermark

Country Status (1)

Country Link
CN (1) CN112613001A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114006733A (en) * 2021-10-08 2022-02-01 北卡科技有限公司 Method and system for verifying data transmission integrity
CN114647824A (en) * 2022-05-23 2022-06-21 南京信息工程大学 Active protection method and system for neural network, storage medium and computing equipment
CN116881871A (en) * 2023-09-06 2023-10-13 腾讯科技（深圳）有限公司 Model watermark embedding method, device, computer equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
XIQUAN GUAN 等: "Reversible Watermarking in Deep Convolutional Neural Networks for Integrity Authentication", 《PROCEEDINGS OF THE 28TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA (MM’20)》 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114006733A (en) * 2021-10-08 2022-02-01 北卡科技有限公司 Method and system for verifying data transmission integrity
CN114006733B (en) * 2021-10-08 2023-10-20 北卡科技有限公司 Method and system for verifying data transmission integrity
CN114647824A (en) * 2022-05-23 2022-06-21 南京信息工程大学 Active protection method and system for neural network, storage medium and computing equipment
CN116881871A (en) * 2023-09-06 2023-10-13 腾讯科技（深圳）有限公司 Model watermark embedding method, device, computer equipment and storage medium
CN116881871B (en) * 2023-09-06 2023-11-24 腾讯科技（深圳）有限公司 Model watermark embedding method, device, computer equipment and storage medium

Similar Documents

Publication Publication Date Title
Li et al. How to prove your model belongs to you: A blind-watermark based framework to protect intellectual property of DNN
Li et al. Concealed attack for robust watermarking based on generative model and perceptual loss
CN112613001A (en) Method for realizing integrity authentication of convolutional neural network through reversible watermark
Guan et al. Reversible watermarking in deep convolutional neural networks for integrity authentication
Lydia et al. Application of discrete transforms with selective coefficients for blind image watermarking
Ali et al. An image watermarking scheme in wavelet domain with optimized compensation of singular value decomposition via artificial bee colony
Wang et al. Data hiding with deep learning: A survey unifying digital watermarking and steganography
US7324662B2 (en) Method, software, and device for hiding data in binary image, while preserving image quality
MaungMaung et al. A protection method of trained CNN model with a secret key from unauthorized access
Barni et al. DNN watermarking: Four challenges and a funeral
Zhu et al. Fragile neural network watermarking with trigger image set
Liu et al. An invisible and robust watermarking scheme using convolutional neural networks
Meenakshi et al. A hybrid matrix factorization technique to free the watermarking scheme from false positive and negative problems
Lou et al. Ownership verification of dnn architectures via hardware cache side channels
Barani et al. Image forgery detection in contourlet transform domain based on new chaotic cellular automata
Ito et al. Access control using spatially invariant permutation of feature maps for semantic segmentation models
Abdulmunem et al. Advanced Intelligent Data Hiding Using Video Stego and Convolutional Neural Networks
CN116523725A (en) Watermark processing method and device of neural network model
Bravo-Solorio et al. Watermarking with lowembedding distortion and self-propagating restoration capabilities
Olliaro et al. Empirical analysis of the impact of queries on watermarked relational databases
Vybornova et al. Copyright protection for image classification models using pseudo-holographic watermarks
Shady et al. Local features-based watermarking for image security in social media
Ito et al. Access control of semantic segmentation models using encrypted feature maps
Zhang et al. Mitigating targeted bit-flip attacks via data augmentation: An empirical study
Lin et al. Protecting IP of deep neural networks with watermarking using logistic disorder generation trigger sets

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210406