JP6908553B2 - Information processing equipment, information processing methods, and programs - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

In recent years, the speed of CPU (Central Processing Unit) and GPU (Graphics Processing Unit) has been increased, the capacity of memory has been increased, and machine learning technology has been rapidly advanced. For this reason, machine learning using learning data on the order of hundreds of thousands to millions has become possible, and highly accurate identification technology and classification technology are being established (see Non-Patent Document 1).

Further, a technique for identifying the source by embedding a digital watermark such as a source or a provider in the model parameter of the learning model generated by the above machine learning has also been proposed (see Non-Patent Document 2).

Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia (pp. 675-678) ). ACM. Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin’ichi Satoh.: Embedding Watermarks into Deep Neural Networks. In Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, ser. ICMR 2017 (pp.269-277), 2017.

The technique disclosed in Non-Patent Document 2 is a vector obtained by binarizing a vector obtained by multiplying a fixed private watermark detection matrix by a vector composed of a part of model parameters of a learning model. , The training model is generated so that it becomes a watermark vector. Since the watermark detection matrix is not open to the public, it is possible that a third party may forge a fake watermark matrix different from the original watermark detection matrix after the fact.

The present invention has been made in view of these points, and an object of the present invention is to provide a learning model capable of identifying the source and to provide a technique capable of showing that the watermark is not a fake watermark.

The first aspect of the present invention is an information processing device. This device is selected from a weight update unit that updates the weight of each layer of a neural network having a plurality of layers and a layer that constitutes the neural network based on training data including input data for learning a target task. A weight vector generation unit that generates a weight vector having weights of one or more layers as elements, a matrix generation unit that generates a matrix for watermark detection for detecting a watermark vector to be embedded in the neural network, and the watermark detection. It includes a watermark vector generation unit that generates a vector obtained by inputting a component of a neural network into a predetermined hash function as a watermark vector. The weight update unit updates the weights of each layer of the neural network on the condition that the error between the vector obtained from the weight vector and the watermark detection matrix and the watermark vector is reduced as a constraint condition.

The matrix generation unit may generate the watermark detection matrix by embedding additional information about the neural network in the initial matrix.

The matrix generation unit may generate additional information itself regarding the neural network as the watermark detection matrix.

The watermark vector generation unit is a vector obtained by the exclusive OR of the vector obtained by inputting the components of the watermark detection matrix into the predetermined hash function and the additional vector which is additional information about the neural network. May be set as the watermark vector.

The information processing device obtains the exclusive OR of the additional vector and the data obtained by multiplying the weight vector generated from the detection target neural network by the watermark detection matrix and applying binarization to the watermark. A watermark detection unit that detects a vector may be further provided.

When the length of the weight vector is P and the length of the vector which is the hash value output by the hash function is Q, the matrix generator has a column length of P and a row length of Q. The matrix may be generated as the watermark detection matrix.

A second aspect of the present invention is an information processing method executed by a processor. In this method, the processor selects from a step of updating the weight of each layer of a neural network including a plurality of layers and a layer constituting the neural network based on training data including data for learning a target task. A step of generating a weight vector having the weight of one or more layers as an element, a step of generating a watermark detection matrix for detecting a watermark to be embedded in the neural network, and a component of the watermark detection matrix. It includes a step of generating a vector obtained by inputting into a predetermined hash function as a watermark vector. In the step of updating the weight, the weight is updated with the constraint condition that the error between the weight vector, the vector obtained from the watermark detection matrix, and the watermark vector is reduced.

A third aspect of the present invention is a program that causes a computer to realize an information processing function. This program has a function of updating the weight of each layer of a neural network having a plurality of layers based on training data including data for learning a target task on the computer, and a function of selecting from the layers constituting the neural network. A function of generating a weight vector having the weight of one or more layers as an element, a function of generating a watermark detection matrix for detecting a watermark embedded in the neural network, and a component of the watermark detection matrix. The computer realizes a function of generating a vector obtained by inputting into a predetermined hash function as a watermark vector. The function of updating the weight updates the weight of each layer of the neural network on the constraint condition that the error between the vector obtained from the weight vector and the watermark detection matrix and the watermark vector is reduced.

According to the present invention, it is possible to provide a learning model capable of identifying the source and a technique capable of showing that the watermark is not a fake watermark.

It is a figure which shows typically the general functional structure of a convolutional neural network. It is a figure for demonstrating the relationship of the convolution of the input data and a feature map. It is a figure for demonstrating the generation of a feature map using a weighting filter. It is a figure for demonstrating the relationship between N weight filters and N-stage feature map. It is a schematic diagram for demonstrating the outline of the learning process executed by the information processing apparatus which concerns on embodiment. It is a figure which shows typically the functional structure of the information processing apparatus which concerns on embodiment. It is a figure for demonstrating the weight setting process by the weight vector generation part which concerns on embodiment. It is a figure for demonstrating embedding of additional information different from a watermark vector. It is a flowchart for demonstrating the flow of information processing executed by the information processing apparatus which concerns on embodiment.

<Convolutional neural network>
The information processing device according to the embodiment is a device for embedding watermark information in a model parameter of a neural network, and a convolutional neural network (CNN) is particularly suitable. Therefore, as a prerequisite technology of the information processing apparatus according to the embodiment, first, a convolutional neural network will be briefly described.

FIG. 1 is a diagram schematically showing a general functional configuration of a convolutional neural network.

Currently, neural networks with various configurations have been proposed, but these basic configurations are common. The basic configuration of a neural network is represented by a superposition (or graph structure) of a plurality of types of layers. The neural network learns the model parameters so that the output result for the input data is an appropriate value. In other words, the neural network learns the model parameters to minimize the loss function defined so that the output result for the input data has an appropriate value.

In FIG. 1, as a feedforward neural network, it is composed of three layers, an input layer, a hidden layer, and an output layer, from the input layer to the output layer. Propagate in one direction. The hidden layer can be composed of a plurality of layers in a graph shape. Each layer has multiple units (neurons). In each layer, the parameter of the function that connects the unit in the front layer to the unit in the rear layer is called "weight". Learning in the present specification is to calculate an appropriate "weight" as a parameter of this function.

FIG. 1 illustrates a convolutional neural network. A convolutional neural network is composed of an input layer, a convolutional layer, a pooling layer, a full-connected layer, and an output layer. In FIG. 1, the first layer L1 is an input layer, and the second layer L2 is a convolution layer. Similarly, the m-th layer Lm is the output layer. In a convolutional neural network, only specific units in the anterior layer are connected to units in the posterior layer. That is, in the convolutional neural network, not all the units in the front layer are connected to the units in the rear layer.

Learning in the neural network according to the embodiment means that the weight of each layer is optimally updated by using the error between the output value from the output layer and the label of the training data with respect to the training data. A "loss function" is defined to calculate the error. The error is propagated one after another from the output layer side to the input layer side by the "error backpropagation method", and the weight of each layer is updated little by little. Finally, a convergence calculation is performed that adjusts the weights of each layer to the appropriate values so that the error is small. Specifically, in learning in a neural network (that is, in the generation stage of a new model parameter), the model parameter is updated by the gradient in which the error is back-propagated.

FIG. 2 is a diagram for explaining the convolutional relationship between the input data and the feature map. The process in FIG. 2 is performed by the convolution layer and the fully connected layer. In the example shown in FIG. 2, a feature map is generated by applying one weight filter to the input data. In FIG. 2, the sizes of the input data, the weight filter, and the feature map are as follows.
Input data: 32 x 32 x 3 elements Weight filter: 5 x 5 x 3 elements (model parameters)
Feature map: 28 x 28 elements

N weight filters (N is an integer of 1 or more) are prepared, and these are model parameters. That is, "weight" means N weight filters. However, the bias term is not considered here.

FIG. 3 is a diagram for explaining the generation of the feature map using the weight filter.
In the example shown in FIG. 3, one weight filter composed of 5 × 5 × 3 elements is applied to the input data, and the sum of the products of the elements is used as the value of one element of the feature map. Then, by moving the same weight filter with respect to the input data, one feature map is generated. Here, the number of elements (movement amount) for moving the weight filter is referred to as "stride". A zero-padding area filled with element 0 is provided at the peripheral edge of the input data. As a result, the same number of weight filters can be applied to the edge elements of the input data.

FIG. 4 is a diagram for explaining the relationship between the N weight filters and the N-stage feature map. In the example shown in FIG. 4, the number of weight filters is N. 2 and 3 show an example in which one feature map generated by one weight filter is generated. On the other hand, the example shown in FIG. 4 shows an example in which an N-stage feature map is generated by N weight filters. In neural network learning, the feature map in one layer becomes the input data in the next layer. By executing the training of the neural network, the error based on the loss function is propagated one after another from the output layer side to the input layer side, and the weight of each layer is updated by the known backpropagation method.

<Outline of the embodiment>
On the premise of the above, the outline of the embodiment will be described.
The information processing device according to the embodiment is a device for generating a learning model for achieving a target task by using a neural network and at the same time embedding an electronic watermark in the learning model. Further, the information processing device according to the embodiment is a device that can indicate that the device that detects the electronic watermark is not created later.

The information processing device according to the embodiment uses a processor such as a CPU or GPU, a work memory such as a DRAM (Dynamic Random Access Memory), and a large-capacity storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). It is composed. The information processing device 1 may be a single device such as a PC (Personal Computer), a workstation, or a server, or may be composed of a plurality of devices such as a cloud server.

FIG. 5 is a schematic diagram for explaining an outline of the learning process executed by the information processing apparatus according to the embodiment. The information processing apparatus according to the embodiment is the first neural network N1 having a plurality of layers L based on the training data including the input data D for learning the target task and the first label T1, similarly to the conventional neural network. The weight of each layer L of is updated. For example, when there are a plurality of target tasks such as "cat detection", "mountain detection", and "automobile detection", the input data D for each target task and the first label T1 different for each input data D are assigned. The assigned data set becomes training data.

Further, the information processing apparatus according to the embodiment also includes a second neural network N2 having one or more layers L selected from the layers L constituting the first neural network as input data. The second neural network N2 is a neural network for embedding a watermark vector in the layer L selected from the layers L constituting the first neural network. Note that FIG. 5 shows an example in which the second layer L2 in the first neural network N1 is selected as the weight update target in the second neural network N2.

Although the details will be described later, the information processing apparatus according to the embodiment generates the weight vector W based on the layer L selected from the first neural network N1. The information processing apparatus according to the embodiment uses the output of the fully connected layer with respect to the generated weight vector W as the second output O2, which is the output of the second neural network N2. That is, the second output O2 corresponds to a vector generated by multiplying the weight vector W by the “watermark detection matrix X”, the bias value is added, and the activation function is applied. Here, the description is made on the assumption that the second neural network N2 is a single-layer perceptron, but it can be applied to a multi-layer perceptron and other networks as long as a plurality of watermark detection matrices are prepared.

The information processing apparatus according to the embodiment updates the weight vector W based on the value obtained by applying the activation function s to the second output O2 and the second error E2 with the second label T2. When the bias value is 0, the information processing apparatus sets the value S (XW) obtained by multiplying the weight vector W by the watermark detection matrix X and applying the activation function s to the vector generated so that the second label T2 is obtained. For learning, the second label T2 becomes the watermark vector to be embedded. Therefore, in the present specification, the "second label T2" and the "watermark vector to be embedded" shall indicate the same vector. Hereinafter, the watermark vector to be embedded is also described as “watermark vector T2” using the same code as the second label T2. The detected watermark vector is represented by D2.

Here, in the information processing apparatus according to the embodiment, the watermark detection matrix X and the second label T2 are fixed before the start of learning of the first neural network. Further, the watermark detection matrix X and the second label T2 are not independent of each other, and the second label T2 is a hash value H (X) generated by inputting the watermark detection matrix X into a predetermined hash function H. ). The information processing apparatus according to the embodiment can extract the watermark vector D2 embedded in the model parameter by multiplying the weight vector extracted from the model parameter by the watermark detection matrix and performing the threshold processing at the time of watermark detection. The information processing apparatus according to the embodiment is based on the fact that the watermark vector D2 and the hash value H (X) of the watermark detection matrix are equal to each other, which is the basis for not being a watermark or a watermark detection matrix forged after learning.

The information processing apparatus according to the embodiment has an error between the first output O1 which is the output of the mth layer Lm which is the final layer of the first neural network N1 and the first label T1 as in the conventional neural network. Basically, the weight of each layer L is updated based on a certain first error E1. However, the information processing apparatus according to the embodiment updates the weights of the second layer L2 selected as the weights constituting the second neural network N2 based on the first error E1 and the second error E2. That is, with respect to the second layer L2 selected as the weight constituting the second neural network N2, the information processing apparatus according to the embodiment requires that the vector obtained by multiplying the watermark detection matrix X becomes the watermark vector. As a result, the weight will be updated.

As a result, the information processing apparatus according to the embodiment can simultaneously realize learning for the target task and learning for watermark embedding. Further, since the watermark detection matrix X and the second label T2 which is the watermark vector are connected by a hash function, it is desired to multiply the weight vector by some matrix after the learning is completed and the weight vector is fixed. It becomes difficult to generate the watermark vector T2 _{atk of.} As a result, it is possible to prevent a third party from forging the watermark detection matrix X and the watermark vector in order to claim their ownership after the fact.

A person who has generated a model parameter using the information processing apparatus according to the embodiment can extract the watermark vector embedded in the model parameter by multiplying the weight vector extracted from the model parameter by the watermark detection matrix and performing threshold processing. Can be done. At the time of detection, the information processing apparatus according to the embodiment can extract the watermark vector D2 embedded in the model parameter by multiplying the weight vector extracted from the model parameter by the watermark detection matrix and performing the threshold processing. If the watermark vector D2 and the hash value H (X) of the watermark detection matrix are equal to each other, the information processing apparatus according to the embodiment can indicate that the watermark or the watermark detection matrix is not forged after learning. As described above, the information processing apparatus according to the embodiment can provide a learning model capable of identifying the source and can show that it is not a fake watermark.

<Functional configuration of the information processing device according to the embodiment>
Hereinafter, the information processing apparatus according to the embodiment will be described in more detail.
FIG. 6 is a diagram schematically showing a functional configuration of the information processing device 1 according to the embodiment. The information processing device 1 includes a storage unit 10 and a control unit 11. The control unit 11 includes a weight update unit 110, a weight vector generation unit 111, a matrix generation unit 112, a watermark vector generation unit 113, and a watermark detection unit 114.

The storage unit 10 includes a ROM (Read Only Memory) for storing the BIOS (Basic Input Output System) of the computer that realizes the information processing device 1, a RAM (Random Access Memory) that serves as a work area for the information processing device 1, and an OS (OS). An operating system), an application program, and a large-capacity storage device such as an HDD or SSD that stores various information referred to when the application program is executed.

The control unit 11 is a processor such as a CPU or GPU of the information processing device 1, and by executing a program stored in the storage unit 10, the weight update unit 110, the weight vector generation unit 111, the matrix generation unit 112, and the watermark It functions as a vector generator 113.

The weight update unit 110 reads out the training data including the input data D for learning the target task from the storage unit 10 and acquires it. The weight updating unit 110 updates the weight of each layer L of the first neural network N1 including the plurality of layers L based on the acquired training data.

The weight vector generation unit 111 selects one or more layers L from the layers L constituting the first neural network N1. The weight vector generation unit 111 generates a weight vector W having the weight of the selected layer L as an element.

The matrix generation unit 112 generates a watermark detection matrix X for detecting the watermark vector T2 to be embedded in the first neural network N1. Although the details will be described later, the watermark detection matrix X is also used for embedding the watermark vector T2 in the watermark detection matrix X itself. In this sense, the watermark detection matrix X can be said to be a "watermark embedding and watermark detection matrix", but hereinafter, it is simply referred to as a "watermark detection matrix X".

When the second neural network N2 is a fully connected single-layer perceptron, the matrix generator 112 has a column length of P, where P is the length of the weight vector W and Q is the length of the watermark vector T2. A matrix having a row length of Q is generated as a watermark detection matrix X. The matrix generation unit 112 generates the watermark detection matrix X only once before the weight update unit 110 learns the target task. After that, while the weight update unit 110 is learning the target task, the watermark detection matrix X is fixed and is not changed.

The watermark vector generation unit 113 generates a hash value obtained by inputting the components of the watermark detection matrix X into a predetermined hash function as the watermark vector T2. Therefore, the length Q of the watermark vector T2 is equal to the length of the vector, which is the hash value output by the hash function, and the length depends on the hash function.

The weight update unit 110 multiplies the weight vector W by the watermark detection matrix X, and applies the activation function s to the first neural network on the condition that the error between the vector O2 and the watermark vector T2 is reduced. The weight of each layer L of N1 is updated. As a result, when the learning of the first neural network N1 is completed, the watermark detection unit 114 multiplies the weight vector W generated from the neural network for which the watermark is detected by the watermark detection matrix X, and binarizes the watermark vector. D2 can be detected. If the watermark is embedded correctly, D2 = T2. As a result, the information processing device 1 can provide a learning model that can identify the source with a watermark that is hard to be spoofed. The watermark detection unit 114 applies, for example, an activation function s to the data obtained by multiplying the weight vector W generated from the neural network for which the watermark is detected by the watermark detection matrix X. Binarize.

The watermark vector generation unit 113 may create a "watermark vector T2" from the hash value of the watermark detection matrix X and the first additional information I1. For example, the exclusive OR of the hash value of the watermark detection matrix X and the first additional information I1 may be the watermark vector T2. The first additional information I1 may pad 0 or use the same information repeatedly according to the length Q of the watermark vector T2. In this case, the watermark detection unit 114 can extract the right-related information by calculating the exclusive OR of the hash value of the watermark detection matrix X and the detected watermark vector D2, and the watermark detection matrix X can be used. It is possible to show at the same time that it is not a spoofed matrix later and rights-related information.

7 (a)-(c) are diagrams for explaining the weight setting process by the weight vector generation unit 111 according to the embodiment. Specifically, FIGS. 7A to 7C are diagrams showing a process in which the weight vector generation unit 111 generates the weight vector W based on the weight filter composed of 5 × 5 × 3 elements. ..

FIG. 7A is a diagram schematically showing a weight filter F composed of 5 × 5 × 3 elements. The weight vector generation unit 111 first converts a weight filter consisting of 5 × 5 × 3 elements into three two-dimensional weight filters (F1, F2, and F3) consisting of 25 elements in total of 5 in the vertical direction and 5 in the horizontal direction. To divide. FIG. 7B is a diagram schematically showing a first weight filter F1, a second weight filter F2, and a third weight filter F3 obtained by dividing a weight filter composed of 5 × 5 × 3 elements. be.

Subsequently, the weight vector generation unit 111 uses the first weight filter F1, the second weight filter F2, and the third weight filter F3 as three column vectors (V1, V2, etc.) composed of 5 × 5 = 25 elements, respectively. And V3). FIG. 7C shows a first column vector V1, a second column vector V2, and a third column vector V3 obtained by expanding each of the first weight filter F1, the second weight filter F2, and the third weight filter F3. Is a diagram schematically showing.

Finally, the weight vector generation unit 111 connects the first column vector V1, the second column vector V2, and the third column vector V3 in order to generate one column vector V composed of 25 × 3 = 75 elements. do. Here, as described with reference to FIG. 4, when the layer L selected by the weight vector generation unit 111 includes N weight filters N, the weight vector generation unit 111 similarly converts each filter into a column vector. After expanding and connecting them in order, a new column vector consisting of 75 × N elements is generated. The weight vector generation unit 111 uses the generated column vector as the weight vector W. In this case, the length P of the weight vector is P = 75 × N.

The watermark detection matrix X can also be vectorized by the same method as described with reference to FIG. 7 (c). Here, assuming that the hash function is H and the vector obtained by using the vectorized watermark detection matrix X as an argument of the hash function is H (X), the watermark vector T2 can be expressed by the following equation (1).
T2 = H (X) (1)

Further, when the weight vector W is multiplied by the watermark detection matrix X and the activation function s is applied, the second output O2 is obtained, and when the threshold value processing is performed, the detected watermark vector D2 is obtained. If it can be detected correctly, the watermark vector T2 is obtained, so that the following equations (2) and (3) are established.
s (XW) = O2 (2)
th (O2) = T2 (3)
Here, th () indicates threshold processing.

Equation (4) is obtained from equations (1), (2), and (3).
th (s (XW)) = H (X) (4)
In many cases, the application of the activation function s can be omitted at the time of detection.

From equation (4), the weight vector W is constrained by the watermark detection matrix X.

In the example of FIG. 5, the weight updating unit 110 updates the weights of the selected layer L and other layers under the constraint condition that the value of the second layer L2 becomes the weight vector W satisfying the equation (4). Means. In general, a multi-layer neural network has a lot of redundancy, and even if the degrees of freedom of the weights of some layers L are constrained, it is considered that the training data can be learned by the weights of other layers L. ..

Since the watermark detection matrix X is fixed if it is not spoofed, the degree of freedom in determining the weight vector W that satisfies the equation (4) is small.

Here, consider a situation in which a third party who illegally obtains the model parameters _{tries to detect the watermark vector T2 atk for spoofing from the model parameters.}

First, consider the first case. If you want to detect attacks watermark vector T2 _atk, the weight vector W obtained from the first neural network N1 as input, obtains by learning detector matrix X _atk fake was output attack watermark vector T2 _atk. Third party, when the the attack watermark vector _{T2 atk} the weight vector W is _{fixed, th (s (X atk ·} W)) = T2 atk or _{th (X atk · W) =} T2 atk satisfy _{X atk} Will be easy to find. Further, at this time, since the false detector matrix X _atk is changed without modifying the weight vector W such as re-learning, the correct answer rate of the first task does not decrease. However, it would be difficult to embed H (X _atk ) obtained from the fake detector matrix X _{atk as the watermark vector T2.} This is because when the false detector matrix X _atk is updated by learning, the watermark vector T2 = H (X _atk ) to be embedded becomes a different value, and the fake detector matrix X _atk can be obtained by learning. Because it is difficult. Moreover, the probability that a false detector matrix X _{atk satisfying T2 atk} = H (X _atk ) is obtained by chance from the characteristics of the hash function is extremely low.

Next, consider the second case. In the second case, after the third party fixes the _{X atk value satisfying th (s (X atk} · W)) = T2 _atk = H (X _atk ), _{the hash value of X atk} is calculated and the hash value is calculated. _{Suppose the} weight vector W is changed to embed the value T2 atk. In this case, since the element of the weight vector W is also the weight of the layer L of the neural network, it is considered that the correct answer rate of the neural network is lowered. In this case, the value of using the model parameters by a third party is also reduced.

From the above, it is possible to prevent a third party from detecting another watermark vector from the model parameters after the fact.

<Embedding and detection of the first additional information I1 in the watermark vector T2>
Subsequently, the method of embedding the first additional information I1 in the watermark vector T2 and the method of detecting the embedded first additional information I1 will be described.

8 (a)-(b) are diagrams for explaining a method of embedding the first additional information I1 in the watermark vector T2. Specifically, FIG. 8A is a diagram showing a first method of embedding the first additional information I1 in the watermark vector T2.

In the method shown in FIG. 8A, the watermark vector generation unit 113 divides the Q bits of the watermark vector T2 into a (a <Q) bits and Q-a bits, and assigns a hash value to the a bits. The first additional information I1 such as copyright is assigned to the Q-a bit. As a result, the first additional information I1 can be embedded in the watermark vector T2. In FIG. 8A, the portion of the watermark vector T2 indicated by a white circle is a bit to which a hash value is assigned, and the shaded circle is a bit to which the first additional information I1 is assigned.

FIG. 8B is a diagram showing a second method of embedding the first additional information I1 in the watermark vector T2. The second method is a method of embedding the first additional information I1 while effectively utilizing the bits of the watermark vector T2. Specifically, the watermark vector generation unit 113 does not use the hash value as it is as the watermark vector T2, but repeats or zero-pads the first additional information I1 so as to have the same number of digits as the hash value, and then the first Performs a predetermined calculation with the additional information of 1 and the hash value, and sets it as the watermark vector T2.

For example, a predetermined operation is an exclusive OR. In this case, the watermark vector generation unit 113 acquires the exclusive OR of the first additional information I1 and the hash value H (X), and adopts it as the watermark vector T2. At the time of detection, the watermark detection unit 114 multiplies the weight vector W by the watermark detection matrix X to perform threshold processing. The watermark detection unit 114 acquires an exclusive OR with the hash value of the watermark detection matrix X for the detection result D2 = th (s (X · W)) or th (X · W). As a result, the watermark detection unit 114 can also obtain the first additional information I1. In this case, the watermark detection unit 114 forges the first additional information I1 by obtaining the first additional information I1 from the detected watermark and at the same time decoding the first additional information I1 with the hash value of the watermark detection matrix X. You can also show that you haven't.

Further, since the hash value of the watermark detection matrix X and the first additional information I1 are embedded with a small number of embedded bits, the watermark bits can be effectively used as compared with the first method. In general, the watermark bit is limited, and the accuracy of the first task deteriorates as the number of embedded bits increases, so that the effective use of the bit is useful. The first additional information I1 often embeds the author information of the learning model, the source information such as the model provider (illegal outflow source), the date, the URL (Uniform Resource Locator), and the like, but is not limited to this.

<Embed the second additional information I2 in the watermark detection matrix>
Next, a method of embedding the second additional information I2 in the watermark detection matrix X or using the watermark detection matrix X itself as the second additional information I2 will be described.

If the watermark detection matrix X is a random value and has no meaning, even if it is a legitimate watermark detection matrix X, once it is leaked or made public, anyone can use the watermark detection matrix X to create a new first. 1 A watermark can be embedded in the neural network N1. That is, it is also possible to create a new first neural network model _Natk and a weight vector _{Watk by impersonating the legitimate owner of X.}

Therefore, the matrix generation unit 112 may include the second additional information I2 in the watermark detection matrix X. The matrix generation unit 112 may generate the watermark detection matrix X by embedding the second additional information I2 in the initial matrix, or may generate the second additional information I2 itself as the watermark detection matrix X. ..

Here, the second additional information I2 embedded in the watermark detection matrix X by the matrix generation unit 112 is information about parameters and programs in the range to which the watermark detection matrix X is to be applied. Specifically, the author name of the model parameter, a part of the URL scheduled to be released to the model, and the like are effective as the second additional information I2.

The matrix generation unit 112 may use the second additional information I2 such as a character string as the watermark detection matrix X by converting the second additional information I2 into a numerical type such as a double-precision floating-point number or a single-precision floating-point number as it is. When there is a problem in the randomness of the data obtained by converting the second additional information I2 into a numerical type such as a double-precision floating-point number or a single-precision floating-point number, the matrix generation unit 112 sets the weight of the watermark detection matrix X. The type-converted values may be applied to the bits at a fixed position of the value (high-order number bits, low-order number bits, etc.) in order, or the second additional information I2 is scrambled or randomized using a key by some method. May be done.

The embedding of the second additional information I2 in the watermark detection matrix X by the matrix generation unit 112 and the embedding of the first additional information I1 in the weight vector W by the weight vector generation unit 111 can be used together. In this case, the watermark detection unit 114 indicates that the trained model of the first neural network N1 is a legitimate model to be detected, and the detected first additional information I1 is not information that has been spoofed later. Can also be shown. As a result, the reliability of the watermark can be improved.

<Processing flow of information processing executed by the information processing device 1 according to the embodiment>
FIG. 9 is a flowchart for explaining the flow of information processing executed by the information processing apparatus 1 according to the embodiment. The process in this flowchart starts, for example, when the information processing device 1 is activated.

The matrix generation unit 112 generates a watermark detection matrix X for detecting the watermark vector T2 to be embedded in the first neural network N1 (S2). The watermark vector generation unit 113 generates a vector obtained by inputting the components of the watermark detection matrix X into the hash function as the watermark vector T2 (S4).

The weight update unit 110 reads out the target task learning data and the label from the storage unit 10 and acquires them (S6). The weight vector generation unit 111 selects a layer L to be used as an input layer of the second neural network N2 from the layers L constituting the first neural network N1 (S8). The weight vector generation unit 111 generates a weight vector W having the weight of the selected layer L as an element (S10).

The weight updating unit 110 updates the weights of each layer L of the first neural network N1 on the condition that the vector obtained by multiplying the weight vector W by the watermark detection matrix X becomes the watermark vector T2 (S12). When the weight updating unit 110 finishes updating the weight of each layer L of the first neural network N1, the process in this flowchart ends.

<Effects of the information processing device 1 according to the embodiment>
As described above, according to the information processing device 1 according to the embodiment, it is possible to provide a learning model capable of identifying the source and showing that it is not a spoofing.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist thereof. be. For example, the specific embodiment of the distribution / integration of the device is not limited to the above embodiment, and all or a part thereof may be functionally or physically distributed / integrated in any unit. Can be done. Also included in the embodiments of the present invention are new embodiments resulting from any combination of the plurality of embodiments. The effect of the new embodiment produced by the combination has the effect of the original embodiment together.

<Modification example>
In the above description, the case where the information processing apparatus 1 embeds the watermark in the convolutional neural network has been mainly described. However, those skilled in the art can easily understand that the present invention can be applied to neural networks other than convolutional neural networks.

1 ... Information processing device 10 ... Storage unit 11 ... Control unit 110 ... Weight update unit 111 ... Weight vector generation unit 112 ... Matrix generation unit 113 ... Watermark vector generation unit 114・ ・ ・ Watermark detection unit

Claims (8)

A weight update unit that updates the weight of each layer of a neural network having a plurality of layers based on training data including input data for objective task learning.
A weight vector generation unit that generates a weight vector having the weight of one or more layers selected from the layers constituting the neural network as an element, and a weight vector generation unit.
A matrix generator that generates a watermark detection matrix for detecting the watermark vector to be embedded in the neural network,
A watermark vector generation unit that generates a vector obtained by inputting a component of the watermark detection matrix into a predetermined hash function as a watermark vector is provided.
The weight updating unit is an information processing device that updates the weights of each layer of the neural network on the condition that the error between the vector obtained from the weight vector and the watermark detection matrix and the watermark vector is reduced as a constraint condition. The matrix generation unit generates the watermark detection matrix by embedding additional information about the neural network in the initial matrix.
The information processing device according to claim 1. The matrix generation unit generates additional information itself regarding the neural network as the watermark detection matrix.
The information processing device according to claim 1. The watermark vector generation unit is a vector obtained by the exclusive OR of the vector obtained by inputting the components of the watermark detection matrix into the predetermined hash function and the additional vector which is additional information about the neural network. Is set as the watermark vector,
The information processing device according to any one of claims 1 to 3. Watermark detection that detects the watermark vector by acquiring the exclusive OR of the data obtained by multiplying the weight vector generated from the detection target neural network by the watermark detection matrix and applying binarization to the additional vector. With more parts,
The information processing device according to claim 4. When the length of the weight vector is P and the length of the vector which is the hash value output by the hash function is Q, the matrix generator has a column length of P and a row length of Q. Generate a matrix as the watermark detection matrix,
The information processing device according to any one of claims 1 to 5. Information processing method executed by the processor
Objective A step of updating the weight of each layer of a neural network with multiple layers based on training data including data for learning tasks.
A step of generating a weight vector having the weight of one or more layers selected from the layers constituting the neural network as an element, and
A step of generating a watermark detection matrix for detecting a watermark embedded in the neural network, and
Including a step of inputting the components of the watermark detection matrix into a predetermined hash function and generating a vector obtained as a watermark vector.
In the step of updating the weight, the weight is updated with the constraint condition that the error between the weight vector, the vector obtained from the watermark detection matrix, and the watermark vector is reduced.
Information processing method. A program that enables a computer to realize information processing functions
Objective A function to update the weight of each layer of a neural network with multiple layers based on training data including data for task learning, and
A function to generate a weight vector having the weight of one or more layers selected from the layers constituting the neural network as an element, and
A function to generate a watermark detection matrix for detecting a watermark embedded in the neural network, and
The computer realizes a function of inputting the components of the watermark detection matrix into a predetermined hash function and generating a vector obtained as a watermark vector.
The function of updating the weight updates the weight of each layer of the neural network on the constraint condition that the error between the vector obtained from the weight vector and the watermark detection matrix and the watermark vector is reduced.
program.

Images