KR102654687B1 - Efficient Verifiable Convolutional Neural Netwrok Model - Google Patents

Abstract

An efficient integrity verification method for the operation results of CNN, a deep learning model based on convolution operation proposed in the present invention, is that the prover in the first verification step determines the result of a specific univariate function from the multivariate function related to the convolution operation. A step of transmitting a plurality of result values among the values to a verifier, where the verifier in the first verification step performs a connectivity check between the plurality of result values and the previous function using a convolution operation, and the verifier generates a new random value. Transmitting to the prover, the prover in the second verification step transmitting a plurality of result values of another single variable function to the verifier, the verifier in the second verification step using a convolution operation A connection check is performed between the plurality of result values and the previous function, the verifier transmits a new random value to the prover, and a verification formula is calculated in the second verification step. If the verification equation is satisfied, the verifier Includes the step of accepting the protocol.

Description

Efficient integrity verification technique for the calculation results of CNN, a deep learning model based on convolution calculation {Efficient Verifiable Convolutional Neural Netwrok Model}

The present invention relates to an efficient integrity verification technique for the calculation results of CNN, a deep learning model based on convolution calculation.

Deep learning technology, one of the technologies that has been attracting attention recently, is a technology that creates and applies a predictive model by learning from given sample data. The amount of computation of a deep learning model built using a large amount of sample data is considerable, making it difficult to perform on a PC with relatively low computing power. Therefore, deep learning models are used to perform consignment calculations through cloud computing by using servers with excellent computing capabilities rather than performing calculations directly. Companies that support this include Amazon and Microsoft. However, since none of the relevant services support integrity verification of calculation results, the only way to use the service is with the belief that the service provider performed the calculation honestly.

A solution to this problem is verifiable computation. A verifiable operation consists of a process in which the server performing the entrusted operation generates an integrity proof for the operation, and a process in which the user verifies the result of the operation through the operation result and proof received from the server. Among deep learning models, a technique called Safetynet is being studied to verify the integrity of the CNN (Convolution Neural Network) model, which is widely used in image recognition. However, this technique has the disadvantage of requiring a large amount of computation to generate a proof, making it difficult to apply it directly to actual applications.

Korean Patent Publication No. 10-2020-0093418 2020.08.05)

The technical task to be achieved by the present invention is to provide an integrity verification technique for convolution operation, one of the core operations of CNN model. The present invention seeks to reduce the proof generation cost for convolution, which is a problem in Safetynet, by utilizing the mathematical characteristics of convolution operations, and to provide an integrity verification technique that is the core of the CNN model that efficiently supports integrity verification.

In one aspect, an efficient integrity verification method for the calculation results of CNN, a deep learning model based on convolution operation proposed in the present invention, is performed when the prover in the first verification step specifies a multivariate function related to the convolution operation. A step of transmitting a plurality of result values among the result values of a single variable function to a verifier, where the verifier in the first verification step checks the connectivity of the plurality of result values with the previous function, and the verifier proves a new random value. transmitting to the verifier, the prover in the second verification step transmitting a plurality of result values of another single variable function to the verifier, the verifier in the second verification step transmitting the plurality of result values and the transfer It includes performing a connectivity check with the function, the verifier transmitting a new random value to the prover, calculating a verification equation in the second verification step, and accepting the protocol if the verification equation is satisfied. do.

In the first verification step, the verifier performs a connectivity check between the plurality of result values and the previous function, and the verifier transmits a new random value to the prover. After performing a connectivity check and repeating the process of the verifier transmitting a new random value to the prover, finally, the verification equation for the first verification step is calculated, and the process moves to the second verification step.

The verifier in the first verification step checks the connectivity between the plurality of result values and the previous function, and the verifier in the second verification step checks the connectivity between the plurality of result values and the previous function. A function is used that checks the relationship between the two vectors with respect to the previous function and outputs 1 if the condition is satisfied and 0 if the condition is not satisfied. The function can be expressed as Equation (5) for specific details for carrying out the invention.

Using the function of Equation (5) according to an embodiment of the present invention, the convolution operation applying the thumb-check protocol can be expressed as Equation (6) with specific details for carrying out the invention.

In the convolution operation verification technique applying the thumb-check protocol according to an embodiment of the present invention, the functions in the summation are products of multi linear polynomials, and are fully connected and thumb pooling. It can be applied to CNN verification techniques consisting of (Sum pooling) and Quadratic activation.

Through an efficient integrity verification technique for the operation results of CNN, a deep learning model based on convolution operation according to embodiments of the present invention, the proof generation cost for convolution, which is a problem in Safetynet, can be reduced through the mathematical method of convolution operation. It is possible to provide an integrity verification technique that is the core of a CNN model that utilizes characteristics to reduce and efficiently supports integrity verification.

1 is a diagram for explaining a naive access technique according to the prior art.
Figure 2 is a diagram for explaining an integrity verification technique based on a convolution operation according to an embodiment of the present invention.
Figure 3 is a flowchart illustrating an efficient integrity verification method for the operation results of CNN, a deep learning model based on convolution operation, according to an embodiment of the present invention.
Figure 4 is a diagram showing a convolution verification algorithm applying the thumb-check protocol according to an embodiment of the present invention.
Figure 5 is a diagram showing a verification protocol algorithm of a CNN consisting of convolution, fully connected, thumb pooling, and quadratic activation according to an embodiment of the present invention.

According to the prior art, the Sum-check protocol for matrix multiplication is used to verify the convolution operation in Safetynet. The core idea of Safetynet is as follows. dimension , the channel , the size of the batch is person The dimension in the second layer is As a result of the convolution operation on the weight matrix, in dimension channel having a dog The second layer appears, and this convolution equation is converted to a two-dimensional matrix multiplication operation and then verified. At this time A certain amount of prover computation is required.

Inefficiencies may occur in the process of converting such convolution relations into a two-dimensional matrix. Therefore, the present invention proposes a method that can verify convolution relations in a three-dimensional array without converting them to a two-dimensional matrix. According to an embodiment of the present invention, It can be confirmed that it is more efficient than Safetynet by requiring only the amount of prover calculations. Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

First, terms and symbols used in embodiments of the present invention will be explained.

(1) Mathematical structure/set

is the length represents a decimal number. is modulo represents a finite field.

bold symbol are each finite field vector represents.

two natural numbers About subscript Is from Represents a vector made up of consecutive integers. i.e. vector About is established.

Hereinafter, the log symbol that appears represents log with base 2.

Matrix The first row and column of It is expressed as , and the preceding index starts from 0. one by one until Indicates the th row, with subsequent indices starting from 0. one by one until Indicates the second column.

The CNN (Convolutional Neural Network) model is an artificial neural network that uses convolution operations and is a widely used model for image learning. In the present invention, it can be applied in the inference process of a CNN model, that is, in the process of attempting a classification task through the model after the model is determined through the training process. The CNN model is composed of functions that connect each layer, and when an input value is input, it passes through several layers through each function and then outputs the result. Functions used at this time include convolution, activation, pooling, and fully connected.

In the present invention, CNN The number of components for the width and height of the th layer It is expressed as and the number of channels is It is expressed as and the batch size is It will be expressed as In particular, the present invention was designed to use quadratic activation and sum pooling to verify functions other than convolution operations using only the sum-check protocol.

The convolution operation used in CNN according to the prior art is a two-dimensional convolution operation in which the weight matrix moves and performs an inner product. Matrix representing the previous layer , weight matrix , the resulting matrix is This satisfies the following equation:

Equation (1)

The thumb-check protocol is a protocol that verifies the result of the sum of a multivariate function. The thumb-check protocol is one of the building blocks used in computation verification techniques because the computation required for verification drops to the logarithm level of performing the computation directly. Thumb-check protocol is a multivariate function It is a protocol that verifies the integrity of the following operations:

Equation (2)

The thumb-check protocol can be applied to verify matrix products, and the matrix is converted into a multivariate polynomial through multi linear extension to proceed with the thumb-check protocol. At this time, the multivariate polynomial obtained by multiply linearly extending Matrix M is also It is written as a multivariate polynomial. is a function that receives an index vector expressed in binary form and returns an element corresponding to the index. That is, multivariate function about The value is the element of the 2nd row and 4th column of Matrix M. represents. Because multiple linear extensions to matrices guarantee uniqueness and are widely known natural transformations, polynomial and matrix forms are sometimes used interchangeably.

By default, matrices are displayed in uppercase letters. especially the matrix represents the square identity matrix and matrix satisfies the following: The matrix satisfies the following:

Equation (3)

1 is a diagram for explaining a naive access technique according to the prior art.

Naive approach according to the prior art is a method used in Safetynet. As mentioned above, layers consisting of a three-dimensional array are lined up in a row, and the convolution operation is expressed as the product of two matrices, and then the matrix product is performed. Use a thumb-check for . In the thumb-check for matrix multiplication, the operation that the prover must perform is proportional to the size of the matrix, so the size of the center weight matrix is It becomes proportional to At this time, since the size increases in the process of converting to a two-dimensional matrix, a technique that uses the thumb-check technique without going through the conversion is needed.

Figure 2 is a diagram for explaining an integrity verification technique based on a convolution operation according to an embodiment of the present invention.

In the present invention, the equation is re-expressed by introducing a vector representing the position without converting the layer. in other words Feature map of the th layer , weight matrix And obtained by convolution the second layer This satisfies the following equation:

Equation (4)

Here, the square plus sign means binary addition of two vectors. Since binary addition is not a linear operation, if the matrix is multi-linearly extended, there is a problem that the operation is not preserved, so another method must be used.

In order to solve this problem, the present invention introduces a new function that checks the relationship between two vectors and outputs 1 if the condition is satisfied, and 0 if not. The newly introduced functions are as follows:

Equation (5)

The convolution operation can be expressed using the above function as follows:

Equation (6)

For the above equation, the sum-check protocol can be applied because the functions in the summation are products of multi linear polynomials. here am.

Figure 3 is a flowchart illustrating an efficient integrity verification method for the operation results of CNN, a deep learning model based on convolution operation, according to an embodiment of the present invention.

The efficient integrity verification method for the operation results of CNN, a deep learning model based on the proposed convolution operation, includes a first verification step (310) and a second verification step (320).

First, in the first verification step 310, the prover in the first verification step transmits to the verifier a plurality of result values among the result values of a specific univariate function in the multivariate function related to the convolution operation (311), In the first verification step, the verifier checks the connectivity of the plurality of result values with the previous function, and the verifier transmits a new random value to the prover (312).

First, the input is the dimensions of the layer and weight matrix. and a public parameter and feature map matrix containing security parameters. , weighted matrix , randomly generated And the result of substituting the corresponding random value into a multivariate function that is a multi-linear extension of the matrix of the output layer. There is.

In step 311, the prover in the first verification step transmits to the verifier a plurality of result values among the result values of a specific univariate function in the multivariate function related to the convolution operation.

According to an embodiment of the present invention, a multivariate function It is a type of thumb-check method for , where the prover is a multivariate function related to the convolution operation. at A single variable function that leaves only the first variable 3 results transmit.

In step 320, the verifier in the first verification step checks the connectivity of the plurality of result values with the previous function, and the verifier transmits a new random value to the prover.

The verifier performs a connectivity check between the plurality of result values and the previous function, and the verifier selects a new random value. After repeating the process of transmitting to the prover, finally, the verification formula of the first verification step ( ) is calculated and moves to the second verification step.

In the second verification step 320, the prover in the second verification step transmits a plurality of result values of another one-variable function to the verifier (321), and the verifier in the second verification step transmits the above results to the verifier. Checking the connectivity of a plurality of result values and the previous function, the verifier transmits a new random value to the prover (322), calculating the verification equation in the second verification step, and verifying if the verification equation is satisfied. It includes a step 323 in which the user accepts the protocol.

In step 321, the prover in the second verification step transmits a plurality of result values of another one-variable function to the verifier.

In step 322, the verifier in the second verification step checks the connectivity of the plurality of result values with the previous function, and the verifier transmits a new random value to the prover.

In step 323, the verification equation in the second verification step is calculated, and if the verification equation is satisfied, the verifier accepts the protocol.

The verifier in the first verification step checks the connectivity between the plurality of result values and the previous function, and the verifier in the second verification step checks the connectivity between the plurality of result values and the previous function. A function is used that checks the relationship between the two vectors with respect to the previous function and outputs 1 if the condition is satisfied, and 0 if the condition is not satisfied. The function can be expressed as Equation (5) above.

According to an embodiment of the present invention, a convolution operation applying the thumb-check protocol can be expressed as the following equation (6) using equation (5).

The convolution operation using the thumb-check protocol according to an embodiment of the present invention is fully connected and sum pooling because the functions in the summation are products of multi linear polynomials. It is possible to apply the thumb-check protocol for pooling and quadratic activation.

Figure 4 is a diagram showing a convolution algorithm applying the thumb-check protocol according to an embodiment of the present invention.

The convolution algorithm applying the thumb-check protocol to equation (6) is shown in Figure 4.

First, the inputs to the algorithm are the dimensions of the layer and weight matrix. and a public parameter and feature map matrix containing security parameters. , weighted matrix , randomly generated And the result of substituting the corresponding random value into a multivariate function that is a multi-linear extension of the matrix of the output layer. There is.

The first step is a multivariate function It is a type of thumb-check method for which the prover is a multivariate function. at A single variable function that leaves only the first variable 3 results is sent (lines 1 and 4), the verifier checks the connectivity of the values with the previous function and generates a new random value. is sent to the prover and this process is repeated (lines 2 and 5). After repeating the process, the verifier After calculating (line 6), enter the second step.

The second step is also a thumb-check protocol and proceeds similarly to the above method. When the prover sends the function value (lines 7 and 10), the verifier checks it and sends a random value (lines 8 and 11). Finally, the verifier checks the verification expression (line 12) and passes all verifications. The verifier accepts the protocol.

This protocol is a protocol that verifies the operation of the convolution layer, and based on this protocol, an operation verification protocol for the entire CNN model can be designed. By combining the inventor's convolution algorithm with the fully connected, sum pooling, and quadratic activation verification techniques used in Safetynet, a CNN consisting of Convolution - Quadratic activation - Sum pooling - fully connected The verification protocol is shown in Figure 5.

Figure 5 is a diagram showing a verification protocol algorithm of a CNN consisting of convolution, fully connected, thumb pooling, and quadratic activation according to an embodiment of the present invention.

The inputs to the algorithm are the dimensions of the layer and weight matrix. and classification class number , public parameters containing security parameters and the input layer matrix of the CNN to be verified. and output layer matrix , hidden layer matrix , , , convolutional weighted matrix , fully connected weighted matrix There is.

The thumb-check for pulley connection, thumb-check for thumb pulling, and thumb-check for quadratic activation in FIG. 5 apply the idea of Safetynet and each consists of a thumb-check protocol. Using the thumb-check protocol in order at each step, the prover sends three function values to the verifier, and the verifier checks the operational relationship and repeats the process of sending a random value to the prover, and finally verifies the final relationship. It proceeds as follows. Using the algorithm in Figure 5, it is possible to design a verification protocol even for more complex CNN models.

Currently, deep learning services using cloud computing are provided by Amazon, Microsoft, etc., but there are no cases of actual service due to technical problems and lack of awareness regarding integrity verification of computational results through cloud computing. However, computational integrity verification through cloud computing is a necessary technology in terms of trust in service providers and user rights.

In order for the present invention to be put into practical use, it is important to have efficiency, especially the low cost of generating a proof for the calculation result on the server. This is because if the cost of generating a proof is too large compared to performing the calculation requested by the user, the service provider will inevitably be reluctant to introduce the technology.

Therefore, the present invention has the advantage of being able to produce proofs that must be performed by the entrusted computation service provider for the CNN model more simply than existing techniques, thereby reducing the computational burden on the integrity entrusted computation service provider.

The device described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general-purpose or special-purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. It can be embodied in . Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

As described above, although the embodiments have been described with limited examples and drawings, various modifications and variations can be made by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (5)

Through a deep learning model to perform a convolution operation-based integrity verification method, the prover in the first verification stage transmits a plurality of result values of a specific univariate function to the verifier in a multivariate function related to the convolution operation. steps;
A verifier in a first verification step checks connectivity between the plurality of result values and a previous function through the deep learning model, and the verifier transmits a new random value to the prover;
A prover in a second verification step transmitting a plurality of result values of another one-variable function to a verifier through the deep learning model;
A verifier in a second verification step checks connectivity between the plurality of result values and a previous function through the deep learning model, and the verifier transmits a new random value to the prover; and
Calculating a verification equation in the second verification step through the deep learning model, and if the verification equation is satisfied, the verifier accepts the protocol.
Including,
The verifier in the first verification step checks the connectivity between the plurality of result values and the previous function, and the verifier in the second verification step checks the connectivity between the plurality of result values and the previous function. Use a function that checks the relationship between the two vectors with respect to the previous function and outputs 1 if the condition is satisfied and 0 if it is not satisfied, and the function is expressed as the following equation (c-1)
Equation (c-1)
Integrity verification method based on convolution operation. According to paragraph 1,
The step of the verifier in the first verification step checking the connectivity of the plurality of result values and the previous function through the deep learning model, and the verifier transmitting a new random value to the prover,
The verifier checks the connectivity of the plurality of result values with the previous function, repeats the process of the verifier transmitting a new random value to the prover, and finally calculates the verification equation of the first verification step, Moving to the second verification stage
Integrity verification method based on convolution operation. delete According to paragraph 1,
The convolution operation applying the thumb-check protocol using the above function is expressed as the following equation (c-2),
Equation (c-2)
here, The feature map of the th layer is , the weight matrix is and obtained by convolution operation. The second layer is When, satisfying the above convolution operation
Integrity verification method based on convolution operation. According to clause 4,
Integrity verification based on convolution operation applying the thumb-check protocol is,
The functions in Summation are products of multi linear polynomials and are applied to CNN verification techniques consisting of fully connected, sum pooling, and quadratic activation. possible
Integrity verification method based on convolution operation.

Images