CN115345307A - Secure convolution neural network reasoning method and system on ciphertext image - Google Patents

Secure convolution neural network reasoning method and system on ciphertext image Download PDF

Info

Publication number
CN115345307A
CN115345307A CN202211263823.4A CN202211263823A CN115345307A CN 115345307 A CN115345307 A CN 115345307A CN 202211263823 A CN202211263823 A CN 202211263823A CN 115345307 A CN115345307 A CN 115345307A
Authority
CN
China
Prior art keywords
ciphertext
image
random
neural network
relu
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211263823.4A
Other languages
Chinese (zh)
Other versions
CN115345307B (en
Inventor
王世晞
吴志刚
张亮
王勋
朱东海
马文瑞
张翠
李娇娇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Shiping Information & Technology Co ltd
Zhejiang Gongshang University
Original Assignee
Hangzhou Shiping Information & Technology Co ltd
Zhejiang Gongshang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Shiping Information & Technology Co ltd, Zhejiang Gongshang University filed Critical Hangzhou Shiping Information & Technology Co ltd
Priority to CN202211263823.4A priority Critical patent/CN115345307B/en
Publication of CN115345307A publication Critical patent/CN115345307A/en
Application granted granted Critical
Publication of CN115345307B publication Critical patent/CN115345307B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Facsimile Transmission Control (AREA)

Abstract

The invention discloses a safe convolution neural network reasoning method and a system on a ciphertext image, and designs a splitting method for splitting an input image; based on the calculation specificity of the convolutional neural network, distributing the inference calculation to three servers for execution, wherein two servers are main calculation nodes, and one server is a ReLU auxiliary calculation node; the calculation protocol ensures that the calculation results of the two main calculation nodes are added to be a real convolution neural network reasoning result. Compared with the prior art, the invention does not need any encryption means and has smaller time overhead. For a user, simple operations such as splitting and merging are only needed to be performed on an input image, and the operation is more convenient and efficient compared with encryption and decryption operations. Server-side pre-computation is avoided, which also reduces overhead. The inference method provided by the invention does not need to use any encrypted primitive, so that the time for inference calculation is shorter; and no cryptographic computing power is required for the user. Compared with the prior art, the invention has stronger practicability.

Description

Secure convolution neural network reasoning method and system on ciphertext image
Technical Field
The invention belongs to the field of image processing, and relates to a secure convolution neural network reasoning method and system on a ciphertext image.
Background
The deep convolutional neural network becomes one of the most active and effective machine learning models, has become a basic model in the research fields of image recognition, image understanding, image generation and the like, and is widely applied to medical treatment, security, social networks and the like. The deep convolutional neural network is a multi-layer computational model, which is composed of convolutional layers, pooling layers, batch normalization (BatchNorm) layers, modified linear unit (ReLU) activation function layers, and the like. In the inference stage, the ReLU activation function and the max pooling layer perform non-linear calculations, and the rest of the layers are linear calculations, according to the calculation type.
Cloud computing service is widely applied to production and life as a basic software information service mode, and a brand-new service mode is generated by combining a deep neural network and a cloud computing technology: deep learning as a service (DLaaS), taking a convolutional neural network as an example, the basic service steps are as follows: deploying a deep convolutional neural network model at the cloud end by a cloud service provider or an enterprise user; the personal user uploads the private image data to the cloud; the cloud service provider uses data of an individual user as input to calculate a deep convolutional neural network model; the cloud service obtains a calculation result and sends the result to the individual user.
For individual users, the above service model faces a potential privacy disclosure risk. After the private data are uploaded to the cloud end by the individual user, access control to the data is lost, and the cloud service provider can obtain complete original data which are not subjected to encryption and other processing. Against this background, it is therefore highly desirable to propose techniques that can protect the privacy of the private data of the user when using deep learning as a service.
Aiming at the problems, a safe multi-party computing technology in cryptography is mainly adopted to design a safe reasoning scheme at present, and basic primitives comprise homomorphic encryption and a garbled circuit. Homomorphic encryption is a novel cryptosystem supporting cryptographic operation, and can enable a calculation executive party to complete algebraic operation (such as addition, multiplication and the like) on the premise of not contacting real data. The garbled circuits can also perform the same function, generally in two steps: the circuit construction and circuit execution generally need symmetric encryption, inadvertent transmission and other technologies to assist in completing the calculation.
The most advanced technical scheme at present is Delphi, which is a safe neural network reasoning scheme of a mixed homomorphic encryption and garbled circuit technology. The scheme has two participants: the user and the server provide data, the server provides a model, and the user and the server cooperate to complete inference calculation. Wherein, the linear part of the convolutional neural network is safely calculated by adopting homomorphic encryption; the obfuscation circuit is used for carrying out safe calculation on the nonlinear part of the convolutional neural network, and in addition, in order to assist in completing the protocol, a server is required to generate a large number of multiplication triples off line. However, the above schemes are all based on encryption technology, so the calculation cost is huge, and the practical use is not facilitated.
The main disadvantages of the prior art are mainly the following three points:
the time consumption is calculated. As is well known, the encryption algorithm has huge time overhead, and the deep convolutional neural network is a computation intensive model, so that the encryption-based deep convolutional neural network security inference scheme is slower by multiple orders of magnitude compared with the common inference scheme.
The user needs to participate in the calculation. In order to complete a computing protocol, users are required to participate, such as some encryption and decryption operations, and certain computing resources are also required for such computing. Many users do not have installed cryptographic software or insufficient computing resources and are therefore limited in use.
The server needs to pre-process the computation. For the cryptographic multiplication in the scheme, the server needs to pre-compute the generated multiplication triple, which brings additional computation overhead.
Disclosure of Invention
The invention aims to solve the problems that the inference method in the prior art is large in calculated amount and users need to master the encryption calculation capacity, and provides a secure convolution neural network inference method and a secure convolution neural network inference system on a ciphertext image.
In order to achieve the purpose, the invention adopts the following technical scheme to realize the purpose:
in a first aspect, the present invention provides a secure convolutional neural network inference method on a ciphertext image, including the following steps:
the main computing node receives ciphertext image information processed by a user, and computes a linear layer of the deep convolutional neural network according to the ciphertext image information to obtain a linear layer result;
the main computing node sends the linear layer result to the ReLU auxiliary computing node so that the ReLU auxiliary computing node can compute the computation result of the nonlinear layer ReLU of the deep convolutional neural network according to the linear layer result;
and the main computing node receives the computation result of the nonlinear layer ReLU of the deep convolutional neural network and sends the computation result of the nonlinear layer ReLU to the user, so that the inference of the ciphertext image is realized.
Preferably, the ciphertext image information processing by the user comprises the following steps:
splitting an original image into a plurality of ciphertext sub-images, and randomly selecting a random number as a random seed;
and combining the plurality of ciphertext sub-images with the random seeds respectively to obtain a plurality of sets, namely ciphertext image information.
Preferably, the step of splitting the original image into a plurality of ciphertext sub-images is as follows:
step 1, calculating an original image
Figure 683569DEST_PATH_IMAGE001
Mean value of
Figure 402126DEST_PATH_IMAGE002
，
Figure 652979DEST_PATH_IMAGE003
Calculating the original image
Figure 290633DEST_PATH_IMAGE001
Variance of (2)
Figure 626937DEST_PATH_IMAGE004
，
Figure 883606DEST_PATH_IMAGE005
Wherein
Figure 988965DEST_PATH_IMAGE006
As an original image
Figure 52648DEST_PATH_IMAGE001
Each pixel in total
Figure 610668DEST_PATH_IMAGE007
A plurality of;
step 2, calculating a standardized preprocessed image
Figure 671028DEST_PATH_IMAGE008
；
Step 3, generating
Figure 630894DEST_PATH_IMAGE009
Sum original image
Figure 610351DEST_PATH_IMAGE010
Random matrix with same size
Figure 921247DEST_PATH_IMAGE011
The specific method comprises the following steps: original image
Figure 519718DEST_PATH_IMAGE010
Comprises
Figure 334091DEST_PATH_IMAGE007
Individual pixels, in turn, based on uniform distribution
Figure 500761DEST_PATH_IMAGE012
Generating
Figure 33373DEST_PATH_IMAGE007
The random pixels form a random matrix
Figure 169957DEST_PATH_IMAGE011
Wherein, in the process,
Figure 104415DEST_PATH_IMAGE013
；
step 4, selecting a proportion parameter
Figure 691254DEST_PATH_IMAGE014
；
Step 5, before calculation
Figure 445583DEST_PATH_IMAGE009
A ciphertext sub-image
Figure 651437DEST_PATH_IMAGE015
(ii) a First, the
Figure 440401DEST_PATH_IMAGE016
A ciphertext sub-image
Figure 181830DEST_PATH_IMAGE017
，
Figure 689035DEST_PATH_IMAGE016
The number of nodes is calculated for the master,
Figure 433000DEST_PATH_IMAGE018
。
preferably according to a uniform distribution, normal distribution or log positiveMethod for generating random matrix by state distribution
Figure 76471DEST_PATH_IMAGE019
。
Preferably, the linear layer results
Figure 5112DEST_PATH_IMAGE020
The following:
Figure 734034DEST_PATH_IMAGE021
wherein the content of the first and second substances,
Figure 281690DEST_PATH_IMAGE022
in order to be the weight of the model,
Figure 45247DEST_PATH_IMAGE023
in order to bias the model in a way that,
Figure 629943DEST_PATH_IMAGE024
is the ciphertext image information of the current layer.
Preferably, the result of the ReLU auxiliary computation node computing the ReLU of the non-linear layer of the deep convolutional neural network according to the result of the linear layer is as follows:
a. generating random disorder tables
Figure 111740DEST_PATH_IMAGE025
Simultaneously recording the random disorder table of each element in the original sequence
Figure 197507DEST_PATH_IMAGE025
Position in (2) to generate a reduced table
Figure 815571DEST_PATH_IMAGE026
；
b. Computing mask ciphertext subimages
Figure 820436DEST_PATH_IMAGE027
According to a disorder table
Figure 523949DEST_PATH_IMAGE025
Computing out-of-order ciphertext sub-images
Figure 413408DEST_PATH_IMAGE028
Out-of-order ciphertext subimages
Figure 885978DEST_PATH_IMAGE029
Sending the data to a ReLU auxiliary computing node; wherein the content of the first and second substances,
Figure 842170DEST_PATH_IMAGE030
，
Figure 970663DEST_PATH_IMAGE031
is composed of
Figure 460550DEST_PATH_IMAGE032
Each of the plurality of ciphertext sub-images,
Figure 115523DEST_PATH_IMAGE033
in order to be a random mask, the mask is,
Figure 665453DEST_PATH_IMAGE034
as mask ciphertext sub-image
Figure 15662DEST_PATH_IMAGE035
Each of the elements of (a) to (b),
Figure 309241DEST_PATH_IMAGE036
as ciphertext sub-image of mask
Figure 569452DEST_PATH_IMAGE037
To (1)
Figure 290283DEST_PATH_IMAGE038
The value of each of the elements is,
Figure 393368DEST_PATH_IMAGE039
is an out-of-order table
Figure 225058DEST_PATH_IMAGE040
To (1)
Figure 589043DEST_PATH_IMAGE041
A value of each element;
c. ReLU auxiliary computing node selects a random seed
Figure 480776DEST_PATH_IMAGE042
In uniform distribution
Figure 805578DEST_PATH_IMAGE043
Generating and
Figure 440959DEST_PATH_IMAGE032
a ciphertext sub-image
Figure 174297DEST_PATH_IMAGE044
Random matrix of the same size as random perturbation
Figure 236931DEST_PATH_IMAGE045
Each random disturbance
Figure 49029DEST_PATH_IMAGE045
The specific generation method comprises the following steps:
Figure 222522DEST_PATH_IMAGE032
a ciphertext sub-image
Figure 561099DEST_PATH_IMAGE044
Comprises
Figure 794634DEST_PATH_IMAGE046
Individual pixels, in turn, based on uniform distribution
Figure 94029DEST_PATH_IMAGE047
Generating
Figure 71212DEST_PATH_IMAGE007
The random pixels form random disturbance
Figure 280608DEST_PATH_IMAGE045
；
d. The ReLU auxiliary computing node receives the out-of-order ciphertext sub-images sent by each main computing node
Figure 685044DEST_PATH_IMAGE048
Calculating intermediate parameters
Figure 206155DEST_PATH_IMAGE049
；
e. If the intermediate parameter
Figure 987030DEST_PATH_IMAGE050
Before calculation
Figure 300199DEST_PATH_IMAGE009
The out-of-order ciphertext output is:
Figure 875537DEST_PATH_IMAGE051
calculating the first
Figure 883944DEST_PATH_IMAGE032
The out-of-order ciphertext output is:
Figure 202930DEST_PATH_IMAGE052
wherein, in the step (A),
Figure 614015DEST_PATH_IMAGE053
is as follows
Figure 360254DEST_PATH_IMAGE054
An out-of-order sub-image of the ciphertext,
Figure 121536DEST_PATH_IMAGE055
is as follows
Figure 978634DEST_PATH_IMAGE032
An out-of-order ciphertext sub-image; if the intermediate parameter
Figure 266396DEST_PATH_IMAGE056
Before calculation
Figure 183536DEST_PATH_IMAGE009
The out-of-order ciphertext output is:
Figure 166536DEST_PATH_IMAGE057
calculating the first
Figure 827324DEST_PATH_IMAGE032
The out-of-order ciphertext output is:
Figure 720325DEST_PATH_IMAGE058
will be calculated
Figure 808367DEST_PATH_IMAGE032
Out-of-order ciphertext output
Figure 278662DEST_PATH_IMAGE059
Respectively sending the data to each main computing node;
f. then according to the reduction table
Figure 743142DEST_PATH_IMAGE026
Calculating to obtain a mask cipher text output
Figure 739917DEST_PATH_IMAGE060
Wherein, in the step (A),
Figure 998860DEST_PATH_IMAGE061
is composed of
Figure 956451DEST_PATH_IMAGE032
Out-of-order ciphertext output
Figure 224622DEST_PATH_IMAGE062
Each of the elements of (a) to (b),
Figure 325170DEST_PATH_IMAGE063
is composed of
Figure 755015DEST_PATH_IMAGE064
Out-of-order ciphertext output
Figure 199903DEST_PATH_IMAGE065
To (1)
Figure 6185DEST_PATH_IMAGE066
The value of each of the elements is,
Figure 711972DEST_PATH_IMAGE066
to restore the watch
Figure 578297DEST_PATH_IMAGE026
To
Figure 244902DEST_PATH_IMAGE061
A value of each element; computing a ciphertext output result based on the mask ciphertext output
Figure 854875DEST_PATH_IMAGE067
Completing the calculation of nonlinear layer ReLU to obtain the cipher text output result
Figure 165902DEST_PATH_IMAGE068
(ii) a Wherein the true inference result
Figure 203128DEST_PATH_IMAGE069
。
Preferably, a random out-of-order table is generated
Figure 357029DEST_PATH_IMAGE025
The steps are as follows:
each master computing node using the same random number
Figure 505113DEST_PATH_IMAGE070
As random seeds, according to uniform distribution
Figure 185493DEST_PATH_IMAGE071
Generating and
Figure 393621DEST_PATH_IMAGE016
a ciphertext sub-image
Figure 34818DEST_PATH_IMAGE072
Using random matrix with same size as random mask
Figure 986593DEST_PATH_IMAGE073
(ii) a Then the sequences are combined
Figure 770747DEST_PATH_IMAGE074
Random disorder, generating random disorder table
Figure 149776DEST_PATH_IMAGE025
(ii) a Wherein the content of the first and second substances,
Figure 278269DEST_PATH_IMAGE075
is composed of
Figure 96052DEST_PATH_IMAGE016
A ciphertext sub-image
Figure 688708DEST_PATH_IMAGE076
The number of the elements in the Chinese character 'Zhongqin'.
In a second aspect, the present invention provides a secure convolutional neural network inference system on a ciphertext image, comprising:
the linear result acquisition module is used for receiving ciphertext image information processed by a user through the main computing node, and calculating a linear layer of the deep convolutional neural network according to the ciphertext image information to obtain a linear layer result;
the nonlinear result acquisition module is used for sending the linear layer result to the ReLU auxiliary computing node by the main computing node so that the ReLU auxiliary computing node can compute the computation result of the nonlinear layer ReLU of the deep convolutional neural network according to the linear layer result;
and the ciphertext image inference module is used for receiving the calculation result of the nonlinear layer ReLU of the deep convolutional neural network by the main calculation node and sending the calculation result of the nonlinear layer ReLU to a user to realize the inference of the ciphertext image.
In a third aspect, the present invention provides a computer device comprising a memory storing a computer program and a processor implementing the steps of the secure convolutional neural network inference method on a ciphertext image when executing the computer program.
In a fourth aspect, the present invention provides a computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of a secure convolutional neural network inference method on a ciphertext image.
Compared with the prior art, the invention has the following beneficial effects:
compared with the prior art, the safe convolutional neural network reasoning method on the ciphertext image, provided by the invention, has the advantages that the main computing node receives ciphertext image information processed by a user, calculates the linear layer of the deep convolutional neural network, and sends the linear layer result to the ReLU auxiliary computing node so that the ReLU auxiliary computing node can calculate the ReLU of the nonlinear layer according to the linear layer result. For a user, the input image is only needed to be simply processed, and compared with encryption and decryption operations, the method is more convenient and efficient, pre-calculation of a main calculation node is avoided, and the total cost is reduced. The inference method does not need to use any encryption primitive, is shorter in inference calculation time and has no requirement on encryption calculation capacity of a user.
Furthermore, simple operations such as splitting and merging are carried out on the input image, and compared with encryption and decryption operations, the operation is more convenient and efficient.
The secure convolution neural network reasoning system on the ciphertext image, provided by the invention, divides the system into a linear result acquisition module, a nonlinear result acquisition module and a ciphertext image reasoning module, adopts a modularization idea to enable the modules to be mutually independent, and is convenient for uniformly managing the modules.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention, and therefore should not be considered as limiting the scope, and those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
FIG. 1 is a flow chart of a secure convolutional neural network inference method on a ciphertext image.
Fig. 2 is a schematic diagram of the interaction flow between the user and the server in the security inference scheme of the present invention.
FIG. 3 is a diagram of a secure convolutional neural network inference system on a ciphertext image of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. are used for indicating the orientation or positional relationship based on the orientation or positional relationship shown in the drawings or the orientation or positional relationship which is usually arranged when the product of the present invention is used, the description is merely for convenience and simplicity, and the indication or suggestion that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and thus, cannot be understood as limiting the present invention. Furthermore, the terms "first," "second," and the like are used merely to distinguish one description from another, and are not to be construed as indicating or implying relative importance.
The invention is described in further detail below with reference to the accompanying drawings:
the invention provides a secure convolution neural network reasoning method on a ciphertext image, which comprises the following steps as shown in figure 1:
s1, a main computing node receives ciphertext image information processed by a user, and a linear layer of a deep convolutional neural network is computed according to the ciphertext image information to obtain a linear layer result;
the steps of the ciphertext image information processed by the user are as follows:
splitting an original image into a plurality of ciphertext sub-images, and randomly selecting a random number as a random seed;
and combining the plurality of ciphertext sub-images with the random seeds respectively to obtain a plurality of sets, namely ciphertext image information.
The steps of splitting the original image into a plurality of ciphertext sub-images are as follows:
step 1, calculating an original image
Figure 910741DEST_PATH_IMAGE010
Mean value of
Figure 588847DEST_PATH_IMAGE002
，
Figure 695475DEST_PATH_IMAGE077
Calculating the original image
Figure 142637DEST_PATH_IMAGE010
Variance of (2)
Figure 801151DEST_PATH_IMAGE004
，
Figure 700974DEST_PATH_IMAGE078
Wherein
Figure 594981DEST_PATH_IMAGE079
As an original image
Figure 162228DEST_PATH_IMAGE010
Each pixel in total
Figure 991644DEST_PATH_IMAGE007
A plurality of;
step 2, calculating a standardized preprocessed image
Figure 378763DEST_PATH_IMAGE008
；
Step 3, generating
Figure 591307DEST_PATH_IMAGE009
Sum original image
Figure 747482DEST_PATH_IMAGE080
Random matrix with same size
Figure 13379DEST_PATH_IMAGE019
The specific method comprises the following steps: original image
Figure 950110DEST_PATH_IMAGE080
Comprises
Figure 123603DEST_PATH_IMAGE007
Pixels, in turn, based on uniform distribution
Figure 337547DEST_PATH_IMAGE012
Generating
Figure 571082DEST_PATH_IMAGE007
The random pixels form a random matrix
Figure 745842DEST_PATH_IMAGE019
Wherein, in the process,
Figure 723026DEST_PATH_IMAGE013
；
step 4, selecting a proportion parameter
Figure 791476DEST_PATH_IMAGE014
；
Step 5, before calculation
Figure 195912DEST_PATH_IMAGE009
A ciphertext sub-image
Figure 107236DEST_PATH_IMAGE015
(ii) a First, the
Figure 888111DEST_PATH_IMAGE016
A ciphertext sub-image
Figure 811067DEST_PATH_IMAGE081
，
Figure 386405DEST_PATH_IMAGE016
The number of nodes is calculated for the master,
Figure 40152DEST_PATH_IMAGE018
。
generating random matrix according to uniform distribution, normal distribution or log-normal distribution method
Figure 359138DEST_PATH_IMAGE082
。
The linear layer results
Figure 402181DEST_PATH_IMAGE083
The following:
Figure 148420DEST_PATH_IMAGE084
wherein, the first and the second end of the pipe are connected with each other,
Figure 34336DEST_PATH_IMAGE022
in order to be the weight of the model,
Figure 891434DEST_PATH_IMAGE023
in order to bias the model in a way that,
Figure 54562DEST_PATH_IMAGE024
is the ciphertext image information of the current layer.
S2, the main computing node sends the linear layer result to the ReLU auxiliary computing node so that the ReLU auxiliary computing node can compute the computation result of the nonlinear layer ReLU of the deep convolutional neural network according to the linear layer result;
the ReLU auxiliary computing node computes the computation result of the nonlinear layer ReLU of the deep convolutional neural network according to the linear layer result as follows:
a. generating random disorder tables
Figure 50331DEST_PATH_IMAGE025
Simultaneously recording the random disorder table of each element in the original sequence
Figure 95647DEST_PATH_IMAGE025
Position in (2) to generate a reduction table
Figure 694119DEST_PATH_IMAGE026
；
b. Computing mask ciphertext sub-image
Figure 774070DEST_PATH_IMAGE027
According to a disorder table
Figure 924429DEST_PATH_IMAGE025
Computing out-of-order ciphertext sub-images
Figure 457041DEST_PATH_IMAGE028
Out-of-order ciphertext subimages
Figure 859204DEST_PATH_IMAGE029
Sending the data to a ReLU auxiliary computing node; wherein the content of the first and second substances,
Figure 793662DEST_PATH_IMAGE030
，
Figure 364189DEST_PATH_IMAGE031
is composed of
Figure 384098DEST_PATH_IMAGE032
Each of the plurality of ciphertext sub-images,
Figure 324372DEST_PATH_IMAGE033
in order to be a random mask, the mask is,
Figure 113337DEST_PATH_IMAGE034
as mask ciphertext sub-image
Figure 871077DEST_PATH_IMAGE035
Each of the elements of (a) to (b),
Figure 581544DEST_PATH_IMAGE036
as ciphertext sub-image of mask
Figure 122247DEST_PATH_IMAGE037
To
Figure 844346DEST_PATH_IMAGE038
The value of each of the elements is,
Figure 445092DEST_PATH_IMAGE039
is an out-of-order table
Figure 377276DEST_PATH_IMAGE040
To (1)
Figure 721669DEST_PATH_IMAGE041
A value of an element;
c. ReLU auxiliary computing node selects a random seed
Figure 547543DEST_PATH_IMAGE042
In uniform distribution
Figure 584769DEST_PATH_IMAGE043
Generating and
Figure 738670DEST_PATH_IMAGE032
a ciphertext sub-image
Figure 886755DEST_PATH_IMAGE044
Random matrix of the same size as random perturbation
Figure 816402DEST_PATH_IMAGE045
Each random disturbance
Figure 24530DEST_PATH_IMAGE045
The specific generation method comprises the following steps:
Figure 665726DEST_PATH_IMAGE032
a ciphertext sub-image
Figure 351923DEST_PATH_IMAGE044
Comprises
Figure 152389DEST_PATH_IMAGE046
Individual pixels, in turn, based on uniform distribution
Figure 531417DEST_PATH_IMAGE047
Generating
Figure 659910DEST_PATH_IMAGE007
The random pixels form random disturbance
Figure 149797DEST_PATH_IMAGE045
；
d. The ReLU auxiliary computing node receives the out-of-order ciphertext sub-images sent by each main computing node
Figure 289923DEST_PATH_IMAGE048
Calculating intermediate parameters
Figure 839853DEST_PATH_IMAGE085
；
e. If the intermediate parameter
Figure 455642DEST_PATH_IMAGE050
Then before calculation
Figure 77116DEST_PATH_IMAGE009
The out-of-order ciphertext output is:
Figure 524278DEST_PATH_IMAGE051
calculating the first
Figure 917213DEST_PATH_IMAGE032
The out-of-order ciphertext output is:
Figure 82615DEST_PATH_IMAGE052
wherein, in the process,
Figure 225890DEST_PATH_IMAGE053
is a first
Figure 793137DEST_PATH_IMAGE054
An out-of-order sub-image of the ciphertext,
Figure 356974DEST_PATH_IMAGE055
is a first
Figure 9672DEST_PATH_IMAGE032
An out-of-order ciphertext sub-image; if the intermediate parameter
Figure 707370DEST_PATH_IMAGE056
Before calculation
Figure 863544DEST_PATH_IMAGE009
The out-of-order ciphertext output is:
Figure 863861DEST_PATH_IMAGE057
calculating the first
Figure 738277DEST_PATH_IMAGE032
The out-of-order ciphertext output is:
Figure 990397DEST_PATH_IMAGE086
will be calculated
Figure 266658DEST_PATH_IMAGE032
Out-of-order ciphertext output
Figure 703456DEST_PATH_IMAGE059
Respectively sending the data to each main computing node;
f. then according to a reduction table
Figure 127484DEST_PATH_IMAGE026
Calculating to obtain a mask cipher text output
Figure 104667DEST_PATH_IMAGE087
Wherein, in the step (A),
Figure 173117DEST_PATH_IMAGE088
is composed of
Figure 577554DEST_PATH_IMAGE016
Out-of-order ciphertext output
Figure 333803DEST_PATH_IMAGE089
Each of the elements of (a) to (b),
Figure 317939DEST_PATH_IMAGE090
is composed of
Figure 37634DEST_PATH_IMAGE091
Out-of-order ciphertext output
Figure 81813DEST_PATH_IMAGE092
To
Figure 11592DEST_PATH_IMAGE066
The value of each of the elements is,
Figure 799419DEST_PATH_IMAGE066
to restore the watch
Figure 373620DEST_PATH_IMAGE026
To
Figure 588701DEST_PATH_IMAGE093
A value of an element; computing a ciphertext output result based on the mask ciphertext output
Figure 490929DEST_PATH_IMAGE094
Completing the calculation of the nonlinear layer ReLU to obtain the ciphertext output result
Figure 82447DEST_PATH_IMAGE095
(ii) a Wherein the true inference results
Figure 511154DEST_PATH_IMAGE096
。
Generating random disorder tables
Figure 756191DEST_PATH_IMAGE025
The steps are as follows:
each master computing node using the same random number
Figure 535928DEST_PATH_IMAGE097
As random seeds, according to uniform distribution
Figure 665558DEST_PATH_IMAGE098
Generating and
Figure 948772DEST_PATH_IMAGE016
a ciphertext sub-image
Figure 145136DEST_PATH_IMAGE072
Random matrix with same size as random mask
Figure 146590DEST_PATH_IMAGE099
(ii) a Then the sequence is processed
Figure 79911DEST_PATH_IMAGE074
Random disorder, generating random disorder table
Figure 217631DEST_PATH_IMAGE025
(ii) a Wherein the content of the first and second substances,
Figure 335629DEST_PATH_IMAGE075
is composed of
Figure 824379DEST_PATH_IMAGE016
A ciphertext sub-image
Figure 295811DEST_PATH_IMAGE076
The number of the elements in the Chinese character 'Zhongqin'.
And S3, the main computing node receives the calculation result of the nonlinear layer ReLU of the deep convolutional neural network and sends the calculation result of the nonlinear layer ReLU to the user, so that the reasoning of the ciphertext image is realized.
The invention provides a secure convolution neural network reasoning method on a ciphertext image, which is suitable for n main computing nodes (a, b and c)
Figure 553617DEST_PATH_IMAGE018
) The complete steps are as follows:
firstly, executing a data transmitting stage of a user, and comprising the following 3 steps:
1. user will private original image
Figure 593249DEST_PATH_IMAGE100
Is split into
Figure 303716DEST_PATH_IMAGE016
A ciphertext sub-image
Figure 578839DEST_PATH_IMAGE072
The resolution method comprises the following steps:
a. computing an original image
Figure 815786DEST_PATH_IMAGE100
Mean value of
Figure 885373DEST_PATH_IMAGE002
，
Figure 348715DEST_PATH_IMAGE101
Calculating the original image
Figure 161950DEST_PATH_IMAGE100
Variance of (2)
Figure 768250DEST_PATH_IMAGE004
，
Figure 8738DEST_PATH_IMAGE102
In which
Figure 959377DEST_PATH_IMAGE103
As an original image
Figure 576303DEST_PATH_IMAGE100
Each pixel of the pixel
Figure 787841DEST_PATH_IMAGE007
And (4) respectively.
b. Computing normalized pre-processed images
Figure 464810DEST_PATH_IMAGE008
。
c. Generating
Figure 637166DEST_PATH_IMAGE009
Sum original image
Figure 792204DEST_PATH_IMAGE100
Random matrix with same size
Figure 874560DEST_PATH_IMAGE082
The specific method comprises the following steps: original image
Figure 456851DEST_PATH_IMAGE100
Comprises
Figure 116503DEST_PATH_IMAGE007
Pixels, in turn, based on uniform distribution
Figure 934286DEST_PATH_IMAGE012
Generating
Figure 995783DEST_PATH_IMAGE007
The random pixels form a random matrix
Figure 14555DEST_PATH_IMAGE082
Wherein, in the step (A),
Figure 161502DEST_PATH_IMAGE013
(ii) a The random matrix may also be generated by replacing the uniform distribution in the above process with a normal distribution or a log-normal distribution
Figure 32244DEST_PATH_IMAGE082
As long as the random matrix is guaranteed
Figure 948247DEST_PATH_IMAGE082
Just as random.
d. The user selects a scale parameter
Figure 137920DEST_PATH_IMAGE014
。
e. Before calculation
Figure 772164DEST_PATH_IMAGE009
A ciphertext sub-image
Figure 197329DEST_PATH_IMAGE104
(ii) a First, the
Figure 967839DEST_PATH_IMAGE032
A ciphertext sub-image
Figure 328413DEST_PATH_IMAGE105
，
Figure 449953DEST_PATH_IMAGE018
。
2. The user selects a random number
Figure 163962DEST_PATH_IMAGE106
As a random seed for the master computing node.
3. User sends to each master computing node
Figure 54558DEST_PATH_IMAGE107
。
Secondly, executing a server computing stage, which comprises the following 3 steps:
1. each master computing node receives
Figure 586033DEST_PATH_IMAGE108
。
2. Each layer of the deep convolutional neural network is computed in turn. Wherein, for the linear layer of the deep convolutional neural network, the linear layer result executed by each main computing node
Figure 929290DEST_PATH_IMAGE109
The following:
Figure 696258DEST_PATH_IMAGE110
，
Figure 441360DEST_PATH_IMAGE111
in order to be the weight of the model,
Figure 409316DEST_PATH_IMAGE112
in order to bias the model, the bias of the model,
Figure 239868DEST_PATH_IMAGE024
is the ciphertext image information of the current layer,
Figure 534496DEST_PATH_IMAGE113
calculating the number of nodes for the master; for the nonlinear layer ReLU of the deep convolutional neural network, the computation performed by the ReLU-aided compute node is as follows:
a. each master computing node using the same random number
Figure 399683DEST_PATH_IMAGE114
As random seeds, according to uniform distribution
Figure 272961DEST_PATH_IMAGE115
Generating and
Figure 715444DEST_PATH_IMAGE032
a ciphertext sub-image
Figure 699581DEST_PATH_IMAGE072
Random matrix with same size as random mask
Figure 419275DEST_PATH_IMAGE099
(ii) a Then the sequences are combined
Figure 463454DEST_PATH_IMAGE074
Random disorder, generating random disorder table
Figure 143966DEST_PATH_IMAGE025
(ii) a Wherein the content of the first and second substances,
Figure 931793DEST_PATH_IMAGE116
is composed of
Figure 505994DEST_PATH_IMAGE032
A ciphertext sub-image
Figure 721074DEST_PATH_IMAGE076
The number of the elements in the original sequence is recorded, and the random disorder table of each element in the original sequence is recorded at the same time
Figure 606991DEST_PATH_IMAGE025
Position in (2) to generate a reduced table
Figure 198509DEST_PATH_IMAGE026
。
b. Computing mask ciphertext subimages
Figure 627217DEST_PATH_IMAGE117
According to a disorder table
Figure 387100DEST_PATH_IMAGE025
Computing out-of-order ciphertext sub-images
Figure 166837DEST_PATH_IMAGE028
Out-of-order ciphertext subimages
Figure 296467DEST_PATH_IMAGE029
Sending the data to a ReLU auxiliary computing node; wherein, the first and the second end of the pipe are connected with each other,
Figure 579681DEST_PATH_IMAGE030
，
Figure 526777DEST_PATH_IMAGE031
is composed of
Figure 528231DEST_PATH_IMAGE032
Each of the plurality of ciphertext sub-images,
Figure 195973DEST_PATH_IMAGE033
in order to be a random mask, the mask is,
Figure 599272DEST_PATH_IMAGE034
as mask ciphertext sub-image
Figure 468002DEST_PATH_IMAGE035
Each of the elements of (a) to (b),
Figure 691173DEST_PATH_IMAGE036
as mask ciphertext sub-image
Figure 428185DEST_PATH_IMAGE118
To
Figure 685991DEST_PATH_IMAGE038
The value of each of the elements is,
Figure 709311DEST_PATH_IMAGE039
is an out-of-order table
Figure 685357DEST_PATH_IMAGE040
To
Figure 694901DEST_PATH_IMAGE041
A value of each element;
c. ReLU auxiliary computing node selects a random seed
Figure 181115DEST_PATH_IMAGE042
In uniform distribution
Figure 250703DEST_PATH_IMAGE119
Generating and
Figure 448466DEST_PATH_IMAGE016
a ciphertext sub-image
Figure 651914DEST_PATH_IMAGE120
Random matrix of the same size as random disturbance
Figure 618733DEST_PATH_IMAGE045
Each random disturbance
Figure 859221DEST_PATH_IMAGE045
The specific generation method comprises the following steps:
Figure 419647DEST_PATH_IMAGE016
a ciphertext sub-image
Figure 36573DEST_PATH_IMAGE120
Comprises
Figure 123478DEST_PATH_IMAGE046
Individual pixels, in turn, based on uniform distribution
Figure 659501DEST_PATH_IMAGE119
Generating
Figure 831857DEST_PATH_IMAGE007
The random pixels form random disturbance
Figure 252474DEST_PATH_IMAGE045
；
d. The ReLU auxiliary computing node receives the out-of-order secret transmitted by each main computing nodeText image
Figure 193885DEST_PATH_IMAGE048
Calculating intermediate parameters
Figure 415656DEST_PATH_IMAGE121
；
e. If the intermediate parameter
Figure 75308DEST_PATH_IMAGE122
Before calculation
Figure 34037DEST_PATH_IMAGE009
The out-of-order ciphertext output is:
Figure 95534DEST_PATH_IMAGE123
calculating the first
Figure 238939DEST_PATH_IMAGE016
The out-of-order ciphertext output is:
Figure 854728DEST_PATH_IMAGE124
wherein, in the process,
Figure 617148DEST_PATH_IMAGE125
is as follows
Figure 408517DEST_PATH_IMAGE054
An out-of-order sub-image of the ciphertext,
Figure 598190DEST_PATH_IMAGE055
is as follows
Figure 232434DEST_PATH_IMAGE016
An out-of-order ciphertext sub-image; if the intermediate parameter
Figure 532965DEST_PATH_IMAGE126
Before calculation
Figure 428109DEST_PATH_IMAGE009
The out-of-order ciphertext output is:
Figure 788683DEST_PATH_IMAGE127
calculating the first
Figure 644644DEST_PATH_IMAGE016
The out-of-order ciphertext output is:
Figure 748866DEST_PATH_IMAGE128
will be calculated
Figure 7503DEST_PATH_IMAGE016
Out-of-order ciphertext output
Figure 538979DEST_PATH_IMAGE059
Respectively sending the information to each main computing node;
f. then according to the reduction table
Figure 882236DEST_PATH_IMAGE026
Calculating to obtain a mask cipher text output
Figure 524570DEST_PATH_IMAGE087
Wherein, in the step (A),
Figure 394305DEST_PATH_IMAGE061
is composed of
Figure 96682DEST_PATH_IMAGE016
Out-of-order ciphertext output
Figure 661656DEST_PATH_IMAGE129
Each of the elements of (a) to (b),
Figure 983047DEST_PATH_IMAGE130
is composed of
Figure 848235DEST_PATH_IMAGE064
Out-of-order ciphertext output
Figure 721513DEST_PATH_IMAGE092
To (1)
Figure 773782DEST_PATH_IMAGE066
The value of each of the elements is,
Figure 148132DEST_PATH_IMAGE066
to restore the watch
Figure 867826DEST_PATH_IMAGE026
To (1)
Figure 912006DEST_PATH_IMAGE061
A value of an element; computing a ciphertext output result based on the mask ciphertext output
Figure 451571DEST_PATH_IMAGE067
And completing the calculation of the nonlinear layer ReLU.
3. Each main computing node respectively obtains a ciphertext output result
Figure 613300DEST_PATH_IMAGE068
And the results are sent back to the user.
And (III) finally executing a user result reduction stage, which comprises the following 2 steps:
1. user reception
Figure 187501DEST_PATH_IMAGE032
A ciphertext output result
Figure 402582DEST_PATH_IMAGE068
。
2. Computing true inference results
Figure 695023DEST_PATH_IMAGE131
To obtain the real reasoning result
Figure 145596DEST_PATH_IMAGE132
。
The invention provides a secure convolution neural network reasoning method on a ciphertext image, which comprises two main computing nodes and comprises the following complete steps: four entities are involved: the safe convolutional neural network reasoning method comprises three stages of: the method comprises a user data sending stage, a server calculation stage and a user result restoring stage.
Firstly, executing a data sending stage of a user, and comprising the following 3 steps:
1. user will privately make the original image
Figure 839882DEST_PATH_IMAGE133
Split into ciphertext sub-images 1
Figure 225864DEST_PATH_IMAGE134
And ciphertext sub-image 2
Figure 349809DEST_PATH_IMAGE135
The resolution method comprises the following steps:
a. computing an original image
Figure 479439DEST_PATH_IMAGE133
Is recorded as the mean value of
Figure 28232DEST_PATH_IMAGE136
，
Figure 585115DEST_PATH_IMAGE101
Calculating the original image
Figure 711203DEST_PATH_IMAGE133
The variance of (A) is recorded as
Figure 644524DEST_PATH_IMAGE137
，
Figure 47824DEST_PATH_IMAGE102
Wherein
Figure 775608DEST_PATH_IMAGE103
As an original image
Figure 638260DEST_PATH_IMAGE100
Each pixel of the pixel
Figure 375272DEST_PATH_IMAGE007
And (4) respectively.
b. Computing normalized pre-processed images
Figure 633078DEST_PATH_IMAGE138
。
c. Generating a sum of original images in a uniform distribution
Figure 531763DEST_PATH_IMAGE139
Random matrix with same size
Figure 632443DEST_PATH_IMAGE140
The specific method comprises the following steps: original image
Figure 907567DEST_PATH_IMAGE100
Comprises
Figure 19880DEST_PATH_IMAGE046
Individual pixels, in turn, based on uniform distribution
Figure 355046DEST_PATH_IMAGE012
Generating
Figure 428175DEST_PATH_IMAGE007
Each random pixel forms a random matrix
Figure 506990DEST_PATH_IMAGE140
Alternatively, the random matrix may be generated according to a normal distribution or a lognormal distribution method
Figure 473809DEST_PATH_IMAGE140
As long as the random matrix is guaranteed
Figure 838931DEST_PATH_IMAGE140
Just as random.
d. User selection of a scale parameter
Figure 523990DEST_PATH_IMAGE141
。
e. Computing ciphertext sub-image 1
Figure 140916DEST_PATH_IMAGE142
Ciphertext subimage 2
Figure 227821DEST_PATH_IMAGE143
。
2. The user selects a random number
Figure 278691DEST_PATH_IMAGE144
As a random seed for the master computing node.
3. User sending to the host computing node 1
Figure 451047DEST_PATH_IMAGE145
(ii) a Sending to the master computing node 2
Figure 871664DEST_PATH_IMAGE146
。
Secondly, executing a server computing stage, which comprises the following 3 steps:
1. master computing node 1 receives
Figure 813075DEST_PATH_IMAGE145
The master computing node 2 receives
Figure 785579DEST_PATH_IMAGE147
。
2. Each layer of the deep convolutional neural network is computed in turn. Wherein, for the linear layer of the deep convolutional neural network, the linear layer result executed by each main computing node
Figure 445231DEST_PATH_IMAGE148
The following were used:
Figure 403959DEST_PATH_IMAGE149
，
Figure 465456DEST_PATH_IMAGE111
in order to be the weight of the model,
Figure 94015DEST_PATH_IMAGE112
in order to bias the model, the bias of the model,
Figure 240962DEST_PATH_IMAGE024
ciphertext image information of a current layer; for the nonlinear layer ReLU of the deep convolutional neural network, the computation performed by the ReLU-aided compute node is as follows:
a. each master computing node using the same random number
Figure 737803DEST_PATH_IMAGE150
As random seeds, first according to a uniform distribution
Figure 778440DEST_PATH_IMAGE151
Generating and ciphertext sub-images
Figure 968113DEST_PATH_IMAGE152
Using random matrix with same size as random mask
Figure 336777DEST_PATH_IMAGE153
(ii) a Then the sequences are combined
Figure 637309DEST_PATH_IMAGE154
Random disorder, generating random disorder table
Figure 53158DEST_PATH_IMAGE155
Simultaneously recording the random disorder table of each element in the original sequence
Figure 413733DEST_PATH_IMAGE155
Position in (2) to generate a reduction table
Figure 269693DEST_PATH_IMAGE156
(ii) a Wherein the content of the first and second substances,
Figure 373915DEST_PATH_IMAGE157
for ciphertext sub-images
Figure 123566DEST_PATH_IMAGE158
The number of the elements in the Chinese herbal medicine composition,
Figure 655041DEST_PATH_IMAGE159
。
b. computing mask ciphertext sub-image
Figure 998298DEST_PATH_IMAGE160
According to a disorder table
Figure 515998DEST_PATH_IMAGE155
Computing out-of-order ciphertext sub-images
Figure 261100DEST_PATH_IMAGE161
Out-of-order ciphertext subimages
Figure 963477DEST_PATH_IMAGE162
Sending the data to a ReLU auxiliary computing node; wherein the content of the first and second substances,
Figure 794030DEST_PATH_IMAGE163
in order to be a random mask, the mask is,
Figure 364688DEST_PATH_IMAGE164
as mask ciphertext sub-image
Figure 964297DEST_PATH_IMAGE165
Each of the elements of (a) to (b),
Figure 837575DEST_PATH_IMAGE166
as ciphertext sub-image of mask
Figure 155424DEST_PATH_IMAGE165
To (1)
Figure 779041DEST_PATH_IMAGE167
The value of each of the elements is,
Figure 233156DEST_PATH_IMAGE167
is an out-of-order table
Figure 277335DEST_PATH_IMAGE155
To
Figure 82480DEST_PATH_IMAGE061
A value of each element.
c. ReLU auxiliary computing node selects a random seed
Figure 994941DEST_PATH_IMAGE168
In uniform distribution
Figure 303563DEST_PATH_IMAGE169
Generating and ciphertext subimage
Figure 518644DEST_PATH_IMAGE170
Random matrix of the same size as random perturbation
Figure 686451DEST_PATH_IMAGE171
Random perturbation
Figure 12390DEST_PATH_IMAGE171
The specific generation method comprises the following steps: ciphertext subimage
Figure 441097DEST_PATH_IMAGE170
Comprises
Figure 92659DEST_PATH_IMAGE007
Individual pixels based on uniform distribution
Figure 731450DEST_PATH_IMAGE047
Generating
Figure 861080DEST_PATH_IMAGE007
The random pixels constitute random disturbance
Figure 409873DEST_PATH_IMAGE171
。
d. The ReLU auxiliary computing node receives the out-of-order ciphertext sub-images sent by the two main computing nodes
Figure 966757DEST_PATH_IMAGE172
Calculating intermediate parameters
Figure 76533DEST_PATH_IMAGE173
。
e. If the intermediate parameter
Figure 9854DEST_PATH_IMAGE174
Then calculate the out-of-order ciphertext output 1 as:
Figure 413153DEST_PATH_IMAGE175
the out-of-order ciphertext output 2 is:
Figure 265572DEST_PATH_IMAGE176
(ii) a If the intermediate parameter
Figure 754322DEST_PATH_IMAGE177
Then calculate the out-of-order ciphertext output 1 as:
Figure 225754DEST_PATH_IMAGE178
the out-of-order ciphertext output 2 is:
Figure 483561DEST_PATH_IMAGE179
. Outputting the two out-of-order ciphertexts obtained by calculation
Figure 257613DEST_PATH_IMAGE180
Respectively, to two master computing nodes, wherein,
Figure 233659DEST_PATH_IMAGE181
in order to scramble the ciphertext image 1,
Figure 243203DEST_PATH_IMAGE182
is the scrambled ciphertext image 2.
f. Then according to the reduction table
Figure 745729DEST_PATH_IMAGE183
Calculating to obtain a mask cipher text output
Figure 815316DEST_PATH_IMAGE184
Wherein
Figure 278658DEST_PATH_IMAGE034
Outputting for two out-of-order ciphertexts
Figure 91893DEST_PATH_IMAGE185
Each of the elements of (a) to (b),
Figure 698193DEST_PATH_IMAGE186
output for two out-of-order ciphertexts
Figure 204261DEST_PATH_IMAGE185
To
Figure 623741DEST_PATH_IMAGE066
The value of each of the elements is,
Figure 365301DEST_PATH_IMAGE066
to restore the watch
Figure 452205DEST_PATH_IMAGE026
To
Figure 863595DEST_PATH_IMAGE034
A value of each element; computing a ciphertext output result based on the mask ciphertext output
Figure 911317DEST_PATH_IMAGE187
And completing the calculation of the nonlinear layer ReLU.
3. Two main computing nodes respectively obtain ciphertext output results 1
Figure 800775DEST_PATH_IMAGE188
And ciphertext output result 2
Figure 7766DEST_PATH_IMAGE189
And sending the cipher text output result back to the user.
And (III) finally executing a user result reduction stage, which comprises the following 2 steps:
1. user receiving ciphertext output result 1
Figure 714691DEST_PATH_IMAGE188
And ciphertext output result 2
Figure 374342DEST_PATH_IMAGE189
。
2. Computing true inference results
Figure 333071DEST_PATH_IMAGE190
To obtain the real reasoning result
Figure 520468DEST_PATH_IMAGE191
。
As shown in Table 1, is based on an out-of-order table
Figure 539240DEST_PATH_IMAGE025
Out-of-order and restore input schematic.
TABLE 1 ciphertext image reasoning case
Figure 686187DEST_PATH_IMAGE192
As shown in fig. 2, which is a schematic flow of interaction between a user and a server in the security inference scheme of the present invention, for the case of two main computing nodes: smiley faces represent users and clouds represent cloud providers, requiring three servers to participate. First, the user will encrypt the subimage 1
Figure 448607DEST_PATH_IMAGE193
Ciphertext subimage 2
Figure 489244DEST_PATH_IMAGE194
And a random number
Figure 413338DEST_PATH_IMAGE195
One and sent to both servers (primary compute nodes); then, the two servers are at the third server (ReLU auxiliary computing node)) With the help of (2), the inference calculation of the deep convolutional neural network is completed according to the step (II), and ciphertext output results 1 are respectively obtained
Figure 47581DEST_PATH_IMAGE196
And ciphertext output result 2
Figure 348113DEST_PATH_IMAGE197
And the result is sent back to the user. The user outputs the ciphertext to the result 1 according to the steps in the stage (three)
Figure 259568DEST_PATH_IMAGE196
And ciphertext output result 1
Figure 354563DEST_PATH_IMAGE197
Adding to obtain true inference result
Figure 476103DEST_PATH_IMAGE198
. Thus, all reasoning is completed.
The invention provides a secure convolution neural network inference system on a ciphertext image, as shown in fig. 3, comprising:
the linear result acquisition module is used for receiving ciphertext image information processed by a user through the main computing node, and calculating a linear layer of the deep convolutional neural network according to the ciphertext image information to obtain a linear layer result;
the nonlinear result acquisition module is used for sending the linear layer result to the ReLU auxiliary calculation node by the main calculation node so that the ReLU auxiliary calculation node can calculate the calculation result of the ReLU of the nonlinear layer of the deep convolutional neural network according to the linear layer result;
and the ciphertext image reasoning module is used for receiving the calculation result of the nonlinear layer ReLU of the deep convolutional neural network by the main calculation node and sending the calculation result of the nonlinear layer ReLU to the user so as to realize the reasoning of the ciphertext image.
Compared with Delphi, the secure convolutional neural network reasoning method and system on the ciphertext image, provided by the invention, do not need any encryption means, and are relatively low in time overhead. For a user, simple operations such as splitting and merging are only needed to be performed on an input image, and the operation is more convenient and efficient compared with encryption and decryption operations. Server-side pre-computation is avoided, which also reduces the overhead. The inference method provided by the invention does not need to use any encryption primitive, so that the time for inference calculation is shorter; and no cryptographic computing power is required for the user. Compared with the prior art, the invention has stronger practicability.
The invention splits the input image, and designs a splitting method; based on the calculation specificity of the convolutional neural network, distributing the inference calculation to three server systems for execution, wherein two are main calculation nodes, and one is a ReLU auxiliary calculation node; designing a safe ReLU calculation protocol based on confusion and disorder; the calculation protocol of the stage (II) ensures that the addition of the calculation results of the two main calculation nodes is the true inference result of the convolutional neural network.
In an embodiment of the present invention, a terminal device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. The processor realizes the steps of the above method embodiments when executing the computer program. Alternatively, the processor implements the functions of the modules/units in the above device embodiments when executing the computer program.
The computer program may be partitioned into one or more modules/units that are stored in the memory and executed by the processor to implement the invention.
The terminal device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The terminal device may include, but is not limited to, a processor, a memory.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc.
The memory may be used to store the computer programs and/or modules, and the processor may implement various functions of the terminal device by executing or executing the computer programs and/or modules stored in the memory and calling data stored in the memory.
The modules/units integrated in the terminal device may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer-readable medium may contain suitable additions or subtractions depending on the requirements of legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer-readable media may not include electrical carrier signals or telecommunication signals in accordance with legislation and patent practice.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A secure convolution neural network reasoning method on a ciphertext image is characterized by comprising the following steps:
the main computing node receives ciphertext image information processed by a user, and computes a linear layer of the deep convolutional neural network according to the ciphertext image information to obtain a linear layer result;
the main computing node sends the linear layer result to the ReLU auxiliary computing node, so that the ReLU auxiliary computing node can compute the computation result of the ReLU of the nonlinear layer of the deep convolutional neural network according to the linear layer result;
and the main computing node receives the computation result of the nonlinear layer ReLU of the deep convolutional neural network and sends the computation result of the nonlinear layer ReLU to the user, so that the inference of the ciphertext image is realized.
2. The secure convolutional neural network inference method on ciphertext images as claimed in claim 1, wherein the user-processed ciphertext image information steps are as follows:
splitting an original image into a plurality of ciphertext sub-images, and randomly selecting a random number as a random seed;
and combining the plurality of ciphertext sub-images with the random seeds respectively to obtain a plurality of sets, namely ciphertext image information.
3. The secure convolutional neural network inference method on ciphertext images as claimed in claim 2, wherein the step of splitting the original image into a plurality of ciphertext sub-images is as follows:
step 1, calculating an original image
Figure 668122DEST_PATH_IMAGE001
Mean value of
Figure 784983DEST_PATH_IMAGE002
，
Figure 480406DEST_PATH_IMAGE003
Calculating the original image
Figure 925294DEST_PATH_IMAGE001
Variance of (2)
Figure 465997DEST_PATH_IMAGE004
，
Figure 188096DEST_PATH_IMAGE005
Wherein
Figure 788842DEST_PATH_IMAGE006
As an original image
Figure 721026DEST_PATH_IMAGE007
Each pixel of the pixel
Figure 65419DEST_PATH_IMAGE008
A plurality of;
step 2, calculating a standardized preprocessed image
Figure 891293DEST_PATH_IMAGE009
；
Step 3, generating
Figure 928519DEST_PATH_IMAGE010
Sum original image
Figure 82420DEST_PATH_IMAGE007
Random matrix with same size
Figure 230505DEST_PATH_IMAGE011
The specific method comprises the following steps: original image
Figure 425731DEST_PATH_IMAGE007
Comprises
Figure 571542DEST_PATH_IMAGE008
Pixels, in turn, based on uniform distribution
Figure 275056DEST_PATH_IMAGE012
Generating
Figure 23569DEST_PATH_IMAGE008
Each random pixel forms a random matrix
Figure 761718DEST_PATH_IMAGE011
Wherein, in the process,
Figure 78430DEST_PATH_IMAGE013
；
step 4, selecting a proportion parameter
Figure 269239DEST_PATH_IMAGE014
；
Step 5, before calculation
Figure 572176DEST_PATH_IMAGE010
A ciphertext sub-image
Figure 899252DEST_PATH_IMAGE015
(ii) a First, the
Figure 386865DEST_PATH_IMAGE016
A ciphertext sub-image
Figure 64971DEST_PATH_IMAGE017
，
Figure 420866DEST_PATH_IMAGE018
The number of nodes is calculated for the master,
Figure 868028DEST_PATH_IMAGE019
。
4. the secure convolution neural network reasoning method for ciphertext image of claim 3, wherein the random matrix is generated according to a uniform distribution method, a normal distribution method or a lognormal distribution method
Figure 260963DEST_PATH_IMAGE011
。
5. The secure convolutional neural network inference method on ciphertext images of claim 3, wherein the linear layer result
Figure 426365DEST_PATH_IMAGE020
The following:
Figure 569640DEST_PATH_IMAGE021
wherein, the first and the second end of the pipe are connected with each other,
Figure 136887DEST_PATH_IMAGE022
in order to be the weight of the model,
Figure 700724DEST_PATH_IMAGE023
in order to bias the model, the bias of the model,
Figure 353422DEST_PATH_IMAGE024
is the ciphertext image information of the current layer.
6. The secure convolutional neural network inference method on ciphertext images as claimed in claim 5, wherein the ReLU auxiliary computation node computes the computation result of the ReLU of the nonlinear layer of the deep convolutional neural network according to the linear layer result as follows:
a. generating random disorder tables
Figure 51120DEST_PATH_IMAGE025
At the same timeRecording each element in the original sequence in a random disorder table
Figure 410557DEST_PATH_IMAGE025
Position in (2) to generate a reduction table
Figure 473191DEST_PATH_IMAGE026
；
b. Computing mask ciphertext sub-image
Figure 160655DEST_PATH_IMAGE027
According to a disorder table
Figure 599727DEST_PATH_IMAGE025
Computing out-of-order ciphertext sub-images
Figure 813670DEST_PATH_IMAGE028
Out-of-order ciphertext subimages
Figure 47206DEST_PATH_IMAGE029
Sending the data to a ReLU auxiliary computing node; wherein, the first and the second end of the pipe are connected with each other,
Figure 471234DEST_PATH_IMAGE030
，
Figure 448417DEST_PATH_IMAGE031
is composed of
Figure 516867DEST_PATH_IMAGE016
A number of the ciphertext sub-images,
Figure 921304DEST_PATH_IMAGE032
in order to be a random mask, the mask is,
Figure 177553DEST_PATH_IMAGE033
as ciphertext sub-image of mask
Figure 630531DEST_PATH_IMAGE034
Each of the elements of (a) to (b),
Figure 881384DEST_PATH_IMAGE035
as mask ciphertext sub-image
Figure 519038DEST_PATH_IMAGE036
To
Figure 855342DEST_PATH_IMAGE037
The value of each of the elements is,
Figure 112011DEST_PATH_IMAGE038
is an out-of-order table
Figure 217370DEST_PATH_IMAGE039
To (1)
Figure 776658DEST_PATH_IMAGE040
A value of an element;
c. ReLU auxiliary computing node selects a random seed
Figure 334679DEST_PATH_IMAGE041
In uniform distribution
Figure 395039DEST_PATH_IMAGE042
Generating and
Figure 354904DEST_PATH_IMAGE016
a ciphertext sub-image
Figure 334362DEST_PATH_IMAGE043
Random matrix of the same size as random perturbation
Figure 645257DEST_PATH_IMAGE044
Each random disturbance
Figure 243729DEST_PATH_IMAGE044
The specific generation method comprises the following steps:
Figure 58101DEST_PATH_IMAGE016
a ciphertext sub-image
Figure 723307DEST_PATH_IMAGE043
Comprises
Figure 255919DEST_PATH_IMAGE008
Pixels, in turn, based on uniform distribution
Figure 658082DEST_PATH_IMAGE045
Generating
Figure 326960DEST_PATH_IMAGE008
The random pixels constitute random disturbance
Figure 913799DEST_PATH_IMAGE044
；
d. The ReLU auxiliary computing node receives the out-of-order ciphertext sub-images sent by each main computing node
Figure 136970DEST_PATH_IMAGE046
Calculating intermediate parameters
Figure 139561DEST_PATH_IMAGE047
；
e. If the intermediate parameter
Figure 741575DEST_PATH_IMAGE048
Then before calculation
Figure 436999DEST_PATH_IMAGE010
The out-of-order ciphertext output is:
Figure 616307DEST_PATH_IMAGE049
calculating the first
Figure 422589DEST_PATH_IMAGE050
The out-of-order ciphertext output is:
Figure 393956DEST_PATH_IMAGE051
wherein, in the step (A),
Figure 994702DEST_PATH_IMAGE052
is a first
Figure 926886DEST_PATH_IMAGE053
An out-of-order sub-image of the ciphertext,
Figure 271280DEST_PATH_IMAGE054
is as follows
Figure 346421DEST_PATH_IMAGE050
An out-of-order ciphertext sub-image; if the intermediate parameter
Figure 118068DEST_PATH_IMAGE055
Before calculation
Figure 537548DEST_PATH_IMAGE010
The out-of-order ciphertext output is:
Figure 685632DEST_PATH_IMAGE056
calculating the first
Figure 366012DEST_PATH_IMAGE050
The out-of-order ciphertext output is:
Figure 574140DEST_PATH_IMAGE057
will be calculated
Figure 215337DEST_PATH_IMAGE050
Out-of-order ciphertext output
Figure 901533DEST_PATH_IMAGE058
Respectively sending the data to each main computing node;
f. then according to the reduction table
Figure 452731DEST_PATH_IMAGE026
Calculating to obtain a mask cipher text output
Figure 566181DEST_PATH_IMAGE059
Wherein, in the step (A),
Figure 694674DEST_PATH_IMAGE060
is composed of
Figure 512457DEST_PATH_IMAGE050
Out-of-order ciphertext output
Figure 105112DEST_PATH_IMAGE061
Each of the elements of (a) to (b),
Figure 592725DEST_PATH_IMAGE062
is composed of
Figure 270831DEST_PATH_IMAGE063
Out-of-order ciphertext output
Figure 610415DEST_PATH_IMAGE064
To (1)
Figure 57577DEST_PATH_IMAGE065
The value of each of the elements is,
Figure 716091DEST_PATH_IMAGE065
to restore the watch
Figure 881493DEST_PATH_IMAGE026
To
Figure 775500DEST_PATH_IMAGE066
A value of an element; computing a ciphertext output result based on the mask ciphertext output
Figure 77168DEST_PATH_IMAGE067
Completing the calculation of nonlinear layer ReLU to obtain the cipher text output result
Figure 906584DEST_PATH_IMAGE068
(ii) a Wherein the true inference results
Figure 559282DEST_PATH_IMAGE069
。
7. The secure convolutional neural network inference method on ciphertext images of claim 6, wherein a random disorder table is generated
Figure 742133DEST_PATH_IMAGE025
Comprises the following steps:
each master computing node using the same random number
Figure 163887DEST_PATH_IMAGE070
As random seeds, according to uniform distribution
Figure 164204DEST_PATH_IMAGE071
Generating and
Figure 38619DEST_PATH_IMAGE050
a ciphertext sub-image
Figure 540008DEST_PATH_IMAGE072
Random matrix with same size as random mask
Figure 816268DEST_PATH_IMAGE073
(ii) a Then the sequences are combined
Figure 987486DEST_PATH_IMAGE074
Random disorder, generating random disorder table
Figure 349198DEST_PATH_IMAGE025
(ii) a Wherein the content of the first and second substances,
Figure 643825DEST_PATH_IMAGE075
is composed of
Figure 977854DEST_PATH_IMAGE016
A ciphertext sub-image
Figure 382291DEST_PATH_IMAGE076
The number of the elements in the Chinese character 'Zhongqin'.
8. A secure convolutional neural network inference system on a ciphertext image, comprising:
the linear result acquisition module is used for receiving ciphertext image information processed by a user through the main computing node, and calculating a linear layer of the deep convolutional neural network according to the ciphertext image information to obtain a linear layer result;
the nonlinear result acquisition module is used for sending the linear layer result to the ReLU auxiliary computing node by the main computing node so that the ReLU auxiliary computing node can compute the computation result of the nonlinear layer ReLU of the deep convolutional neural network according to the linear layer result;
and the ciphertext image inference module is used for receiving the calculation result of the nonlinear layer ReLU of the deep convolutional neural network by the main calculation node and sending the calculation result of the nonlinear layer ReLU to a user to realize the inference of the ciphertext image.
9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor when executing the computer program implements the steps of the secure convolutional neural network inference method on ciphertext images as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the steps of the secure convolutional neural network inference method on ciphertext images of any of claims 1 to 7.
CN202211263823.4A 2022-10-17 2022-10-17 Secure convolution neural network reasoning method and system on ciphertext image Active CN115345307B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211263823.4A CN115345307B (en) 2022-10-17 2022-10-17 Secure convolution neural network reasoning method and system on ciphertext image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211263823.4A CN115345307B (en) 2022-10-17 2022-10-17 Secure convolution neural network reasoning method and system on ciphertext image

Publications (2)

Publication Number Publication Date
CN115345307A true CN115345307A (en) 2022-11-15
CN115345307B CN115345307B (en) 2023-02-14

Family

ID=83957095

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211263823.4A Active CN115345307B (en) 2022-10-17 2022-10-17 Secure convolution neural network reasoning method and system on ciphertext image

Country Status (1)

Country Link
CN (1) CN115345307B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140164772A1 (en) * 2012-12-07 2014-06-12 At&T Intellectual Property I, L.P. Augmented reality based privacy and decryption
US20190065974A1 (en) * 2017-08-30 2019-02-28 Axell Corporation Inference device, inference system, and inference method
US20200235908A1 (en) * 2017-11-27 2020-07-23 Mitsubishi Electric Corporation Homomorphic inference device, homomorphic inference method, computer readable medium, and privacy-preserving information processing system
CN111444522A (en) * 2020-03-19 2020-07-24 南昌大学 Random blocking chaotic image encryption method
CN112906715A (en) * 2021-02-19 2021-06-04 电子科技大学 Safety image feature extraction and classification method based on deep neural network
CN114003961A (en) * 2021-12-03 2022-02-01 青岛大学 Deep neural network reasoning method with privacy protection
CN114912132A (en) * 2022-05-11 2022-08-16 南京大学 Method for realizing privacy protection convolutional neural network reasoning based on model conversion

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140164772A1 (en) * 2012-12-07 2014-06-12 At&T Intellectual Property I, L.P. Augmented reality based privacy and decryption
US20190065974A1 (en) * 2017-08-30 2019-02-28 Axell Corporation Inference device, inference system, and inference method
US20200235908A1 (en) * 2017-11-27 2020-07-23 Mitsubishi Electric Corporation Homomorphic inference device, homomorphic inference method, computer readable medium, and privacy-preserving information processing system
CN111444522A (en) * 2020-03-19 2020-07-24 南昌大学 Random blocking chaotic image encryption method
CN112906715A (en) * 2021-02-19 2021-06-04 电子科技大学 Safety image feature extraction and classification method based on deep neural network
CN114003961A (en) * 2021-12-03 2022-02-01 青岛大学 Deep neural network reasoning method with privacy protection
CN114912132A (en) * 2022-05-11 2022-08-16 南京大学 Method for realizing privacy protection convolutional neural network reasoning based on model conversion

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
VIKTOR M. LIDKEA 等: "Convolutional Neural Network Framework for Encrypted Image Classification in Cloud-Based ITS", 《IEEE》 *
刘飞: "安全的神经网络计算及应用", 《硕士电子期刊》 *
石晓玲等: "基于卷积神经网络的交通监控模糊图像复原技术", 《智能城市》 *
谢四江等: "基于同态加密的卷积神经网络前向传播方法", 《计算机应用与软件》 *

Also Published As

Publication number Publication date
CN115345307B (en) 2023-02-14

Similar Documents

Publication Publication Date Title
Giacomelli et al. Privacy-preserving ridge regression with only linearly-homomorphic encryption
EP3075098B1 (en) Server-aided private set intersection (psi) with data transfer
US9331984B2 (en) Secret sharing method and system
CN107196926B (en) Cloud outsourcing privacy set comparison method and device
CN113518092B (en) Set intersection method for realizing multi-party privacy
CN110580409B (en) Model parameter determining method and device and electronic equipment
CN109214201A (en) A kind of data sharing method, terminal device and computer readable storage medium
Gupta et al. Single secret image sharing scheme using neural cryptography
WO2018099577A1 (en) System and method for providing a collective decentralized authority for sharing sensitive data
CN113179158B (en) Multi-party combined data processing method and device for controlling bandwidth
CN112668046A (en) Feature interleaving method, apparatus, computer-readable storage medium, and program product
Roman’kov Cryptanalysis of a combinatorial public key cryptosystem
Lyu Lightweight crypto-assisted distributed differential privacy for privacy-preserving distributed learning
CN115037439A (en) Multi-party privacy set intersection method and system suitable for small set
CN115994559A (en) Efficient method for converting unintentional neural network
CN115345307B (en) Secure convolution neural network reasoning method and system on ciphertext image
CN117355834A (en) Privacy-secure bulk retrieval using private information retrieval and secure multiparty computing
TWI746296B (en) Homomorphic multi-level visual image encryption system and method and its application
Zhou et al. A survey of security aggregation
CN115150055A (en) Privacy protection ridge regression method based on homomorphic encryption
Debbarma et al. 2D Chaos based color image encryption using pseudorandom key generation
Thanikaiselvan et al. Encrypting multiple images using stacked autoencoders
Mancy et al. Protection of encrypted medical image using consent based access control
CN111368309A (en) Information processing method, system and equipment
Wang et al. Improving the proof of “Privacy-preserving attribute-keyword based data publish-subscribe service on cloud platforms”

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant