CN110543901A

CN110543901A - image recognition method, device and equipment

Info

Publication number: CN110543901A
Application number: CN201910780015.7A
Authority: CN
Inventors: 郑龙飞; 陈超超; 巫锡斌; 王力; 周俊
Original assignee: Alibaba Group Holding Ltd
Current assignee: Advanced New Technologies Co Ltd; Advantageous New Technologies Co Ltd
Priority date: 2019-08-22
Filing date: 2019-08-22
Publication date: 2019-12-06

Abstract

The embodiment of the specification provides an image identification method, an image identification device and image identification equipment. The encrypted image data is obtained by encrypting original image data of an image to be identified by a client by adopting a fully homomorphic encryption algorithm. And inputting the encrypted image data into a convolutional neural network, and obtaining an identification result. In the convolutional neural network, a polynomial function is used to replace an activation function for activation processing. The polynomial function is fit determined to the activation function based on a predefined fit condition. The predefined fitting conditions include at least: the polynomial function has the same function value and differential value as the activation function at the argument value range boundary of the activation function. And returning the identification result to the client so that the client decrypts the identification result by adopting a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a final identification result.

Description

Image recognition method, device and equipment

Technical Field

One or more embodiments of the present disclosure relate to the field of computer technologies, and in particular, to an image recognition method, an image recognition apparatus, and a device.

background

with the popularization of cloud computing technology and the urgent need of large-scale image recognition for computing capability, the image recognition technology based on cloud computing is gradually becoming a research hotspot due to its huge application value and market potential, and is gradually widely applied in various fields, such as human face recognition of internet finance, disease screening of medical images, and the like.

The image recognition technology based on cloud computing requires a user to upload local image data to a cloud computing platform (also called a cloud server), and the image data is easy to leak in the uploading process. In addition, the image characteristics of the human face or the medical image characteristics have uniqueness, and are difficult to change once leaked, so that the privacy of the image data of the user is protected, and the technical problem to be solved is to provide a safe image identification method.

Disclosure of Invention

One or more embodiments of the present specification describe an image recognition method, apparatus, and device, which can accurately recognize an image while ensuring security of user image data.

in a first aspect, an image recognition method is provided, including:

receiving encrypted image data sent by a client; the encrypted image data is obtained by encrypting original image data of an image to be identified by the client by adopting a fully homomorphic encryption algorithm;

Inputting the encrypted image data into a convolutional neural network, and obtaining an identification result; in the convolutional neural network, a polynomial function is adopted to replace an activation function for activation processing; the polynomial function is fit-determined to the activation function based on a predefined fit condition; the predefined fitting conditions include at least: the polynomial function has the same function value and differential value as the activating function at the boundary of the independent variable value range of the activating function;

And returning the identification result to the client so that the client decrypts the identification result by adopting a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a final identification result.

In a second aspect, an image recognition method is provided, including:

acquiring original image data of an image to be identified;

Encrypting the original image data by adopting a fully homomorphic encryption algorithm to obtain encrypted image data;

Sending the encrypted image data to a cloud server;

Receiving an identification result sent by the cloud server; the identification result is obtained by the cloud server based on the encrypted image data and a convolutional neural network; in the convolutional neural network, a polynomial function is adopted to replace an activation function for activation processing; the polynomial function is fit-determined to the activation function based on a predefined fit condition; the predefined fitting conditions include at least: the polynomial function has the same function value and differential value as the activating function at the boundary of the independent variable value range of the activating function;

and decrypting the identification result by adopting a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a final identification result.

in a third aspect, an image recognition apparatus is provided, including:

the receiving unit is used for receiving the encrypted image data sent by the client; the encrypted image data is obtained by encrypting original image data of an image to be identified by the client by adopting a fully homomorphic encryption algorithm;

The input unit is used for inputting the encrypted image data received by the receiving unit into a convolutional neural network and obtaining an identification result; in the convolutional neural network, a polynomial function is adopted to replace an activation function for activation processing; the polynomial function is fit-determined to the activation function based on a predefined fit condition; the predefined fitting conditions include at least: the polynomial function has the same function value and differential value as the activating function at the boundary of the independent variable value range of the activating function;

And the sending unit is used for returning the identification result to the client so that the client decrypts the identification result by adopting a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a final identification result.

in a fourth aspect, there is provided an image recognition apparatus comprising:

the device comprises an acquisition unit, a recognition unit and a processing unit, wherein the acquisition unit is used for acquiring original image data of an image to be recognized;

the encryption unit is used for encrypting the original image data acquired by the acquisition unit by adopting a fully homomorphic encryption algorithm to obtain encrypted image data;

the sending unit is used for sending the encrypted image data obtained by the encryption unit to a cloud server;

the receiving unit is used for receiving the identification result sent by the cloud server; the identification result is obtained by the cloud server based on the encrypted image data and a convolutional neural network; in the convolutional neural network, a polynomial function is adopted to replace an activation function for activation processing; the polynomial function is fit-determined to the activation function based on a predefined fit condition; the predefined fitting conditions include at least: the polynomial function has the same function value and differential value as the activating function at the boundary of the independent variable value range of the activating function;

and the decryption unit is used for decrypting the identification result received by the receiving unit by adopting a decryption algorithm corresponding to the fully homomorphic encryption algorithm so as to obtain a final identification result.

In a fifth aspect, there is provided an image recognition apparatus comprising:

a memory;

one or more processors; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs when executed by the processors implement the steps of:

in a sixth aspect, there is provided an image recognition apparatus comprising:

a memory;

one or more processors; and

acquiring original image data of an image to be identified;

sending the encrypted image data to a cloud server;

the image identification method, the image identification device and the image identification equipment provided by one or more embodiments of the specification receive encrypted image data sent by a client. The encrypted image data is obtained by encrypting original image data of an image to be identified by a client by adopting a fully homomorphic encryption algorithm. And inputting the encrypted image data into a convolutional neural network, and obtaining an identification result. In the convolutional neural network, a polynomial function is used to replace an activation function for activation processing. The polynomial function is fit determined to the activation function based on a predefined fit condition. The predefined fitting conditions include at least: the polynomial function has the same function value and differential value as the activation function at the argument value range boundary of the activation function. And returning the identification result to the client so that the client decrypts the identification result by adopting a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a final identification result. That is, in the scheme provided in this specification, the client uploads the encrypted image data to the cloud server, and the cloud server can identify the image based on the encrypted image data. Therefore, the image can be accurately identified under the condition that the safety of the user image data is ensured.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.

fig. 1 is a schematic view of an application scenario of an image recognition method provided in the present specification;

FIG. 2 is a flow chart of an image recognition method provided in one embodiment of the present description;

FIG. 3 is a graphical illustration of a polynomial function provided herein;

Fig. 4 is a flowchart of a method for training a CNN model provided in the present specification;

FIG. 5 is a flow chart of an image recognition method according to another embodiment of the present disclosure;

FIG. 6 is a schematic diagram of an image recognition apparatus according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of an image recognition apparatus according to another embodiment of the present disclosure;

FIG. 8 is a schematic diagram of an image recognition device provided in one embodiment of the present description;

fig. 9 is a schematic diagram of an image recognition apparatus according to another embodiment of the present disclosure.

Detailed Description

The scheme provided by the specification is described below with reference to the accompanying drawings.

before describing the solution provided in the present specification, the inventive concept of the present solution will be explained below.

currently, the research directions of image recognition can be divided into the following two types: first, based on differential privacy techniques; second, based on cryptographic techniques. Image recognition based on differential privacy technology is converted into a set of unreal data sets by adding some preset data noise to an original training data set while preserving some statistical properties. And then training the model based on the group of unreal data sets, and carrying out image recognition based on the trained model. The image recognition based on the encryption technology mainly utilizes an encryption algorithm to encrypt an original data set, uploads the original data set to a cloud computing platform to perform corresponding model training, and then performs image recognition based on a trained model.

in the image recognition method of the differential privacy technology, the data added with noise also has the problem of privacy disclosure. Furthermore, models trained based on unrealistic datasets tend to be less accurate. Therefore, the scheme provided by the specification is realized based on the cryptography technology.

Specifically, the cloud server may train a Convolutional Neural Network (CNN), hereinafter referred to as a CNN model, in advance based on the encrypted sample image data. Then, when image recognition is carried out, encrypted image data of an image to be recognized is received from the client. And then obtaining an encrypted recognition result based on the encrypted image data and the trained CNN model. And finally, returning the encrypted identification result to the client, and decrypting the encrypted identification result by the client to obtain a final identification result.

in order to ensure the accuracy of the model, the scheme can adopt a fully homomorphic encryption algorithm to encrypt the image data. A fully homomorphic encryption algorithm is an encryption algorithm that decrypts an operation result obtained after performing some operation (e.g., any number of addition and/or multiplication operations) on encrypted data to exactly equal the operation result obtained after performing the operation on the data before encryption.

Furthermore, it is known from the definition of the fully homomorphic encryption algorithm described above that it only supports an arbitrary number of addition and/or multiplication operations. Therefore, the structure of the CNN model can be improved accordingly.

The CNN model mainly comprises a plurality of layers of networks such as convolution layers, pooling layers and full-connection layers, wherein the convolution layers and the pooling layers are generally arranged in an alternating mode, namely one convolution layer is connected with one pooling layer, the convolution layer is connected with the other convolution layer after the pooling layer, and the like. In addition, an activation function may be provided on one or more layers of the network of the CNN model. The activation function herein may include, but is not limited to, Sigmoid function, Tanh function, and RELU function. Taking the activation function on the fully-connected layer as an example, specifically, the activation function may be connected after each of a plurality of neurons constituting the fully-connected layer, and used to perform activation processing on an output result of the connected neuron. The processing result of the activation function is inputted to each neuron of the next layer network.

For convolutional and fully-connected layers, the two-layer network may not be improved because it mainly implements point-product (combination of multiplication and addition) operations.

in the pooling layer, the raw pooling function includes both the maximum pooling function and the average pooling function. And both do not meet the operation requirement of fully homomorphic encryption. In this specification, a summation function is employed as the pooling function. The summing function conforms to the operating requirements of a fully homomorphic encryption algorithm. The reason for the use of the summing function is explained later.

for the activation function. To the extent that the RELU function is taken as an example in this specification, it may be replaced by a polynomial function. The polynomial function is obtained by finite multiplication and addition of constants and arguments, and also meets the operation requirement of the fully homomorphic encryption algorithm. The specific polynomial functional form is explained later.

In conclusion, the identification of the image can be safely and accurately realized by combining the fully homomorphic encryption algorithm and the improved CNN model structure. It is emphasized that the CNN model or convolutional neural network referred to below in this specification is the above-described modified CNN model.

Fig. 1 is a schematic view of an application scenario of the image recognition method provided in this specification. In fig. 1, the cloud server may employ a fully homomorphic encryption algorithm to encrypt the sample image data, and then may train the CNN model based on the encrypted sample image data. The client can encrypt the original image data of the image to be identified by adopting a fully homomorphic encryption algorithm, and obtain encrypted image data. The client can then upload the encrypted image data to the cloud server. The cloud server can output a recognition result based on the encrypted image data and the trained CNN model, and return the recognition result to the client. And after receiving the identification result, the client decrypts the received identification result by adopting a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a final identification result.

The image recognition method provided in this specification is described below with the cloud server or the client in fig. 1 as an execution subject.

fig. 2 is a flowchart of an image recognition method according to an embodiment of the present disclosure. The execution subject of the method may be a device with processing capabilities: the server or system or apparatus, for example, may be the cloud server in fig. 1. As shown in fig. 2, the method may specifically include:

Step 202, receiving the encrypted image data sent by the client.

The encrypted image data can be obtained by encrypting original image data of an image to be identified by a client by adopting a fully homomorphic encryption algorithm.

and step 204, inputting the encrypted image data into the convolutional neural network, and obtaining an identification result.

The convolutional neural network here may be a convolutional neural network whose training has been completed (the training process thereof will be described later). In the convolutional neural network, a polynomial function is used to replace an activation function for activation processing. The polynomial function is fit determined to the activation function based on a predefined fit condition. The predefined fitting conditions may include at least: the polynomial function has the same function value and differential value as the activation function at the argument value range boundary of the activation function. In addition, the method can further comprise the following steps: the polynomial function has the same function value and differential value as the activation function at the middle of the argument range of the activation function.

the polynomial function may be set in advance manually, or may be generated by:

A functional form of a polynomial function to be generated is determined. And determining a plurality of equality relations which need to be satisfied among a plurality of undetermined coefficients in the polynomial function to be generated according to a predefined fitting condition. And calculating respective values of the undetermined coefficients based on the equation relations. And determining a polynomial function based on the determined function form and the respective values of the plurality of undetermined coefficients.

the substituted activation function is taken as a RELU function, and the argument value range of the RELU function is as follows: the steps involved in the generation of the polynomial function will be described in detail with [ p, q ] as an example. The reason that the argument range of the RELU function can be set here is because the scheme provided in this specification is applied to an image recognition method in which the input data is generally the pixel values of an image, and the pixel values are generally between [0,255], and therefore, the data input to the RELU function is also within a specified range.

in the step of determining the functional form of the polynomial function to be generated, the specific determination process of the functional form may be: and determining the maximum degree of the polynomial function to be generated according to the predefined fitting condition. And determining the frequency value range of the polynomial function to be generated based on the maximum frequency. Randomly selecting one time from the time value range as a target time. Based on the target degree, a functional form of a polynomial function to be generated is determined.

For example, assume that the predefined fitting conditions described above would include only: the polynomial function has the same function value and differential value as the activation function at the argument value range boundary of the activation function. Then four equality relationships can be obtained. Since based on the four equality relationships, four variable values can be calculated. Whereas in general a cubic polynomial function may comprise four coefficients. So that the maximum degree of the polynomial function to be generated can be 3. Thus, the determined number value range can be as follows: [2,3]. If 2 is selected from the frequency value range as the target frequency, the function form of the polynomial function to be generated may be:

g(x)＝ax+bx+c

Wherein a, b and c are undetermined coefficients of a polynomial function to be generated.

In the step of determining a plurality of equation relationships, when the argument of the RELU function takes a p value, the corresponding function value is 0, that is, the RELU function takes a value in the argument range: the function value at the lower boundary of [ p, q ] is 0, and therefore, the equation relationship can be obtained: g (p) 0. In addition, when the argument of the RELU function takes the q value, the corresponding function value is q, that is, the RELU function takes the value range of the argument: the function value at the upper boundary of [ p, q ] is q, and therefore, the equation relationship can be obtained: g (q) ═ q. Furthermore, when the argument of the RELU function takes the value p, the corresponding differential value is 0, that is, the RELU function takes the value range of the argument: the differential value at the lower boundary of [ p, q ] is 0, and therefore, the equation relationship can be obtained: g' (p) ═ 0. Finally, when the argument of the RELU function takes the q value, the corresponding differential value is 1, that is, the RELU function takes the argument range: the differential value at the upper boundary of [ p, q ] is 1, and therefore, the equation relationship can be obtained: g' (q) ═ 1.

in summary, the following four equations can be obtained:

In the step of calculating the values of the undetermined coefficients, the function form only comprises three undetermined coefficients, so that the values of the undetermined coefficients a, b and c can be calculated by selecting three of the four equation relations. Specifically, the values of the three coefficients to be determined are as follows:

based on the determined function form and the respective values of the plurality of undetermined coefficients, the determined polynomial function may be:

Wherein the curve plotted based on the polynomial function may be as shown in fig. 3. In fig. 3, curve a corresponds to a polynomial function and curve b corresponds to a RELU function. As can be seen from fig. 3, the polynomial function satisfies the predefined fitting condition, i.e., the polynomial function has the same function value and differential value as the activation function at the boundary of the argument range of the activation function. .

It should be understood that the above description only describes the generation process of the polynomial function by taking the replaced activation function as the RELU function and the polynomial function to be generated as the second order. In practical applications, the replaced activation function may be a Sigmoid function, a Tanh function, or the like, and the number of times of generating the polynomial function may be three times, or the like, which is not limited in the present specification.

Since in CNN models, the activation function is typically required to be differentiable. The differentiability of the generated polynomial function is explained below: a polynomial function can also be viewed as a linear superposition of power functions consisting of a plurality of exponentials being non-negative integers. The polynomial function is derived because the exponential function, which is a non-negative integer, is an elementary function that is derivable over any interval and linear superposition does not change the function's conductibility. In addition, since the polynomial function is a unary function, and the unary function may be differentiable, the polynomial function may also be differentiable.

Returning to fig. 2, the above is a description of the generation process of the polynomial function. As described in the above-mentioned summary of the present invention, the pooling layer is further improved to accommodate the processing of encrypted image data, and the following describes the data processing procedure of the improved pooling layer.

In the improved pooling layer, a summation function is used as a pooling function to perform pooling operation on output data of a previous network of the pooling layer, so that the output data of the previous network is screened.

Specifically, the summation operation may be repeatedly performed on data in different pooling windows in the output data of the previous layer network of the pooling layer based on the summation function for a plurality of times to obtain a plurality of summation results. And taking the plurality of summation results as output data to a next layer of network.

in one example, the summation function can be expressed as:

The following illustrates the principle of using a summation function as a pooling function:

for the maximum pooling function, it can be expressed as: max (x1, x2, …, xn).

assuming that xk is maximum, max (x1, x2, …, xn) is xk.

order to

when xk is the maximum, lim d → ∞ (xkd)1/d ≦ f ≦ lim d → ∞ (n. xkd)1/d may be present.

let f1 ═ lim d → ∞ (xkd)1/d, then f1 ═ xk.

Let f2 ═ lim d → ∞ (n. xkd)1/d,

Then there is a change in the number of,

so, f ═ xk, i.e.

Thus, it is possible to prevent the occurrence of,

in the above formula, if d is made 1, a summation function is obtained. That is, the above summation function is obtained based on a maximized pooling function.

when the summation function is used as the pooling function, the summation function is used for carrying out summation operation on a plurality of numbers, so that the operation requirement of the fully homomorphic encryption algorithm is met.

And step 206, returning the identification result to the client so that the client decrypts the identification result by adopting a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a final identification result.

it is understood that the final recognition result obtained here should be consistent with the recognition result obtained after the original image data is processed based on the CNN model.

it should be noted that the image recognition method provided in this specification is particularly suitable for binary scenes. Under the two-classification scene, the classification result only comprises two types: positive and negative. In an example, the identification result returned to the client by the cloud server may be 0 or 1 after the homomorphic encryption. And then, after the client decrypts the identification result by adopting a decryption algorithm corresponding to the fully homomorphic encryption algorithm, the obtained final identification result can be 0 or 1. Where 0 may be used to indicate a negative result and 1 may be used to indicate a positive result.

In summary, the image identification method provided in the embodiments of the present description can identify the image to be identified based on the encrypted image data, so that the confidentiality of the image data of the user can be ensured. In addition, the image identification method can identify various types of images, so that the method has good applicability.

it should be further noted that the CNN model may be obtained by training based on a training sample set. The samples in the training sample set may include: image data of the sample image and a tag value, which may indicate a true recognition result of the sample image. Wherein, in the binary scenario, the tag values may be 0 and 1, for example. Where 0 may be the label value of a negative exemplar and 1 may be the label value of a positive exemplar.

The following training process of the CNN model is explained.

fig. 4 is a flowchart of a training method of the CNN model provided in this specification. As shown in fig. 4, the method may include:

step 402, receiving encrypted sample image data sent by a client.

The encrypted sample image data is obtained by encrypting the image data of the sample image by the client by adopting the fully homomorphic encryption algorithm.

step 404, inputting the encrypted sample image data into a convolutional neural network to obtain a prediction result of the sample image.

it should be understood that the convolutional neural network here is the CNN model with the improved structure. That is, in the CNN model, a polynomial function is used instead of an activation function for activation processing. Further, in the pooling layer of the CNN model, a summation function is employed as the pooling function.

In the binary classification scenario, the prediction result may be 0 or 1 after being encrypted in a fully homomorphic manner.

And 406, decrypting the prediction result by using a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a decrypted prediction result.

Here, the decrypted prediction result may be 0 and 1 as described above. Where 0 may be used to indicate a negative result and 1 may be used to indicate a positive result.

and step 408, determining a prediction error according to the decrypted prediction result and the label value.

It should be noted that, in the process of determining the prediction error, a model optimization algorithm such as a random gradient descent method may be combined, which is not limited in this specification.

and step 410, updating parameters of each layer of the convolutional neural network according to the prediction error.

it should be appreciated that steps 402-410 described above may be performed iteratively a plurality of times.

The model training method provided by the specification can be used for training the convolutional neural network directly based on the encrypted sample image data. Therefore, the accuracy of the identification result can be ensured when the image to be identified is identified based on the encrypted image data of the image to be identified.

in this embodiment, for the above-mentioned trained convolutional neural network, it may be tested based on the samples in the test sample set. The test sample set and the training sample set may be obtained by dividing the collected initial sample set according to a predetermined ratio (e.g., 8: 2).

the test process can be specifically as follows: and testing the trained convolutional neural network based on the samples in the test sample set to obtain a test result. And determining the identification accuracy of the trained convolutional neural network based on the test result. And if the identification accuracy is greater than the threshold value, taking the trained convolutional neural network as the finally trained convolutional neural network.

it should be noted that, if the recognition accuracy is not greater than the threshold, the recognition accuracy of the convolutional neural network can be improved by increasing the training sample set, increasing the number of training iteration rounds, regularizing the data, and the like.

the image recognition method and the CNN model training method provided in this specification are described above with the cloud server in fig. 1 as an execution subject. The image recognition method provided in the present specification will be described below with reference to the client in fig. 1 as an execution subject.

Fig. 5 is a flowchart of an image recognition method according to another embodiment of the present disclosure. The execution subject of the method may be the client in fig. 1. As shown in fig. 5, the method may specifically include:

Step 502, obtaining original image data of an image to be identified.

The image to be recognized here may be various types of images such as a face image, a medical image, and the like. The raw image data may refer to pixel values of an image. Taking the image to be recognized as a gray image as an example, the original image data may be a gray value of the image.

And step 504, encrypting the original image data by adopting a fully homomorphic encryption algorithm to obtain encrypted image data.

Step 506, sending the encrypted image data to the cloud server.

After receiving the encrypted image data, the cloud server may obtain a recognition result based on the encrypted image data and the CNN model with the improved structure. In the CNN model, a polynomial function is adopted to replace an activation function for activation processing. Further, in the pooling layer of the CNN model, a summation function is employed as the pooling function. The determination method of the polynomial function and the summation function are as described above, and are not repeated herein.

step 508, receiving the recognition result sent by the cloud server.

After receiving the identification result sent by the cloud server, the client can decrypt the identification result by adopting a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a final identification result. It is understood that the final recognition result obtained here should be consistent with the recognition result obtained after the original image data is processed based on the CNN model.

And 510, decrypting the identification result by adopting a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a final identification result.

In summary, in the image identification method provided in the embodiment of the present description, the client may send the encrypted image data to the cloud server, and in addition, the received identification result is also the encrypted identification result, so that the security of the user image data can be ensured. Therefore, the image can be accurately identified under the condition that the safety of the user image data is ensured.

in correspondence with the image recognition method, an embodiment of the present specification further provides an image recognition apparatus, as shown in fig. 6, the apparatus may include:

a receiving unit 602, configured to receive encrypted image data sent by a client. The encrypted image data is obtained by encrypting original image data of an image to be identified by a client by adopting a fully homomorphic encryption algorithm.

an input unit 604, configured to input the encrypted image data received by the receiving unit 602 into a convolutional neural network, and obtain a recognition result. In the convolutional neural network, a polynomial function is used to replace an activation function for activation processing. The polynomial function is determined based on a predefined fitting condition to the activation function, the predefined fitting condition comprising at least: the polynomial function has the same function value and differential value as the activation function at the argument value range boundary of the activation function.

The activation function may be a RELU function.

The sending unit 606 is configured to return the identification result to the client, so that the client decrypts the identification result by using a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a final identification result.

optionally, the apparatus may further include:

and the operation unit (not shown in the figure) is used for performing pooling operation on the output data of the previous layer of network in the pooling layer of the convolutional neural network by using the summation function as a pooling function so as to realize screening of the output data of the previous layer of network.

The arithmetic unit may be specifically configured to:

And repeatedly carrying out summation operation on data in different pooling windows in the output data of the previous layer network of the pooling layer for multiple times based on the summation function to obtain multiple summation results.

the plurality of summation results are taken as output data to the latter layer network.

optionally, the apparatus may further include:

A collection unit (not shown) for collecting the sample set.

a dividing unit (not shown in the figure) for dividing the sample set collected by the collecting unit into a training sample set and a testing sample set.

a training unit (not shown) for training the convolutional neural network based on the samples in the training sample set.

And the testing unit (not shown in the figure) is used for testing the trained convolutional neural network based on the samples in the test sample set to obtain a test result.

And the first determining unit (not shown in the figure) is used for determining the identification accuracy of the convolutional neural network trained by the training unit based on the test result of the test unit.

the first determining unit is further configured to use the trained convolutional neural network as the finally trained convolutional neural network if the recognition accuracy is greater than the threshold.

Optionally, the convolutional neural network may be trained based on a training sample set, where the samples in the training sample set include: image data of the sample image and a tag value indicating a true recognition result of the sample image. The apparatus may further include:

The receiving unit 602 is further configured to receive the encrypted sample image data sent by the client. The encrypted sample image data is obtained by encrypting the image data of the sample image by adopting a fully homomorphic encryption algorithm by the client.

And the input unit is further configured to input the encrypted sample image data received by the receiving unit 602 into a convolutional neural network to obtain a prediction result of the sample image.

and a decryption unit (not shown in the figure) for decrypting the prediction result by using a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a decrypted prediction result.

and a second determining unit (not shown in the figure) for determining a prediction error according to the prediction result decrypted by the decrypting unit and the tag value.

and an updating unit (not shown in the figure) for updating the parameters of each layer of the convolutional neural network according to the prediction error determined by the second determining unit.

Optionally, the predefined fitting condition may further include: the polynomial function has the same function value and differential value as the activation function at the middle of the argument range of the activation function.

optionally, the apparatus may further include:

A third determining unit (not shown in the figure) for determining a functional form of the polynomial function to be generated.

and the third determining unit is further configured to determine, according to a predefined fitting condition, a plurality of equality relationships to be satisfied between a plurality of undetermined coefficients in the polynomial function to be generated.

and a calculating unit (not shown in the figure) configured to calculate respective values of the multiple undetermined coefficients based on the multiple equation relationships determined by the third determining unit.

And the third determining unit is also used for determining a polynomial function based on the function form and the respective values of the plurality of coefficients to be determined.

The third determining unit may specifically be configured to:

And determining the maximum degree of the polynomial function to be generated according to the predefined fitting condition.

And determining the frequency value range of the polynomial function to be generated based on the maximum frequency.

Randomly selecting one time from the time value range as a target time.

based on the target degree, a functional form of a polynomial function to be generated is determined.

The functions of each functional module of the device in the above embodiments of the present description may be implemented through each step of the above method embodiments, and therefore, a specific working process of the device provided in one embodiment of the present description is not repeated herein.

In the image recognition apparatus provided in one embodiment of the present specification, the receiving unit 602 receives encrypted image data sent by a client. The encrypted image data is obtained by encrypting original image data of an image to be identified by a client by adopting a fully homomorphic encryption algorithm. The input unit 604 inputs the encrypted image data to the convolutional neural network, and obtains a recognition result. In the convolutional neural network, a polynomial function is used to replace an activation function for activation processing. The polynomial function is determined based on a predefined fitting condition to the activation function, the predefined fitting condition comprising at least: the polynomial function has the same function value and differential value as the activation function at the argument value range boundary of the activation function. The sending unit 606 returns the identification result to the client, so that the client decrypts the identification result by using the decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain the final identification result. Therefore, the image can be accurately identified under the condition that the safety of the user image data is ensured.

The image recognition device provided by one embodiment of the present specification may be a module or a unit of the cloud server in fig. 1.

In correspondence with the image recognition method, as shown in fig. 7, an image recognition apparatus according to an embodiment of the present specification may include:

An obtaining unit 702 is configured to obtain original image data of an image to be recognized.

The encrypting unit 704 is configured to encrypt the original image data acquired by the acquiring unit 702 by using a fully homomorphic encryption algorithm, so as to obtain encrypted image data.

A sending unit 706, configured to send the encrypted image data obtained by the encrypting unit 704 to the cloud server.

the receiving unit 708 is configured to receive the identification result sent by the cloud server. The recognition result is obtained by the cloud server based on the encrypted image data and the convolutional neural network. In the convolutional neural network, a polynomial function is used to replace an activation function for activation processing. The polynomial function is fit-determined to the activation function based on a predefined fit condition. The predefined fitting conditions include at least: the polynomial function has the same function value and differential value as the activation function at the argument value range boundary of the activation function.

The decryption unit 710 is configured to decrypt the identification result received by the receiving unit by using a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a final identification result.

An embodiment of the present specification provides an image recognition apparatus that can accurately recognize an image while ensuring security of user image data.

The image recognition device provided by one embodiment of the present specification may be a module or unit of the client in fig. 1.

corresponding to the image recognition method, an embodiment of the present specification further provides an image recognition apparatus, as shown in fig. 8, the apparatus may include: memory 802, one or more processors 804, and one or more programs. Wherein the one or more programs are stored in the memory 802 and configured to be executed by the one or more processors 804, the programs when executed by the processors 804 implement the steps of:

And receiving the encrypted image data sent by the client. The encrypted image data is obtained by encrypting original image data of an image to be identified by a client by adopting a fully homomorphic encryption algorithm.

and inputting the encrypted image data into a convolutional neural network, and obtaining an identification result. In the convolutional neural network, a polynomial function is used to replace an activation function for activation processing. The polynomial function is fit-determined to the activation function based on a predefined fit condition. The predefined fitting conditions include at least: the polynomial function has the same function value and differential value as the activation function at the argument value range boundary of the activation function.

The image recognition device provided by one embodiment of the present specification can accurately recognize an image while ensuring security of user image data.

Corresponding to the image recognition method, an embodiment of the present specification further provides an image recognition apparatus, as shown in fig. 9, the apparatus may include: memory 902, one or more processors 904, and one or more programs. Wherein the one or more programs are stored in the memory 902 and configured to be executed by the one or more processors 904, the programs when executed by the processors 904 performing the steps of:

And acquiring original image data of the image to be identified.

And encrypting the original image data by adopting a fully homomorphic encryption algorithm to obtain encrypted image data.

And sending the encrypted image data to a cloud server.

And receiving an identification result sent by the cloud server. The recognition result is obtained by the cloud server based on the encrypted image data and the convolutional neural network. In the convolutional neural network, a polynomial function is used to replace an activation function for activation processing. The polynomial function is fit-determined to the activation function based on a predefined fit condition. The predefined fitting conditions include at least: the polynomial function has the same function value and differential value as the activation function at the argument value range boundary of the activation function.

the embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The steps of a method or algorithm described in connection with the disclosure herein may be embodied in hardware or may be embodied in software instructions executed by a processor. The software instructions may consist of corresponding software modules that may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an ASIC. Additionally, the ASIC may reside in a server. Of course, the processor and the storage medium may reside as discrete components in a server.

Those skilled in the art will recognize that, in one or more of the examples described above, the functions described in this invention may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The above-mentioned embodiments, objects, technical solutions and advantages of the present specification are further described in detail, it should be understood that the above-mentioned embodiments are only specific embodiments of the present specification, and are not intended to limit the scope of the present specification, and any modifications, equivalent substitutions, improvements and the like made on the basis of the technical solutions of the present specification should be included in the scope of the present specification.

Claims

1. An image recognition method, performed by a cloud server, the method comprising:

2. The method of claim 1, wherein,

And in the pooling layer of the convolutional neural network, performing pooling operation on the output data of the previous layer of network of the pooling layer by adopting a summation function as a pooling function so as to realize screening of the output data of the previous layer of network.

3. The method of claim 2, wherein performing a pooling operation on output data of a network of a previous layer of the pooling layer using a summation function as a pooling function comprises:

repeatedly performing summation operation on data in different pooling windows in the output data of the previous layer of network of the pooling layer for multiple times based on a summation function to obtain multiple summation results;

And taking the plurality of summation results as output data to a next layer of network.

4. The method of claim 1, wherein the convolutional neural network is trained by:

collecting a sample set;

Dividing the sample set into a training sample set and a testing sample set;

Training a convolutional neural network based on samples in the training sample set;

testing the trained convolutional neural network based on the samples in the test sample set to obtain a test result;

Determining the recognition accuracy of the trained convolutional neural network based on the test result;

and if the identification accuracy is greater than a threshold value, taking the trained convolutional neural network as the finally trained convolutional neural network.

5. The method of claim 1, the convolutional neural network trained based on a set of training samples, the samples in the set of training samples comprising: image data of a sample image and a tag value indicating a genuine recognition result of the sample image;

the convolutional neural network is obtained by training the following steps:

Receiving encrypted sample image data sent by the client; the encrypted sample image data is obtained by encrypting the image data of the sample image by the client side by adopting the fully homomorphic encryption algorithm;

inputting the encrypted sample image data into a convolutional neural network to obtain a prediction result of the sample image;

Decrypting the prediction result by adopting a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a decrypted prediction result;

determining a prediction error according to the decrypted prediction result and the label value;

And updating parameters of each layer of network of the convolutional neural network according to the prediction error.

6. The method of claim 1, the predefined fitting condition further comprising: the polynomial function has the same function value and differential value as the activation function at the middle of the argument range of the activation function.

7. The method of claim 1 or 6, the polynomial function being generated by:

determining a function form of a polynomial function to be generated;

Determining a plurality of equality relations which need to be satisfied among a plurality of undetermined coefficients in a polynomial function to be generated according to the predefined fitting condition;

Calculating respective values of the undetermined coefficients based on the equality relations;

And determining the polynomial function based on the function form and the respective values of the plurality of coefficients to be determined.

8. the method of claim 7, the determining a functional form of a polynomial function to be generated, comprising:

determining the maximum times of a polynomial function to be generated according to the predefined fitting condition;

determining the frequency value range of a polynomial function to be generated based on the maximum frequency;

Randomly selecting a frequency from the frequency value range as a target frequency;

And determining the function form of the polynomial function to be generated based on the target times.

9. The method of claim 1, the activation function being a RELU function.

10. an image recognition method, the method being performed by a client, the method comprising:

acquiring original image data of an image to be identified;

sending the encrypted image data to a cloud server;

11. an image recognition apparatus comprising:

12. The apparatus of claim 11, further comprising:

and the operation unit is used for performing pooling operation on the output data of the previous layer of network in the pooling layer of the convolutional neural network by adopting a summation function as a pooling function so as to realize screening of the output data of the previous layer of network.

13. The apparatus of claim 12, the arithmetic unit being specifically configured to:

14. The apparatus of claim 11, further comprising:

A collection unit for collecting a sample set;

a dividing unit, configured to divide the sample set collected by the collecting unit into a training sample set and a testing sample set;

The training unit is used for training the convolutional neural network based on the samples in the training sample set;

The test unit is used for testing the trained convolutional neural network based on the samples in the test sample set to obtain a test result;

the first determining unit is used for determining the identification accuracy of the convolutional neural network trained by the training unit based on the test result of the testing unit;

The first determining unit is further configured to take the trained convolutional neural network as a convolutional neural network which is finally trained if the recognition accuracy is greater than a threshold.

15. the apparatus of claim 11, the convolutional neural network trained based on a set of training samples, the samples in the set of training samples comprising: image data of a sample image and a tag value indicating a genuine recognition result of the sample image; the device further comprises:

the receiving unit is further configured to receive encrypted sample image data sent by the client; the encrypted sample image data is obtained by encrypting the image data of the sample image by the client side by adopting the fully homomorphic encryption algorithm;

The input unit is further configured to input the encrypted sample image data received by the receiving unit into a convolutional neural network to obtain a prediction result of the sample image;

the decryption unit is used for decrypting the prediction result by adopting a decryption algorithm corresponding to the fully homomorphic encryption algorithm to obtain a decrypted prediction result;

a second determining unit, configured to determine a prediction error according to the prediction result decrypted by the decrypting unit and the tag value;

and the updating unit is used for updating the parameters of each layer of network of the convolutional neural network according to the prediction error determined by the second determining unit.

16. The apparatus of claim 11, the predefined fitting condition further comprising: the polynomial function has the same function value and differential value as the activation function at the middle of the argument range of the activation function.

17. The apparatus of claim 11 or 16, further comprising:

A third determining unit for determining a functional form of a polynomial function to be generated;

The third determining unit is further configured to determine, according to the predefined fitting condition, a plurality of equality relationships to be satisfied between a plurality of undetermined coefficients in the polynomial function to be generated;

A calculating unit, configured to calculate respective values of the multiple undetermined coefficients based on the multiple equation relationships determined by the third determining unit;

the third determining unit is further configured to determine the polynomial function based on the function form and respective values of the plurality of coefficients to be determined.

18. The apparatus according to claim 17, wherein the third determining unit is specifically configured to:

19. The apparatus of claim 11, the activation function being a RELU function.

20. an image recognition apparatus comprising:

21. an image recognition apparatus comprising:

A memory;

one or more processors; and

22. An image recognition apparatus comprising:

a memory;

One or more processors; and

Acquiring original image data of an image to be identified;

sending the encrypted image data to a cloud server;