CN110059501A - A kind of safely outsourced machine learning method based on difference privacy - Google Patents

Abstract

The invention discloses a security outsourcing machine learning method based on differential privacy, which belongs to the field of cyberspace security. After uploading to the cloud server, the cloud server stores and adds noise to the encrypted data, and obtains the query function through interaction with the machine learning model provider to perform machine learning. This method effectively combines outsourcing computing with differential privacy, which not only ensures the security and privacy of machine learning, but also greatly reduces computing overhead and computing cost and improves computing efficiency, effectively alleviating the traditional outsourcing machine learning method. inefficiencies and security issues.

Description

A secure outsourcing machine learning method based on differential privacy

technical field

The invention belongs to the field of cyberspace security, in particular to a security outsourcing machine learning method based on differential privacy.

Background technique

With the development of the Internet and information technology, more and more data are generated and utilized. According to statistics, the current growth rate of global data is about 40% per year, and the global big data industry will develop strongly in the next five years. In the face of the increasing surge of massive data, cloud computing technology, as a new data computing and storage mode, can greatly meet its storage and processing requirements. Through the storage and computing outsourcing capabilities of cloud computing technology, users can transfer local computing and storage requirements to the cloud, and use the powerful computing and storage capabilities of cloud servers to improve the efficiency of data processing. Therefore, cloud computing technology with powerful computing power has become a good partner of big data technology.

At the same time, machine learning is based on efficient learning algorithms, abundant and huge data and powerful computing environment, and is widely used in pattern recognition, computer vision, data mining and other scenarios by using a large amount of data accumulated by humans. Driven by scientific research and industrial development, machine learning involves more and more fields and applications, especially in medicine, finance, business and other fields. For example, in medical diagnosis, we train a machine learning model by collecting massive case data, which can accurately analyze the probability of a patient suffering from a certain disease.

Although cloud computing outsourcing services use its powerful storage and computing capabilities to solve problems such as user computing difficulties, as an incompletely trusted third party, our personal sensitive information will face many new security challenges, including outsourcing data storage and computing, etc. Service security and privacy protection issues. For example: On March 17, 2018, The New York Times reported that Cambridge Consulting had obtained access to the data of more than 50 million Facebook users, causing Facebook, which has 2 billion users, to fall into the largest personal information leak in history.

To deal with the above privacy challenges, the traditional solution is for data providers to protect the privacy of data by using encryption technology, but the final result is not ideal. Differential privacy, as one of the most popular privacy protection techniques, has been widely used and studied. Its main idea is that for two data sets that differ by only one record, the probability of querying them to obtain the same value is negligible. The most common method is to randomize the query results by adding noise that satisfies a certain distribution to the query results. As an alternative, differential privacy not only protects the privacy of data, but also improves the efficiency of data processing. Therefore, the data provider outsources the data to the cloud server, and then interacts with the machine learning model provider through the cloud server to complete safe and effective machine learning tasks.

In the research of existing methods, it is found that traditional methods have at least the following problems:

1) In order to adapt to different applications and privacy budgets, different types of noise must be added to the data applied to different query tasks, which inevitably increases the computational overhead and interaction, and increases the computational cost.

2) When data providers publish their data, the public entity i.e. cloud server must exist where all different types of datasets can be stored with different types of noise, posing a great challenge to the storage space of cloud servers.

SUMMARY OF THE INVENTION

In order to solve the problem of low efficiency caused by adding different types of noise to the data set in the traditional solution, the present invention provides a secure outsourcing machine learning method based on differential privacy, which combines cloud computing technology and differential privacy technology to convert complex computing and storage tasks. Outsourcing not only ensures the security and privacy of machine learning, but also greatly reduces computing overhead and cost and improves computing efficiency, effectively alleviating the inefficiency and security problems faced by traditional outsourcing machine learning methods.

The present invention adopts the following technical solutions to realize: the differential privacy-based security outsourcing machine learning method includes the steps:

S1. The data provider selects the Paillier encryption algorithm with the property of additive homomorphic encryption, and negotiates with the machine learning model provider DE to generate a pair of keys (sk, pk), in which the data provider holds the public key pk, and the machine learning model provides The owner DE holds the private key sk;

S2. The data provider will preprocess its data according to the attribute distinction before uploading, and then encrypt the preprocessed data M=(m ₁ , m ₂ , . . . , m _n ) with the public key pk. The ciphertext data ||m ₁ || _pk , ||m ₂ || _pk , ..., ||m _n || _pk are sent to the cloud server CSP;

S3. The cloud server CSP receives the uploaded ciphertext data, and obtains the query function F from the machine learning model provider DE, and calculates the noise η that conforms to the ε-differential privacy standard. Add(||M||,||η ||) algorithm adds it to the ciphertext data in step S2, and sends the noised data ||F(M)+η|| _pk to the machine learning model provider DE;

S4. The machine learning model provider DE receives the noised data, and after decrypting Dec(||m ₁ || _pk , ||m ₂ || _pk , ..., ||m _n || _pk , sk), Obtain the noise data (F(M)+η), and use it as an input to analyze the noise data using machine learning algorithms to complete the machine learning task.

Compared with the traditional full homomorphic encryption, the present invention does not need to spend a large amount of storage space and computing space on the cloud server; using the properties of homomorphic encryption, the cloud server in the third step can safely add noise to the encrypted data, which solves the problem of data encryption. safe question. Compared with the existing methods, the beneficial effects obtained are mainly as follows:

1) The data provider does not need to add noise locally, and the addition of noise is done with the help of a powerful cloud server using cloud computing technology.

2) Through the addition of homomorphic encryption technology, the property of adding operations to ciphertext data without affecting its data integrity ensures that the data will not be leaked between the cloud server and the machine learning model provider during the operation and storage process; Compared with fully homomorphic encryption, the communication complexity is greatly reduced, the interactive operations in the encryption process are reduced, the computational overhead is reduced, and the computational efficiency is improved. At the same time, the differential privacy technology is used to achieve privacy protection by adding noise to sensitive data.

3) The security of outsourced machine learning is guaranteed, and private data is not disclosed to untrusted third parties to realize machine learning.

Description of drawings

Fig. 1 is the flow chart of the outsourcing machine learning method of the present invention;

FIG. 2 is a comparison diagram of the effect of using the data of the method of the present invention and the original non-noise-added data to perform the same machine learning task.

Detailed ways

As a new type of data computing and storage mode, cloud-based data computing has very powerful data processing capabilities and larger storage space. In the present invention, through cloud computing technology, a large number of local computing operations (including using differential privacy technology to add noise) can be completed with the help of cloud servers; machine learning tasks are completed through the interaction between cloud servers and machine learning model providers, thereby realizing safe and efficient. Outsource machine learning tasks. In order to facilitate the understanding of the present invention by those skilled in the art, the present invention will be described in detail below with reference to the accompanying drawings and embodiments, but the embodiments of the present invention are not limited thereto.

Some basic concepts involved in the present invention are as follows:

1) Paillier Homomorphic Encryption: Homomorphic encryption technology, like general encryption technology, performs encryption operations on the encrypted message, that is, without decrypting the ciphertext, by performing operations on the ciphertext, the plaintext data can be encrypted. Various calculations meet the security requirements of privacy protection. Secondly, homomorphic encryption technology has natural properties that general encryption technology does not have. In general, direct calculation of encrypted data will destroy the corresponding plaintext, while ciphertext data using homomorphic encryption can be directly calculated without destroying the integrity and confidentiality of the corresponding plaintext information. In summary, homomorphic encryption is a form of encryption that allows certain types of computations to encrypt ciphertext, and perform operations on the plaintext to obtain an encrypted result during decryption. Paillier homomorphic encryption is additive homomorphic encryption, which has a good application in the calculation of ciphertext space and is also suitable for the method of the present invention.

2) ε-Differential Privacy: is a framework for formalizing privacy in statistical databases, a technique used to prevent de-anonymization. Under this definition, the calculation and processing results of the database are insensitive to changes in a specific record, and a single record in the data set or not in the data set has little impact on the calculation results. Since differential privacy is a probabilistic concept, any differential privacy mechanism is necessarily random. For this method, we adopt the Laplace mechanism, which mainly disturbs the data by adding Laplace noise based on ΔF and privacy budget ε.

3) Outsourced computing: Outsourced computing is a technology that outsources expensive and computationally complex computations to untrusted servers, which allows resource-constrained data providers to outsource their computing load to cloud servers with unlimited computing resources .

4) Machine learning: Arthur Samuel, an American expert in artificial intelligence, described machine learning as follows: Machine learning is a research field that gives computers the ability to learn without directly programming problems. The field of machine learning is broadly divided into three subfields: supervised learning, unsupervised learning, and reinforcement learning. At the same time machine learning as a service, large internet companies now offer machine learning as a service on their cloud platforms. Such as Google Prediction API, Amazon Machine Learning (AmazonML), Microsoft Azure Machine Learning (Azure ML), etc. We can accomplish our machine learning tasks by using machine learning applications on cloud platforms.

As shown in Figure 1, there are 3 entities in this method, namely users (data providers), cloud servers (CSP), and machine learning model providers Data Evaluators (DE); among them, users own data and provide these data Applied to machine learning; CSP interacts with users to provide cloud storage and outsourced computing services, and adds noise to the data; DE interacts with CSP to obtain noisy data and perform corresponding machine learning tasks. Specific steps are as follows:

S1. The user selects the Paillier encryption algorithm with the property of additive homomorphic encryption, and negotiates with the DE to generate a pair of keys (sk, pk). Randomly take large prime numbers p and q, let n=pq, λ(n)=lcm(p-1,q-1), and define the function L(n)=(n-1)/n, then randomly take g∈ (Z/n2Z)* so that it satisfies gcd(L(g ^λ(n) mod n ² ))=1, where lcm and gcd represent the least common multiple and the greatest common divisor, respectively. The public key is pk=(n, g), and the private key is sk=(p, λ); the user holds the public key pk, and the DE holds the private key sk.

S2. The user will preprocess his data according to the attribute distinction before uploading. Then, encrypt the preprocessed data M=(m ₁ , m ₂ , . . . , m _n ) with the public key pk, and encrypt the encrypted ciphertext data ||m ₁ || _pk , ||m ₂ || _pk , ..., ||m _n || _pk is sent to the cloud server.

The encryption process is: the plaintext to be encrypted is M∈Z _n , select a random number r<n, and then calculate the ciphertext C=g ^M r ⁿ mod n ² .

S3. The cloud server CSP receives the ciphertext data uploaded in step S2, obtains the query function F from the machine learning model provider DE, uses the Laplace mechanism to calculate the noise η that conforms to the ε-differential privacy standard, and uses Add(||M| |,||η||) algorithm adds it to the ciphertext data in step S2, and sends the noised data ||F(M)+η|| _pk to DE.

Due to the use of the additive homomorphic encryption Paillier algorithm, it is guaranteed that the privacy of the data will not be leaked through the cloud server after being encrypted by the user. At the same time, the property of additive homomorphic encryption E(x+y)=Eval(E(x), E(y) ) enables the cloud server to directly operate on the ciphertext data without destroying the integrity and confidentiality of the corresponding plaintext information.

The steps for cloud server CSP to use Laplace mechanism to calculate the noise η conforming to the ε-differential privacy standard are as follows:

S31. The cloud server CSP first interacts with the machine learning model provider DE to obtain a query function F, and calculates the sensitivity ΔF.

S32, calculate b=ΔF/ε according to the set privacy budget parameter.

S33. Generate Laplace noise η.

S4. DE receives the noise-added data uploaded in step S3, decrypts Dec(||m ₁ || _pk , ||m ₂ || _pk , ..., ||m _n || _pk , sk), and obtains Noise data (F(M)+η) is used as input, and machine learning algorithm is used to classify and regress the noise data, so as to complete the machine learning task. Because it is noisy data, the machine learning model provider cannot know its original data, which realizes the privacy protection in the machine learning process.

The decryption process is: calculating F(M)+η=L(C ^λ(n) mod n ² )*μmod n, where C is the ciphertext information.

It can be seen from the above implementation process that the present invention is a secure outsourcing machine learning method based on ε-differential privacy, combines cloud computing technology with differential privacy, and utilizes the property of homomorphic encryption to outsource the noise addition operation to a cloud server. Through the interaction between the cloud server and the machine learning model provider, the data can safely participate in machine learning, which not only protects the user's privacy, but also does not cause privacy leakage, and greatly improves the computing efficiency compared with the traditional solution. This method has strong applicability and can be widely used in various scenarios where privacy-preserving machine learning needs to be implemented.

In this embodiment, the user is a hospital, and the hospital, as a data provider, hopes to use the patient's personal case, that is, the private data, to implement intelligent diagnosis by using a machine learning algorithm. However, it requires both providing useful data for machine learning and protecting patient privacy. The other two entities, the cloud server and the machine learning model provider, are not completely trusted, and we can implement machine learning without revealing privacy to other entities by using the method provided by the present invention.

First, as a user or data provider, the hospital uses the Paillier addition homomorphic encryption algorithm to generate a pair of keys (sk, pk), in which the hospital holds the public key pk, and uses the public key pk to pair the sensitive data used to train the machine learning model M=(m ₁ , m ₂ ,..., m _n ) performs encryption processing to generate ciphertext, and the private key sk is handed over to the machine learning model provider for safekeeping.

Then the hospital uploads the encrypted ciphertext data to the cloud server, the cloud server receives the uploaded ciphertext, and obtains the query function F from the machine learning model provider DE, and uses the Laplace mechanism to calculate the noise that conforms to the ε-differential privacy standard η, use the Add(||M||,||η||) algorithm to add it to the generated ciphertext, and send the noised data ||F(M)+η|| _pk to the machine learning model provider.

The machine learning provider decrypts the noisy data with the private key sk, and can perform machine learning tasks according to the needs of the hospital, such as the diagnosis of specific diseases.

As shown in Figure 2, by comparing the original data without noise through experiments, in the process of machine learning, using this method to add noise to the original data has little impact on the classification task using different machine learning algorithms. Therefore, this method does not greatly affect the accuracy of machine learning under the premise of ensuring security and privacy, and experiments can prove that this method is completely effective.

The embodiments described in this patent are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Claims (5)

1. a security outsourcing machine learning method based on differential privacy, is characterized in that, comprises the steps: S1. The data provider selects the Paillier encryption algorithm with the property of additive homomorphic encryption, and negotiates with the machine learning model provider DE to generate a pair of keys (sk, pk), in which the data provider holds the public key pk, and the machine learning model provides The owner DE holds the private key sk; S2. The data provider will preprocess its data according to the attribute distinction before uploading, and then encrypt the preprocessed data M=(m ₁ , m ₂ , . . . , m _n ) with the public key pk. The ciphertext data ∥m ₁ ∥ _pk , ∥m ₂ ∥ _pk , ..., ∥m _n ∥ _pk are sent to the cloud server CSP; S3. The cloud server CSP receives the uploaded ciphertext data, and obtains the query function F from the machine learning model provider DE, and calculates the noise η that conforms to the ε-differential privacy standard, and uses the Add(∥M∥,∥η∥) algorithm Add it to the ciphertext data in step S2, and send the noised data ∥F(M)+ _η∥pk to the machine learning model provider DE; S4. The machine learning model provider DE receives the noised data, decrypts Dec(∥m ₁ ∥ _pk , ∥m ₂ ∥ _pk , ..., ∥m _n ∥ _pk , sk), and obtains the noise data (F( M)+η), and as the input, use the machine learning algorithm to analyze the noise data to complete the machine learning task. 2. the security outsourcing machine learning method based on differential privacy according to claim 1, is characterized in that, in step S3, cloud server CSP utilizes Laplace mechanism to calculate the noise η that meets ε-differential privacy standard, and steps are as follows: S31. The cloud server CSP first interacts with the machine learning model provider DE to obtain a query function F, and calculates the sensitivity ΔF; S32, calculate b=ΔF/ε according to the set privacy budget parameter; S33. Generate noise η. 3. The safe outsourcing machine learning method based on differential privacy according to claim 1, is characterized in that, the generation process of key (sk, pk) is: take large prime numbers p and q at random, let n=pq, λ ( n)=lcm(p-1,q-1), and define the function L(n)=(n-1)/n; then randomly take g∈(Z/n ² Z)* to satisfy gcd(L( g ^λ(n) mod n ² ))=1; where lcm and gcd represent the least common multiple and the greatest common divisor, respectively, the public key is pk=(n, g), and the private key is sk=(p, λ). 4. The security outsourcing machine learning method based on differential privacy according to claim 3, characterized in that, in the encryption process in step S2: the plaintext to be encrypted is M ∈ Z _n , a random number r<n is selected, and then the encryption is calculated. The text C=g Mr ⁿ mod ^{n 2} . 5. The secure outsourcing machine learning method based on differential privacy according to claim 3 or 4, wherein in step S4, decryption process: calculate F(M)+η=L(C ^λ(n) mod n ² ) *μmod n, where C is the ciphertext information.