CN115426205B - Encrypted data generation method and device based on differential privacy - Google Patents
Encrypted data generation method and device based on differential privacy Download PDFInfo
- Publication number
- CN115426205B CN115426205B CN202211380238.2A CN202211380238A CN115426205B CN 115426205 B CN115426205 B CN 115426205B CN 202211380238 A CN202211380238 A CN 202211380238A CN 115426205 B CN115426205 B CN 115426205B
- Authority
- CN
- China
- Prior art keywords
- data
- gradient value
- privacy
- loss gradient
- network layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000012549 training Methods 0.000 claims abstract description 94
- 238000012545 processing Methods 0.000 claims abstract description 65
- 238000004088 simulation Methods 0.000 claims abstract description 23
- 238000004806 packaging method and process Methods 0.000 claims abstract description 7
- 230000006870 function Effects 0.000 claims description 18
- 238000005315 distribution function Methods 0.000 claims description 11
- 238000013507 mapping Methods 0.000 claims description 10
- 238000003064 k means clustering Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims 2
- 230000002708 enhancing effect Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 5
- 230000008030 elimination Effects 0.000 description 4
- 238000003379 elimination reaction Methods 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 2
- 239000006185 dispersion Substances 0.000 description 2
- 235000002566 Capsicum Nutrition 0.000 description 1
- 239000006002 Pepper Substances 0.000 description 1
- 235000016761 Piper aduncum Nutrition 0.000 description 1
- 235000017804 Piper guineense Nutrition 0.000 description 1
- 244000203593 Piper nigrum Species 0.000 description 1
- 235000008184 Piper nigrum Nutrition 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/16—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using machine learning or artificial intelligence
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/20—Network architectures or network communication protocols for network security for managing network security; network security policies in general
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Biophysics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computer Hardware Design (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Storage Device Security (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for generating encrypted data based on differential privacy, relates to the technical field of data processing, and solves the privacy safety problem of shared data, and the method comprises the following steps: inputting the shared data training set into a generation network layer of the model to obtain simulation data; calculating a loss gradient value according to output results obtained by forward outputting the simulation data to other network layers of the model; inputting the loss gradient value into a differential privacy processing layer for privacy removal processing, returning the loss gradient value meeting the differential privacy condition to a generated network layer updating parameter, and training a model; and extracting the generated network layer of the trained model, packaging the generated network layer into a sampler, and processing the structured data input into the sampler to generate privacy-removed encrypted data. The invention encapsulates the trained generation network layer into the sampler, and generates the privacy-removed encrypted data by processing the data through the sampler. The requirement of enhancing the shared data on the basis of privacy protection is met, and meanwhile the safety of the shared data can be protected.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to an encrypted data generation method and device based on differential privacy, electronic equipment and a computer readable medium.
Background
Structured data is data logically expressed and implemented by a two-dimensional table structure, which can distinguish the dimensions of a record by fields, so that the data has a definite relationship and is widely used. However, the structured data usually contains some user privacy information, which can limit the shared publishing of such data, and the structured data is used as a sample of a machine learning model, so that the risk of privacy disclosure exists in each link of data acquisition, model training and the like. For example, it has been demonstrated that convolutional neural network-based disease recognition classifiers can recover personal private information in training data sets by iteratively querying output probabilities. In addition, training of machine learning models generally requires a large amount of structured data as samples, and how to enhance shared data on the basis of privacy protection is a challenge.
Disclosure of Invention
In view of the above, the present invention is directed to a method, an apparatus, an electronic device and a computer-readable medium for generating encrypted data based on differential privacy, so as to at least partially solve at least one of the above technical problems.
In order to solve the above technical problem, a first aspect of the present invention provides an encrypted data generation method based on differential privacy, where the method includes:
acquiring historical shared structured data among different terminals to generate a shared data training set;
inputting the shared data training set into a generation network layer of a model to obtain simulation data;
calculating a loss gradient value according to output results obtained by forward outputting the simulation data to other network layers of the model;
inputting the loss gradient value into a differential privacy processing layer to carry out privacy removal processing to obtain a loss gradient value meeting differential privacy conditions;
transmitting the loss gradient value meeting the difference privacy condition back to a generated network layer updating parameter, and performing iterative training until the loss gradient value meets the training target, and finishing training the model;
and extracting the trained generation network layer of the model, packaging the generation network layer into a sampler, and processing the structured data input into the sampler based on the sampler to generate privacy-removed encrypted data.
According to a preferred embodiment of the present invention, the inputting the loss gradient value into the differential privacy processing layer for privacy elimination processing includes:
configuring preset hyper-parameters;
truncating the loss gradient value according to a preset hyper-parameter;
and adding noise data into the intercepted loss gradient value to obtain the loss gradient value meeting the difference privacy condition.
According to a preferred embodiment of the present invention, before adding noise data to the truncated loss gradient values, the method further comprises:
generating noise data according to the noise distribution function;
or clustering the data in the shared data training set according to the K-means clustering, and then determining noise data according to the data volume of the data in each category;
or determining noise data according to the discrete degree of each data in the shared data training set;
or fitting the data in the shared data training set to a smooth curve, and determining the noise data according to the smooth curve.
According to a preferred embodiment of the invention, the method further comprises: calculating a loss function according to output results of other network layers, wherein the training target is as follows: a predetermined number of iterations is reached or the loss function is less than a threshold.
According to a preferred embodiment of the present invention, before inputting the training set of shared data into the generation network layer of the model to obtain the simulation data, the method further comprises:
and mapping the enumerated data in the shared data training set into numerical data, and filling missing values.
To solve the above technical problem, a second aspect of the present invention provides an encrypted data generating apparatus based on differential privacy, the apparatus including:
the acquisition module is used for acquiring historical shared structured data among different terminals and generating a shared data training set;
the input module is used for inputting the shared data training set into a generation network layer of a model to obtain simulation data;
the output calculation module is used for calculating a loss gradient value according to output results obtained by forward outputting the simulation data to other network layers of the model;
the privacy removing processing module is used for inputting the loss gradient value into a differential privacy processing layer to carry out privacy removing processing so as to obtain the loss gradient value meeting the differential privacy condition;
the training module is used for transmitting the loss gradient value meeting the difference privacy condition back to a generated network layer updating parameter, carrying out iterative training, and finishing training the model until the training target is met;
and the generation module is used for extracting the trained generation network layer of the model, packaging the generation network layer into a sampler, and processing the structured data input into the sampler based on the sampler to generate privacy-removed encrypted data.
According to a preferred embodiment of the present invention, the privacy elimination processing module includes:
the configuration module is used for configuring preset hyper-parameters;
the setting module is used for truncating the loss gradient value according to a preset hyper-parameter;
and the sub-processing module is used for adding noise data into the truncated loss gradient value to obtain the loss gradient value meeting the differential privacy condition.
According to a preferred embodiment of the present invention, the apparatus further comprises:
the sub-generation module is used for generating noise data according to the noise distribution function;
or the sub-generation module is used for clustering data in the shared data training set according to the K-means clustering, and then determining noise data according to the data quantity of the data in each category;
or, the sub-generation module is used for determining noise data according to the discrete degree of each data in the shared data training set;
or, the sub-generation module is configured to fit data in the shared data training set to a smooth curve, and determine noise data according to the smooth curve.
According to a preferred embodiment of the present invention, the training module further calculates a loss function according to output results of other network layers, and the training target is: a predetermined number of iterations is reached or the loss function is less than a threshold.
According to a preferred embodiment of the invention, the device further comprises:
and the mapping module is used for mapping the enumerated data in the shared data training set into numerical data and filling missing values.
To solve the above technical problem, a third aspect of the present invention provides an electronic device, comprising:
a processor; and
a memory storing computer-executable instructions that, when executed, cause the processor to perform the method of any of the above.
To solve the above technical problems, a fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs which, when executed by a processor, implement the above method.
The method combines the data generation capability of a generation network layer with the privacy protection capability of a difference privacy processing layer, inputs a shared data training set into the generation network layer of a model to obtain simulation data in the training process, and forwards outputs the simulation data to other network layers to calculate a loss gradient value; the loss gradient value is subjected to privacy removal processing through a differential privacy processing layer to obtain the loss gradient value meeting the differential privacy conditions; and then the loss gradient value meeting the difference privacy condition is transmitted back to the generated network layer to update parameters, and the generated network layer with the privacy protection function is trained, so that the trained generated network layer can generate privacy-removed structured data after being packaged into a sampler. On one hand, more data can be generated to be used by other machine learning models, and the requirement of enhancing the shared data training set on the basis of privacy protection is met. On the other hand, the method can generate a privacy-removed public data set, is beneficial to the sharing of structured data, and protects the security of shared data.
Drawings
In order to make the technical problems solved by the present invention, the technical means adopted and the technical effects obtained more clear, the following will describe in detail the embodiments of the present invention with reference to the accompanying drawings. It should be noted, however, that the drawings described below are only illustrations of exemplary embodiments of the invention, from which other embodiments can be derived by those skilled in the art without inventive step.
Fig. 1 is a schematic flowchart of an encrypted data generation method based on differential privacy according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an embodiment of the present invention for generating encrypted data without privacy based on an encrypted data generation method with differential privacy;
fig. 3 is a schematic structural framework diagram of an encrypted data generation apparatus based on differential privacy according to an embodiment of the present invention;
FIG. 4 is a block diagram of an exemplary embodiment of an electronic device in accordance with the present invention;
FIG. 5 is a diagrammatic representation of one embodiment of a computer-readable medium of the present invention.
Detailed Description
Exemplary embodiments of the present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments of the invention may be embodied in many specific forms, and should not be construed as limited to the embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept to those skilled in the art.
The same reference numerals denote the same or similar elements, components, or portions throughout the drawings, and thus, a repetitive description thereof may be omitted hereinafter. It will be further understood that, although the terms first, second, third, etc. may be used herein to describe various elements, components, or sections, these elements, components, or sections should not be limited by these terms. That is, these phrases are used only to distinguish one from another. For example, a first device may also be referred to as a second device without departing from the spirit of the present invention. Furthermore, the term "and/or", "and/or" is intended to include all combinations of any one or more of the listed items.
Referring to fig. 1, fig. 1 is a method for generating encrypted data based on differential privacy according to the present invention, as shown in fig. 1, the method includes:
s1, acquiring historical shared structured data among different terminals, and generating a shared data training set;
in this embodiment, the structured data is data logically expressed and implemented by a two-dimensional table structure, and is data in a tabular form, and dimensions of one record can be distinguished by each field. The structured data is mainly stored and managed through a relational database. Therefore, the structured data X and the label data Y corresponding to the structured data X can be obtained from the relational database as a training set.
In this embodiment, the history shared structured data may be structured data, and the structured data may be used for training and processing of a machine learning model, so the structured data may be numerical data or enumerated data. In order to facilitate subsequent processing, the enumerated data in the shared data training set can be mapped into numerical data according to a preset mapping rule, and missing values generated after mapping are filled. Thereby ensuring standard consistency of the structured data.
Further, the history sharing structured data may also be categorized type data. The classification type data may be sequential number data (sequential classification data, such as day of the week) or nominal data (non-sequential classification data). In this embodiment, enumerated data, ordered data, and nominal data are classified into discrete variables, and the discrete variables have corresponding unique numerical values, and can be further mapped into numerical data to facilitate subsequent model processing.
S2, inputting the shared data training set into a generation network layer of a model to obtain simulation data;
referring to fig. 2, an embodiment of the present invention combines the data generation capability of the generation network layer 21 with the privacy protection capability of the differential privacy processing layer 23 to construct a model including the generation network layer 21, the other network layers 22, and the differential privacy processing layer 23. In the training process, the loss gradient value obtained by processing the structured data through the generation network layer 21 and other network layers 22 is subjected to privacy removal processing through the differential privacy processing layer 23 to obtain the loss gradient value meeting the differential privacy conditions, and then the parameters of the generation network layer 21 are updated through the loss gradient value meeting the differential privacy conditions, so that the parameters of the generation network layer 21 after training do not have privacy information, and the privacy protection function is achieved.
Wherein: the generation network layer 21 may implicitly learn the distribution of the samples, sample the simulated data y from the learned distribution according to the input data z, and forward output the simulated data y to the other network layer 22. Illustratively, generating the network layer 21 may include: two fully connected hidden layers, a batch normalization layer and a relu activation layer. As shown in fig. 2, in the present embodiment, the structured data X and the corresponding label data Y are input into the generation network layer 21, and the generation network layer 21 generates a large amount of simulation data according to the structured data X and the corresponding label data Y, so as to enhance the shared data training set. The analog data is then forwarded to other network layers 22.
S3, calculating a loss gradient value according to output results obtained by forward outputting the simulation data to other network layers of the model;
wherein: the other network layers 22 process the simulation data according to the task type to obtain a prediction result of the simulation data y as an output result, and in this step, a loss gradient value is calculated according to the output result of the other network layers 22, and the loss gradient value is output to the differential privacy processing layer 23. Specifically, when calculating the loss gradient value, a loss function is calculated according to the output results of other network layers 22 and the label data in the simulation data, and then a derivative is obtained for the parameters of the generated network layer 21 of the iterative training through the loss function, so as to obtain the loss gradient value of the iterative training. The loss function is determined according to the structure of the generation network layer 21, and may be Jensen-Shannon divergence, wasserstein distance, and the like.
Among other network layers 22 may include: full-link layers, convolutional layers, pooling layers, and the like. The task type may be image recognition, voice recognition, data security recognition, etc., i.e., the other network layer 22 may be an image recognition network layer, a voice recognition network layer, a data security recognition network layer, etc.
S4, inputting the loss gradient value into a differential privacy processing layer for privacy removal processing to obtain a loss gradient value meeting a differential privacy condition;
the difference privacy processing layer 23 performs privacy elimination processing on the loss gradient value to obtain a loss gradient value satisfying the difference privacy condition, and returns the loss gradient value satisfying the difference privacy condition to the generation network layer 21. Noise data is added to the loss gradient value in the differential privacy processing layer 23, so that the loss gradient value to which the noise data is added and the loss gradient value to which no noise is added have the same statistical characteristics, and due to the addition of the noise data to the loss gradient value, the original loss gradient value cannot be known even if other people illegally acquire the loss gradient value to which the noise data is added, and therefore the privacy-removing processing of the loss gradient value is realized.
For example, inputting the loss gradient value into the differential privacy processing layer for de-privacy processing may include:
s41, configuring a preset hyper-parameter,
illustratively, the preset hyper-parameter may be a learning rate, an iteration number, and the like. The present embodiment uses a preset hyperparameter as a threshold value of the truncation loss gradient value.
S42, cutting off the loss gradient value according to the preset hyper-parameter,
specifically, the input loss gradient value is compared with a preset hyper-parameter, the loss gradient value smaller than the preset hyper-parameter is set to be zero, and the loss gradient value larger than or equal to the preset hyper-parameter is not processed.
And S43, adding noise data into the intercepted loss gradient value to obtain the loss gradient value meeting the difference privacy condition.
Preferably, prior to this step, noise data may be generated in advance, such as: a noise distribution function may be selected, and the first noise data may be generated by the selected noise distribution function. Wherein: the noise distribution function may be: gaussian distribution function, poisson distribution function, salt and pepper distribution function, etc. Or, clustering data in the shared data training set according to the K-means clustering, and then determining second noise data according to the data amount of the data in each category; such as: in a noisy environment, data corresponding to a category whose data amount is larger than the first threshold value is regarded as noise data, and in a normal environment, data corresponding to a category whose data amount is smaller than the second threshold value is regarded as noise data. Alternatively, the degree of dispersion of the respective data in the training set of shared data may be calculated, and data having a degree of dispersion larger than a threshold value may be regarded as the third noise data. For example, an average value of the respective data is calculated, and data having a deviation from the average value larger than a threshold value is regarded as noise data. Furthermore, the data in the training set of shared data may be fitted to a smooth curve, and the data deviating from the smooth curve may be used as the fourth noise data. The noise data generated in different modes can be added into the loss gradient value after being cut off independently or can be added into the loss gradient value after being cut off after being combined randomly.
For example, the step may add noise data to the truncated loss gradient value, and calculate a difference value between the loss gradient value to which the noise data is added and the loss gradient value to which the noise data is not added, until the difference value is smaller than a preset value, to obtain a loss gradient value satisfying the differential privacy condition. Wherein: the disparity value may be calculated using a euclidean distance, a mahalanobis distance, or a vector difference.
S5, transmitting the loss gradient value meeting the difference privacy condition back to a generated network layer updating parameter, and performing iterative training until the loss gradient value meets a training target, and finishing training the model;
exemplarily, the step returns the loss gradient value meeting the difference privacy condition to the generation network layer 21 to update the parameter, and then determines whether the training target is met, if not, the steps S2 to S4 are executed to perform iterative training, and if the training target is met, the model training is ended. Wherein: the training targets may be: a predetermined number of iterations is reached or the loss function in step S3 is less than a threshold.
In this embodiment, the loss gradient value is subjected to privacy elimination processing by the differential privacy processing layer 23, and then the parameters of the generated network layer 21 are updated by the loss gradient value satisfying the differential privacy conditions, so that the trained parameters of the generated network layer 21 do not have privacy information, and the trained generated network layer 21 has a privacy protection function.
And S6, extracting the trained generation network layer of the model, packaging the model into a sampler, and processing the structured data input into the sampler based on the sampler to generate privacy-removed encrypted data.
Illustratively, as shown in fig. 2, the generation network layer 21 in the trained model is derived and encapsulated into a sampler. Upon entering the structured data to be processed, the de-privacy encrypted data may be generated to be distributed in conformity with the original structured data (i.e., to contain similar statistical information). Wherein: the amount of generated encrypted data may be set according to the preset hyper-parameter in step S41, thereby generating statistically significant and privacy-removed encrypted data.
Fig. 3 is an encrypted data generating apparatus based on differential privacy according to the present invention, as shown in fig. 3, the apparatus includes:
an obtaining module 31, configured to obtain historical shared structured data between different terminals, and generate a shared data training set;
the input module 32 is configured to input the shared data training set into a generation network layer of a model to obtain simulation data;
the output calculation module 33 is configured to calculate a loss gradient value according to an output result obtained by outputting the simulation data to other network layers of the model in a forward direction;
a privacy removal processing module 34, configured to input the loss gradient value into a differential privacy processing layer for privacy removal processing, so as to obtain a loss gradient value meeting a differential privacy condition;
the training module 35 is configured to transmit the loss gradient value meeting the difference privacy condition back to the generated network layer update parameter, perform iterative training, and end training the model until a training target is met;
and the generating module 36 is configured to extract a generated network layer of the trained model, package the generated network layer into a sampler, and process the structured data input to the sampler based on the sampler to generate privacy-removed encrypted data.
In one embodiment, the de-privacy processing module 34 includes:
the configuration module is used for configuring preset hyper-parameters;
the setting module is used for truncating the loss gradient value according to a preset hyper-parameter;
and the sub-processing module is used for adding noise data into the truncated loss gradient value to obtain the loss gradient value meeting the differential privacy condition.
The training module 35 further calculates a loss function according to output results of other network layers, where the training target is: a predetermined number of iterations is reached or the loss function is less than a threshold.
Further, the apparatus further comprises:
the sub-generation module is used for generating noise data according to the noise distribution function;
or the sub-generation module is used for clustering the data in the shared data training set according to the K-means clustering, and then determining noise data according to the data volume of the data in each category;
or, the sub-generation module is used for determining noise data according to the discrete degree of each data in the shared data training set;
or, the sub-generation module is configured to fit data in the shared data training set to a smooth curve, and determine noise data according to the smooth curve.
The device further comprises:
and the mapping module is used for mapping the enumerated data in the shared data training set into numerical data and filling missing values.
Those skilled in the art will appreciate that the modules in the above-described embodiments of the apparatus may be distributed as described in the apparatus, and that corresponding variations may be made in one or more apparatus other than the above-described embodiments. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
In the following, embodiments of the electronic device of the present invention are described, which may be regarded as an implementation in physical form for the above-described embodiments of the method and apparatus of the present invention. The details described in the embodiments of the electronic device of the invention are to be regarded as supplementary for the embodiments of the method or the apparatus described above; for details which are not disclosed in embodiments of the electronic device of the invention, reference may be made to the above-described embodiments of the method or the apparatus.
Fig. 4 is a block diagram of an exemplary embodiment of an electronic device according to the present invention. The electronic device shown in fig. 4 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 4, the electronic apparatus 400 of this exemplary embodiment is represented in the form of a general-purpose data processing apparatus. The components of electronic device 400 may include, but are not limited to: at least one processing unit 410, at least one memory unit 420, a bus 430 connecting different electronic device components (including the memory unit 420 and the processing unit 410), a display unit 440, and the like.
The storage unit 420 stores a computer-readable program, which may be a code of a source program or a read-only program. The program may be executed by the processing unit 410 such that the processing unit 410 performs the steps of various embodiments of the present invention. For example, the processing unit 410 may perform the steps as shown in fig. 1.
The memory unit 420 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM) 4201 and/or a cache memory unit 4202, and may further include a read only memory unit (ROM) 4203. The storage unit 420 may also include a program/utility 4204 having a set (at least one) of program modules 4205, such program modules 4205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data.
The electronic device 400 may also communicate with one or more external devices 100 (e.g., a keyboard, a display, a network device, a bluetooth device, etc.), enable a user to interact with the electronic device 400 via the external devices 100, and/or enable the electronic device 400 to communicate with one or more other data processing devices (e.g., a router, a modem, etc.). Such communication may occur via input/output (I/O) interfaces 450, and may also occur via network adapter 460 with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network.) network adapter 460 may communicate with the other modules of electronic device 400 via bus 430.
FIG. 5 is a schematic diagram of one computer-readable medium embodiment of the present invention. As shown in fig. 5, the computer program may be stored on one or more computer readable media. The computer readable medium may be a readable signal medium or a readable storage medium. The computer program, when executed by one or more data processing devices, enables the computer-readable medium to implement the above-described method of the invention, namely: acquiring historical shared structured data among different terminals, and generating a shared data training set; inputting the shared data training set into a generation network layer of a model to obtain simulation data; calculating a loss gradient value according to output results obtained by forward outputting the simulation data to other network layers of the model; inputting the loss gradient value into a differential privacy processing layer for privacy removal processing to obtain a loss gradient value meeting a differential privacy condition; transmitting the loss gradient value meeting the difference privacy condition back to a generated network layer updating parameter, and performing iterative training until the loss gradient value meets a training target, and finishing training the model; and extracting the trained generation network layer of the model, packaging the generation network layer into a sampler, and processing the structured data input into the sampler based on the sampler to generate privacy-removed encrypted data.
While the foregoing embodiments have described the objects, aspects and advantages of the present invention in further detail, it should be understood that the present invention is not inherently related to any particular computer, virtual machine or electronic device, and various general-purpose machines may be used to implement the present invention. The invention is not to be considered as limited to the specific embodiments thereof, but is to be understood as being modified in all respects, all changes and equivalents that come within the spirit and scope of the invention.
Claims (10)
1. A method for generating encrypted data based on differential privacy, the method comprising:
acquiring historical shared structured data among different terminals, and generating a shared data training set;
constructing a model comprising a generation network layer, other network layers and a difference privacy processing layer, wherein: generating the distribution of network layer learning sample data, and sampling from the learned distribution according to an input shared data training set to obtain analog data; inputting the shared data training set into a generation network layer of a model to obtain simulation data;
calculating a loss gradient value according to output results obtained by forward outputting the simulation data to other network layers of the model;
configuring preset hyper-parameters; comparing the loss gradient value of the backward input differential privacy processing layer with a preset hyper-parameter, setting the loss gradient value smaller than the preset hyper-parameter to be zero, and not processing the loss gradient value larger than or equal to the preset hyper-parameter; adding noise data into the loss gradient value which is set to be zero or is not processed to obtain the loss gradient value meeting the difference privacy condition;
transmitting the loss gradient value meeting the difference privacy condition back to a generated network layer updating parameter, and performing iterative training until the loss gradient value meets a training target, and finishing training the model;
and extracting the trained generation network layer of the model, packaging the model into a sampler, and processing the structured data input into the sampler based on the sampler to generate privacy-removed encrypted data.
2. The method of claim 1, wherein prior to adding noise data to the truncated loss gradient values, the method further comprises:
generating noise data according to the noise distribution function;
or clustering the data in the shared data training set according to the K-means clustering, and then determining noise data according to the data volume of the data in each category;
or determining noise data according to the discrete degree of each data in the shared data training set;
or fitting the data in the shared data training set to a smooth curve, and determining the noise data according to the smooth curve.
3. The method of claim 1, further comprising: calculating a loss function according to output results of other network layers, wherein the training target is as follows: a predetermined number of iterations is reached or the loss function is less than a threshold.
4. The method of claim 1, wherein prior to inputting the shared training set of data into a generating network layer of a model to obtain simulated data, the method further comprises:
and mapping enumerated data in the shared data training set into numerical data, and filling missing values.
5. An apparatus for generating encrypted data based on differential privacy, the apparatus comprising:
the acquisition module is used for acquiring historical shared structured data among different terminals and generating a shared data training set;
an input module for constructing a model comprising a generation network layer, other network layers, and a differential privacy processing layer, wherein: generating the distribution of network layer learning sample data, and sampling from the learned distribution according to an input shared data training set to obtain analog data; inputting the shared data training set into a generation network layer of a model to obtain simulation data;
the output calculation module is used for calculating a loss gradient value according to output results obtained by forward outputting the simulation data to other network layers of the model;
the privacy removal processing module is used for configuring preset hyper-parameters; comparing the loss gradient value of the backward input differential privacy processing layer with a preset hyper-parameter, setting the loss gradient value smaller than the preset hyper-parameter to be zero, and not processing the loss gradient value larger than or equal to the preset hyper-parameter; adding noise data to the loss gradient value which is set to be zero or is not processed to obtain the loss gradient value meeting the difference privacy condition;
the training module is used for transmitting the loss gradient value meeting the difference privacy condition back to a generated network layer updating parameter, carrying out iterative training, and finishing training the model until the training target is met;
and the generating module is used for extracting the trained generating network layer of the model, packaging the generating network layer into a sampler, and processing the structured data input into the sampler based on the sampler to generate privacy-removed encrypted data.
6. The apparatus of claim 5, further comprising:
the sub-generation module is used for generating noise data according to the noise distribution function;
or the sub-generation module is used for clustering the data in the shared data training set according to the K-means clustering, and then determining noise data according to the data volume of the data in each category;
or, the sub-generation module is used for determining noise data according to the discrete degree of each data in the shared data training set;
or the sub-generation module is used for fitting the data in the shared data training set into a smooth curve and determining the noise data according to the smooth curve.
7. The apparatus of claim 5, wherein the training module further calculates a loss function according to output results of other network layers, and the training objective is: a predetermined number of iterations is reached or the loss function is less than a threshold.
8. The apparatus of claim 5, further comprising:
and the mapping module is used for mapping the enumerated data in the shared data training set into numerical data and filling missing values.
9. An electronic device, comprising:
a processor; and
a memory storing computer-executable instructions that, when executed, cause the processor to perform the method of any of claims 1 to 4.
10. A computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211380238.2A CN115426205B (en) | 2022-11-05 | 2022-11-05 | Encrypted data generation method and device based on differential privacy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211380238.2A CN115426205B (en) | 2022-11-05 | 2022-11-05 | Encrypted data generation method and device based on differential privacy |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115426205A CN115426205A (en) | 2022-12-02 |
| CN115426205B true CN115426205B (en) | 2023-02-10 |
Family
ID=84207717
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211380238.2A Active CN115426205B (en) | 2022-11-05 | 2022-11-05 | Encrypted data generation method and device based on differential privacy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115426205B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117235665B (en) * | 2023-09-18 | 2024-06-25 | 北京大学 | Self-adaptive privacy data synthesis method, device, computer equipment and storage medium |
| CN117056979B (en) * | 2023-10-11 | 2024-03-29 | 杭州金智塔科技有限公司 | Service processing model updating method and device based on user privacy data |
| CN118094012A (en) * | 2024-03-26 | 2024-05-28 | 佛山的度云企业管理有限公司 | Information recommendation method and device based on privacy protection |
| CN118410521B (en) * | 2024-06-27 | 2024-09-13 | 山东云海国创云计算装备产业创新中心有限公司 | Multi-client data privacy processing method, system, equipment, medium and product |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108520181B (en) * | 2018-03-26 | 2022-04-22 | 联想(北京)有限公司 | Data model training method and device |
| CN111242290B (en) * | 2020-01-20 | 2022-05-17 | 福州大学 | Lightweight privacy protection generation countermeasure network system |
| CN113642731A (en) * | 2020-05-06 | 2021-11-12 | 支付宝(杭州)信息技术有限公司 | Training method and device of data generation system based on differential privacy |
| CN113435583B (en) * | 2021-07-05 | 2024-02-09 | 平安科技（深圳）有限公司 | Federal learning-based countermeasure generation network model training method and related equipment thereof |
| CN113642717B (en) * | 2021-08-31 | 2024-04-02 | 西安理工大学 | Convolutional neural network training method based on differential privacy |
| CN114925213A (en) * | 2022-05-16 | 2022-08-19 | 北京航空航天大学 | Industrial knowledge map supplementing method based on layered cross-domain knowledge collaborative deduction |
-
2022
- 2022-11-05 CN CN202211380238.2A patent/CN115426205B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN115426205A (en) | 2022-12-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115426205B (en) | Encrypted data generation method and device based on differential privacy | |
| AU2017322386B2 (en) | Updating attribute data structures to indicate joint relationships among attributes and predictive outputs for training automated modeling systems | |
| CN112085565B (en) | Deep learning-based information recommendation method, device, equipment and storage medium | |
| Nazarenko et al. | Features of application of machine learning methods for classification of network traffic (features, advantages, disadvantages) | |
| Chruściński et al. | Generalized semi-Markov quantum evolution | |
| Grigoriu | Linear random vibration by stochastic reduced‐order models | |
| CN113935050B (en) | Feature extraction method and device based on federal learning, electronic equipment and medium | |
| CN117811801A (en) | Model training method, device, equipment and medium | |
| CN117094412A (en) | Federal learning method and device aiming at non-independent co-distributed medical scene | |
| CN111475511A (en) | Data storage method, data access method, data storage device, data access device and data access equipment based on tree structure | |
| CN112396310B (en) | Social credit risk assessment system based on machine learning | |
| WO2019167240A1 (en) | Information processing device, control method, and program | |
| WO2022161624A1 (en) | Candidate machine learning model identification and selection | |
| CN112948582A (en) | Data processing method, device, equipment and readable medium | |
| CN114897588B (en) | Order management method and device based on data analysis | |
| Babatunde et al. | On The Application Of Genetic Probabilistic Neural Networksand Cellular Neural Networks In Precision Agriculture | |
| CN116108286A (en) | False information detection method, device and equipment based on propagation reconstruction | |
| Del Moral et al. | On the Robustness of the Snell envelope | |
| CN113591969A (en) | Face similarity evaluation method, device, equipment and storage medium | |
| CN113806600B (en) | Method, device, equipment and medium for constructing family relation network of service data | |
| CN118570588B (en) | Image generation method, device, equipment and storage medium | |
| CN111125685A (en) | Method and device for predicting network security situation | |
| Li et al. | Cloud platform protocol data graph structure modeling method based on spectral clustering | |
| CN116383884B (en) | Data security protection method and system based on artificial intelligence | |
| CN113282740B (en) | Content recommendation method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |