CN114547687A - Question-answering system model training method and device based on differential privacy technology - Google Patents
Question-answering system model training method and device based on differential privacy technology Download PDFInfo
- Publication number
- CN114547687A CN114547687A CN202210159711.8A CN202210159711A CN114547687A CN 114547687 A CN114547687 A CN 114547687A CN 202210159711 A CN202210159711 A CN 202210159711A CN 114547687 A CN114547687 A CN 114547687A
- Authority
- CN
- China
- Prior art keywords
- question
- privacy
- target
- training
- answering system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 135
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000005516 engineering process Methods 0.000 title claims abstract description 18
- 230000007246 mechanism Effects 0.000 claims abstract description 83
- 230000006870 function Effects 0.000 claims description 24
- 238000009826 distribution Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 description 15
- 239000013598 vector Substances 0.000 description 12
- 238000013135 deep learning Methods 0.000 description 10
- 238000013527 convolutional neural network Methods 0.000 description 6
- 238000011156 evaluation Methods 0.000 description 6
- 241000282414 Homo sapiens Species 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 230000002457 bidirectional effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000035945 sensitivity Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000007423 decrease Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 241000238366 Cephalopoda Species 0.000 description 1
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Bioethics (AREA)
- Human Computer Interaction (AREA)
- Medical Informatics (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
The invention provides a method and a device for training a question-answering system model based on a differential privacy technology, which relate to the technical field of question-answering systems and comprise the steps of obtaining privacy mechanisms corresponding to target language models in the question-answering system, wherein the privacy mechanisms comprise at least one, and the privacy mechanisms protect differential privacy of a data set of the question-answering system through intervening the target language models; acquiring a corresponding training sample according to each privacy mechanism, and adding a target disturbance amount to each preset batch of training samples corresponding to each privacy mechanism to obtain a target training sample corresponding to each privacy mechanism; target training samples are respectively input into the target language models, the target language models of the question-answering system are trained, the technical problem that data are easily stolen by question-answering in the question-answering system is solved, and the safety of the question-answering system is improved.
Description
Technical Field
The invention relates to the technical field of question-answering systems, in particular to a question-answering system model training method and device based on a differential privacy technology.
Background
The question-answering system is one of ways for realizing human and machine communication, and can return a simple and accurate answer to a user by analyzing questions provided by the user. Compared with the traditional search engine technology, the question answering system can give more accurate answers to the user, meets the retrieval requirements of the user and improves the retrieval efficiency.
The intelligent question answering systems in different fields can store different question set data and are used for training a more targeted deep learning language model. The current question-answering system based on the deep learning language model and the retraining and updating process of the model are easy to attack, and an attacker can steal more private question-answering data used for training, such as case data of a medical consultation question-answering system, user chatting data of a chatting robot system and the like, and has serious threat to the safety of the question-answering system.
Disclosure of Invention
The invention aims to provide a method and a device for training a question-answering system model based on a differential privacy technology, so as to relieve the technical problem that data is easily stolen by question-answering in a question-answering system and improve the safety of the question-answering system.
In a first aspect, an embodiment of the present invention provides a question-answering system model training method based on a differential privacy technology, where the method includes:
the method comprises the steps that privacy mechanisms corresponding to target language models in a question-answering system are obtained, wherein the privacy mechanisms comprise at least one privacy mechanism, and the privacy mechanisms are used for protecting differential privacy of data sets of the question-answering system by intervening in the target language models;
acquiring a corresponding training sample according to each privacy mechanism, and adding a target disturbance amount to each preset batch of training samples corresponding to each privacy mechanism to obtain a target training sample corresponding to each privacy mechanism;
and respectively inputting the target training samples into the target language model, and training the target language model of the question-answering system.
With reference to the first aspect, an embodiment of the present invention provides a first possible implementation manner of the first aspect, where the step of obtaining a corresponding training sample according to each privacy mechanism, and adding a target perturbation amount to each preset batch of training samples corresponding to each privacy mechanism to obtain a target training sample corresponding to each privacy mechanism includes:
obtaining a group of training samples corresponding to a current privacy mechanism;
presetting divided batches according to a current privacy mechanism and training samples corresponding to the current privacy mechanism, and determining a target disturbance amount added to the training samples of each batch;
adding the target disturbance amount to training samples of corresponding batches to obtain target training samples corresponding to the current privacy mechanism;
and repeating the steps until a target training sample corresponding to each privacy mechanism is obtained.
With reference to the first aspect, an embodiment of the present invention provides a second possible implementation manner of the first aspect, where the step of determining the target perturbation amount added to each batch of training samples further includes:
and adjusting the target disturbance amount added to each batch of training samples based on the gradient of a rating function in the privacy mechanism, wherein the training samples comprise question texts and output answers, and the rating function is used for calculating the matching degree of the output answers and answer sets corresponding to the question texts in the data sets.
With reference to the first aspect, an embodiment of the present invention provides a third possible implementation manner of the first aspect, wherein the step of adjusting the target perturbation amount added to each batch of training samples based on the gradient of the evaluation function in the privacy mechanism includes:
with the increase of the training times of the target language model, if the absolute value descending speed of the gradient is lower than a first speed threshold value, reducing the target disturbance amount; and if the absolute value descending speed of the gradient is higher than a second speed threshold value, increasing the target disturbance quantity, wherein the second speed threshold value is larger than the first speed threshold value.
With reference to the first aspect, an embodiment of the present invention provides a fourth possible implementation manner of the first aspect, where the method further includes:
and verifying the trained target language model based on the accurate matching rate, the harmonic mean value of the accurate rate and the recall rate of the question-answering system and the target disturbance quantity.
With reference to the first aspect, an embodiment of the present invention provides a fifth possible implementation manner of the first aspect, where the disturbance amount includes noise conforming to a laplacian distribution, a gaussian distribution, or an exponential distribution.
With reference to the first aspect, an embodiment of the present invention provides a sixth possible implementation manner of the first aspect, where the target language model is a bidirectional attention flow model.
In a second aspect, an embodiment of the present invention further provides a question-answering system model training apparatus based on a differential privacy technology, where the apparatus includes:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring privacy mechanisms corresponding to a target language model in a question-answering system, the privacy mechanisms comprise at least one, and the privacy mechanisms are used for protecting the differential privacy of a data set of the question-answering system by intervening the target language model;
the adding module is used for obtaining corresponding training samples according to each privacy mechanism and adding target disturbance quantities to the training samples of each preset divided batch corresponding to each privacy mechanism to obtain target training samples corresponding to each privacy mechanism;
and the training module is used for respectively inputting the target training samples into the target language model and training the target language model of the question-answering system.
In a third aspect, an embodiment provides an electronic device, including a memory and a processor, where the memory stores a computer program operable on the processor, and the processor implements the steps of the method described in any one of the foregoing embodiments when executing the computer program.
In a fourth aspect, embodiments provide a machine-readable storage medium having stored thereon machine-executable instructions that, when invoked and executed by a processor, cause the processor to carry out the steps of the method of any preceding embodiment.
The question-answering system based on the deep learning language model is easy to be attacked by data stealing of a malicious attacker when the language model is updated and trained. An attacker obtains a plurality of time slices of the language model, analyzes the model parameters and the input and output results, and calculates corresponding difference scores to capture the probability difference of model distribution before and after training, so that the difference data used for retraining the model each time is stolen, and the data privacy of the retraining model is stolen.
In order to improve the data privacy security of a deep learning question-answering system, training data is prevented from being stolen by a malicious attacker in the process of training a language model. The embodiment of the invention provides a question-answering system model training method and device based on a differential privacy technology. According to the invention, through a differential privacy protection technology, appropriate disturbance is added to data used for retraining each time, so that the purposes of protecting training data and resisting data stealing attack are achieved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a question-answering system model training method based on a differential privacy technology according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a basic language model BIDAF of the question-answering system according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of Q2C and C2Q of an attention flow layer in a BIDAF language model according to an embodiment of the present invention;
fig. 4 is a schematic functional block diagram of a question-answering system model training device based on a differential privacy technology according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a hardware architecture of an electronic device according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
With the development of deep learning, the question-answering system based on deep learning is widely applied. These systems can accept questions in natural language by a user and can query or infer answers from a large amount of heterogeneous data that are desired by the user. Compared with the traditional question-answering system based on the question set and encyclopedia knowledge, the intelligent question-answering system based on the deep learning language model can analyze the questions proposed by the user by utilizing the language model, better understand the semantics of the questions, match the proper questions in the question set and output corresponding answers. Such deep learning based intelligent question-answering systems can also generate answers that better conform to grammatical rules and are closer to human speaking habits when encountering question-answer pair data that are not stored in the library. The generated answers can be stored in a question set together with the questions of the user as a pair of new question-answer pair data, a question-answer database of a question-answer system is expanded, and the efficiency of subsequent retrieval is improved.
Currently, in order to continuously improve the software quality of the question-answering system and update (such as an input method) according to the use habits of users, the question-answering system software based on deep learning must be regularly retrained and the core model is updated. However, in the process of updating the model, the question answering system is vulnerable to data leakage. For example, an attacker can reveal a lot of detailed information about the training data changes through the difference analysis of the language model snapshots before and after the update. Even more serious can reason out the partial challenge-pair data used to retrain the model. Therefore, the data leakage attack has a serious threat to the question answering system.
Based on this, the method and the device for training the question-answering system model based on the differential privacy technology provided by the embodiment of the invention can improve the data security of the question-answering system and reduce the possibility of attack and stealing.
In order to facilitate understanding of the embodiment, a method for training a question-answering system model based on a differential privacy technology disclosed in the embodiment of the present invention is first described in detail.
Fig. 1 is a flowchart of a question-answering system model training method based on a differential privacy technology according to an embodiment of the present invention.
Referring to fig. 1, the method includes the steps of:
step S102, a privacy mechanism corresponding to a target language model in the question-answering system is obtained, wherein the target language model corresponds to at least one privacy mechanism, and the privacy mechanism protects differential privacy of a data set of the question-answering system through intervening the target language model.
Here, the privacy mechanism refers to a mode of adding disturbance, and is generally laplacian noise, gaussian noise, and exponential noise. The main principle of the differential privacy technique is to add noise conforming to laplacian distribution, gaussian distribution, or exponential distribution to the raw data, the output of the query function, or the gradient in model training, etc. The data after noise addition can keep the main characteristics of the data, so that the model is ensured to keep certain prediction precision, but certain uncertainty exists, and an attacker is difficult to acquire specific data information. Two data with small difference are input into the same model, and the obtained output results are generally similar. The invention adds noise into the training sample to realize the differential privacy protection of the training data.
And step S104, acquiring corresponding training samples according to each privacy mechanism, and adding target disturbance quantities to the training samples of each preset divided batch corresponding to each privacy mechanism to obtain target training samples corresponding to each privacy mechanism.
In the application process, a data set of the question-answering system is divided into a training sample set and a verification sample set in a ratio of 6: 1, each sample comprises a question and a plurality of answers made by human beings to the question, and if an answer output by a model is similar to one of written answers given by the question to the human beings in the data set, the model is given credit. The training data is used to train language models in the intelligent question-answering system, and the validation data is used to test the usability of the question-answering system under normal conditions.
It should be noted that training samples corresponding to different privacy mechanisms may have partial overlap, and there is no inclusion relationship. Assuming that i want to use 3 privacy mechanisms (M ═ 3), three sets of data sets with no relation to each other are needed, and each privacy mechanism trains N batches independently, and the data set used is unchanged.
And step S106, respectively inputting the target training samples into the target language model, and training the target language model of the question-answering system.
And calculating the occurrence frequency of words for the questioning data in the training set, constructing a new questioning by using the words with the minimum word frequency of 5%, and drawing up proper answer data for the questioning data to serve as simulated question and answer pair data for verifying the data leakage attack and testing the defense effect of the model added with the differential privacy noise on the data leakage attack.
These simulated question-answer data and the original training data are used to compose new data samples. An attacker can obtain a model snapshot M trained on an original dataset DDAnd adding the model snapshot M after retraining on the data set D' of the simulated question-answer pairD′. In attack, the random sequence s is inquired to form an element T*And observe the corresponding probability distribution MD(s) and MD′(s) to infer differences between the data set D of the pre-trained model and the data set D' that complements the simulated challenge-response pair samples. So that the approximation reasons new pairs of questions and answers to the data information each time it is added for updating the model.
In order to defend against the data leakage attack, in the process of retraining the model in each batch, the differential privacy noise of an exponential mechanism is added to the return value of the query function of the language model. Let the output domain of the query function be R, and each value R ∈ R in the domain be an entity object. Several physical objects are combined together as part of the answer return value of the question-answering system. Under the exponential mechanism, the input of the language model M is a data set D, and a function q (D, R) → R becomes a usability function of the output value R for evaluating the degree of goodness of the output value R, and Δ q is a sensitivity of the function q (D, R) → R. If language model M selects from R and outputs R with a probability proportional to exp (ε q (D, R)/2 Δ q), then language model M provides ε -differential privacy protection.
Retraining the language model epsilon-M added with the differential privacy noise on the original data set D and the data set D' added with the simulated question-answer pair again, and calculating corresponding probability distribution epsilon-MD(s) and ε -MD′(s) difference between(s). Re-inference of data sets of pre-trained models
And a data set supplemented with simulated question-answer pair samples
The variation of the difference between them. By reducing the output difference of the model before and after retraining, the training data is realizedAnd (5) protection.
In a preferred embodiment of practical application, a target disturbance amount is added to training samples of each privacy mechanism through the privacy mechanisms included in a target language model of a question-and-answer system, wherein the training samples are divided according to preset divided batches, a corresponding target disturbance amount is added to each batch of training samples to obtain target training samples, the target language model is trained based on the target training samples to which the target disturbance amount is added, and the artificial interaction function of the question-and-answer system is realized on the basis of effectively protecting training data.
In some embodiments, the target language model may be a bidirectional attention flow model (BIDAF).
Referring to fig. 2 and fig. 3, in the embodiment of the present invention, a bidirectional attention flow language understanding model is used as a basic language model of a question-answering system, and a basic question-answering system framework is built by training the language model on a stanford question-answering dataset (squid) as follows:
1) constructing a target model data set:
in an embodiment of the present invention, the Stanford question-answer dataset (SQuAD) is used to train the basic language model of the question-answer system. SQuAD is a machine-understood data set based on a large number of Wikipedia articles, where the answer to each question is always a span in context. The SquAD contains more than 500 articles, and more than 100,000 question-answer-to-text data are contained, wherein the ratio of the training set to the verification set is 6: 1, which is significantly larger than the previous reading comprehension data set.
2) Training a machine understanding model in a question-and-answer system:
the model architecture of the embodiment of the invention is similar to the model architecture used by the SquAD data set, and only has small changes so as to adapt to the complete filling test. In addition, embodiments of the present invention mask all non-entity words in the final classification level to exclude them from possible answers. During training, the most likely answer p is obtained1Thereafter, all probability values of the entity instances in the context corresponding to the correct answer are added. The loss function is then calculated based on the probability of the sum. Setting the size of the minimum batchAt 48, 8 epochs were trained, stopping early when the accuracy of the validation data began to decline. Each article is divided into short sentences, where each sentence is a window of 19 words around each entity. The recurrent neural networks in BIDAF do not feed forward or back propagate across sentences, which speeds up the training process through parallelization. The sum of cross-entropy losses of the start and end indices is used as the loss function for the model, and an Adam Optimizer optimization function is used to minimize the loss function.
2.1) Structure of the target language model BIDAF
The basic BIDAF question-answering system language model is a layered multi-stage process, and consists of 6 layers, namely:
1. character embedding layer: each word is mapped to vector space using character level CNN (Char-CNN). In particular, each word is mapped to a high-dimensional vector space, let { x }1，...，xTAnd q1，...，qJRepresents the words in the input context paragraph and query, respectively. Character-level embedding of each word is obtained using a Convolutional Neural Network (CNN). The characters are embedded in a vector, which can be considered a one-dimensional input to CNN, whose size is the input channel size of CNN. The output of the CNN is maximally pooled across the width to obtain a fixed size vector for each word.
2. Word embedding layer: each word is mapped to vector space using a pre-trained word embedding model.
3. Context embedding layer: the embedding of words is refined using contextual cues from surrounding words. The first three layers apply to queries and contexts simultaneously.
4. Attention is paid to the flow layer: the query vector and the context vector are combined and a set of query-aware feature vectors is generated for each word in the context.
5. A modeling layer: the context is scanned using a Recurrent Neural Network (RNN), and the LSTM structure is used in the present invention to improve the ability of language model feature extraction.
6. An output layer: an answer to the query is provided. Constituted by softmax, probability p for finding the start and end indices of the answer intervalstartAnd pendAnd (5) vector quantity. The invention combines the context-hidden state with the attention vectors of previous layers to obtain a mixed result. The result of these blends will eventually become the input to the fully connected layer.
The most important of which is the attention flow layer, 2.2) the structural features of which will be described in detail.
2.2) C2Q and Q2C attention mechanism Using attention-flow layer
Firstly, calculating a similarity matrix S epsilon RN×MContaining each pair of context and question hiding state (c)i，qj) The similarity score of (a).
Where c isi，qjIt is shown that the array elements are multiplied correspondingly,
is a weight vector.
The C2Q attention was then calculated. Calling the softmax function to the similarity matrix S line by line to obtain the attention distribution alphaiWith which the problem hidden state q is obtainedjFinally, the output a of C2Q attention is obtainedi：
Q2C attention is then performed. For each context location i e { 1.,. N }, taking the maximum value of the corresponding row of the similarity matrix:
for the result vector m ∈ RNInvoking the softmax function will give an attention distribution β ∈ R with respect to the contextual positionN. Then, a weighted sum c of the context hidden states is obtained using βiThis is also the output c' of Q2C attention. The correlation formula is as follows:
β＝softmax(m)∈RN
finally for each context position ciThe output of the C2Q attention and the Q2C attention were combined.
3) Data leakage attack
3.1) modeling model Generation Structure in BIDAF
Generating language models typically operate on a fixed set of known tokents and are autoregressive. To token1…tn∈TnModeling, probability p (t)1…tn) Conditional probability p (t) prefixed by all tokens preceding iti|t1…ti-1) And calculating to obtain the following results:
therefore, training the autoregressive generative language model M requires learning a function (also referred to as M) that maps token sequences of arbitrary length to a probability distribution on a vocabulary T, modeling the probability of each token occurring at the next position, using M (T) in this paper＜i) Is shown in the reading sequence t1…ti-1∈T*The probability distribution of tokens is then calculated from the model M. M (t)＜i)(ti) Representing the probability t of a particular tokeni。
Data set with a defined BIDAF model architecture
Need to be obtained as training dataAnd (5) concrete models. In the embodiment of the invention, the model M is used for testing the data set t1…tnPerp of the above standardM(t1…tn)＝pM(t1…tn)-1/nProbability p assigned to a sequence using model MM(t1…tn). It only captures the correctness of the most likely selection, this metric captures the most likely outcome of the model output.
It should be noted that the BIDAF language model in the question-answering system is updated periodically by adding or deleting data from the training set. Each update is trained to obtain a new BIDAF model.
Implementing an attack on the BIDAF model requires obtaining access to two model snapshots, before and after the update, of the language model trained on the datasets D and D', respectivelyDAnd MD′Wherein
By querying the sequence ∈ T*Observe the corresponding probability distributions M(s) and M'(s).
And (3) creating question-answer pair texts which do not appear in a plurality of original data sets, and simulating private data which is tried to be extracted during attack. Different word frequency characteristics are considered to control the influence on the used vocabulary. Specifically, the length of the simulated question-answer pairs is fixed, an effective question-answer pair grammar structure is selected, and each placeholder is instantiated with a tag in the dataset vocabulary. The word frequency in the created simulated question-answer pairs text is low in all data sets (all tokens are from the one-fifth word with the lowest word frequency). The amount of private data C is changed by inserting the number of times the simulated question-answer pair text s is inserted in proportion to the number of tokens in the training corpus. Model M is trained on dataset D, and model M' is trained on D using k samples of simulated question-and-answer pair text s. Differential ratings of simulated question-answer versus text for different values of k are then calculated. And the problem data aiming at the corresponding answers can be recovered by sequentially matching the vocabulary in the database to be selected through differential level sequencing.
In some embodiments, step S104 may also be implemented by steps comprising:
step a, obtaining a group of training samples corresponding to the current privacy mechanism.
And b, presetting and dividing batches according to the current privacy mechanism and the training samples corresponding to the current privacy mechanism, and determining the target disturbance amount added to the training samples of each batch.
Wherein, it can also be based on the evaluation function in the privacy mechanism
The gradient of (2) adjusts the amount of disturbance of the target added to each batch of training samples, wherein the training samples include the question text D and the output answer
The evaluation function is used for calculating the matching degree of the output answer and an answer set corresponding to the question text in the data set.
Illustratively, embodiments of the present invention add perturbations to the raw data, such as laplacian noise based on the parameters λ and μ.
Taking epsilon as the weight and multiplying the gradient direction (more than 0 is set as 1, less than 0 is set as-1, and 0 is set when the weight is equal to 0), meanwhile, the iteration times (preset dividing batches) are represented by N, N can be selected according to experience, and N can be taken as 2 initially, namely, only two batches are iterated. The average distribution of the total noise amplitude to each iteration is realized, so that the parameters epsilon and N can also be directly set by epsilon as e/N on the premise of giving the noise amplitude e.
And c, adding the target disturbance amount to the training samples of the corresponding batches to obtain the target training samples corresponding to the current privacy mechanism.
And repeating the steps a-c until a target training sample corresponding to each privacy mechanism is obtained.
In the application process, along with the increase of the training times of the target language model, if the absolute value descending speed of the gradient is lower than a first speed threshold, the target disturbance amount is reduced; and if the absolute value descending speed of the gradient is higher than a second speed threshold value, increasing the target disturbance quantity, wherein the second speed threshold value is larger than the first speed threshold value.
In general, the magnitude of the absolute value of the gradient decreases as the number of model training increases until the final convergence. If the absolute value of the gradient is difficult to converge in the training process, the main task performance of the original model of the question-answering system is poor, and disturbance needs to be reduced. On the contrary, if the gradient descent speed is too fast, the model is easily subjected to differential attack, and the disturbance needs to be increased.
Specifically, if the biaf algorithm satisfies the privacy mechanism M, then for any event E:
Pr(M(S)∈E)≤eεPr(M(S′)∈E)+δ
where S is the question text of the input model. The smaller the epsilon and delta of the model means the higher the privacy, and the specific way to improve the privacy of the BIDAF model is to measure the sensitivity of the gradient and apply a proportional noise according to the magnitude of the gradient. After adding noise to the gradient, an attacker cannot determine whether specific question data corresponding to an answer exists in the training set. In deep neural networks, each iteration sacrifices a portion of privacy in exchange for performance improvement. In the embodiment of the invention, the noise disturbance is added according to the gradient measurement of each batch, and the disturbances with different sizes are added in each batch of data in the training process, so that the purposes of confusing attackers and protecting the training data are achieved.
Specifically, the whole privacy budget epsilon of the language model is set firstly, the corresponding budget can be deducted every time the data is accessed, and the data cannot be accessed when the budget is used up. Sequentially executing a group of privacy mechanisms M-M on the data set SquAD by using a privacy budget composition method of sequential composition1，...MmAnd each MiProviding an epsilon-differential privacy guarantee, the master mechanism M provides an (M x epsilon) differential privacy guarantee.
For query f: d → R and the data sets D and D' before and after the update, the sensitivity Δ f is defined as Δ f ═ maxD，D′| f (D) -f (D') |. Sensitivity is only related to the type of query f, which measures the maximum difference between the query results on the two data sets.
Output results using an exponential mechanism on BIDAF language models
Performing randomization process and performing evaluation function
And correspond to each other. The evaluation function is used for calculating the matching degree of the output answer and the answer set corresponding to the question in the data set. Δ q represents
The exponential mechanism M satisfies the epsilon-differential privacy requirement that the following conditions are satisfied:
for the BIDAF question-answering system, differential privacy in the training process is achieved by modifying an optimizer. In particular, a random gradient is decreased to set the differential privacy perturbation. The privacy of the training set is protected by the parameter gradient that the intervention model uses to update the weights. By adding noise to the gradient in each iteration, the library may prevent the model from remembering the training samples. Over multiple batches of the training process, the unbiased noise will be naturally cancelled.
As an alternative embodiment, the magnitude of the additive perturbation may be controlled by looking at the gradient norm. The gradient of each sample in the smallest lot was calculated. Each gradient is gradient clipped separately, accumulated to a gradient tensor, and then noise is added thereto.
In some embodiments, the aforementioned method further comprises: and verifying the trained target language model based on the accurate matching rate, the harmonic mean value of the accurate rate and the recall rate of the question-answering system and the target disturbance quantity.
It should be noted that the embodiment of the present invention uses three indexes to evaluate the model: and the precision matching (EM), the F1 score and the disturbance size epsilon are respectively used for measuring the weighted average values of the accuracy and the recall rate of the character level, and the prediction precision change of the original model is evaluated under the condition of adding the differential privacy disturbance. The disturbance magnitude is used to measure the amount of noise that needs to be added to achieve the data protection effect.
Exact match ratio (EM): the exact match rate is a common evaluation index of the question-answering system, is used for evaluating the percentage of correct answers matched in the prediction, and is one of the main indexes of the question-answering system based on the SquAD data set.
F1 score: the F1 score is a harmonic mean of the precision ratio and the recall ratio, where the precision ratio represents the proportion of all "correctly discriminated samples (TP)" to all "actually discriminated samples (TP + FP)", that is, the proportion judged to be correct among the samples judged to be member samples.
The recall represents the percentage of successfully predicted member samples to the total number of member samples.
Wherein TP indicates that the positive class is determined as the positive class, FP indicates that the negative class is determined as the positive class, FN indicates that the positive class is determined as the negative class, and TN indicates that the negative class is determined as the negative class. Integrating the accuracy and the recall rate, and obtaining an F1 score to represent:
disturbance size ε: a measure is made of how much noise is added to the gradient during training. The less noise added, the more effective the differential privacy method is in achieving the same protection.
The embodiment of the invention uses a question-answering system with better performance and based on a bidirectional attention flow language understanding model (BIDAF) as a basic model. The method comprises the steps of firstly considering the problem of data privacy disclosure possibly existing in an intelligent question-answering system based on deep learning, reasoning the text data of the question-answering pair for training by comparing different scores before and after model training, and verifying the problem of data privacy disclosure. Aiming at the data leakage attack method, the idea of differential privacy is used, and proper noise is added to the model gradient in training, so that the privacy security of a target model is improved, and the function of protecting the training data of the question-answering system is realized.
As shown in fig. 4, an embodiment of the present invention provides a question-answering system model training apparatus based on a differential privacy technology, where the apparatus includes:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring privacy mechanisms corresponding to a target language model in a question-answering system, the privacy mechanisms comprise at least one, and the privacy mechanisms are used for protecting the differential privacy of a data set of the question-answering system by intervening the target language model;
the adding module is used for obtaining corresponding training samples according to each privacy mechanism and adding target disturbance quantities to the training samples of each preset divided batch corresponding to each privacy mechanism to obtain target training samples corresponding to each privacy mechanism;
and the training module is used for respectively inputting the target training samples into the target language model and training the target language model of the question answering system.
In this embodiment, the electronic device may be, but is not limited to, a Computer device with analysis and processing capabilities, such as a Personal Computer (PC), a notebook Computer, a monitoring device, and a server.
As an exemplary embodiment, referring to fig. 5, the electronic device 110 includes a communication interface 111, a processor 112, a memory 113, and a bus 114, wherein the processor 112, the communication interface 111, and the memory 113 are connected by the bus 114; the memory 113 is used for storing computer programs that support the processor 112 to execute the above-mentioned methods, and the processor 112 is configured to execute the programs stored in the memory 113.
A machine-readable storage medium as referred to herein may be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: a RAM (random Access Memory), a volatile Memory, a non-volatile Memory, a flash Memory, a storage drive (e.g., a hard drive), any type of storage disk (e.g., an optical disk, a dvd, etc.), or similar storage medium, or a combination thereof.
The non-volatile medium may be non-volatile memory, flash memory, a storage drive (e.g., a hard drive), any type of storage disk (e.g., an optical disk, dvd, etc.), or similar non-volatile storage medium, or a combination thereof.
It can be understood that, for the specific operation method of each functional module in this embodiment, reference may be made to the detailed description of the corresponding step in the foregoing method embodiment, and no repeated description is provided herein.
The computer-readable storage medium provided in the embodiments of the present invention stores a computer program, and when executed, the computer program code may implement the method described in any of the above embodiments, and for specific implementation, reference may be made to the method embodiment, which is not described herein again.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In addition, in the description of the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as being fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: those skilled in the art can still make modifications or changes to the embodiments described in the foregoing embodiments, or make equivalent substitutions for some features, within the scope of the disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein.
Claims (10)
1. A question-answering system model training method based on a differential privacy technology is characterized by comprising the following steps:
obtaining privacy mechanisms corresponding to target language models in a question-answering system, wherein the privacy mechanisms comprise at least one privacy mechanism, and the privacy mechanisms intervene in the target language models to protect differential privacy of data sets of the question-answering system;
acquiring a corresponding training sample according to each privacy mechanism, and adding a target disturbance amount to each preset batch of training samples corresponding to each privacy mechanism to obtain a target training sample corresponding to each privacy mechanism;
and respectively inputting the target training samples into the target language model, and training the target language model of the question-answering system.
2. The method according to claim 1, wherein the step of obtaining a corresponding training sample according to each of the privacy mechanisms, and adding a target perturbation amount to each of the preset batch of training samples corresponding to each of the privacy mechanisms to obtain a target training sample corresponding to each of the privacy mechanisms comprises:
obtaining a group of training samples corresponding to a current privacy mechanism;
presetting divided batches according to a current privacy mechanism and training samples corresponding to the current privacy mechanism, and determining a target disturbance amount added to the training samples of each batch;
adding the target disturbance amount to training samples of corresponding batches to obtain target training samples corresponding to the current privacy mechanism;
and repeating the steps until a target training sample corresponding to each privacy mechanism is obtained.
3. The method of claim 2, wherein the step of determining the amount of target disturbance added to each batch of training samples further comprises:
and adjusting the target disturbance amount added to each batch of training samples based on the gradient of a rating function in the privacy mechanism, wherein the training samples comprise question texts and output answers, and the rating function is used for calculating the matching degree of the output answers and answer sets corresponding to the question texts in the data sets.
4. The method of claim 3, wherein the step of adjusting the amount of target perturbation added to each batch of training samples based on the gradient of the merit function in the privacy mechanism comprises:
with the increase of the training times of the target language model, if the absolute value descending speed of the gradient is lower than a first speed threshold value, reducing the target disturbance amount; and if the absolute value descending speed of the gradient is higher than a second speed threshold value, increasing the target disturbance quantity, wherein the second speed threshold value is larger than the first speed threshold value.
5. The method of claim 1, further comprising:
and verifying the trained target language model based on the accurate matching rate, the harmonic mean value of the accurate rate and the recall rate of the question-answering system and the target disturbance quantity.
6. The method of claim 1, wherein the disturbance amount comprises noise conforming to a laplacian distribution, a gaussian distribution, or an exponential distribution.
7. The method of claim 1, wherein the target language model is a bi-directional attention flow model.
8. A question-answering system model training device based on a differential privacy technology is characterized by comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring privacy mechanisms corresponding to a target language model in a question-answering system, the privacy mechanisms comprise at least one, and the privacy mechanisms are used for protecting the differential privacy of a data set of the question-answering system by intervening the target language model;
the adding module is used for obtaining corresponding training samples according to each privacy mechanism and adding target disturbance quantities to the training samples of each preset divided batch corresponding to each privacy mechanism to obtain target training samples corresponding to each privacy mechanism;
and the training module is used for respectively inputting the target training samples into the target language model and training the target language model of the question-answering system.
9. An electronic device comprising a memory, a processor, and a program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1 to 7 when executing the program.
10. A computer-readable storage medium, characterized in that a computer program is stored in the readable storage medium, which computer program, when executed, implements the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210159711.8A CN114547687A (en) | 2022-02-22 | 2022-02-22 | Question-answering system model training method and device based on differential privacy technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210159711.8A CN114547687A (en) | 2022-02-22 | 2022-02-22 | Question-answering system model training method and device based on differential privacy technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114547687A true CN114547687A (en) | 2022-05-27 |
Family
ID=81677447
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210159711.8A Pending CN114547687A (en) | 2022-02-22 | 2022-02-22 | Question-answering system model training method and device based on differential privacy technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114547687A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111079938A (en) * | 2019-11-28 | 2020-04-28 | 百度在线网络技术（北京）有限公司 | Question-answer reading understanding model obtaining method and device, electronic equipment and storage medium |
| CN112199479A (en) * | 2020-09-15 | 2021-01-08 | 北京捷通华声科技股份有限公司 | Method, device and equipment for optimizing language semantic understanding model and storage medium |
| CN113204793A (en) * | 2021-06-09 | 2021-08-03 | 辽宁工程技术大学 | Recommendation method based on personalized differential privacy protection |
| CN113342953A (en) * | 2021-06-18 | 2021-09-03 | 北京理工大学东南信息技术研究院 | Government affair question and answer method based on multi-model integration |
| CN113360917A (en) * | 2021-07-07 | 2021-09-07 | 浙江工业大学 | Deep reinforcement learning model security reinforcement method and device based on differential privacy |
| CN113961967A (en) * | 2021-12-13 | 2022-01-21 | 支付宝(杭州)信息技术有限公司 | Method and device for jointly training natural language processing model based on privacy protection |
-
2022
- 2022-02-22 CN CN202210159711.8A patent/CN114547687A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111079938A (en) * | 2019-11-28 | 2020-04-28 | 百度在线网络技术（北京）有限公司 | Question-answer reading understanding model obtaining method and device, electronic equipment and storage medium |
| CN112199479A (en) * | 2020-09-15 | 2021-01-08 | 北京捷通华声科技股份有限公司 | Method, device and equipment for optimizing language semantic understanding model and storage medium |
| CN113204793A (en) * | 2021-06-09 | 2021-08-03 | 辽宁工程技术大学 | Recommendation method based on personalized differential privacy protection |
| CN113342953A (en) * | 2021-06-18 | 2021-09-03 | 北京理工大学东南信息技术研究院 | Government affair question and answer method based on multi-model integration |
| CN113360917A (en) * | 2021-07-07 | 2021-09-07 | 浙江工业大学 | Deep reinforcement learning model security reinforcement method and device based on differential privacy |
| CN113961967A (en) * | 2021-12-13 | 2022-01-21 | 支付宝(杭州)信息技术有限公司 | Method and device for jointly training natural language processing model based on privacy protection |
Non-Patent Citations (8)
| Title |
|---|
| PEI LIU等: "Attention-Based Memory Network for Sentence-Level Question Answering", 《SPRINGLINK》 * |
| 任琪宇: "基于机器阅读理解的开放域问答技术研究", 《中国优秀硕士学位论文全文数据库》 * |
| 刘宝锺: "《大数据分类模型和算法研究》", 31 January 2020, 云南大学出版社 * |
| 吉久明: "《知识发现 科技文献内容挖掘技术研究》", 31 July 2017, 上海科学技术文献出版社 * |
| 王宏力: "《惯性测量组合智能故障诊断及预测技术》", 31 May 2017, 国防工业出版社 * |
| 纪守领: "机器学习模型安全与隐私研究综述", 《软件学报》 * |
| 许斌等: "大数据环境中非交互式查询差分隐私保护模型", 《计算机工程与应用》 * |
| 高志强: "《边缘智能 关键技术与落地实践》", 31 May 2021, 中国铁道出版社 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Thiesson et al. | Learning mixtures of DAG models | |
| CN107506414B (en) | Code recommendation method based on long-term and short-term memory network | |
| Kadlec et al. | Text understanding with the attention sum reader network | |
| CN110929515B (en) | Reading understanding method and system based on cooperative attention and adaptive adjustment | |
| CN109783817B (en) | Text semantic similarity calculation model based on deep reinforcement learning | |
| Guu et al. | Traversing knowledge graphs in vector space | |
| CN110188272B (en) | Community question-answering website label recommendation method based on user background | |
| CN110598206A (en) | Text semantic recognition method and device, computer equipment and storage medium | |
| US20220129621A1 (en) | Bert-based machine-learning tool for predicting emotional response to text | |
| US11397892B2 (en) | Method of and system for training machine learning algorithm to generate text summary | |
| EP3619651A1 (en) | System and method for batch-normalized recurrent highway networks | |
| CN107579821B (en) | Method for generating password dictionary and computer-readable storage medium | |
| CN110516210B (en) | Text similarity calculation method and device | |
| CN112417153A (en) | Text classification method and device, terminal equipment and readable storage medium | |
| Markou et al. | Ex machina lex: Exploring the limits of legal computability | |
| Thomas et al. | Chatbot using gated end-to-end memory networks | |
| CN115146068B (en) | Method, device, equipment and storage medium for extracting relation triples | |
| CN114648032B (en) | Training method and device of semantic understanding model and computer equipment | |
| Vu et al. | dpUGC: Learn Differentially Private Representation for User Generated Contents (Best Paper Award, Third Place, Shared) | |
| CN116432184A (en) | Malicious software detection method based on semantic analysis and bidirectional coding characterization | |
| CN115496144A (en) | Power distribution network operation scene determining method and device, computer equipment and storage medium | |
| US20220374606A1 (en) | Systems and methods for utility-preserving deep reinforcement learning-based text anonymization | |
| CN116720519B (en) | Seedling medicine named entity identification method | |
| CN114547687A (en) | Question-answering system model training method and device based on differential privacy technology | |
| CN112738098A (en) | Anomaly detection method and device based on network behavior data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220527 |
|
| RJ01 | Rejection of invention patent application after publication |