CN115374863A - Sample generation method, sample generation device, storage medium and equipment - Google Patents
Sample generation method, sample generation device, storage medium and equipment Download PDFInfo
- Publication number
- CN115374863A CN115374863A CN202211024828.1A CN202211024828A CN115374863A CN 115374863 A CN115374863 A CN 115374863A CN 202211024828 A CN202211024828 A CN 202211024828A CN 115374863 A CN115374863 A CN 115374863A
- Authority
- CN
- China
- Prior art keywords
- sample
- encoder
- samples
- target
- variational self
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The application discloses a sample generation method, a sample generation device, a storage medium and equipment, wherein the method comprises the following steps: identifying a sample input by a user in advance as a sample to be processed; inputting a sample to be processed into a variational self-encoder obtained by pre-training to obtain a sample set output by the variational self-encoder; wherein, the variational self-encoder comprises an encoder and a decoder; the sample set includes other samples similar to the sample to be processed. The application inputs the sample to be processed into the variational self-encoder obtained by pre-training, the sample set output by the variational self-encoder is obtained, in the training process of the variational self-encoder, the difference between the target sample and each sample is taken as a training target, and the parameter of the loss function of the variational self-encoder is adjusted, so that the calculation complexity of the variational self-encoder is lower, compared with the existing sample generation model, the calculation process of the variational self-encoder is simpler, and the quality of the generated sample set is higher.
Description
Technical Field
The present application relates to the field of data processing, and in particular, to a method, an apparatus, a storage medium, and a device for generating a sample.
Background
With the development of information technology, machine learning is increasingly applied to various business fields of banks, machine learning often needs a large amount of millions or even tens of millions of data supports, in a production environment of banks, data sets in certain fields and occasions have the problems of small sample number, data loss and the like, and the machine learning is difficult to apply due to the small sample (i.e., the sample with small number) training problem.
At present, the mainstream sample generation model mainly generates a countermeasure network and a flow model. The generation of the countermeasure network needs complex parameter adjustment, and the generated samples are large in size and poor in diversity and are not suitable for small sample generation scenes. The sample generating function of the flow model is complex, accurate design and calculation are needed, the universality is poor, and the method is not applicable.
Therefore, how to improve the sample quality while reducing the computational complexity of the sample generation process is an urgent problem to be solved in the art.
Disclosure of Invention
The application provides a sample generation method, a sample generation device, a storage medium and a sample generation device, which are used for improving the quality of a sample under the condition of reducing the calculation complexity of a sample generation process.
In order to achieve the above object, the present application provides the following technical solutions:
a sample generation method, comprising:
identifying a sample input by a user in advance as a sample to be processed;
inputting the sample to be processed into a variational self-encoder obtained by pre-training to obtain a sample set output by the variational self-encoder; wherein the variational self-encoder comprises an encoder and a decoder; the sample set comprises other samples similar to the sample to be processed; the training process of the variational self-encoder comprises the following steps:
inputting a plurality of samples belonging to the same field, which are acquired in advance, into the encoder to obtain an encoding result output by the encoder; the coding result comprises a mean and a variance of the hidden variables associated with each of the samples;
obtaining a target hidden variable distribution based on the mean and variance of the hidden variables associated with each of the samples;
inputting the target hidden variable distribution into the decoder, sampling the target hidden variable distribution through the decoder to obtain target samples, and adjusting parameters of a loss function of the variational self-encoder by taking the difference between the target samples and each sample as a training target; the parameter comprises an expected value of the difference between the target sample and each of the samples;
and determining that the training of the variational self-encoder is finished under the condition that the value of the loss function is smaller than a preset threshold value.
Optionally, the encoder includes a first encoder and a second encoder;
the inputting a plurality of samples belonging to the same field, which are obtained in advance, into the encoder to obtain an encoding result output by the encoder includes:
grouping a plurality of samples belonging to the same field, which are obtained in advance, to obtain a first sample set and a second sample set;
inputting the samples in the first sample set into the first encoder, and learning the samples in the first sample set through the first encoder to obtain a first encoding result; the first encoding result comprises a mean and a variance of implicit variables associated with each sample in the first set of samples;
inputting the samples in the second sample set into the second encoder, and learning the samples in the second sample set through the second encoder to obtain a second encoding result; the second encoding result includes a mean and a variance of an implicit variable associated with each sample in the second set of samples.
Optionally, the obtaining a target hidden variable distribution based on the mean and the variance of the hidden variables associated with each sample includes:
generating a first hidden variable distribution based on a mean and variance of hidden variables associated with each sample in the first set of samples;
generating a second hidden variable distribution based on the mean and variance of the hidden variables associated with each sample in the second set of samples;
and generating target hidden variable distribution based on the first hidden variable distribution and the second hidden variable distribution.
Optionally, the method further includes:
and displaying other samples similar to the sample to be processed to the user through a preset interface.
A sample generation device, comprising:
the device comprises a sample acquisition unit, a processing unit and a processing unit, wherein the sample acquisition unit is used for identifying a sample input by a user in advance as a sample to be processed;
the sample generating unit is used for inputting the sample to be processed into a variational self-encoder obtained by pre-training to obtain a sample set output by the variational self-encoder; wherein the variational self-encoder comprises an encoder and a decoder; the sample set comprises other samples similar to the sample to be processed; the training process of the variational self-encoder comprises the following steps:
inputting a plurality of samples which are acquired in advance and belong to the same field into the encoder to obtain an encoding result output by the encoder; the coding result comprises a mean and a variance of the hidden variables associated with each of the samples;
obtaining a target hidden variable distribution based on the mean and variance of the hidden variables associated with each of the samples;
inputting the target hidden variable distribution into the decoder, sampling the target hidden variable distribution through the decoder to obtain target samples, and adjusting parameters of a loss function of the variational self-encoder by taking the difference between the target samples and each sample as a training target; the parameter comprises an expected value of the difference between the target sample and each of the samples;
and determining that the training of the variational self-encoder is finished under the condition that the value of the loss function is smaller than a preset threshold value.
Optionally, the encoder includes a first encoder and a second encoder;
the sample generation unit is specifically configured to:
grouping a plurality of pre-acquired samples belonging to the same field to obtain a first sample set and a second sample set;
inputting the samples in the first sample set into the first encoder, and learning the samples in the first sample set through the first encoder to obtain a first encoding result; the first encoding result comprises a mean and a variance of implicit variables associated with each sample in the first set of samples;
inputting the samples in the second sample set into the second encoder, and learning the samples in the second sample set through the second encoder to obtain a second encoding result; the second encoding result includes a mean and a variance of the hidden variables associated with each sample in the second set of samples.
Optionally, the sample generating unit is specifically configured to:
generating a first hidden variable distribution based on a mean and a variance of hidden variables associated with each sample in the first set of samples;
generating a second latent variable distribution based on the mean and variance of latent variables associated with each sample in the second set of samples;
and generating target hidden variable distribution based on the first hidden variable distribution and the second hidden variable distribution.
Optionally, the method further includes:
and the sample display unit is used for displaying other samples similar to the sample to be processed to the user through a preset interface.
A computer-readable storage medium comprising a stored program, wherein the program performs the sample generation method.
A sample generation device, comprising: a processor, memory, and a bus; the processor and the memory are connected through the bus;
the memory is used for storing a program, and the processor is used for executing the program, wherein the program executes the sample generation method during running.
According to the technical scheme, the samples input by the user in advance are marked as the samples to be processed.
Inputting a sample to be processed into a variational self-encoder obtained by pre-training to obtain a sample set output by the variational self-encoder; wherein, the variational self-encoder comprises an encoder and a decoder; the sample set comprises other samples similar to the sample to be processed; the training process of the variational self-encoder comprises the following steps: inputting a plurality of samples belonging to the same field, which are acquired in advance, into an encoder to obtain an encoding result output by the encoder; the coding result includes the mean and variance of the hidden variables associated with each sample; obtaining a target hidden variable distribution based on the mean and variance of the hidden variables associated with each sample; inputting the target hidden variable distribution into a decoder, sampling the target hidden variable distribution through the decoder to obtain target samples, and adjusting parameters of a loss function of a variational self-encoder by taking the difference between the target samples and each sample as a training target; the parameters include expected values of differences between the target sample and the respective samples; and determining that the training of the variational self-encoder is finished under the condition that the value of the loss function is smaller than a preset threshold value. The application inputs the sample to be processed into the variational self-encoder obtained by pre-training, the sample set output by the variational self-encoder is obtained, in the training process of the variational self-encoder, the difference between the target sample and each sample is taken as a training target, and the parameter of the loss function of the variational self-encoder is adjusted, so that the calculation complexity of the variational self-encoder is lower, compared with the existing sample generation model, the calculation process of the variational self-encoder is simpler, and the quality of the generated sample set is higher.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1a is a schematic flowchart of a sample generation method according to an embodiment of the present disclosure;
fig. 1b is a schematic flow chart of a sample generation method according to an embodiment of the present disclosure;
fig. 2 is a schematic flow chart of another sample generation method provided in the embodiment of the present application;
fig. 3 is a schematic diagram of an architecture of a sample generation apparatus according to an embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
As shown in fig. 1a and fig. 1b, a schematic flow chart of a sample generation method provided in an embodiment of the present application includes the following steps.
S101: grouping a plurality of samples belonging to the same field, which are obtained in advance, to obtain a first sample set and a second sample set.
The number of samples included in the first sample set and the number of samples included in the second sample set can be set by a technician according to actual situations.
It should be noted that the samples shown in the embodiments of the present application include, but are not limited to, sample data in the financial field, such as alarm data in the financial field, customer preference data, and the like.
S102: a first Encoder, a second Encoder and a decoder included in a preset Variational Auto Encoder (VAE) are obtained.
The specific implementation principle of the variational self-encoder is as follows:
assuming a data set
The number of data set samples is N, x is a discrete or continuous variable, and can be observed as a variable. Assuming that x can be generated by existence of a hidden variable z, we can obtain the variable x by solving the probability distribution of z, but since only x can be observed, the probability distribution of z cannot be solved, so that the probability distribution of z, namely p (z | x), is solved under the condition of given x, and the formula (1) can be obtained according to the Bayesian formula.
However, since p (x) cannot be calculated, the posterior probability p (z | x) is not solved, and for this reason, equation (1) is further derived from the total probability equation to obtain equation (2).
p(x)＝∫p(x|z)p(z)dz (2)
For the solution problem of equation (2), there are generally two solutions. The first way is the monte carlo method, which approximates the true distribution of p (x) z by continually sampling, the more sampling is taken, the closer to the true distribution of p (x). The second way is to solve the maximum likelihood function, because the log-likelihood function cannot be solved accurately, the lower bound of the log-likelihood function can be solved, and the log-likelihood function is made to be maximum through the maximum lower bound. The specific method comprises the following steps: introducing a new probability distribution q φ (z|x (i) ) To approximate p θ (x (i) ) To make q equal φ (z|x (i) ) As close as possible to p θ (x (i) ) The log-likelihood function at this time is shown in equation (3).
Although the three parameters on the right side of the middle number in the formula (3) cannot be used for the technique, the D can be obtained according to the property of the K-L divergence KL (q φ (z|x (i) )||p θ (z|x (i) ) Is prepared fromPositive real numbers, for which purpose equation (3) can be converted to that shown in equation (4).
log[p θ (x (i) )]≥E z [logp θ (x (i) |z)]-D KL (q φ (z|x (i) )||p θ (z)) (4)
The right side of equation (4) is called the lower variational bound, denoted L (x) (i) (ii) a θ, φ), the lower bound of variation is used as a loss function to obtain equation (5).
The second term of the lower bound of the variation is to calculate the prior distribution p of the hidden variable θ (z) and approximate distribution q of a posterior distribution φ (z|x (i) ) K-L divergence of (1). Two assumptions can be made here, the first being that the prior distribution of the hidden variables is a standard normal distribution N (0, 1) in d-dimension. The second is for each true sample x (i) All have a special assignment of x (i) Approximate distribution q of the posterior distribution of (1) φ (z|x (i) ) I.e. for each sample x (i) All correspond to a normal distribution N (mu, sigma) 2 ；x (i) ). So now only mu (x) needs to be solved (i) ) And σ 2 (x (i) ) The K-L divergence can be obtained, the embodiment of the application uses two neural networks to fit and solve the mean value and the variance, but when in solution, because the value range of the logarithm of the variance is the whole real number, the value range of the activation function does not need to be considered when in solution by using the neural network fitting, and the logarithm of the mean value and the variance is finally solved. Mean μ (x) since each dimension of the hidden variables is independent of each other (i) ) The output of the neural network responsible for calculating the mean value is shown in equation (6), for a vector of dimension d.
[μ 1 ,μ 2 ,μ 3 ,...,μ d ] (6)
Variance σ 2 (x (i) ) Then it is a diagonal matrix of d dimensions, as shown in equation (7).
The range of the variance is the total number of positive real numbers, the range of the activation function needs to be accurately designed when the variance is solved by using neural network fitting, and the range of the logarithm of the variance is the total number of real numbers, so that the logarithm of the variance is solved, and the output of the neural network responsible for calculating the variance can be shown in formula (8).
The formula (6) and the formula (8) together constitute an encoder as shown in the formula (9).
Since each dimension of the two Gaussian distributions is independent of the other, the K-L divergence can be calculated separately, with the K-L divergence for the d-th dimension shown in equation (10).
The mean value and the variance of the approximate distribution of the posterior distribution of the hidden variables are fitted by learning respectively through the two coders, so that the approximate distribution q can be obtained φ (z|x (i) ) Thus, the K-L divergence can be calculated. The goal of passing the training coder for VAE is to expect the K-L divergence to be minimal, i.e., for each sample, the approximate distribution q of the posterior distribution φ (z|x (i) ) Approaching the standard normal distribution N (0, 1).
The calculation of the first term of the lower bound of the variation requires the use of an empirical approximation, as shown in equation (11).
E z [logp θ (x (i) |z)]≈logp θ (x (i) |z) (11)
That is to say in the calculationWhen the first term of the lower bound of the variation is divided, all z are not required to be sampled for calculation, and only one sampling is required. To calculate logp θ (x (i) Z), suppose p θ (x (i) I z) satisfies Bernoulli distribution or normal distribution according to p θ (x (i) I z) satisfy different distributions, the calculation of the decoder is different accordingly.
If p is θ (x (i) I z) is Bernoulli distribution, corresponding x (i) The vectors are binary, q-dimensional, independent vectors, and when q parameters of bernoulli distribution are learned through a neural network, the neural network is a decoder, the input is a hidden variable z, and the output is shown in formula (12).
[ρ 1 ,ρ 2 ,...,ρ q ] (12)
The representation of the decoder can be shown in equation (13).
ρ(z)＝dec(z) (13)
The calculation formula of the likelihood of the sample is shown as formula (14).
The corresponding formula for calculating the log-likelihood is shown in formula (15).
When designing the decoder neural network, only the Sigmod function is needed to be used as the activation function of the last layer of the encoder, and then the loss function of the decoder is set as the two-class cross entropy.
If p is θ (x (i) | z) normal distribution, corresponding to x (i) Is a vector with real number and q dimensions independent from each other, and the variance of each dimension of the normal distribution is a constant and is marked as sigma 2 When q mean parameters are learned through the neural network, the neural network is a decoder, the input is a hidden variable z, and the output is shown as formula (16).
[μ 1 ,μ 2 ,μ 3 ,...,μ d ] (16)
The representation of the decoder can be shown in equation (17).
μ(z)＝dec(z) (17)
The calculation formula of the likelihood of the sample is shown as formula (18).
The corresponding formula for calculating the log-likelihood is shown in formula (19).
Only the activation function with the value range of the whole real number is used as the activation function, and then the MSE is used as the loss function.
The forward derivation process of the entire VAE is: sample x (i) Sending into a coder to calculate q φ (z|x (i) ) The mean and variance of the hidden variables are obtained, and the approximate distribution of the posterior distribution of the hidden variables is obtained, so that a hidden variable z can be sampled from the approximate distribution, and then the z is sent to a decoder to calculate the update parameters of the loss function.
However, in the whole process, there is a problem that the sampling process from the approximate distribution of the posterior distribution of the hidden variables is not conductive, that is, the mean and variance calculated by the encoder cannot be accurately transmitted to the decoder, and that z obtained from which normal distribution is sampled is not known and therefore cannot be propagated backward.
To solve this problem, VAE uses a heavily parametric technique, i.e., sampling directly in the standard normal distribution N (0, 1) during the sampling process, and then passing the parameters of mean and variance learned by the encoder, as shown in equation (20).
z＝μ+ε×σ (20)
Therefore, the sampling process is changed into sampling from the standard normal distribution N (0, 1), and then the result is obtained through parameter transformation, the sampling process does not need to participate in gradient descent, so that the parameters can be propagated reversely, and the whole model becomes trainable.
S103: and inputting the samples in the first sample set into a first encoder, and learning the samples in the first sample set through the first encoder to obtain a first encoding result.
The first encoder includes two neural networks, and specifically, the expression form of the first encoder may be shown in formula (21).
In equation (21), μ (X) represents the mean of the hidden variables associated with sample X, log σ 2 (X) represents the variance, enc, of the hidden variable associated with sample X 1 (X) and Enc 2 (X) each represents a neural network.
In an embodiment of the present application, the first encoding result includes a mean of the hidden variables associated with each sample in the first set of samples and a variance of the hidden variables associated with each sample in the first set of samples.
It should be noted that, when learning samples in the first sample set, the essence is: and performing mean fitting on the normal distribution of each sample in the first sample set by using one neural network to obtain a mean value of the hidden variable associated with each sample, and performing variance fitting on the normal distribution of each sample in the first sample set by using the other neural network to obtain a variance of the hidden variable associated with each sample. Generally, the hidden variables associated with a sample are sampled from the normal distribution of the sample.
S104: and inputting the samples in the second sample set into a second encoder, and learning the samples in the second sample set through the second encoder to obtain a second encoding result.
The second encoder comprises two neural networks, and the first encoder and the second encoder have the same structure, so that the second encoder has the same expression form as the first encoder.
In an embodiment of the present application, the second encoding result includes a mean of the hidden variables associated with each sample in the second set of samples and a variance of the hidden variables associated with each sample in the second set of samples.
It should be noted that, the essence of learning the samples in the second sample set is: and performing mean fitting on the normal distribution of each sample in the second sample set by using one neural network to obtain a mean value of the hidden variable associated with each sample, and performing variance fitting on the normal distribution of each sample in the second sample set by using the other neural network to obtain a variance of the hidden variable associated with each sample.
S105: a first hidden variable distribution is generated based on the mean and variance of the hidden variables associated with each sample in the first set of samples.
S106: a second hidden variable distribution is generated based on the mean and variance of the hidden variables associated with each sample in the second set of samples.
S107: and generating target hidden variable distribution based on the first hidden variable distribution and the second hidden variable distribution.
The first hidden variable distribution and the second hidden variable distribution respectively represent data information of different samples in a hidden space, and if one hidden variable distribution is used independently to learn the variational self-encoder, the learning process of the variational self-encoder is incomplete.
Specifically, assume that the first hidden variable distribution is
The second hidden variable distribution is
Will be provided with
And
combining to generate target hidden variable distribution
It should be noted that the dimension of the target hidden variable distribution is equal to the sum of the dimension of the first hidden variable distribution and the dimension of the second hidden variable distribution, and specifically, assuming that the dimension of the first hidden variable distribution is 10 and the dimension of the second hidden variable distribution is 10, the dimension of the target hidden variable distribution is 20.
S108: and inputting the target hidden variable distribution into a decoder, sampling the target hidden variable distribution through the decoder to obtain target samples, and adjusting parameters of a loss function of the variational self-encoder by taking the difference between the target samples and each sample as a training target.
Wherein the decoder comprises a neural network and the loss function of the variational self-encoder comprises a combination of a K-L divergence, a first reconstruction loss function and a second reconstruction loss function.
In the embodiment of the present application, the K-L divergence is expressed in the following formula (22).
In the formula (22), the parameters of the K-L divergence may refer to the specific implementation principle of the variational self-encoder mentioned in the above S102, and are not described herein again.
The first reconstruction loss function is expressed in the following equation (23).
In equation (23), res 1 Representing a first reconstruction loss function of the first set of coefficients,
an expected value, X, representing the difference between the target sample and each sample in the first set of samples ti Represents a sample in the first set of samples and X represents a target sample.
The second reconstruction loss function is expressed in the form shown in equation (24).
In equation (24), res 2 Representing the second reconstruction loss function and,
representing an expected value, X, for the difference between the target sample and each sample in the second set of samples si Represents a sample in the second set of samples and X represents a target sample.
The combination of the K-L divergence, the first reconstruction loss function, and the second reconstruction loss function is specifically shown in equation (25).
Loss＝KL(q φ (Z|X)N(0,1))+λRes 1 +(1-λ)Res 2 (25)
In equation (25), loss represents the Loss function of the variational self-encoder, KL (q) φ (Z | X) | N (0, 1)) represents K-L divergence, i.e., KL (q) | N (0, 1)) φ (Z | X) N (0, 1)) is equal to
λ represents a preset hyper-parameter, and the value range of λ is [0, 1')]And λ is used to adjust the respective weights of the first reconstruction loss function and the second reconstruction loss function.
S109: and determining that the training of the variational self-encoder is finished under the condition that the value of the loss function is smaller than a preset threshold value.
S110: and identifying the samples input by the user in advance as samples to be processed.
S111: and inputting the sample to be processed into a variational self-encoder obtained by pre-training to obtain a sample set output by the variational self-encoder.
Wherein the sample set comprises other samples similar to the sample to be processed.
S112: and displaying other samples similar to the sample to be processed to the user through a preset interface.
Based on the above-mentioned process shown in S101-S112, this embodiment can improve the implicit variable structure and the loss function of the variational self-encoder from the data characteristics of the sample, fully utilize the prior knowledge of the sample, and train out a variational self-encoder with better generalization capability and lower computation complexity, so as to improve the quality of the sample set generated by the variational self-encoder.
In summary, in the embodiment, the samples to be processed are input into the variational self-encoder obtained by pre-training, so as to obtain the sample set output by the variational self-encoder, and in the training process of the variational self-encoder, the difference between the target sample and each sample is used as a training target, and the parameter of the loss function of the variational self-encoder is adjusted, so that the computational complexity of the variational self-encoder is low.
It should be noted that, in the above embodiment, reference to S101 is an optional implementation manner of the sample generation method shown in the embodiment of the present application. In addition, S112 mentioned in the above embodiments is also an optional implementation manner of the sample generation method shown in the embodiments of the present application. For this reason, the flow mentioned in the above embodiment can be summarized as the method shown in fig. 2.
As shown in fig. 2, a schematic flow chart of another sample generation method provided in the embodiment of the present application includes the following steps.
S201: and identifying the sample input by the user in advance as a sample to be processed.
S202: and inputting the sample to be processed into a variational self-encoder obtained by pre-training to obtain a sample set output by the variational self-encoder.
The variational self-encoder comprises an encoder and a decoder.
The sample set comprises other samples similar to the sample to be processed; the training process of the variational self-encoder comprises the following steps: inputting a plurality of samples belonging to the same field, which are acquired in advance, into an encoder to obtain an encoding result output by the encoder; the coding result includes the mean and variance of the hidden variables associated with each sample; obtaining a target hidden variable distribution based on the mean and variance of the hidden variables associated with each sample; inputting the target hidden variable distribution into a decoder, sampling the target hidden variable distribution through the decoder to obtain target samples, and adjusting parameters of a loss function of a variational self-encoder by taking the difference between the target samples and each sample as a training target; the parameters include expected values of differences between the target sample and the respective samples; and determining that the training of the variational self-encoder is finished under the condition that the value of the loss function is smaller than a preset threshold value.
In summary, in the embodiment, the samples to be processed are input into the variational self-encoder obtained by pre-training, so as to obtain the sample set output by the variational self-encoder, and in the training process of the variational self-encoder, the difference between the target sample and each sample is used as a training target, and the parameter of the loss function of the variational self-encoder is adjusted, so that the computational complexity of the variational self-encoder is low.
Corresponding to the sample generation method provided by the embodiment of the application, the embodiment of the application also provides a sample generation device.
Fig. 3 is a schematic diagram of an architecture of a sample generation apparatus provided in an embodiment of the present application, which includes the following units.
The sample acquiring unit 100 is configured to identify a sample input by a user in advance as a sample to be processed.
The sample generating unit 200 is configured to input a sample to be processed into a variational self-encoder obtained through pre-training, so as to obtain a sample set output by the variational self-encoder; wherein, the variational self-encoder comprises an encoder and a decoder; the sample set comprises other samples similar to the sample to be processed; the training process of the variational self-encoder comprises the following steps: inputting a plurality of samples belonging to the same field, which are acquired in advance, into an encoder to obtain an encoding result output by the encoder; the coding result includes the mean and variance of the hidden variables associated with each sample; obtaining a target hidden variable distribution based on the mean and variance of the hidden variables associated with each sample; inputting the target hidden variable distribution into a decoder, sampling the target hidden variable distribution through the decoder to obtain target samples, and adjusting parameters of a loss function of a variational self-encoder by taking the difference between the target samples and each sample as a training target; the parameters include expected values of differences between the target sample and the respective samples; and determining that the training of the variational self-encoder is finished under the condition that the value of the loss function is smaller than a preset threshold value.
Optionally, the encoder comprises a first encoder and a second encoder.
The sample generation unit 200 is specifically configured to: grouping a plurality of pre-acquired samples belonging to the same field to obtain a first sample set and a second sample set; inputting the samples in the first sample set into a first encoder, and learning the samples in the first sample set through the first encoder to obtain a first encoding result; the first encoding result includes a mean and a variance of the hidden variables associated with each sample in the first set of samples; inputting the samples in the second sample set into a second encoder, and learning the samples in the second sample set through the second encoder to obtain a second encoding result; the second encoding result includes a mean and a variance of the hidden variable associated with each sample in the second set of samples.
The sample generation unit 200 is specifically configured to: generating a first hidden variable distribution based on the mean and variance of the hidden variables associated with each sample in the first set of samples; generating a second hidden variable distribution based on the mean and variance of the hidden variables associated with each sample in the second set of samples; and generating target hidden variable distribution based on the first hidden variable distribution and the second hidden variable distribution.
The sample display unit 300 is configured to display other samples similar to the sample to be processed to the user through a preset interface.
In summary, in the embodiment, the samples to be processed are input into the variational self-encoder obtained by pre-training, so as to obtain the sample set output by the variational self-encoder, and in the training process of the variational self-encoder, the difference between the target sample and each sample is used as a training target, and the parameter of the loss function of the variational self-encoder is adjusted, so that the computational complexity of the variational self-encoder is low.
The present application also provides a computer-readable storage medium comprising a stored program, wherein the program performs the sample generation method provided herein above.
The present application further provides a sample generation device comprising: a processor, a memory, and a bus. The processor is connected with the memory through a bus, the memory is used for storing programs, and the processor is used for running the programs, wherein when the programs are run, the sample generation method provided by the application is executed, and the method comprises the following steps:
identifying a sample input by a user in advance as a sample to be processed;
inputting the sample to be processed into a variational self-encoder obtained by pre-training to obtain a sample set output by the variational self-encoder; wherein the variational self-encoder comprises an encoder and a decoder; the sample set comprises other samples similar to the sample to be processed; the training process of the variational self-encoder comprises the following steps:
inputting a plurality of samples belonging to the same field, which are acquired in advance, into the encoder to obtain an encoding result output by the encoder; the coding result comprises a mean and a variance of the hidden variables associated with each of the samples;
obtaining a target hidden variable distribution based on the mean and variance of the hidden variables associated with each of the samples;
inputting the target hidden variable distribution into the decoder, sampling the target hidden variable distribution through the decoder to obtain target samples, and adjusting parameters of a loss function of the variational self-encoder by taking the difference between the target samples and each sample as a training target; the parameter comprises an expected value of the difference between the target sample and each of the samples;
and determining that the training of the variational self-encoder is finished under the condition that the value of the loss function is smaller than a preset threshold value.
Specifically, on the basis of the above embodiment, the encoder includes a first encoder and a second encoder;
the inputting a plurality of pre-acquired samples belonging to the same field into the encoder to obtain an encoding result output by the encoder includes:
grouping a plurality of samples belonging to the same field, which are obtained in advance, to obtain a first sample set and a second sample set;
inputting the samples in the first sample set into the first encoder, and learning the samples in the first sample set through the first encoder to obtain a first encoding result; the first encoding result comprises a mean and a variance of implicit variables associated with each sample in the first set of samples;
inputting the samples in the second sample set into the second encoder, and learning the samples in the second sample set through the second encoder to obtain a second encoding result; the second encoding result includes a mean and a variance of an implicit variable associated with each sample in the second set of samples.
Specifically, on the basis of the foregoing embodiment, the obtaining a target hidden variable distribution based on the mean and the variance of the hidden variables associated with each sample includes:
generating a first hidden variable distribution based on a mean and variance of hidden variables associated with each sample in the first set of samples;
generating a second latent variable distribution based on the mean and variance of latent variables associated with each sample in the second set of samples;
and generating target hidden variable distribution based on the first hidden variable distribution and the second hidden variable distribution.
Specifically, on the basis of the above embodiment, the method further includes:
and displaying other samples similar to the sample to be processed to the user through a preset interface.
The functions described in the method of the embodiment of the present application, if implemented in the form of software functional units and sold or used as independent products, may be stored in a storage medium readable by a computing device. Based on such understanding, part of the contribution to the prior art of the embodiments of the present application or part of the technical solution may be embodied in the form of a software product stored in a storage medium and including several instructions for causing a computing device (which may be a personal computer, a server, a mobile computing device or a network device) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: u disk, removable hard disk, read only memory, random access memory, magnetic or optical disk, etc. for storing program codes.
In the present specification, the embodiments are described in a progressive manner, and each embodiment focuses on differences from other embodiments, and the same or similar parts between the embodiments are referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method of generating a sample, comprising:
identifying a sample input by a user in advance as a sample to be processed;
inputting the sample to be processed into a variational self-encoder obtained by pre-training to obtain a sample set output by the variational self-encoder; wherein the variational self-encoder comprises an encoder and a decoder; the sample set comprises other samples similar to the sample to be processed; the training process of the variational self-encoder comprises the following steps:
inputting a plurality of samples belonging to the same field, which are acquired in advance, into the encoder to obtain an encoding result output by the encoder; the coding result comprises a mean and a variance of the hidden variables associated with each of the samples;
obtaining a target hidden variable distribution based on the mean and variance of the hidden variables associated with each of the samples;
inputting the target hidden variable distribution into the decoder, sampling the target hidden variable distribution through the decoder to obtain target samples, and adjusting parameters of a loss function of the variational self-encoder by taking the difference between the target samples and each sample as a training target; the parameter comprises an expected value of the difference between the target sample and each of the samples;
and determining that the training of the variational self-encoder is finished under the condition that the value of the loss function is smaller than a preset threshold value.
2. The method of claim 1, wherein the encoder comprises a first encoder and a second encoder;
the inputting a plurality of samples belonging to the same field, which are obtained in advance, into the encoder to obtain an encoding result output by the encoder includes:
grouping a plurality of pre-acquired samples belonging to the same field to obtain a first sample set and a second sample set;
inputting the samples in the first sample set into the first encoder, and learning the samples in the first sample set through the first encoder to obtain a first encoding result; the first encoding result comprises a mean and a variance of implicit variables associated with each sample in the first set of samples;
inputting the samples in the second sample set into the second encoder, and learning the samples in the second sample set through the second encoder to obtain a second encoding result; the second encoding result includes a mean and a variance of an implicit variable associated with each sample in the second set of samples.
3. The method of claim 2, wherein obtaining a target hidden variable distribution based on the mean and variance of the hidden variables associated with each of the samples comprises:
generating a first hidden variable distribution based on a mean and variance of hidden variables associated with each sample in the first set of samples;
generating a second latent variable distribution based on the mean and variance of latent variables associated with each sample in the second set of samples;
and generating target hidden variable distribution based on the first hidden variable distribution and the second hidden variable distribution.
4. The method of claim 1, further comprising:
and displaying other samples similar to the sample to be processed to the user through a preset interface.
5. A sample generation device, comprising:
the device comprises a sample acquisition unit, a processing unit and a processing unit, wherein the sample acquisition unit is used for identifying a sample input by a user in advance as a sample to be processed;
the sample generating unit is used for inputting the sample to be processed into a variational self-encoder obtained by pre-training to obtain a sample set output by the variational self-encoder; wherein the variational self-encoder comprises an encoder and a decoder; the sample set comprises other samples similar to the sample to be processed; the training process of the variational self-encoder comprises the following steps:
inputting a plurality of samples belonging to the same field, which are acquired in advance, into the encoder to obtain an encoding result output by the encoder; the coding result comprises a mean and a variance of the hidden variables associated with each of the samples;
obtaining a target hidden variable distribution based on the mean and variance of the hidden variables associated with each of the samples;
inputting the target hidden variable distribution into the decoder, sampling the target hidden variable distribution through the decoder to obtain target samples, and adjusting parameters of a loss function of the variational self-encoder by taking the difference between the target samples and each sample as a training target; the parameter comprises an expected value of the difference between the target sample and each of the samples;
and determining that the training of the variational self-encoder is finished under the condition that the value of the loss function is smaller than a preset threshold value.
6. The apparatus of claim 5, wherein the encoder comprises a first encoder and a second encoder;
the sample generation unit is specifically configured to:
grouping a plurality of samples belonging to the same field, which are obtained in advance, to obtain a first sample set and a second sample set;
inputting the samples in the first sample set into the first encoder, and learning the samples in the first sample set through the first encoder to obtain a first encoding result; the first encoding result comprises a mean and a variance of implicit variables associated with each sample in the first set of samples;
inputting the samples in the second sample set into the second encoder, and learning the samples in the second sample set through the second encoder to obtain a second encoding result; the second encoding result includes a mean and a variance of an implicit variable associated with each sample in the second set of samples.
7. The apparatus according to claim 6, wherein the sample generation unit is specifically configured to:
generating a first hidden variable distribution based on a mean and variance of hidden variables associated with each sample in the first set of samples;
generating a second latent variable distribution based on the mean and variance of latent variables associated with each sample in the second set of samples;
and generating target hidden variable distribution based on the first hidden variable distribution and the second hidden variable distribution.
8. The apparatus of claim 5, further comprising:
and the sample display unit is used for displaying other samples similar to the sample to be processed to the user through a preset interface.
9. A computer-readable storage medium, comprising a stored program, wherein the program performs the sample generation method of any one of claims 1-4.
10. A sample generation device, comprising: a processor, a memory, and a bus; the processor and the memory are connected through the bus;
the memory is configured to store a program and the processor is configured to execute the program, wherein the program when executed performs the method of generating samples of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211024828.1A CN115374863A (en) | 2022-08-25 | 2022-08-25 | Sample generation method, sample generation device, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211024828.1A CN115374863A (en) | 2022-08-25 | 2022-08-25 | Sample generation method, sample generation device, storage medium and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115374863A true CN115374863A (en) | 2022-11-22 |
Family
ID=84067794
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211024828.1A Pending CN115374863A (en) | 2022-08-25 | 2022-08-25 | Sample generation method, sample generation device, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115374863A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116758378A (en) * | 2023-08-11 | 2023-09-15 | 小米汽车科技有限公司 | Method for generating model, data processing method, related device, vehicle and medium |
-
2022
- 2022-08-25 CN CN202211024828.1A patent/CN115374863A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116758378A (en) * | 2023-08-11 | 2023-09-15 | 小米汽车科技有限公司 | Method for generating model, data processing method, related device, vehicle and medium |
| CN116758378B (en) * | 2023-08-11 | 2023-11-14 | 小米汽车科技有限公司 | Method for generating model, data processing method, related device, vehicle and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | Quantized kernel least mean square algorithm | |
| Chuang et al. | Hybrid robust support vector machines for regression with outliers | |
| Egloff et al. | A dynamic look-ahead Monte Carlo algorithm for pricing Bermudan options | |
| Tang et al. | Automatic sparse connectivity learning for neural networks | |
| Norets et al. | Adaptive Bayesian estimation of conditional densities | |
| CN111461445B (en) | Short-term wind speed prediction method and device, computer equipment and storage medium | |
| Tang et al. | Bayesian augmented Lagrangian algorithm for system identification | |
| Gutoski et al. | Parallel approximation of min-max problems | |
| Ibragimovich et al. | Effective recognition of pollen grains based on parametric adaptation of the image identification model | |
| CN115374863A (en) | Sample generation method, sample generation device, storage medium and equipment | |
| CN115496144A (en) | Power distribution network operation scene determining method and device, computer equipment and storage medium | |
| Chen et al. | Sparse general non-negative matrix factorization based on left semi-tensor product | |
| CN114492823A (en) | Method and apparatus for eliminating quantum noise, electronic device, and medium | |
| CN114429218A (en) | Method and apparatus for eliminating quantum noise, electronic device and medium | |
| Luo et al. | Pruning method for dendritic neuron model based on dendrite layer significance constraints | |
| Roy | Convergence rates for MCMC algorithms for a robust Bayesian binary regression model | |
| Pervez et al. | Spectral smoothing unveils phase transitions in hierarchical variational autoencoders | |
| Rieder et al. | Robust consumption-investment problems with random market coefficients | |
| Ma et al. | RDP-GAN: A Rényi-differential privacy based generative adversarial network | |
| Alquier et al. | Sparsity considerations for dependent variables | |
| Ao et al. | Entropy estimation via normalizing flow | |
| CN116384471A (en) | Model pruning method, device, computer equipment, storage medium and program product | |
| CN115859048A (en) | Noise processing method and device for partial discharge signal | |
| CN113065659B (en) | Method and apparatus for eliminating quantum noise, electronic device, and medium | |
| CN111832277B (en) | Training method of reading understanding model and reading understanding processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |