CN109344873A - A kind of the training sample method for digging and device of deep neural network - Google Patents
A kind of the training sample method for digging and device of deep neural network Download PDFInfo
- Publication number
- CN109344873A CN109344873A CN201811010624.6A CN201811010624A CN109344873A CN 109344873 A CN109344873 A CN 109344873A CN 201811010624 A CN201811010624 A CN 201811010624A CN 109344873 A CN109344873 A CN 109344873A
- Authority
- CN
- China
- Prior art keywords
- sample
- value
- probability
- neural network
- deep neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Probability & Statistics with Applications (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The present invention provides a kind of training sample method for digging of deep neural network, it include: to obtain sample image in the probability of miscarriage of justice value of the propagated forward Jing Guo deep neural network, in Gauss variance maximum, first is calculated using probability value and as weight, first typical sample is chosen using Weighted random sampling algorithm, deep neural network is trained；Sample image is obtained in the probability of miscarriage of justice value of the propagated forward Jing Guo deep neural network, in Gaussian mean maximum, second is calculated using probability value and as weight, chooses the second typical sample, deep neural network is trained；Sample image is obtained in the probability of miscarriage of justice value of the propagated forward Jing Guo deep neural network, in Gaussian mean maximum, variance minimum, third is calculated using probability value and as weight, chooses third typical sample, deep neural network is trained, until deconditioning after training convergence.Compared with prior art, the present invention can excavate typical sample, improve network training effect.
Description
Technical field
The present invention relates to depth learning technology field, in particular to the training sample method for digging of a kind of deep neural network
And device.
Background technique
In recent years, deep learning is achieved in computer vision fields such as image classification, target detection, target followings
Quantum jump.
Due to needing the data of magnanimity when deep neural network training, the quality of data generates final mask effect non-
Often big influence.However, existing sample data excavates Shortcomings, effective sample data cannot be obtained.
In conclusion there is an urgent need to propose a kind of training sample method for digging of deep neural network at present.
Summary of the invention
In view of this, it is a primary object of the present invention to realize that the effective sample of deep neural network excavates.
In order to achieve the above objectives, first aspect according to the invention provides a kind of training sample of deep neural network
This method for digging, this method comprises:
First step obtains sample image in the probability of miscarriage of justice value of the propagated forward Jing Guo deep neural network, is sample
It generates and uses probability value in Gaussian Profile, in Gauss variance maximum, calculate gauss change mean value and corresponding first and use
Probability value uses probability value as weight, chooses the first typical sample using Weighted random sampling algorithm, to depth mind using first
It is trained through network；
Second step obtains sample image in the probability of miscarriage of justice value of the propagated forward Jing Guo deep neural network, is sample
It generates and uses probability value in Gaussian Profile, in Gaussian mean maximum, calculate gauss change variance and corresponding second and use
Probability value uses probability value as weight, chooses the second typical sample using Weighted random sampling algorithm, to depth mind using second
It is trained through network；
Third step obtains sample image in the probability of miscarriage of justice value of the propagated forward Jing Guo deep neural network, is sample
Generate and use probability value in Gaussian Profile, Gaussian mean is maximum, variance minimum when, calculate third and use probability value, with the
Three carry out deep neural network using Weighted random sampling algorithm selection third typical sample as weight using probability value
Training, until deconditioning after training convergence.
Further, the first step includes:
Probability of miscarriage of justice value calculates step, chooses N1A sample image, by the forward direction of sample image input deep neural network
It propagates, obtains N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtaining step, according to the sequence of probability of miscarriage of justice value from small to large, to N1A sample image is ranked up,
Obtain sample sequence
Change mean obtaining step generates for sample and uses probability value in Gaussian Profile, in Gauss variance maximum, meter
It calculates gauss change mean value and corresponding first and uses probability value, calculate gauss change mean valueIts
Middle j is iteration j, μminFor minimum mean, μmaxFor Largest Mean, T1For Change in Mean maximum number of iterations；
First calculates step using probability value, according to normalization Gaussian distribution formula, calculates i-th of sample in sample sequence
Image SiFirst use probability valueWherein σmaxFor maximum variance；
First typical sample selecting step is used probability value as weight, is selected using Weighted random sampling algorithm using first
Take N2A first typical sample；
First typical sample training step, according to N2A first typical sample, is trained deep neural network.
Further, the second step includes:
Probability of miscarriage of justice value calculates step, according to N1A sample image, by the forward direction of sample image input deep neural network
It propagates, obtains N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtaining step, according to the sequence of probability of miscarriage of justice value from small to large, to N1A sample image is ranked up,
Obtain sample sequence
Change variance obtaining step, calculates gauss change varianceWherein k is that kth time changes
Generation, σminAnd σmaxRespectively minimum variance and maximum variance, T2Change minimum the number of iterations for variance；
Second calculates step using probability value, according to normalization Gaussian distribution formula, calculates i-th of sample in sample sequence
Image Si' second use probability value
Second typical sample selecting step is used probability value as weight, is selected using Weighted random sampling algorithm using second
Take N3A second typical sample；
Second typical sample training step, according to N3A second typical sample, is trained deep neural network.
Further, the third step includes:
Probability of miscarriage of justice value calculates step, according to N1A sample image, by the forward direction of sample image input deep neural network
It propagates, obtains N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtaining step, according to the sequence of probability of miscarriage of justice value from small to large, to N1A sample image is ranked up,
Obtain sample sequence
Third using probability value calculate step, for sample generate in Gaussian Profile use probability value, Gaussian mean most
Greatly, when variance minimum, according to normalization Gaussian distribution formula, i-th of sample image S in sample sequence is calculatediThird use
Probability value
Third typical sample selecting step chooses N using Weighted random sampling algorithm4A third typical sample；
Third typical sample training step, according to N4A third typical sample, is trained deep neural network, until
Deconditioning after training convergence.
Other side according to the invention provides a kind of training sample excavating gear of deep neural network, the dress
It sets and includes:
Change in Mean sample excavates and network training module, for obtaining sample image before by deep neural network
To the probability of miscarriage of justice value of propagation, first calculated under gauss change mean value and maximum variance uses probability value, with first using general
Rate value chooses the first typical sample as weight, using Weighted random sampling algorithm, is trained to deep neural network；
Variance changes sample excavation and network training module, for obtaining sample image before by deep neural network
It to the probability of miscarriage of justice value of propagation, is generated for sample and uses probability value in Gaussian Profile, in Gaussian mean maximum, calculate Gauss
Change variance and corresponding second and use probability value, uses probability value as weight using second, using Weighted random sampling algorithm
The second typical sample is chosen, deep neural network is trained；
Largest Mean and the excavation of minimum variance sample and network training module, for obtaining sample image by depth mind
The probability of miscarriage of justice value of propagated forward through network generates for sample and uses probability value in Gaussian Profile, Gaussian mean it is maximum,
It when variance minimum, calculates third and uses probability value, selected as weight using Weighted random sampling algorithm using third using probability value
Third typical sample is taken, deep neural network is trained, until deconditioning after training convergence.
Further, the Change in Mean sample excavates and network training module includes:
Probability of miscarriage of justice value computing module, for choosing N1A sample image, by sample image input deep neural network
Propagated forward obtains N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtains module, for the sequence according to probability of miscarriage of justice value from small to large, to N1A sample image carries out
Sequence obtains sample sequence
Change mean obtains module, uses probability value in Gaussian Profile for generating for sample, maximum in Gauss variance
When, it calculates gauss change mean value and corresponding first and uses probability value, calculate gauss change mean valueWherein j is iteration j, μminFor minimum mean, μmaxFor Largest Mean, T1For mean value change
Change maximum number of iterations；
First uses probability value computing module, for calculating in sample sequence i-th according to normalization Gaussian distribution formula
Sample image SiFirst use probability valueWherein σmaxFor maximum variance；
First typical sample chooses module, for using probability value as weight using first, is sampled and is calculated using Weighted random
Method chooses N2A first typical sample；
First typical sample training module, for according to N2A first typical sample, is trained deep neural network.
Further, the variance variation sample excavates and network training module includes:
Probability of miscarriage of justice value computing module, for according to N1A sample image, by sample image input deep neural network
Propagated forward obtains N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtains module, for the sequence according to probability of miscarriage of justice value from small to large, to N1A sample image carries out
Sequence obtains sample sequence
Change variance and obtain module, for calculating gauss change varianceWherein k is kth
Secondary iteration, σminAnd σmaxRespectively minimum variance and maximum variance, T2Change minimum the number of iterations for variance；
Second uses probability value computing module, for calculating in sample sequence i-th according to normalization Gaussian distribution formula
Sample image Si' second use probability value
Second typical sample chooses module, for using probability value as weight using second, is sampled and is calculated using Weighted random
Method chooses N3A second typical sample；
Second typical sample training module, for according to N3A second typical sample, is trained deep neural network.
Further, the Largest Mean and minimum variance sample excavate and network training module includes:
Probability of miscarriage of justice value computing module, for according to N1A sample image, by sample image input deep neural network
Propagated forward obtains N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtains module, for the sequence according to probability of miscarriage of justice value from small to large, to N1A sample image carries out
Sequence obtains sample sequence
Third uses probability value computing module, equal in Gauss for generating the probability value that uses in Gaussian Profile for sample
When value maximum, variance minimum, according to normalization Gaussian distribution formula, i-th of sample image S in sample sequence is calculatediThird
Using probability value
Third typical sample chooses module, for choosing N using Weighted random sampling algorithm4A third typical sample；
Third typical sample training module, for according to N4A third typical sample, is trained deep neural network,
Until deconditioning after training convergence.
Compared with existing sample training technology, a kind of training sample method for digging of deep neural network of the invention and
Device uses probability value using sample, is obtained respectively by changing mean value, change variance and fixed mean variance using general
Rate value, what be will acquire uses probability value as weight, realizes effective excavation of the sample of network, improves training effect.
Detailed description of the invention
Fig. 1 shows the flow chart of the training sample method for digging of deep neural network according to the invention.
Fig. 2 shows the frame diagrams of the training sample excavating gear of deep neural network according to the invention.
Specific embodiment
To enable those skilled in the art to further appreciate that structure of the invention, feature and other purposes, now in conjunction with institute
Detailed description are as follows for attached preferred embodiment, and illustrated preferred embodiment is only used to illustrate the technical scheme of the present invention, and not limits
The fixed present invention.
Fig. 1 gives the flow chart of the training sample method for digging of deep neural network according to the invention.Such as Fig. 1 institute
Show, the training sample method for digging of deep neural network according to the invention includes:
First step S1 obtains sample image in the probability of miscarriage of justice value of the propagated forward Jing Guo deep neural network, is sample
This generation uses probability value in Gaussian Profile, in Gauss variance maximum, calculates gauss change mean value and corresponding first and adopts
With probability value, uses probability value as weight using first, the first typical sample is chosen using Weighted random sampling algorithm, to depth
Neural network is trained；
Second step S2 obtains sample image in the probability of miscarriage of justice value of the propagated forward Jing Guo deep neural network, is sample
This generation uses probability value in Gaussian Profile, in Gaussian mean maximum, calculates gauss change variance and corresponding second and adopts
With probability value, uses probability value as weight using second, the second typical sample is chosen using Weighted random sampling algorithm, to depth
Neural network is trained；
Third step S3 obtains sample image in the probability of miscarriage of justice value of the propagated forward Jing Guo deep neural network, is sample
This generation uses probability value in Gaussian Profile, in Gaussian mean maximum, variance minimum, calculates third and uses probability value, with
Third using probability value be used as weight, using Weighted random sampling algorithm selection third typical sample, to deep neural network into
Row training, until deconditioning after training convergence.
Further, the first step S1 includes:
Probability of miscarriage of justice value calculates step S11, chooses N1A sample image, before sample image is inputted deep neural network
To propagation, N is obtained1The probability of miscarriage of justice value of a sample image；
Sample sequence obtaining step S12, according to the sequence of probability of miscarriage of justice value from small to large, to N1A sample image is arranged
Sequence obtains sample sequence
Change mean obtaining step S13 is generated for sample and is used probability value in Gaussian Profile, maximum in Gauss variance
When, it calculates gauss change mean value and corresponding first and uses probability value, calculate gauss change mean valueWherein j is iteration j, μminFor minimum mean, μmaxFor Largest Mean, T1For mean value change
Change maximum number of iterations；
First calculates step S14 using probability value, according to normalization Gaussian distribution formula, calculates in sample sequence i-th
Sample image SiFirst use probability valueWherein σmaxFor maximum variance；
First typical sample selecting step S15 uses probability value as weight, using Weighted random sampling algorithm using first
Choose N2A first typical sample；
First typical sample training step S16, according to N2A first typical sample, is trained deep neural network.
Further, the N1Value range be 100~5000, the N2Value range be N1/ 100~N1/ 5, institute
State μminInterval range be [0, λ1×N1], the μmaxInterval range be [λ2×N1, λ3×N1], σmaxInterval range be
[λ4×N1,λ5×N1]。
Further, the λ1Value range be 0.005~0.02, the λ2Value range be 0.8~1.0, it is described
λ3Value range be 1.0~1.2, the λ4Value range be 0.28~0.4, the λ5Value range be 0.45~
0.65, the T1Value range be 1000~5000.Illustratively, the λ1It is selected as 0.01, λ2It is selected as 0.9, λ3It is selected as 1, λ4
It is selected as 0.33, λ5It is selected as 0.5.
The sample image can be chosen according to actual scene demand or application field, including but not limited to: people
Face image, license plate image, pedestrian image, vehicle image etc..
The deep neural network includes: convolutional neural networks, deepness belief network, recurrent neural network or biology
Neural network, or combinations thereof.
The Weighted random sampling algorithm can be real using existing Weighted random sampling algorithm or Weighted random algorithm
It is existing.Illustratively, use probability value as weight using first, using " Weighted random sampling with a
Reservoir.PS Efraimidis, PG Spirakis. " Information Processing Letters ", 2006,97
(5): the Weighted random method of sampling in 181-185 " document chooses N2A first typical sample.
Illustratively, for field of face identification, the first step S1 are as follows: choose 1000 marked facial images
As sample image, sample image is input to the propagated forward of convolutional neural networks, obtains the corresponding mistake of each sample image
Sentence probability value；According to the sequence of probability of miscarriage of justice value from small to large, 1000 sample images are ranked up, obtain sample sequenceAccording toCalculate sample sequence first uses probability value；With first
Using probability value as weight, 50 the first typical samples are chosen using Weighted random sampling algorithm；Finally according to 50 first
Typical sample is trained convolutional neural networks.
Further, the second step S2 includes:
Probability of miscarriage of justice value calculates step S21, according to N1A sample image, before sample image is inputted deep neural network
To propagation, N is obtained1The probability of miscarriage of justice value of a sample image；
Sample sequence obtaining step S22, according to the sequence of probability of miscarriage of justice value from small to large, to N1A sample image is arranged
Sequence obtains sample sequence
Change variance obtaining step S23, generated for sample and use probability value in Gaussian Profile, in Gaussian mean maximum
When, calculate gauss change varianceWherein k is kth time iteration, σminAnd σmaxIt is respectively minimum
Variance and maximum variance, T2Change minimum the number of iterations for variance；
Second calculates step S24 using probability value, according to normalization Gaussian distribution formula, calculates in sample sequence i-th
Sample image Si' second use probability value
Second typical sample selecting step S25 uses probability value as weight, using Weighted random sampling algorithm using second
Choose N3A second typical sample；
Second typical sample training step S26, according to N3A second typical sample, is trained deep neural network.
Wherein, the σminInterval range be [λ6×N1,λ7×N1], the N3Value range be N1/ 100~N1/5。
Further, the λ6Value range be 0.1~0.3, the λ7Value range be 0.3~0.8, the T2
Value range be 5000~20000.Illustratively, the λ6It is selected as 0.2, λ7It is selected as 0.5, T2It is selected as 10000.
Further, the third step S3 includes:
Probability of miscarriage of justice value calculates step S31, according to N1A sample image, before sample image is inputted deep neural network
To propagation, N is obtained1The probability of miscarriage of justice value of a sample image；
Sample sequence obtaining step S32, according to the sequence of probability of miscarriage of justice value from small to large, to N1A sample image is arranged
Sequence obtains sample sequence
Third calculates step S33 using probability value, generates for sample and uses probability value in Gaussian Profile, in Gaussian mean
When maximum, variance minimum, according to normalization Gaussian distribution formula, i-th of sample image S in sample sequence is calculatediThird adopt
Use probability value
Third typical sample selecting step S34 chooses N using Weighted random sampling algorithm4A third typical sample；
Third typical sample training step S35, according to N4A third typical sample, is trained deep neural network,
Until deconditioning after training convergence.
Fig. 2 gives the frame diagram of the training sample excavating gear of deep neural network according to the invention.Such as Fig. 2 institute
Show, the training sample excavating gear of deep neural network according to the invention includes:
Change in Mean sample excavates and network training module 1, for obtaining sample image by deep neural network
The probability of miscarriage of justice value of propagated forward generates for sample and uses probability value in Gaussian Profile, in Gauss variance maximum, calculates high
This change mean and corresponding first uses probability value, uses probability value as weight using first, is sampled and calculated using Weighted random
Method chooses the first typical sample, is trained to deep neural network；
Variance changes sample excavation and network training module 2, for obtaining sample image by deep neural network
The probability of miscarriage of justice value of propagated forward generates for sample and uses probability value in Gaussian Profile, in Gaussian mean maximum, calculates high
This variation variance and corresponding second uses probability value, uses probability value as weight using second, is sampled and calculated using Weighted random
Method chooses the second typical sample, is trained to deep neural network；
Largest Mean and the excavation of minimum variance sample and network training module 3, for obtaining sample image by depth
The probability of miscarriage of justice value of the propagated forward of neural network, for sample generate in Gaussian Profile use probability value, Gaussian mean most
Greatly, it when variance minimum, calculates third and uses probability value, sampled and calculated using Weighted random as weight using probability value using third
Method chooses third typical sample, is trained to deep neural network, until deconditioning after training convergence.
Further, the Change in Mean sample excavates and network training module 1 includes:
Probability of miscarriage of justice value computing module 11, for choosing N1Sample image is inputted deep neural network by a sample image
Propagated forward, obtain N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtains module 12, for the sequence according to probability of miscarriage of justice value from small to large, to N1A sample image into
Row sequence, obtains sample sequence
Change mean obtain module 13, for for sample generate in Gaussian Profile use probability value, Gauss variance most
When big, calculate gauss change mean value and corresponding first and use probability value, calculate gauss change mean valueWherein j is iteration j, μminFor minimum mean, μmaxFor Largest Mean, T1For mean value change
Change maximum number of iterations；
First uses probability value computing module 14, for calculating i-th in sample sequence according to normalization Gaussian distribution formula
A sample image SiFirst use probability valueWherein σmaxFor maximum variance；
First typical sample chooses module 15, for using probability value as weight using first, is sampled using Weighted random
Algorithm picks N2A first typical sample；
First typical sample training module 16, for according to N2A first typical sample, instructs deep neural network
Practice.
Further, the variance variation sample excavates and network training module 2 includes:
Probability of miscarriage of justice value computing module 21, for according to N1Sample image is inputted deep neural network by a sample image
Propagated forward, obtain N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtains module 22, for the sequence according to probability of miscarriage of justice value from small to large, to N1A sample image into
Row sequence, obtains sample sequence
Change variance and obtain module 23, for calculating gauss change varianceWherein k is
Kth time iteration, σminAnd σmaxRespectively minimum variance and maximum variance, T2Change minimum the number of iterations for variance；
Second uses probability value computing module 24, for calculating i-th in sample sequence according to normalization Gaussian distribution formula
A sample image Si' second use probability value
Second typical sample chooses module 25, for using probability value as weight using second, is sampled using Weighted random
Algorithm picks N3A second typical sample；
Second typical sample training module 26, for according to N3A second typical sample, instructs deep neural network
Practice.
Further, the Largest Mean and minimum variance sample excavate and network training module 3 includes:
Probability of miscarriage of justice value computing module 31, for according to N1Sample image is inputted deep neural network by a sample image
Propagated forward, obtain N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtains module 32, for the sequence according to probability of miscarriage of justice value from small to large, to N1A sample image into
Row sequence, obtains sample sequence
Third uses probability value computing module 33, probability value is used in Gaussian Profile for generating for sample, in Gauss
When mean value maximum, variance minimum, according to normalization Gaussian distribution formula, i-th of sample image S in sample sequence is calculatedi?
Three use probability value
Third typical sample chooses module 34, for choosing N using Weighted random sampling algorithm4A third typical sample；
Third typical sample training module 35, for according to N4A third typical sample, instructs deep neural network
Practice, until deconditioning after training convergence.
Further, the N4Value range be N1/ 100~N1/5。
Compared with existing sample training technology, a kind of training sample method for digging of deep neural network of the invention and
Device uses probability value using sample, is obtained respectively by changing mean value, change variance and fixed mean variance using general
Rate value, what be will acquire uses probability value as weight, realizes effective excavation of the sample of network, improves training effect.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention, should
Understand, the present invention is not limited to implementation as described herein, the purpose of these implementations description is to help this field
In technical staff practice the present invention.Any those of skill in the art are easy to do not departing from spirit and scope of the invention
In the case of be further improved and perfect, therefore the present invention is only by the content of the claims in the present invention and the limit of range
System, intention, which covers, all to be included the alternative in the spirit and scope of the invention being defined by the appended claims and waits
Same scheme.
Claims (11)
1. a kind of training sample method for digging of deep neural network, which is characterized in that this method comprises:
First step obtains sample image in the probability of miscarriage of justice value of the propagated forward Jing Guo deep neural network, generates for sample
Probability value is used in Gaussian Profile, in Gauss variance maximum, gauss change mean value and corresponding first is calculated and uses probability
Value uses probability value as weight, the first typical sample is chosen using Weighted random sampling algorithm, to depth nerve net using first
Network is trained；
Second step obtains sample image in the probability of miscarriage of justice value of the propagated forward Jing Guo deep neural network, generates for sample
Probability value is used in Gaussian Profile, in Gaussian mean maximum, gauss change variance and corresponding second is calculated and uses probability
Value uses probability value as weight, the second typical sample is chosen using Weighted random sampling algorithm, to depth nerve net using second
Network is trained；
Third step obtains sample image in the probability of miscarriage of justice value of the propagated forward Jing Guo deep neural network, generates for sample
Probability value is used in Gaussian Profile, in Gaussian mean maximum, variance minimum, third is calculated and uses probability value, adopted with third
It uses probability value as weight, third typical sample is chosen using Weighted random sampling algorithm, deep neural network is trained,
Until deconditioning after training convergence.
2. the method as described in claim 1, which is characterized in that the first step includes:
Probability of miscarriage of justice value calculates step, chooses N1Sample image is inputted the propagated forward of deep neural network by a sample image,
Obtain N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtaining step, according to the sequence of probability of miscarriage of justice value from small to large, to N1A sample image is ranked up, and is obtained
Sample sequence
Change mean obtaining step generates for sample and uses probability value in Gaussian Profile, in Gauss variance maximum, calculates high
This change mean and corresponding first uses probability value, calculates gauss change mean valueWherein j
For iteration j, μminFor minimum mean, μmaxFor Largest Mean, T1For Change in Mean maximum number of iterations；
First calculates step using probability value, according to normalization Gaussian distribution formula, calculates i-th of sample image in sample sequence
SiFirst use probability valueWherein σmaxFor maximum variance；
First typical sample selecting step uses probability value as weight, chooses N using Weighted random sampling algorithm using first2It is a
First typical sample；
First typical sample training step, according to N2A first typical sample, is trained deep neural network.
3. method according to claim 2, further, the deep neural network include: convolutional neural networks, depth letter
Network, recurrent neural network or biological neural network are read, or combinations thereof.
4. the method as described in claim 1, which is characterized in that the second step includes:
Probability of miscarriage of justice value calculates step, according to N1Sample image is inputted the propagated forward of deep neural network by a sample image,
Obtain N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtaining step, according to the sequence of probability of miscarriage of justice value from small to large, to N1A sample image is ranked up, and is obtained
Sample sequence
Change variance obtaining step, calculates gauss change varianceWherein k is kth time iteration,
σminAnd σmaxRespectively minimum variance and maximum variance, T2Change minimum the number of iterations for variance；
Second calculates step using probability value, according to normalization Gaussian distribution formula, calculates i-th of sample image in sample sequence
Si' second use probability value
Second typical sample selecting step uses probability value as weight, chooses N using Weighted random sampling algorithm using second3It is a
Second typical sample；
Second typical sample training step, according to N3A second typical sample, is trained deep neural network.
5. the method as described in claim 1, which is characterized in that the third step includes:
Probability of miscarriage of justice value calculates step, according to N1Sample image is inputted the propagated forward of deep neural network by a sample image,
Obtain N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtaining step, according to the sequence of probability of miscarriage of justice value from small to large, to N1A sample image is ranked up, and is obtained
Sample sequence
Third is generated for sample using probability value calculating step and is used probability value in Gaussian Profile, in Gaussian mean maximum, side
When poor minimum, according to normalization Gaussian distribution formula, i-th of sample image S in sample sequence is calculatediThird use probability value
Third typical sample selecting step chooses N using Weighted random sampling algorithm4A third typical sample；
Third typical sample training step, according to N4A third typical sample, is trained deep neural network, until training
Deconditioning after convergence.
6. the method as described in Claims 1 to 5, further, the N1Value range be 100~5000, the N2、N3
And N4Value range be N1/ 100~N1/ 5, the μminInterval range be [0, λ1×N1], the μmaxInterval range be
[λ2×N1,λ3×N1], σmaxInterval range be [λ4×N1,λ5×N1]；The σminInterval range be [λ6×N1,λ7×
N1]。
7. method as claimed in claim 6, further, the λ1Value range be 0.005~0.02, the λ2Take
Being worth range is 0.8~1.0, the λ3Value range be 1.0~1.2, the λ4Value range be 0.28~0.4, the λ5
Value range be 0.45~0.65, the T1Value range be 1000~5000；The λ6Value range be 0.1~
0.3, the λ7Value range be 0.3~0.8, the T2Value range be 5000~20000.
8. a kind of training sample excavating gear of deep neural network, which is characterized in that the device includes:
Change in Mean sample excavates and network training module, passes for obtaining sample image in the forward direction Jing Guo deep neural network
The probability of miscarriage of justice value broadcast generates for sample and uses probability value in Gaussian Profile, in Gauss variance maximum, calculates gauss change
Mean value and corresponding first uses probability value, uses probability value as weight using first, is chosen using Weighted random sampling algorithm
First typical sample, is trained deep neural network；
Variance changes sample excavation and network training module, passes for obtaining sample image in the forward direction Jing Guo deep neural network
The probability of miscarriage of justice value broadcast generates for sample and uses probability value in Gaussian Profile, in Gaussian mean maximum, calculates gauss change
Variance and corresponding second uses probability value, uses probability value as weight using second, is chosen using Weighted random sampling algorithm
Second typical sample, is trained deep neural network；
Largest Mean and the excavation of minimum variance sample and network training module are passing through depth nerve net for obtaining sample image
The probability of miscarriage of justice value of the propagated forward of network generates for sample and uses probability value in Gaussian Profile, in Gaussian mean maximum, variance
It when minimum, calculate third and uses probability value, weight is used as using probability value using third, using Weighted random sampling algorithm selection the
Three typical samples, are trained deep neural network, until deconditioning after training convergence.
9. device as claimed in claim 8, which is characterized in that the Change in Mean sample excavates and network training module packet
It includes:
Probability of miscarriage of justice value computing module, for choosing N1A sample image passes the forward direction of sample image input deep neural network
It broadcasts, obtains N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtains module, for the sequence according to probability of miscarriage of justice value from small to large, to N1A sample image is ranked up,
Obtain sample sequence
Change mean obtains module, generates for sample and uses probability value in Gaussian Profile, in Gauss variance maximum, calculates high
This change mean and corresponding first uses probability value, calculates gauss change mean valueWherein j
For iteration j, μminFor minimum mean, μmaxFor Largest Mean, T1For Change in Mean maximum number of iterations；
First uses probability value computing module, for calculating i-th of sample in sample sequence according to normalization Gaussian distribution formula
Image SiFirst use probability valueWherein σmaxFor maximum variance；
First typical sample chooses module, for using probability value as weight using first, is selected using Weighted random sampling algorithm
Take N2A first typical sample；
First typical sample training module, for according to N2A first typical sample, is trained deep neural network.
10. device as claimed in claim 9, which is characterized in that the variance variation sample excavates and network training module packet
It includes:
Probability of miscarriage of justice value computing module, for according to N1A sample image passes the forward direction of sample image input deep neural network
It broadcasts, obtains N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtains module, for the sequence according to probability of miscarriage of justice value from small to large, to N1A sample image is ranked up,
Obtain sample sequence
Change variance and obtain module, for calculating gauss change varianceWherein k is that kth time changes
Generation, σminAnd σmaxRespectively minimum variance and maximum variance, T2Change minimum the number of iterations for variance；
Second uses probability value computing module, for calculating i-th of sample in sample sequence according to normalization Gaussian distribution formula
Image Si' second use probability value
Second typical sample chooses module, for using probability value as weight using second, is selected using Weighted random sampling algorithm
Take N3A second typical sample；
Second typical sample training module, for according to N3A second typical sample, is trained deep neural network.
11. device as claimed in claim 8, which is characterized in that the Largest Mean and the excavation of minimum variance sample and network
Training module includes:
Probability of miscarriage of justice value computing module, for according to N1A sample image passes the forward direction of sample image input deep neural network
It broadcasts, obtains N1The probability of miscarriage of justice value of a sample image；
Sample sequence obtains module, for the sequence according to probability of miscarriage of justice value from small to large, to N1A sample image is ranked up,
Obtain sample sequence
Third use probability value computing module, for for sample generate in Gaussian Profile use probability value, Gaussian mean most
Greatly, when variance minimum, according to normalization Gaussian distribution formula, i-th of sample image S in sample sequence is calculatediThird use
Probability value
Third typical sample chooses module, for choosing N using Weighted random sampling algorithm4A third typical sample；
Third typical sample training module, for according to N4A third typical sample, is trained deep neural network, until
Deconditioning after training convergence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811010624.6A CN109344873B (en) | 2018-08-31 | 2018-08-31 | Training sample mining method and device for deep neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811010624.6A CN109344873B (en) | 2018-08-31 | 2018-08-31 | Training sample mining method and device for deep neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109344873A true CN109344873A (en) | 2019-02-15 |
| CN109344873B CN109344873B (en) | 2021-07-09 |
Family
ID=65292095
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811010624.6A Active CN109344873B (en) | 2018-08-31 | 2018-08-31 | Training sample mining method and device for deep neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109344873B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113655457A (en) * | 2021-08-24 | 2021-11-16 | 中国电子科技集团公司第十四研究所 | Radar target detection capability self-evolution method and device based on sample mining |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105787513A (en) * | 2016-03-01 | 2016-07-20 | 南京邮电大学 | Transfer learning design method and system based on domain adaptation under multi-example multi-label framework |
| CN105787557A (en) * | 2016-02-23 | 2016-07-20 | 北京工业大学 | Design method of deep nerve network structure for computer intelligent identification |
| WO2016207731A2 (en) * | 2015-06-25 | 2016-12-29 | Sentient Technologies (Barbados) Limited | Alife machine learning system and method |
| US20170329048A1 (en) * | 2016-05-12 | 2017-11-16 | The Climate Corporation | Statistical blending of weather data sets |
| CN107463951A (en) * | 2017-07-19 | 2017-12-12 | 清华大学 | A kind of method and device for improving deep learning model robustness |
| CN107784312A (en) * | 2016-08-24 | 2018-03-09 | 腾讯征信有限公司 | Machine learning model training method and device |
-
2018
- 2018-08-31 CN CN201811010624.6A patent/CN109344873B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016207731A2 (en) * | 2015-06-25 | 2016-12-29 | Sentient Technologies (Barbados) Limited | Alife machine learning system and method |
| CN105787557A (en) * | 2016-02-23 | 2016-07-20 | 北京工业大学 | Design method of deep nerve network structure for computer intelligent identification |
| CN105787513A (en) * | 2016-03-01 | 2016-07-20 | 南京邮电大学 | Transfer learning design method and system based on domain adaptation under multi-example multi-label framework |
| US20170329048A1 (en) * | 2016-05-12 | 2017-11-16 | The Climate Corporation | Statistical blending of weather data sets |
| CN107784312A (en) * | 2016-08-24 | 2018-03-09 | 腾讯征信有限公司 | Machine learning model training method and device |
| CN107463951A (en) * | 2017-07-19 | 2017-12-12 | 清华大学 | A kind of method and device for improving deep learning model robustness |
Non-Patent Citations (2)
| Title |
|---|
| JIE JI 等: ""Crucial Data Selection Based on Random Weight Neural Network"", 《2015 IEEE INTERNATIONAL CONFERENCE ON SYSTEMS, MAN, AND CYBERNETICS》 * |
| 杨泽平: ""基于神经网络的不平衡数据分类方法研究"", 《中国博士学位论文全文数据库-信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113655457A (en) * | 2021-08-24 | 2021-11-16 | 中国电子科技集团公司第十四研究所 | Radar target detection capability self-evolution method and device based on sample mining |
| CN113655457B (en) * | 2021-08-24 | 2023-11-24 | 中国电子科技集团公司第十四研究所 | Self-evolution method and device for radar target detection capability based on sample mining |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109344873B (en) | 2021-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tang et al. | An automatic cost learning framework for image steganography using deep reinforcement learning | |
| Li et al. | Adaptive square attack: Fooling autonomous cars with adversarial traffic signs | |
| Wang et al. | Invisible adversarial attack against deep neural networks: An adaptive penalization approach | |
| Zhang et al. | Auxiliary training: Towards accurate and robust models | |
| CN111666925B (en) | Training method and device for face recognition model | |
| CN106372656B (en) | Obtain method, image-recognizing method and the device of the disposable learning model of depth | |
| Pu et al. | Stein variational autoencoder | |
| Fang et al. | Visual acuity inspired saliency detection by using sparse features | |
| CN109344873A (en) | A kind of the training sample method for digging and device of deep neural network | |
| Zhang et al. | Preserving data privacy in federated learning through large gradient pruning | |
| Liu et al. | Jamming recognition based on feature fusion and convolutional neural network | |
| Huang et al. | Global-local fusion based on adversarial sample generation for image-text matching | |
| Su et al. | Reconstruction-assisted and distance-optimized adversarial training: A defense framework for remote sensing scene classification | |
| CN104299197A (en) | Single image defogging method by means of 2D CCA | |
| Terry et al. | Locating Hidden Exoplanets in ALMA Data Using Machine Learning | |
| Shi et al. | Coded diffraction imaging via double sparse regularization model | |
| CN106530319B (en) | A kind of the video object collaboration dividing method based on track digraph | |
| CN106934339A (en) | A kind of target following, the extracting method of tracking target distinguishing feature and device | |
| Zhang et al. | Visible light polarization image desmogging via cycle convolutional neural network | |
| Qi et al. | HSNet: Crowd counting via hierarchical scale calibration and spatial attention | |
| Xiong et al. | SCFFNet: Spatial context feature fusion network for understanding the highly congested scenes | |
| Ribeiro et al. | Optimizing the Wasserstein GAN for TeV Gamma Ray Detection with VERITAS | |
| Li et al. | A lightweight multi-feature fusion structure for automatic modulation classification | |
| Wang et al. | Perspective-aware density regression for crowd counting | |
| Li et al. | Scale-informed density estimation for dense crowd counting |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |