CN109583485A - It is a kind of that supervision deep learning method is had based on feedback training - Google Patents
It is a kind of that supervision deep learning method is had based on feedback training Download PDFInfo
- Publication number
- CN109583485A CN109583485A CN201811367393.4A CN201811367393A CN109583485A CN 109583485 A CN109583485 A CN 109583485A CN 201811367393 A CN201811367393 A CN 201811367393A CN 109583485 A CN109583485 A CN 109583485A
- Authority
- CN
- China
- Prior art keywords
- sample
- deep learning
- weight parameter
- training
- prediction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000013135 deep learning Methods 0.000 title claims abstract description 19
- 230000006870 function Effects 0.000 claims description 10
- 238000005070 sampling Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 8
- 238000013136 deep learning model Methods 0.000 abstract description 6
- 239000011159 matrix material Substances 0.000 description 5
- 238000012952 Resampling Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 2
- 241000208340 Araliaceae Species 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012428 routine sampling Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention relates to a kind of to have supervision deep learning method based on feedback training, this method is during training has supervision deep learning model, when each iteration starts, each sample in training set is sampled with a sampled probability, the sampled probability is adjusted with the prediction penalty values dynamic of each sample.Compared with prior art, the present invention passes through the prediction penalty values of sample each in supervised learning training process are associated by sample frequency with its own, the probability that adjustment outliers are sampled is realized in the way of feedback training, has many advantages, such as to improve training effect.
Description
Technical field
The present invention relates to deep learning fields, have supervision deep learning side based on feedback training more particularly, to a kind of
Method.
Background technique
Present has supervision deep learning method in use, needs to be learnt using a large amount of sample data,
Demand to hardware when in order to reduce supervision deep learning model training generallys use small lot sampling or single sample input
Mode training pattern.Common sample mode is that uniform sampling or use sequentially input.
In that case, a large amount of conventional sample has equal probability with a small amount of outliers and is admitted to model instruction
Practice, model is caused to be difficult to acquire the spatial distribution of a small amount of outliers.When the training objective of model needs to detect or identify in a small amount
When outliers, the accuracy rate of model is not only reduced by the Training of routine sampling mode, and reduces model
Training speed.
To solve the above-mentioned problems, existing settling mode is usually data resampling, classification aligned sample, cost-sensitive
Matrix and the mode of the method for cost-sensitive vector are trained.The mode of resampling and classification aligned sample is by inhomogeneity
The identical number of other specimen sample is trained.Class inherited is big, small different types of of difference in class solving for this method
Effect is preferable on sample size imbalance problem.However difference is big in the class, that is, when there are a small amount of outliers, extremely difficult of model
To its sample distribution.The method of cost-sensitive matrix or cost-sensitive vector can pass through building confusion matrix or cost-sensitive square
Battle array increases learning rate to by the classification of mistake point, thus study of the acceleration model to outliers.But when outliers are present in greatly
When in sample size classification, since the probability that outliers are pumped to is very little, the effect of this method almost be can be ignored.
Therefore, in order to promote the learning efficiencies of outliers, sample size is unbalanced between not only solving the problems, such as class, and
And to solve the problems, such as that sample size is unbalanced in class.And the prior art is difficult to solve the above problems.
Summary of the invention
It is an object of the present invention to overcome the above-mentioned drawbacks of the prior art and provide one kind to be based on feedback training
Have supervision deep learning method.
The purpose of the present invention can be achieved through the following technical solutions:
A kind of to have supervision deep learning method based on feedback training, this method has supervision deep learning model in training
In the process, when each iteration starts, each sample in training set is sampled with a sampled probability, the sampled probability with
The prediction penalty values dynamic of each sample adjusts.
Further, the process of the sampled probability dynamic adjustment specifically includes:
1) each sample weight parameter is initialized；
2) according to various kinds should have before weight parameter calculate corresponding sampled probability:
Wherein, P (i) is the sampled probability of sample i, and α is priority factor, piFor the weight parameter of sample i；
3) after carrying out an iteration, the prediction penalty values of each sample are obtained, update weight ginseng based on the prediction penalty values
Number；
4) when next iteration starts, p is enabledi=p (i), return step 2).
Further, when each sample weight parameter of the initialization, enabling each sample weight parameter is 1.
It is further, described to update weight parameter based on the prediction penalty values specifically:
P (i)=| δ (i) |+ε
Wherein, p (i) is the weight parameter of updated sample i, and δ (i) is the prediction penalty values of sample i, and ε is modifying factor
Son.
Further, the modifying factor ε is a positive number for being greater than 0.
Further, the expression formula of prediction penalty values δ (i) are as follows:
δ (i)=L (yi,f(xi))
Wherein, xiFor input, yiFor xiCorresponding true value label, function f are by inputting xiThe function of prediction label, letter
Number L is to calculate true value label yiWith prediction label f (xi) difference loss function.
Further, when the update weight parameter based on the prediction penalty values, weight parameter and prediction penalty values
It is reciprocal directly proportional.
Compared with prior art, the present invention have with following the utility model has the advantages that
First, present invention firstly provides the methods that dynamic sampling rate adjusting is used in having supervision deep learning, pass through
Increase the probability that outliers are learnt so that model quickly acquires whole sample space distribution, to reduce model training
Simultaneously model training effect can be improved in time.
Second, the present invention can be in conjunction with other sample mode (sides such as resampling, classification aligned sample, cost-sensitive matrix
Formula) to reach the better training effect of effect.
Third, the present invention can be used inversely, increase model learning routine by reducing the probability that outliers are sampled
The ability of sample characteristics.
Detailed description of the invention
Fig. 1 is the flow diagram that present invention training has supervision deep learning model.
Specific embodiment
The present invention is described in detail with specific embodiment below in conjunction with the accompanying drawings.The present embodiment is with technical solution of the present invention
Premised on implemented, the detailed implementation method and specific operation process are given, but protection scope of the present invention is not limited to
Following embodiments.
The present invention provide it is a kind of based on feedback training having supervision deep learning method, run in GPU, be applied to image
Treatment process, this method is during training has supervision deep learning model, when each iteration starts, with a sampled probability
Each sample in training set is sampled, the sampled probability is adjusted with the prediction penalty values dynamic of each sample.
The process of sampled probability dynamic adjustment specifically includes:
1) each sample weight parameter p is initializedi=1；
2) according to various kinds should have before weight parameter calculate corresponding sampled probability:
Wherein, P (i) is the sampled probability of sample i, piFor the weight parameter of sample i, α is priority factor, value more it is big then
It is bigger to represent priority, is uniform sampling when α takes 0；
3) after carrying out an iteration, the prediction penalty values of each sample are obtained, update weight parameter:
P (i)=| δ (i) |+ε
Wherein, p (i) is the weight parameter of updated sample i, and δ (i) is the prediction penalty values of sample i, and ε is modifying factor
Son can take 10-5Etc. very littles normal number, x when preventing δ (i)=00It will not be sampled again；
4) when next iteration starts, p is enabledi=p (i), return step 2).
Predict the expression formula of penalty values δ (i) are as follows:
δ (i)=L (yi,f(xi))
Wherein, xiFor input, yiFor xiCorresponding true value label, function f are by inputting xiThe function of prediction label, letter
Number L is to calculate true value label yiWith prediction label f (xi) difference loss function.
The above method can combine other sample mode (such as resampling, classification aligned sample, cost-sensitive matrix sides
Formula) to reach the better training effect of effect.For in conjunction with classification aligned sample, from a large amount of sample class and a small amount of samples
The sample of same amount is acquired in this class respectively, acquisition probability presses weighted value calculating in class.
The above method can be used inversely, increase model learning routine sample by reducing the probability that outliers are sampled
The ability of feature.Such as when using autocoder (Auto-encoder), need to learn more arm's length standard samples
Feature needs to sample more normal samples at this time, by peeling off using the inverse of penalty values as when itself weight calculation probability
Sample can be sampled less.
As shown in Figure 1, having supervision deep learning model training process based on above-mentioned sampled probability dynamic adjustment specifically:
In step 401, the information of the corresponding tag along sort of all picture samples is read in advance；
In step 402, weight initialization, initialization value 1 are acquired to the image pattern information of all readings；
In step 403, the acquisition probability of each image pattern is calculated；
In step 404, the tag along sort corresponding according to the acquisition probability acquisition image of each image pattern；
In step 405, acquired image feeding there is into supervision deep learning network model training, and is lost with it
Value；
In a step 406, judging this has whether supervision deep learning network model reaches the trained the number of iterations upper limit, if reaching
Training is then terminated to the upper limit, it is no to then follow the steps 407；
In step 407, the penalty values of each image pattern of gained are calculated using step 405；
In a step 408, the weight of each sample is updated, executes step 403 after the completion.
The preferred embodiment of the present invention has been described in detail above.It should be appreciated that those skilled in the art without
It needs creative work according to the present invention can conceive and makes many modifications and variations.Therefore, all technologies in the art
Personnel are available by logical analysis, reasoning, or a limited experiment on the basis of existing technology under this invention's idea
Technical solution, all should be within the scope of protection determined by the claims.
Claims (7)
1. a kind of have supervision deep learning method based on feedback training, which is characterized in that this method has supervision depth in training
During learning model, when each iteration starts, each sample in training set is sampled with a sampled probability, it is described
Sampled probability is adjusted with the prediction penalty values dynamic of each sample.
2. according to claim 1 have supervision deep learning method based on feedback training, which is characterized in that the sampling
The process of probability dynamic adjustment specifically includes:
1) each sample weight parameter is initialized；
2) according to various kinds should have before weight parameter calculate corresponding sampled probability:
Wherein, P (i) is the sampled probability of sample i, and α is priority factor, piFor the weight parameter of sample i；
3) after carrying out an iteration, the prediction penalty values of each sample are obtained, update weight parameter based on the prediction penalty values；
4) when next iteration starts, p is enabledi=p (i), return step 2).
3. according to claim 2 have supervision deep learning method based on feedback training, which is characterized in that described initial
When changing each sample weight parameter, enabling each sample weight parameter is 1.
4. according to claim 2 have supervision deep learning method based on feedback training, which is characterized in that described to be based on
The prediction penalty values update weight parameter specifically:
P (i)=| δ (i) |+ε
Wherein, p (i) is the weight parameter of updated sample i, and δ (i) is the prediction penalty values of sample i, and ε is modifying factor.
5. according to claim 4 have supervision deep learning method based on feedback training, which is characterized in that the amendment
Factor ε is a positive number for being greater than 0.
6. according to claim 4 have supervision deep learning method based on feedback training, which is characterized in that the prediction
The expression formula of penalty values δ (i) are as follows:
δ (i)=L (yi,f(xi))
Wherein, xiFor input, yiFor xiCorresponding true value label, function f are by inputting xiThe function of prediction label, function L are
Calculate true value label yiWith prediction label f (xi) difference loss function.
7. according to claim 2 have supervision deep learning method based on feedback training, which is characterized in that described to be based on
When the prediction penalty values update weight parameter, weight parameter is directly proportional to the inverse of prediction penalty values.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811367393.4A CN109583485B (en) | 2018-11-16 | 2018-11-16 | Supervised deep learning method based on feedback training |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811367393.4A CN109583485B (en) | 2018-11-16 | 2018-11-16 | Supervised deep learning method based on feedback training |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109583485A true CN109583485A (en) | 2019-04-05 |
| CN109583485B CN109583485B (en) | 2023-12-08 |
Family
ID=65922667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811367393.4A Active CN109583485B (en) | 2018-11-16 | 2018-11-16 | Supervised deep learning method based on feedback training |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109583485B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112052900A (en) * | 2020-09-04 | 2020-12-08 | 京东数字科技控股股份有限公司 | Machine learning sample weight adjusting method and device and storage medium |
| CN113420792A (en) * | 2021-06-03 | 2021-09-21 | 阿波罗智联(北京)科技有限公司 | Training method of image model, electronic equipment, road side equipment and cloud control platform |
| CN116484744A (en) * | 2023-05-12 | 2023-07-25 | 北京百度网讯科技有限公司 | Object simulation method, model training method, device, equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104102700A (en) * | 2014-07-04 | 2014-10-15 | 华南理工大学 | Categorizing method oriented to Internet unbalanced application flow |
| US20150332169A1 (en) * | 2014-05-15 | 2015-11-19 | International Business Machines Corporation | Introducing user trustworthiness in implicit feedback based search result ranking |
| CN105096375A (en) * | 2014-05-09 | 2015-11-25 | 三星电子株式会社 | Image processing method and apparatus |
-
2018
- 2018-11-16 CN CN201811367393.4A patent/CN109583485B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105096375A (en) * | 2014-05-09 | 2015-11-25 | 三星电子株式会社 | Image processing method and apparatus |
| US20150332169A1 (en) * | 2014-05-15 | 2015-11-19 | International Business Machines Corporation | Introducing user trustworthiness in implicit feedback based search result ranking |
| CN104102700A (en) * | 2014-07-04 | 2014-10-15 | 华南理工大学 | Categorizing method oriented to Internet unbalanced application flow |
Non-Patent Citations (1)
| Title |
|---|
| 余成波;田桐;熊递恩;许琳英;: "中心损失与Softmax损失联合监督下的人脸识别", 重庆大学学报, no. 05 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112052900A (en) * | 2020-09-04 | 2020-12-08 | 京东数字科技控股股份有限公司 | Machine learning sample weight adjusting method and device and storage medium |
| CN112052900B (en) * | 2020-09-04 | 2024-05-24 | 京东科技控股股份有限公司 | Machine learning sample weight adjustment method and device, and storage medium |
| CN113420792A (en) * | 2021-06-03 | 2021-09-21 | 阿波罗智联(北京)科技有限公司 | Training method of image model, electronic equipment, road side equipment and cloud control platform |
| CN116484744A (en) * | 2023-05-12 | 2023-07-25 | 北京百度网讯科技有限公司 | Object simulation method, model training method, device, equipment and storage medium |
| CN116484744B (en) * | 2023-05-12 | 2024-01-16 | 北京百度网讯科技有限公司 | Object simulation method, model training method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109583485B (en) | 2023-12-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105335756B (en) | A kind of image classification method and image classification system based on Robust Learning model | |
| CN108416370A (en) | Image classification method, device based on semi-supervised deep learning and storage medium | |
| CN107065842B (en) | A kind of fault detection method based on particle group optimizing core independent component analysis model | |
| CN109583485A (en) | It is a kind of that supervision deep learning method is had based on feedback training | |
| CN110084221A (en) | A kind of serializing face critical point detection method of the tape relay supervision based on deep learning | |
| CN106228182B (en) | SAR image classification method based on SPM and depth increments SVM | |
| CN108229532A (en) | Image-recognizing method, device and electronic equipment | |
| CN114155397B (en) | Small sample image classification method and system | |
| CN108664986A (en) | Based on lpThe multi-task learning image classification method and system of norm regularization | |
| CN109325513A (en) | A kind of image classification network training method based on magnanimity list class single image | |
| CN113128478A (en) | Model training method, pedestrian analysis method, device, equipment and storage medium | |
| CN108564569B (en) | A kind of distress in concrete detection method and device based on multicore classification learning | |
| CN113989519B (en) | Long-tail target detection method and system | |
| CN113109782B (en) | Classification method directly applied to radar radiation source amplitude sequence | |
| CN110288026A (en) | A kind of image partition method and device practised based on metric relation graphics | |
| CN109902589A (en) | A kind of target identification method and system based on Radar range profile's | |
| Hang et al. | Surface defect detection in sanitary ceramics based on lightweight object detection network | |
| CN110929809A (en) | Soft measurement method for key water quality index of sewage by using characteristic self-enhanced circulating neural network | |
| CN109614999A (en) | A kind of data processing method, device, equipment and computer readable storage medium | |
| CN116486151A (en) | Image classification model training method, image classification method, device and storage medium | |
| CN103761530B (en) | Hyperspectral image unmixing method based on relevance vector machine | |
| CN112348700B (en) | Line capacity prediction method combining SOM clustering and IFOU equation | |
| CN109977797A (en) | The optimization method of single order object detector based on sequence loss function | |
| CN109740109A (en) | A kind of PolSAR image broad object decomposition method based on unitary transformation | |
| CN114078203A (en) | Image recognition method and system based on improved PATE |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |