CN107463953A - Image classification method and system based on quality insertion in the case of label is noisy - Google Patents
Image classification method and system based on quality insertion in the case of label is noisy Download PDFInfo
- Publication number
- CN107463953A CN107463953A CN201710599924.1A CN201710599924A CN107463953A CN 107463953 A CN107463953 A CN 107463953A CN 201710599924 A CN201710599924 A CN 201710599924A CN 107463953 A CN107463953 A CN 107463953A
- Authority
- CN
- China
- Prior art keywords
- label
- model
- msub
- mrow
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The present invention provides a kind of image classification method and system based on quality insertion in the case of label is noisy, including：Network picture tag collection step；Label quality factor Embedded step：In the image classification model for have supervision introduce the label quality factor, for control band make an uproar label predicted value generation and absorb the error back information from error label；Using log-likelihood function is maximized, the optimization object function added after the label quality factor is designed；Network model construction step：Optimization object function is modeled using deep neural network；Network parameter training step：Picture and label with noise will be trained to input network model, linked end to end training pattern using the stochastic gradient descent method of mutation, while update model parameter；Image classification step.Picture true tag, user are provided label and picture tag quality three variable unified Modelings by the present invention, are formed the supervised learning to noisy label, can be obtained accurate image classification result.
Description
Technical field
The present invention relates to computer vision and Data Mining, and specifically, more particularly to label contains under noise situations
Picture tag learning method and system.
Background technology
Image recognition is a basis of artificial intelligence field and important task, its application span natural science, is cured
The multiple fields such as pharmacy, industry.With the fast development of deep learning, the image classification for training to obtain using convolutional neural networks
Device obtains unprecedented success.But training number of the image classification study dependent on extensive high-quality under deep learning framework
According to, including clearly image and accurate label.For such training data often from artificially collecting and marking, this will consumption
Substantial amounts of manpower and materials so that handling the problem of image recognition of frontier becomes relatively expensive and poorly efficient.
Due to network technology, social media develop rapidly and people are had deep love for from media network, in internet
Innumerable image data be present.Picture social platform such as Flickr and Netease LOFTER possess nearly millions upon millions of user and provided
Image data and label information.If these pictures and label data can be used for the training of deep neural network model,
The type and quantity of data set will be greatly promoted, help deep neural network faster to move to the image recognition of different field
In problem.
The picture and label uploaded using Internet user can be very good solve handmarking's data as training data
Limitation, but also bring along the problem of corresponding and challenge.Large-sized artificial flag data collection provide image data quality better and
Label is complete, therefore high based on the neural network classifier model accuracy rate that such data set trains to obtain.By contrast, network
The inaccurate characteristic of the bad label of quality be present in picture and user tag.If utilize the picture tag number that much noise be present
According to the prediction reliability of model can be substantially reduced.Therefore, research how to make full use of network picture and user provide label this
One inexhaustible data resource, carry out effective picture tag and learn to have obtained more concerns.
It is traditional to there is the loss function of design robust, statistics to look into using the band method that label carries out picture tag study of making an uproar
Inquiry, analogue noise characteristic etc..Some of methods need a part of clean label data to carry out supplemental training Image Classifier；
Other attempts to establish the distribution of modeling data noise, and true tag and user provide the difference between label in picture
The noise brought, the picture quality quality not accounted for but, and user provide the order of accuarcy of label, its Classification and Identification
Effect does not reach expection.
The content of the invention
The purpose of the present invention is overcome the deficiencies in the prior art, there is provided one kind is embedded in the case of label is noisy based on quality
Image classification method and system, with solve use the label picture training image grader with noise in the prior art when do not examine
The problem of considering picture tag quality itself.
According to an aspect of the present invention, there is provided a kind of image classification side based on quality insertion in the case of label is noisy
Method, including：
Network picture tag collection step：A large amount of pictures and the label of user's offer are provided from network picture sharing platform
Information, filtered and arranged according to required species, for use in the training of Image Classifier；
Label quality factor Embedded step：The label quality factor is introduced in the image classification model for have supervision, for controlling
Band processed make an uproar label predicted value generation and absorb the error back information from error label；Using maximizing log-likelihood letter
Number, the optimization object function that design is added after the label quality factor；
Network model construction step：Optimization object function is modeled using deep neural network, obtains overall network
Model, it includes four submodels, respectively encoding model, sampling model, decoded model and disaggregated model；
Network parameter training step：The training picture that network picture tag collection step is obtained and the label with noise are defeated
Enter the overall network model that network model construction step obtains, linked end to end training using the stochastic gradient descent method of mutation
Aforementioned four submodel, while update the model parameter of four submodels；
Image classification step：New picture for requiring classification, inputs to the disaggregated model trained, obtains true to picture
The prediction of real label.
Preferably, the network picture tag collection step, has used web crawlers technology, is received on picture social network sites
Collection required for a large amount of pictures and user annotation label, and according to required species to label and picture carry out filtering and it is whole
Reason, for example retain the picture containing one or more labels in total m classes.
Preferably, the label quality factor Embedded step, there is the image classification model of supervision existing, add figure
The insertion of the piece label quality factor, makes the new optimization object function be：
Wherein xmAnd ymBe respectively m pictures pixel set and relative users provide noise label, zmAnd smRespectively
It is the hidden variable of representative picture true tag and label quality, M represents the picture sum for training.New optimization aim letter
For number due to adding the label quality factor, the harmful effect to caused by the label that training data concentrates mistake has absorption.Together
When, because the gradient function of the object function is difficult to calculate, therefore transfer to optimize its evidence lower bound (ELBO) first, utilize simultaneously
Join skill again and simplify the required computing resource of training, obtain final optimization object function formula combinations.
Preferably, the network model construction step, using deep neural network to final optimization object function formula
Combination is modeled respectively, obtains overall network model, it includes four submodels：Encoding model, sampling model, decoded model
And disaggregated model；
Wherein, the encoding model, using convolutional neural networks, for generating noise label from image content XPriori
PredictionAnd combine noise label y q (S | X, Y) and true tag distribution q (Z | X, Y) are distributed to label quality and be predicted.
Wherein, the sampling model, for the label quality distribution q (S | X, Y) and true tag for generating encoding model
Distribution q (Z | X, Y) it is mapped as explicit value S and Z.
Wherein, the decoded model, used method are neutral net, and it inputs the output label matter for sampling model
S and true tag Z is measured, q (Y | Z, S) is predicted the posteriority of noise label for generating.
Wherein, the disaggregated model, used method are convolutional neural networks, and it is generated to true tag using picture
Z prediction.
Preferably, the network parameter training step, the noise label posteriority prediction q recovered using decoded model (Y | Z,
S the model training for having supervision) is carried out, calculation code model, sampling model, the passback gradient of decoded model, updates these three sons
The parameter of model, meanwhile, it is distributed q (Z | X, Y) using the true tag that is obtained in encoding model and prison has been carried out to disaggregated model
The model training superintended and directed, neutral net passback gradient is calculated, update the parameter of disaggregated model.
Preferably, described image classifying step, the classification mould that the picture input of required progress image classification is trained
In type, the prediction to image true tag is obtained, while produces the classification results of image.
According to the second aspect of the invention, there is provided a kind of image classification system based on quality insertion in the case of label is noisy
System, including：
Network picture tag collection module：A large amount of pictures and the label of user's offer are provided from network picture sharing platform
Information is simultaneously filtered and arranged according to required species；
The label quality factor is embedded in module：The label quality factor is introduced in the image classification model that tradition has supervision to control
Band processed make an uproar label predicted value generation and absorb the error back information from error label, calculate image classification model corresponding to
Optimization object function of the log-likelihood function as training；
Network model builds module：For being modeled using deep neural network to the optimization object function, respectively
Obtain encoding model, sampling model and decoded model and four submodels of disaggregated model；
Network parameter training module：Training picture and the label with noise are inputted into overall network model, use mutation
Stochastic gradient descent method links end to end trains four submodels, while updates the parameter of this four submodels；
New images classification task processing module：New picture for requiring classification, inputs to the disaggregated model trained, obtains
To the prediction to picture true tag.
Preferably, label quality factor insertion module, there is the image classification model of supervision existing, add figure
The insertion of the piece label quality factor, makes the new optimization object function be：
Wherein xmAnd ymBe respectively m pictures pixel set and relative users provide noise label, zmAnd smRespectively
It is the hidden variable of representative picture true tag and label quality, M represents the picture sum for training；
New optimization object function is due to adding the label quality factor, caused by the label for concentrate mistake to training data
Harmful effect has absorption, meanwhile, the gradient function of the new optimization object function is difficult to calculate, therefore transfers optimization first
Its evidence lower bound, while simplify the required computing resource of training using skill is joined again, obtain final optimization object function formula
Combination.
Preferably, the network model structure module, using deep neural network to final optimization object function formula
Combination is modeled respectively, obtains four models：Encoding model, sampling model, decoded model and disaggregated model；Wherein：
The encoding model, using convolutional neural networks, for generating noise label from image content XPriori predictionAnd combine noise label y q (S | X, Y) and true tag distribution q (Z | X, Y) are distributed to label quality and be predicted；
The sampling model, label quality distribution q (S | X, Y) and true tag for encoding model to be generated are distributed q
(Z | X, Y) it is mapped as explicit value S and Z；
The decoded model, used method are neutral net, its input for sampling model output label quality S and
True tag Z, q (Y | Z, S) is predicted the posteriority of noise label for generating；
The disaggregated model, using convolutional neural networks, it generates the prediction to true tag Z using picture.
Preferably, the network parameter training module, the noise label posteriority prediction q recovered using decoded model (Y | Z,
S the model training for having supervision) is carried out, calculation code model, sampling model, the passback gradient of decoded model, updates these three sons
The parameter of model, meanwhile, it is distributed q (Z | X, Y) using the true tag that is obtained in encoding model and prison has been carried out to disaggregated model
The model training superintended and directed, neutral net passback gradient is calculated, update the parameter of disaggregated model.
Preferably, the network picture tag collection module, has used web crawlers technology, is received on picture social network sites
The label of a large amount of pictures and user annotation required for collection.
The present invention is to provide picture true tag, user to three label, picture tag quality variable unified Modelings, training
The true tag of input image data every time is not only predicted during grader, and deduces the matter for the picture tag that user is uploaded
Amount, and then the supervised learning to noisy label is formed, continuous iteration restrains until training, obtains required Image Classifier, uses
In the classification task of new image.
Compared with prior art, the present invention has following beneficial effect：
The present invention is embedded in image classification model to represent mark by the picture tag data in deep excavation social media
The hidden variable of quality is signed, improves the grader mode of learning of existing label of being made an uproar using band.Returned by redesigning error
Gradient formula and corresponding neural network model is constructed, the noisy mark provided true tag, picture tag quality and user
Label are predicted simultaneously, so as to effectively absorb the wrong back information in neural metwork training caused by label noise, favorably
In the correct study of picture classification device.
It will be helpful in social media will largely exist using the present invention and cheap available picture tag data be used to scheme
As the training of grader, so as to effectively save the manpower and materials needed for specialty mark, while label noise is avoided to instruct grader
Experienced influence, obtain accurate image classification result.
Brief description of the drawings
The detailed description made by reading with reference to the following drawings to non-limiting example, further feature of the invention,
Objects and advantages will become more apparent upon：
Fig. 1 is the method flow diagram in one embodiment of the invention；
Fig. 2 has supervision image classification model for one embodiment of the invention introducing label quality factor；
Fig. 3 is the structural map of the deep neural network model modules used in one embodiment of the invention；
Fig. 4 is system block diagram in one embodiment of the invention.
Embodiment
With reference to specific embodiment, the present invention is described in detail.Following examples will be helpful to the technology of this area
Personnel further understand the present invention, but the invention is not limited in any way.It should be pointed out that the ordinary skill to this area
For personnel, without departing from the inventive concept of the premise, various modifications and improvements can be made.These belong to the present invention
Protection domain.
The present invention, which considers, provides picture true tag, user to three label, picture tag quality variables, proposes
Image Classfication Technology based on quality insertion in the case of label is noisy.Divided according to the realization of general technical, main point four
Point：
(1) network picture tag is collected；
(2) the label quality factor is embedded in；
(3) network model structure and parameter training；
(4) new image classification task processing.
Above-mentioned four part constitutes image classification method and system in the present invention, in order to be better understood on the present invention, with
Under in conjunction with the embodiments to the present invention method and system realize be introduced.
As shown in figure 1, the flow chart of the sorting technique provided for the present embodiment, wherein：
(1) network picture tag is collected；
Screening zone has required tag along sort on data set YFCC100M disclosed in based on photo sharing website Flickr
Picture, and using web crawlers technology be collected download and arrange, altogether obtain M pictures.
(2) the label quality factor is embedded in；
As shown in Fig. 2 introducing label quality factor S in the existing image classification model for having supervision, label quality is established
The factor and the relation of its dependent variable.It is theoretical according to graph model, log-likelihood function lnP (Y | X) is rewritten as：
Wherein X representative pictures properties collection, Y represent the noise tag set of user's offer, xmAnd ymIt is picture m respectively
The noise label that content and relative users provide, zmAnd smIt is the hidden variable of representative picture true tag and label quality respectively,
E, which is represented, it is expected.
In this step, provide label quality and true tag and user to label unified Modeling, be advantageous to correct understanding
Association between noise label and true tag, the influence that noise label is trained to picture classification device is reduced, and then improve and marking
Training obtains the degree of accuracy of picture classification device under label tape noise situations.The target letter for needing to optimize is calculated by the log-likelihood function
Number:
1. the thought inferred according to variation, and it is excellent using Jensen ineguality (Jensen's inequality) calculating needs
The evidence lower bound (ELBO) of the object function of change, shares three, is respectively
And
So as to simplify the amount of calculation needed for training grader；
Wherein xmAnd ymBe respectively picture m content and relative users provide noise label, zmAnd smIt is representative graph respectively
The hidden variable of piece true tag and label quality, DKL[| |] relative entropy is represented, E, which is represented, it is expected.
2. calculate the explicit expression of two relative entropy expression formulas, it is assumed that true tag variable obeys q (zm|xm,ym) and P
(zm|xm) K dimension multinomial distribution, label quality variable probability P (sm), obedience average is μ (xm,ym), covariance is diag (σ
(xm,ym)) multivariate Gaussian distribution；
3. build true tag z using skill is joined againmWith label quality smTo supplement stochastic variable γmAnd ζmMapping, can
To solve the sample variance problems of too that traditional Monte Carlo method is brought, the stability of training grader is improved.
Utilize (Geng Beier-normalization exponential function, Gumbel-SoftMax Function) construction of function true tag zm
To Geng Beier variables γmMapping, zm=g (γm)；
Construct label quality smTo standard normal variable ζm~N (0,1) mapping, sm=μ (xm,ym)+σ2(xm,ym)eζm,
Wherein μ (xm,ym) it is label quality smAverage, σ2(xm,ym) it is label quality smVariance.
Accordingly, calculate in evidence lower bound and it is expected item's
Explicit expression, the object function that this is optimized as needs.
(3) network model structure and parameter training；
1. as shown in figure 3, the evidence lower bound expression of the object function optimized as needed, is carried out whole using neutral net
Volume modeling, obtained network model include four submodels, are divided into 4 submodels：Encoding model, sampling model, decode mould
Type, disaggregated model.When a pictures and its noisy label input overall network model, overall network model are first begin to positive biography
Broadcast：
A) encoding model according to image content x and band make an uproar label y to true tag z and label quality s distribution q (z | x,
Y) made prediction with q (s | x, y).
Probability distribution q that b) sampling model provides according to encoding model (z | x, y) and q (s | x, y), sample out true tag
Z and label quality s occurrence.
C) the true tag z and label quality s occurrence input neutral net that decoded model obtains sampling model, are obtained
To the prediction to noise label, so as to intersect entropy loss with the noisy tag computation that picture provides.
D) one neural network classification model of stand-alone training, for the true tag P (z) of predicted pictures, and calculate and compile
Relative entropy between the true tag z distribution q (z | x, y) that code model provides, so as to couple the sub- model of its above-mentioned excess-three.
Wherein, the encoding model, using convolutional neural networks, for generating the priori prediction P of label by image content x
(y), and combine noise label y q (s | x, y) and true tag distribution q (z | x, y) are distributed to label quality and be predicted.Wherein,
The distribution of generation label quality is realized that the distribution for generating true tag is realized by addition layer by equivalent beds.
Wherein, the sampling model, for the label quality distribution q (s | x, y) and true tag for generating encoding model
Distribution q (z | x, y) it is mapped as explicit value, including label quality s and true tag z.Taken in sampling model and join skill again, from
And reduce the variance of sampled result so that model training is more stable.
Wherein, the decoded model, using neutral net, it is inputted as the output label quality s of sampling model and true
Label z, for recovering the prediction of noise label by noise floor
Wherein, the neural network classification model, using convolutional neural networks, its input is image content x, and output is pair
The prediction P (z) of picture true tag, and utilize the true tag distribution q obtained in encoding model, decoded model running
(z | x, y) relative entropy is calculated, the training for having supervision is carried out, obtains required Image Classifier.
2. by the output of decoded model, i.e. band is made an uproar Tag EstimationBand is provided with user to make an uproar label y, together entrance loss
Layer counting loss, and utilize the network parameter of each submodel of stochastic gradient descent method renewal.
3. through excessive wheel feedback iteration, until neutral net restrains, training is completed.
(4) new image classification task processing.
With the neural network image disaggregated model that training is completed in (three), when the unlabelled new picture of classification in need
When, the disaggregated model trained is input to, obtains the prediction to picture true tag.
As shown in figure 4, in another embodiment, corresponding to the above method, one kind is based on quality in the case of label is noisy
The embodiment of embedded image classification system, including：
Network picture tag collection module：A large amount of pictures and the label of user's offer are provided from network picture sharing platform
Information is simultaneously filtered and arranged according to required species；
The label quality factor is embedded in module：For introducing the label quality factor in traditional image classification model for having supervision
Come control band make an uproar label predicted value generation and absorb the error back information from error label, calculate overall network model
Optimization object function of the log-likelihood function as training corresponding to (the image classification model after embedded quality factor)；
Network model builds module：For being modeled using deep neural network to optimization object function, respectively obtain
Encoding model, sampling model and decoded model and four submodels of disaggregated model；
Network parameter training module：Training picture and the label with noise are inputted into overall network model, use mutation
Stochastic gradient descent method links end to end trains four submodels, while updates the parameter of four submodels；
New image classification task processing module：New picture for requiring classification, input to the disaggregated model trained,
Obtain the prediction to picture true tag.
The specific module of the above-mentioned image classification system system based on quality insertion in the case of label is noisy realizes skill
Art feature is corresponding with each step of the image classification method based on quality insertion in the case of label is noisy.
It should be noted that the step in method provided by the invention, can utilize corresponding module, dress in the system
Put, the step of unit etc. is achieved, and the technical scheme that those skilled in the art are referred to the system realizes methods described
Flow, i.e. the embodiment in the system can be regarded as realizing the preference of methods described, will not be described here.
One skilled in the art will appreciate that except realizing system provided by the invention in a manner of pure computer readable program code
And its beyond each device, completely can by by method and step carry out programming in logic come system provided by the invention and its
Each device is in the form of gate, switch, application specific integrated circuit, programmable logic controller (PLC) and embedded microcontroller etc.
To realize identical function.So system provided by the invention and its every device are considered a kind of hardware component, and it is right
What is included in it is used to realize that the device of various functions can also to be considered as the structure in hardware component；It will can also be used to realize respectively
The device of kind of function, which is considered as, not only can be the software module of implementation method but also can be the structure in hardware component.
The specific embodiment of the present invention is described above.It is to be appreciated that the invention is not limited in above-mentioned
Particular implementation, those skilled in the art can make a variety of changes or change within the scope of the claims, this not shadow
Ring the substantive content of the present invention.In the case where not conflicting, the feature in embodiments herein and embodiment can any phase
Mutually combination.
Claims (10)
- A kind of 1. image classification method based on quality insertion in the case of label is noisy, it is characterised in that：Including：Network picture tag collection step：A large amount of pictures and the label letter of user's offer are provided from network picture sharing platform Breath, is filtered and is arranged according to required species, for use in the training of Image Classifier；Label quality factor Embedded step：The label quality factor is introduced in the image classification model for have supervision, for controlling band Make an uproar label predicted value generation and absorb the error back information from error label；Using maximizing log-likelihood function, if The optimization object function that meter is added after the label quality factor；Network model construction step：Optimization object function is modeled using deep neural network, obtains four models, respectively For encoding model, sampling model, decoded model and disaggregated model；Network parameter training step：The training picture that network picture tag collection step is obtained and the label with noise input net The above-mentioned network model that network model construction step obtains, being linked end to end using the stochastic gradient descent method of mutation, it is above-mentioned to train Four models, while model parameter is updated, the network model trained；Image classification step：New picture for requiring classification, inputs to the disaggregated model trained, obtains to the true mark of picture The prediction of label, while produce the classification results of image.
- 2. the image classification method based on quality insertion in the case of label is noisy according to claim 1, it is characterised in that： In the label quality factor Embedded step, have the image classification model of supervision existing, add picture tag quality because The insertion of son, makes the new optimization object function be：<mrow> <mi>ln</mi> <mi> </mi> <mi>P</mi> <mrow> <mo>(</mo> <mi>Y</mi> <mo>|</mo> <mi>X</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mo>&Sigma;</mo> <mrow> <mi>m</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>ln</mi> <mi> </mi> <mi>P</mi> <mrow> <mo>(</mo> <msub> <mi>y</mi> <mi>m</mi> </msub> <mo>|</mo> <msub> <mi>x</mi> <mi>m</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mo>&Sigma;</mo> <mrow> <mi>m</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>ln</mi> <mi> </mi> <msub> <mi>E</mi> <mrow> <mi>P</mi> <mrow> <mo>(</mo> <msub> <mi>z</mi> <mi>m</mi> </msub> <mo>|</mo> <msub> <mi>x</mi> <mi>m</mi> </msub> <mo>)</mo> </mrow> <mo>,</mo> <mi>P</mi> <mrow> <mo>(</mo> <msub> <mi>s</mi> <mi>m</mi> </msub> <mo>)</mo> </mrow> </mrow> </msub> <mo>&lsqb;</mo> <mi>P</mi> <mrow> <mo>(</mo> <msub> <mi>y</mi> <mi>m</mi> </msub> <mo>|</mo> <msub> <mi>z</mi> <mi>m</mi> </msub> <mo>,</mo> <msub> <mi>s</mi> <mi>m</mi> </msub> <mo>)</mo> </mrow> <mo>&rsqb;</mo> </mrow>Wherein xmAnd ymBe respectively m pictures pixel set and relative users provide noise label, zmAnd smIt is generation respectively The hidden variable of table picture true tag and label quality, M represent the picture sum for training；New optimization object function is bad caused by the label for concentrate mistake to training data due to adding the label quality factor Influence has absorption, meanwhile, the gradient function of the new optimization object function is difficult to calculate, therefore transfers to optimize its card first Simplify the required computing resource of training according to lower bound, while using skill is joined again, obtain final optimization object function formula combinations.
- 3. the image classification method based on quality insertion in the case of label is noisy according to claim 2, it is characterised in that： The network model construction step, final optimization object function formula combinations are built respectively using deep neural network Mould, obtain four models：Encoding model, sampling model, decoded model and disaggregated model；Wherein：The encoding model, using convolutional neural networks, for generating noise label from image content XPriori prediction And combine noise label y q (S | X, Y) and true tag distribution q (Z | X, Y) are distributed to label quality and be predicted；The sampling model, label quality distribution q (S | X, Y) and true tag for encoding model to be generated be distributed q (Z | X, Y explicit value S and Z) are mapped as；The decoded model, using neutral net, it inputs the output label quality S and true tag Z for sampling model, is used for Generate and predict the posteriority of noise label q (Y | Z, S)；The disaggregated model, using convolutional neural networks, it generates the prediction to true tag Z using picture.
- 4. the image classification method based on quality insertion in the case of label is noisy according to claim 3, it is characterised in that： The network parameter training step, the noise label posteriority recovered using decoded model predict that q (Y | Z, S) carries out the mould for having supervision Type training, calculation code model, sampling model, the passback gradient of decoded model, model parameter is updated, meanwhile, using encoding The true tag distribution q (Z | X, Y) obtained in model carries out the model training for having supervision to disaggregated model, calculates neutral net and returns Gradient is passed, updates model parameter.
- 5. the image classification method according to claim any one of 1-4 based on quality insertion in the case of label is noisy, its It is characterised by：In the network picture tag collection step, web crawlers technology has been used, institute is collected on picture social network sites The label of a large amount of pictures and user annotation that need.
- A kind of 6. image classification system based on quality insertion in the case of label is noisy, it is characterised in that：Including：Network picture tag collection module：A large amount of pictures and the label information of user's offer are provided from network picture sharing platform And filtered and arranged according to required species；The label quality factor is embedded in module：The label quality factor is introduced in the image classification model that tradition has supervision to control band Make an uproar label predicted value generation and absorb the error back information from error label, calculate image classification model corresponding to logarithm Optimization object function of the likelihood function as training；Network model builds module：For being modeled using deep neural network to the optimization object function, respectively obtain Encoding model, sampling model and decoded model and four models of disaggregated model；Network parameter training module：Training picture and the label with noise are inputted into network model, use the stochastic gradient of mutation Descent method links end to end trains four models, while updates model parameter；New images classification task processing module：New picture for requiring classification, input are obtained pair to the disaggregated model trained The prediction of picture true tag.
- 7. the image classification system based on quality insertion in the case of label is noisy according to claim 6, it is characterised in that： The label quality factor is embedded in module, has the image classification model of supervision existing, adds picture tag quality factor Insertion, make the new optimization object function be：<mrow> <mi>ln</mi> <mi> </mi> <mi>P</mi> <mrow> <mo>(</mo> <mi>Y</mi> <mo>|</mo> <mi>X</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mo>&Sigma;</mo> <mrow> <mi>m</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>ln</mi> <mi> </mi> <mi>P</mi> <mrow> <mo>(</mo> <msub> <mi>y</mi> <mi>m</mi> </msub> <mo>|</mo> <msub> <mi>x</mi> <mi>m</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mo>&Sigma;</mo> <mrow> <mi>m</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>ln</mi> <mi> </mi> <msub> <mi>E</mi> <mrow> <mi>P</mi> <mrow> <mo>(</mo> <msub> <mi>z</mi> <mi>m</mi> </msub> <mo>|</mo> <msub> <mi>x</mi> <mi>m</mi> </msub> <mo>)</mo> </mrow> <mo>,</mo> <mi>P</mi> <mrow> <mo>(</mo> <msub> <mi>s</mi> <mi>m</mi> </msub> <mo>)</mo> </mrow> </mrow> </msub> <mo>&lsqb;</mo> <mi>P</mi> <mrow> <mo>(</mo> <msub> <mi>y</mi> <mi>m</mi> </msub> <mo>|</mo> <msub> <mi>z</mi> <mi>m</mi> </msub> <mo>,</mo> <msub> <mi>s</mi> <mi>m</mi> </msub> <mo>)</mo> </mrow> <mo>&rsqb;</mo> </mrow>Wherein xmAnd ymBe respectively m pictures pixel set and relative users provide noise label, zmAnd smIt is generation respectively The hidden variable of table picture true tag and label quality, M represent the picture sum for training；New optimization object function is bad caused by the label for concentrate mistake to training data due to adding the label quality factor Influence has absorption, meanwhile, the gradient function of the new optimization object function is difficult to calculate, therefore transfers to optimize its card first Simplify the required computing resource of training according to lower bound, while using skill is joined again, obtain final optimization object function formula combinations.
- 8. the image classification method based on quality insertion in the case of label is noisy according to claim 7, it is characterised in that： The network model builds module, and final optimization object function formula combinations are built respectively using deep neural network Mould, obtain four models：Encoding model, sampling model, decoded model and disaggregated model；Wherein：The encoding model, using convolutional neural networks, for generating noise label from image content XPriori prediction And combine noise label y q (S | X, Y) and true tag distribution q (Z | X, Y) are distributed to label quality and be predicted；The sampling model, label quality distribution q (S | X, Y) and true tag for encoding model to be generated be distributed q (Z | X, Y explicit value S and Z) are mapped as；The decoded model, used method are neutral net, and it is inputted as the output label quality S of sampling model and true Label Z, q (Y | Z, S) is predicted the posteriority of noise label for generating；The disaggregated model, used method are convolutional neural networks, and it generates the prediction to true tag Z using picture.
- 9. the image classification method based on quality insertion in the case of label is noisy according to claim 8, it is characterised in that： The network parameter training module, the noise label posteriority recovered using decoded model predict that q (Y | Z, S) carries out the mould for having supervision Type training, calculation code model, sampling model, the passback gradient of decoded model, model parameter is updated, meanwhile, using encoding The true tag distribution q (Z | X, Y) obtained in model carries out the model training for having supervision to disaggregated model, calculates neutral net and returns Gradient is passed, updates model parameter.
- 10. the image classification system according to claim any one of 6-7 based on quality insertion in the case of label is noisy, its It is characterised by：The network picture tag collection module, has used web crawlers technology, on picture social network sites needed for collection The label of a large amount of pictures and user annotation wanted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710599924.1A CN107463953B (en) | 2017-07-21 | 2017-07-21 | Image classification method and system based on quality insertion in the noisy situation of label |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710599924.1A CN107463953B (en) | 2017-07-21 | 2017-07-21 | Image classification method and system based on quality insertion in the noisy situation of label |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107463953A true CN107463953A (en) | 2017-12-12 |
| CN107463953B CN107463953B (en) | 2019-11-19 |
Family
ID=60543879
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710599924.1A Active CN107463953B (en) | 2017-07-21 | 2017-07-21 | Image classification method and system based on quality insertion in the noisy situation of label |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107463953B (en) |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734227A (en) * | 2018-06-13 | 2018-11-02 | 北京宏岸图升网络技术有限公司 | A kind of sorting technique and device of picture |
| CN109189767A (en) * | 2018-08-01 | 2019-01-11 | 北京三快在线科技有限公司 | Data processing method, device, electronic equipment and storage medium |
| CN109242106A (en) * | 2018-09-07 | 2019-01-18 | 百度在线网络技术（北京）有限公司 | sample processing method, device, equipment and storage medium |
| CN109976153A (en) * | 2019-03-01 | 2019-07-05 | 北京三快在线科技有限公司 | Control the method, apparatus and electronic equipment of unmanned equipment and model training |
| CN110110780A (en) * | 2019-04-30 | 2019-08-09 | 南开大学 | A kind of picture classification method based on confrontation neural network and magnanimity noise data |
| CN110188791A (en) * | 2019-04-18 | 2019-08-30 | 南开大学 | Based on the visual emotion label distribution forecasting method estimated automatically |
| CN110415094A (en) * | 2019-06-18 | 2019-11-05 | 重庆金融资产交易所有限责任公司 | Asset-liabilities intelligent management, device and computer readable storage medium |
| CN110738264A (en) * | 2019-10-18 | 2020-01-31 | 上海眼控科技股份有限公司 | Abnormal sample screening, cleaning and training method, device, equipment and storage medium |
| CN110751170A (en) * | 2019-09-06 | 2020-02-04 | 武汉精立电子技术有限公司 | Panel quality detection method, system, terminal device and computer readable medium |
| CN110837926A (en) * | 2019-11-04 | 2020-02-25 | 四川省烟草公司广元市公司 | Tobacco main pest and disease damage prediction method based on big data |
| CN110852983A (en) * | 2018-07-27 | 2020-02-28 | 三星电子株式会社 | Method for detecting defects in semiconductor device |
| CN110910356A (en) * | 2019-11-08 | 2020-03-24 | 北京华宇信息技术有限公司 | Method for generating image noise detection model, image noise detection method and device |
| WO2020107264A1 (en) * | 2018-11-28 | 2020-06-04 | 深圳市大疆创新科技有限公司 | Neural network architecture search method and apparatus |
| CN111507419A (en) * | 2020-04-22 | 2020-08-07 | 腾讯科技（深圳）有限公司 | Training method and device of image classification model |
| CN111797854A (en) * | 2019-04-09 | 2020-10-20 | Oppo广东移动通信有限公司 | Scene model establishing method and device, storage medium and electronic equipment |
| CN112364993A (en) * | 2021-01-13 | 2021-02-12 | 深圳市友杰智新科技有限公司 | Model joint training method and device, computer equipment and storage medium |
| CN112418327A (en) * | 2020-11-25 | 2021-02-26 | Oppo广东移动通信有限公司 | Training method and device of image classification model, electronic equipment and storage medium |
| CN112633310A (en) * | 2019-09-24 | 2021-04-09 | 博世有限公司 | Method and system for classifying sensor data with improved training robustness |
| CN113206824A (en) * | 2021-03-23 | 2021-08-03 | 中国科学院信息工程研究所 | Dynamic network abnormal attack detection method and device, electronic equipment and storage medium |
| CN113284142A (en) * | 2021-07-16 | 2021-08-20 | 腾讯科技（深圳）有限公司 | Image detection method, image detection device, computer-readable storage medium and computer equipment |
| CN113673591A (en) * | 2021-08-13 | 2021-11-19 | 上海交通大学 | Image classification method, device and medium for self-adjusting sampling optimization |
| WO2022073414A1 (en) * | 2020-10-09 | 2022-04-14 | 腾讯科技（深圳）有限公司 | Image recognition method and apparatus, computing device and computer-readable storage medium |
| CN114769072A (en) * | 2022-06-16 | 2022-07-22 | 深圳徕科技术有限公司 | High-speed injection valve control method and device, electronic equipment and storage medium |
| CN116047987A (en) * | 2023-03-31 | 2023-05-02 | 福建天甫电子材料有限公司 | Intelligent control system for producing electronic-grade buffer oxide etching solution |
| CN117523213A (en) * | 2024-01-04 | 2024-02-06 | 南京航空航天大学 | Noise tag identification method based on meta denoising and negative learning |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7657102B2 (en) * | 2003-08-27 | 2010-02-02 | Microsoft Corp. | System and method for fast on-line learning of transformed hidden Markov models |
| CN105160866A (en) * | 2015-08-07 | 2015-12-16 | 浙江高速信息工程技术有限公司 | Traffic flow prediction method based on deep learning nerve network structure |
| CN105224948A (en) * | 2015-09-22 | 2016-01-06 | 清华大学 | A kind of generation method of the largest interval degree of depth generation model based on image procossing |
| CN105612514A (en) * | 2013-08-05 | 2016-05-25 | 脸谱公司 | Systems and methods for image classification by correlating contextual cues with images |
-
2017
- 2017-07-21 CN CN201710599924.1A patent/CN107463953B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7657102B2 (en) * | 2003-08-27 | 2010-02-02 | Microsoft Corp. | System and method for fast on-line learning of transformed hidden Markov models |
| CN105612514A (en) * | 2013-08-05 | 2016-05-25 | 脸谱公司 | Systems and methods for image classification by correlating contextual cues with images |
| CN105160866A (en) * | 2015-08-07 | 2015-12-16 | 浙江高速信息工程技术有限公司 | Traffic flow prediction method based on deep learning nerve network structure |
| CN105224948A (en) * | 2015-09-22 | 2016-01-06 | 清华大学 | A kind of generation method of the largest interval degree of depth generation model based on image procossing |
Non-Patent Citations (4)
| Title |
|---|
| CHAO GAO 等: "Minimax Optimal Convergence Rates for Estimating Ground Truth from Crowdsourced Labels", 《ARXIV》 * |
| ZICHAO YANG 等: "Improved Variational Autoencoders for Text Modeling using Dilated Convolutions", 《ARXIV》 * |
| 余涛: "基于稀疏自编码器的手写体数字识别", 《数字技术与应用》 * |
| 虎晓红 等: "基于Vague融合的图像分类方法", 《计算机工程》 * |
Cited By (35)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734227A (en) * | 2018-06-13 | 2018-11-02 | 北京宏岸图升网络技术有限公司 | A kind of sorting technique and device of picture |
| CN110852983B (en) * | 2018-07-27 | 2024-03-08 | 三星电子株式会社 | Method for detecting defect in semiconductor device |
| CN110852983A (en) * | 2018-07-27 | 2020-02-28 | 三星电子株式会社 | Method for detecting defects in semiconductor device |
| CN109189767A (en) * | 2018-08-01 | 2019-01-11 | 北京三快在线科技有限公司 | Data processing method, device, electronic equipment and storage medium |
| CN109189767B (en) * | 2018-08-01 | 2021-07-23 | 北京三快在线科技有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN109242106A (en) * | 2018-09-07 | 2019-01-18 | 百度在线网络技术（北京）有限公司 | sample processing method, device, equipment and storage medium |
| CN109242106B (en) * | 2018-09-07 | 2022-07-26 | 百度在线网络技术（北京）有限公司 | Sample processing method, device, equipment and storage medium |
| WO2020107264A1 (en) * | 2018-11-28 | 2020-06-04 | 深圳市大疆创新科技有限公司 | Neural network architecture search method and apparatus |
| CN109976153A (en) * | 2019-03-01 | 2019-07-05 | 北京三快在线科技有限公司 | Control the method, apparatus and electronic equipment of unmanned equipment and model training |
| CN111797854B (en) * | 2019-04-09 | 2023-12-15 | Oppo广东移动通信有限公司 | Scene model building method and device, storage medium and electronic equipment |
| CN111797854A (en) * | 2019-04-09 | 2020-10-20 | Oppo广东移动通信有限公司 | Scene model establishing method and device, storage medium and electronic equipment |
| CN110188791A (en) * | 2019-04-18 | 2019-08-30 | 南开大学 | Based on the visual emotion label distribution forecasting method estimated automatically |
| CN110188791B (en) * | 2019-04-18 | 2023-07-07 | 南开大学 | Visual emotion label distribution prediction method based on automatic estimation |
| CN110110780A (en) * | 2019-04-30 | 2019-08-09 | 南开大学 | A kind of picture classification method based on confrontation neural network and magnanimity noise data |
| CN110110780B (en) * | 2019-04-30 | 2023-04-07 | 南开大学 | Image classification method based on antagonistic neural network and massive noise data |
| CN110415094A (en) * | 2019-06-18 | 2019-11-05 | 重庆金融资产交易所有限责任公司 | Asset-liabilities intelligent management, device and computer readable storage medium |
| CN110751170A (en) * | 2019-09-06 | 2020-02-04 | 武汉精立电子技术有限公司 | Panel quality detection method, system, terminal device and computer readable medium |
| CN112633310A (en) * | 2019-09-24 | 2021-04-09 | 博世有限公司 | Method and system for classifying sensor data with improved training robustness |
| CN110738264A (en) * | 2019-10-18 | 2020-01-31 | 上海眼控科技股份有限公司 | Abnormal sample screening, cleaning and training method, device, equipment and storage medium |
| CN110837926A (en) * | 2019-11-04 | 2020-02-25 | 四川省烟草公司广元市公司 | Tobacco main pest and disease damage prediction method based on big data |
| CN110837926B (en) * | 2019-11-04 | 2022-08-12 | 四川省烟草公司广元市公司 | Tobacco main pest and disease damage prediction method based on big data |
| CN110910356A (en) * | 2019-11-08 | 2020-03-24 | 北京华宇信息技术有限公司 | Method for generating image noise detection model, image noise detection method and device |
| CN111507419A (en) * | 2020-04-22 | 2020-08-07 | 腾讯科技（深圳）有限公司 | Training method and device of image classification model |
| WO2022073414A1 (en) * | 2020-10-09 | 2022-04-14 | 腾讯科技（深圳）有限公司 | Image recognition method and apparatus, computing device and computer-readable storage medium |
| CN112418327A (en) * | 2020-11-25 | 2021-02-26 | Oppo广东移动通信有限公司 | Training method and device of image classification model, electronic equipment and storage medium |
| CN112418327B (en) * | 2020-11-25 | 2024-08-13 | Oppo广东移动通信有限公司 | Training method and device for image classification model, electronic equipment and storage medium |
| CN112364993A (en) * | 2021-01-13 | 2021-02-12 | 深圳市友杰智新科技有限公司 | Model joint training method and device, computer equipment and storage medium |
| CN113206824A (en) * | 2021-03-23 | 2021-08-03 | 中国科学院信息工程研究所 | Dynamic network abnormal attack detection method and device, electronic equipment and storage medium |
| CN113284142A (en) * | 2021-07-16 | 2021-08-20 | 腾讯科技（深圳）有限公司 | Image detection method, image detection device, computer-readable storage medium and computer equipment |
| CN113673591A (en) * | 2021-08-13 | 2021-11-19 | 上海交通大学 | Image classification method, device and medium for self-adjusting sampling optimization |
| CN113673591B (en) * | 2021-08-13 | 2023-12-01 | 上海交通大学 | Self-adjusting sampling optimization image classification method, device and medium |
| CN114769072A (en) * | 2022-06-16 | 2022-07-22 | 深圳徕科技术有限公司 | High-speed injection valve control method and device, electronic equipment and storage medium |
| CN116047987A (en) * | 2023-03-31 | 2023-05-02 | 福建天甫电子材料有限公司 | Intelligent control system for producing electronic-grade buffer oxide etching solution |
| CN117523213A (en) * | 2024-01-04 | 2024-02-06 | 南京航空航天大学 | Noise tag identification method based on meta denoising and negative learning |
| CN117523213B (en) * | 2024-01-04 | 2024-03-29 | 南京航空航天大学 | Noise tag identification method based on meta denoising and negative learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107463953B (en) | 2019-11-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107463953B (en) | Image classification method and system based on quality insertion in the noisy situation of label | |
| CN107229904A (en) | A kind of object detection and recognition method based on deep learning | |
| CN103003846B (en) | Articulation region display device, joint area detecting device, joint area degree of membership calculation element, pass nodular region affiliation degree calculation element and joint area display packing | |
| CN108647226A (en) | A kind of mixing recommendation method based on variation autocoder | |
| Zhang et al. | Unifying generative models with GFlowNets and beyond | |
| CN111199216B (en) | Motion prediction method and system for human skeleton | |
| CN109543100B (en) | User interest modeling method and system based on collaborative learning | |
| Lei et al. | RETRACTED: Sports image detection based on particle swarm optimization algorithm | |
| CN114528490B (en) | Self-supervision sequence recommendation method based on long-term and short-term interests of user | |
| CN108171324A (en) | A kind of variation own coding mixed model | |
| CN113887501A (en) | Behavior recognition method and device, storage medium and electronic equipment | |
| Nepomuceno et al. | On the use of interval extensions to estimate the largest Lyapunov exponent from chaotic data | |
| CN102708294A (en) | Self-adaptive parameter soft measuring method on basis of semi-supervised local linear regression | |
| Zhang et al. | PointOT: Interpretable geometry-inspired point cloud generative model via optimal transport | |
| Xu et al. | A review of image inpainting methods based on deep learning | |
| CN108536844A (en) | A kind of network representation learning method of Text enhancement | |
| Wang et al. | Intercontrol: Generate human motion interactions by controlling every joint | |
| WO2024169276A1 (en) | Trajectory information processing method and apparatus, and computer device and readable storage medium | |
| CN117994011A (en) | E-commerce dynamic perception data recommendation method based on memory updating and neighbor transfer | |
| He et al. | Generative Structural Design Integrating BIM and Diffusion Model | |
| CN103839280A (en) | Method for tracking human body posture based on visual information | |
| Yi et al. | Differential evolutionary cuckoo-search-integrated tabu-adaptive pattern search (DECS-TAPS): a novel multihybrid variant of swarm intelligence and evolutionary algorithm in architectural design optimization and automation | |
| CN113158051A (en) | Label sorting method based on information propagation and multilayer context information modeling | |
| Li et al. | D‐Cloth: Skinning‐based Cloth Dynamic Prediction with a Three‐stage Network | |
| CN109902870A (en) | Electric grid investment prediction technique based on AdaBoost regression tree model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20181016 Address after: 200240 No. 800, Dongchuan Road, Shanghai, Minhang District Applicant after: Zhang Ya Applicant after: Wang Yanfeng Address before: 200240 No. 800, Dongchuan Road, Shanghai, Minhang District Applicant before: Shanghai Jiao Tong University |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20181121 Address after: Room 387, Building 333, Hongqiao Road, Xuhui District, Shanghai 200030 Applicant after: SHANGHAI MEDIA INTELLIGENCE Co.,Ltd. Address before: 200240 No. 800, Dongchuan Road, Shanghai, Minhang District Applicant before: Zhang Ya Applicant before: Wang Yanfeng |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Image Classification Method and System Based on Quality Embedding in Noisy Tags Effective date of registration: 20230329 Granted publication date: 20191119 Pledgee: The Bank of Shanghai branch Caohejing Limited by Share Ltd. Pledgor: SHANGHAI MEDIA INTELLIGENCE Co.,Ltd. Registration number: Y2023310000098 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |