CN109034186A - The method for establishing DA-RBM sorter model - Google Patents
The method for establishing DA-RBM sorter model Download PDFInfo
- Publication number
- CN109034186A CN109034186A CN201810595182.XA CN201810595182A CN109034186A CN 109034186 A CN109034186 A CN 109034186A CN 201810595182 A CN201810595182 A CN 201810595182A CN 109034186 A CN109034186 A CN 109034186A
- Authority
- CN
- China
- Prior art keywords
- rbm
- data
- model
- layer
- domain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to a kind of methods for establishing DA-RBM sorter model, and described method includes following steps: obtaining source domain data XsAnd source domain data XsCorresponding label Ys, target numeric field data XTAnd label YT；RBM model parameter is initialized, by data Xs、XTIt is input in RBM network, finds out single order feature；Using the single order feature as the input of lower first order network, RBM training is carried out；The hidden layer of RBM is exported into Hs、HTSoftmax recurrence layer is input to classify；The constraint of source domain data and aiming field data distribution is carried out using MMD in the output of RBM hidden layer；The constraint of prediction result is carried out using MMD in the top-level categories layer of RBM model；The total cost function J (θ) for constructing model, by optimizing the total cost function come the parameter of Optimum Classification device model.Model built of the present invention can effectively identify cross-domain data.
Description
Technical field
The present invention relates to deep learnings to identify field, in particular to a kind of side for establishing DA-RBM sorter model
Method.
Background technique
Character recognition information is generally divided into two kinds, one is the identification for text information, mainly country variant or not
With the identification of the printed matters such as national text, such as newspaper and periodical information, the information such as handwriting.Another kind is the knowledge of data information
Not, there is critically important application in digital information identification field, such as enterprise report data, bank return data, postcode data
Etc. a series of data.In this series of digital information, requires the very big manpower and material resources of consuming and supervision is gone to handle these
Data, especially financial circles increasingly developed today, the data volume of processing is also increasing, if only using artificial place to go
Reason, then will appear inefficiency, and error rate is bigger.If processing these information that can be automated are undoubtedly very convenient
, this can not only reduce out thick probability or save a large amount of time.If carrying out the identification of handwritten numeral, need
To classify to these data.Suitable recognizer is selected to have very great shadow for the discrimination for improving handwritten numeral
It rings.Friendship is beaten for financial field and other and data if it is possible to handle the technology of the identification of handwritten numeral font well
There is very big benefit in the field in road.And for current social, all work require intelligence, are known by handwritten numeral
Other technology can simplify the processes of work such as logging data, school inspection data and improve working efficiency.With mentioning for computer technology
The progress for rising especially machine learning is gradually taken seriously using the identification that machine learning algorithm carries out handwritten numeral,
This is that the identification handwritten numeral of automation brings Gospel.In machine learning field, deep learning algorithm is by more and more
Concern, therefore deep learning algorithm be applied to Handwritten Digit Recognition technology also become hot spot, and also take in this regard
The good achievement obtained, such as Google laboratory can will almost know when identifying handwritten numeral using convolutional neural networks
Not rate reaches 99% or more.This provides test basis for business use, and deep learning algorithm is applied to Handwritten Digit Recognition
Concern is placed in the method processing Handwritten Digit Recognition using deep learning by the technique extension direction of digital identification.
Traditional machine learning algorithm good implementation effect when handling Handwritten Digit Recognition in deep learning field, but
It is that the data that the requirements such as traditional deep learning algorithm such as RBM (limited Boltzmann machine) are handled belong to same distribution, also
It is training data and test data from same data set.And the identification of handwritten numeral font is from different numbers in real world
According to collection, that is, their distribution is different, therefore is going classification mixing numeric field data that can then go out using traditional deep learning algorithm
Existing unconformable situation.And traditional one reliable model of machine learning algorithm training needs a large amount of training sampling,
In real world, be sometimes difficult to get it is enough have a label can be used for trained data.Use the data for having label
Collection go training pattern then by this model be applied to one it is related to there is label data collection but be different on goal task
There is critically important application in the life of reality.
Summary of the invention
The present invention is not suitable in order to brought when overcoming the problems, such as training data and test data from different data collection, mentions
A kind of method for establishing DA-RBM sorter model is supplied, described method includes following steps:
S110, source domain data X is obtainedsAnd source domain data XsCorresponding label Ys, target numeric field data XTAnd label YT；
S120, initialization RBM model parameter, by data Xs、XTIt is input in RBM network, finds out single order feature；
S130, using the single order feature as the input of lower first order network, carry out RBM training；
S140, the hidden layer of RBM is exported into Hs、HTSoftmax recurrence layer is input to classify；
S150, the constraint for carrying out source domain data and aiming field data distribution using MMD in the output of RBM hidden layer；
S160, the constraint for carrying out prediction result using MMD in the top-level categories layer of RBM model；
S170, the total cost function J (θ) for constructing model, by optimizing the total cost function come Optimum Classification device model
Parameter.
Further, in the step S130, RBM training is carried out using Gibbs sampling and contrast divergence algorithm.
Further, the step S130 further includes being for source domain data and target numeric field data setting Hidden unit number
M, learning rate areMaximum cycle of training is T, and the parameter setting of RBM network is respectively that connection weight is set as W, and layer biasing is set
It is set to b, hidden layer biasing is set as c；RBM network is initialized, then the hidden node all for source domain data calculates its activation
Probability activates probability P (hs=1 | vs) its calculation formula is as follows:
P(hsj=1 | vs)=σ (c+ ∑iv1iwij)
It is sampled according to the conditional probability of hidden node using Gibbs, the output for acquiring hidden node is asked by following form
Solution:
hs~P (hs|v)
The hidden node all for target numeric field data calculates it and activates probability P (hT=1 | vT), calculation formula is as follows:
P(hTj=1 | vT)=σ (c+ ∑iv1iwij)
It is sampled according to the conditional probability of hidden node using Gibbs, the output for the aiming field hidden node asked passes through as follows
Mode solves: hT~P (hT|v)。
Further, the step S140 include: softmax return layer in classification solve calculation formula it is as follows:
Following formula is used for source domain data:
Following formula is used for aiming field:
Further, the last layer is classification layer in the DA-RBM model, and output is the probability for belonging to each class, institute
The output measurement source domain and aiming field that DA-RBM classifier is stated in feature extraction layer and classifier are lost based on the distribution of MMD；?
In feature distribution, the feature distribution difference on two domains is measured by the feature MMD in objective function:
Wherein H represents the output of the hidden layer of RBM；
MMD loss is added in the level of classifier, defining MMD loss is conditional MMD, the calculating of MMD loss
Formula is as follows:
Wherein C is the classification number of label, and q corresponds to the vector that certain a kind of all output is constituted.
Further, in the step S150, the formula of the total cost J (θ) is as follows:
Wherein L (θ) is the loss function of classifier.
Further, the source domain data are MNIST data set, and the target numeric field data is USPS data set.
Technical solution through the foregoing embodiment, the present invention establish a kind of model that can handle cross-domain hand-written data,
Recognition effect is preferable.
Detailed description of the invention
The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, and attached drawing is schematically without that should manage
Solution is carries out any restrictions to the present invention, in the accompanying drawings:
Fig. 1 is the RBM model schematic schematic diagram in some embodiments of the invention；
Fig. 2 is the domain adaptive learning model schematic in some embodiments of the invention；
Fig. 3 is the DA-RBM sorter model schematic diagram in some embodiments of the invention；
Fig. 4 is the classification results schematic diagram in some embodiments of the invention；
Fig. 5 is the USPS classification results schematic diagram in some embodiments of the invention；
Fig. 6 is the classification results schematic diagram in some embodiments of the invention；
Fig. 7 is the result schematic diagram that the DA-RBM in some embodiments of the invention classifies to cross-domain data；
Fig. 8 is the method flow diagram for establishing DA-RBM sorter model in some embodiments of the invention.
Specific embodiment
To better understand the objects, features and advantages of the present invention, with reference to the accompanying drawing and specific real
Applying mode, the present invention is further described in detail.It should be noted that in the absence of conflict, the implementation of the application
Feature in example and embodiment can be combined with each other.
In the following description, numerous specific details are set forth in order to facilitate a full understanding of the present invention, still, the present invention may be used also
To be implemented using other than the one described here other modes, therefore, protection scope of the present invention is not by described below
Specific embodiment limitation.
In the field of the depth of investigation study, use MNIST database as the experimental data of various algorithms, and obtain
Good experiment effect, traditional deep learning field achieve good effect in the data processing for solving this single domain
Fruit, but deep learning still exists in the data processing for solving the problems, such as some cross-domain aspects.Limited Boltzmann machine
The application of (Restricted Boltzmann Machine, RBM) in pattern-recognition and recurrence has shown that it is a kind of
Practical and efficient technology.However, the unsupervised learning of RBM concern, semi-supervised learning, supervised learning is all in single domain
(namely source domain (source data)).It is studied not yet for the learning ability of the limited Boltzmann machine of hybrid domain, be
The learning ability of RBM is promoted, RBM and domain are adapted to (Domain Adaptation, DA) and extension RBM combined to learn by the present invention
Ability.The invention proposes a unified frame, referred to as domain adapts to limited Boltzmann machine (DA-RBM), passes through study
Source domain data and aiming field (target data) data construct a healthy and strong classifier.The present invention passes through MNIST is this
Handwritten numeral data regard source domain data as, regard USPS handwritten numeral data as target numeric field data, go to learn by DA-RBM frame
A composite character library is practised, the data for identifying other aiming fields are then gone.It is returned in the classification layer present invention using softmax
(Softmax Regression, SR) classifies, while mixing the classifier of numeric field data to be formed to classify, using most
Big mean difference (Maximum Mean Discrepancy, MMD) algorithm is constrained.Concrete mode is come by using MMD
The feature distribution extracted on two domains (source domain and aiming field) is constrained, to make the feature distribution on two domains about
It is as identical as possible in the case where beam, while the SR classification results on two domains are constrained with MMD, so that two classification results
Distribution it is as identical as possible.Main contributions of the invention are by combining DA algorithm and RBM algorithm and SR sorting algorithm shape
At a kind of method suitable for solving hybrid domain data classification.
RBM is an a kind of probability distribution for learning about input data, and supervised learning or nothing can be used
The mode of supervised learning is trained, and is widely used in classification using the diversified neural network that RBM learns to be formed, is built
In the diversified practical problem such as mould, recurrence.
RBM belongs to directionless probability graph model, and undirected probability graph model is also referred to as Markov random field model, this mould
Type is made of a series of stochastic variable, and the relationship between stochastic variable can be indicated by non-directed graph, while also to be expired
The characteristic of sufficient markov random file, that is, following the sum of the state in a random change procedure conditional probability distribution
Current state is related and other states are not related.Markovian Markov characteristics exhibit is when a given node
Other neighbours several points after, keep conditional sampling between each node, that is, present node is connected directly only and with it
Node be related, it is unrelated with other unconnected nodes.A series of mistakes that reasoning for undirected probability graph model solves
Journey is related to the correlation techniques such as energy function, probability density and parameter Estimation.
Energy function is for describing estimating for a system mode, when its system mode is more being ordered into or more probability distribution
It concentrates, the energy of system will be smaller；Its system mode is more unordered or probability distribution more tends to be uniformly distributed form, is
The energy of system will be bigger.Since undirected probability graph model belongs to one kind of random network, comparison and dynamics side and statistics
Energy function in mechanics and introduce, the state of the smaller corresponding system of energy function is also more stable.For undirected probability graph model
Global variable can be provided for whole network by defining energy model, provide objective function for the study of model, and can
With the principle of optimality of Definition Model, even if the numerical value of energy function tends to be minimum, model entire in this way is namely the most stable
When, the parameter of model is also just the parameter optimized at this time, this is provided to solve probability-distribution function represented by model
A possibility that can solving.Therefore, non-directed graph probabilistic model design energy function is facilitated from the angle of mathematics up
Solution is provided.For RBM therefore a kind of its Markov random field model for also belonging to special shape is being instructed
Energy function model is also defined when practicing model.
RBM generally forms the connection weight between its aobvious layer and hidden layer by aobvious layer v and hidden layer h double-layer structure and is indicated with w, and
And indicate that the biasing of hidden layer is indicated with c with b in the biasing of aobvious layer.RBM model schematic is as shown in Figure 1.
In limited Boltzmann machine model since there is no aobvious layer unit and aobvious layer unit connection and hidden layer with it is hidden
The connection of layer unit, then for the E of the energy of RBM (v, h | θ) function is defined as:
Wherein θ={ wij,bi,cjCan pass through during training pattern Optimal Parameters for the parameter of RBM model
The joint probability distribution for being dependent on (v, h) of the energy function is acquired, wherein the following institute of the joint probability distribution function of (v, h)
Show:
Wherein Z (θ) is normalization factor.
By understanding the joint probability distribution of above-mentioned aobvious hidden layer, the present invention can for example be summed by mathematical method or product
Point etc. forms obtain the marginal probability distribution or their conditional probability distribution of aobvious layer or hidden layer, show the marginal probability of layer v
The condition relative to aobvious layer v of conditional probability distribution and hidden layer h of distribution, the marginal probability of hidden layer h, aobvious layer v relative to hidden layer h
Probability distribution is as follows:
Gibbs sampling is one kind of Markov chain Monte-Carlo sampling algorithm, is mainly used to construct multivariate probability point
The random sample of cloth, such as construct the joint probability distribution etc. of two or more variables.For RBM model due to
The joint probability distribution that the presence of normalization factor shows between layer and hidden layer is difficult to acquire by calculating, it is therefore desirable to pass through one kind
Approximate mode is solved, and Gibbs sampling is just possessing this ability, when integral, expectation and joint probability distribution cannot
If calculating, it can be sampled by Gibbs and acquire approximate solution.The basic principle of Gibbs sampling is: it is assumed that one has D
The random vector sample X=(X of dimension1,...,XD), there is great difficulty for the calculating of its joint probability, that is, pass through routine
The calculation of joint probability is asked to be difficult to solve.According to existing knowledge present invention assumes that knowing a certain component in sample vector
For the conditional probability of other sample components, Ke YiyongIt indicates, whereinSo
The present invention can use above-mentioned condition probability, and iteration samples any vector in sample vector, when of the invention changes
When the number in generation is enough, the probability distribution of heap stochastic variable will converge on the joint probability distribution P (X) of X.It is adopted by Gibbs
Sample invention can be sampled in the case where not knowing the Joint Distribution of sample X.This ability of Gibbs sampling just may be used
It can not be sampled with solving the problems, such as to show in RBM model in the case that joint probability is not known between layer and hidden layer.
RBM model has the characteristics that between symmetrical in structure and its aobvious layer and hidden layer relative to other side to be all condition
It is independent, therefore the present invention can sample to obtain by Gibbs and meet the random sample that RBM model defines distribution.Given RBM
Model, the specific algorithm for carrying out k step sampling operation using Gibbs sampling method are as follows: (or aobvious using training sample
Any stochastic regime of layer) initialize the state v for showing layer0, alternately such as down-sampling:
h0~p (h | v0), v1~p (v | h0)
h1~p (h | v1), v2~p (v | h1)
..., hk+1~p (v | hk+1)
It is available to meet sample distribution required for RBM learning model in the case where sampling sufficiently large, while can also
To solve the problems, such as to show Joint Distribution between layer and hidden layer about RBM can not be solved in above-mentioned maximum likelihood function.
It although can solve the problem of joint probability distribution can not solve using Gibbs sampling algorithm, but for reality
Calculating in especially when data of the invention be more high-dimensional data and training sample size it is bigger when, use
The sampling that Gibbs sampling algorithm carries out data sample will become extremely difficult.It is pungent in order to improve the training effectiveness of RBM model
Pausing, it is a kind of for solving the lower method of RBM model training efficiency, that is, contrast divergence algorithm to propose in 2002.With
Gibbs sampling algorithm is not both, for contrast divergence algorithm, when the present invention initializes v using sample data0When, only
As long as a step or a few step Gibbs sampling can obtain a relatively good approximation.
In contrast divergence algorithm, the present invention is given to show layer using sample data deinitialization, then goes to calculate hidden layer again
State, in RBM model, due to being conditional sampling between each hidden layer state, therefore under the premise of given aobvious layer state
The probability for solving the hidden node activation of jth at this time is as follows:
Similarly in the case that all hidden layer states all determine, the probability of each aobvious layer state activation is all conditional sampling
, thus for solved in the case where determining hidden layer show for i-th of aobvious layer node layer activation probability it is as follows:
By above-mentioned calculating, the present invention can regard the state of aobvious node layer as the reconstruct of aobvious layer data, because
The update rule of this each parameter in RBM model in the above cases is as follows:
Δwij=η (< vihj>data-<vihj>recon),
Δbi=η (< vi>data-<vi>recon),
Δcj=(< hj>data-<hj>recon)
It is sampled in contrast to traditional Gibbs, the present invention is come using the reconstructed distribution of sample data in contrast divergence algorithm
Instead of approximate model profile in original Gibbs sampling, RBM model efficiency is not only optimized, but also make full use of sample
The characteristic information of data often obtains good effect in actual training.
For the RBM model of sample given for one, that is, suitable parameter is found in order to remove to be fitted trained sample
This, that is, in the case where determining this parameter, the probability distribution of the generation of RBM model meets data sample as far as possible.It is logical
Maximal possibility estimation is crossed present invention may determine that parameter, the core of maximal possibility estimation is to form model by study, so as to
The maximum probability for allowing training sample to be observed in the model of study.The training namely made for undirected probability graph model
Obtained joint probability distribution can meet with the distribution of sample data as far as possible.Current invention assumes that T is the total of training sample
Amount, and between each training sample be all it is independent identically distributed, the key of training RBM is exactly to maximize following likelihood function:
Since being to solve for maximum likelihood, seek to solve maximum value to likelihood function, the process solved is namely to ginseng
Number derivation, then constantly promotes objective function using gradient rise method, to reach final stop condition.The present invention is first
Need to solve the logarithm of its maximum likelihood function, then again to its derivation, calculation formula is as follows:
Due to still having the presence of normalization factor in the above-mentioned formula for asking it, can't directly be asked
Solution is solved just need Gibbs use proposed above and contrast divergence algorithm herein.
As the other neural networks of training, for RBM prototype network, also there are many parameter needs to go to carry out
Setting, such as the number of hidden nodes, learning rate and initialization of parameter etc..The setting of these parameters what a stablizes for training
RBM model it is most important.It is usually the increase hidden layer being arranged according to the feature of data simply for the number of hidden nodes
Number of nodes is not very good, and it is the 10 of weight that general setting, which is the amount of the update of weight, for learning rate-3Left and right, for
The initial setting up of biasing etc. is usually using the random value for being just distributed very much generation.
Domain adaptive learning also should comply with the core of transfer learning as a kind of its algorithm core naturally of transfer learning, right
Say that invention is frequently utilized that the experience of forefathers goes to solve existing relevant issues from psychologic for transfer learning,
This behavior is exactly a kind of one kind of transfer learning in fact, the model for learning other field from the angle present invention of machine learning
Knowledge is applied to correlation but is different in the learning model of other field.Therefore for the definition sheet of the concept of domain adaptive learning
Invention can do description below: give a source domain data DsWith source domain learning tasks Ts, target numeric field data DTLearn with aiming field
Task TT, utilize DsAnd TsStudy anticipation function f (), constraint condition therein on aiming field are source domain and target domain characterization
Space is identical, while the classification space of source domain and aiming field is also identical, but source domain and the data distribution of aiming field are
Different.
The purpose that domain adapts to is to solve a problem concerning study in aiming field to pass through using the training data in source domain,
And different data distributions is possible on source domain and aiming field.The data for being typically due to label are to be difficult on aiming field
It obtains, the knowledge by learning source domain data is gone to solve the problems, such as on aiming field for solving the problems, such as the of crucial importance of this change.
Domain adaptive learning is typically considered a kind of study of specific transfer, transfer learning be usually exactly migrate shared knowledge difference still
Between relevant domain.The main problem of its concern is how to go to reduce the distribution before of source domain and aiming field for domain adapts to
Difference, trained in hybrid domain character representation be it is particularly important, a good character representation should be reduced as far as possible
The difference of source domain and aiming field data distribution, therefore its final judgment criteria is sought in character representation for domain adapts to
On make the data distribution of source domain and aiming field similar as far as possible.
Domain adapt in the domain present invention consider that it contains two layers of meaning, one be input data X feature space another
It is the probability distribution p (x) of input data, wherein X={ x1..., xnIt is a series of learning samples.Usually in domain adaptive learning
In, the present invention usually assumes that they usually may be by different feature spaces for different domains there are two not same area
Or there are different data probability distributions.Source domain and the data field feature having the same that the present invention is paid close attention in this film paper are empty
Between, their difference is that the probability distribution of data is different.Present invention assumes that the data containing label in the adaptive model of domain
DSFor source domain data, present invention assumes that the data D without labelTFor target numeric field data.Wherein source domain data DS=((xs1,
ys1),...,(xsn1,ysn1)) similar for target numeric field dataThe present invention uses P (XS) and
Indicate the data distribution from source domain and aiming field, wherein P andIt is different.This hair for domain of the invention adapts to
Bright task is to predict the label yT of the data of aiming fieldi.For most number field adaptive learning the present invention usually do as
Lower hypothesis:
P(YS|XS)=P (YT|XT)
For domain adapts to, the present invention is also by the way that source domain training knowledge to be applied on aiming field, and domain adapts to learn
It is as shown in Figure 2 to practise model.
Knowledge migration by learning in source domain is into aiming field for domain adapts to, and can allow machine learning model
More added with extending space.For the present invention, the RBM learning model that the present invention can be allowed to use is more healthy and strong, can go
Data suitable for not same area.
Domain adaptive learning is strictly to have very big performance space, therefore how to go to realize domain on solving cross-domain problem concerning study
Adaptive learning just becomes particularly important, and again includes other different fields for domain adapts to, such as ask about multiple domain
The study of topic and the study of single domain problem.The domain adaptive learning of supervised or unsupervised formula, various domains adapt to classification,
Inevitable solution that just should be different.For the adaptive learning task of domain, the present invention can also be with model layer
It goes from feature space or from every aspects such as relationships to solve domain adjustment.Therefore, for the adaptive learning of domain, one is selected
A suitable learning model is for going to solve the most important of cross-domain problem concerning study change.In the adaptive learning of domain, present invention concern
Data set it is multifarious, that is, for its knowledge that may be migrated be also it is different, selected according to data characteristics
The extended capability of training pattern can be improved in different domain adaptive learning methods for specific data.Solve what domain adapted to
Learning method emphasis should be paid close attention in its source domain and the data distribution similarity of aiming field, if the model construction of study is shared
Knowledge base can be compatible with the knowledge of source domain and aiming field, this will provide very big side for solving the problem concerning study of cross-domain data
It helps.
If source domain data and target numeric field data are if there is many sharing features in the adaptive learning of domain, the present invention is at this time
It can go to seek solution by migration feature.Therefore when the present invention may determine that source domain data and target numeric field data exist
Having many similarities in feature is that the present invention can be written over source domain data first, is filtered out similar to target numeric field data
It spends high data and then is trained study.This targetedly mode of learning for solution possess many common traits across
Problem tool in domain has an enormous advantage.Therefore, the migration pattern that Case-based Reasoning can be used in the present invention goes to solve this domain adaptation
Problem concerning study.If source domain data and target numeric field data have some public cross features, the present invention can pass through transformation at this time
Then source domain data and target numeric field data are transformed to same feature space again by traditional machine learning algorithm by feature space
Learnt.Analysis of the main mode that uses of the present invention based on migration ingredient for the present invention, that is, by source domain
And target numeric field data transforms to identical space and then removes constraint learning model between them in a manner of distance to minimize.This hair
Bright usually used the way of restraint is maximum mean difference (Maximum mean discrepancy, MMD).Therefore next
Some knowledge of the article present invention by introduction about MMD algorithm.
Maximum mean difference, which mentions, to be made to solve double sample and solves the problems, such as, is also just used to solve asking for two data distributions
Topic, in domain adaptive learning they can be used to the problem of judging source domain data distribution and aiming field data distribution.Maximum mean difference
The different distributional difference that data are mainly measured by the difference of population mean.Therefore, one is looked for for the adaptive learning of domain
A kind of method of situation in a public data distribution space is exactly to pass through maximum mean difference to go to constrain, and passes through the constraint sheet of MMD
The problem of invention can solve the common data distribution space that domain is met the needs of.The present invention has different for MMD algorithm
The solution present invention can go to understand from arbitrarily spatially MMD, and the present invention can also be by reproducing kernel Hilbert space
MMD go to understand.Its basic assumption is the generating function f of all sample spaces for maximum mean difference, if sample
It is same point that notebook data, which is considered as the two data distributions situations such as the corresponding picture of f function above has enough mean value phases,
Cloth.
Mainly pass through regeneration Hilbert when the present invention carries out the constraint of domain adaptive learning data characteristics distribution using MMD
Space MMD is constrained, therefore, in next introduction the present invention will emphasis introduction regenerate Hilbert space MMD
Definition and basic concepts.The present invention assumes initially that the data set there are two different distributions, and one of data meet P
It, is defined as source domain data X by distribution(s)=[x(s)1,...,x(s)ns], another data meet Q distribution, and the present invention is it
It is defined as target numeric field data X(t)=[x(t)1,...,x(t)nt], the present invention indicates regeneration Hilbert space using H, in fact again
Raw Hilbert space is obtained by mapping function, and mapping function is indicated that its effect is by original number with φ ()
According to being mapped on feature space.When the primitive character of data is mapped to feature space, following formula is can be used in the present invention
It is indicated maximum mean difference:
The present invention indicates source domain data using the difference of the population mean between source domain and aiming field in the adaptive learning of domain
Difference between distribution and aiming field data distribution.It therefore, can be in feature space layer using MMD algorithm in the adaptive learning of domain
The data distribution between two domains of constraint is gone in face.This restriction behavior of the MMD algorithm on feature space can be conventional machines
Study gets involved in and provides opportunity in the adaptive learning of domain, and the mapping mode of conventional machines study can be used by original number in the present invention
According to the feature space for being mapped on feature space and then reusing MMD constrained procedure and go one same distribution of training.
Logistic regression be a kind of efficient classification it not only can be only used for the prediction of data label and can also be counted
It calculates various labels and the size of possibility occurs.To do is to the identifications of handwritten numeral by the present invention in the present invention, know because to do
Suitable sorting algorithm is not selected to have a very big impact the efficiency of identification.This hair in kernel model proposed by the present invention
The bright top layer addition classification layer needed in deep neural network, present invention selection softmax returns classifier and makees in this model
For sorter model of the invention, softmax returns one kind that classifier is logistic regression in fact, since logistic regression can be into
Capable classification must be two classification problems, but handwritten numeral Character Font Recognition of the invention is more classification problems, therefore use biography
The Logic Regression Models of system go classify just to become extremely improper.Therefore, the mutation for choosing logistic regression at this time goes to solve
More classification problems just become most important.
Present invention assumes that there is a c classification problem in softmax recurrence, tag along sort therein is yi∈
{ 1 ..., c }, for softmax return in how to go to carry out the identification of more classification problems, that is, by two points of logistic regression
How class problem is extended to more classification problems, i.e., the present invention first selects any type to regard another as remaining again as one kind first
One kind constructs a binary classifier, can thus construct c classifier, and in this way, the present invention can be by logic
Two classification problems returned are extended to more classification problems.This c classifier present invention can be defined with such as under type:
...,
It can easily be returned using softmax by this division class method for distinguishing and carry out more classification problems, therefore
The present invention will use softmax model to go to solve classification problem in core of the invention model.
DA-RBM sorter model will form public feature database into aiming field by migrating source domain data knowledge, more
The situation of the knowledge deficiency occurred when complementary modulus type learning objective numeric field data, by way of MMD come respectively feature space and point
It is constrained in class result.DA-RBM disaggregated model extends the learning ability of RBM, in order to which RBM goes to solve Cross-domain problem, especially
It is, by migrating previous correlation but being different data set knowledge, to be reduced in mesh in the case that aiming field marker samples are few
Mark the cost of field mark.DA-RBM sorter model is proposing that the advantages that feature expressive force is fast with speed by force is counted by RBM
According to the proposition of feature, then using domain adaptive method carry out feature space constraint so that source domain data characteristics spatial distribution and
Aiming field data characteristics spatial distribution is as similar as possible, while will will use classifier algorithm in the top layer present invention and classify,
The probability distribution present invention of classification results is equally constrained using domain adaptive method so that source domain data classification result
Distribution and aiming field data classification distribution of results are as similar as possible, and the ginseng of entire model is then updated by above-mentioned constraint
Number allows DA-RBM sorter model to obtain the ability of processing cross-domain data.
The tradition study in Handwritten Digit Recognition field is that the present invention divides instruction on the same data set namely in source domain
Practice set test set, training pattern is gone by training set, then will train the model come again and be applied in test data set, lead to
Reason condition lowers whole Machine Learning Parameter, can be very good identification handwritten numeral.It is this to be trained and test by same domain
Behavior be conventional machines study premise, also require that training dataset and test data set from same for RBM
Domain will appear the undesirable situation of modeling effect if the case where they are different domain.RBM handles Handwritten Digit Recognition
Conventional rule is to extract feature first to hidden layer, is then returned on hidden layer top plus classification layer such as softmax.But
RBM only only solves the learning ability in single domain, for solving the problems, such as cross-domain to seem it is not the roadmap for meeting very much him.
Therefore, for solving cross-domain Handwritten Digit Recognition RBM, theoretically there is also limitations.This limitation of RBM be usually by
Occur not applicable situation on being applied to aiming field in the property data base extracted using source domain data, therefore how to utilize
The mode that the present invention learns to design a kind of building public characteristic library to a large amount of model knowledge in the past goes the processing for solving RBM cross-domain
The undesirable situation of Handwritten Digit Recognition is most important.
Require training data and test data from same data when solving Handwritten Digit Recognition for traditional RBM algorithm
The case where distribution, removes the identification for handling cross-domain handwritten numeral, this algorithm the invention proposes a kind of DA-RBM algorithm model
It is intended to excavate the data that information useful in source domain removes identification aiming field.Although the data distribution of aiming field and the data point of source domain
Cloth is different, but still can will be in the study of the knowledge migration of source domain to aiming field.The identification of handwritten numeral is carried out by RBM
Key be to construct suitable network structure, therefore should also establish suitable study in DA-RBM model of the invention
Model.Main thinking of the invention is exactly to carry out the constraint of the data distribution of source domain and aiming field in feature extraction layer so as to two
The data distribution of person is as similar as possible, while constrain classification results in classification layer, is constructed by constraining twice
A kind of public feature model library realizes domain adaptive learning.
The main task of the present invention is to carry out Handwritten Digit Recognition, therefore the present invention constructs DA-RBM and carries out hand-written script
Identification, the present invention using RBM carry out feature extraction, then feature top addition multi-categorizer softmax recurrence divided
Class constrains the data probability distributions of source domain and aiming field in the feature present invention by MMD algorithm, uses in the result of classification layer
MMD algorithm carries out result constraint, and the cost of a totality is constructed by the cost function of this two layers constraint and neural network itself
Then function goes the parameter of optimization whole network in turn by optimizing total cost function, come so that whole network trains
Model can be used to classify cross-domain data.DA-RBM sorter model schematic diagram is as shown in Figure 3.
Above-mentioned model has given the scheme of the processing of the invention handled from the process of data processing, and the following present invention will be from
The angle of mathematics goes to handle this process.Assuming that the present invention provides source domain data DsAnd label value YsWith target numeric field data DTWith it is few
The label Y of amountT, it is m for source domain data and target numeric field data setting Hidden unit number, learning rate isMaximum cycle of training is
The parameter setting of T, RBM network is respectively that connection weight is set as W, can bias layer and be set as b, at the beginning of hidden layer biasing is set as c.
Beginningization random network, then the hidden node all for source domain data calculates it and activates probability, calculates P (hs=1 | vs) its meter
It is as follows to calculate formula:
P(hsj=1 | vs)=σ (c+ ∑iv1iwij), wherein i, j indicate the label of node, wijIndicate weight, v indicates aobvious
Layer；
It is sampled according to the conditional probability of hidden node using Gibbs, the output for acquiring hidden node can be by following shape
Formula solves:
hs~P (hs|v)
The hidden node all for target numeric field data calculates it and activates probability, calculates P (hT=1 | vT) its calculation formula is such as
Under:
P(hTj=1 | vT)=σ (c+ ∑iv1iwij)
It is sampled according to the conditional probability of hidden node using Gibbs, asks the output of aiming field hidden node can be by such as
Under type solves:
hT~P (hT|v)
Above-mentioned formula is the solution in feature level, because the domain adaptive model that the present invention constructs also is needed in classification results
Therefore upper progress solves its calculation formula for the solution present invention of classification results by way of solution during softmax is returned
It is as follows:
For source domain data:
The result formula of its solution classified for aiming field are as follows:
The solution formula of above-mentioned classification layer is abbreviated as by the present invention in DA-RBM modelWithIt has been provided substantially in the mode for the calculating data for solving entire model in this way, the following present invention needs structure
Build in constraint condition and then provide the cost function of entire sorter model.
The present invention carries out feature extraction using RBM in DA-RBM model, is classification layer in the last layer present invention addition,
Output is the probability for belonging to each class, this classifier measures source domain and aiming field in the output of feature extraction layer and classifier
Distribution loss based on MMD.In feature distribution, the spy on two domains is measured by the feature MMD in objective function
Sign distribution difference:
Wherein H represents the output of the hidden layer of RBM, M expression parameter matrix, wherein nsIndicate source domain node total number, nTIndicate mesh
Domain node sum is marked, i, j indicate that nodal scheme, Tr () indicate mark, hi, hjIndicate node output.
MMD is added in the level present invention of classifier to lose so that two domains are identical as far as possible in condition distribution, calmly
Its conditional MMD of justice:
Wherein C is the classification number of label, qcThe vector that certain corresponding a kind of all output are constituted, M expression parameter matrix.
Finally the objective function of whole network is just constituted plus the Classification Loss in network:
Wherein L (θ) is the loss function of classifier, and λ and μ indicate proportionality coefficient.
Explanation has been carried out in the basic mathematical logic of DA-RBM disaggregated model, and the following present invention will be from algorithm flow
Angle be introduced.DA-RBM algorithm is summarized as follows:
Input: source data and label Xs,YsWith target data XTAnd a small amount of label YT
Output: the prediction label yT of the aiming field of the parameter w, b, c and prediction of RBM
Start:
Pass through RBM initialization feature space Hs,HT；
1. RBM model parameter is initialized, by initial data Xs,XTIt is input to the single order character representation asked in RBM network；
2. carrying out RBM training then using single order feature as the input of lower first order network with this；
3. the hidden layer of RBM is exported Hs,HTSoftmax recurrence layer is input to classify；
4. carrying out source domain data and aiming field data distribution using maximum mean difference (MMD) in the output of RBM hidden layer
Constraint；
5. using the constraint of MMD progress prediction result in the top-level categories layer of model；
6. constructing total cost function J (θ) of model, then pass through the ginseng of optimization cost function Optimum Classification device model
Number；
Terminate.
DA-RBM is used to solve the problems, such as the identification of the hand-written script of Cross-domain problem, therefore when testing used in the present invention
Data set should also be cross-domain data set, therefore the present invention has chosen MNIST handwritten numeral data acquisition system USPS handwritten numeral
Data set is tested, wherein it is of the invention using MNIST data set as source domain data, using USPS hand-written script data set as mesh
Mark numeric field data collection.DA-RBM model be exactly by MNIST study to knowledge migration to USPS on go identification USPS data set.
MNIST handwritten numeral data set learns common handwritten numeral data set as conventional machines and has had been used for many experiments
The present invention selects MNIST data set as source domain data when suffering, therefore testing the fact doing the identification of cross-domain hand-written data, for
The USPS handwritten numeral data set present invention being of little use chooses it as target numeric field data.Above-mentioned data set is applied to by the present invention
The single domain sorting algorithm identification of RBM adapts to convolutional neural networks Classification and Identification applied to domain and is applied to above-mentioned data set
On DA-RBM disaggregated model.
The MNIST hand-written script data set used is referred to as source domain data by the present invention, and wherein MNIST data set possesses
60000 training sample data set, 10000 test sample data compositions.These digital pictures have been typically canonicalized, and will be counted
Word is placed in picture centre, and makes the in the same size of image.Image size in MNIST data set of the invention be 16*16 also
Be its characteristic dimension be 256.USPS data set is the United States Post Office handwritten numeral collection present invention as in DA-RBM model
Aiming field data set wherein contains 7291 training sample data in USPS data set, contain 2007 test sample data.This hair
Image size in bright USPS data set is 16*16, that is, its characteristic dimension is 256.The present invention extracts a part of data
In source domain and aiming field, experiment is done.
The test present invention of DA-RBM will be considered first from single domain, that is, by the MNIST data in single domain and
USPS data application is classified in RBM.When being classified using RBM, training data and test data are all chosen from same domain
The case where data set, the present invention should be tested the RBM done on MNIST data set first, the present invention choose hidden node
It is 200, choosing learning rate is that 0.01 its classification results are as shown in Figure 4.
If can see training data from the above-mentioned experimental result present invention and test data both be from MNIST data set
When its discrimination be 96%
The following present invention will use RBM to go identification USPS handwritten numeral data set namely in test data and training number
According to USPS data set both is from, the case where RBM sorter model of the invention constructed, is at this time, and Hidden unit number is 200
Habit rate is 0.01.The result run when the USPS data application of above-mentioned selection to above-mentioned RBM is classified is as shown in Figure 5.
The discrimination that can be seen that RBM classifier identification USPS data set from the above-mentioned experimental result present invention is 96%.
Next the identification that the present invention will use RBM sorter model to remove processing cross-domain data, that is, using MNIST as source domain number
According to using USPS as target numeric field data, the present invention sets Hidden unit number and is set as 200 learning rates in sorter model
0.01, by above-mentioned data application, to cross-domain Handwritten Digit Recognition, its result is as shown in Figure 6.
Can be seen that from the above-mentioned experimental result present invention goes classification cross-domain data as a result, discrimination is using RBM
24%, therefore, for for cross-domain identification handwritten numeral this discrimination be low-down.
Next the present invention will use DA-RBM to go to handle the identification problem of handwritten numeral, wherein of the invention use will
As source domain data using USPS data as target numeric field data, the number of hidden nodes that network model is arranged in the present invention is MNIST
200, setting learning rate is 0.01, above-mentioned data is carried out its processing of Classification and Identification using DA-RBM, as a result as shown in Figure 7.
It can be seen that down to identify handwritten numeral in cross-domain situation using DA-RBM by the above-mentioned experimental data present invention
Discrimination promising 92%.With from the point of view of the identification of the RBM algorithm of no application domain adaptive learning algorithm to compare hand-written script this
The recognition capability of the DA-RBM algorithm of invention has promotion.
The next experiment present invention will compare the performance of DA-RBM by comparison algorithm, adapt to convolutional Neural using domain
Network goes to execute cross-domain handwritten numeral Classification and Identification.Because convolutional neural networks are advantageous on processing image, and handwritten word
Volume data is usually to be stored in the form of picture, therefore using convolutional neural networks go to be tested more representative.Below
Be various experiments correlation data it is as shown in table 1
1 Experimental comparison results' table of table
Present invention CNN algorithm first removes the hand of processing hybrid domain under the premise of no domain adaptability in comparative experiments
Digital identification is write, then in the identification for the handwritten numeral for going processing hybrid domain using DA-CNN algorithm, the RBM from the point of view of comparing result
Its discrimination improves 68% in cross-domain identification after application domain adaptive learning, while its learning ability and DA-CNN algorithm
Learning ability is essentially identical, since CNN has advantage in processing image data.Therefore, from the point of view of above-mentioned experimental result, this
The learning ability of the DA-RBM of invention, which has, to be obviously improved.
Present invention employs MNIST handwritten numeral data set and USPS handwritten numeral data set, then tests and used RBM
MNIST data set and USPS data set are identified respectively, while going to handle cross-domain handwritten numeral data set using DA-RBM model
Identification, the identification for handling cross-domain handwritten numeral data set is then gone using DA-CNN, can be obtained according to the experimental result present invention
DA-RBM model can promote the learning ability of RBM on handling cross-domain Handwritten Digit Recognition out, compare the experiment of DA-CNN
As a result, DA-RBM is identical with DA-CNN on promoting discrimination.Therefore, DA- can consider according to the experiments experiment result present invention
RBM model is effective on handling cross-domain Handwritten Digit Recognition.
Method of the invention can be summarized as follows: as shown in figure 8, establishing DA-RBM sorter model the present invention provides a kind of
Method, described method includes following steps:
S110, source domain data X is obtainedsAnd source domain data XsCorresponding label Ys, target numeric field data XTAnd label YT；
S120, initialization RBM model parameter, by data Xs、XTIt is input in RBM network, finds out single order feature；
S130, using the single order feature as the input of lower first order network, carry out RBM training；
S140, the hidden layer of RBM is exported into Hs、HTSoftmax recurrence layer is input to classify；
S150, the constraint for carrying out source domain data and aiming field data distribution using MMD in the output of RBM hidden layer；
S160, the constraint for carrying out prediction result using MMD in the top-level categories layer of RBM model；
S170, the total cost function J (θ) for constructing model, by optimizing the total cost function come Optimum Classification device model
Parameter.
In the step S130, RBM training is carried out using Gibbs sampling and contrast divergence algorithm.
The step S130 further includes, and is m for source domain data and target numeric field data setting Hidden unit number, learning rate isMaximum cycle of training is T, and the parameter setting of RBM network is respectively that connection weight is set as W, sets b, hidden layer for layer biasing
Biasing is set as c；RBM network is initialized, then the hidden node all for source domain data calculates it and activate probability, and activation is general
Rate P (hs=1 | vs) its calculation formula is as follows:
P(hsj=1 | vs)=σ (c+ ∑iv1iwij)
It is sampled according to the conditional probability of hidden node using Gibbs, the output for acquiring hidden node is asked by following form
Solution:
hs~P (hs|v)
The hidden node all for target numeric field data calculates it and activates probability P (hT=1 | vT), calculation formula is as follows:
P(hTj=1 | vT)=σ (c+ ∑iv1iwij)
It is sampled according to the conditional probability of hidden node using Gibbs, the output for the aiming field hidden node asked passes through as follows
Mode solves: hT~P (hT|v)。
The step S140 include: softmax return layer in classification solve calculation formula it is as follows:
Following formula is used for source domain data:
Following formula is used for aiming field:
The last layer is classification layer in the DA-RBM model, and output is the probability for belonging to each class, the DA-RBM
Classifier is measured source domain and aiming field in the output of feature extraction layer and classifier and is lost based on the distribution of MMD；In feature distribution
On, the feature distribution difference on two domains is measured by the feature MMD in objective function:
Wherein H represents the output of the hidden layer of RBM；
MMD loss is added in the level of classifier, defining MMD loss is conditional MMD, the calculating of MMD loss
Formula is as follows:
Wherein C is the classification number of label, and q corresponds to the vector that certain a kind of all output is constituted.
In the step S150, the formula of the total cost J (θ) is as follows:
Wherein L (θ) is the loss function of classifier.
The source domain data are MNIST data set, and the target numeric field data is USPS data set.
DA-RMB modeling method of the invention can effectively identify cross-domain hand-written data.
In the present invention, term " first ", " second ", " third " are used for description purposes only, and should not be understood as instruction or
Imply relative importance.Term " multiple " refers to two or more, unless otherwise restricted clearly.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.
Claims (7)
1. a kind of method for establishing DA-RBM sorter model, which is characterized in that described method includes following steps:
S110, source domain data X is obtainedsAnd source domain data XsCorresponding label Ys, target numeric field data XTAnd label YT；
S120, initialization RBM model parameter, by data Xs、XTIt is input in RBM network, finds out single order feature；
S130, using the single order feature as the input of lower first order network, carry out RBM training；
S140, the hidden layer of RBM is exported into Hs、HTSoftmax recurrence layer is input to classify；
S150, the constraint for carrying out source domain data and aiming field data distribution using MMD in the output of RBM hidden layer；
S160, the constraint for carrying out prediction result using MMD in the top-level categories layer of RBM model；
S170, the total cost function J (θ) for constructing model, by optimizing the total cost function come the ginseng of Optimum Classification device model
Number.
2. the method according to claim 1 for establishing DA-RBM sorter model, which is characterized in that the step S130
In, RBM training is carried out using Gibbs sampling and contrast divergence algorithm.
3. the method according to claim 2 for establishing DA-RBM sorter model, which is characterized in that the step S130 is also
Including being m for source domain data and target numeric field data setting Hidden unit number, learning rate isMaximum cycle of training is T, RBM
The parameter setting of network is respectively that connection weight is set as W, sets b for layer biasing, hidden layer biasing is set as c；Initialize RBM
Network, then the hidden node all for source domain data calculates it and activates probability, activates probability P (hs=1 | vs) it calculates public affairs
Formula is as follows:
P(hsj=1 | vs)=σ (c+ ∑iv1iwij)
It is sampled according to the conditional probability of hidden node using Gibbs, the output for acquiring hidden node is solved by following form:
hs~P (hs|v)
The hidden node all for target numeric field data calculates it and activates probability P (hT=1 | vT), calculation formula is as follows:
P(hTj=1 | vT)=σ (c+ ∑iv1iwij)
It is sampled according to the conditional probability of hidden node using Gibbs, the output for the aiming field hidden node asked is in the following way
It solves: hT~P (hT|v)。
4. the method according to claim 3 for establishing DA-RBM sorter model, which is characterized in that the step S140 packet
Include: it is as follows that softmax returns classification solution calculation formula in layer:
Following formula is used for source domain data:
Following formula is used for aiming field:
5. the method according to claim 4 for establishing DA-RBM sorter model, which is characterized in that the DA-RBM model
Middle the last layer is classification layer, and output is the probability for belonging to each class, and the DA-RBM classifier is in feature extraction layer and divides
The output of class device is measured source domain and aiming field and is lost based on the distribution of MMD；In feature distribution, by objective function
Feature MMD distinguishes to measure the feature distribution on two domains:
Wherein H represents the output of the hidden layer of RBM；
MMD loss is added in the level of classifier, defining MMD loss is conditional MMD, the calculation formula of MMD loss
It is as follows:
Wherein C is the classification number of label, and q corresponds to the vector that certain a kind of all output is constituted.
6. the method according to claim 5 for establishing DA-RBM sorter model, which is characterized in that the step S150
In, the formula of the total cost J (θ) is as follows:
Wherein L (θ) is the loss function of classifier.
7. any method for establishing DA-RBM sorter model according to claim 1~6, which is characterized in that the source
Numeric field data is MNIST data set, and the target numeric field data is USPS data set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810595182.XA CN109034186B (en) | 2018-06-11 | 2018-06-11 | Handwriting data identification method based on DA-RBM classifier model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810595182.XA CN109034186B (en) | 2018-06-11 | 2018-06-11 | Handwriting data identification method based on DA-RBM classifier model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109034186A true CN109034186A (en) | 2018-12-18 |
| CN109034186B CN109034186B (en) | 2022-05-24 |
Family
ID=64612623
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810595182.XA Active CN109034186B (en) | 2018-06-11 | 2018-06-11 | Handwriting data identification method based on DA-RBM classifier model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109034186B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109829804A (en) * | 2019-01-10 | 2019-05-31 | 西安交通大学 | A kind of tax risk recognition methods towards marker samples missing administrative region |
| CN110659744A (en) * | 2019-09-26 | 2020-01-07 | 支付宝(杭州)信息技术有限公司 | Training event prediction model, and method and device for evaluating operation event |
| CN110781970A (en) * | 2019-10-30 | 2020-02-11 | 腾讯科技（深圳）有限公司 | Method, device and equipment for generating classifier and storage medium |
| CN110866536A (en) * | 2019-09-25 | 2020-03-06 | 西安交通大学 | Cross-regional enterprise tax evasion identification method based on PU learning |
| WO2020134409A1 (en) * | 2018-12-28 | 2020-07-02 | 深圳光启空间技术有限公司 | Cross-domain face recognition algorithm, storage medium, and processor |
| CN111738298A (en) * | 2020-05-27 | 2020-10-02 | 哈尔滨工业大学 | Data classification method based on depth-width-variable multi-core learning |
| US11675976B2 (en) | 2019-07-07 | 2023-06-13 | International Business Machines Corporation | Exploitation of domain restrictions for data classification |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101866417A (en) * | 2010-06-18 | 2010-10-20 | 西安电子科技大学 | Method for identifying handwritten Uigur characters |
| CN106203520A (en) * | 2016-07-17 | 2016-12-07 | 西安电子科技大学 | SAR image sorting technique based on degree of depth Method Using Relevance Vector Machine |
| CN107330480A (en) * | 2017-07-03 | 2017-11-07 | 贵州大学 | Hand-written character Computer Identification |
| CN108022589A (en) * | 2017-10-31 | 2018-05-11 | 努比亚技术有限公司 | Aiming field classifier training method, specimen discerning method, terminal and storage medium |
-
2018
- 2018-06-11 CN CN201810595182.XA patent/CN109034186B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101866417A (en) * | 2010-06-18 | 2010-10-20 | 西安电子科技大学 | Method for identifying handwritten Uigur characters |
| CN106203520A (en) * | 2016-07-17 | 2016-12-07 | 西安电子科技大学 | SAR image sorting technique based on degree of depth Method Using Relevance Vector Machine |
| CN107330480A (en) * | 2017-07-03 | 2017-11-07 | 贵州大学 | Hand-written character Computer Identification |
| CN108022589A (en) * | 2017-10-31 | 2018-05-11 | 努比亚技术有限公司 | Aiming field classifier training method, specimen discerning method, terminal and storage medium |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020134409A1 (en) * | 2018-12-28 | 2020-07-02 | 深圳光启空间技术有限公司 | Cross-domain face recognition algorithm, storage medium, and processor |
| CN109829804A (en) * | 2019-01-10 | 2019-05-31 | 西安交通大学 | A kind of tax risk recognition methods towards marker samples missing administrative region |
| US11675976B2 (en) | 2019-07-07 | 2023-06-13 | International Business Machines Corporation | Exploitation of domain restrictions for data classification |
| CN110866536A (en) * | 2019-09-25 | 2020-03-06 | 西安交通大学 | Cross-regional enterprise tax evasion identification method based on PU learning |
| CN110866536B (en) * | 2019-09-25 | 2022-06-07 | 西安交通大学 | Cross-regional enterprise tax evasion identification method based on PU learning |
| CN110659744A (en) * | 2019-09-26 | 2020-01-07 | 支付宝(杭州)信息技术有限公司 | Training event prediction model, and method and device for evaluating operation event |
| CN110659744B (en) * | 2019-09-26 | 2021-06-04 | 支付宝(杭州)信息技术有限公司 | Training event prediction model, and method and device for evaluating operation event |
| CN110781970A (en) * | 2019-10-30 | 2020-02-11 | 腾讯科技（深圳）有限公司 | Method, device and equipment for generating classifier and storage medium |
| CN110781970B (en) * | 2019-10-30 | 2024-04-26 | 腾讯科技（深圳）有限公司 | Classifier generation method, device, equipment and storage medium |
| CN111738298A (en) * | 2020-05-27 | 2020-10-02 | 哈尔滨工业大学 | Data classification method based on depth-width-variable multi-core learning |
| CN111738298B (en) * | 2020-05-27 | 2023-09-12 | 哈尔滨工业大学 | MNIST handwriting digital data classification method based on deep-wide variable multi-core learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109034186B (en) | 2022-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Xu | Understanding graph embedding methods and their applications | |
| CN109034186A (en) | The method for establishing DA-RBM sorter model | |
| Tang et al. | Multi-label patent categorization with non-local attention-based graph convolutional network | |
| CN110046671A (en) | A kind of file classification method based on capsule network | |
| Aggarwal et al. | An insight into machine learning techniques for predictive analysis and feature selection | |
| Kumar et al. | A benchmark to select data mining based classification algorithms for business intelligence and decision support systems | |
| CN112800229B (en) | Knowledge graph embedding-based semi-supervised aspect-level emotion analysis method for case-involved field | |
| Ruby et al. | Analysis of influencing factors in predicting students performance using MLP-A comparative study | |
| Li et al. | Explain graph neural networks to understand weighted graph features in node classification | |
| Navarro et al. | Learning feature representation of Iberian ceramics with automatic classification models | |
| Rabbi et al. | An Approximation For Monitoring The Efficiency Of Cooperative Across Diverse Network Aspects | |
| Khayyat et al. | A deep learning based prediction of arabic manuscripts handwriting style. | |
| Pan et al. | Learning first-order rules with relational path contrast for inductive relation reasoning | |
| Ricci et al. | Thinned random measures for sparse graphs with overlapping communities | |
| Kumari et al. | Retail forecasting using neural network and data mining technique: a review and reflection | |
| Li et al. | University Students' behavior characteristics analysis and prediction method based on combined data mining model | |
| Kadam et al. | Data mining in finance | |
| Zhao et al. | A supervised learning community detection method based on attachment graph model | |
| Gul et al. | Analyzing complex networks: Extracting key characteristics and measuring structural similarities | |
| Patil et al. | Efficient processing of decision tree using ID3 & improved C4. 5 algorithm | |
| Federico et al. | Complex networks and deep learning for copper flow across countries | |
| Wang | Statistical Mechanics for Network Structure and Evolution | |
| Vijay et al. | Hand Written Text Recognition and Global Translation | |
| Song et al. | Prior-guided multi-scale fusion transformer for face attribute recognition | |
| Breu | Point Cloud Classification as a Support for BIM-based Progress Tracking |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |