CN110533193A - Feature and example combine transfer learning method under semi-supervised scene - Google Patents
Feature and example combine transfer learning method under semi-supervised scene Download PDFInfo
- Publication number
- CN110533193A CN110533193A CN201910770868.2A CN201910770868A CN110533193A CN 110533193 A CN110533193 A CN 110533193A CN 201910770868 A CN201910770868 A CN 201910770868A CN 110533193 A CN110533193 A CN 110533193A
- Authority
- CN
- China
- Prior art keywords
- data
- matrix
- aiming field
- sample
- label
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses features under a kind of semi-supervised scene and example to combine transfer learning method, when certain domain classification model trainings aiming at the problem that tape label data deficiencies, and introduce other field data and the field without label data supplemental training, while considering the difference of data distribution between field, for a small amount of tape label data existing in aiming field, there is situation largely without label data again, the present invention proposes mixed balanced distribution adaptive method and self study instance migration method；And construction feature and example combine transfer learning method FSJT based on this.
Description
Technical field
The present invention relates to transfer learning technical fields in machine learning, in particular to feature and reality under a kind of semi-supervised scene
Example joint transfer learning method.
Background technique
It is formal in the bulletin of US Department of Defense Advanced Research Projects Agency (DARPA) Information Processing Technology Office in 2005
Transfer learning one definition is provided, i.e., the knowledge and skills acquired in other tasks are applied to the ability of new task.With more
Business study is compared, transfer learning more concerned be goal task, rather than learn all originating task and goal task simultaneously.Source
Role of the task and goal task in transfer learning is no longer symmetrical.The task space for needing to study in transfer learning claims
For aiming field (Target domain), and task space associated before is known as source domain (Source domain).Existing rank
The development of section transfer learning mainly has following direction, and the transfer learning of Case-based Reasoning, the transfer learning based on feature are based on
The migration etc. of parameter.
The migration of Case-based Reasoning is more intuitive, main thought are as follows: although source domain data cannot be utilized directly, data certain
A little parts still can be used together with the existing training data in aiming field.How the source domain data that can be used are screened
As the emphasis of instance migration study, such as classical instance migration method TrAdaBoost algorithm.In addition from the ratio of distribution
Angle carries out correlative study, proposes transmitting transfer learning method (Transitive Transfer Learning, TTL) etc..
Migration based on feature, which refers to, migrates the data field by way of eigentransformation, to reduce source domain
With the gap of aiming field.Component analyzing method (Transfer Component Analysis, TCA) is migrated as classical spy
Shifting method of relocating residents from locations to be used for construction of new buildings or factories is proposed that this method utilizes Largest Mean difference (Maximum Mean Discrepancy, MMD) by Pan et al.
As the measurement criterion of probability distribution between field, target is to minimize distributional difference.
Currently, there is scholar to make exploration in the moving method for combining example and feature.Long et al. proposition is minimizing
While distribution distance, migration joint matching (Tranfer Joint Matching, TJM) method of example selection is added, it will
Example and feature transfer learning method have carried out organic combination.But current transfer learning method mainly utilizes in aiming field
A small amount of tape label data, or directly utilize no label data.And traditional semi-supervised learning utilizes in a field simultaneously
Therefore two class data if considering semi-supervised scene in transfer learning, can learn to more target domains are existing to know
Know.
Summary of the invention
The purpose of the present invention is to solve above-mentioned background technique there are the problem of, and propose a kind of semi-supervised scene under
Feature and example joint transfer learning method (Feature and Sample Jointed Transfer, abbreviation FSJT), for
Aiming field has a small amount of tape label data and largely without the situation of label data, and the situation that source domain and aiming field differ greatly, with
Phase promotes the accuracy rate classified on aiming field.
To achieve the above object, feature and example combine transfer learning method under the semi-supervised scene designed by the present invention,
It is characterized in that, the described method comprises the following steps:
Step 1: data in Definition Model: giving a field, it is known that each data generic uses " 0 " in the field
Or " 1 " indicates, is denoted as source domain Ds, the n sample eigenmatrix and class label vector form for including be expressed as
It is abbreviated as { xs,ys}；Another field is given, aiming field D is denoted ast, wherein m sample composition tape label data set is denoted asIt is denoted as without label data collectionTwo class data sets are denoted as altogetherInclude m+
K training sample, and assumeMiddle sample size is not enough to one reliable classifier of training；
Assuming that feature space Xs=Xt, i.e., the feature type in two fields is identical as quantity, classification space Ys=Yt, but side
Fate cloth Ps(xs)≠Pt(xt), condition is distributed Ps(ys|xs)≠Pt(yt|xt), target is to utilize source domain DsMiddle data set { xs,ys}
With aiming field DtMiddle data setUnlabeled data in learning objective domainClassificationWherein r=
1 ..., l, n, m, k, l are the natural number greater than 1；
Step 2: balance parameters μ is utilized, objective function is constructed and carries out abbreviation and solve to obtain:
Wherein objective function minimum value is sought in min expression, and s.t. indicates bound for objective function.Matrix is sought in tr () expression
Mark, the input matrix that X is made of source domain and aiming field data set, and meet X=[xs,xt], wherein
Mapping function ψ:By kernel function K=ψ (X)Tψ (X) is provided, and λ is regularization coefficient,Indicate the flat of F norm
Side, balance factor μ, A indicate that transition matrix, H=I- (1/n) 1 are center matrix, I ∈ R(n+m+k+l)×(n+m+k+l)For unit square
Battle array, M0, McFor MMD matrix, make is as follows:
Wherein { 0,1 } c ∈,WithIt is illustrated respectively in the number for belonging to the sample of c class in source domain and aiming field, nc
Indicate source domain data set { xs,ysIn belong to c class sample number；mcIndicate aiming field tape label data setIn
Belong to the number of the sample of c class；kcIndicate aiming field without label data collectionIn belong to c class sample number；lcTable
Show aiming field test data set { xtestIn belong to c class sample number；
Step 3: being solved using method of Lagrange multipliers, set Lagrangian as Φ=(φ1,φ2,...,φd), d
For the minimal eigenvalue of A, the local derviation that transition matrix A is sought after Lagrangian is derived:
Step 4: calculating transformed matrix A using MatlabTψ(X)；
Step 5: be directed to transformed source domain and target numeric field data, carry out instance migration study, with filter out in source domain with
The sample that aiming field differs greatly, specifically:
5.1: data preparation, transformed source domain training dataset { zs,ys, aiming field tape label training datasetAiming field not tape label training datasetWith the test data set { x of aiming fieldtest}；
5.2: initialization weight vectorsWherein 1 to n weight is 1/n；The weight of n+1 to n+m is
1/m, settingN is the number of iterations；
5.3: iterative calculation updates sample weights vector wv+1, wherein source domain DsOn weight be updated to
Wherein i=1 ..., n, aiming field DtOn weight be updated toWherein i=n+1 ..., n+m；
Confidence level is calculated using such as following formulaScreen the δ that confidence level is greater than γuA sample, wherein γ indicates confidence
Threshold value is spent,It is addedData set, wherein u=1 ..., U, U indicate the number of iterations, update m=m+
δu, and fromMiddle these samples of deletion；
5.5: step 5.2~5.4 are repeated, until u=U；
5.6: it is each primary using the data set repetition step 5.2 and 5.3 that update completion, calculate { ztestPrediction label hf
(ztest)。
Preferably, the step 4 includes following sub-step:
4.1: data preparation, source domain training dataset { xs,ys, aiming field training dataset dataWith
Aiming field test data set { xtest}；
4.2: data set merges, by source domain eigenmatrix xsWith target domain characterization matrixIt is combined into square
Battle array X=[xs,xt]；
4.3: utilizing existing tag setCalculate initial mc, with source domain data { xs,ysOne classifier of training, in advance
It surveys in aiming field without label data collectionWith test data set { xtestInitial pseudo label collectionWithAnd it calculates
Initial kcAnd lc, calculate initial M0And Mc；
4.4: iteration T times, T are the natural number greater than 1, update McMatrix, until iteration terminates；
4.5: using the formula calculating matrix A of step 3, obtaining transformed matrix { ATψ(xs),ys, training is same on it
The classifier of one type, prediction aiming field have label data collectionPseudo label collectionAccording to
On accuracy rate repeat step 3, using PSO optimization algorithm adjusting parameter μ, d, and obtain optimal parameter combination；
4.6 repeat step 3 using parameter after updating, and calculate ATψ (X) matrix, and be decomposed intoShape
Formula.
Preferably, the calculating step of each iteration includes: in the step 5.3
(1) it calculatesThe component of weight is set to add up to 1, v=1 ..., N；
(2) sample weights ω is utilizedvTraining base classifier hv, predict in aiming field without label training datasetMark
Label, are denoted asAnd calculate the error ε in aiming field tape label datav:
(3) β is setv=εv/(1-εv), and update sample weights vector wv+1, source domain DsOn weight be updated toWherein i=1 ..., n, aiming field DtOn weight be updated toWherein i=n
+1,...,n+m。
Preferably, in the step 4.4 each iteration the following steps are included:
(1) setup parameter λ, kernel function K are fixed value, and the initial value of setup parameter μ, d utilize function eigs solution matrix
A；
(2) in { ATψ(xs),ysOn the same classifier of training, prediction{ AT(xtest) tally setWithAnd update kcAnd lc, calculateUpper classification accuracy η；
(3) M is updatedcMatrix, until iteration terminates.
Preferably, it is calculated in the step 5.3The prediction label of middle dataFormula are as follows:
Preferably, confidence level is calculated in the step 5.3Formula are as follows:
The present invention introduces other field when certain domain classification model trainings aiming at the problem that tape label data deficiencies
Data and the field without label data supplemental training, while considering the difference of data distribution between field, propose a kind of half
Supervise feature and example joint transfer learning model (Feature and Sample Jointed Transfer, abbreviation under scene
FSJT).In terms of novelty, for a small amount of tape label data existing in aiming field, and there is situation largely without label data, mention
The balanced distribution adaptive method and self study instance migration method mixed out；And construction feature and example are combined and are moved based on this
Move learning method FSJT.
Compared with the existing technology, of the invention to have the advantages that
1, the present invention is directed to the feature space of source domain and aiming field data set, carries out migration feature transformation, to reduce two necks
The difference in characteristic of field space, the situation to differ greatly suitable for source domain and aiming field；
2, the present invention considers in semi-supervised scene the marginal probability distribution of data and conditional probability distribution and prison in aiming field
The difference in scene is superintended and directed, in order to minimize the statistical distribution difference of data between field, is translated into and solves objective function most
The problem of small value；Using balance parameters by distance and conditional distribution function between the edge feature distribution function of two FIELD Datas
Between distance introduce objective function simultaneously；
3, present invention building method of the learning objective domain without label data information, will introduce instance migration without label data
Algorithm is practised, to be suitable for semi-supervised learning scene
4, the present invention expands instance migration method using self study thought, and model is made to be suitable for semi-supervised learning field
Scape is conducive to the example weight of target classification task by improving in training, reduces the example for being unfavorable for target classification task
Weight removes the sample to differ greatly in source domain.
Detailed description of the invention
Fig. 1 is the flow chart that feature and example combine transfer learning method under the semi-supervised scene of the present invention.
Fig. 2 to realize the present invention under semi-supervised scene feature and example joint transfer learning method computer model structure
Schematic diagram.
Specific embodiment
Present invention will now be described in further detail with reference to the embodiments and the accompanying drawings, but embodiments of the present invention are unlimited
In this.
Basic procedure of the invention is as shown in Figure 1, feature and reality under a kind of semi-supervised scene provided in an embodiment of the present invention
Example joint transfer learning method, comprising the following steps:
Step 1: data are as follows in Definition Model:
Give a field, it is known that each data generic is indicated with " 0 " or " 1 " in the field, is denoted as source domain Ds。
The n sample eigenmatrix and class label vector form for including are expressed asIt is abbreviated as { xs,ys}.It is given another
One field, is denoted as aiming field Dt.Wherein m sample composition tape label data set is denoted asWithout label data collection
It is denoted asWherein q=1 ..., k.Two class data sets are denoted as altogetherComprising m+k training sample, and assumeMiddle sample size is not enough to one reliable classifier of training.
Assuming that feature space Xs=Xt, i.e., the feature type in two fields is identical as quantity, classification space Ys=Yt, but side
Fate cloth Ps(xs)≠Pt(xt), condition is distributed Ps(ys|xs)≠Pt(yt|xt).Institute is consideration is that utilize source domain DsMiddle data
Collect { xs,ysAnd aiming field DtMiddle data setUnlabeled data in learning objective domainClassificationIts
Middle r=1 ..., l.
Step 2: utilizing balance parameters μ, using balance parameters μ, construct objective function and carry out abbreviation solution.
2.1: utilizing balance parameters μ, while considering distance and condition between the edge feature distribution function of two FIELD Datas
Distance between distribution function.It converts the difference for making aiming field and source domain minimum to and solves objective function minimum problems, structure
The objective function built is as follows:
Wherein, as a=1 ..., m,WithIt respectively indicatesWithWork as a=m+1 ..., m
When+k,It indicatesPseudo label,WithIt respectively indicatesWith It indicatesPseudo- mark
Label.As a=m+k+1 ..., m+k+l,It indicatesPseudo label,WithIt respectively indicatesWith It indicatesPseudo label.Expression condition is distributed P (yt|xt) estimated value.
2.2: using MMD distance metric by objective function abbreviation in step 2 be numerical expression form:
Wherein, H indicates reproducing kernel Hilbert space (reproducing kernel Hilbert space abbreviation
RKHS)。Indicate square of the norm (abbreviation H norm) on reproducing kernel Hilbert space.C ∈ { 0,1 },WithPoint
The number of the sample of c class Biao Shi not be belonged in source domain and aiming field.ncIndicate source domain data set { xs,ysIn belong to the sample of c class
This number；mcIndicate aiming field tape label data setIn belong to c class sample number；kcIndicate aiming field not
Tape label data setIn belong to c class sample number；lcIndicate aiming field test data set { xtestIn belong to c class
The number of sample.
2.3: utilize matrix skill and regularization by its abbreviation:
Wherein objective function minimum value is sought in min expression, and s.t. indicates bound for objective function.Matrix is sought in tr () expression
Mark,.The input matrix that X is made of source domain and aiming field data set, and meet X=[xs,xt], wherein
Mapping function ψ:By kernel function K=ψ (X)Tψ (X) is provided.λ is regularization coefficient,Indicate the flat of F norm
Side, balance factor μ.Constraint condition ensures the data A after conversionTThe built-in attribute of ψ (X) reservation initial data.Wherein A is indicated
Transition matrix, H=I- (1/n) 1 are center matrix, I ∈ R(n+m+k+l)×(n+m+k+l)For unit matrix.M0, McFor MMD matrix, construction
Mode is as follows:
Step 3: being solved using method of Lagrange multipliers, set Lagrangian as Φ=(φ1,φ2,...,φd),
The local derviation of A is asked to obtain following result after deriving Lagrangian:
The formula is generalized eigenvalue problem by solving the available matrix A of d minimal eigenvalue of A, and then acquires change
Data A after changingTψ(X)。
Step 4: calculating transformed matrix A using MatlabTψ (X) calculates A using parameter after updateTψ (X) matrix, and
It is decomposed intoForm.
Specifically:
4.1: data preparation, source domain training dataset { xs,ys, aiming field training dataset dataWith
Aiming field test data set { xtest}。
4.2: data set merges, by source domain eigenmatrix xsWith target domain characterization matrixIt is combined into square
Battle array X=[xs,xt]。
4.3: utilizing existing tag setCalculate initial mc, with source domain data { xs,ysOne classifier of training, in advance
It surveys in aiming field without label data collectionWith test data set { xtestInitial pseudo label collectionWithAnd it calculates
Initial kcAnd lc, utilize M in step 40And McExpression formula calculates initial M0And Mc。
4.4: iteration T times, each operation are as follows:
1) setup parameter λ, kernel function K are fixed value, and the initial value of setup parameter μ, d utilize function eigs solution matrix A.
2) is in { ATψ(xs),ysOn the same classifier of training, prediction{ AT(xtest) tally set
WithAnd update kcAnd lc, calculateUpper classification accuracy η.
3) updates McMatrix, until iteration terminates.
4.5: using the calculated matrix A of the formula of step 3, transformed matrix { A can be obtainedTψ(xs),ys, on it
The same type of classifier of training, prediction aiming field have label data collectionPseudo label collectionAccording toOn accuracy rate repeat step (4), using PSO optimization algorithm adjusting parameter μ, d, and obtain optimal parameter group
It closes.
4.6: repeating step (4) using parameter after updating, calculate ATψ (X) matrix, and be decomposed into's
Form.
Step 5: be directed to transformed source domain and target numeric field data, carry out instance migration study, with filter out in source domain with
The sample that aiming field differs greatly, specifically:
5.1: data preparation, transformed source domain training dataset { zs,ys, aiming field tape label training datasetAiming field not tape label training datasetWith the test data set { x of aiming fieldtest}。
5.2: initialization weight vectorsWherein 1 to n weight is 1/n；The weight of n+1 to n+m is
1/m.SettingWherein N is the number of iterations.
5.3: iterative calculation updates sample weights vector wv+1, wherein source domain DsOn weight be updated to
Wherein i=1 ..., n, aiming field DtOn weight be updated toWherein i=n+1 ..., n+m.
Need to calculate following several steps (v=1 ..., N) for each iteration:
1) is calculatedThe component of weight is set to add up to 1.
2) utilizes sample weights ωvTraining base classifier hv, predict in aiming field without label training datasetMark
Label, are denoted asAnd the error ε in aiming field tape label data is calculated using such as following formulav。
3) β is arranged inv=εv/(1-εv), and update sample weights vector wv+1.Source domain DsOn weight be updated toWherein i=1 ..., n.Aiming field DtOn weight be updated toWherein i=n+
1,...,n+m。
5.4: being calculated using such as following formulaThe prediction label of middle data
Confidence level is calculated using such as following formulaScreen the δ that confidence level is greater than γuA sample, wherein γ indicates confidence
Spend threshold value.It is addedData set, wherein u=1 ..., U, U indicate the number of iterations.Update m=m+
δu, and fromMiddle these samples of deletion.
5.5: step 5.2~5.4 are repeated, until u=U；
5.6: it is each primary using the data set repetition step 5.2 and 5.3 that update completion, calculate { ztestPrediction label hf
(ztest)。
By above-mentioned process, can to after eigentransformation source domain and aiming field training data carry out self study example and move
It moves, and predicts the label of aiming field data to be tested.
Instance analysis:
1. data set prepares:
Integrated using transfer learning general data as OFFICE, it is made of the data in three fields, be followed successively by Amazon,
Webcam and DSLR.There are 985,295,295 pictures in each field respectively.The discrete feature for turning to 800 dimensions of each picture is constituted
Vector.These three fields are abbreviated as A, W, D respectively.
2. data prediction:
The selection of source domain and aiming field sample.Altogether there are four types of to experimental program, being followed successively by A is source domain sample, and D is aiming field
Sample；A is source domain sample, and W is aiming field sample；D is source domain sample, and W is aiming field sample；W is source domain sample, and D is target
Domain sample.In each experimental program, aiming field training sample is 90% data randomly selected, and 10% data, which are used as, to be tested
Card.
3. the setting of parameter:
Basic classification device is set as k nearest neighbor classifier, and parameter k is set as 8.The number of iterations T is set as 10, and parameter lambda is excellent
In change problemCoefficient, i.e. the regular parameter of F norm is set as λ=0.01, Selection of kernel function gaussian kernel function here.μ
It is that tuning is needed in algorithm with d, μ balances M in solution procedure0、McThe importance of matrix, further μ is used to balance side
The shared weight of fate cloth and condition distribution, and obtained matrix A is made of d dimensional feature vector.Initial μ and d are set first
Parameter adjusts the two parameters by the accuracy rate on aiming field tape label training data.After setting parameter, input is each
The classification accuracy in test data can be obtained in the data of scheme, operation program.
4. the precision of different models compares:
An existing transfer learning method is selected, distribution adaptive method (BDA) algorithm of balance is compared, but due to
BDA is applicable in unsupervised scene, when training BDA aiming field only choose that MBDA uses without label training data.
While in order to verify the performance of proposed method difference between process field, with the sorting algorithm in three kinds of machine learning
It compares.Here support vector machines (SVM), three kinds of logistic regression (LR), decision tree (DT) algorithms are selected.Every kind of algorithm uses
Data it is consistent with FSJT model, utilize the training of data to practice, the prediction of another data set.
Since there are three fields for OFFICE data set, several groups of different fields are provided respectively as trained and test data
Model as a result, as shown in table 1 when collection.
The classifying quality comparing result of each model of table 1
The result from table, can it is found that migration effect of the FSJT and BDA method in field A to other two fields is all poor
It larger can be caused by the gap in A data set and other two fields.But the transfer learning effect between the field W and D is preferable, FSJT
Reach 88.41% and 89.27%, and is above the accuracy rate 85.37% and 86.02% of D → W and W → D in BDA method.By
A small amount of tape label data on aiming field are utilized in FSJT, therefore may learn more target domain characterizations, there is precision
It is improved.Accuracy rate highest of the SVM model in four kinds of data in the result of three kinds of machine learning methods is utilizing W data collection
The precision of training, the test of D data set has reached 77.71%, followed by the 72.33% of LR model and DT model.And comprehensive five classes
Method as a result, two class transfer learning methods are all higher than three kinds of machine learning algorithms in the result of four kinds of situations.It may be due to machine
Device study needs to guarantee training data and the same distributional assumption of test data, and transfer learning method can handle the difference of two class data
It is different, so that the generalization ability of model has obtained certain promotion.To sum up, performance of the FSJT method on normal data demonstrates the mould
The validity of type.
It should be understood that it is above-mentioned for the more detailed of preferred embodiment, can not therefore it be considered to the present invention
The limitation of scope of patent protection, those skilled in the art under the inspiration of the present invention, want not departing from right of the present invention
It asks under protected ambit, replacement or deformation can also be made, fallen within the protection scope of the present invention, of the invention asks
Ask that the scope of protection shall be subject to the appended claims.
Claims (6)
1. feature and example combine transfer learning method under a kind of semi-supervised scene, it is characterised in that: the method includes following
Step:
Step 1: data in Definition Model: giving a field, it is known that each data generic uses " 0 " or " 1 " in the field
It indicates, is denoted as source domain Ds, the n sample eigenmatrix and class label vector form for including be expressed asIt is abbreviated as
{xs,ys}；Another field is given, aiming field D is denoted ast, wherein m sample composition tape label data set is denoted asNo
Tape label data set is denoted asQ=1 ..., k, two class data sets are denoted as altogetherInclude m+k trained sample
This, and assumeMiddle sample size is not enough to one reliable classifier of training；
Assuming that feature space Xs=Xt, i.e., the feature type in two fields is identical as quantity, classification space Ys=Yt, but edge distribution
Ps(xs)≠Pt(xt), condition is distributed Ps(ys|xs)≠Pt(yt|xt), target is to utilize source domain DsMiddle data set { xs,ysAnd target
Domain DtMiddle data setUnlabeled data in learning objective domainClassificationWherein r=1 ..., l,
N, m, k, l are the natural number greater than 1；
Step 2: balance parameters μ is utilized, objective function is constructed and carries out abbreviation and solve to obtain:
Wherein objective function minimum value is sought in min expression, and tr () indicates to seek the mark of matrix, and X is by source domain and aiming field data set group
At input matrix, and meet X=[xs,xt], whereinMapping function ψ:By kernel function
K=ψ (X)Tψ (X) is provided, and λ is regularization coefficient,Indicate that square of F norm, balance factor μ, A indicate transition matrix, H
=I- (1/n) 1 is center matrix, I ∈ R(n+m+k+l)×(n+m+k+l)For unit matrix, M0, McFor MMD matrix, make is as follows:
Wherein { 0,1 } c ∈,WithIt is illustrated respectively in the number for belonging to the sample of c class in source domain and aiming field, ncExpression source
Numeric field data collection { xs,ysIn belong to c class sample number；mcIndicate aiming field tape label data setIn belong to c
The number of the sample of class；kcIndicate aiming field without label data collectionIn belong to c class sample number；lcIndicate target
Domain test data set { xtestIn belong to c class sample number；
Step 3: being solved using method of Lagrange multipliers, set Lagrangian as Φ=(φ1,φ2,...,φd), d A
Minimal eigenvalue, derive the local derviation that transition matrix A is sought after Lagrangian:
Step 4: calculating transformed matrix A using MatlabTψ(X)；
Step 5: be directed to transformed source domain and target numeric field data, carry out instance migration study, with filter out in source domain with target
The sample that domain differs greatly, specifically:
5.1: data preparation, transformed source domain training dataset { zs,ys, aiming field tape label training dataset
Aiming field not tape label training datasetWith the test data set { x of aiming fieldtest}；
5.2: initialization weight vectorsWherein 1 to n weight is 1/n；The weight of n+1 to n+m is 1/m,
SettingN is the number of iterations；
5.3: iterative calculation updates sample weights vector wv+1, wherein source domain DsOn weight be updated toIts
Middle i=1 ..., n, aiming field DtOn weight be updated toWherein i=n+1 ..., n+m；
5.4: being calculated using following formulaThe prediction label of middle data
Confidence level is calculated using such as following formulaScreen the δ that confidence level is greater than γuA sample, wherein γ indicates confidence level threshold
Value,It is addedData set, wherein u=1 ..., U, U indicate the number of iterations, update m=m+ δu, and
FromMiddle these samples of deletion,
5.5: step 5.2~5.4 are repeated, until u=U；
5.6: it is each primary using the data set repetition step 5.2 and 5.3 that update completion, calculate { ztestPrediction label hf
(ztest)。
2. feature and example combine transfer learning method under semi-supervised scene according to claim 1, it is characterised in that: institute
Stating step 4 includes following sub-step:
4.1: data preparation, source domain training dataset { xs,ys, aiming field training dataset dataAnd aiming field
Test data set { xtest}；
4.2: data set merges, by source domain eigenmatrix xsWith target domain characterization matrixIt is combined into matrix X
=[xs,xt]；
4.3: utilizing existing tag setCalculate initial mc, with source domain data { xs,ysOne classifier of training, predict mesh
It marks in domain without label data collectionWith test data set { xtestInitial pseudo label collectionWithAnd it calculates initial
kcAnd lc, calculate initial M0And Mc；
4.4: iteration T times, T are the natural number greater than 1, update McMatrix, until iteration terminates；
4.5: using the formula calculating matrix A of step 3, obtaining transformed matrix { ATψ(xs),ys, same class is trained on it
The classifier of type, prediction aiming field have label data collectionPseudo label collectionAccording toOn
Accuracy rate repeats step 3, using PSO optimization algorithm adjusting parameter μ, d, and obtains optimal parameter combination；
4.6 repeat step 3 using parameter after updating, and calculate ATψ (X) matrix, and be decomposed intoForm.
3. feature and example combine transfer learning method under semi-supervised scene according to claim 1, it is characterised in that: institute
The calculating step for stating each iteration in step 5.3 includes:
(1) it calculatesThe component of weight is set to add up to 1, v=1 ..., N；
(2) sample weights ω is utilizedvTraining base classifier hv, predict in aiming field without label training datasetLabel, note
ForAnd calculate the error ε in aiming field tape label datav:
(3) β is setv=εv/(1-εv), and update sample weights vector wv+1, source domain DsOn weight be updated to
Wherein i=1 ..., n, aiming field DtOn weight be updated toWherein i=n+1 ..., n+m.
4. feature and example combine transfer learning method under semi-supervised scene according to claim 1, it is characterised in that: institute
State each iteration in step 4.4 the following steps are included:
(1) setup parameter λ, kernel function K are fixed value, and the initial value of setup parameter μ, d utilize function eigs solution matrix A；
(2) in { ATψ(xs),ysOn the same classifier of training, prediction{ AT(xtest) tally setWithAnd update kcAnd lc, calculateUpper classification accuracy η；
(3) M is updatedcMatrix, until iteration terminates.
5. feature and example combine transfer learning method under semi-supervised scene according to claim 1, it is characterised in that: institute
It states in step 5.3 and calculatesThe prediction label of middle dataFormula are as follows:
6. feature and example combine transfer learning method under semi-supervised scene according to claim 1, it is characterised in that: institute
It states and calculates confidence level in step 5.3Formula are as follows:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910770868.2A CN110533193A (en) | 2019-08-20 | 2019-08-20 | Feature and example combine transfer learning method under semi-supervised scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910770868.2A CN110533193A (en) | 2019-08-20 | 2019-08-20 | Feature and example combine transfer learning method under semi-supervised scene |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110533193A true CN110533193A (en) | 2019-12-03 |
Family
ID=68662299
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910770868.2A Pending CN110533193A (en) | 2019-08-20 | 2019-08-20 | Feature and example combine transfer learning method under semi-supervised scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110533193A (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111312403A (en) * | 2020-01-21 | 2020-06-19 | 山东师范大学 | Disease prediction system, device and medium based on instance and feature sharing cascade |
| CN111581380A (en) * | 2020-04-29 | 2020-08-25 | 南京理工大学紫金学院 | Single-point and double-point smooth combination manifold regularization semi-supervised text classification method |
| CN111667000A (en) * | 2020-06-05 | 2020-09-15 | 成都理工大学 | Earthquake early warning method of adaptive field deep neural network |
| CN111724487A (en) * | 2020-06-19 | 2020-09-29 | 广东浪潮大数据研究有限公司 | Flow field data visualization method, device, equipment and storage medium |
| CN111914708A (en) * | 2020-07-23 | 2020-11-10 | 杭州电子科技大学 | Electroencephalogram signal classification method for migration semi-supervised width learning |
| CN112070236A (en) * | 2020-09-11 | 2020-12-11 | 福州大学 | Sparse feature learning method for solving online complex optimization calculation based on transfer learning |
| CN112085085A (en) * | 2020-09-03 | 2020-12-15 | 哈尔滨工业大学 | Multi-source transfer learning method based on graph structure |
| CN112288004A (en) * | 2020-10-28 | 2021-01-29 | 香港中文大学（深圳） | Semi-supervision method without consistency constraint and mobile terminal |
| CN112285565A (en) * | 2020-09-21 | 2021-01-29 | 电子科技大学 | Method for predicting SOH (State of health) of battery by transfer learning based on RKHS (remote keyless entry) domain matching |
| CN112861929A (en) * | 2021-01-20 | 2021-05-28 | 河南科技大学 | Image classification method based on semi-supervised weighted migration discriminant analysis |
| CN112883994A (en) * | 2020-12-28 | 2021-06-01 | 重庆邮电大学 | Rotating machinery variable working condition fault diagnosis method with balanced distribution adaptation |
| CN113030197A (en) * | 2021-03-26 | 2021-06-25 | 哈尔滨工业大学 | Gas sensor drift compensation method |
| CN113283489A (en) * | 2021-05-19 | 2021-08-20 | 浙大宁波理工学院 | Semi-supervised domain adaptive learning classification method based on joint distribution matching |
| CN113420775A (en) * | 2021-03-31 | 2021-09-21 | 中国矿业大学 | Image classification method under extremely small quantity of training samples based on adaptive subdomain field adaptation of non-linearity |
| CN114500325A (en) * | 2022-01-27 | 2022-05-13 | 重庆邮电大学 | SDN controller fault self-adaptive intelligent detection method based on unsupervised transfer learning |
| CN116167288A (en) * | 2023-04-19 | 2023-05-26 | 中国科学技术大学 | UA2FT semi-supervised transfer learning modeling method, medium and equipment |
| CN116488158A (en) * | 2023-04-24 | 2023-07-25 | 国能日新科技股份有限公司 | Method and device for predicting medium-long-term electric quantity of wind power based on transfer learning |
| CN117096070A (en) * | 2023-10-19 | 2023-11-21 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Semiconductor processing technology abnormality detection method based on field self-adaption |
-
2019
- 2019-08-20 CN CN201910770868.2A patent/CN110533193A/en active Pending
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111312403A (en) * | 2020-01-21 | 2020-06-19 | 山东师范大学 | Disease prediction system, device and medium based on instance and feature sharing cascade |
| CN111581380A (en) * | 2020-04-29 | 2020-08-25 | 南京理工大学紫金学院 | Single-point and double-point smooth combination manifold regularization semi-supervised text classification method |
| CN111667000A (en) * | 2020-06-05 | 2020-09-15 | 成都理工大学 | Earthquake early warning method of adaptive field deep neural network |
| CN111667000B (en) * | 2020-06-05 | 2023-02-07 | 成都理工大学 | Earthquake early warning method of adaptive field deep neural network |
| CN111724487A (en) * | 2020-06-19 | 2020-09-29 | 广东浪潮大数据研究有限公司 | Flow field data visualization method, device, equipment and storage medium |
| CN111724487B (en) * | 2020-06-19 | 2023-05-16 | 广东浪潮大数据研究有限公司 | Flow field data visualization method, device, equipment and storage medium |
| WO2021253666A1 (en) * | 2020-06-19 | 2021-12-23 | 广东浪潮智慧计算技术有限公司 | Flow field data visualization method, apparatus and device, and storage medium |
| CN111914708A (en) * | 2020-07-23 | 2020-11-10 | 杭州电子科技大学 | Electroencephalogram signal classification method for migration semi-supervised width learning |
| CN111914708B (en) * | 2020-07-23 | 2023-09-12 | 杭州电子科技大学 | Electroencephalogram signal classification method for migration semi-supervised width learning |
| CN112085085B (en) * | 2020-09-03 | 2024-02-23 | 哈尔滨工业大学 | Multi-source migration learning method based on graph structure |
| CN112085085A (en) * | 2020-09-03 | 2020-12-15 | 哈尔滨工业大学 | Multi-source transfer learning method based on graph structure |
| CN112070236A (en) * | 2020-09-11 | 2020-12-11 | 福州大学 | Sparse feature learning method for solving online complex optimization calculation based on transfer learning |
| CN112285565A (en) * | 2020-09-21 | 2021-01-29 | 电子科技大学 | Method for predicting SOH (State of health) of battery by transfer learning based on RKHS (remote keyless entry) domain matching |
| CN112285565B (en) * | 2020-09-21 | 2021-07-13 | 电子科技大学 | Method for predicting SOH (State of health) of battery by transfer learning based on RKHS (remote keyless entry) domain matching |
| CN112288004A (en) * | 2020-10-28 | 2021-01-29 | 香港中文大学（深圳） | Semi-supervision method without consistency constraint and mobile terminal |
| CN112883994A (en) * | 2020-12-28 | 2021-06-01 | 重庆邮电大学 | Rotating machinery variable working condition fault diagnosis method with balanced distribution adaptation |
| CN112883994B (en) * | 2020-12-28 | 2022-05-10 | 重庆邮电大学 | Rotating machinery variable working condition fault diagnosis method with balanced distribution adaptation |
| CN112861929A (en) * | 2021-01-20 | 2021-05-28 | 河南科技大学 | Image classification method based on semi-supervised weighted migration discriminant analysis |
| CN113030197A (en) * | 2021-03-26 | 2021-06-25 | 哈尔滨工业大学 | Gas sensor drift compensation method |
| CN113420775A (en) * | 2021-03-31 | 2021-09-21 | 中国矿业大学 | Image classification method under extremely small quantity of training samples based on adaptive subdomain field adaptation of non-linearity |
| CN113420775B (en) * | 2021-03-31 | 2024-03-29 | 中国矿业大学 | Picture classification method based on nonlinearity self-adaptive subdomain field adaptation under extremely small quantity of training samples |
| CN113283489B (en) * | 2021-05-19 | 2023-06-30 | 浙大宁波理工学院 | Classifying method for semi-supervised domain adaptive learning based on joint distribution matching |
| CN113283489A (en) * | 2021-05-19 | 2021-08-20 | 浙大宁波理工学院 | Semi-supervised domain adaptive learning classification method based on joint distribution matching |
| CN114500325B (en) * | 2022-01-27 | 2023-07-18 | 重庆邮电大学 | SDN controller fault self-adaptive intelligent detection method based on unsupervised transfer learning |
| CN114500325A (en) * | 2022-01-27 | 2022-05-13 | 重庆邮电大学 | SDN controller fault self-adaptive intelligent detection method based on unsupervised transfer learning |
| CN116167288A (en) * | 2023-04-19 | 2023-05-26 | 中国科学技术大学 | UA2FT semi-supervised transfer learning modeling method, medium and equipment |
| CN116488158A (en) * | 2023-04-24 | 2023-07-25 | 国能日新科技股份有限公司 | Method and device for predicting medium-long-term electric quantity of wind power based on transfer learning |
| CN116488158B (en) * | 2023-04-24 | 2023-09-15 | 国能日新科技股份有限公司 | Method and device for predicting medium-long-term electric quantity of wind power based on transfer learning |
| CN117096070A (en) * | 2023-10-19 | 2023-11-21 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Semiconductor processing technology abnormality detection method based on field self-adaption |
| CN117096070B (en) * | 2023-10-19 | 2024-01-05 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Semiconductor processing technology abnormality detection method based on field self-adaption |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110533193A (en) | Feature and example combine transfer learning method under semi-supervised scene | |
| Gauci et al. | Horizon: Facebook's open source applied reinforcement learning platform | |
| CN110363344B (en) | Probability integral parameter prediction method for optimizing BP neural network based on MIV-GP algorithm | |
| US20200074246A1 (en) | Capturing network dynamics using dynamic graph representation learning | |
| Liu et al. | Energy-based imitation learning | |
| Yao et al. | RDAM: A reinforcement learning based dynamic attribute matrix representation for virtual network embedding | |
| CN109002845A (en) | Fine granularity image classification method based on depth convolutional neural networks | |
| CN111881342A (en) | Recommendation method based on graph twin network | |
| CN113191484A (en) | Federal learning client intelligent selection method and system based on deep reinforcement learning | |
| CN114841257B (en) | Small sample target detection method based on self-supervision comparison constraint | |
| CN106157154B (en) | Complex network community discovery method under adaptive evolution bat algorithm for self-media network data | |
| CN111931801B (en) | Dynamic route network learning method based on path diversity and consistency | |
| CN115270007B (en) | POI recommendation method and system based on mixed graph neural network | |
| CN112115993A (en) | Zero sample and small sample evidence photo anomaly detection method based on meta-learning | |
| CN113807176A (en) | Small sample video behavior identification method based on multi-knowledge fusion | |
| CN114556364A (en) | Neural architecture search based on similarity operator ordering | |
| CN115269861A (en) | Reinforced learning knowledge graph reasoning method based on generative confrontation and imitation learning | |
| J. Toal et al. | Performance of an ensemble of ordinary, universal, non-stationary and limit Kriging predictors | |
| CN114819091A (en) | Multi-task network model training method and system based on self-adaptive task weight | |
| Lin et al. | Localization-aware meta tracker guided with adversarial features | |
| Deng et al. | Label propagation on k-partite graphs with heterophily | |
| CN116595543A (en) | Processing system for developing application data by software based on Internet platform | |
| CN114254199A (en) | Course recommendation method based on bipartite graph projection and node2vec | |
| CN111612572A (en) | Adaptive local low-rank matrix approximate modeling method based on recommendation system | |
| CN111369124A (en) | Image aesthetic prediction method based on self-generation global features and attention |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191203 |