CN107958287A - Towards the confrontation transfer learning method and system of big data analysis transboundary - Google Patents
Towards the confrontation transfer learning method and system of big data analysis transboundary Download PDFInfo
- Publication number
- CN107958287A CN107958287A CN201711183974.8A CN201711183974A CN107958287A CN 107958287 A CN107958287 A CN 107958287A CN 201711183974 A CN201711183974 A CN 201711183974A CN 107958287 A CN107958287 A CN 107958287A
- Authority
- CN
- China
- Prior art keywords
- data
- tensor
- loss function
- predetermined depth
- neutral net
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Machine Translation (AREA)
Abstract
The present invention provides a kind of confrontation transfer learning method and system towards big data analysis transboundary.This method includes：The source domain and the respective unlabeled data collection of target domain are corresponded to the corresponding random polyteny fusion of each tensor in tensor set to represent to substitute into the primary loss function of discriminator, obtain the current loss function of discriminator, and utilize backpropagation, adjust the parameter of the discriminator, to minimize the current loss function, the current optimal loss function as the discriminator；Tensor is the tensor product of the data vector of all data Layers in preset data layer set in the predetermined depth neutral net in the tensor set；The predetermined depth neural network parameter is updated based on the current optimal loss function and enters the renewal of the predetermined depth neural network parameter next time until parameter restrains.The Joint Distribution of multiple data Layers, which deviates, in the predetermined depth neutral net preset data layer set obtained by the present invention reduces, preferable applied to target domain effect.
Description
Technical field
The present invention relates to data analysis technique field, more particularly, to a kind of confrontation towards big data analysis transboundary
Transfer learning method and system.
Background technology
In numerous machine learning task processing, deep neural network method is current effect the best way.It is but deep
Degree neutral net only obtain it is enough abundant have label data after, could obtain good by supervised learning training
Business effect.Preferably it is used to complete target in the case that labeled data is less, remain to obtain effect in target domain
The deep neural network of task, the cross-cutting study of generally use, the labeled data that source domain is enriched are used for target domain
The acquisition of deep neural network.Under the deep neural network obtained based on the labeled data that source domain enriches, the number of source domain
According to the data with target domain there are the problem of distributions shift, thus the deep neural network application target field is completed target and is appointed
It is ineffective during business.
For this problem, generally use transfer learning method solves, that is, trains a discriminator to be used for percentage regulation god
Parameter through network so that under the deep neural network after parameter adjustment, between the data of source domain and the data of target domain
Distributions shift reduces, so that deep neural network application target field completes have preferable effect during goal task.Wherein, resist
Transfer learning method is one of best transfer learning method of current effect, it passes through the list according to deep neural network intermediate layer
The data vector of a data Layer, builds the loss function of discriminator, and the loss function for minimizing the discriminator obtains the discriminator
Parameter, and fix the parameter of the loss function of the discriminator, minimize the loss function and the discriminator of deep neural network
Loss function difference, obtain the continuous percentage regulation neutral net of this mode of the parameter of deep neural network parameter until receive
Hold back.
By resisting transfer learning method, under some data Layers of the deep neural network top layer after parameter adjustment, source neck
The data in domain and the data of target domain may possibly still be present data distribution offset, and then deep neural network application target field
Completing effect during goal task may be bad.Especially, when the complexity of multi-mode is presented in the data distribution of source domain and target domain
During structure, according to the data vector of the single data layer in deep neural network intermediate layer, the loss function for building discriminator is used for
The parameter of percentage regulation neutral net, deep neural network after parameter adjustment to be likely difficult to catch numerous and diverse data distribution special
Sign is alignd with that will be distributed fine granularity so that the offset of the data distribution of source domain and target domain is still larger, and deep neural network should
It is ineffective during with target domain completion goal task.
The content of the invention
The present invention provides a kind of confrontation transfer learning method and system towards big data analysis transboundary, existing right to overcome
Under some data Layers for the deep neural network top layer that anti-migration learning method obtains, the data of source domain and the number of target domain
Deviated according to may possibly still be present data distribution, and when the complicated knot of multi-mode is presented in the data distribution of source domain and target domain
During structure, obtain deep neural network and be likely difficult to catch numerous and diverse data distribution characteristics so that distribution fine granularity to be alignd so that source
The offset of the data distribution of field and target domain is still larger, and the problem of ineffective during goal task is completed in application target field.
According to the first aspect of the invention, there is provided a kind of confrontation transfer learning method towards big data analysis transboundary, should
Method includes：Step 1, source domain and the respective unlabeled data collection of target domain inputted to predetermined depth neutral net and just
To propagation, the corresponding tensor set of the respective unlabeled data collection of the source domain and target domain is obtained；Opened in the tensor set
When measuring as corresponding unlabeled data as input, all data Layers in preset data layer set in the predetermined depth neutral net
Data vector tensor product；Step 2, by the source domain and the corresponding tensor set of the respective unlabeled data collection of target domain
In the corresponding random polyteny fusion of each tensor represent, substitute into the primary loss function of discriminator, obtain the current of discriminator
Loss function, and using the parameter of the backpropagation adjustment discriminator, to minimize the current loss function, as described
The current optimal loss function of discriminator；Step 3, using backpropagation, the predetermined depth neutral net is led in the source
The loss function in domain subtracts balance parameters with being minimized after the product of the current optimal loss function, obtains the predetermined depth
The new parameter of neutral net, updates the parameter of the predetermined depth neutral net with the new parameter and carries out the forward direction again
Propagate to update the parameter of the predetermined depth neutral net again, until parameter restrains；The balance parameters are described pre-
If deep neural network is in the balance parameters of the loss function and the current optimal loss function of the source domain.
Wherein, the step 1 specifically includes：Step 11, by the source domain and the respective unlabeled data of target domain
The each unlabeled data concentrated is sequentially input to predetermined depth neutral net and forward-propagating, and acquisition is described each not to mark number
According to the data vector of each data Layer in preset data layer set in the lower predetermined depth neutral net；Step 12, institute is calculated
The tensor product of data vector is stated, using the tensor product as the corresponding tensor of each unlabeled data；Step 13, according to institute
The unlabeled data for stating source domain concentrates the corresponding tensor of each unlabeled data, obtains the unlabeled data collection of the source domain
Corresponding tensor set, and the corresponding tensor of each unlabeled data is concentrated according to the unlabeled data of the target domain, obtain
The corresponding tensor set of unlabeled data collection of the target domain.
Wherein, in step 1, the preset data layer set is by the predetermined depth neutral net top layer and intermediate layer
Some data Layers form.
Wherein, in step 2, the discriminator is that an input is d dimensional vectors, output connecting entirely on section [0,1]
Connect predetermined depth neutral net discriminator；The primary loss function sets of the discriminator are：
Wherein,WithThe unlabeled data collection of respectively described source domain and the target domain is corresponding
Tensor set,WithThe tensor of i-th of unlabeled data of respectively described source domain and the target domain, nsAnd ntRespectively
The number of unlabeled data is concentrated for the unlabeled data of the source domain and the target domain,WithRespectively
Merge and represent for the random polyteny of the source domain and the tensor of i-th of unlabeled data of the target domain；To be described
Preset data layer set,Z is tensor, and ⊙ is Hadamard product,It is that a dimension is
Random matrix,It is data LayerDimension, d be discriminator input vector dimension,For data LayerData vector.
Wherein, in step 3, the predetermined depth neutral net is to be led in the source in the loss function of the source domain
The labeled data in domain concentrates the average of the intersection entropy loss of all labeled data.
Wherein, the predetermined depth neutral net is set as in the loss function of the source domain：
Wherein, nsThe number of labeled data is concentrated for the labeled data of the source domain,For the source domain
The feature vector of i-th of labeled data,For the decision function of the predetermined depth neutral net,Led for the source
The label of the labeled data of i-th of domain, J () is cross entropy loss function.
According to the second aspect of the invention, there is provided a kind of confrontation transfer learning system towards big data analysis transboundary, its
It is characterized in that, including：The current optimal loss function acquisition module and update module of tensor set acquisition module and discriminator；It is described
Tensor set acquisition module, for inputting the unlabeled data collection of the unlabeled data collection of source domain and target domain to default depth
Spend neutral net and forward-propagating, obtain the corresponding tensor set of unlabeled data collection of the source domain and the target domain
The corresponding tensor set of unlabeled data collection；It is described default when tensor is that corresponding unlabeled data is used as input in the tensor set
In deep neural network in preset data layer set the data vector of all data Layers tensor product；The discriminator it is current most
Good loss function acquisition module, for by the corresponding tensor set of the unlabeled data collection of the source domain and the target domain
The corresponding fusion of polyteny at random of each tensor represents to substitute into the primary loss function of discriminator, obtains the current of discriminator and lose
Function, and backpropagation is utilized, the parameter of the discriminator is adjusted, to minimize the current loss function, as the mirror
The current optimal loss function of other device；The update module, for utilizing the backpropagation, minimizes the predetermined depth god
Through network the source domain loss function and the current optimal loss function tradeoff difference, obtain the default depth
Spend the new parameter of neutral net, with the new parameter update the parameter of the predetermined depth neutral net and carry out again it is described just
To propagating to update the parameter of the predetermined depth neutral net again, until parameter restrains；The current optimal loss letter
Several tradeoffs is loss function of the predetermined depth neutral net in the source domain and the current optimal loss function
Balance parameters be multiplied by the current optimal loss function.
According to the third aspect of the invention we, there is provided a kind of computer program product, it is characterised in that the computer program
Product includes the computer program being stored on non-transient computer readable storage medium storing program for executing, and the computer program refers to including program
Order, when described program instruction is computer-executed, makes the method for the computer execution as described in relation to the first aspect.
According to the fourth aspect of the invention, there is provided a kind of non-transient computer readable storage medium storing program for executing, it is characterised in that described
Non-transient computer readable storage medium storing program for executing stores computer instruction, and the computer instruction makes the computer perform such as first party
Method described in face.
Confrontation transfer learning method and system proposed by the present invention towards big data analysis transboundary, by by predetermined depth
In neutral net in preset data layer set the tensor product of the data vector of all data Layers as the corresponding tensor of input data,
Each tensor in the corresponding tensor set of the unlabeled data collection of the source domain and the target domain is corresponding random multi-thread
Property fusion represent substitute into discriminator primary loss function, obtain the current loss function of discriminator, and then be based on the discriminator
Current loss function the parameter of predetermined depth neutral net is updated so that obtained predetermined depth neutral net is preset
In data Layer set multiple data Layers Joint Distribution offset reduce, applied to target domain complete goal task when effect compared with
It is good, meanwhile, tensor is represented using the fusion of random polyteny, follow-up data calculation amount is substantially reduced, accelerates to anti-migration
Learning process.If in addition, the invention avoids the predetermined depth neutral net top layer obtained based on existing transfer learning method
The data distribution offset that dry data Layer may still suffer from, meanwhile, it also avoid when the data distribution of source domain and target domain is in
During the labyrinth of existing multi-mode, the predetermined depth neutral net that is obtained based on existing transfer learning method is difficult to catch numerous and diverse
Data distribution characteristics cause data distribution offset larger, complete target applied to target domain and appoint so that distribution fine granularity to be alignd
It is ineffective during business.
Brief description of the drawings
Fig. 1 is a kind of confrontation transfer learning method flow towards big data analysis transboundary according to the embodiment of the present invention
Figure；
Fig. 2 is to be worked as according to discriminator under the respective unlabeled data collection of source domain and target domain of the embodiment of the present invention
The acquisition flow chart of preceding loss function；
Fig. 3 is a kind of confrontation transfer learning system flow towards big data analysis transboundary according to the embodiment of the present invention
Figure.
Embodiment
With reference to the accompanying drawings and examples, the embodiment of the present invention is described in further detail.Implement below
Example is used to illustrate the present invention, but is not limited to the scope of the present invention.
As shown in Figure 1, the embodiment of the present invention provides a kind of confrontation transfer learning method towards big data analysis transboundary, should
Method includes：
Step 1, source domain and the respective unlabeled data collection of target domain inputted to predetermined depth neutral net and just
To propagation, the corresponding tensor set of the respective unlabeled data collection of the source domain and target domain is obtained；Opened in the tensor set
When measuring as corresponding unlabeled data as input, all data Layers in preset data layer set in the predetermined depth neutral net
Data vector tensor product.
In the present embodiment, predetermined depth neutral net can be applied to numerous machine learning tasks, such as:Image recognition and language
Sound identification etc..By taking non-supervisory image recognition transfer learning task as an example, predetermined depth can be realized under Caffe deep learning frames
Neutral net AlexNet or ResNet, can be before transfer learning be carried out first in order to improve the performance of predetermined depth neutral net
Pre-training is carried out to predetermined depth neutral net using the ImageNet data sets marked；The unlabeled data collection of target domain
For the set of eigenvectors of image to be classified.
In the present embodiment, specifically, do not marked what source domain and the respective unlabeled data of target domain were concentrated each
Note data are sequentially input to predetermined depth neutral net and forward-propagating, obtain preset data in the predetermined depth neutral net
The data vector of each data Layer in layer set.The tensor product of the data vector is calculated, the tensor product is as input
The corresponding tensor of unlabeled data.The corresponding tensor of each unlabeled data is concentrated according to the unlabeled data of the source domain,
The corresponding tensor set of unlabeled data collection of the source domain is obtained, and is concentrated often according to the unlabeled data of the target domain
The corresponding tensor of a unlabeled data, obtains the corresponding tensor set of unlabeled data collection of the target domain.Wherein, it is described pre-
If data Layer set is made of some data Layers in the predetermined depth neutral net top layer and intermediate layer.For example, when default
When deep neural network is AlexNet, preset data layer set is setWhen predetermined depth nerve net
When network is ResNet, preset data layer set is set
Inputted according to by source domain and the respective unlabeled data collection of target domain to predetermined depth neutral net and forward direction
Propagate, obtain the respective unlabeled data collection of the source domain and target domain under each data Layer of preset data layer set
The process of data vector is as shown in Figure 2.Wherein, XsFor the unlabeled data collection of source domain, XtFor the unlabeled data of target domain
Collection, ZsiFor data vector of the unlabeled data collection under i-th of data Layer of preset data layer set of source domain, ZtiFor mesh
Data vector of the unlabeled data collection in mark field under i-th of data Layer of preset data layer set.
Step 2, by each tensor in the source domain and the corresponding tensor set of the respective unlabeled data collection of target domain
Corresponding random polyteny fusion represents, substitutes into the primary loss function of discriminator, obtains the current loss function of discriminator, and
The parameter of the discriminator is adjusted using backpropagation, to minimize the current loss function, as working as the discriminator
Preceding optimal loss function.
In the present embodiment, the discriminator is that an input is the full connection of d dimensional vectors, output on section [0,1]
Predetermined depth neutral net discriminator.The primary loss function sets of the discriminator are：
Wherein,WithThe unlabeled data collection of respectively described source domain and the target domain is corresponding
Tensor set,WithThe tensor of i-th of unlabeled data of respectively described source domain and the target domain, nsAnd ntRespectively
The number of unlabeled data is concentrated for the unlabeled data of the source domain and the target domain,WithRespectively
Merge and represent for the random polyteny of the source domain and the tensor of i-th of unlabeled data of the target domain；For institute
Preset data layer set is stated,Z is tensor, and ⊙ is Hadamard product,It is that a dimension isRandom matrix,It is data LayerDimension, d be discriminator input vector dimension,For data LayerData
Vector.For random matrixEach matrix elementDistribution need to meet
Available matrix element distribution includes Gaussian Profile, Bernoulli Jacob's distribution and Uniformly distributed.
The respective unlabeled data collection of source domain and target domain is under each data Layer of preset data layer set
Tensor, substitutes into the primary loss function of discriminator, and the process for obtaining the current loss function of discriminator is as shown in Figure 2.Wherein, Ri
For the corresponding random matrix of i-th of data Layer of preset data layer set, ⊙ is Hadamard product, and D is the current loss of discriminator
Function.YsFor the corresponding output collection of unlabeled data collection of source domain, YtFor the corresponding output of unlabeled data collection of target domain
Collection.
In the present embodiment, after the current loss function for obtaining discriminator, the discriminator is adjusted using backpropagation
Parameter, to minimize the current loss function, uses small during the current optimal loss function as the discriminator
Momentum is simultaneously set to 0.9 by batch stochastic gradient descent.In each round iteration of stochastic gradient descent, learning rate isWherein, parameter p linearly rises to 1, η with iteration wheel number from 00=0.01, α=10, β=0.75.
Step 3, using backpropagation, the predetermined depth neutral net is subtracted in the loss function of the source domain flat
Weighing apparatus parameter with minimized after the current most preferably product of loss function, obtain the new parameter of the predetermined depth neutral net,
The parameter of the predetermined depth neutral net is updated with the new parameter and carries out the forward-propagating again to update again
The parameter of the predetermined depth neutral net, until parameter restrains；The balance parameters exist for the predetermined depth neutral net
The balance parameters of the loss function of the source domain and the current optimal loss function.
In the present embodiment, the predetermined depth neutral net is in the source domain in the loss function of the source domain
Labeled data concentrate all labeled data intersection entropy loss average, the predetermined depth neutral net is in the source
The loss function in field is set as：
Wherein, nsThe number of labeled data is concentrated for the labeled data of the source domain,For the source domain
The feature vector of i-th of labeled data,For the decision function of the predetermined depth neutral net,Led for the source
The label of the labeled data of i-th of domain, J () is cross entropy loss function.
In the present embodiment, the predetermined depth neutral net the loss function of the source domain subtract balance parameters with
The product of the current optimal loss function is：
Wherein, λ is loss function and the current optimal loss of the predetermined depth neutral net in the source domain
The balance parameters of function.
In the present embodiment, using the backpropagation, the predetermined depth neutral net is minimized in the source domain
Loss function and the current optimal loss function tradeoff difference during, using small lot stochastic gradient descent simultaneously
Momentum is set to 0.9.In each round iteration of stochastic gradient descent, learning rate isWherein, parameter p
As iteration wheel number linearly rises to 1, η from 00=0.01, α=10, β=0.75.Balance parameters λ is again set at gradual change ginseng
Number,Wherein δ=10.In the process, it specifically have adjusted predetermined depth neutral net F's (x)
All convolved data layers, pond data Layer and classifier data layer, and the learning rate of classifier data layer is other data Layers
10 times.
Confrontation transfer learning method proposed by the present invention towards big data analysis transboundary, by by predetermined depth nerve net
The tensor product of the data vector of all data Layers is as the corresponding tensor of input data in preset data layer set in network, by described in
The corresponding random polyteny fusion of each tensor in the corresponding tensor set of unlabeled data collection of source domain and the target domain
Represent the primary loss function of substitution discriminator, obtain the current loss function of discriminator, and then based on the current of the discriminator
Loss function is updated the parameter of predetermined depth neutral net so that obtained predetermined depth neutral net preset data layer
In set multiple data Layers Joint Distribution offset reduce, applied to target domain complete goal task when effect it is preferable, meanwhile,
Tensor is represented using the fusion of random polyteny, follow-up data calculation amount is substantially reduced, accelerates confrontation transfer learning process.
In addition, some data Layers the invention avoids the predetermined depth neutral net top layer obtained based on existing transfer learning method can
The data distribution offset that can be still suffered from, meanwhile, it also avoid when multi-mode is presented in the data distribution of source domain and target domain
During labyrinth, the predetermined depth neutral net obtained based on existing transfer learning method is difficult to catch numerous and diverse data distribution spy
Sign with will distribution fine granularity alignment, cause data distribution offset it is larger, applied to target domain complete goal task when effect not
It is good.
As shown in figure 3, the embodiment of the present invention provides a kind of confrontation transfer learning system towards big data analysis transboundary, bag
Include：The current optimal loss function acquisition module and update module of tensor set acquisition module and discriminator；The tensor set obtains
Module, for inputting the unlabeled data collection of the unlabeled data collection of source domain and target domain to predetermined depth neutral net
And forward-propagating, obtain the corresponding tensor set of unlabeled data collection of the source domain and the unlabeled data of the target domain
Collect corresponding tensor set；When tensor is that corresponding unlabeled data is used as input in the tensor set, the predetermined depth nerve net
In network in preset data layer set the data vector of all data Layers tensor product；The current optimal loss function of the discriminator
Acquisition module, for by each tensor pair in the corresponding tensor set of the unlabeled data collection of the source domain and the target domain
The random polyteny fusion answered represents the primary loss function of substitution discriminator, obtains the current loss function of discriminator, and profit
With backpropagation, the parameter of the discriminator is adjusted, to minimize the current loss function, as the current of the discriminator
Optimal loss function；The update module, for utilizing the backpropagation, minimizes the predetermined depth neutral net in institute
The difference of the loss function of source domain and the tradeoff of the current optimal loss function is stated, obtains the predetermined depth neutral net
New parameter, update the parameter of the predetermined depth neutral net with the new parameter and carry out the forward-propagating again with again
The parameter of the predetermined depth neutral net is once updated, until parameter restrains；The tradeoff of the current optimal loss function
It is balance parameters of the predetermined depth neutral net in the loss function and the current optimal loss function of the source domain
It is multiplied by the current optimal loss function.
Confrontation transfer learning system proposed by the present invention towards big data analysis transboundary, by by predetermined depth nerve net
The tensor product of the data vector of all data Layers is as the corresponding tensor of input data in preset data layer set in network, by described in
The corresponding random polyteny fusion of each tensor in the corresponding tensor set of unlabeled data collection of source domain and the target domain
Represent the primary loss function of substitution discriminator, obtain the current loss function of discriminator, and then based on the current of the discriminator
Loss function is updated the parameter of predetermined depth neutral net so that obtained predetermined depth neutral net preset data layer
In set multiple data Layers Joint Distribution offset reduce, applied to target domain complete goal task when effect it is preferable, meanwhile,
Tensor is represented using the fusion of random polyteny, follow-up data calculation amount is substantially reduced, accelerates confrontation transfer learning process.
In addition, some data Layers the invention avoids the predetermined depth neutral net top layer obtained based on existing transfer learning method can
The data distribution offset that can be still suffered from, meanwhile, it also avoid when multi-mode is presented in the data distribution of source domain and target domain
During labyrinth, the predetermined depth neutral net obtained based on existing transfer learning method is difficult to catch numerous and diverse data distribution spy
Sign with will distribution fine granularity alignment, cause data distribution offset it is larger, applied to target domain complete goal task when effect not
It is good.
The embodiment of the present invention provides a kind of computer program product, and the computer program product includes being stored in non-transient
Computer program on computer-readable recording medium, the computer program include programmed instruction, when described program instructs quilt
When computer performs, computer is able to carry out the method that above method embodiment is provided, such as including：By not marking for source domain
The unlabeled data collection of note data set and target domain is inputted to predetermined depth neutral net and forward-propagating, obtains the source neck
The corresponding tensor set of unlabeled data collection in domain and the corresponding tensor set of unlabeled data collection of the target domain；The tensor
It is when corresponding to unlabeled data as input to concentrate tensor, is owned in the predetermined depth neutral net in preset data layer set
The tensor product of the data vector of data Layer；By the corresponding tensor set of the unlabeled data collection of the source domain and the target domain
In the corresponding random polyteny fusion of each tensor represent to substitute into the primary loss function of discriminator, obtain the current damage of discriminator
Function is lost, and utilizes backpropagation, the parameter of the discriminator is adjusted, to minimize the current loss function, as described
The current optimal loss function of discriminator；Using the backpropagation, the predetermined depth neutral net is minimized in the source
The difference of the tradeoff of the loss function in field and the current optimal loss function, obtains the new of the predetermined depth neutral net
Parameter, updates the parameter of the predetermined depth neutral net with the new parameter and carries out the forward-propagating again with again
The parameter of the predetermined depth neutral net is updated, until parameter restrains；The tradeoff of the current optimal loss function is institute
The balance parameters that predetermined depth neutral net is stated in the loss function and the current optimal loss function of the source domain are multiplied by
The current optimal loss function.
The embodiment of the present invention provides a kind of non-transient computer readable storage medium storing program for executing, the non-transient computer readable storage
Medium storing computer instructs, and the computer instruction makes the computer perform the method that above method embodiment is provided,
Such as including：The unlabeled data collection of the unlabeled data collection of source domain and target domain is inputted to predetermined depth neutral net
And forward-propagating, obtain the corresponding tensor set of unlabeled data collection of the source domain and the unlabeled data of the target domain
Collect corresponding tensor set；When tensor is that corresponding unlabeled data is used as input in the tensor set, the predetermined depth nerve net
In network in preset data layer set the data vector of all data Layers tensor product；By the source domain and the target domain
The corresponding random polyteny fusion of each tensor represents to substitute into the original damage of discriminator in the corresponding tensor set of unlabeled data collection
Function is lost, obtains the current loss function of discriminator, and utilizes backpropagation, adjusts the parameter of the discriminator, to minimize
The current loss function, the current optimal loss function as the discriminator；Using the backpropagation, described in minimum
Predetermined depth neutral net the source domain loss function and the current optimal loss function tradeoff difference, obtain
The new parameter of the predetermined depth neutral net, the parameter and again of the predetermined depth neutral net is updated with the new parameter
The forward-propagating is carried out to update the parameter of the predetermined depth neutral net again, until parameter restrains；It is described current
The tradeoff of optimal loss function for the predetermined depth neutral net the source domain loss function with it is described it is current most
The balance parameters of good loss function are multiplied by the current optimal loss function.
One of ordinary skill in the art will appreciate that：Realizing all or part of step of above method embodiment can pass through
The relevant hardware of programmed instruction is completed, and foregoing program can be stored in a computer read/write memory medium, the program
Upon execution, the step of execution includes above method embodiment；And foregoing storage medium includes：ROM, RAM, magnetic disc or light
Disk etc. is various can be with the medium of store program codes.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment can
Realized by the mode of software plus required general hardware platform, naturally it is also possible to pass through hardware.Based on such understanding, on
The part that technical solution substantially in other words contributes to the prior art is stated to embody in the form of software product, should
Computer software product can store in a computer-readable storage medium, such as ROM/RAM, magnetic disc, CD, including some fingers
Order is used so that a computer equipment (can be personal computer, server, or network equipment etc.) performs each implementation
Method described in some parts of example or embodiment.
Finally it should be noted that：The above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although
The present invention is described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that：It still may be used
To modify to the technical solution described in foregoing embodiments, or equivalent substitution is carried out to which part technical characteristic；
And these modification or replace, do not make appropriate technical solution essence depart from various embodiments of the present invention technical solution spirit and
Scope.
Claims (9)
- A kind of 1. confrontation transfer learning method towards big data analysis transboundary, it is characterised in that including：Step 1, source domain and the respective unlabeled data collection of target domain are inputted to predetermined depth neutral net and positive biography Broadcast, obtain the corresponding tensor set of the respective unlabeled data collection of the source domain and target domain；Tensor is in the tensor set When corresponding unlabeled data is as input, in the predetermined depth neutral net in preset data layer set all data Layers number According to the tensor product of vector；Step 2, each tensor in the source domain and the corresponding tensor set of the respective unlabeled data collection of target domain is corresponded to Random polyteny fusion represent, substitute into the primary loss function of discriminator, obtain the current loss function of discriminator, and utilize Backpropagation adjusts the parameter of the discriminator, to minimize the current loss function, as the discriminator it is current most Good loss function；Step 3, using backpropagation, the predetermined depth neutral net is subtracted into balance ginseng in the loss function of the source domain Number obtains the new parameter of the predetermined depth neutral net, uses institute with being minimized after the product of the current optimal loss function State that new parameter updates the parameter of the predetermined depth neutral net and to carry out the forward-propagating again described to update again The parameter of predetermined depth neutral net, until parameter restrains；The balance parameters are the predetermined depth neutral net described The balance parameters of the loss function of source domain and the current optimal loss function.
- 2. according to the method described in claim 1, it is characterized in that, the step 1 specifically includes：Step 11, each unlabeled data that the source domain and the respective unlabeled data of target domain are concentrated is sequentially input To predetermined depth neutral net and forward-propagating, obtain pre- in the predetermined depth neutral net under each unlabeled data If the data vector of each data Layer in data Layer set；Step 12, the tensor product of the data vector is calculated, the tensor product is corresponding as each unlabeled data Tensor；Step 13, the corresponding tensor of each unlabeled data is concentrated according to the unlabeled data of the source domain, obtains the source The corresponding tensor set of unlabeled data collection in field, and concentrated according to the unlabeled data of the target domain and each do not mark number According to corresponding tensor, the corresponding tensor set of unlabeled data collection of the target domain is obtained.
- 3. according to the method described in claim 1, it is characterized in that, in step 1, the preset data layer set is by described pre- If some data Layers in deep neural network top layer and intermediate layer are formed.
- 4. according to the method described in claim 1, it is characterized in that, in step 2, the discriminator is that an input is d dimensions The full connection predetermined depth neutral net discriminator of vector, output on section [0,1]；The primary loss function sets of the discriminator are：Wherein,WithThe corresponding tensor of unlabeled data collection of respectively described source domain and the target domain Collection,WithThe tensor of i-th of unlabeled data of respectively described source domain and the target domain, nsAnd ntRespectively institute The number of the unlabeled data concentration unlabeled data of source domain and the target domain is stated,WithRespectively institute The random polyteny fusion for stating the tensor of i-th of unlabeled data of source domain and the target domain represents；To be described pre- If data Layer set,Z is tensor, and ⊙ is that Hadamard accumulates, RlIt is that a dimension is d × dl's Random matrix, dlIt is the dimension of data Layer l, d is the dimension of discriminator input vector, zlFor the data vector of data Layer l.
- 5. according to the method described in claim 1, it is characterized in that, in step 3, the predetermined depth neutral net is described The loss function of source domain is in the equal of the intersection entropy loss of all labeled data of the labeled data of source domain concentration Value.
- 6. according to the method described in claim 5, it is characterized in that, the predetermined depth neutral net the source domain damage Losing function sets is：<mrow> <mfrac> <mn>1</mn> <msub> <mi>n</mi> <mi>s</mi> </msub> </mfrac> <msubsup> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <msub> <mi>n</mi> <mi>s</mi> </msub> </msubsup> <mi>J</mi> <mrow> <mo>(</mo> <mi>F</mi> <mo>(</mo> <msubsup> <mi>x</mi> <mi>i</mi> <mi>s</mi> </msubsup> <mo>)</mo> <mo>,</mo> <msubsup> <mi>y</mi> <mi>i</mi> <mi>s</mi> </msubsup> <mo>)</mo> </mrow> <mo>,</mo> </mrow>Wherein, nsThe number of labeled data is concentrated for the labeled data of the source domain,For i-th of the source domain The feature vector of labeled data,For the decision function of the predetermined depth neutral net,For the source domain The label of i-th of labeled data, J () is cross entropy loss function.
- A kind of 7. confrontation transfer learning system towards big data analysis transboundary, it is characterised in that including：Tensor set acquisition module With the current optimal loss function acquisition module and update module of discriminator；The tensor set acquisition module, for the unlabeled data collection of the unlabeled data collection of source domain and target domain to be inputted To predetermined depth neutral net and forward-propagating, the corresponding tensor set of unlabeled data collection of the source domain and the mesh are obtained The corresponding tensor set of unlabeled data collection in mark field；When tensor is that corresponding unlabeled data is used as input in the tensor set, In the predetermined depth neutral net in preset data layer set the data vector of all data Layers tensor product；The current optimal loss function acquisition module of the discriminator, for not marking the source domain and the target domain The corresponding fusion of polyteny at random of each tensor in the corresponding tensor set of data set is noted to represent to substitute into the primary loss letter of discriminator Number, obtains the current loss function of discriminator, and utilizes backpropagation, the parameter of the discriminator is adjusted, with described in minimum Current loss function, the current optimal loss function as the discriminator；The update module, for utilizing the backpropagation, minimizes the predetermined depth neutral net in the source domain Loss function and the current optimal loss function tradeoff difference, obtain the new ginseng of the predetermined depth neutral net Number, updates the parameter of the predetermined depth neutral net with the new parameter and carries out the forward-propagating again with again more The parameter of the new predetermined depth neutral net, until parameter restrains；The tradeoff of the current optimal loss function is described Predetermined depth neutral net is multiplied by institute in the balance parameters of the loss function and the current optimal loss function of the source domain State current optimal loss function.
- 8. a kind of computer program product, it is characterised in that the computer program product includes being stored in non-transient computer Computer program on readable storage medium storing program for executing, the computer program include programmed instruction, when described program is instructed by computer During execution, the computer is set to perform such as claim 1 to 6 any one of them method.
- 9. a kind of non-transient computer readable storage medium storing program for executing, it is characterised in that the non-transient computer readable storage medium storing program for executing is deposited Computer instruction is stored up, the computer instruction makes the computer perform such as claim 1 to 6 any one of them method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711183974.8A CN107958287A (en) | 2017-11-23 | 2017-11-23 | Towards the confrontation transfer learning method and system of big data analysis transboundary |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711183974.8A CN107958287A (en) | 2017-11-23 | 2017-11-23 | Towards the confrontation transfer learning method and system of big data analysis transboundary |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107958287A true CN107958287A (en) | 2018-04-24 |
Family
ID=61961754
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711183974.8A Pending CN107958287A (en) | 2017-11-23 | 2017-11-23 | Towards the confrontation transfer learning method and system of big data analysis transboundary |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107958287A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109165562A (en) * | 2018-07-27 | 2019-01-08 | 深圳市商汤科技有限公司 | Training method, crosswise joint method, apparatus, equipment and the medium of neural network |
| CN109948645A (en) * | 2019-01-23 | 2019-06-28 | 西安交通大学 | A kind of enterprise based on depth confrontation transfer learning evades the tax recognition methods |
| CN110234085A (en) * | 2019-05-23 | 2019-09-13 | 深圳大学 | Based on the indoor location fingerprint to anti-migration network drawing generating method and system |
| CN110489661A (en) * | 2019-07-24 | 2019-11-22 | 武汉大学 | A kind of social networks prediction technique based on generation confrontation network and transfer learning |
| WO2019228358A1 (en) * | 2018-05-31 | 2019-12-05 | 华为技术有限公司 | Deep neural network training method and apparatus |
| CN110751207A (en) * | 2019-10-18 | 2020-02-04 | 四川大学 | Fault diagnosis method for anti-migration learning based on deep convolution domain |
| CN110827258A (en) * | 2019-10-31 | 2020-02-21 | 西安交通大学 | Screening device and method for diabetic retinopathy based on semi-supervised antagonistic learning |
| CN111488979A (en) * | 2019-01-28 | 2020-08-04 | 斯特拉德视觉公司 | Method and apparatus for continuously learning neural network for analyzing input data on device |
| CN112313044A (en) * | 2018-06-15 | 2021-02-02 | 谷歌有限责任公司 | Deep reinforcement learning for robotic manipulation |
| CN113869342A (en) * | 2020-06-30 | 2021-12-31 | 微软技术许可有限责任公司 | Mark offset detection and adjustment in predictive modeling |
-
2017
- 2017-11-23 CN CN201711183974.8A patent/CN107958287A/en active Pending
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019228358A1 (en) * | 2018-05-31 | 2019-12-05 | 华为技术有限公司 | Deep neural network training method and apparatus |
| CN112313044A (en) * | 2018-06-15 | 2021-02-02 | 谷歌有限责任公司 | Deep reinforcement learning for robotic manipulation |
| CN109165562A (en) * | 2018-07-27 | 2019-01-08 | 深圳市商汤科技有限公司 | Training method, crosswise joint method, apparatus, equipment and the medium of neural network |
| CN109165562B (en) * | 2018-07-27 | 2021-06-04 | 深圳市商汤科技有限公司 | Neural network training method, lateral control method, device, equipment and medium |
| CN109948645A (en) * | 2019-01-23 | 2019-06-28 | 西安交通大学 | A kind of enterprise based on depth confrontation transfer learning evades the tax recognition methods |
| CN111488979B (en) * | 2019-01-28 | 2023-11-07 | 斯特拉德视觉公司 | Method and device for continuously learning neural network for analyzing input data on equipment |
| CN111488979A (en) * | 2019-01-28 | 2020-08-04 | 斯特拉德视觉公司 | Method and apparatus for continuously learning neural network for analyzing input data on device |
| CN110234085A (en) * | 2019-05-23 | 2019-09-13 | 深圳大学 | Based on the indoor location fingerprint to anti-migration network drawing generating method and system |
| CN110489661A (en) * | 2019-07-24 | 2019-11-22 | 武汉大学 | A kind of social networks prediction technique based on generation confrontation network and transfer learning |
| CN110751207B (en) * | 2019-10-18 | 2022-08-05 | 四川大学 | Fault diagnosis method for anti-migration learning based on deep convolution domain |
| CN110751207A (en) * | 2019-10-18 | 2020-02-04 | 四川大学 | Fault diagnosis method for anti-migration learning based on deep convolution domain |
| CN110827258A (en) * | 2019-10-31 | 2020-02-21 | 西安交通大学 | Screening device and method for diabetic retinopathy based on semi-supervised antagonistic learning |
| CN110827258B (en) * | 2019-10-31 | 2022-02-11 | 西安交通大学 | Device and method for screening diabetic retinopathy based on counterstudy |
| CN113869342A (en) * | 2020-06-30 | 2021-12-31 | 微软技术许可有限责任公司 | Mark offset detection and adjustment in predictive modeling |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107958287A (en) | Towards the confrontation transfer learning method and system of big data analysis transboundary | |
| CN109960759B (en) | Recommendation system click rate prediction method based on deep neural network | |
| US11551098B2 (en) | Generative adversarial network-based optimization method and application | |
| EP3570222B1 (en) | Information processing device and method, and computer readable storage medium | |
| Pietrzak et al. | Framework of Strategic Learning: The PDCA Cycle. | |
| Chandra et al. | DIVACE: Diverse and accurate ensemble learning algorithm | |
| CN107766794A (en) | The image, semantic dividing method that a kind of Fusion Features coefficient can learn | |
| CN104933428B (en) | A kind of face identification method and device based on tensor description | |
| CN110168572A (en) | Information processing method, information processing unit, computer readable storage medium | |
| CN106203628A (en) | A kind of optimization method strengthening degree of depth learning algorithm robustness and system | |
| US11526733B2 (en) | Neural network execution block and transfer learning | |
| CN112560991A (en) | Personalized federal learning method based on hybrid expert model | |
| CN107665339A (en) | A kind of method changed by neural fusion face character | |
| Pal et al. | Selection of optimal set of weights in a layered network using genetic algorithms | |
| Whitelam et al. | Training neural networks using Metropolis Monte Carlo and an adaptive variant | |
| Liu et al. | Adaptive transfer learning for PINN | |
| CN109948589A (en) | Facial expression recognizing method based on quantum deepness belief network | |
| Nam et al. | Predicting airline passenger volume | |
| CN109117946A (en) | Neural computing handles model | |
| CN112884160A (en) | Meta learning method and related device | |
| Lacko | From perceptrons to deep neural networks | |
| Didandeh et al. | AVLR-EBP: a variable step size approach to speed-up the convergence of error back-propagation algorithm | |
| Toro-Zapata et al. | Conditions on the energy market diversification from adaptive dynamics | |
| Siu | Automatic induction of neural network decision tree algorithms | |
| Abrahamson | A nested decomposition approach for solving staircase structured linear programs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180424 |
|
| RJ01 | Rejection of invention patent application after publication |