CN107609566A - Dual network deep learning method based on a small amount of personalized sample - Google Patents
Dual network deep learning method based on a small amount of personalized sample Download PDFInfo
- Publication number
- CN107609566A CN107609566A CN201610546022.7A CN201610546022A CN107609566A CN 107609566 A CN107609566 A CN 107609566A CN 201610546022 A CN201610546022 A CN 201610546022A CN 107609566 A CN107609566 A CN 107609566A
- Authority
- CN
- China
- Prior art keywords
- network
- mrow
- sample
- msub
- personalized
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Machine Translation (AREA)
Abstract
The present invention relates to a kind of dual network deep learning method based on a small amount of personalized sample, the dual network includes reconstructed network and depth network, and this method includes：Gather a small amount of personalized sample；Wherein, the personalized sample includes sample data and its label；The reconstructed network is trained using the sample data in personalized sample, then the label in personalized sample is inputted in the reconstructed network after training, generates new reconstruct data and its label；Based on the new reconstruct data from reconstructed network and its label training depth network；By data input to be tested into trained depth network, the output of sigmoid function is obtained, then response of the result as personalization is selected from the output of the sigmoid function by soft maximum returns.
Description
Technical field
The present invention relates to communications and data process field, more particularly to the dual network depth based on a small amount of personalized sample
Learning method.
Background technology
With the fast development of network technology, capacity, personalization and the diversity of data quickly increase, and processing data
Algorithm complex is but difficult to improve, and relies on personal experience and manual operations to describe data, labeled data, selection feature, extraction
Feature, the method for processing data, be difficult to meet the needs of individuation data rapid growth, how efficient process personalization number
According to having become a urgent problem.
The research of deep learning method is broken through, and a direction for being worth exploring is specified to solve data processing problem.It is deep
Degree study can automatically extract feature from big data, and obtain good treatment effect by the sample training of magnanimity.It is actual
On, the research of the rapid growth and deep learning of big data is complementary, and the rapid growth of one side big data needs one
The method of kind efficient process mass data, the training of another aspect deep learning system need the sample data of magnanimity.
But individuation data, mainly as caused by the user of network edge, current method is first to these data
It is collected, then these data is analyzed and handled on server beyond the clouds again.This can produce substantial amounts of data transfer,
Meanwhile for terminal user, the response results obtained from server end are that the model trained by global data responds
Arrive, it is relatively low for the response accuracy of personalized user.If the structure of deep learning network is carried out in terminal, due to terminal
The data volume of personalized user very little, can not be trained to deep learning network, and here it is the big number for being present in current network
According to awkward antinomy.
The Hadoop framework of Apache exploitations at present, the parameter server framework of Baidu's exploitation, the Mariana of Tengxun's exploitation
Framework etc., some trials made both for distributed extensive, deep learning system, by being split to model,
Data are carried out to tear grading mode open, these systems solve the process problem of large-scale data to a certain extent.More than however,
These distributed systems be to be handled large-scale model and data on multimachine, substantial amounts of data between multimachine be present
Communication, this results in these distributed systems and can be only applied to LAN at present.And terminal user goes for response and also must
First it must transfer data in center cluster, then the response of generation is sent to terminal user, therefore user by a center group of planes again
End delay response is very high.
In Wide Area Network, the limitation of network bandwidth be present, big data face a difficult choice antinomy, individual demand the problems such as, if
The problem of communication cost in distributed big data real time processing system is high, edge customer data are lacked is can solve the problem that, then can be had
Distributed big data real time processing system is applied to Wide Area Network by effect ground, is increased network utilization, and improves Consumer's Experience.
Therefore, for a small amount of personalized sample, it is necessary to provide at a kind of distributed data more suitable for Wide Area Network
Reason method, it is big to solve the transmission quantity of training data and model parameter, and when distributed data has customized information, place
The problem of reason precision is difficult to lifting.
The content of the invention
It is an object of the invention to overcome existing distributed data processing method due to edge customer sample data is few, from
And processing accuracy is difficult to the defects of lifting, so as to provide a kind of dual network deep learning method based on a small amount of personalized sample.
To achieve these goals, the invention provides a kind of dual network deep learning side based on a small amount of personalized sample
Method, the dual network include reconstructed network and depth network, and this method includes：
Step 1), a small amount of personalized sample of collection；Wherein, the personalized sample includes sample data and its label；
Step 2), using the sample data in personalized sample the reconstructed network is trained, then by personalized sample
Label input in reconstructed network after training, generate new reconstruct data and its label；
Step 3), based on the new reconstruct data and its label training depth from reconstructed network obtained by step 2)
Spend network；
Step 4), by data input to be tested into trained depth network, obtain the output of sigmoid function, then
Response of the result as personalization is selected from the output of the sigmoid function by soft maximum returns.
In above-mentioned technical proposal, the number of a small amount of personalized sample is designated as m, and m ≈ M/J, M therein are to use big data
The total sample number during depth network of same precision is trained, J is the number by the sample that property sample reconstructs one by one.
In above-mentioned technical proposal, in step 2), the new reconstruct data and its set expression of label that are generated are
{xr k,yr k}；Wherein,
xr kRepresent any one reconstruct data, yr kFor label corresponding to the reconstruct data, k value is 1≤k≤m × J,
M is personalized sample number, and J is the number of the sample reconstructed by any personalized sample；J and personalized sample number m
Relation it is as follows：M values are smaller, and J value is bigger, to ensure to have enough reconstructed sample m × J to train depth network.
In above-mentioned technical proposal, the reconstructed network is including Bayesian network and without any including leveled neural net
A kind of learning network with random reconstruct great amount of samples function.
In above-mentioned technical proposal, the reconstructed network is Bayes's programmed instruction programmed learning network with disposable learning functionality.
In above-mentioned technical proposal, the depth network is to include depth feedforward network, depth confidence net, depth Boltzmann
Any one including machine, depth convolutional network has high accuracy deep learning network.
In above-mentioned technical proposal, the step 4) further comprises：
Step 4-1), calculate by the hidden layer of depth network the output of each neuron, its calculation formula is：
Wherein, yjIt is the output of j-th of neuron, f is a given activation primitive, wijBe from i-th of neuron to
The connection weight of j-th of neuron, xiIt is the input from i-th of neuron, bjIt is biasing weight；
Step 4-2), calculate sigmoid function output；Wherein, the expression formula of sigmoid function is：
The output of resulting sigmoid function is the vector (o of a P dimension1,o2,…,oj,…,oP), vector in each
Element oj=sigmoid (yj) all it is real number between 0 and 1, P here is the sum of personalized labels；
Step 4-3), by soft maximum return that to select a result be used as from the output of the sigmoid function personalized
Response；Wherein, the formula of soft maximum recurrence is as follows：
Soft maximum return is represented respectively with (softmax (o1),softmax(o2),…,softmax(oj),…,
softmax(oP)) probability sigmoid function output (o1,o2,…,oj,…,oP) in choose an ojResponded as personalization.
The advantage of the invention is that：
The present invention is converted a small amount of personalized sample by introducing an auxiliary reconstructed network for supporting an inquiry learning
For mass data, so as to solve the problems, such as that a small amount of personalized sample can not directly train a high accuracy depth network.Together
When, method of the invention is effectively prevented from the data transfer between center and peripheral, and it is carried out to existing deep learning system
Extension, can be applied it in Wide Area Network, and it is not only solved when distributed data has customized information, processing
Precision is difficult to the problem of lifting, and improves the transmission cost of existing distributed big data real time processing system.
Brief description of the drawings
Fig. 1 is the flow chart of the dual network deep learning method based on a small amount of personalized sample of the present invention；
Fig. 2 is the structural representation of dual network involved in the present invention.
Embodiment
In conjunction with accompanying drawing, the invention will be further described.
As depicted in figs. 1 and 2, the dual network deep learning method of the invention based on a small amount of personalized sample, including：
Step 1), a small amount of personalized sample of collection, the input as dual network system；Wherein, a small amount of personalized sample
Sum is designated as m, and span is m >=1；Preferably, m ≈ M/J, M therein are the depth net that same precision is trained with big data
Total sample number during network, J are the number by the sample that property sample reconstructs one by one.Sample in the personalized sample gathered
The set of notebook data can be designated asMiddle i span is 1≤i≤m, and superscript p represents personalized
(personalized)。
In this step, the personalized sample that is gathered except sample data in itself in addition to, in addition to corresponding to the sample
Label.Personalized sample label refers to for arbitrary dataCorrect outputThe set of personalized sample label can be designated asThe personalized sample that this step is gathered is the set of sample data and label, can be designated as
Step 2), the sample data of gathered personalized sample is utilized to train the reconstructed network in dual network (to be designated asAn inquiry learning of a small amount of sample is completed, and the label of personalized sample (is designated as) be input to it is trained
Reconstructed network, to generate a large amount of new reconstruct data and its label, the set of new reconstruct data and its label is designated asWhereinAny one reconstruct data,For label corresponding to the reconstruct data, k value for 1≤k≤m ×
J, m are personalized sample number, and J is the number of the sample reconstructed by any personalized sample, J and personalized sample number
M relation is as follows：M values are smaller, and J value is bigger, to ensure to have enough reconstructed sample m × J to train depth network.
Wherein, the reconstructed network E in dual networka (i)Can be any with random reconstruct great amount of samples function
Practise network, including Bayesian network and without leveled neural net including；Reconstructed network Ea (i)Preferably there is an inquiry learning work(
Bayes's programmed instruction programmed learning network of energy；The priori that described Bayes's programmed instruction programmed learning network first obtains according to background knowledge storehouse
Input data is split, input data is split into small part and tuple, then enters these small parts and tuple
Row reconfigures, and generates new reconstruct dataAgain from multiple reconstruct data of generation, choose and former data similarity highest
Output of the J data as module；Wherein J is positive integer, for representing by property sample reconstructs one by one sample
Number.
Step 3), the set based on a large amount of new reconstruct data and its labels from reconstructed networkWith now
There is technology to train the depth network in dual network (to be designated as Ed (i))；
Wherein, the depth network E in dual networkd (i)Can be that any one has high accuracy deep learning network, it is main
To include depth feedforward network, depth confidence net, depth Boltzmann machine, depth convolutional network.
Step 4), trained depth net will be input to for the personalized sample of test or from outside new data
Network Ed (i)In, the output of sigmoid function is obtained, then a result is selected from the output of sigmoid function by soft maximum returns,
Response as personalization.
Wherein, the expression formula of sigmoid function is：
Wherein, yjIt is the output of j-th of neuron, its value is calculated by the hidden layer of depth network, specific meter
It is as follows to calculate formula：
Wherein, f is a given activation primitive, wijIt is the connection weight from i-th of neuron to j-th of neuron,
xiIt is the input from i-th of neuron, bjIt is biasing weight.
The output of sigmoid function is the vector (o of a P dimension1,o2,…,oj,…,oP), vector in each element oj=
sigmoid(yj) all it is real number between 0 and 1, P here is the sum of personalized labels.
Wherein, the formula of soft maximum recurrence is as follows：
Soft maximum return is represented respectively with (softmax (o1),softmax(o2),…,softmax(oj),…,
softmax(oP)) probability in (o1,o2,…,oj,…,oP) choose an ojResponded as personalization.
It should be noted last that the above embodiments are merely illustrative of the technical solutions of the present invention and it is unrestricted.Although ginseng
The present invention is described in detail according to embodiment, it will be understood by those within the art that, to the technical side of the present invention
Case is modified or equivalent substitution, and without departure from the spirit and scope of technical solution of the present invention, it all should cover in the present invention
Right among.
Claims (7)
1. a kind of dual network deep learning method based on a small amount of personalized sample, the dual network include reconstructed network and depth
Network, this method include：
Step 1), a small amount of personalized sample of collection；Wherein, the personalized sample includes sample data and its label；
Step 2), using the sample data in personalized sample the reconstructed network is trained, then by the mark in personalized sample
In reconstructed network of the label input after training, new reconstruct data and its label are generated；
Step 3), based on the new reconstruct data and its label training depth net from reconstructed network obtained by step 2)
Network；
Step 4), by data input to be tested into trained depth network, obtain the output of sigmoid function, then pass through
One soft maximum returns selects response of the result as personalization from the output of the sigmoid function.
2. the dual network deep learning method according to claim 1 based on a small amount of personalized sample, it is characterised in that institute
The number for stating a small amount of personalized sample is designated as m, and m ≈ M/J, M therein are when training the depth network of same precision with big data
Total sample number, J is by the number of the sample that property sample reconstructs one by one.
3. the dual network deep learning method according to claim 1 based on a small amount of personalized sample, it is characterised in that
In step 2), the new reconstruct data and its set expression of label that are generated are { xr k,yr k}；Wherein,
xr kRepresent any one reconstruct data, yr kFor label corresponding to the reconstruct data, k value is 1≤k≤m × J, and m is individual
Property number of samples, J is the number of sample reconstructed by any personalized sample；J and personalized sample number m relation
It is as follows：M values are smaller, and J value is bigger, to ensure to have enough reconstructed sample m × J to train depth network.
4. the dual network deep learning method according to claim 1 based on a small amount of personalized sample, it is characterised in that institute
It is to have random reconstruct great amount of samples including Bayesian network and without any one including leveled neural net to state reconstructed network
The learning network of function.
5. the dual network deep learning method according to claim 4 based on a small amount of personalized sample, it is characterised in that institute
It is Bayes's programmed instruction programmed learning network with disposable learning functionality to state reconstructed network.
6. the dual network deep learning method according to claim 1 based on a small amount of personalized sample, it is characterised in that institute
It is any including depth feedforward network, depth confidence net, depth Boltzmann machine, depth convolutional network to state depth network
One kind has high accuracy deep learning network.
7. the dual network deep learning method according to claim 1 based on a small amount of personalized sample, it is characterised in that institute
Step 4) is stated to further comprise：
Step 4-1), calculate by the hidden layer of depth network the output of each neuron, its calculation formula is：
<mrow>
<msub>
<mi>y</mi>
<mi>j</mi>
</msub>
<mo>=</mo>
<mi>f</mi>
<mrow>
<mo>(</mo>
<munder>
<mo>&Sigma;</mo>
<mi>i</mi>
</munder>
<msub>
<mi>w</mi>
<mrow>
<mi>i</mi>
<mi>j</mi>
</mrow>
</msub>
<msub>
<mi>x</mi>
<mi>i</mi>
</msub>
<mo>+</mo>
<msub>
<mi>b</mi>
<mi>j</mi>
</msub>
<mo>)</mo>
</mrow>
</mrow>
Wherein, yjIt is the output of j-th of neuron, f is a given activation primitive, wijIt is from i-th of neuron to j-th
The connection weight of neuron, xiIt is the input from i-th of neuron, bjIt is biasing weight；
Step 4-2), calculate sigmoid function output；Wherein, the expression formula of sigmoid function is：
<mrow>
<mi>s</mi>
<mi>i</mi>
<mi>g</mi>
<mi>m</mi>
<mi>o</mi>
<mi>i</mi>
<mi>d</mi>
<mrow>
<mo>(</mo>
<msub>
<mi>y</mi>
<mi>j</mi>
</msub>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mfrac>
<mn>1</mn>
<mrow>
<mn>1</mn>
<mo>+</mo>
<mi>exp</mi>
<mrow>
<mo>(</mo>
<msub>
<mi>y</mi>
<mi>j</mi>
</msub>
<mo>)</mo>
</mrow>
</mrow>
</mfrac>
<mo>;</mo>
</mrow>
The output of resulting sigmoid function is the vector (o of a P dimension1,o2,…,oj,…,oP), vector in each element
oj=sigmoid (yj) all it is real number between 0 and 1, P here is the sum of personalized labels；
Step 4-3), pass through soft maximum return and select a result be used as personalized response from the output of the sigmoid function；
Wherein, the formula of soft maximum recurrence is as follows：
<mrow>
<mi>s</mi>
<mi>o</mi>
<mi>f</mi>
<mi>t</mi>
<mi> </mi>
<mi>m</mi>
<mi>a</mi>
<mi>x</mi>
<mrow>
<mo>(</mo>
<msub>
<mi>o</mi>
<mi>j</mi>
</msub>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mfrac>
<mrow>
<mi>exp</mi>
<mrow>
<mo>(</mo>
<msub>
<mi>o</mi>
<mi>j</mi>
</msub>
<mo>)</mo>
</mrow>
</mrow>
<mrow>
<munder>
<mo>&Sigma;</mo>
<mi>i</mi>
</munder>
<mi>exp</mi>
<mrow>
<mo>(</mo>
<msub>
<mi>o</mi>
<mi>i</mi>
</msub>
<mo>)</mo>
</mrow>
</mrow>
</mfrac>
</mrow>
Soft maximum return is represented respectively with (softmax (o1),softmax(o2),…,softmax(oj),…,softmax
(oP)) probability sigmoid function output (o1,o2,…,oj,…,oP) in choose an ojResponded as personalization.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610546022.7A CN107609566A (en) | 2016-07-12 | 2016-07-12 | Dual network deep learning method based on a small amount of personalized sample |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610546022.7A CN107609566A (en) | 2016-07-12 | 2016-07-12 | Dual network deep learning method based on a small amount of personalized sample |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107609566A true CN107609566A (en) | 2018-01-19 |
Family
ID=61055027
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610546022.7A Pending CN107609566A (en) | 2016-07-12 | 2016-07-12 | Dual network deep learning method based on a small amount of personalized sample |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107609566A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110223785A (en) * | 2019-05-28 | 2019-09-10 | 北京师范大学 | A kind of infectious disease transmission network reconstruction method based on deep learning |
-
2016
- 2016-07-12 CN CN201610546022.7A patent/CN107609566A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110223785A (en) * | 2019-05-28 | 2019-09-10 | 北京师范大学 | A kind of infectious disease transmission network reconstruction method based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Lim et al. | Time-series forecasting with deep learning: a survey | |
| Wu et al. | Evolving RBF neural networks for rainfall prediction using hybrid particle swarm optimization and genetic algorithm | |
| Makridakis et al. | Statistical, machine learning and deep learning forecasting methods: Comparisons and ways forward | |
| Gong et al. | Multi-objective parameter optimization of common land model using adaptive surrogate modeling | |
| Chen et al. | Dictionary learning over distributed models | |
| Corchado et al. | Ibr retrieval method based on topology preserving mappings | |
| CN107944559A (en) | A kind of entity relationship automatic identifying method and system | |
| CN103942749B (en) | A kind of based on revising cluster hypothesis and the EO-1 hyperion terrain classification method of semi-supervised very fast learning machine | |
| Konomi et al. | Bayesian treed multivariate gaussian process with adaptive design: Application to a carbon capture unit | |
| Chauhan et al. | Approximating parameters of photovoltaic models using an amended reptile search algorithm | |
| Said et al. | AI-based solar energy forecasting for smart grid integration | |
| Wang et al. | An approach of recursive timing deep belief network for algal bloom forecasting | |
| CN109670927A (en) | The method of adjustment and its device of credit line, equipment, storage medium | |
| Sahoo et al. | Space–time forecasting of groundwater level using a hybrid soft computing model | |
| Bakhashwain | Prediction of global solar radiation using support vector machines | |
| Zhang et al. | Wind speed forecasting based on wavelet decomposition and wavelet neural networks optimized by the Cuckoo search algorithm | |
| Wen et al. | A solar forecasting framework based on federated learning and distributed computing | |
| Fang et al. | A novel application of artificial neural network for wind speed estimation | |
| Abed et al. | Data mining in web personalization using the blended deep learning model | |
| Dunbar et al. | Ensemble‐based experimental design for targeting data acquisition to inform climate models | |
| Zhao et al. | Short‐term load forecasting of multi‐scale recurrent neural networks based on residual structure | |
| Zhang et al. | Digital twin empowered PV power prediction | |
| Al-Hajj et al. | Estimating solar irradiance using genetic programming technique and meteorological records. | |
| Gouthamkumar et al. | Hybridized gravitational search algorithm for short-term hydrothermal scheduling | |
| Semero et al. | A GA-PSO hybrid algorithm based neural network modeling technique for short-term wind power forecasting |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180119 |