CN109670622A - Neural net prediction method and device - Google Patents
Neural net prediction method and device Download PDFInfo
- Publication number
- CN109670622A CN109670622A CN201710960503.7A CN201710960503A CN109670622A CN 109670622 A CN109670622 A CN 109670622A CN 201710960503 A CN201710960503 A CN 201710960503A CN 109670622 A CN109670622 A CN 109670622A
- Authority
- CN
- China
- Prior art keywords
- predicted
- prediction
- module
- prediction result
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Economics (AREA)
- General Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Biomedical Technology (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Game Theory and Decision Science (AREA)
- Development Economics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Operations Research (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The disclosure provides a kind of neural net prediction method and device, which comprises obtains the characteristic information of multiple objects to be predicted respectively;It will be handled in the characteristic information input prediction model, obtain the prediction result for being directed to the multiple object to be predicted;Export the prediction result of at least one object to be predicted in the multiple object to be predicted, wherein the prediction model includes shot and long term memory network LSTM module, deep neural network DNN module, relational network RN module and multi-task learning network MTL module.The disclosure can capture timing between the characteristic information of multiple objects to be predicted, object to be predicted characteristic information between relevance, the relevance between each object to be predicted while the prediction result for exporting multiple objects to be predicted, and improve the accuracy rate of prediction result.
Description
Technical field
This disclosure relates to nerual network technique field more particularly to a kind of neural net prediction method and device.
Background technique
In actual life, need to predict various events, for example, for will show film prediction box office,
For will show TV play prediction audience ratings, America, presidential elections, the winning digit of current lottery, football match victory or defeat
Deng, various events to be predicted the incidence relation that influence factor is complicated, different between each influence factor also result in it is different pre-
Survey as a result, but in traditional predicting means, limited data can only be directed to, carried out using related algorithms such as probability statistics pre-
It surveys, the accuracy rate of prediction result is low.
Summary of the invention
In view of this, the present disclosure proposes a kind of neural net prediction method and device, it can be simultaneously to multiple to be predicted
The characteristic information of object is handled, and the accuracy rate of prediction result is improved.
According to the one side of the disclosure, a kind of neural net prediction method is provided, which comprises
The characteristic information of multiple objects to be predicted is obtained respectively;
It will be handled in the characteristic information input prediction model, obtain the prediction for being directed to the multiple object to be predicted
As a result;
The prediction result of at least one object to be predicted in the multiple object to be predicted is exported,
Wherein, the prediction model includes shot and long term memory network LSTM module, deep neural network DNN module, relationship
Network RN module and multi-task learning network MTL module.
In one possible implementation, it will be handled in the characteristic information input prediction model, acquisition is directed to
The prediction result of the multiple object to be predicted, comprising:
The characteristic information is inputted in the LSTM module and is handled, determines the length of the multiple object to be predicted
Phase recall info;
The shot and long term recall info is inputted in the DNN module and is handled, determines the multiple object to be predicted
Depth information;
The depth information is inputted in the RN module and is handled, determines the relationship letter of the multiple object to be predicted
Breath;
The relation information is inputted in the MTL module and is handled, is determined for the multiple object to be predicted
Prediction result.
In one possible implementation, the prediction model further includes insertion module,
Wherein, it will be handled in the characteristic information input prediction model, obtain and be directed to the multiple object to be predicted
Prediction result, further includes:
Initial information is inputted into the insertion module and carries out vectorization processing, determines the vector information of the initial information;
The initial information and the vector information are determined as the characteristic information.
In one possible implementation, the prediction result includes:
Main prediction result and correlation predictive as a result,
Wherein, in the objective function of the prediction model, parameters weighting associated with the main prediction result is greater than
Parameters weighting associated with the correlation predictive result.
In one possible implementation, the pre- of at least one object to be predicted in the multiple object to be predicted is exported
It surveys as a result, including any of the following:
Export the main prediction result of at least one object to be predicted in the multiple object to be predicted;
Export the main prediction result and correlation predictive knot of at least one object to be predicted in the multiple object to be predicted
Fruit.
In one possible implementation, the method also includes:
The characteristic information of multiple sample object current periods is obtained respectively;
The training prediction result of the characteristic information of the current period and previous cycle is inputted in initial predicted model
It is handled, obtains the training prediction result for being directed to the multiple sample object current period;
It is pre- according to the training prediction result in multiple periods of the multiple sample object and the expectation in the multiple period
It surveys as a result, determining multiple modulus of periodicity types loss of the multiple sample object respectively;
It is lost according to the model, adjusts the parameters weighting in the initial predicted model, determine prediction mould adjusted
Type;
In the case where model loss meets training condition, prediction model adjusted is determined as to final prediction
Model.
In one possible implementation, it is lost according to the model, adjusts the parameter in the initial predicted model
Weight determines prediction model adjusted, comprising:
According to the multiple modulus of periodicity type losses of the multiple sample object, it is sequentially adjusted in the MTL module, the RN mould
Parameters weighting in block, the DNN module, the LSTM module and the insertion module, determines prediction model adjusted.
In one possible implementation, the characteristic information of the multiple object to be predicted is identical.
According to another aspect of the present disclosure, a kind of neural network prediction device is provided, which is characterized in that described device packet
It includes:
Characteristic acquisition unit, for obtaining the characteristic information of multiple objects to be predicted respectively;
Prediction result acquiring unit obtains for will handle in the characteristic information input prediction model and is directed to institute
State the prediction result of multiple objects to be predicted;
Prediction result output unit, for exporting the prediction of at least one object to be predicted in the multiple object to be predicted
As a result,
Wherein, the prediction model includes shot and long term memory network LSTM module, deep neural network DNN module, relationship
Network RN module and multi-task learning network MTL module.
In one possible implementation, the prediction result acquiring unit includes:
LSTM handles subelement, handle for inputting the characteristic information in the LSTM module, described in determination
The shot and long term recall info of multiple objects to be predicted;
DNN handles subelement, handles for inputting the shot and long term recall info in the DNN module, determines
The depth information of the multiple object to be predicted;
RN handles subelement, handles, determines the multiple for inputting the depth information in the RN module
The relation information of object to be predicted;
MTL handles subelement, handles for inputting the relation information in the MTL module, determines and is directed to institute
State the prediction result of multiple objects to be predicted.
In one possible implementation, the prediction model further includes insertion module,
The prediction result acquiring unit further include:
It is embedded in subelement, carries out vectorization processing for initial information to be inputted the insertion module, is determined described initial
The vector information of information;
Information determines subelement, for the initial information and the vector information to be determined as the characteristic information.
In one possible implementation, the prediction result includes:
Main prediction result and correlation predictive as a result,
Wherein, in the objective function of the prediction model, parameters weighting associated with the main prediction result is greater than
Parameters weighting associated with the correlation predictive result.
In one possible implementation, the prediction result output unit for export it is following any one:
Export the main prediction result of at least one object to be predicted in the multiple object to be predicted;
Export the main prediction result and correlation predictive knot of at least one object to be predicted in the multiple object to be predicted
Fruit.
In one possible implementation, further includes:
Sample characteristics information acquisition unit, for obtaining the characteristic information of multiple sample object current periods respectively;
Training prediction result acquiring unit, for the training of the characteristic information of the current period and previous cycle is pre-
It surveys in result input initial predicted model and is handled, obtain the training prediction knot for the multiple sample object current period
Fruit;
Model loses determination unit, for according to the training prediction result in multiple periods of the multiple sample object and
The expectation prediction result in the multiple period determines multiple modulus of periodicity types loss of the multiple sample object respectively;
Weight adjustment unit adjusts the parameters weighting in the initial predicted model, really for losing according to the model
Fixed prediction model adjusted;
Prediction model determination unit, for the model loss meet training condition in the case where, will be adjusted pre-
It surveys model and is determined as final prediction model.
In one possible implementation, the weight adjustment unit includes:
Weight adjusts subelement, for losing according to the model of the multiple sample object current period, is sequentially adjusted in institute
The parameters weighting in MTL module, the RN module, the DNN module, the LSTM module and the insertion module is stated, really
Prediction model after settled preceding period modulation.
In one possible implementation, the characteristic information of the multiple object to be predicted is identical.
According to another aspect of the present disclosure, a kind of neural network prediction device is provided, comprising: processor;For storing
The memory of processor-executable instruction;Wherein, the processor is configured to executing the above method.
According to another aspect of the present disclosure, a kind of non-volatile computer readable storage medium storing program for executing is provided, is stored thereon with
Computer program instructions, wherein the computer program instructions realize the above method when being executed by processor.
According to the neural net prediction method and device of embodiment of the present disclosure various aspects, by remembering using including shot and long term
The prediction model of network LSTM, deep neural network DNN, relational network RN and multi-task learning network MTL, to multiple to pre-
The characteristic information for surveying object is handled, and is obtained the prediction result for being directed to multiple objects to be predicted, can be captured multiple to be predicted
Timing between the characteristic information of object, the relevance between the characteristic information of object to be predicted, between each object to be predicted
Relevance, to improve the accuracy rate of prediction result.
According to below with reference to the accompanying drawings to detailed description of illustrative embodiments, the other feature and aspect of the disclosure will become
It is clear.
Detailed description of the invention
Comprising in the description and constituting the attached drawing of part of specification and specification together illustrates the disclosure
Exemplary embodiment, feature and aspect, and for explaining the principles of this disclosure.
Fig. 1 is a kind of flow chart of neural net prediction method shown according to an exemplary embodiment.
Fig. 2 is the flow chart of step S12 in a kind of neural net prediction method shown according to an exemplary embodiment.
Fig. 3 is the flow chart of step S12 in a kind of neural net prediction method shown according to an exemplary embodiment.
Fig. 4 is a kind of flow chart of neural net prediction method shown according to an exemplary embodiment.
Fig. 5 is a kind of block diagram of prediction model shown according to an exemplary embodiment.
Fig. 6 is a kind of three-layer neural network structural schematic diagram shown according to an exemplary embodiment.
Fig. 7 is a kind of structural schematic diagram of multilayer perceptron shown according to an exemplary embodiment.
Fig. 8 is a kind of showing for the multi-task learning network of neural net prediction method shown according to an exemplary embodiment
It is intended to.
Fig. 9 is the input schematic diagram of vector information and initial information shown according to an exemplary embodiment.
Figure 10 is a kind of schematic diagram of the application scenarios of neural net prediction method shown according to an exemplary embodiment.
Figure 11 is a kind of block diagram of neural network prediction device shown according to an exemplary embodiment.
Figure 12 is a kind of block diagram of neural network prediction device shown according to an exemplary embodiment.
Figure 13 is a kind of block diagram of device for neural network prediction shown according to an exemplary embodiment.
Specific embodiment
Various exemplary embodiments, feature and the aspect of the disclosure are described in detail below with reference to attached drawing.It is identical in attached drawing
Appended drawing reference indicate element functionally identical or similar.Although the various aspects of embodiment are shown in the attached drawings, remove
It non-specifically points out, it is not necessary to attached drawing drawn to scale.
Dedicated word " exemplary " means " being used as example, embodiment or illustrative " herein.Here as " exemplary "
Illustrated any embodiment should not necessarily be construed as preferred or advantageous over other embodiments.
In addition, giving numerous details in specific embodiment below to better illustrate the disclosure.
It will be appreciated by those skilled in the art that without certain details, the disclosure equally be can be implemented.In some instances, for
Method, means, element and circuit well known to those skilled in the art are not described in detail, in order to highlight the purport of the disclosure.
Fig. 1 is a kind of flow chart of neural net prediction method shown according to an exemplary embodiment.This method can answer
For in server, as shown in Figure 1, the neural net prediction method includes:
Step S11 obtains the characteristic information of multiple objects to be predicted respectively.
For example, the object to be predicted may include that film, TV play, ball match, lottery ticket, presidential election etc. will occur
Event, the characteristic information include may to the various influence factors that the prediction result of object to be predicted has an impact, for example,
When predicting the box office etc. for the film that will be shown, object to be predicted be film, prediction result can for the film box office,
Viewing number etc., characteristic information may include the associated various information such as the subject matter of the film, working days, director and performer.
And when predicting the result of the match of ball match, object to be predicted be ball match, prediction result can for the ball match result (victory or defeat),
Score etc., characteristic information may include the associated various information such as both sides coach, sportsman, weather, history victory or defeat.For it is each to
It predicts object, a variety of (various dimensions) characteristic informations of the object can be obtained as much as possible, in actual prediction, each to pre-
The characteristic information for surveying object can be, for example, 100 dimensions to 200 dimensions.
In one possible implementation, characteristic information may include characteristic information itself and relevant to characteristic information
Numerical value, the numerical value may include the real number value of discrete type or continuous type.By taking the scene that the box office of film is predicted as an example, characteristic information can
Including subject matter and performer, wherein subject matter can indicate whether to belong to this feature of theme of love, 1 table of value with a Boolean
Show that current sample (film) belongs to theme of love, the expression of value 0 is not belonging to theme of love;And quantifiable index can be used in performer
Indicate, for example, using performer movie show for the previous period in volumes of searches in a search engine.
Step S12 will be handled in the characteristic information input prediction model, be obtained for the multiple to be predicted right
The prediction result of elephant, wherein the prediction model include shot and long term memory network LSTM module, deep neural network DNN module,
Relational network RN module and multi-task learning network MTL module.
For example, the prediction model includes trained neural network prediction model, and the neural network prediction
Model includes shot and long term memory network LSTM (Long Short-Term Memory) module, deep neural network DNN (Deep
Neural Network) module, relational network RN (Relation Network) module and multi-task learning network MTL
(Multi-Task Learning) module.
In actual prediction, the prediction result of t moment often with the t-1 moment and earlier before prediction result have one
Fixed relevance.For example, (t-1 moment) according to characteristic informations such as the subject matter of a certain film, director, performers, obtains before
Corresponding prediction result (box office) is arrived;And there may be variation (such as performers for the characteristic information of current (t moment) film
Front or negative press etc.).In this case, before the prediction result of current (t moment) is based on (t-1 moment)
Prediction result and generate certain variation.That is, may have between current prediction result and prediction result before
Temporal relevance.LSTM module can be used in the treatment process of prediction result introducing this timing information.
There is complicated incidence relation between multiple/multidimensional characteristic information of same object to be predicted, and incidence relation is
It is no accurately to be captured, the accuracy of prediction result is directly influenced, the DNN module based on deep neural network is for capturing not
The integrated mode of same characteristic information.And more (depth is bigger) its learning ability of the number of plies of DNN module are stronger, eventually form
The stability of prediction model is higher, and capability of fitting is also stronger.
The capture of incidence relation between the achievable difference object to be predicted of RN module.Such as one of them is being directed to pre-
When survey object A is predicted, RN module can be obviously improved the relation inference between object A to be predicted and other objects to be predicted
Effect.
To obtain multiple prediction results of multiple objects to be predicted simultaneously, or the multiple of an object to be predicted are exported simultaneously
Prediction result is handled multiple prediction results using MTL module simultaneously.
By the characteristic information of multiple objects to be predicted, inputs after trained prediction model handled, can obtain multiple
One or more prediction results of object to be predicted, for example, the characteristic information of multi-section film is inputted trained prediction model
After being handled, the prediction result at the box office of multi-section film can be obtained, box office and the public praise of multi-section film can also be obtained simultaneously
Etc. multiple prediction results.
It is understood that can flexibly set prediction result according to demand according to the characteristic information of multiple objects to be predicted
Acquisition situation, including obtain a prediction result of an object to be predicted, multiple prediction results of object to be predicted,
One identical prediction result of multiple objects to be predicted and multiple prediction results of multiple objects to be predicted, in prediction model
It is set in training process.The disclosure to this with no restriction.
Step S13 exports the prediction result of at least one object to be predicted in the multiple object to be predicted.
For example, what is got is the prediction result for multiple objects to be predicted, and according to demand, selection output is wherein
The prediction result of one object to be predicted, or selection export the prediction result of multiple objects to be predicted.
In the present embodiment, by utilizing the prediction mould for including LSTM module, DNN module, RN module and MTL module
Type after handling the characteristic information of multiple objects to be predicted, obtains the prediction result for being directed to the multiple object to be predicted,
According to all aspects of this disclosure, timing between the characteristic information of multiple objects to be predicted, object to be predicted can be captured
The relevance between relevance, each object to be predicted between characteristic information, the prediction knot for exporting multiple objects to be predicted simultaneously
Fruit, and improve the accuracy rate of prediction result.
Fig. 2 is the flow chart of step S12 in a kind of neural net prediction method shown according to an exemplary embodiment, should
Method can be applied in server, on the basis of the above embodiments, as shown in Fig. 2, step S12 includes:
The characteristic information is inputted in the LSTM module and is handled, determined the multiple to be predicted by step S121
The shot and long term recall info of object.
For example, the characteristic information of the multiple object to be predicted needs to input LSTM module respectively and is handled.Fig. 5
It is a kind of block diagram of prediction model shown according to an exemplary embodiment.By taking three objects to be predicted as an example, Fig. 5 is provided pre-
It surveys in model, LSTM module can be respectively set for three objects to be predicted and carry out parallel processing, it is ensured that processing speed.It can be with
Understand, to save hardware cost, may also set up the feature of a LSTM module successively three objects to be predicted of serial process
Information.
In the embodiment that Fig. 5 is provided, by the characteristic information of multiple objects to be predicted, respectively input with it is described to be predicted right
As one-to-one LSTM model is handled, the shot and long term recall info of the multiple object to be predicted is obtained.Due in reality
In the prediction on border, the prediction result of object t moment to be predicted often with the t-1 moment and earlier before prediction result have one
Fixed correlation can use Recognition with Recurrent Neural Network (Recurrent to introduce this timing information during prediction
Neural Network, RNN) it is handled.
In the structure of RNN, the output of a sequence current output and front is also related.The specific form of expression is net
Network can remember the information of front and is applied in the calculating currently exported, i.e., the node between hidden layer is no longer connectionless
But there is connection, and it further includes the output of last moment hidden layer that the input of hidden layer, which not only includes the output of input layer,.
Wherein, one group of weight parameter is shared between input layer and hidden layer, hidden layer and output layer, hidden layer and hidden layer respectively.This
A little weight parameters can be solved by BPTT (Back Propagation Through Time) algorithm.
Although theoretically RNN can be handled the sequence data of any length, increase in practice as the length of sequence
Add, RNN can encounter the gradient disappearance problem of deep neural network.Therefore, LSTM structure can be introduced to alleviate the gradient of RNN and disappear
Problem.
LSTM structure is used to replace the neuron of hidden layer in RNN structure, using out gate, input gate, forgets door
Determine whether information is retained in transmission process, each design parameter is equally obtained by BPTT learning algorithm.
LSTM module can be used for handling the letter of the feature on the multiple time points of multiple objects to be predicted in a period of time
Breath, captures the timing information between the characteristic information on multiple time points.The output information of LSTM module is multiple to be predicted right
The shot and long term recall info of elephant, LSTM module export shot and long term recall info to DNN module.
The shot and long term recall info is inputted in the DNN module and is handled by step S122, determine it is the multiple to
Predict the depth information of object.
For example, DNN module includes the full articulamentum of multilayer, and DNN module is used based on coding (AutoEncoder) certainly
Pre-training (Pre-training) solve the problems, such as the gradient disappearance of DNN.Fig. 6 is shown according to an exemplary embodiment one
Kind three-layer neural network structural schematic diagram.As shown in fig. 6, for each layer of input of DNN, first using three layers of mind in Fig. 6
It is fitted itself through network structure, wherein output layer is the duplication of input layer, and the target of network structure study is by " input
Layer-hidden layer " (code weight W1) and " hidden layer-output layer " (decoding weight W2) this two groups of weight parameters are fitted as much as possible
Input layer itself.The nodal point number of hidden layer is identical as next layer in DNN of nodal point number.Finally use code weight W1 as current
The initial value of weight parameter between input layer and next hidden layer, and so on, each layer in DNN of weight parameter is all logical
The mode for crossing above-mentioned pre-training is initialized, and entire DNN may finally be made to obtain good convergence.
In one possible implementation, according to the DNN model after pre-training, DNN module can be constructed.It will be multiple
The shot and long term recall info of object to be predicted is inputted respectively in DNN model, can respectively multidimensional to each object to be predicted it is special
Incidence relation between reference breath carries out capture processing, so that it is determined that the depth information of multiple objects to be predicted.
The depth information is inputted in the RN module and is handled by step S123, and it is the multiple to be predicted right to determine
The relation information of elephant.
For example, the depth information of multiple objects to be predicted can be input in RN module and is handled.Network of personal connections
Network model RN has significant effect promoting to the problem of relation inference (relational reasoning) correlation.
It is handled for example, the depth information of film A1, film A2 and film A3 are input in RN module, it can be with base
The relation information of film A1, film A2 and film A3 are determined in the trained RN module.As it was noted above, target is to be predicted
Object can be one, for example, can be film A1, at this point, RN module is by film A1 and film A2, film A1 with film A3's
Depth information relationship determines the relation information of film A1 Yu film A2 and film A3 as input.If target object to be predicted
For film A1, film A2 and film A3, then RN module by film A1 and film A2, film A1 and film A3 and film A2 and
The depth information of film A3 determines the relation information of film A1, film A2 and film A3 as input.
Fig. 7 is a kind of structural schematic diagram of multilayer perceptron shown according to an exemplary embodiment.As shown in fig. 7, such as
Fruit needs to object x4It is predicted, then object x can be determined by multilayer perceptron g respectively in RN network1、x2And x3With x4
Between relationship, and sum, then the relation information after summation handled by multilayer perceptron f, final output
Multiple object (x1、x2And x3) it is directed to object x to be predicted4Relation information.
In one possible implementation, it can also be respectively processed between all objects to be predicted of input,
Determine the relationship between all objects to be predicted, such as to object x in Fig. 71~x4Between all compositions of relations located
It manages, the relation information between all objects to be predicted of final output.Tool of the disclosure to the relation information of multiple objects to be predicted
Body composition of relations is with no restriction.
An illustrative relation information is given below and determines formula (1):
Assuming that concern xn, and x1, x2 ..., xn-1 are and the associated input of xn.In order to predict the output of xn, consider
N-1 binary group (xi, xn), i=1,2 ... n-1.
Can define relational network RN is following compound function:
In formula (1), RN (x1, x2 ..., xn) indicates that multiple objects (x1, x2 to xn-1) are directed to object xn to be predicted
Relation information,And gθIt is multilayer perceptron, i is variable, and the value of i is 1 between n-1.
In this way, after being handled in RN module, the relation information of multiple objects to be predicted can be inputted.
The relation information is inputted in the MTL module and is handled by step S124, is determined for the multiple to pre-
Survey the prediction result of object.
For example, the relation information of object to be predicted can be input in MTL module and is handled.Wherein, MTL mould
For block for realizing multi-task learning, multi-task learning is to indicate (shared representation) based on shared, allows main task
(main tasks) uses inter-related task (related tasks), a kind of machine learning side of the extensive effect of Lai Tisheng main task
Method.For example, the prediction for video flow, main task is accurately prediction video flow VV (video view), but pre- simultaneously
The predictablity rate of main task can be promoted by surveying multiple inter-related tasks, and inter-related task includes video playing amount, click volume, playback volume
Deng.Multi-task learning is divided into hard parameter sharing (hard parameter sharing) and soft parameter sharing (soft
Parameter sharing) two kinds, the present embodiment uses hard parameter sharing.
Fig. 8 is a kind of showing for the multi-task learning network of neural net prediction method shown according to an exemplary embodiment
It is intended to.As shown in figure 8, the multi-task learning network can be realized simultaneously main task output and inter-related task output, obtain to
Predict the corresponding main prediction result and correlation predictive result of object.It will be understood by those skilled in the art that for multitask
The specific structure and parameter sharing mode for practising network can be configured as needed, as long as can carry out by the MTL module
Multi-task learning, the disclosure to this with no restriction.
In MTL module, inter-related task is equivalent to relative to main task introduces noise, and introduces noise and mould can be improved
The generalization ability of type.Canonical effect between main task and inter-related task prevents over-fitting, lift scheme generalization ability.And MTL
The attention mechanism (attention) of module, can aid forecasting model focus more on relevant feature.
In the present embodiment, by utilizing the prediction mould for including LSTM module, DNN module, RN module and MTL module
Type after handling the characteristic information of multiple objects to be predicted, obtains the prediction result for being directed to the multiple object to be predicted,
According to all aspects of this disclosure, timing between the characteristic information of multiple objects to be predicted, object to be predicted can be captured
The relevance between relevance, each object to be predicted between characteristic information, the prediction knot for exporting multiple objects to be predicted simultaneously
Fruit, and improve the accuracy rate of prediction result.
Fig. 3 is the flow chart of step S12 in a kind of neural net prediction method shown according to an exemplary embodiment, should
Method can be applied in server, and on the basis of the above embodiments, the prediction model further includes insertion module.Such as Fig. 3 institute
Show, step S12 further include:
Initial information is inputted the insertion module and carries out vectorization processing, determines the initial information by step S1211
Vector information.
For example, in actual prediction, characteristic information includes the history box office of director, directs and be good at subject matter, performer A
Baidu's index, the history box office of performer A etc., wherein the history box office directed and director, which are good at subject matter, belongs to this master of director
Body, and the history box office of Baidu's index of performer A, performer A belong to this main body of performer A.I.e. an object to be predicted includes
Multiple main bodys, multiple characteristic informations belong to the same main body.
Initial information is to handle the discrete type obtained or successional reality by simple mathematical after directly acquiring various information
Numerical value, but the simple characteristic information for relying on numerical value, are typically only capable to the influence from a dimension response feature information.For example, performer A
Movie show for the previous period in Baidu index of the volumes of searches as performer A in a search engine, for predicting film
Box office.What Baidu's index substantially reflected is the temperature of chief creating, and is difficult the had an impact flow of exhaustion by Feature Engineering merely
Dimension, in addition to this, numerical value is also more serious to the dependence of external data.
In insertion module, all characteristic informations for each main body being involved in are spliced, with a fixed length
The vector of degree indicates, obtains the vector information of each main body, and by taking volume forecasting as an example, each chief creating, subject matter are corresponding
The vector of one regular length.
In the input layer of prediction model, in addition to inputting initial information, also input is generated after being spliced using initial information
The vector information of each main body, the vector letter in the training process of embeding layer, as the node in hiding, in input layer
Breath also needs to update, and it is shared that the corresponding vector information of the characteristic information of the same main body carries out weight.
Fig. 9 is the input schematic diagram of vector information and initial information shown according to an exemplary embodiment, such as Fig. 9 institute
Show, by taking TV play volume forecasting as an example, inputs the multiple features 1, feature 2 and feature 3 of TV play, the numeric type mark sheet in figure
Show the traditional characteristic of TV play, increase embeding layer (embedding layer) input on this basis, i.e., by current television play
The features such as director, protagonist are indicated with the vector of a regular length respectively, and every one-dimensional in random initialization vector, output
The flow that can be current sample can increase output, such as scoring under the frame of multi-task learning.Middle layer can be used
Arbitrary neural network structure (being DNN used in figure), constantly adjusts nerve by Back Propagation Algorithm in learning process
The numerical value of vector in the parameter and embeding layer of network.After the completion of training, each feature obtains a vector and indicates, obtains simultaneously
The parameter of final neural network.It is in addition to traditional numeric type feature, the feature of current television play is corresponding special when being predicted
The vector of sign is spliced into embeding layer, and the parameter obtained together with numeric type feature based on training carries out propagated forward, final to calculate
Depth information Y1, Y2 and the Y3 of each feature in output layer out.
The vectorization of characteristic information indicates, more preferable compared to traditional numerical representation stability.Vector information is advantageous
In portraying the similitude between characteristic information, such as subject matter " magical " and " magic " have a degree of similitude, according to biography
Boolean's representation of system can not portray this similitude, and can use the similarity between vector after being embedded in portray
This similitude.Also, current signature is often contained in multiple dimensions for prediction by being embedded in the vector that training obtains
The information content of structure reduces the engineering burden for extracting characteristic information.
The initial information and the vector information are determined as the characteristic information by step S1212.
For example, the prediction model that joined embeding layer needs simultaneously to include initial letter in the input of LSTM module
Breath and vector information.
In the present embodiment, it due to joined embeding layer, so that the stability of numerical representation is more preferable, more conducively captures
Similitude between characteristic information has contained current characteristic information in multiple dimensions for pre- by the vector information after training
The information content for surveying result reduces the burden that information characteristics extract engineering.
In one possible implementation, the prediction result include main prediction result and correlation predictive as a result, its
In, in the objective function of the prediction model, to the main associated parameters weighting of prediction result be greater than with it is described related
The associated parameters weighting of prediction result.
Multiple prediction results of prediction model output, in the objective function of training pattern, the settable parameter power first closed
Weight, to adjust the accuracy between different prediction results.For example, theme to be predicted is film, main prediction result is box office, related
Prediction result is public praise, and in objective function, by adjusting separately the parameters weighting at box office and public praise, adjusting main prediction result is
Box office or public praise.
In the present embodiment, prediction result includes main prediction result and correlation predictive as a result, by adjusting prediction model
Parameters weighting in objective function can adjust the accuracy of main prediction result and correlation predictive result.
In one possible implementation, the pre- of at least one object to be predicted in the multiple object to be predicted is exported
It surveys as a result, including any of the following: the main prediction knot of at least one object to be predicted in the multiple object to be predicted of output
Fruit;Export the main prediction result and correlation predictive result of at least one object to be predicted in the multiple object to be predicted.
Since the accuracy of main prediction result is higher than correlation predictive as a result, when focusing more on the accuracy of prediction result,
Main prediction result can be only exported, the prediction result at box office is such as only exported;When paying close attention to the correlation between each prediction result simultaneously,
Main prediction result and correlation predictive are exported simultaneously as a result, exporting related prediction result while exporting main prediction result box office
Public praise.
Further, prediction result includes the prediction result at a time point, when also may include different in a period
Between the prediction result put.
In the present embodiment, by choose whether output prediction result in main prediction result and correlation predictive as a result, can
To get the combination of relevant prediction result according to different concern demands, so that the applicability of the present embodiment is more extensive.
Fig. 4 is a kind of flow chart of neural net prediction method shown according to an exemplary embodiment, and this method can answer
For in server.As shown in figure 4, the method also includes:
Step S14 obtains the characteristic information of multiple sample object current periods respectively.
For example, the characteristic information of a certain number of sample objects is used to train prediction model, such as will
The characteristic information of the film at the known box office shown, as the characteristic information of sample object, for initial prediction model into
Row training.
Step S15, the input of the training prediction result of the characteristic information of the current period and previous cycle is initial pre-
It surveys in model and is handled, obtain the training prediction result for being directed to the multiple sample object current period.
For example, using LSTM module, the characteristic information of multiple time cycles of sample object can be handled.Wherein, exist
In the treatment process of the prediction result of current period, except the characteristic information of input current period, it is also necessary to input the previous later period
Training prediction result is jointly processed by.
Step S16, according to the training prediction result in multiple periods of the multiple sample object and the multiple period
Expectation prediction result, determine respectively the multiple sample object multiple modulus of periodicity types loss.
For example, it such as needs to calculate the training prediction result in three periods, obtains preset initial sample pair first
As, initially it is expected prediction result, by initial sample object input initial predicted model handle, obtain initial training prediction knot
Fruit.
Then the sample object of the initial training prediction result and a cycle is inputted into initial predicted adjusted
Model is handled, and the training prediction result of a cycle is obtained.
Further according to the training prediction result of a cycle and the characteristic information of second period, initial predicted model is utilized
It is handled, obtains the training prediction result of second period, and training prediction result and third according to second period
The characteristic information in a period is handled using initial predicted model, obtains the training prediction result in third period, final to obtain
To the training prediction result in each period.
Since sample object has a known actual result in each period, according to sample object each
The training prediction result and actual result in period, available initial predicted model lose in each modulus of periodicity type.
Step S17 loses according to the model, adjusts the parameters weighting in the initial predicted model, after determining adjustment
Prediction model.
For example, after above-mentioned each modulus of periodicity type loss being added, gradient is sought to all-network parameter, then basis
Back-propagation algorithm BPTT (Back Propagation Through Time) adjusts gradient, updates in initial predicted model
Parameters weighting.
In one possible implementation, it is lost according to the model, adjusts the parameter in the initial predicted model
Weight determines prediction model adjusted, comprising: is lost according to the multiple modulus of periodicity types of the multiple sample object, is successively adjusted
Parameter power in the whole MTL module, the RN module, the DNN module, the LSTM module and the insertion module
Weight, determines prediction model adjusted.
Equally by taking the training process in above three period as an example, according to the loss of the model of the period 1 got, second
The loss of modulus of periodicity type and the model of period 3 lose, and using back-propagation algorithm, are sequentially adjusted in the MTL module, described
Parameters weighting in RN module, the DNN module, the LSTM module and the insertion module, determines prediction adjusted
Model.
When prediction model carries out the repetitive exercise number for meeting setting, in each repetitive exercise, in repetition
State step.
Prediction model adjusted is determined as most by step S18 in the case where model loss meets training condition
Whole prediction model.
For example, training condition includes the repetitive exercise number of setting, and/or the condition of convergence of setting.
In the present embodiment, it is trained according to the characteristic information of different cycles and initial predicted model, obtains meeting instruction
Practice condition prediction model, obtained prediction model can preferably reflect the characteristic information of object to be predicted timing,
Relevance, and accurate prediction result can be obtained.
In the present embodiment, it is sequentially adjusted in the parameters weighting of the modules in the initial predicted model, is finally obtained
Meet the prediction model of training condition, the adjustment of the parameters weighting of modules ensure that the prediction result of prediction model output
Accuracy.
In one possible implementation, the characteristic information of the multiple object to be predicted is identical.It is different types of to
The characteristic information for predicting object needs to filter out the common characteristic information between different objects to be predicted, or will be each to be predicted
The characteristic information of object is converted into identical characteristic information.
In the present embodiment, identical characteristic information can guarantee the accuracy of the prediction result of object to be predicted.
Using example:
Below in conjunction with " predicting film A1 " property application scenarios as an example, provide according to the embodiment of the present disclosure
Application example, in order to understand the process of neural net prediction method.It will be understood by those skilled in the art that applying example below
Merely for the sake of the purpose for being easy to understand the embodiment of the present invention, it is not construed as the limitation to the embodiment of the present invention.
Figure 10 is a kind of schematic diagram of the application scenarios of neural net prediction method shown according to an exemplary embodiment.
As shown in Figure 10, prediction model (step 900) is trained using in example at this.It is applied in example at this, obtains film A1 respectively
With the characteristic information of the current period with film A2, film A1 is identical with the characteristic information of film A2, for example, film A1
Characteristic information is 300 dimensions, and the characteristic information of film A2 is 300 dimensions.In one possible implementation, when selection and film A1
There is the TV programme B1 of certain incidence relation, when as another object to be predicted, the feature of film A1 and TV programme B1 are believed
Breath has 150 dimensions identical, can be merely with the characteristic information of identical 150 dimension, as the spy for predicting film A1 and TV programme B1
Reference breath.
In application example of the film A1 and film A2 as object to be predicted, by the characteristic information of the current period with
And handled in the training prediction result input initial predicted model of previous cycle, it obtains and works as the multiple groups sample object
The training prediction result in preceding period.For example, the characteristic information of multiple time cycles of sample object is handled using LSTM module,
Wherein, in the treatment process of the prediction result of current period, except the characteristic information of input current period, it is also necessary to input previous
The training prediction result in period is jointly processed by.For example, by the characteristic information of second round and the training prediction knot of period 1
It is handled in fruit input initial predicted model, obtains the training prediction result for being directed to two groups of sample object current periods.
This using in example, according to the training prediction result in multiple periods of the multiple groups sample object and described more
The expectation prediction result in a period determines multiple modulus of periodicity types loss of the multiple groups sample object respectively.
It applies in example at this, is lost according to the model, the initial predicted model is adjusted using back-propagation algorithm
In parameters weighting, determine prediction model adjusted.For example, according to film A1 and film A2 sample object current period
Model loss, is sequentially adjusted in the MTL module, the RN module, the DNN module and the LSTM module, the insertion
Parameters weighting in module determines current period prediction model adjusted.
This using in example, in the case where model loss meets training condition, by prediction model adjusted
It is determined as final prediction model.For example, training condition includes the repetitive exercise number of setting, and/or the condition of convergence of setting.
When meeting repetitive exercise number and/or meeting the condition of convergence of setting, prediction model adjusted can be determined as finally
Prediction model.
This using in example, the characteristic information (step of the available film A1 to be predicted of server, film A2
901), and characteristic information is inputted into the prediction model and is handled (step 902), to get at least one in object to be predicted
The prediction result of a object to be predicted the, for example, prediction result (step of the prediction result of film A1 or film A1 and film A2
903).This using in example, the prediction result may include main prediction result (box office) and correlation predictive result (public praise,
Broadcasting rate etc.).
It, can be by the prediction model including multiple neural network modules, to multiple to be predicted according to the embodiment of the present disclosure
The characteristic information of object is handled, to obtain the prediction knot at least one object to be predicted in multiple objects to be predicted
Fruit improves the accuracy of the prediction result.
Figure 11 is a kind of block diagram of neural network prediction device shown according to an exemplary embodiment, as shown in figure 11,
The neural network prediction device includes:
Characteristic acquisition unit 11, for obtaining the characteristic information of multiple objects to be predicted respectively.
Prediction result acquiring unit 12, for will handle in the characteristic information input prediction model, acquisition is directed to
The prediction result of the multiple object to be predicted.Wherein, the prediction model includes shot and long term memory network LSTM module, depth
Neural network DNN module, relational network RN module and multi-task learning network MTL module.
Prediction result output unit 13, for exporting the pre- of at least one object to be predicted in the multiple object to be predicted
Survey result.
In the present embodiment, by using include shot and long term memory network LSTM module, deep neural network DNN module,
The prediction model of relational network RN module and multi-task learning network MTL module, to the characteristic information of multiple objects to be predicted
After being handled, the prediction result obtained for the multiple object to be predicted can capture more according to all aspects of this disclosure
It is timing between the characteristic information of a object to be predicted, the relevance between the characteristic information of object to be predicted, each to be predicted
Relevance between object exports the prediction result of multiple objects to be predicted simultaneously, and improves the accuracy rate of prediction result.
Figure 12 is a kind of block diagram of neural network prediction device shown according to an exemplary embodiment, in above-described embodiment
On the basis of, in one possible implementation, as shown in figure 12, prediction result acquiring unit 12 includes:
LSTM handles subelement 121, handles for inputting the characteristic information in the LSTM module, determines institute
State the shot and long term recall info of multiple objects to be predicted;
DNN handles subelement 122, handles for inputting the shot and long term recall info in the DNN module, really
The depth information of fixed the multiple object to be predicted;
RN handles subelement 123, handles, determines described more for inputting the depth information in the RN module
The relation information of a object to be predicted;
MTL handles subelement 124, handles for inputting the relation information in the MTL module, and determination is directed to
The prediction result of the multiple object to be predicted.
On the basis of the above embodiments, in one possible implementation, as shown in figure 12, the characteristic information packet
Include initial information, the prediction result acquiring unit 12 further include:
It is embedded in subelement 125, vectorization processing is carried out for the initial information to be inputted the insertion module, determines institute
State the vector information of initial information;
Information determines subelement 126, for the initial information and the vector information to be determined as the characteristic information.
In the present embodiment, it due to joined embeding layer, so that the stability of numerical representation is more preferable, more conducively captures
Similitude between characteristic information has contained current characteristic information in multiple dimensions for pre- by the vector information after training
The information content for surveying result reduces the burden that information characteristics extract engineering.
In one possible implementation, the prediction result include main prediction result and correlation predictive as a result, its
In, in the objective function of the prediction model, to the main associated parameters weighting of prediction result be greater than with it is described related
The associated parameters weighting of prediction result.
In one possible implementation, the prediction result output unit, for export it is following any one: output
The main prediction result of at least one object to be predicted in the multiple object to be predicted;It exports in the multiple object to be predicted extremely
The main prediction result and correlation predictive result of a few object to be predicted.
On the basis of the above embodiments, in one possible implementation, as shown in figure 12, the neural network is pre-
Survey device, further includes:
Sample characteristics information acquisition unit 14, for obtaining the characteristic information of multiple sample object current periods respectively.
Training prediction result acquiring unit 15, for by the training of the characteristic information of the current period and previous cycle
It is handled in prediction result input initial predicted model, obtains the training prediction for the multiple sample object current period
As a result.
Model loses determination unit 16, for according to the training prediction result in multiple periods of the multiple sample object with
And the expectation prediction result in the multiple period, multiple modulus of periodicity types loss of the multiple sample object is determined respectively.
Weight adjustment unit 17, for adjusting the parameters weighting in the initial predicted model according to model loss,
Determine prediction model adjusted.
Prediction model determination unit 18, for the model loss meet training condition in the case where, will be adjusted
Prediction model is determined as final prediction model.
In the present embodiment, it is trained according to the characteristic information of different cycles and initial predicted model, obtains meeting instruction
Practice condition prediction model, obtained prediction model can preferably reflect the characteristic information of object to be predicted timing,
Relevance, and accurate prediction result can be obtained.
In one possible implementation, the weight adjustment unit includes: weight adjustment subelement, for according to institute
State the multiple modulus of periodicity types loss of multiple sample objects, be sequentially adjusted in the MTL module, the RN module, the DNN module,
Parameters weighting in the LSTM module and the insertion module, determines prediction model adjusted.
In one possible implementation, the characteristic information of the multiple object to be predicted is identical.
In one possible implementation, a kind of neural network prediction device is also provided, comprising:
Processor;
Memory for storage processor executable instruction;
Wherein, the processor is configured to: execute the nerve net of any one of the claim of this application the method
Network prediction steps.
In one possible implementation, a kind of non-volatile computer readable storage medium storing program for executing is also provided, is stored thereon
There are computer program instructions, any one of the claim of this application is realized when the computer program instructions are executed by processor
The method.
Figure 13 is a kind of block diagram of device 1900 for neural network prediction shown according to an exemplary embodiment.Example
Such as, device 1900 may be provided as a server.Referring to Fig.1 3, device 1900 includes processing component 1922, is further wrapped
One or more processors and memory resource represented by a memory 1932 are included, it can be by processing component for storing
The instruction of 1922 execution, such as application program.The application program stored in memory 1932 may include one or one with
On each correspond to one group of instruction module.In addition, processing component 1922 is configured as executing instruction, to execute above-mentioned side
Method.
Device 1900 can also include that a power supply module 1926 be configured as the power management of executive device 1900, and one
Wired or wireless network interface 1950 is configured as device 1900 being connected to network and input and output (I/O) interface
1958.Device 1900 can be operated based on the operating system for being stored in memory 1932, such as Windows ServerTM, Mac
OS XTM, UnixTM, LinuxTM, FreeBSDTM or similar.
In the exemplary embodiment, a kind of non-volatile computer readable storage medium storing program for executing is additionally provided, for example including calculating
The memory 1932 of machine program instruction, above-mentioned computer program instructions can be executed by the processing component 1922 of device 1900 to complete
The above method.
The disclosure can be system, method and/or computer program product.Computer program product may include computer
Readable storage medium storing program for executing, containing for making processor realize the computer-readable program instructions of various aspects of the disclosure.
Computer readable storage medium, which can be, can keep and store the tangible of the instruction used by instruction execution equipment
Equipment.Computer readable storage medium for example can be-- but it is not limited to-- storage device electric, magnetic storage apparatus, optical storage
Equipment, electric magnetic storage apparatus, semiconductor memory apparatus or above-mentioned any appropriate combination.Computer readable storage medium
More specific example (non exhaustive list) includes: portable computer diskette, hard disk, random access memory (RAM), read-only deposits
It is reservoir (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable
Compact disk read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical coding equipment, for example thereon
It is stored with punch card or groove internal projection structure and the above-mentioned any appropriate combination of instruction.Calculating used herein above
Machine readable storage medium storing program for executing is not interpreted that instantaneous signal itself, the electromagnetic wave of such as radio wave or other Free propagations lead to
It crosses the electromagnetic wave (for example, the light pulse for passing through fiber optic cables) of waveguide or the propagation of other transmission mediums or is transmitted by electric wire
Electric signal.
Computer-readable program instructions as described herein can be downloaded to from computer readable storage medium it is each calculate/
Processing equipment, or outer computer or outer is downloaded to by network, such as internet, local area network, wide area network and/or wireless network
Portion stores equipment.Network may include copper transmission cable, optical fiber transmission, wireless transmission, router, firewall, interchanger, gateway
Computer and/or Edge Server.Adapter or network interface in each calculating/processing equipment are received from network to be counted
Calculation machine readable program instructions, and the computer-readable program instructions are forwarded, for the meter being stored in each calculating/processing equipment
In calculation machine readable storage medium storing program for executing.
Computer program instructions for executing disclosure operation can be assembly instruction, instruction set architecture (ISA) instructs,
Machine instruction, machine-dependent instructions, microcode, firmware instructions, condition setup data or with one or more programming languages
The source code or object code that any combination is write, the programming language include the programming language-of object-oriented such as
Smalltalk, C++ etc., and conventional procedural programming languages-such as " C " language or similar programming language.Computer
Readable program instructions can be executed fully on the user computer, partly execute on the user computer, be only as one
Vertical software package executes, part executes on the remote computer or completely in remote computer on the user computer for part
Or it is executed on server.In situations involving remote computers, remote computer can pass through network-packet of any kind
It includes local area network (LAN) or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as benefit
It is connected with ISP by internet).In some embodiments, by utilizing computer-readable program instructions
Status information carry out personalized customization electronic circuit, such as programmable logic circuit, field programmable gate array (FPGA) or can
Programmed logic array (PLA) (PLA), the electronic circuit can execute computer-readable program instructions, to realize each side of the disclosure
Face.
Referring herein to according to the flow chart of the method, apparatus (system) of the embodiment of the present disclosure and computer program product and/
Or block diagram describes various aspects of the disclosure.It should be appreciated that flowchart and or block diagram each box and flow chart and/
Or in block diagram each box combination, can be realized by computer-readable program instructions.
These computer-readable program instructions can be supplied to general purpose computer, special purpose computer or other programmable datas
The processor of processing unit, so that a kind of machine is produced, so that these instructions are passing through computer or other programmable datas
When the processor of processing unit executes, function specified in one or more boxes in implementation flow chart and/or block diagram is produced
The device of energy/movement.These computer-readable program instructions can also be stored in a computer-readable storage medium, these refer to
It enables so that computer, programmable data processing unit and/or other equipment work in a specific way, thus, it is stored with instruction
Computer-readable medium then includes a manufacture comprising in one or more boxes in implementation flow chart and/or block diagram
The instruction of the various aspects of defined function action.
Computer-readable program instructions can also be loaded into computer, other programmable data processing units or other
In equipment, so that series of operation steps are executed in computer, other programmable data processing units or other equipment, to produce
Raw computer implemented process, so that executed in computer, other programmable data processing units or other equipment
Instruct function action specified in one or more boxes in implementation flow chart and/or block diagram.
The flow chart and block diagram in the drawings show system, method and the computer journeys according to multiple embodiments of the disclosure
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
One module of table, program segment or a part of instruction, the module, program segment or a part of instruction include one or more use
The executable instruction of the logic function as defined in realizing.In some implementations as replacements, function marked in the box
It can occur in a different order than that indicated in the drawings.For example, two continuous boxes can actually be held substantially in parallel
Row, they can also be executed in the opposite order sometimes, and this depends on the function involved.It is also noted that block diagram and/or
The combination of each box in flow chart and the box in block diagram and or flow chart, can the function as defined in executing or dynamic
The dedicated hardware based system made is realized, or can be realized using a combination of dedicated hardware and computer instructions.
The presently disclosed embodiments is described above, above description is exemplary, and non-exclusive, and
It is not limited to disclosed each embodiment.Without departing from the scope and spirit of illustrated each embodiment, for this skill
Many modifications and changes are obvious for the those of ordinary skill in art field.The selection of term used herein, purport
In the principle, practical application or technological improvement to the technology in market for best explaining each embodiment, or lead this technology
Other those of ordinary skill in domain can understand each embodiment disclosed herein.
Claims (18)
1. a kind of neural net prediction method, which is characterized in that the described method includes:
The characteristic information of multiple objects to be predicted is obtained respectively;
It will be handled in the characteristic information input prediction model, obtain the prediction knot for being directed to the multiple object to be predicted
Fruit;
The prediction result of at least one object to be predicted in the multiple object to be predicted is exported,
Wherein, the prediction model includes shot and long term memory network LSTM module, deep neural network DNN module, relational network
RN module and multi-task learning network MTL module.
2. the method according to claim 1, wherein by the characteristic information input prediction model
Reason obtains the prediction result for being directed to the multiple object to be predicted, comprising:
The characteristic information is inputted in the LSTM module and is handled, determines the shot and long term note of the multiple object to be predicted
Recall information;
The shot and long term recall info is inputted in the DNN module and is handled, determines the depth of the multiple object to be predicted
Spend information;
The depth information is inputted in the RN module and is handled, determines the relation information of the multiple object to be predicted;
The relation information is inputted in the MTL module and is handled, determines the prediction for being directed to the multiple object to be predicted
As a result.
3. according to the method described in claim 2, it is characterized in that, the prediction model further include insertion module,
Wherein, it will handle, obtained for the pre- of the multiple object to be predicted in the characteristic information input prediction model
Survey result, further includes:
Initial information is inputted into the insertion module and carries out vectorization processing, determines the vector information of the initial information;
The initial information and the vector information are determined as the characteristic information.
4. the method according to claim 1, wherein the prediction result includes:
Main prediction result and correlation predictive as a result,
Wherein, in the objective function of the prediction model, parameters weighting associated with the main prediction result is greater than and institute
State the associated parameters weighting of correlation predictive result.
5. according to the method described in claim 4, it is characterized in that, at least one in the multiple object to be predicted of output wait for it is pre-
The prediction result for surveying object, includes any of the following:
Export the main prediction result of at least one object to be predicted in the multiple object to be predicted;
Export the main prediction result and correlation predictive result of at least one object to be predicted in the multiple object to be predicted.
6. method according to claim 1 or 3, which is characterized in that the method also includes:
The characteristic information of multiple sample object current periods is obtained respectively;
The training prediction result of the characteristic information of the current period and previous cycle is inputted in initial predicted model and is carried out
Processing obtains the training prediction result for being directed to the multiple sample object current period;
According to the training prediction result in multiple periods of the multiple sample object and the expectation in the multiple period prediction knot
Fruit determines multiple modulus of periodicity types loss of the multiple sample object respectively;
It is lost according to the model, adjusts the parameters weighting in the initial predicted model, determine prediction model adjusted;
In the case where model loss meets training condition, prediction model adjusted is determined as to final prediction mould
Type.
7. according to the method described in claim 6, it is characterized in that, adjusting the initial predicted mould according to model loss
Parameters weighting in type determines prediction model adjusted, comprising:
According to the multiple modulus of periodicity type losses of the multiple sample object, it is sequentially adjusted in the MTL module, the RN module, institute
The parameters weighting in DNN module, the LSTM module and the insertion module is stated, determines prediction model adjusted.
8. the method according to claim 1, wherein the characteristic information of the multiple object to be predicted is identical.
9. a kind of neural network prediction device, which is characterized in that described device includes:
Characteristic acquisition unit, for obtaining the characteristic information of multiple objects to be predicted respectively;
Prediction result acquiring unit is obtained for will handle in the characteristic information input prediction model for described more
The prediction result of a object to be predicted;
Prediction result output unit, for exporting the prediction knot of at least one object to be predicted in the multiple object to be predicted
Fruit,
Wherein, the prediction model includes shot and long term memory network LSTM module, deep neural network DNN module, relational network
RN module and multi-task learning network MTL module.
10. device according to claim 9, which is characterized in that the prediction result acquiring unit includes:
LSTM handles subelement, handles, determines the multiple for inputting the characteristic information in the LSTM module
The shot and long term recall info of object to be predicted;
DNN handles subelement, handle for inputting the shot and long term recall info in the DNN module, described in determination
The depth information of multiple objects to be predicted;
RN handles subelement, handles, determines the multiple to pre- for inputting the depth information in the RN module
Survey the relation information of object;
MTL handles subelement, handles for inputting the relation information in the MTL module, determines for described more
The prediction result of a object to be predicted.
11. device according to claim 10, which is characterized in that the prediction model further includes insertion module,
The prediction result acquiring unit further include:
It is embedded in subelement, vectorization processing is carried out for initial information to be inputted the insertion module, determines the initial information
Vector information;
Information determines subelement, for the initial information and the vector information to be determined as the characteristic information.
12. device according to claim 9, which is characterized in that the prediction result includes:
Main prediction result and correlation predictive as a result,
Wherein, in the objective function of the prediction model, parameters weighting associated with the main prediction result is greater than and institute
State the associated parameters weighting of correlation predictive result.
13. device according to claim 12, which is characterized in that the prediction result output unit is for exporting following
It anticipates one kind:
Export the main prediction result of at least one object to be predicted in the multiple object to be predicted;
Export the main prediction result and correlation predictive result of at least one object to be predicted in the multiple object to be predicted.
14. the device according to claim 9 or 11, which is characterized in that further include:
Sample characteristics information acquisition unit, for obtaining the characteristic information of multiple sample object current periods respectively;
Training prediction result acquiring unit, for tying the training prediction of the characteristic information of the current period and previous cycle
It is handled in fruit input initial predicted model, obtains the training prediction result for being directed to the multiple sample object current period;
Model loses determination unit, for according to the training prediction result in multiple periods of the multiple sample object and described
The expectation prediction result in multiple periods determines multiple modulus of periodicity types loss of the multiple sample object respectively;
Weight adjustment unit adjusts the parameters weighting in the initial predicted model, determines and adjust for being lost according to the model
Prediction model after whole;
Prediction model determination unit, for the model loss meet training condition in the case where, by prediction mould adjusted
Type is determined as final prediction model.
15. device according to claim 14, which is characterized in that the weight adjustment unit includes:
Weight adjusts subelement, for losing according to the multiple modulus of periodicity types of the multiple sample object, is sequentially adjusted in the MTL
Parameters weighting in module, the RN module, the DNN module, the LSTM module and the insertion module, determines adjustment
Prediction model afterwards.
16. device according to claim 9, which is characterized in that the characteristic information of the multiple object to be predicted is identical.
17. a kind of neural network prediction device characterized by comprising
Processor;
Memory for storage processor executable instruction;
Wherein, the processor is configured to: perform claim require any one of 1 to 8 described in neural net prediction method.
18. a kind of non-volatile computer readable storage medium storing program for executing, is stored thereon with computer program instructions, which is characterized in that institute
It states and realizes method described in any one of claim 1 to 8 when computer program instructions are executed by processor.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710960503.7A CN109670622A (en) | 2017-10-16 | 2017-10-16 | Neural net prediction method and device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710960503.7A CN109670622A (en) | 2017-10-16 | 2017-10-16 | Neural net prediction method and device |
Publications (1)
Publication Number | Publication Date |
---|---|
CN109670622A true CN109670622A (en) | 2019-04-23 |
Family
ID=66140114
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710960503.7A Pending CN109670622A (en) | 2017-10-16 | 2017-10-16 | Neural net prediction method and device |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109670622A (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110321601A (en) * | 2019-06-14 | 2019-10-11 | 山东大学 | A kind of overhead transmission line dynamic current-carrying capability advanced prediction method and system |
CN112561174A (en) * | 2020-12-18 | 2021-03-26 | 西南交通大学 | Method for predicting geothermal energy production based on LSTM and MLP superimposed neural network |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106960252A (en) * | 2017-03-08 | 2017-07-18 | 深圳市景程信息科技有限公司 | Methods of electric load forecasting based on long Memory Neural Networks in short-term |
-
2017
- 2017-10-16 CN CN201710960503.7A patent/CN109670622A/en active Pending
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106960252A (en) * | 2017-03-08 | 2017-07-18 | 深圳市景程信息科技有限公司 | Methods of electric load forecasting based on long Memory Neural Networks in short-term |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110321601A (en) * | 2019-06-14 | 2019-10-11 | 山东大学 | A kind of overhead transmission line dynamic current-carrying capability advanced prediction method and system |
CN112561174A (en) * | 2020-12-18 | 2021-03-26 | 西南交通大学 | Method for predicting geothermal energy production based on LSTM and MLP superimposed neural network |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111966698B (en) | Block chain-based trusted federation learning method, system, device and medium | |
CN109670569A (en) | Neural net prediction method and device | |
Li et al. | Emergent world representations: Exploring a sequence model trained on a synthetic task | |
Rere et al. | Metaheuristic algorithms for convolution neural network | |
CN109670621A (en) | Neural net prediction method and device | |
CN109670623A (en) | Neural net prediction method and device | |
Zhong et al. | Density-based evolutionary framework for crowd model calibration | |
CN109960761A (en) | Information recommendation method, device, equipment and computer readable storage medium | |
CN109325516A (en) | A kind of integrated learning approach and device towards image classification | |
CN109670567A (en) | Neural net prediction method and device | |
Jafar et al. | Hyperparameter optimization for deep residual learning in image classification | |
Xue et al. | Attention‐based spatial–temporal hierarchical ConvLSTM network for action recognition in videos | |
CN109670622A (en) | Neural net prediction method and device | |
Broad et al. | Amplifying the uncanny | |
Pan et al. | VA-RED $^ 2$: Video Adaptive Redundancy Reduction | |
CN109670590A (en) | Neural net prediction method and device | |
CN109670571A (en) | Neural net prediction method and device | |
CN109670588A (en) | Neural net prediction method and device | |
CN109670572A (en) | Neural net prediction method and device | |
Li | Visual classification of music style transfer based on PSO-BP rating prediction model | |
Jain et al. | Soccer result prediction using deep learning and neural networks | |
Ji et al. | Improving decision-making efficiency of image game based on deep Q-learning | |
CN109670568A (en) | Neural net prediction method and device | |
WO2023235346A1 (en) | Prompting machine-learned models using chains of thought | |
US20230394328A1 (en) | Prompting Machine-Learned Models Using Chains of Thought |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
TA01 | Transfer of patent application right | ||
TA01 | Transfer of patent application right |
Effective date of registration: 20200610 Address after: 310052 room 508, floor 5, building 4, No. 699, Wangshang Road, Changhe street, Binjiang District, Hangzhou City, Zhejiang Province Applicant after: Alibaba (China) Co.,Ltd. Address before: 100080 Beijing Haidian District city Haidian street A Sinosteel International Plaza No. 8 block 5 layer A, C Applicant before: Youku network technology (Beijing) Co.,Ltd. |
|
RJ01 | Rejection of invention patent application after publication | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190423 |