CN107992938B - Space-time big data prediction technique and system based on positive and negative convolutional neural networks - Google Patents
Space-time big data prediction technique and system based on positive and negative convolutional neural networks Download PDFInfo
- Publication number
- CN107992938B CN107992938B CN201711192977.8A CN201711192977A CN107992938B CN 107992938 B CN107992938 B CN 107992938B CN 201711192977 A CN201711192977 A CN 201711192977A CN 107992938 B CN107992938 B CN 107992938B
- Authority
- CN
- China
- Prior art keywords
- convolution
- moment
- deconvolution
- positive
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Game Theory and Decision Science (AREA)
- Development Economics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Operations Research (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Complex Calculations (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention provides a kind of space-time big data prediction technique and system based on positive and negative convolutional neural networks, and method includes: that space-time big data is inputted trained positive and negative convolutional neural networks model, obtains prediction result；Trained positive and negative convolutional neural networks model obtains as follows: obtaining the output of any moment convolution according to the memory of any moment convolution and any moment convolution out gate, constructs any moment convolution length time memory unit；Any moment deconvolution output is obtained according to any moment deconvolution memory and any moment deconvolution out gate, constructs any moment deconvolution length time memory unit；Build positive and negative convolutional neural networks model；The tensor sequence data being made of observation is inputted positive and negative convolutional neural networks model to be trained, obtains trained positive and negative convolutional neural networks model.The data that the present invention is observed in the past by analysis and study, learn the hidden feature of space-time data, predict following space-time big data sequence.
Description
Technical field
The present invention relates to computer data analysis field, more particularly, to a kind of based on positive and negative convolutional neural networks
Space-time big data prediction technique and system.
Background technique
The research of data mining is realized by modeling and is hidden in mass data based on the characteristic of data itself
Information utilizes, and therefore, can the various implicit connections that sufficiently capture in data be the important mark for evaluating model superiority and inferiority
It is quasi-.As the biology lived in time and space, the number of two kinds of dimension hierarchies of time and space can be collected into while had
According to.Such as when being embodied not only in a certain specific time in precipitation data, the space of Rainfall distribution situation point in a certain range
Cloth data are also included in the precipitation Annual distribution data of a certain particular spatial location.If only from Spatial Dimension or when
Between dimension analyze data, necessarily will cause significant ground information loss.
A period of time recently, the research of the field of data mining forefront are being related to merely time dimension or space dimension
Breakthrough is realized on the problem of spending, such as using recurrent neural network as the data time series analysis method of representative, He Yijuan
Product neural network is the spatial data analysis method of representative.But for by time data together with spatial data integration into
The data digging method of row analysis is also far from reaching expected.Meanwhile be for the demand of space-time data analysis method it is huge,
The true application scenarios such as weather forecast, visual classification, image prediction need to handle space-time data.
Summary of the invention
The present invention provides a kind of a kind of space-time big data prediction based on positive and negative convolutional neural networks for overcoming the above problem
Method and system.
According to an aspect of the present invention, a kind of space-time big data prediction side based on positive and negative convolutional neural networks is provided
Method, comprising: space-time big data is inputted into trained positive and negative convolutional neural networks model, obtains prediction result；Wherein, the instruction
The positive and negative convolutional neural networks model perfected obtains as follows: according to the memory of any moment convolution and any moment convolution
Out gate obtains the output of any moment convolution, and based on any moment convolution output building any moment convolution long short time
Memory unit；Any moment deconvolution output is obtained according to any moment deconvolution memory and any moment deconvolution out gate,
And based on any moment deconvolution output building any moment deconvolution length time memory unit；It is long by all convolution
Short time memory unit and all deconvolution length time memory units build the positive and negative convolutional neural networks model, wherein
The positive and negative convolutional neural networks model is using moment and layer as the two dimensional model of dimension, and each layer be equal in length is each
Moment is equal in length；The tensor sequence data being made of the observation input positive and negative convolutional neural networks model is carried out
Training obtains the trained positive and negative convolutional neural networks model.
Preferably, described to be taken by all convolution length time memory units and all deconvolution length time memory units
Building the positive and negative convolutional neural networks model further comprises: all convolution length time memory units are placed in the positive warp
All deconvolution length time memory units are placed in the positive and negative convolutional neural networks mould by the lower level of product neural network model
The higher level of type, wherein the number of plies locating for any deconvolution length time memory unit is longer in short-term than any convolution
Between the number of plies locating for memory unit it is high, and the quantity of the convolution length time memory unit and the deconvolution long short time are remembered
The quantity for recalling unit is equal.
Preferably, any moment convolution memory is obtained by following steps: being obtained by the following formula convolution input
Door:
Wherein, itFor convolution input gate, σ is S type nonlinear activation function Sigmoid, WxiWhen to calculate convolution input gate
WithThe convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhiTo calculate convolution input
Men HeThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, WciTo calculate convolution
When input gate andThe convolution kernel of convolution operation is done, ⊙ is Hadamard operator,For the volume of the last moment of any moment
Product memory, biFor convolution input gate deviation；
It is obtained by the following formula convolution and forgets door:
Wherein, ftDoor is forgotten for convolution, and σ is S type nonlinear activation function Sigmoid, WxfWhen forgeing door to calculate convolution
WithThe convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhfForget to calculate convolution
Men HeThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, WcfTo calculate convolution
Forget door when andThe convolution kernel of convolution operation is done, ⊙ is Hadamard operator,For the volume of the last moment of any moment
Product memory, bfDoor deviation is forgotten for convolution；
The input quantity of any moment and the output of the convolution of the last moment of any moment are integrated into one by following formula
A convolution tensor:
Wherein, gtFor convolution tensor, φ is tanh activation primitive tangent, WxgFor calculate convolution tensor when and
The convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhgTo calculate convolution tensor sumThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, bgFor convolution tensor deviation；
Door and the convolution tensor are forgotten according to the convolution input gate, the convolution, are obtained by following formula described any
The memory of moment convolution:
Wherein,For the memory of any moment convolution, ftDoor is forgotten for convolution, ⊙ is Hadamard operator,When being any
The convolution of the last moment at quarter is remembered, itFor convolution input gate, gtFor convolution tensor.
Preferably, described defeated according to the memory of any moment convolution and any moment convolution out gate acquisition any moment convolution
Further comprise out: being obtained by the following formula any moment convolution out gate:
Wherein, otFor any moment convolution out gate, σ is S type nonlinear activation function Sigmoid, WxoTo calculate convolution
When out gate andThe convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhoTo calculate
When convolution out gate andThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, Wco
For calculate convolution out gate when andThe convolution kernel of convolution operation is done, ⊙ is Hadamard operator,For the upper of any moment
The convolution at one moment is remembered, bfFor convolution out gate deviation；
Remembered according to any moment convolution out gate and any moment convolution, any moment convolution is obtained by following formula
Output:
Wherein,For the output of any moment convolution, otFor any moment convolution out gate, φ is that tanh activates letter
Number tangent, ⊙ are Hadamard operator,For the memory of any moment convolution.
Preferably, any moment deconvolution memory is obtained by following steps: being obtained by the following formula deconvolution
Input gate:
Wherein, itFor deconvolution input gate, σ is S type nonlinear activation function Sigmoid, WxiTo calculate deconvolution input
Door when andThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,For the defeated of any moment
Enter amount, WhiFor calculate deconvolution input gate when andThe convolution kernel of convolution operation is done,For upper a period of time of any moment
The deconvolution at quarter exports, biFor deconvolution input gate deviation；
It is obtained by the following formula deconvolution and forgets door:
Wherein, ftDoor is forgotten for deconvolution, and σ is S type nonlinear activation function Sigmoid, WxfForget to calculate deconvolution
Door when andThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,For the defeated of any moment
Enter amount, WhfFor calculate deconvolution forget door andThe convolution kernel of convolution operation is done,For the last moment of any moment
Deconvolution output, bfDoor deviation is forgotten for deconvolution；
The input quantity of any moment and the output of the convolution of the last moment of any moment are integrated into one by following formula
A deconvolution tensor:
Wherein, gtFor deconvolution tensor, φ is tanh activation primitive tangent, WxgWhen to calculate deconvolution tensor
WithThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,For the input quantity of any moment,
WhgTo calculate deconvolution tensor sumThe convolution kernel of convolution operation is done,For the warp of the last moment of any moment
Product output, bgFor deconvolution tensor deviation；
Door and the deconvolution tensor are forgotten according to the deconvolution input gate, the deconvolution, and institute is obtained by following formula
State any moment deconvolution memory:
Wherein,For any moment deconvolution memory, ftDoor is forgotten for deconvolution, ⊙ is Hadamard operator,To appoint
The deconvolution of the last moment at one moment is remembered, itFor deconvolution input gate, gtFor deconvolution tensor.
Preferably, described anti-according to any moment deconvolution memory and any moment deconvolution out gate acquisition any moment
Convolution output further comprises:
It is obtained by the following formula any moment deconvolution out gate:
Wherein, otFor any moment deconvolution out gate, σ is S type nonlinear activation function Sigmoid, WxoIt is anti-to calculate
When convolution out gate andThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,It is any
The input quantity at moment, WhoFor calculate deconvolution out gate when andThe convolution kernel of convolution operation is done,For any moment
Last moment deconvolution output, boFor deconvolution out gate deviation；
Remembered according to any moment deconvolution out gate and any moment deconvolution, any moment is obtained by following formula
Deconvolution output:
Wherein,For any moment deconvolution output, otFor any moment deconvolution out gate, φ swashs for tanh
Function tangent living, ⊙ are Hadamard operator,For any moment deconvolution memory.
Preferably, it is described by the tensor sequence data being made of observation input the positive and negative convolutional neural networks model into
Row training, obtaining the trained positive and negative convolutional neural networks model further comprises: S1, the tensor that will be made of observation
The first moment tensor sequence data in sequence data inputs the positive and negative convolutional neural networks model；S2, in the positive warp
The first layer of product neural network model extracts corresponding informance by the first moment first layer convolution length time memory unit, and will
The corresponding informance is transferred to lower layer of subsequent time of convolution length time memory unit, wherein the corresponding informance includes
The corresponding convolution output of first moment first layer convolution length time memory unit and the memory of the first moment convolution, acquisition are described just
Prediction result of the deconvolution neural network model in the first moment first layer；S3, the number of plies for extracting corresponding informance is added one, continued
Step S2 is executed, until the number of plies locating for the corresponding informance extracted is equal to total layer comprising the convolution length time memory unit
Number, obtaining remaining of the positive and negative convolutional neural networks model in the first moment in addition to the first layer includes that convolution is long in short-term
Between memory unit layer prediction result；S4, the number of plies locating for the corresponding informance of extraction are equal to long in short-term comprising the convolution
Between memory unit total number of plies when, the number of plies for extracting corresponding informance is added one, the long short time note of the convolution as described in the first moment the
The total number of plies for recalling unit adds one layer of deconvolution length time memory unit to extract corresponding informance, and the corresponding informance is transferred to
Lower layer of subsequent time of deconvolution length time memory unit obtains the positive and negative convolutional neural networks model at the first moment
Total number of plies of the convolution length time memory unit adds one layer of prediction result；S5, the number of plies for extracting corresponding informance is added
One, step S4 is continuously carried out, until the number of plies locating for the corresponding informance extracted is equal to comprising the convolution length time memory list
Total number of plies of member and the sum of total number of plies of the deconvolution length time memory unit, obtain the positive and negative convolutional neural networks mould
Type all layers of prediction result in the first moment.
Preferably, after step S5 further include: S6, by the subsequent time tensor in the tensor sequence data being made of observation
Sequence data inputs the positive and negative convolutional neural networks model, and circulation executes step S2 to S5, until locating for the information extracted
Moment number is greater than or equal to the length of tensor sequence data in the past, obtains the positive and negative convolutional neural networks model and is extracting pair
Each layer of prediction result at the time of answering information；Number is greater than or equal at the time of S7, judgement are worked as locating for the corresponding informance extracted
When the length of past tensor sequence data, at this time at the time of number and fiducial value size, the fiducial value be it is described in the past tensor
Two times of the length of sequence data；Wherein, the length of the tensor sequence data in the past be it is described at this time at the time of number upper one
The numerical value of moment number；If S8, it is described at this time at the time of number be greater than or equal to the fiducial value, by loss function acquisition described in
The gap of prediction result and actual result, and according to back-propagation algorithm to the tensor sequence data being made of observation into
Row updates, and circulation executes step S2 to S7, until the gap of prediction result and actual result is lower than disparity threshold.
Preferably, step S6 further include: if it is described at this time at the time of number be less than the fiducial value, the positive deconvolution mind
It exports through network model to the prediction result at this time, and the prediction result is inputted into the positive and negative convolutional neural networks mould
Type, circulation execute step S2 to S7, and number is greater than or equal to the fiducial value at the time of described at this time.
According to another aspect of the present invention, a kind of space-time big data prediction system based on positive and negative convolutional neural networks is provided
System, comprising: obtain prediction result module, for space-time big data to be inputted trained positive and negative convolutional neural networks model, obtain
Take prediction result；Wherein, the trained positive and negative convolutional neural networks model is obtained by following submodule: building convolution is long
Short time memory unit submodule, for obtaining any moment according to the memory of any moment convolution and any moment convolution out gate
Convolution output, and based on any moment convolution output building any moment convolution length time memory unit；Construct warp
Product length time memory unit submodule, for being obtained according to any moment deconvolution memory and any moment deconvolution out gate
Any moment deconvolution output, and based on any moment deconvolution output building any moment deconvolution length time memory
Unit；Model submodule is built, for passing through all convolution length time memory units and all deconvolution length time memories
Unit builds the positive and negative convolutional neural networks model, wherein the positive and negative convolutional neural networks model is to be with moment and layer
The two dimensional model of dimension, each layer is equal in length, and each moment is equal in length；Acquisition trains model submodule,
Tensor sequence data for will be made of observation inputs the positive and negative convolutional neural networks model and is trained, described in acquisition
Trained positive and negative convolutional neural networks model.
A kind of space-time big data prediction technique and system based on positive and negative convolutional neural networks provided by the invention, uses volume
Product length time memory unit and deconvolution length time memory unit can capture dynamic on time and Spatial Dimension simultaneously
Mechanical information learns the hidden feature of space-time data by analyzing and learning the data observed in the past, predicts future
Space-time big data sequence.This is all extremely helpful for the application of the space-time datas such as video monitoring, weather prognosis, smart city
's.
Detailed description of the invention
Fig. 1 is the flow chart of the trained positive and negative convolutional neural networks model of acquisition in the embodiment of the present invention；
Fig. 2 is the structural schematic diagram of the positive and negative convolutional neural networks model of one of the embodiment of the present invention；
Fig. 3 is the structural schematic diagram of one of embodiment of the present invention convolution length time memory unit；
Fig. 4 is the structural schematic diagram of one of embodiment of the present invention deconvolution length time memory unit；
Fig. 5 is the flow chart that one of embodiment of the present invention obtains trained positive and negative convolutional neural networks model；
Fig. 6 is Moving Mnist data sample and prediction sample figure in the embodiment of the present invention.
Specific embodiment
With reference to the accompanying drawings and examples, specific embodiments of the present invention will be described in further detail.Implement below
Example is not intended to limit the scope of the invention for illustrating the present invention.
Fig. 1 is the flow chart of the trained positive and negative convolutional neural networks model of acquisition in the embodiment of the present invention, such as Fig. 1 institute
Show, space-time big data is inputted into trained positive and negative convolutional neural networks model, obtains prediction result；Wherein, described to train
Positive and negative convolutional neural networks model obtain as follows: according to any moment convolution memory and any moment convolution export
Door obtains the output of any moment convolution, and based on any moment convolution output building any moment convolution length time memory
Unit；Any moment deconvolution output, and base are obtained according to any moment deconvolution memory and any moment deconvolution out gate
Building any moment deconvolution length time memory unit is exported in any moment deconvolution；It is long in short-term by all convolution
Between memory unit and all deconvolution length time memory units build the positive and negative convolutional neural networks model, wherein it is described
Positive and negative convolutional neural networks model is using moment and layer as the two dimensional model of dimension, and each layer is equal in length, each moment
Be equal in length；The tensor sequence data being made of observation is inputted the positive and negative convolutional neural networks model to instruct
Practice, obtains the trained positive and negative convolutional neural networks model.
Specifically, specific explanation is made to the length time memory network in the present embodiment below.
Sequence length has the problem of gap in data to solve the problems, such as sequence, and those skilled in the art devises circulation
Neural network (recurrent neural network, RNN) carrys out processing sequence problem.But there are two to ask by common RNN
Topic.First is that long-distance dependence, second is that gradient disappears and gradient explosion, this problem are particularly evident when handling long sequence.
In order to solve problem above, those skilled in the art proposes length time memory network (Long Short-
Term Memory, LSTM).This RNN framework disappears and gradient explosion issues dedicated for solving the gradient of RNN model.By three
The state of activation of a multiplication gate control block of memory: input gate (input gate), forgets door again at out gate (output gate)
Claim to forget door (forget gate).This structure can be allowed to the preceding information preservation inputted in a network, and the biography that goes ahead
It passs, input gate, which opens stylish input, can just change the historic state saved in network, the history shape that out gate saves when opening
State can be accessed to, and the output after influencing, forget for emptying previously stored historical information.
Further, in this embodiment the tensor sequence data being made of observation proposed is handled based on following thought
: data are pre-processed first, obtain the input of positive and negative convolutional neural networks model.Assuming that constantly monitoring a dynamical system
The P kind measurement amount (such as: temperature) of system (such as: Atmosphere System), and every kind of measurement within the scope of certain space (such as: in
State) every bit have record, be expressed as the matrix of a M × N.For the angle in space, the P measurement amount of any time
It can be expressed as a tensor X ∈ RP×M×N.From the point of view of time angle, a tensor sequence that observation is made of T timestamp
It should also be noted that, any moment convolution length time memory unit and the long short time note of any moment deconvolution
At the time of recalling the correspondence that unit is in positive and negative convolutional neural networks model, and in a moment, each layer has and only one
A convolution length time memory unit or deconvolution length time memory unit.
A kind of space-time big data prediction technique provided by the invention is long using convolution length time memory unit and deconvolution
Short time memory unit can capture the dynamic information on time and Spatial Dimension simultaneously, be seen in the past by analyzing and learning
The data measured learn the hidden feature of space-time data, predict following space-time big data sequence.This supervises video
The application of the space-time datas such as control, weather prognosis, wisdom city city is all extremely helpful.
On the basis of the above embodiments, the present embodiment makes further the structure of positive and negative convolutional neural networks model
Explanation.Fig. 2 is the structural schematic diagram of the positive and negative convolutional neural networks model of one of the embodiment of the present invention, and the present embodiment please refers to
Fig. 2.
It is described built by all convolution length time memory units and all deconvolution length time memory units it is described
Positive and negative convolutional neural networks model further comprises: all convolution length time memory units are placed in the positive and negative convolutional Neural
The lower level of network model, by all deconvolution length time memory units be placed in the positive and negative convolutional neural networks model compared with
It is high-rise, wherein the number of plies locating for any deconvolution length time memory unit is than any convolution length time memory
The number of plies locating for unit is high, and the quantity of the convolution length time memory unit and the deconvolution length time memory unit
Quantity it is equal.
It should be noted that positive and negative convolutional neural networks model perseverance is even level, convolution length time memory unit and anti-
Convolution length time memory unit is symmetric.For example, positive and negative convolutional neural networks model shares 6 layers, three first layers point
Multiple convolution length time memory units are furnished with, latter three layers are distributed with multiple deconvolution length time memory units.
It should also be noted that, in positive and negative convolutional neural networks model, any moment convolution length time memory unit
At the time of remembering corresponding in respective convolution at the time of number.Any moment deconvolution length time memory unit is in respective warp
At the time of product memory is corresponding at the time of number.
Further, positive and negative convolutional neural networks model is end to end.
A kind of space-time big data prediction technique provided by the invention, since positive and negative convolutional neural networks model uses multilayer
The mode that convolution length time memory elementary layer and deconvolution length time memory elementary layer stack, is provided with model stronger
Ability to express is allowed to be more suitable for spatio-temporal prediction.
Based on the above embodiment, the present embodiment is made acquisition any moment convolution memory and is illustrated below.Fig. 3 is
The structural schematic diagram of one of embodiment of the present invention convolution length time memory unit, the present embodiment please refer to Fig. 3.In Fig. 3
Hidden state refer to convolution export.
It is obtained by the following formula convolution input gate:
Wherein, itFor convolution input gate, σ is S type nonlinear activation function Sigmoid, WxiWhen to calculate convolution input gate
WithThe convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhiTo calculate convolution input
Men HeThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, WciTo calculate convolution
When input gate andThe convolution kernel of convolution operation is done, ⊙ is Hadamard operator,For the volume of the last moment of any moment
Product memory, biFor convolution input gate deviation.
It is obtained by the following formula convolution and forgets door:
Wherein, ftDoor is forgotten for convolution, and σ is S type nonlinear activation function Sigmoid, WxfWhen forgeing door to calculate convolution
WithThe convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhfForget to calculate convolution
Men HeThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, WcfTo calculate convolution
Forget door when andThe convolution kernel of convolution operation is done, ⊙ is Hadamard operator,For the volume of the last moment of any moment
Product memory, bfDoor deviation is forgotten for convolution.
The input quantity of any moment and the output of the convolution of the last moment of any moment are integrated into one by following formula
A convolution tensor:
Wherein, gtFor convolution tensor, φ is tanh activation primitive tangent, WxgFor calculate convolution tensor when and
The convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhgTo calculate convolution tensor sumThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, bgFor convolution tensor deviation.
Door and the convolution tensor are forgotten according to the convolution input gate, the convolution, are obtained by following formula described any
The memory of moment convolution:
Wherein,For the memory of any moment convolution, ftDoor is forgotten for convolution, ⊙ is Hadamard operator,When being any
The convolution of the last moment at quarter is remembered, itFor convolution input gate, gtFor convolution tensor.
Specifically, convolution input gate is used to determine the convolution output of the last moment of any moment input quantity and any moment
In which information can be added into the memory of convolution length time memory unit.Convolution forgets door for determining any moment
Which information for being stored in the memory of convolution length time memory unit of last moment need to be retained.
Further, in the present embodiment, Sigmoid is the function of a common S type in biology, also referred to as S
Sigmoid growth curve.In information science, since singly properties, the Sigmoid function such as increasing and the increasing of inverse function list are often used as mind for it
Threshold function table through network, by variable mappings to 0, between 1.
Further, convolution is the common method of image procossing, gives input picture, in the output image each pixel
It is the weighted average of pixel in a zonule in input picture, wherein weight is defined by a function, this function is known as rolling up
Product core.
A kind of space-time big data prediction technique provided by the invention, by the process for obtaining the memory of any moment convolution
In, there is multiple convolution calculating operation, convolution length time memory unit is enabled to carry out dimensionality reduction for higher-dimension sequence data.
Based on the above embodiment, obtain any moment convolution memory after, need according to any moment convolution remember and
Any moment convolution out gate obtains the output of any moment convolution, and then based on any moment convolution output building any moment volume
Product length time memory unit.The present embodiment is made update any moment convolution output and being illustrated.The present embodiment is also asked
Refering to Fig. 3.
It is obtained by the following formula any moment convolution out gate:
Wherein, otFor any moment convolution out gate, σ is S type nonlinear activation function Sigmoid, WxoTo calculate convolution
When out gate andThe convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhoTo calculate
When convolution out gate andThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, Wco
For calculate convolution out gate when andThe convolution kernel of convolution operation is done, ⊙ is Hadamard operator,For the upper of any moment
The convolution at one moment is remembered, bfFor convolution out gate deviation.
Remembered according to any moment convolution out gate and any moment convolution, any moment convolution is obtained by following formula
Output:
Wherein,For the output of any moment convolution, otFor any moment convolution out gate, φ is that tanh activates letter
Number tangent, ⊙ are Hadamard operator,For the memory of any moment convolution.
Specifically, convolution out gate is used to determine in the memory of convolution length time memory unit which information can be by conduct
As a result it exports.
A kind of space-time big data prediction technique provided by the invention, by the process for updating the output of any moment convolution
In, there is multiple convolution calculating operation, convolution length time memory unit is enabled to carry out dimensionality reduction for higher-dimension sequence data.
Based on the above embodiment, the present embodiment is made acquisition any moment deconvolution memory and is illustrated below.Fig. 4
For the structural schematic diagram of one of embodiment of the present invention deconvolution length time memory unit, the present embodiment please refers to Fig. 4.Figure
Hidden state in 4 refers to that deconvolution exports.
It is obtained by the following formula deconvolution input gate:
Wherein, itFor deconvolution input gate, σ is S type nonlinear activation function Sigmoid, WxiTo calculate deconvolution input
Door when andThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,For the defeated of any moment
Enter amount, WhiFor calculate deconvolution input gate when andThe convolution kernel of convolution operation is done,For upper a period of time of any moment
The deconvolution at quarter exports, biFor deconvolution input gate deviation.
It is obtained by the following formula deconvolution and forgets door:
Wherein, ftDoor is forgotten for deconvolution, and σ is S type nonlinear activation function Sigmoid, WxfForget to calculate deconvolution
Door when andThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,For the defeated of any moment
Enter amount, WhfFor calculate deconvolution forget door andThe convolution kernel of convolution operation is done,For the last moment of any moment
Deconvolution output, bfDoor deviation is forgotten for deconvolution.
The input quantity of any moment and the output of the convolution of the last moment of any moment are integrated into one by following formula
A deconvolution tensor:
Wherein, gtFor deconvolution tensor, φ is tanh activation primitive tangent, WxgWhen to calculate deconvolution tensor
WithThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,For the input quantity of any moment,
WhgTo calculate deconvolution tensor sumThe convolution kernel of convolution operation is done,For the warp of the last moment of any moment
Product output, bgFor deconvolution tensor deviation.
Door and the deconvolution tensor are forgotten according to the deconvolution input gate, the deconvolution, and institute is obtained by following formula
State any moment deconvolution memory:
Wherein,For any moment deconvolution memory, ftDoor is forgotten for deconvolution, ⊙ is Hadamard operator,To appoint
The deconvolution of the last moment at one moment is remembered, itFor deconvolution input gate, gtFor deconvolution tensor.
Specifically, deconvolution input gate is used to determine the deconvolution of the last moment of any moment input quantity and any moment
Which information in output can be added into the memory of deconvolution length time memory unit.Deconvolution forgets door for determining
Which information that the last moment of any moment is stored in the memory of deconvolution length time memory unit needs to be retained.
Further, in the present embodiment, Sigmoid is the function of a common S type in biology, also referred to as S
Sigmoid growth curve.In information science, since singly properties, the Sigmoid function such as increasing and the increasing of inverse function list are often used as mind for it
Threshold function table through network, by variable mappings to 0, between 1.
Further, convolution is the common method of image procossing, gives input picture, in the output image each pixel
It is the weighted average of pixel in a zonule in input picture, wherein weight is defined by a function, this function is known as rolling up
Product core.
A kind of space-time big data prediction technique provided by the invention, by the process for obtaining any moment deconvolution memory
In, there is multiple convolution calculating operation, convolution length time memory unit is enabled to carry out dimensionality reduction for higher-dimension sequence data.
Based on the above embodiment, it after obtaining any moment deconvolution memory, needs to be remembered according to any moment deconvolution
Recall and obtain any moment deconvolution output with any moment deconvolution out gate, and then based on any moment deconvolution output building
Any moment deconvolution length time memory unit.The present embodiment makes specifically update any moment deconvolution output
It is bright.The present embodiment also please refers to Fig. 4.
It is obtained by the following formula any moment deconvolution out gate:
Wherein, otFor any moment deconvolution out gate, σ is S type nonlinear activation function Sigmoid, WxoIt is anti-to calculate
When convolution out gate andThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,It is any
The input quantity at moment, WhoFor calculate deconvolution out gate when andThe convolution kernel of convolution operation is done,For any moment
Last moment deconvolution output, boFor deconvolution out gate deviation.
Remembered according to any moment deconvolution out gate and any moment deconvolution, any moment is obtained by following formula
Deconvolution output:
Wherein,For any moment deconvolution output, otFor any moment deconvolution out gate, φ swashs for tanh
Function tangent living, ⊙ are Hadamard operator,For any moment deconvolution memory.
Specifically, deconvolution out gate is used to determine in the memory of deconvolution length time memory unit which information can quilt
It exports as a result.
A kind of space-time big data prediction technique provided by the invention, by the process for updating the output of any moment convolution
In, there is multiple convolution calculating operation, higher-dimension sequence data drops in deconvolution length time memory unit
Dimension.
Based on the above embodiment, the present embodiment is made further for obtaining trained positive and negative convolutional neural networks model
It explains on ground.
Fig. 5 is the flow chart that one of embodiment of the present invention obtains trained positive and negative convolutional neural networks model.This
Embodiment please refers to Fig. 5.It is explained below for the specific parameter in Fig. 5, t represents moment number, and l represents positive and negative convolutional Neural
The number of plies of network model, L represent total number of plies comprising convolution length time memory unit, are similarly represented as long in short-term comprising deconvolution
Between memory unit total number of plies.T represents the length of tensor sequence data in the past.
S1, the first moment tensor sequence data in the tensor sequence data being made of observation is inputted into the positive warp
Product neural network model；S2, in the first layer of the positive and negative convolutional neural networks model, by the first moment first layer convolution length
Time memory unit extracts corresponding informance, and the long short time note of the convolution that the corresponding informance is transferred to lower layer of subsequent time
Recall unit, wherein the corresponding informance includes the corresponding convolution output of the first moment first layer convolution length time memory unit
Remember with the first moment convolution, obtains the positive and negative convolutional neural networks model in the prediction result of the first moment first layer；S3,
The number of plies for extracting corresponding informance is added one, continuously carries out step S2, until extract corresponding informance locating for the number of plies be equal to comprising
Total number of plies of the convolution length time memory unit obtains the positive and negative convolutional neural networks model and removes institute in the first moment
Remaining stated outside first layer includes the prediction result of the layer of convolution length time memory unit；S4, the corresponding informance institute in extraction
When the number of plies at place is equal to total number of plies comprising the convolution length time memory unit, the number of plies for extracting corresponding informance is added one,
Total number of plies of the convolution length time memory unit as described in the first moment the adds one layer of deconvolution length time memory unit to extract
Corresponding informance, and the corresponding informance is transferred to lower layer of subsequent time of deconvolution length time memory unit, obtain institute
The total number of plies for stating positive and negative convolutional neural networks model convolution length time memory unit described in the first moment adds one layer pre-
Survey result；S5, the number of plies for extracting corresponding informance is added one, continuously carries out step S4, until layer locating for the corresponding informance extracted
Number is equal to total layer of total number of plies comprising the convolution length time memory unit and the deconvolution length time memory unit
The sum of number obtains the positive and negative convolutional neural networks model all layers of the prediction result in the first moment.
Further, the present embodiment is that all corresponding informances at the first moment are extracted and predicted, i.e., only
Complete the prediction at a moment.Positive and negative convolutional neural networks model provided by the invention is in the information extraction for carrying out each moment
Consistent with method cited by the present embodiment when predicting with data, the information at each moment includes the history letter of last time
Breath.
Further, in step s 2, the corresponding informance is transferred to lower layer of subsequent time of convolution long short time
Memory unit further comprises: the corresponding informance is transferred to lower layer of synchronization of convolution length time memory unit；
The corresponding informance is transferred to the convolution length time memory unit of subsequent time same layer.
It should be noted that the present embodiment is so that corresponding informance is first transferred to lower layer of synchronization of convolution length in short-term
Between memory unit, then be transferred to for the convolution length time memory unit of subsequent time same layer, but the present invention is not intended to limit
Corresponding informance is transmitted into (N+ from n-hour n-th layer convolution length time memory unit by the transmitting timing of corresponding informance
1) the convolution length time memory unit of moment (N+1) layer.N takes natural number.
Further, in step s 4, the deconvolution for the corresponding informance being transferred to lower layer of subsequent time is long in short-term
Between memory unit further comprise: the deconvolution length time memory that the corresponding informance is transferred to lower layer of synchronization is single
Member；The corresponding informance is transferred to the deconvolution length time memory unit of subsequent time same layer.
It should be noted that the present embodiment is so that corresponding informance to be first transferred to lower layer of synchronization of deconvolution length
Time memory unit, then be transferred to for the deconvolution length time memory unit of subsequent time same layer, but the present invention is not
Corresponding informance, i.e., be transmitted by the transmitting timing for limiting corresponding informance from n-hour n-th layer deconvolution length time memory unit
The deconvolution length time memory unit of (N+1) moment (N+1) layer.N takes natural number.
Based on the above embodiment, the present embodiment also please refers to Fig. 5.After step S5 further include: S6, by what is be made of observation
Subsequent time tensor sequence data in tensor sequence data inputs the positive and negative convolutional neural networks model, and circulation executes step
S2 to S5, until number is greater than or equal to the length of tensor sequence data in the past at the time of locating for the information extracted, acquisition is described just
Deconvolution neural network model at the time of having extracted corresponding informance in each layer of prediction result；S7, judgement are when pair extracted
At the time of answering locating for information number be greater than or equal in the past tensor sequence data length when, at this time at the time of number and fiducial value it is big
Small, the fiducial value is two times of the length of tensor sequence data in the past；Wherein, the length of the tensor sequence data in the past
The numerical value of the last moment number of number at the time of at this time is spent for described in；If S8, it is described at this time at the time of number be greater than or equal to the ratio
Compared with value, then the gap of the prediction result and actual result is obtained by loss function, and according to back-propagation algorithm to described
The tensor sequence data being made of observation is updated, and circulation executes step S2 to S7, until prediction result and actual result
Gap be lower than disparity threshold.
Specifically, the length of past tensor sequence data be it is described at this time at the time of number last moment number numerical value,
That is the length of past tensor sequence data is the last moment corresponding numerical value at this moment, for example, when for first
For quarter, the length of past tensor sequence data is 0, and for n-hour, the length of past tensor sequence data is N-1.
N takes natural number.
Loss function in the present embodiment is preferably MSE, and however, the present invention is not limited thereto, also protection apply other types of damage
Lose the scheme of function.Disparity threshold is setting value.
It should be noted that the fiducial value of the present embodiment is two times of the length of tensor sequence data in the past, this is
A kind of preferred embodiment.Fiducial value is the sum of the length of the length of tensor sequence data and the following tensor sequence data in the past, future
The length of tensor sequence data is the length of tensor sequence data to be predicted, is setting value.In the present embodiment, the setting past
The equal length of the length of tensor sequence data and the following tensor sequence data.
Further, in step s 8, hyper parameter can be adjusted when necessary.
Further, if it is described at this time at the time of number be less than the fiducial value, the positive and negative convolutional neural networks model
Output inputs the positive and negative convolutional neural networks model to the prediction result at this time, and by the prediction result, and circulation is held
Row step S2 to S7, number is greater than or equal to the fiducial value at the time of described at this time.
Below by taking the prediction of Moving Mnist data set as an example, to a kind of space-time big data prediction side provided by the invention
Method, which is made, to be further explained.Fig. 6 is Moving Mnist data sample and prediction sample figure in the embodiment of the present invention, this reality
It applies example and please refers to Fig. 6.
Data are pre-processed first.Each data of Moving Mnist data set are made of 20 frame pictures, and 10
Frame is for inputting, and for 10 frames for predicting, the size of every picture is 64 × 64.Its content is two hand-written numbers of random Mnist
Movement of the word in picture.For each frame picture, tensor X ∈ R can be generated1×64×64.From the point of view of time angle, observation be by
One tensor sequence of the tensor data composition of 20 timing nodes
It is obtained by the following formula convolution input gate again:
Wherein, itFor convolution input gate, σ is S type nonlinear activation function Sigmoid, WxiWhen to calculate convolution input gate
WithThe convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhiTo calculate convolution input
Men HeThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, WciTo calculate convolution
When input gate andThe convolution kernel of convolution operation is done, ⊙ is Hadamard operator,For the volume of the last moment of any moment
Product memory, biFor convolution input gate deviation, the size of convolution kernel W is 5 × 5, and characteristic pattern quantity is 64.
It is obtained by the following formula convolution and forgets door:
Wherein, ftDoor is forgotten for convolution, and σ is S type nonlinear activation function Sigmoid, WxfWhen forgeing door to calculate convolution
WithThe convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhfForget to calculate convolution
Men HeThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, WcfTo calculate convolution
Forget door when andThe convolution kernel of convolution operation is done, ⊙ is Hadamard operator,For the volume of the last moment of any moment
Product memory, bfDoor deviation is forgotten for convolution, and the size of convolution kernel W is 5 × 5, and characteristic pattern quantity is 64.
It is obtained by the following formula any moment convolution out gate:
Wherein, otFor any moment convolution out gate, σ is S type nonlinear activation function Sigmoid, WxoTo calculate convolution
When out gate andThe convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhoTo calculate
When convolution out gate andThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, Wco
For calculate convolution out gate when andThe convolution kernel of convolution operation is done, ⊙ is Hadamard operator,For the upper of any moment
The convolution at one moment is remembered, bfFor convolution out gate deviation, the size of convolution kernel W is 5 × 5, and characteristic pattern quantity is 64.
The input quantity of any moment and the output of the convolution of the last moment of any moment are integrated into one by following formula
A convolution tensor:
Wherein, gtFor convolution tensor, φ is tanh activation primitive tangent, WxgFor calculate convolution tensor when and
The convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhgTo calculate convolution tensor sumThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, bgFor convolution tensor deviation,
The size of convolution kernel W is 5 × 5, and characteristic pattern quantity is 64.
Door and the convolution tensor are forgotten according to the convolution input gate, the convolution, are obtained by following formula described any
The memory of moment convolution:
Wherein,For the memory of any moment convolution, ftDoor is forgotten for convolution, ⊙ is Hadamard operator,When being any
The convolution of the last moment at quarter is remembered, itFor convolution input gate, gtFor convolution tensor.
Remembered according to any moment convolution out gate and any moment convolution, any moment convolution is obtained by following formula
Output:
Wherein,For the output of any moment convolution, otFor any moment convolution out gate, φ is that tanh activates letter
Number tangent, ⊙ are Hadamard operator,For the memory of any moment convolution.
Then it is obtained by the following formula deconvolution input gate:
Wherein, itFor deconvolution input gate, σ is S type nonlinear activation function Sigmoid, WxiTo calculate deconvolution input
Door when andThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,For the defeated of any moment
Enter amount, WhiFor calculate deconvolution input gate when andThe convolution kernel of convolution operation is done,For upper a period of time of any moment
The deconvolution at quarter exports, biFor deconvolution input gate deviation, the size of convolution kernel W is 5 × 5, and characteristic pattern quantity is 64.
It is obtained by the following formula deconvolution and forgets door:
Wherein, ftDoor is forgotten for deconvolution, and σ is S type nonlinear activation function Sigmoid, WxfForget to calculate deconvolution
Door when andThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,For the defeated of any moment
Enter amount, WhfFor calculate deconvolution forget door andThe convolution kernel of convolution operation is done,For the last moment of any moment
Deconvolution output, bfDoor deviation is forgotten for deconvolution, and the size of convolution kernel W is 5 × 5, and characteristic pattern quantity is 64.
It is obtained by the following formula any moment deconvolution out gate:
Wherein, otFor any moment deconvolution out gate, σ is S type nonlinear activation function Sigmoid, WxoIt is anti-to calculate
When convolution out gate andThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,It is any
The input quantity at moment, WhoFor calculate deconvolution out gate when andThe convolution kernel of convolution operation is done,For any moment
Last moment deconvolution output, boSize for deconvolution out gate deviation, convolution kernel W is 5 × 5, and characteristic pattern quantity is
64。
The input quantity of any moment and the output of the convolution of the last moment of any moment are integrated into one by following formula
A deconvolution tensor:
Wherein, gtFor deconvolution tensor, φ is tanh activation primitive tangent, WxgWhen to calculate deconvolution tensor
WithThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,For the input quantity of any moment,
WhgTo calculate deconvolution tensor sumThe convolution kernel of convolution operation is done,For the warp of the last moment of any moment
Product output, bgFor deconvolution tensor deviation, the size of convolution kernel W is 5 × 5, and characteristic pattern quantity is 64.
Door and the deconvolution tensor are forgotten according to the deconvolution input gate, the deconvolution, and institute is obtained by following formula
State any moment deconvolution memory:
Wherein,For any moment deconvolution memory, ftDoor is forgotten for deconvolution, ⊙ is Hadamard operator,To appoint
The deconvolution of the last moment at one moment is remembered, itFor deconvolution input gate, gtFor deconvolution tensor.
Remembered according to any moment deconvolution out gate and any moment deconvolution, any moment is obtained by following formula
Deconvolution output:
Wherein,For any moment deconvolution output, otFor any moment deconvolution out gate, φ swashs for tanh
Function tangent living, ⊙ are Hadamard operator,For any moment deconvolution memory.
Then by all convolution length time memory units and all deconvolution length time memory units with certain
Mode is stacked, is connected, and builds positive and negative convolutional neural networks model end to end, wherein the positive and negative convolutional neural networks model
It is using moment and layer as the two dimensional model of dimension, each layer is equal in length, and each moment is equal in length.
Again through the foregoing embodiment in method obtain trained positive and negative convolutional neural networks model.It will be by observation group
At tensor sequenceIt is read into the corresponding input x of neural network1,x2,…,x10In, use positive deconvolution
Neural network model carries out operation, obtains prediction resultIt is calculated and is transported with specific loss function MSE again
Calculate resultWith correct resultGap, then use Adam algorithm as reversed
Propagation algorithm is trained model with 0.001 training rate, and training 50000 times calculates 16 groups of sequences every time.
The effect of positive and negative convolutional neural networks model is finally tested with untrained Moving Mnist data, if obtaining
Preferably as a result, i.e. prediction is more accurate, then positive and negative convolutional neural networks model is saved, so as to specifically answering afterwards
It is used in scene.
Based on the above embodiment, another embodiment of the present invention discloses a kind of space-time big data forecasting system, comprising: obtains pre-
Object module is surveyed, for space-time big data to be inputted trained positive and negative convolutional neural networks model, obtains prediction result；Its
In, the trained positive and negative convolutional neural networks model is obtained by following submodule: building convolution length time memory is single
First submodule, for obtaining the output of any moment convolution according to the memory of any moment convolution and any moment convolution out gate, and
Building any moment convolution length time memory unit is exported based on any moment convolution；Construct the long short time note of deconvolution
Unit submodule is recalled, for obtaining any moment warp according to any moment deconvolution memory and any moment deconvolution out gate
Product output, and based on any moment deconvolution output building any moment deconvolution length time memory unit；Build mould
Type submodule, it is described for being built by all convolution length time memory units and all deconvolution length time memory units
Positive and negative convolutional neural networks model, wherein the positive and negative convolutional neural networks model is using moment and layer as the two-dimentional mould of dimension
Type, each layer is equal in length, and each moment is equal in length；Acquisition trains model submodule, for will be by observing
The tensor sequence data of value composition inputs the positive and negative convolutional neural networks model and is trained, and obtains described trained positive and negative
Convolutional neural networks model.
A kind of space-time big data prediction technique and system based on positive and negative convolutional neural networks provided by the invention, uses volume
Product length time memory unit and deconvolution length time memory unit can capture dynamic on time and Spatial Dimension simultaneously
Mechanical information learns the hidden feature of space-time data by analyzing and learning the data observed in the past, predicts future
Space-time big data sequence.This is all extremely helpful for the application of the space-time datas such as video monitoring, weather prognosis, wisdom city city
's.Space-time big data prediction technique is in stable condition in training, and convergence is fast, and memory usage is low, can sufficiently capture space-time
Motion feature in data, precision of prediction are high.Since positive and negative convolutional neural networks model uses the long short time note of multilayer convolution
Recall the mode that elementary layer and deconvolution length time memory elementary layer stack, so that model is provided with stronger ability to express, be allowed to
It is more suitable for spatio-temporal prediction.Use variable dimension convolution length time memory unit and deconvolution length time memory unit as
The way of network structure presents splendid performance in spatio-temporal prediction.Use tensor layer regularization (Tensor
LayerNormalization) this exclusively for convolution-deconvolution length time memory structure design method, instruction can be shortened
Practice the time, and hidden state is stablized by the activity of regularization neuron.
Finally, method of the invention is only preferable embodiment, it is not intended to limit the scope of the present invention.It is all
Within the spirit and principles in the present invention, any modification, equivalent replacement, improvement and so on should be included in protection of the invention
Within the scope of.
Claims (10)
1. a kind of space-time big data prediction technique characterized by comprising
Precipitation space-time big data is inputted into trained positive and negative convolutional neural networks model, obtains the precipitation of prediction as a result, institute
Positive and negative convolutional neural networks model is stated for obtaining dynamics of the precipitation space-time big data on time and Spatial Dimension
Information, and extract hidden feature therein and learnt, to improve the predictablity rate of the precipitation result；
Wherein, the trained positive and negative convolutional neural networks model obtains as follows:
The output of any moment convolution is obtained according to the memory of any moment convolution and any moment convolution out gate, and is appointed based on described
One moment convolution output building any moment convolution length time memory unit；
Any moment deconvolution output is obtained according to any moment deconvolution memory and any moment deconvolution out gate, and is based on
Any moment deconvolution output building any moment deconvolution length time memory unit；
The positive deconvolution is built by all convolution length time memory units and all deconvolution length time memory units
Neural network model, wherein the positive and negative convolutional neural networks model is each layer using moment and layer as the two dimensional model of dimension
Be equal in length, each moment is equal in length；
The tensor sequence data being made of observation is inputted the positive and negative convolutional neural networks model to be trained, described in acquisition
Trained positive and negative convolutional neural networks model.
2. prediction technique according to claim 1, which is characterized in that described to pass through all convolution length time memory units
Building the positive and negative convolutional neural networks model with all deconvolution length time memory units further comprises:
All convolution length time memory units are placed in the lower level of the positive and negative convolutional neural networks model, by all warps
Product length time memory unit is placed in the higher level of the positive and negative convolutional neural networks model；
Wherein, the number of plies locating for any deconvolution length time memory unit is more single than any convolution length time memory
The number of plies locating for member is high, and the quantity of the convolution length time memory unit and the deconvolution length time memory unit
Quantity is equal.
3. prediction technique according to claim 1, which is characterized in that any moment convolution memory passes through following steps
It obtains:
It is obtained by the following formula convolution input gate:
Wherein, itFor convolution input gate, σ is S type nonlinear activation function Sigmoid, WxiFor calculate convolution input gate when and
The convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhiFor calculate convolution input gate andThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, WciTo calculate convolution input
Door when andThe convolution kernel of convolution operation is done, ⊙ is Hadamard operator,For the convolution note of the last moment of any moment
Recall, biFor convolution input gate deviation；
It is obtained by the following formula convolution and forgets door:
Wherein, ftDoor is forgotten for convolution, and σ is S type nonlinear activation function Sigmoid, WxfFor calculate convolution forget door when and
The convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhfFor calculate convolution forget door andThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, WcfForget to calculate convolution
Door when andThe convolution kernel of convolution operation is done, ⊙ is Hadamard operator,For the convolution note of the last moment of any moment
Recall, bfDoor deviation is forgotten for convolution；
The input quantity of any moment and the output of the convolution of the last moment of any moment are integrated into a volume by following formula
Product tensor:
Wherein, gtFor convolution tensor, φ is tanh activation primitive tangent, WxgFor calculate convolution tensor when andIt rolls up
The convolution kernel of product operation, * is convolution operator,For the input quantity of any moment, WhgTo calculate convolution tensor sumIt does
The convolution kernel of convolution operation,For the convolution output of the last moment of any moment, bgFor convolution tensor deviation；
Door and the convolution tensor are forgotten according to the convolution input gate, the convolution, and any moment is obtained by following formula
Convolution memory:
Wherein,For the memory of any moment convolution, ftDoor is forgotten for convolution, ⊙ is Hadamard operator,For any moment
The convolution of last moment is remembered, itFor convolution input gate, gtFor convolution tensor.
4. prediction technique according to claim 1, which is characterized in that it is described according to any moment convolution memory and it is any when
It carves convolution out gate and obtains any moment convolution and export and further comprise:
It is obtained by the following formula any moment convolution out gate:
Wherein, otFor any moment convolution out gate, σ is S type nonlinear activation function Sigmoid, WxoTo calculate convolution output
Door when andThe convolution kernel of convolution operation is done, * is convolution operator,For the input quantity of any moment, WhoTo calculate convolution
When out gate andThe convolution kernel of convolution operation is done,For the convolution output of the last moment of any moment, WcoFor meter
Calculate convolution out gate when andThe convolution kernel of convolution operation is done, ⊙ is Hadamard operator,For upper a period of time of any moment
The convolution at quarter is remembered, bfFor convolution out gate deviation；
Remembered according to any moment convolution out gate and any moment convolution, it is defeated to obtain any moment convolution by following formula
Out:
Wherein,For the output of any moment convolution, otFor any moment convolution out gate, φ is tanh activation primitive
Tangent, ⊙ are Hadamard operator,For the memory of any moment convolution.
5. prediction technique according to claim 1, which is characterized in that any moment deconvolution memory passes through following step
It is rapid to obtain:
It is obtained by the following formula deconvolution input gate:
Wherein, itFor deconvolution input gate, σ is S type nonlinear activation function Sigmoid, WxiWhen to calculate deconvolution input gate
WithThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,For the input quantity of any moment,
WhiFor calculate deconvolution input gate when andThe convolution kernel of convolution operation is done,For last moment of any moment
Deconvolution output, biFor deconvolution input gate deviation；
It is obtained by the following formula deconvolution and forgets door:
Wherein, ftDoor is forgotten for deconvolution, and σ is S type nonlinear activation function Sigmoid, WxfWhen forgeing door to calculate deconvolution
WithThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,For the input of any moment
Amount, WhfFor calculate deconvolution forget door andThe convolution kernel of convolution operation is done,For last moment of any moment
Deconvolution output, bfDoor deviation is forgotten for deconvolution；
The input quantity of any moment and the output of the convolution of the last moment of any moment are integrated into one instead by following formula
Convolution tensor:
Wherein, gtFor deconvolution tensor, φ is tanh activation primitive tangent, WxgFor calculate deconvolution tensor when and
The convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,For the input quantity of any moment, WhgFor
Calculate deconvolution tensor sumThe convolution kernel of convolution operation is done,Deconvolution for the last moment of any moment is defeated
Out, bgFor deconvolution tensor deviation；
Door and the deconvolution tensor are forgotten according to the deconvolution input gate, the deconvolution, and described appoint is obtained by following formula
One moment deconvolution memory:
Wherein,For any moment deconvolution memory, ftDoor is forgotten for deconvolution, ⊙ is Hadamard operator,When being any
The deconvolution of the last moment at quarter is remembered, itFor deconvolution input gate, gtFor deconvolution tensor.
6. prediction technique according to claim 1, which is characterized in that described according to any moment deconvolution memory and any
Moment deconvolution out gate obtains any moment deconvolution output:
It is obtained by the following formula any moment deconvolution out gate:
Wherein, otFor any moment deconvolution out gate, σ is S type nonlinear activation function Sigmoid, WxoTo calculate deconvolution
When out gate andThe convolution kernel of convolution operation is done, * is convolution operator,It is accorded with for de-convolution operation,For any moment
Input quantity, WhoFor calculate deconvolution out gate when andThe convolution kernel of convolution operation is done,For the upper of any moment
The deconvolution at one moment exports, boFor deconvolution out gate deviation；
Remembered according to any moment deconvolution out gate and any moment deconvolution, any moment warp is obtained by following formula
Product output:
Wherein,For any moment deconvolution output, otFor any moment deconvolution out gate, φ is that tanh activates letter
Number tangent, ⊙ are Hadamard operator,For any moment deconvolution memory.
7. prediction technique according to claim 1, which is characterized in that the tensor sequence data that will be made of observation
It inputs the positive and negative convolutional neural networks model to be trained, obtains the trained positive and negative convolutional neural networks model into one
Step includes:
S1, the first moment tensor sequence data in the tensor sequence data being made of observation is inputted into the positive deconvolution mind
Through network model；
S2, in the first layer of the positive and negative convolutional neural networks model, by the first first layer convolution length time memory at moment list
Member extracts corresponding informance, and the corresponding informance is transferred to lower layer of subsequent time of convolution length time memory unit,
In, the corresponding informance includes the corresponding convolution output of the first moment first layer convolution length time memory unit and the first moment
Convolution memory obtains the positive and negative convolutional neural networks model in the prediction result of the first moment first layer；
S3, the number of plies for extracting corresponding informance is added one, continuously carries out step S2, until number of plies locating for the corresponding informance extracted etc.
In total number of plies comprising the convolution length time memory unit, the positive and negative convolutional neural networks model is obtained at the first moment
In in addition to the first layer remaining include convolution length time memory unit layer prediction result；
When S4, the number of plies locating for the corresponding informance of extraction are equal to total number of plies comprising the convolution length time memory unit,
The number of plies for extracting corresponding informance is added one, total number of plies of the convolution length time memory unit as described in the first moment the add one layer it is anti-
Convolution length time memory unit extracts corresponding informance, and the corresponding informance is transferred to lower layer of subsequent time of deconvolution
Length time memory unit obtains the positive and negative convolutional neural networks model convolution length time memory described in the first moment the
Total number of plies of unit adds one layer of prediction result；
S5, the number of plies for extracting corresponding informance is added one, continuously carries out step S4, until number of plies locating for the corresponding informance extracted etc.
In total number of plies comprising the convolution length time memory unit and total number of plies of the deconvolution length time memory unit it
With obtain the positive and negative convolutional neural networks model all layers of the prediction result in the first moment.
8. prediction technique according to claim 7, which is characterized in that after step S5 further include:
S6, the subsequent time tensor sequence data in the tensor sequence data being made of observation is inputted into the positive deconvolution mind
Through network model, circulation executes step S2 to S5, until number is greater than or equal to tensor sequence in the past at the time of locating for the information extracted
The length of column data, obtain the positive and negative convolutional neural networks model at the time of having extracted corresponding informance in each layer of prediction
As a result；
S7, when number is greater than or equal to the length of tensor sequence data in the past at the time of judge locating for the corresponding informance of extraction, this
When at the time of number and fiducial value size, the fiducial value is two times of length of the tensor sequence data in the past；Wherein, institute
State over tensor sequence data length be it is described at this time at the time of number last moment number numerical value；
If S8, it is described at this time at the time of number be greater than or equal to the fiducial value, the prediction result is obtained by loss function
With the gap of actual result, and the tensor sequence data being made of observation is updated according to back-propagation algorithm,
Circulation executes step S2 to S7, until the gap of prediction result and actual result is lower than disparity threshold.
9. prediction technique according to claim 8, which is characterized in that step S6 further include:
If it is described at this time at the time of number be less than the fiducial value, the positive and negative convolutional neural networks model output to it is described at this time
Prediction result, and the prediction result is inputted into the positive and negative convolutional neural networks model, circulation executes step S2 to S7, directly
Number is greater than or equal to the fiducial value at the time of described at this time.
10. a kind of pre- measurement equipment of space-time big data characterized by comprising
Precipitation space-time big data is inputted into trained positive and negative convolutional neural networks model, obtains the precipitation of prediction as a result, institute
Positive and negative convolutional neural networks model is stated for obtaining dynamics of the precipitation space-time big data on time and Spatial Dimension
Information, and extract hidden feature therein and learnt, to improve the predictablity rate of the precipitation result；
Wherein, the trained positive and negative convolutional neural networks model obtains as follows:
Convolution length time memory unit submodule is constructed, for exporting according to the memory of any moment convolution and any moment convolution
Door obtains the output of any moment convolution, and based on any moment convolution output building any moment convolution length time memory
Unit；
Deconvolution length time memory unit submodule is constructed, for according to any moment deconvolution memory and any moment warp
Product out gate obtains any moment deconvolution output, and based on any moment deconvolution output building any moment deconvolution
Length time memory unit；
Model submodule is built, for single by all convolution length time memory units and all deconvolution length time memories
Member builds the positive and negative convolutional neural networks model, wherein the positive and negative convolutional neural networks model is with moment and layer for dimension
The two dimensional model of degree, each layer is equal in length, and each moment is equal in length；
Acquisition trains model submodule, and the tensor sequence data for will be made of observation inputs the positive and negative convolutional Neural
Network model is trained, and obtains the trained positive and negative convolutional neural networks model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711192977.8A CN107992938B (en) | 2017-11-24 | 2017-11-24 | Space-time big data prediction technique and system based on positive and negative convolutional neural networks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711192977.8A CN107992938B (en) | 2017-11-24 | 2017-11-24 | Space-time big data prediction technique and system based on positive and negative convolutional neural networks |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107992938A CN107992938A (en) | 2018-05-04 |
| CN107992938B true CN107992938B (en) | 2019-05-14 |
Family
ID=62033013
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711192977.8A Active CN107992938B (en) | 2017-11-24 | 2017-11-24 | Space-time big data prediction technique and system based on positive and negative convolutional neural networks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107992938B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109448361B (en) * | 2018-09-18 | 2021-10-19 | 云南大学 | Resident traffic travel flow prediction system and prediction method thereof |
| CN110322009B (en) * | 2019-07-19 | 2023-04-07 | 南京梅花软件系统股份有限公司 | Image prediction method based on multilayer convolution long-time memory neural network |
| CN111461169B (en) * | 2020-03-04 | 2023-04-07 | 浙江工商大学 | Pedestrian attribute identification method based on forward and reverse convolution and multilayer branch depth network |
| CN111507212A (en) * | 2020-04-03 | 2020-08-07 | 咪咕文化科技有限公司 | Video focus area extraction method, device, equipment and storage medium |
| CN112215406B (en) * | 2020-09-23 | 2024-04-16 | 国网甘肃省电力公司电力科学研究院 | Non-invasive resident electricity load decomposition method based on time convolution neural network |
| CN112183886B (en) * | 2020-10-22 | 2024-03-15 | 南京信息工程大学 | Short-time adjacent rainfall prediction method based on convolution network and attention mechanism |
| CN112257800B (en) * | 2020-10-30 | 2024-05-31 | 南京大学 | Visual identification method based on deep convolutional neural network model-regeneration network |
| CN112862165B (en) * | 2021-01-23 | 2022-04-26 | 西南交通大学 | Travel demand prediction method based on 3D deconvolution |
| CN114936700A (en) * | 2022-05-30 | 2022-08-23 | 北京环境特性研究所 | Construction method and prediction method of aerosol concentration space-time distribution prediction model |
| CN114997490A (en) * | 2022-05-30 | 2022-09-02 | 北京环境特性研究所 | Construction method, prediction method, device and equipment of temperature profile prediction model |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650690A (en) * | 2016-12-30 | 2017-05-10 | 东华大学 | Night vision image scene identification method based on deep convolution-deconvolution neural network |
| CN106782602A (en) * | 2016-12-01 | 2017-05-31 | 南京邮电大学 | Speech-emotion recognition method based on length time memory network and convolutional neural networks |
| CN107292268A (en) * | 2017-06-23 | 2017-10-24 | 西安电子科技大学 | The SAR image semantic segmentation method of quick ridge ripple deconvolution Structure learning model |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170091615A1 (en) * | 2015-09-28 | 2017-03-30 | Siemens Aktiengesellschaft | System and method for predicting power plant operational parameters utilizing artificial neural network deep learning methodologies |
-
2017
- 2017-11-24 CN CN201711192977.8A patent/CN107992938B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106782602A (en) * | 2016-12-01 | 2017-05-31 | 南京邮电大学 | Speech-emotion recognition method based on length time memory network and convolutional neural networks |
| CN106650690A (en) * | 2016-12-30 | 2017-05-10 | 东华大学 | Night vision image scene identification method based on deep convolution-deconvolution neural network |
| CN107292268A (en) * | 2017-06-23 | 2017-10-24 | 西安电子科技大学 | The SAR image semantic segmentation method of quick ridge ripple deconvolution Structure learning model |
Non-Patent Citations (2)
| Title |
|---|
| Merging recovery feature network to faster RCNN for low-resolution images detection;Ruyi Zhang等;《2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)》;20171116;第1230-1234页 |
| 图像特征学习方法与应用研究;杨晨雪;《中国博士学位论文全文数据库信息科技辑》;20170215(第2期);第18-107页 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107992938A (en) | 2018-05-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107992938B (en) | Space-time big data prediction technique and system based on positive and negative convolutional neural networks | |
| CN107958044A (en) | Higher-dimension sequence data Forecasting Methodology and system based on depth space-time memory network | |
| CN109492822B (en) | Air pollutant concentration time-space domain correlation prediction method | |
| CN108805185B (en) | Face recognition method and device, storage medium and computer equipment | |
| CN106778682B (en) | A kind of training method and its equipment of convolutional neural networks model | |
| CN108664893B (en) | Face detection method and storage medium | |
| CN107092870B (en) | A kind of high resolution image Semantic features extraction method | |
| US20190034497A1 (en) | Data2Data: Deep Learning for Time Series Representation and Retrieval | |
| CN107369166B (en) | Target tracking method and system based on multi-resolution neural network | |
| CN107748942B (en) | Radar Echo Extrapolation prediction technique and system based on velocity field sensing network | |
| CN110473592B (en) | Multi-view human synthetic lethal gene prediction method | |
| CN108021947B (en) | A kind of layering extreme learning machine target identification method of view-based access control model | |
| CN108052881A (en) | The method and apparatus of multiclass entity object in a kind of real-time detection construction site image | |
| CN109829541A (en) | Deep neural network incremental training method and system based on learning automaton | |
| CN108090472B (en) | Pedestrian re-identification method and system based on multi-channel consistency characteristics | |
| Braakmann-Folgmann et al. | Sea level anomaly prediction using recurrent neural networks | |
| CN108846473A (en) | Light field depth estimation method based on direction and dimension self-adaption convolutional neural networks | |
| CN110826453A (en) | Behavior identification method by extracting coordinates of human body joint points | |
| CN109064460B (en) | Wheat severe disease prediction method based on multiple time sequence attribute element depth characteristics | |
| CN109886342A (en) | Model training method and device based on machine learning | |
| CN109919246A (en) | Pedestrian's recognition methods again based on self-adaptive features cluster and multiple risks fusion | |
| CN113298129A (en) | Polarized SAR image classification method based on superpixel and graph convolution network | |
| CN112560966A (en) | Polarimetric SAR image classification method, medium and equipment based on scattergram convolution network | |
| CN110188621B (en) | Three-dimensional facial expression recognition method based on SSF-IL-CNN | |
| Rahadian et al. | Image encoding selection based on Pearson correlation coefficient for time series anomaly detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |