CN111950810B - Multi-variable time sequence prediction method and equipment based on self-evolution pre-training - Google Patents
Multi-variable time sequence prediction method and equipment based on self-evolution pre-training Download PDFInfo
- Publication number
- CN111950810B CN111950810B CN202010876972.2A CN202010876972A CN111950810B CN 111950810 B CN111950810 B CN 111950810B CN 202010876972 A CN202010876972 A CN 202010876972A CN 111950810 B CN111950810 B CN 111950810B
- Authority
- CN
- China
- Prior art keywords
- time
- prediction
- sequence
- time sequence
- univariate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000012549 training Methods 0.000 title claims abstract description 29
- 230000015654 memory Effects 0.000 claims abstract description 21
- 238000005457 optimization Methods 0.000 claims abstract description 8
- 238000010586 diagram Methods 0.000 claims description 8
- 238000012417 linear regression Methods 0.000 claims description 8
- 239000013598 vector Substances 0.000 claims description 8
- 230000004927 fusion Effects 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 5
- 230000004044 response Effects 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 2
- 238000002156 mixing Methods 0.000 claims description 2
- 238000010606 normalization Methods 0.000 claims description 2
- 238000007500 overflow downdraw method Methods 0.000 claims description 2
- 238000005311 autocorrelation function Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000012731 temporal analysis Methods 0.000 description 2
- 238000000700 time series analysis Methods 0.000 description 2
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- YHXISWVBGDMDLQ-UHFFFAOYSA-N moclobemide Chemical compound C1=CC(Cl)=CC=C1C(=O)NCCN1CCOCC1 YHXISWVBGDMDLQ-UHFFFAOYSA-N 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Business, Economics & Management (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Evolutionary Biology (AREA)
- Computing Systems (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Mathematics (AREA)
- Molecular Biology (AREA)
- Operations Research (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- General Health & Medical Sciences (AREA)
- Pure & Applied Mathematics (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Strategic Management (AREA)
- Health & Medical Sciences (AREA)
- Development Economics (AREA)
- Quality & Reliability (AREA)
- Marketing (AREA)
- Databases & Information Systems (AREA)
- Tourism & Hospitality (AREA)
- Algebra (AREA)
- Entrepreneurship & Innovation (AREA)
- Probability & Statistics with Applications (AREA)
- General Business, Economics & Management (AREA)
- Game Theory and Decision Science (AREA)
Abstract
The invention discloses a multi-variable time sequence prediction method and equipment based on self-evolution pre-training, which are based on a pre-training strategy, a convolutional network and a long and short memory network depth sequence model, and combine univariate self-evolution information and multi-variable dependency information to model, so that an optimization algorithm of multi-variable time sequence prediction is realized, and meanwhile, the overall prediction precision and the local univariate prediction precision are considered. The method has better overall prediction precision, and is superior to the existing multivariable time sequence prediction method in the aspect of guaranteeing the prediction precision of local univariates.
Description
Technical Field
The invention relates to time sequence prediction, in particular to a multivariable time sequence prediction method.
Background
The time sequence prediction method is adopted to obtain the predicted value of the future moment, and the method has fundamental and important significance for auxiliary decision making, resource allocation optimization, intervention measures taking in advance and the like. Information systems operating in real-time have a wide range of applications. Many systems generate large amounts of time series data, mainly including system performance monitoring data, and many applications in performance prediction of these monitoring data, such as predicting network traffic for resource planning and anomaly discovery, predicting disk failures in data centers for early replacement, etc. Future data predicting system performance thus provides important assistance to the daily operation, etc. of modern information systems. In recent years there has been a great deal of research focused on the problem of time series prediction of real-time systems. Linear statistical class algorithms (e.g., AR, ARIMA) have an impact on the field of temporal sequence prediction. However, nonlinear time series analysis and prediction are proposed and gradually applied due to the lack of applicability of the linear model in many practical applications. In addition to classical statistical method models, machine learning models have become an important algorithm in the field of time series prediction. Researchers apply machine learning models such as linear regression models, support vector machines, decision trees, etc. to time series analysis. The time sequence prediction algorithm based on machine learning takes time sequence prediction as a supervised learning task, wherein the input attribute is time sequence data of historical observation, and the output label is future data. However, the current multivariate time series prediction algorithm focusing on overall prediction target optimization brings about the problem of local variable prediction accuracy loss. For example, in the regression task of multivariate prediction, existing algorithms target the overall prediction error (weighted) sum of all variables, improving overall prediction accuracy by optimizing such a minimization target.
Disclosure of Invention
The invention aims to: in order to ensure the prediction precision of local variables, a multivariate time sequence prediction algorithm based on self-evolution pre-training is provided. Not only can an optimization algorithm of multi-variable time sequence prediction be realized, but also the overall prediction precision and the prediction precision of local univariate can be considered
The technical scheme is as follows: in order to predict the performance of a system running in real time, so as to perform reasonable operation and maintenance aiming at a predicted value, in a first aspect, the invention provides a self-evolution pre-training-based multi-variable time sequence prediction method, which comprises the following steps:
s1, acquiring N dimension index data representing the performance of a real-time operation system, forming N univariate time sequences as historical input sequences, and preprocessing;
s2, inputting N univariate time sequences into an automatic evolution pre-training model SE based on a differential feature structure, establishing a univariate time sequence linear regression model, and explicitly blending multi-order differential information into the linear autoregressive model to form corresponding prediction output;
s3, performing one-dimensional convolution operation with different sizes on a two-dimensional tensor formed by N univariate time sequences, and extracting dependency relationship information among variables to obtain a plurality of feature graphs;
s4, inputting the feature map into a long short memory network (LSTM) to perform time sequence modeling on the multi-scale feature map so as to capture time sequence information of the multi-variable dependency relationship, and forming corresponding output by corresponding multi-variable modeling network components;
s5, carrying out one-to-one correspondence addition fusion on the outputs obtained in the steps S2 and S4 to construct multivariable prediction output, and training a network to obtain a trained model;
s6, inputting the time sequence to be predicted into the trained model obtained in the step S5, so as to obtain a predicted value of the time sequence.
In a second aspect, the present invention is directed to a computer device comprising one or more processors; a memory for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method according to the first aspect of the invention.
The beneficial effects are that: the invention provides a multi-variable time sequence prediction method based on self-evolution pre-training, which is based on a pre-training strategy, a convolutional network and a long and short memory network depth sequence model, combines single-variable self-evolution information and multi-variable dependency information to carry out modeling, realizes an optimization algorithm of multi-variable time sequence prediction, and simultaneously gives consideration to the overall prediction precision and the local single-variable prediction precision. The method has better overall prediction precision, and is superior to the existing multivariable time sequence prediction method in the aspect of guaranteeing the prediction precision of local univariates.
Drawings
FIG. 1 is a general flow diagram of a multivariate time series prediction method according to an embodiment of the invention;
FIG. 2 is a schematic diagram of the main process of a multivariate time series prediction method according to an embodiment of the invention;
FIG. 3 is a flow chart of model training phase processing according to an embodiment of the invention;
FIG. 4 is a flow chart of a model run phase process according to an embodiment of the invention.
Detailed Description
The technical scheme of the invention is further described below with reference to the accompanying drawings. It should be understood that the following examples are provided only for the purpose of thoroughly and completely disclosing the present invention and fully conveying the technical concept of the present invention to those skilled in the art, the present invention may be embodied in many different forms and is not limited to the examples described herein. The terminology used in the exemplary embodiments illustrated in the accompanying drawings is not intended to be limiting of the invention.
In the embodiment of the invention, a multi-variable time sequence prediction method based on self-evolution pre-training is used for predicting time sequences. Based on a pre-training strategy, a convolutional network and a long and short memory network depth sequence model, modeling is carried out by combining univariate self-evolution information and multivariate dependency relationship information, an optimization algorithm of multivariate time sequence prediction is realized, and meanwhile, the overall prediction precision and the prediction precision of local univariate are considered. Referring to fig. 1 and 2, the method of the present invention comprises the steps of:
step S1, corresponding multivariable time series data are collected from an operating system running in real time through a monitoring system, wherein the multivariable time series data possibly comprise a plurality of key indexes such as CPU load, bandwidth utilization rate, bandwidth speed, memory utilization rate, response time delay, IO read-write and the like. In reality, some problems such as data missing and irregular data often occur in the data, so that the data is preprocessed, and then the preprocessed data is used as an input of a model. Furthermore, in order to have a better modeling effect on the time series, it is also necessary to evaluate the periodicity of the time series to obtain the history window T.
The data is preprocessed, namely, the data set is divided in a time dimension, a training set is required to be reserved for training of the model, and the data is subjected to standardization processing so that the data values can be mapped to the same interval, thus being beneficial to learning of the model, such as min-max standardization:
wherein min is x At a minimum value of x, max x Is the maximum value in x. The periodicity of the time series is evaluated by an autocorrelation function ACF to obtain a historical input sequence length T. The CPU load, bandwidth utilization, bandwidth speed, memory utilization, response time delay, IO read-write and other key indexes respectively form a univariate time sequence, the length of each sequence is T, and x represents the vector value of the sequence.
And S2, inputting N univariate time series distributions of the key indexes including CPU load, bandwidth utilization, memory utilization, response time delay and the like after the pretreatment in the step S1 into a Self-Evolution pre-training model SE (Self-Evolution) constructed based on differential features, and establishing a univariate time series linear regression model, wherein the model has the function of ensuring the prediction precision of the univariate time series generated by a real-time system. In addition, considering that the multi-order differential information can improve applicability and robustness in the prediction of a stable and non-stable sequence, the multi-order differential information is explicitly fused into a linear autoregressive model to form corresponding prediction output. The method specifically comprises the following steps:
s2-1, inputting N univariate time sequences into a self-evolution pre-training model SE based on differential feature construction, and establishing a univariate time sequence linear regression model. For the i-th sequence there are:
wherein the method comprises the steps ofRepresent the firstTime sequence value at time t-j in the i sequences,/->For the corresponding linear weights, T is the historical input sequence length obtained by evaluating the periodicity of the time sequence by the autocorrelation function ACF.
Step S2-2, when considering the differential order q=1, includesI.e.It can be seen that the predicted value under the first order difference contains +.>HandleThe equipartition features are explicitly fused into the linear autoregressive model to form corresponding prediction input village +.>
The fusion weight of the differential features is used for training, the limitation is a non-negative real number, softmax is corresponding normalization operation, and Q is the maximum differential order; />Is a standard linear autoregressive term, x t-T：t-1 The time series from time T-1 to time T-T is shown, T being the length of the historical input sequence.
And S3, performing one-dimensional convolution operation with different sizes on a two-dimensional tensor formed by N univariate time sequences, and extracting dependency relationship information among the variables to obtain a plurality of feature graphs.
The invention adopts the time sequence convolution of a one-dimensional CNN layer of a plurality of convolution kernels to express the complex dependency relationship among multi-element time sequences, so as to obtain a feature map (feature map) containing the multi-level information of the local dependency relationship of the multi-element sequences. For input sequencesOne-dimensional convolution operation in variable dimension is performed as follows:
C (k) ＝W (k) *X t-T：t-1
W (k) is the kth convolution kernel, C (k) Is the convolution result, i.e. the feature map extracted from the dependency between the variables. In a convolutional network, typically k convolutional kernels W are employed (1) ，W (2) ，...，W (k) Forming a plurality of channels of feature pattern C (1) ，C (2) ，...，C (k) . Unlike existing algorithms that set all convolution kernels to the same size, the present invention employs convolution kernels of variable size. In contrast to the same-scale convolution kernel, the multi-scale convolution kernel explicitly characterizes associated time periods of different scales. C (C) (1) ，C (2) ，...，C (k) And multi-level information of local dependency relationship of the multi-variable time sequence is contained.
And S4, inputting the feature images into a Long Short-Term Memory (LSTM) network to perform time sequence modeling on the multi-scale feature images so as to capture time sequence information of the multi-variable dependency relationship, and respectively inputting each feature image into a corresponding LSTM and obtaining corresponding output.
LSTM is a classical recurrent neural network that can solve the long-term dependence of sequences. The network component based on the multivariate modeling in the invention is a structure formed by a plurality of LSTM basic memory units, and each LSTM is correspondingly input with a characteristic diagram. Specifically, step S3 is performed to obtain a characteristic diagram C of all channels of the kth convolution kernel (k) Splicing in the time dimension, wherein the number of channels is determined by a time sequenceThe dimension of the data is determined, and the spliced characteristic diagram is used as the input of the corresponding k long and short memory network; hidden output of the corresponding network unit is input to the full connection layer to form an output vector h (k) ∈R N×1 . K identical LSTM total outputs h ( 1 ) ，h (2) ，...，h (K) These outputs contain sequence dependent information. Accordingly, all vectors h (1) ，h (2) ，...，h (K) Adding to form a multivariate dependency modeling output result
And S5, carrying out one-to-one correspondence addition and fusion on the outputs obtained in the steps S2 and S4 to construct multivariable prediction output, and training a network to obtain a trained model. The method comprises the following steps:
and (3) carrying out one-to-one correspondence addition fusion on vector elements on the outputs obtained in the steps S2 and S4, wherein the fusion method is as follows:
W (se) ，W (id) for fusing the weights, by manually setting the super parameters as a model,representing element multiplication. The additive fusion of the two parts has clear practical meaning, the interpretability of the algorithm is improved, and the loss function is minimized, so that the accuracy loss of the local univariate can be controlled without becoming large, and the overall prediction accuracy loss can be controlled, thereby achieving the purpose of the algorithm. For example, the predicted value of the ith variable +.>Then for the prediction of the variable, the relative magnitudes of the two values directly reflect the duty cycle of the two pieces of information in the prediction. Training after the model is builtModeling, constructing a loss function corresponding to an optimization target as follows:
the overall model may be optimized based on a gradient descent method. The weight of model training is the weight of Self-evaluation, CNN and LSTM model. This model is a single step prediction model, then the final outputIs a multivariate vector value.
The steps S1 to S5 are model training stages, as shown in fig. 3, and training the multivariate time series prediction model based on self-evolution pre-training by using historical time series data.
And S6, predicting the time sequence to be predicted by using the trained model.
As shown in fig. 4, in the model operation stage, the time sequence to be predicted is preprocessed and then input into the trained model obtained in step S5, so as to obtain a predicted value of the time sequence, that is, the model finally obtains a predicted output related to the real-time system.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, and the program may be stored in a computer readable storage medium. In the context of the present invention, the computer-readable medium may be considered to be tangible and non-transitory. Non-limiting examples of non-transitory tangible computer readable media include non-volatile memory circuits (e.g., flash memory circuits, erasable programmable read-only memory circuits, or masked read-only memory circuits), volatile memory circuits (e.g., static random access memory circuits or dynamic random access memory circuits), magnetic storage media (e.g., analog or digital magnetic tape or hard disk drives), and optical storage media (e.g., CDs, DVDs, or blu-ray discs), among others.
Program code for carrying out methods of the present invention may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
Moreover, although operations are depicted in a particular order, this should be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the invention. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination.
The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to the specific details of the above embodiments, and various equivalent changes can be made to the technical solution of the present invention within the scope of the technical concept of the present invention, and all the equivalent changes belong to the protection scope of the present invention.
Claims (5)
1. A method for predicting a multivariate time series based on self-evolving pre-training, the method comprising the steps of:
s1, acquiring N dimension index data representing the performance of a real-time operation system, wherein the N dimension index data comprises CPU load, bandwidth utilization rate, bandwidth speed, memory utilization rate, response time delay and IO read-write, forming N univariate time sequences, taking the N univariate time sequences as historical input sequences, and preprocessing the N univariate time sequences;
s2, inputting N univariate time sequences into an automatic evolution pre-training model based on differential feature construction, establishing a univariate time sequence linear regression model, and explicitly blending multi-order differential information into the linear regression model to form corresponding prediction outputComprising the following steps:
s2-1, inputting N univariate time sequences comprising CPU load, bandwidth utilization rate, bandwidth speed, memory utilization rate, response time delay and IO read-write index data into a self-evolution pre-training model SE constructed based on differential characteristics, and establishing a univariate time sequence linear regression model, wherein the ith sequence is as follows:
wherein the method comprises the steps ofTime sequence value representing time t-j in the ith sequence,/for>T is the length of the historical input sequence for the corresponding linear weight;
s2-2, explicitly integrating the multi-order differential information into a linear regression model to form a corresponding prediction output
The method is characterized in that the method is a fusion weight of the differential characteristics, softmax is corresponding normalization operation, and Q is the maximum differential order;is a standard linear autoregressive term, x t-T∶t-1 A time series from time T-1 to time T-T;
s3, carrying out one-dimensional convolution operation with different sizes on a two-dimensional tensor formed by N univariate time sequences, extracting dependency relationship information among variables to obtain a plurality of feature graphs, wherein the one-dimensional convolution operation is as follows:
C (k) ＝W (k) *X t-T:t-1
W (k) is the kth convolution kernel, C (k) Is the convolution result, i.e. the feature diagram extracted from the dependency relationship between the variables, X t-T:t-1 Representing a two-dimensional tensor consisting of N univariate time series,x t-T∶t-1 the time sequence from the time T-1 to the time T-T is represented, and T is the length of the historical input sequence;
s4, inputting the feature map into a long and short memory network to perform time sequence modeling on the multi-scale feature map so as to capture time sequence information of the multi-variable dependency relationship, and forming corresponding output by corresponding multi-variable modeling network components
S5, outputting the output obtained in the steps S2 and S4And->Carrying out one-to-one correspondence addition fusion on vector elements, constructing a multivariable prediction output model, and training to obtain a trained model;
s6, preprocessing the time sequence to be predicted, and inputting the preprocessed time sequence into the trained model obtained in the step S5, so as to obtain a predicted value of the time sequence.
2. The prediction method according to claim 1, wherein the step S4 includes: obtaining a characteristic diagram C of all channels of the kth convolution kernel in the step S3 (l) Splicing according to the time dimension, and then taking the spliced data as the input of a corresponding kth long and short memory network; hidden output of the corresponding network unit is input to the full connection layer to form an output vector h (k) ∈R N×1 K identical LSTM total outputs h (1) ,h (2) ,...,h (K) All vectors h (1) ,h (2) ,...,h (K) Adding to form a multivariate dependency modeling output result
3. The prediction method according to claim 1, wherein in the step S5, the results output in the steps S2 and S4 are weighted and fused, and the fusion method is as follows:
W (se) ,W (id) in order to fuse the weights, the weight values,representing element multiplication.
4. A prediction method according to claim 3, wherein the training model in step S5 constructs a loss function corresponding to the optimization objective as follows:
n is the dimension of the multivariate time series.
5. A computer device, the device comprising:
one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one processor, which when executed by the processor implement the steps of the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010876972.2A CN111950810B (en) | 2020-08-27 | 2020-08-27 | Multi-variable time sequence prediction method and equipment based on self-evolution pre-training |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010876972.2A CN111950810B (en) | 2020-08-27 | 2020-08-27 | Multi-variable time sequence prediction method and equipment based on self-evolution pre-training |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111950810A CN111950810A (en) | 2020-11-17 |
| CN111950810B true CN111950810B (en) | 2023-12-15 |
Family
ID=73366711
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010876972.2A Active CN111950810B (en) | 2020-08-27 | 2020-08-27 | Multi-variable time sequence prediction method and equipment based on self-evolution pre-training |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111950810B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113627591A (en) * | 2021-07-30 | 2021-11-09 | 马上消费金融股份有限公司 | Dynamic graph data processing method and device, electronic equipment and storage medium |
| CN113807951A (en) * | 2021-09-23 | 2021-12-17 | 中国建设银行股份有限公司 | Transaction data trend prediction method and system based on deep learning |
| CN114186711A (en) * | 2021-10-27 | 2022-03-15 | 中山大学 | Industrial raw material consumption prediction method based on multitask time sequence learning |
| CN114039758A (en) * | 2021-11-02 | 2022-02-11 | 中邮科通信技术股份有限公司 | Network security threat identification method based on event detection mode |
| CN114565149A (en) * | 2022-02-23 | 2022-05-31 | 重庆邮电大学 | CGA fusion model-based time series data prediction method and device and computer equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108648457A (en) * | 2018-06-28 | 2018-10-12 | 苏州大学 | A kind of method, apparatus and computer readable storage medium of prediction of speed |
| CN108830430A (en) * | 2018-08-01 | 2018-11-16 | 苏州大学张家港工业技术研究院 | Convolutional neural networks multiple spot regressive prediction model for traffic flow forecasting |
-
2020
- 2020-08-27 CN CN202010876972.2A patent/CN111950810B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108648457A (en) * | 2018-06-28 | 2018-10-12 | 苏州大学 | A kind of method, apparatus and computer readable storage medium of prediction of speed |
| CN108830430A (en) * | 2018-08-01 | 2018-11-16 | 苏州大学张家港工业技术研究院 | Convolutional neural networks multiple spot regressive prediction model for traffic flow forecasting |
Non-Patent Citations (2)
| Title |
|---|
| 一种基于自演化预训练的多变量时间序列预测算法;万晨;计算机学报;第45卷(第3期);513-525 * |
| 基于CNN-LightGBM模型的高速公路交通量预测;张振;曾献辉;;信息技术与网络安全(02);18-23 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111950810A (en) | 2020-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111950810B (en) | Multi-variable time sequence prediction method and equipment based on self-evolution pre-training | |
| US10360517B2 (en) | Distributed hyperparameter tuning system for machine learning | |
| Deka et al. | Learning for DC-OPF: Classifying active sets using neural nets | |
| CN111210024A (en) | Model training method and device, computer equipment and storage medium | |
| JP2022500769A (en) | Power system heat load prediction method and prediction device | |
| CN111277434A (en) | Network flow multi-step prediction method based on VMD and LSTM | |
| KR20210032140A (en) | Method and apparatus for performing pruning of neural network | |
| CN111639783A (en) | Line loss prediction method and system based on LSTM neural network | |
| CN111127246A (en) | Intelligent prediction method for transmission line engineering cost | |
| US20230108920A1 (en) | System and method for providing robust artificial intelligence inference in edge computing devices | |
| CN115618269A (en) | Big data analysis method and system based on industrial sensor production | |
| CN115936159A (en) | Interpretable credit default rate prediction method and system based on automatic feature mining | |
| CN113641525A (en) | Variable exception recovery method, apparatus, medium, and computer program product | |
| CN113159441A (en) | Prediction method and device for implementation condition of banking business project | |
| Shah et al. | A neoteric technique using ARIMA-LSTM for time series analysis on stock market forecasting | |
| Wang et al. | A Transformer-based multi-entity load forecasting method for integrated energy systems | |
| JP7424373B2 (en) | Analytical equipment, analytical methods and analytical programs | |
| CN112667394B (en) | Computer resource utilization rate optimization method | |
| CN114239934A (en) | Power grid load sequence prediction method and system based on generative countermeasure network | |
| Zang | Construction of Mobile Internet Financial Risk Cautioning Framework Based on BP Neural Network | |
| CN114254764B (en) | Feedback-based machine learning model searching method, system, equipment and medium | |
| KR102531291B1 (en) | Method for predicting energy consumption of a building, and computing device performing the method | |
| CN116934041B (en) | Internet of things collaboration-based factory task management method and system | |
| US20220083905A1 (en) | Framework for providing improved predictive model | |
| US20230146635A1 (en) | Method and Systems for Conditioning Data Sets for Efficient Computational Processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |