CN112819178A - Data prediction model training method, device and storage medium - Google Patents
Data prediction model training method, device and storage medium Download PDFInfo
- Publication number
- CN112819178A CN112819178A CN202110158803.XA CN202110158803A CN112819178A CN 112819178 A CN112819178 A CN 112819178A CN 202110158803 A CN202110158803 A CN 202110158803A CN 112819178 A CN112819178 A CN 112819178A
- Authority
- CN
- China
- Prior art keywords
- model
- prediction
- dependent variable
- comprehensive
- variables
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the application provides a data prediction model training method, equipment and a storage medium. In the data prediction model training method, a proper model can be selected from a plurality of selectable prediction models according to the characteristics of the error measurement indexes and the historical data, and a corresponding optimal model is selected for the predictable dependent variables and the related independent variables needing to be predicted. The comprehensive prediction model may be trained based on the relationship between the prediction model of the dependent variable and the prediction model of the independent variable and the actual value of the dependent variable. The comprehensive prediction model can provide a universal prediction method, can flexibly implement various time series related prediction tasks, and meets various prediction requirements of users.
Description
Technical Field
The present application relates to the field of internet technologies, and in particular, to a data prediction model training method, device, and storage medium.
Background
The importance of accurate prediction of future data trends to the success of a business or other organization is readily apparent. In the prior art, prediction of future data trends is generally realized based on integration of historical data. However, the existing prediction method has the defect of poor flexibility. Therefore, a new solution is yet to be proposed.
Disclosure of Invention
Aspects of the present application provide a data prediction model training method, device and storage medium to improve flexibility of data prediction.
The embodiment of the application provides a data prediction model training method, which comprises the following steps: determining a plurality of candidate prediction models and historical data required by training; the historical data comprises a time series of a set of dependent variables, a time series of a plurality of sets of predictable independent variables and a time series of a plurality of sets of unpredictable independent variables; determining an optimal model of the dependent variable and optimal models of the multiple groups of predictable independent variables from the multiple candidate prediction models according to the historical data and the measurement error index adopted by optimization; and obtaining a comprehensive prediction model of the dependent variable according to the optimal model of the dependent variable, the optimal models of the plurality of groups of predictable independent variables and the time sequence of the plurality of groups of unpredictable independent variables.
Further optionally, obtaining a comprehensive predictive model of the dependent variable according to the optimal model of the dependent variable, the optimal models of the plurality of sets of predictable independent variables, and the time series of the plurality of sets of unpredictable independent variables, includes: obtaining an optimal model of the dependent variable, and taking a prediction result of the time sequence of the dependent variable as a dependent variable prediction result; obtaining the prediction results of the respective optimal models of the plurality of groups of the predictable independent variables on the time series of the plurality of groups of the predictable independent variables to obtain a plurality of independent variable prediction results; and performing time series regression calculation on the dependent variable prediction results, the multiple independent variable prediction results and the multiple groups of unpredictable independent variables to obtain the comprehensive prediction model.
Further optionally, the method further comprises: determining a time period to be predicted of the dependent variable; predicting a dependent variable prediction result corresponding to the dependent variable in the time period to be predicted according to the optimal model of the dependent variable; predicting independent variable prediction results corresponding to the plurality of groups of predictable variables in the time period to be predicted according to the respective optimal models of the plurality of groups of predictable independent variables; and inputting the dependent variable prediction results of the dependent variables in the time period to be predicted, the independent variable prediction results of the plurality of groups of predictable variables in the time period to be predicted and unpredictable independent variable values in the time period to be predicted into the comprehensive prediction model to obtain the comprehensive prediction results of the dependent variables in the time period to be predicted.
Further optionally, the method further comprises: adjusting independent variables in the comprehensive model according to the P value corresponding to the comprehensive prediction model so as to optimize the comprehensive prediction model; wherein the P value is a parameter for determining the hypothesis test result.
Further optionally, the method further comprises: calculating an initial error of the comprehensive prediction model according to the error measurement index and a comprehensive prediction result of the comprehensive prediction model; setting a tracking signal according to the initial error; and reconstructing or optimizing the comprehensive prediction model according to the tracking signal.
Further optionally, the method further comprises: according to the comprehensive prediction model, predicting a comprehensive prediction result of the dependent variable in a future time period to be predicted; and updating the tracking signal according to the comprehensive prediction result of the dependent variable in the future time period to be predicted.
Further optionally, the measured error index includes: at least one of a mean error, an absolute deviation, a mean absolute error, a mean absolute percentage error, and a root mean square error.
Further optionally, the plurality of candidate predictive models comprises: one or more of a single and double moving average model, a single exponential smoothing model, a dual exponential smoothing model for Brown and Holter, a seasonal forecasting model for Brown and Holter and Wentt, an autoregressive synthetic moving average model, a Box-Jenkins model, a multivariate ARIMA model, and a neural network model.
An embodiment of the present application further provides an electronic device, including: a memory, a processor, and a communication component; the memory is to store one or more computer instructions; the processor is to execute the one or more computer instructions to: and executing the steps in the data prediction model training method provided by the embodiment of the application.
The embodiment of the present application further provides a computer-readable storage medium storing a computer program, and the computer program, when executed by a processor, can implement the steps in the data prediction model training method provided in the embodiment of the present application.
In the data prediction model training method provided by the embodiment of the application, a proper model can be selected from a plurality of selectable prediction models according to the characteristics of the measured error index and the historical data, and a corresponding optimal model is selected for the predictable dependent variable and the independent variable which needs to be predicted correspondingly. The comprehensive prediction model may be trained based on the relationship between the prediction model of the dependent variable and the prediction model of the independent variable and the actual value of the dependent variable. The comprehensive prediction model can provide a universal prediction method, can flexibly implement various time series related prediction tasks, and meets various prediction requirements of users.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a schematic diagram of a data prediction system according to an exemplary embodiment of the present application;
FIG. 2 is a schematic flow chart diagram illustrating a data prediction model training method according to another exemplary embodiment of the present application;
FIG. 3 is a schematic flow chart diagram illustrating a data prediction model training method according to another exemplary embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The importance of accurately predicting the future trends of the data to the success of a business or other organization is readily apparent. However, the existing prediction method has the defect of low flexibility. In view of this technical problem, embodiments of the present application provide a solution, which will be exemplarily described below with reference to the accompanying drawings.
As shown in FIG. 1, the prediction process of future data trends may include the following steps:
a1, collecting and integrating historical data.
And A2, analyzing and clearing the historical data to determine whether the historical data is suitable for prediction.
A3, determining and optimizing a prediction model and parameters.
A4, performing quantitative prediction of the future time period.
A5, collecting and integrating actual values of the new time period over time and calculating the prediction error.
A6, re-determining and optimizing the prediction model and parameters as necessary according to the error.
Wherein, the historical data can be obtained by storing and integrating data in all relevant various historical data sources. Where the historical data may be stored in a spreadsheet, database, data mart, or data warehouse. These historical data can be retrieved using an OLAP (online analytical processing) engine, a MOLAP (multiple OLAP) engine, a ROLAP (relational OLAP) engine, open database connectivity (ODBC), JDBC data connectivity for Java, open source software modules for Python, and other permissible methods, and further used for data mining and analysis.
After the history data is acquired, the history data may be analyzed to determine whether the read history data has data abnormality, for example, whether the history data has abnormal value, whether a data loss phenomenon exists, or the like. If data anomalies exist, the historical data may be adjusted to "smooth" or "eliminate" the anomalies. The method for smoothing the abnormal data may include: setting the missing data to zero, averaging over some time period before and after to correct an abnormal value or missing data, building a predictive model using historical data from a previous stage to estimate a value of the abnormal value or missing data, or interpolating the abnormal value or missing value using a spline fitting method, etc. The above-described operation of processing the abnormal data may be referred to as a data cleaning operation.
The historical data, after undergoing data cleaning and necessary adjustments, can be used as input data for a variety of predictive methods to predict future expectations. That is, step A3 described above may be performed based on historical data to determine and optimize predictive models and parameters. An alternative embodiment of step a3 will be described in detail below with reference to the drawings.
Fig. 2 is a schematic flow chart of a data prediction model training method according to an exemplary embodiment of the present application, and as shown in fig. 2, the method includes:
And 203, obtaining a comprehensive prediction model of the dependent variable according to the optimal model of the dependent variable, the optimal models of the plurality of groups of predictable independent variables and the time sequence of the plurality of groups of unpredictable independent variables.
In step 201, the plurality of candidate prediction models may include: the model may be at least one of a single and double moving average model, a single exponential smoothing model, a dual exponential smoothing model for brown and holter, a seasonal forecasting model for brown and holter, a warm characteristic model, an autoregressive synthetic moving average model, a Box-Jenkins model, a multivariate ARIMA model, and a neural network model, which is not limited in this embodiment.
And the candidate prediction model is used for realizing a prediction method based on time series prediction. In the prediction method based on "time series prediction", historical data having a historical time attribute may be used. The historical data having the historical time attribute is generally collected in a collection pattern of average interval time. For example, historical data may be collected in an hourly, daily, monthly, or quarterly collection pattern. In the embodiment of the present application, the time series refers to a time-varying and correlated data series arranged in time sequence. For example, an hourly or per minute collection of environmental parameter sequences, energy use parameter sequences, production job parameter sequences, capacity or discharge sequences, personnel shift sequences, and the like may be implemented.
The method for acquiring the time series of the dependent variable or the independent variable may include: one or more of a single and double moving average method, a single exponential smoothing method, a double exponential smoothing method for brown and holter, a seasonal prediction method for brown and holter, a warm season method, an autoregressive integrated moving average (ARIMA) method, a Box-Jenkins method, a multivariate ARIMA (marima), and a neural network method, which is not limited in this embodiment.
The historical data comprises two parts of data: a time series of predicted targets, also called time series of dependent variables, such as a sequence of energy consumption data per time period; and, a time series of any factor data that may affect the predicted objective, also referred to as a time series of arguments. Wherein, each factor data corresponds to a time sequence, and the time interval of the factor data is the same as the time interval of the dependent variable.
Wherein the time series of arguments can be further subdivided into a time series of continuous arguments and a time series of discrete arguments. The two differ in whether their values are enumerable (discrete) or enumerable (continuous).
In the present embodiment, K different values are employed as criteria for distinguishing between discrete or continuous. That is, when a time series of an argument contains 10 or more different values, the time series is considered to be a time series of continuous arguments, and the other is considered to be a time series of discrete arguments. For example, a discrete argument time series may be implemented as: wind (1, 2, 3, 4 levels), wind direction (east, west, south, north, etc.); the continuous type argument time series can be implemented as: continuous temperature values, continuous humidity values, and the like.
Wherein the time series of the independent variable comprises: a time series of predictable arguments, and a time series of unpredictable arguments. When the prediction of the dependent variable is executed, the user specifies whether the system needs to predict the independent variable or not for each independent variable time series so as to use the prediction result of the independent variable for the prediction of the dependent variable. The independent variable that does not need to be predicted by the system may be called an unpredictable independent variable, and the user needs to provide the predicted value of the independent variable when predicting the dependent variable. The arguments that need to be predicted by the system are called predictable arguments, among others. In some embodiments, the predicted values of the independent variables in discrete form may be used by the user; the predicted values of the continuous independent variables can be provided by a user or automatically predicted by the system.
In this step, the history data may be described as: a time series yt of one dependent variable, a time series xkt of K predictable independent variables (K ═ 1, …, K), and a time series zlt of L unpredictable independent variables (L ═ 1, …, L).
In step 202, the metric for optimization may include, but is not limited to: at least one of mean absolute error (MAD), Mean Absolute Error (MAE), mean percent absolute error (MAPE), and Root Mean Square Error (RMSE). The user can specify, customize, and extend any applicable metric of error.
In this step, for any predictable time series of arguments, the time series of arguments may be input into each candidate prediction model to obtain the prediction result of the argument for each candidate prediction model. Then, according to the prediction result and the measure error index, a candidate prediction model with the most reasonable prediction result is selected from the candidate prediction models to serve as the optimal model of the independent variable. Similarly, an optimal model of the dependent variable may be determined based on the time series of the dependent variable and the measure error indicator.
Wherein determining each optimal model comprises determining a type of the optimal model and parameters of the optimal model. Each predictive model (or method) may contain, among other things, one or more parameters that are evaluated by the model and corresponding historical data. Once these parameters are determined, the models can be used to perform a prediction task, i.e., calculate a prediction metric for a future time period.
In this step, the time series in the history data is divided into two parts, one part is used as training data, and the other part is used as test data. For example, the first 75% of the historical data may be used as training data and the second 25% as test data.
And for the yt and K predictable independent variable sequences xkt, the optimal model and parameters of each predictable variable can be screened out by using the corresponding training data and test data according to the candidate predictive model, the applicable conditions of the historical data and the measurement error index adopted by optimization.
In step 203, a comprehensive predictive model of the dependent variable may be obtained according to the optimal model of the dependent variable, the optimal models of the sets of predictable independent variables, and the time series of the sets of unpredictable independent variables.
In some optional embodiments, the prediction result of the optimal model of the dependent variable on the time series of the dependent variable may be obtained as the dependent variable prediction result; obtaining the prediction results of the respective optimal models of the plurality of groups of the predictable independent variables on the time series of the plurality of groups of the predictable independent variables to obtain a plurality of independent variable prediction results; and performing time series regression calculation on the dependent variable prediction results, the multiple independent variable prediction results and the multiple groups of unpredictable independent variables to obtain the comprehensive prediction model.
The regression calculation can be realized based on a multiple linear regression method and/or an improved algorithm of the multiple linear regression method. The improved algorithm of the multiple linear regression method comprises the following steps: at least one of autoregressive, dynamic, stepwise, principal element, canonical correlation, and iterative. Each of the above improved algorithms represents a different mathematical model.
Where multiple linear regression is used to predict information about a dependent variable, it depends on known information about some dependent independent variables. This approach goes beyond simple time series prediction methods because it predicts the information of an ontology by containing information from some related independent other variable.
Wherein the prediction result of the time series yt by the optimal model of the time series yt of the dependent variable can be marked as
As a dependent variable prediction result; tagging predictions of time series xkt by an optimal model of time series xkt of predictable independent variables
Then, the comprehensive prediction model shown in the following formula can be obtained by using the multiple linear regression method:
in the above formula 1, α, β, γl、εtThe regression coefficients respectively represent corresponding variables; if the partial z-sequence is a discrete argument, a dummy variable (dummy variable) may be introduced for processing.
In some alternative embodiments, the comprehensive predictive model may be optimized.
Alternatively, a back-off Selection (backed Selection) may be used to optimize the comprehensive predictive model. Optionally, the independent variables in the comprehensive prediction model may be adjusted according to the P value corresponding to the comprehensive prediction model to optimize the comprehensive prediction model; wherein the P value is a parameter for determining the result of the hypothesis test.
In such an embodiment, a model may first be built that contains all the independent variables, i.e., contains
zltThe multiple regression model of (1). Then checking the maximum independent variable of P value for the built multiple regression model, if the maximum P value is larger than the set threshold value, eliminating the independent variable corresponding to the P value, and re-building and pre-estimating the multiple regression model with some independent variables eliminated until the maximum P valueNot greater than the set threshold. The set threshold is an empirical value, and may be set to 0.05, 0.08, 0.1, and the like, which is not limited in this embodiment. It should be understood that besides the above-mentioned P value optimization method, other methods may be used to optimize the comprehensive prediction model, such as AIC (Akaike Information criterion), BIC (Bayesian Information criteria), etc., which are not described in detail.
In this embodiment, a suitable model may be selected from a plurality of selectable predictive models according to the characteristics of the metric and the historical data, and a corresponding optimal model may be selected for the predictable dependent variables and the dependent variables that need to be predicted. The comprehensive prediction model may be trained based on the relationship between the prediction model of the dependent variable and the prediction model of the independent variable and the actual value of the dependent variable. The comprehensive prediction model can provide a universal prediction method, can flexibly implement various time series related prediction tasks, and meets various prediction requirements of users.
Based on the comprehensive model, the change trend of the dependent variable in the future time period can be predicted. As will be exemplified below.
When performing the prediction function, the user needs to provide the predicted values of the unpredictable independent variables, the system first predicts the predicted values of the predictable independent variables using the existing optimal models and parameters stored in the system, and finally performs prediction of the prediction target (dependent variable) using the comprehensive models and parameters.
Optionally, determining a time period to be predicted of the dependent variable; predicting a dependent variable prediction result corresponding to the dependent variable in the time period to be predicted according to the optimal model of the dependent variable; predicting independent variable prediction results corresponding to the plurality of groups of predictable variables in the time period to be predicted according to the respective optimal models of the plurality of groups of predictable independent variables; and inputting the dependent variable prediction results of the dependent variables in the time period to be predicted, the independent variable prediction results of the multiple groups of predictable variables in the time period to be predicted and unpredictable independent variable values in the time period to be predicted into the comprehensive prediction model to obtain the comprehensive prediction results of the dependent variables in the time period to be predicted.
Assuming that the last period of historical data is T, a time series of dependent variables (T +1, T +2, …, T + T ') for future T' periods needs to be predicted. First, for each predictable independent variable, the independent variable prediction result of the independent variable in the future t' period, namely, the predicted independent variable result can be predicted based on the optimal model and parameters corresponding to the independent variable
T + 1.., T + T'; for a dependent variable, the prediction result of the independent variable of the dependent variable in the future t' period can be predicted based on the parameters of the optimal model corresponding to the dependent variable, namely
T +1, T + T'. At the same time, the value z of each unpredictable independent variable provided by the user can be obtainedltT +1, T. Wherein, for discrete independent variables in the z sequence, the corresponding dummy variable value can be automatically converted. Next, the overall prediction result of the dependent variable may be calculated based on the overall prediction model shown in the foregoing formula 1
As shown in the following equation 2:
in equation 2, T ═ T +1,., T + T',
and (3) showing the comprehensive prediction result of the dependent variable at the time t.
Optionally, calculating an initial error of the comprehensive prediction model according to the measured error index and a comprehensive prediction result of the comprehensive prediction model; setting a tracking signal according to the initial error; and reconstructing or optimizing the comprehensive forecasting model according to the tracking signal.
Assuming that the maximum time period in modeling is T0, the initial error can be determined:
in equation 3, F is the calculated function of the selected metric, ytFor the predicted value of the optimal integrated model obtained in equation 1,
is the predicted value of the optimal comprehensive model obtained by the formula 2. Setting a tracking signal
Over time, historical data stored by the system, including dependent variables and all predictable independent variable time series, may be collected, integrated, and updated, namely: the time series yt of the dependent variable and the time series xkt of the K predictable independent variables. Alternatively, the tracking signal may be updated in accordance with an update of the historical data.
Optionally, predicting a comprehensive prediction result of the dependent variable in a future time period to be predicted according to the comprehensive prediction model; and updating the tracking signal according to the comprehensive prediction result of the dependent variable in the future time period to be predicted.
Alternatively, for each newly updated dependent variable yt, the tracking signal may be updated using equation 4 below:
where θ represents a tracking signal smoothing coefficient, the value of which may be specified by the user.
Wherein, for any period t in the updated history data,
(mu is a user-specifiable threshold), this means that the existing synthesis model and sequence prediction model are not suitableAnd (4) combining historical data, needing to be modeled and optimized again and not repeated.
The above embodiment will be further explained with reference to fig. 3.
As shown in fig. 3, the method of training the comprehensive prediction model and predicting the dependent variable based on the comprehensive prediction model can be described as the following steps:
and B1, collecting and integrating historical data through a historical data source.
B2, determining a prediction target (dependent variable) time sequence, determining a related influence factor (independent variable) time sequence, and determining a continuous independent variable and a discrete independent variable. Predictable independent variables and unpredictable independent variables are determined among the continuous independent variables.
And B3, performing data cleaning on all time series, supplementing missing values and correcting abnormal values. If a deletion value for a sequence exceeds a certain percentage of the total number of sequence data, the sequence will be removed. The cleaned data time series are a dependent variable series yt, K predictable independent variable series xkt (K1.., K), and L unpredictable independent variable series zlt (L1.., L). For example, in an energy consumption prediction scenario, yt may be implemented as a sequence xkt of K predictable independent variables of the energy consumption index, as temperature, production volume, etc. associated with the energy consumption index. In the energy consumption prediction scenario, yt may be implemented as a sequence xkt of K predictable independent variables of the energy consumption index, which may be implemented as predictions of energy consumption related indices such as air temperature, production volume, etc.
B4, identifying that the system needs optimized metric errors (e.g., mean absolute error (MAD), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), and Root Mean Square Error (RMSE) — the user can specify, customize, and extend any applicable metric error.
B5, dividing the historical data correlation sequence (yt, and K predictable independent variable sequences xkt (K ═ 1.., K)) into training data and test data for use. For example, the first 75% are training data and the last 25% are testing data.
B6, confirming various candidate prediction models and historical data applicable conditions used by the system. Further, the user may specify, customize, and extend any applicable candidate predictive model.
B7, for each predicted sequence (yt, xkt), screening out the optimal model and parameters for each predicted sequence according to the application conditions of various candidate prediction models and historical data and the optimized measurement error index by using training data and test data. That is, an optimal model is screened for yt and an optimal model is screened for each predictable xkt. The system will store the resulting optimal model and parameters. Storage methods include, but are not limited to, text file storage and Object Serialization (Object Serialization) storage.
B8, respectively marking the prediction values obtained by the optimal model and the parameters of each prediction sequence
And
the following comprehensive prediction models were found using multiple linear regression:
alpha, beta, and gamma as described abovel、εtRespectively representing the regression coefficients of the corresponding variables. If some of the z-sequences in the above model are discrete arguments, dummy variables (dummy variables) must be introduced to handle them. Introducing dummy variables is a standard approach to dealing with categorical variables of multiple regression (equivalent to discrete independent variables).
The optimization process of the comprehensive model can be realized by a backstepping Selection method (backsward Selection): first, it is established to include all the arguments (i.e.
And
) The multiple regression model of (1). Then checking the independent variable with the maximum p value for the established regression model, and if the maximum p value is greater than 0.05, rejecting the independent variable corresponding to the p valueAnd (4) rebuilding and pre-estimating a new multiple regression model with some independent variables removed until the maximum p value of the obtained model is not more than 0.05. Optimization of the integrated model may also be performed by other methods, such as AIC, BIC, etc.
B9, storing the optimal integrated model obtained from B8 in the system, and calculating the initial error of the optimal integrated model according to the selected error measurement index. Setting the maximum time period in modeling as T0, initial error
Where F is a calculated function of the selected metric,
the predicted value of the optimal comprehensive model obtained for B8. Then setting the tracking signal
And B10, predicting the predicted value of the dependent variable of a plurality of time intervals in the future according to the user requirement.
Assuming that the last period of historical data is T, the dependent variables (T +1, T + 2.., T + T ') for the future T' period need to be predicted, and the system will implement the following steps:
(1) for each predictable independent variable included in the integrated model
(its regression coefficient. beta.)0And betakNon-zero), using the optimal prediction model of the time series stored in the system in B7, predicting
And
(t＝T+1，...，T+t’)；
(2) for each unpredictable independent variable (z) included in the integrated modellt) The user provides its predicted value (i.e., from T + T1.., T + T' period z)ltA value of (d);
(3) for discrete independent variables in the z sequence, the system automatically converts the discrete independent variables into corresponding dummy variable values;
(4) calculating a predicted value of the dependent variable according to a comprehensive model stored in a system in B9 (T ═ T + 1.., T ═ T ═ T'):
(5) outputting the predicted value according to the user requirement
And stored in the system for later use.
B11, collecting, integrating and updating historical data stored by the system over time, including dependent variables and all predictable independent variable time series (yt and K predictable independent variable series xkt).
B12, for each dependent variable yt newly updated in B11, the tracking signal is updated in the following way:
where θ is a user-specifiable tracking signal smoothing coefficient.
B13, for any time period t in the data updating, if
(mu is a threshold value which can be specified by a user), the existing comprehensive model and the sequence prediction model are not suitable for historical data, and need to be modeled and optimized again.
It should be noted that the execution subjects of the steps of the methods provided in the above embodiments may be the same device, or different devices may be used as the execution subjects of the methods. For example, the execution subjects of step 201 to step 204 may be device a; for another example, the execution subject of steps 201 and 202 may be device a, and the execution subject of step 203 may be device B; and so on.
In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations are included in a specific order, but it should be clearly understood that the operations may be executed out of the order presented herein or in parallel, and the sequence numbers of the operations, such as 201, 202, etc., are merely used for distinguishing different operations, and the sequence numbers do not represent any execution order per se. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.
It should be noted that the prediction method provided in the embodiments of the present application may be implemented based on an open platform, and on the open platform, the following operations may be performed:
(1) from a plurality of selectable predictive models, an appropriate model may be selected based on training errors and characteristics of historical data.
(2) Predictive values for the predicable dependent variable and the associated independent variable that needs to be predicted.
(3) And (4) finding the relationship between the predicted values of the dependent variables and the predicted values of the independent variables and the actual values of the dependent variables by using a multivariate linear regression method.
(4) And (3) optimizing various parameters required by the model by using the prediction model found in (1), (2) and (3) and training data for realizing the partition.
(5) And (4) predicting information of a certain period or a plurality of periods in the future by utilizing the comprehensive prediction model formed in the step (4).
(6) And (5) calculating a prediction error and integrating a tracking signal by using the prediction result and the updated actual value to determine whether the current prediction model is suitable for continuously updated historical data or not and whether re-optimization is needed or not.
Based on the above, the open platform can realize a universal prediction method, can flexibly implement various time-series related prediction tasks, and meets various prediction requirements of users.
The invention can be applied, but not limited, to energy-related time series predictions for the following industries and fields: construction sites (offices, IDC, hospitals, hotels, etc.), industrial fields (machining, textile and printing, fine chemical engineering, new energy production, chip manufacturing, etc.), key equipment (general equipment and systems, motors, boilers, air conditioners, lighting, etc.); the traffic field (new energy commercial vehicle and two-wheel vehicle); new energy devices (photovoltaic, wind power, energy storage (electricity, heat, cold), etc.) are not described in detail.
Fig. 4 is a schematic structural diagram of an electronic device provided in an exemplary embodiment of the present application, where the electronic device is adapted to the data prediction model training method provided in the foregoing embodiment. As shown in fig. 4, the electronic apparatus includes: a memory 401 and a processor 402.
The memory 401 is used for storing computer programs and may be configured to store other various data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phonebook data, messages, pictures, videos, and so forth.
The memory 401 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
A processor 402, coupled to the memory 401, for executing the computer program in the memory 401 for: determining a plurality of candidate prediction models and historical data required by training; the historical data comprises a time series of a set of dependent variables, a time series of a plurality of sets of predictable independent variables and a time series of a plurality of sets of unpredictable independent variables; determining an optimal model of the dependent variable and optimal models of the multiple groups of predictable independent variables from the multiple candidate prediction models according to the historical data and the measurement error index adopted by optimization; and obtaining a comprehensive prediction model of the dependent variable according to the optimal model of the dependent variable, the optimal models of the plurality of groups of predictable independent variables and the time sequence of the plurality of groups of unpredictable independent variables.
Further optionally, when obtaining the comprehensive predictive model of the dependent variable according to the optimal model of the dependent variable, the optimal models of the multiple sets of predictable independent variables, and the time series of the multiple sets of unpredictable independent variables, the processor 402 is specifically configured to: obtaining an optimal model of the dependent variable, and taking a prediction result of the time sequence of the dependent variable as a dependent variable prediction result; obtaining the prediction results of the respective optimal models of the plurality of groups of the predictable independent variables on the time series of the plurality of groups of the predictable independent variables to obtain a plurality of independent variable prediction results; and performing time series regression calculation on the dependent variable prediction results, the multiple independent variable prediction results and the multiple groups of unpredictable independent variables to obtain the comprehensive prediction model.
Further optionally, the processor 402 is further configured to: determining a time period to be predicted of the dependent variable; predicting a dependent variable prediction result corresponding to the dependent variable in the time period to be predicted according to the optimal model of the dependent variable; predicting independent variable prediction results corresponding to the plurality of groups of predictable variables in the time period to be predicted according to the respective optimal models of the plurality of groups of predictable independent variables; and inputting the dependent variable prediction results of the dependent variables in the time period to be predicted, the independent variable prediction results of the plurality of groups of predictable variables in the time period to be predicted and unpredictable independent variable values in the time period to be predicted into the comprehensive prediction model to obtain the comprehensive prediction results of the dependent variables in the time period to be predicted.
Further optionally, the processor 402 is further configured to: adjusting independent variables in the comprehensive model according to the P value corresponding to the comprehensive prediction model so as to optimize the comprehensive prediction model; wherein the P value is a parameter for determining the hypothesis test result.
Further optionally, the processor 402 is further configured to: calculating an initial error of the comprehensive prediction model according to the error measurement index and a comprehensive prediction result of the comprehensive prediction model; setting a tracking signal according to the initial error; and reconstructing or optimizing the comprehensive prediction model according to the tracking signal.
Further optionally, the processor 402 is further configured to: according to the comprehensive prediction model, predicting a comprehensive prediction result of the dependent variable in a future time period to be predicted; and updating the tracking signal according to the comprehensive prediction result of the dependent variable in the future time period to be predicted.
Further optionally, the measured error index includes: at least one of a mean error, an absolute deviation, a mean absolute error, a mean absolute percentage error, and a root mean square error.
Further optionally, the plurality of candidate predictive models comprises: one or more of a single and double moving average model, a single exponential smoothing model, a dual exponential smoothing model for Brown and Holter, a seasonal forecasting model for Brown and Holter and Wentt, an autoregressive synthetic moving average model, a Box-Jenkins model, a multivariate ARIMA model, and a neural network model.
Further, as shown in fig. 4, the electronic device further includes: communication components 403, display components 404, power components 405, audio components 406, and the like. Only some of the components are schematically shown in fig. 4, and the electronic device is not meant to include only the components shown in fig. 4.
Wherein the communication component 403 is configured to facilitate communication between the device in which the communication component is located and other devices in a wired or wireless manner. The device in which the communication component is located may access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, or 5G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component may be implemented based on Near Field Communication (NFC) technology, Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
Among other things, the display component 404 includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
The power supply module 405 provides power to various components of the device in which the power supply module is located. The power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device in which the power component is located.
The audio component 406 may be configured to output and/or input audio signals, among other things. For example, the audio component includes a Microphone (MIC) configured to receive an external audio signal when the device in which the audio component is located is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in a memory or transmitted via a communication component. In some embodiments, the audio assembly further comprises a speaker for outputting audio signals.
In this embodiment, a suitable model may be selected from a plurality of selectable prediction models according to the characteristics of the measured error index and the historical data, and a corresponding optimal model may be selected for the predictable dependent variable and the relevant independent variable that needs to be predicted. The comprehensive prediction model may be trained based on the relationship between the prediction model of the dependent variable and the prediction model of the independent variable and the actual value of the dependent variable. The comprehensive prediction model can provide a universal prediction method, can flexibly implement various time series related prediction tasks, and meets various prediction requirements of users.
Accordingly, the present application further provides a computer-readable storage medium storing a computer program, where the computer program is capable of implementing the steps that can be executed by the electronic device in the foregoing method embodiments when executed.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A data prediction model training method is characterized by comprising the following steps:
determining a plurality of candidate prediction models and historical data required by training; the historical data comprises a time series of a set of dependent variables, a time series of a plurality of sets of predictable independent variables and a time series of a plurality of sets of unpredictable independent variables;
determining an optimal model of the dependent variable and optimal models of the multiple groups of predictable independent variables from the multiple candidate prediction models according to the historical data and the measurement error index adopted by optimization;
and obtaining a comprehensive prediction model of the dependent variable according to the optimal model of the dependent variable, the optimal models of the plurality of groups of predictable independent variables and the time sequence of the plurality of groups of unpredictable independent variables.
2. The method of claim 1, wherein obtaining the comprehensive predictive model of the dependent variable from the optimal model of the dependent variable, the optimal models of the sets of predictable independent variables, and the time series of the sets of unpredictable independent variables comprises:
obtaining an optimal model of the dependent variable, and taking a prediction result of the time sequence of the dependent variable as a dependent variable prediction result;
obtaining the prediction results of the respective optimal models of the plurality of groups of the predictable independent variables on the time series of the plurality of groups of the predictable independent variables to obtain a plurality of independent variable prediction results;
and performing time series regression calculation on the dependent variable prediction results, the multiple independent variable prediction results and the multiple groups of unpredictable independent variables to obtain the comprehensive prediction model.
3. The method of claim 2, further comprising:
determining a time period to be predicted of the dependent variable;
predicting a dependent variable prediction result corresponding to the dependent variable in the time period to be predicted according to the optimal model of the dependent variable;
predicting independent variable prediction results corresponding to the plurality of groups of predictable variables in the time period to be predicted according to the respective optimal models of the plurality of groups of predictable independent variables;
and inputting the dependent variable prediction results of the dependent variables in the time period to be predicted, the independent variable prediction results of the plurality of groups of predictable variables in the time period to be predicted and unpredictable independent variable values in the time period to be predicted into the comprehensive prediction model to obtain the comprehensive prediction results of the dependent variables in the time period to be predicted.
4. The method of claim 2, further comprising:
adjusting independent variables in the comprehensive model according to the P value corresponding to the comprehensive prediction model so as to optimize the comprehensive prediction model; wherein the P value is a parameter for determining the hypothesis test result.
5. The method of claim 2, further comprising:
calculating an initial error of the comprehensive prediction model according to the error measurement index and a comprehensive prediction result of the comprehensive prediction model;
setting a tracking signal according to the initial error;
and reconstructing or optimizing the comprehensive prediction model according to the tracking signal.
6. The method of claim 5, further comprising:
according to the comprehensive prediction model, predicting a comprehensive prediction result of the dependent variable in a future time period to be predicted;
and updating the tracking signal according to the comprehensive prediction result of the dependent variable in the future time period to be predicted.
7. The method according to any one of claims 1-6, wherein the measure of the error index comprises: at least one of a mean error, an absolute deviation, a mean absolute error, a mean absolute percentage error, and a root mean square error.
8. The method according to any of claims 1-6, wherein the plurality of candidate predictive models comprises: one or more of a single and double moving average model, a single exponential smoothing model, a dual exponential smoothing model for Brown and Holter, a seasonal forecasting model for Brown and Holter and Wentt, an autoregressive synthetic moving average model, a Box-Jenkins model, a multivariate ARIMA model, and a neural network model.
9. An electronic device, comprising: a memory and a processor;
the memory is to store one or more computer instructions;
the processor is to execute the one or more computer instructions to: performing the steps of the method of any one of claims 1-8.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, is adapted to carry out the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110158803.XA CN112819178A (en) | 2021-02-05 | 2021-02-05 | Data prediction model training method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110158803.XA CN112819178A (en) | 2021-02-05 | 2021-02-05 | Data prediction model training method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112819178A true CN112819178A (en) | 2021-05-18 |
Family
ID=75861597
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110158803.XA Pending CN112819178A (en) | 2021-02-05 | 2021-02-05 | Data prediction model training method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112819178A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113358582A (en) * | 2021-06-04 | 2021-09-07 | 山东国瑞新能源有限公司 | Method, equipment and medium for detecting concrete structure defects |
| CN114330908A (en) * | 2021-12-31 | 2022-04-12 | 中国民航信息网络股份有限公司 | Seat booking demand prediction method and device and revenue management system |
| CN115421465A (en) * | 2022-10-31 | 2022-12-02 | 北京聚新工程技术有限公司 | Optimized self-adaptive control method and system for textile equipment |
| CN116542495A (en) * | 2023-07-05 | 2023-08-04 | 浙江和达科技股份有限公司 | Intelligent water supply scheduling method and device based on data mining and electronic equipment |
-
2021
- 2021-02-05 CN CN202110158803.XA patent/CN112819178A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113358582A (en) * | 2021-06-04 | 2021-09-07 | 山东国瑞新能源有限公司 | Method, equipment and medium for detecting concrete structure defects |
| CN114330908A (en) * | 2021-12-31 | 2022-04-12 | 中国民航信息网络股份有限公司 | Seat booking demand prediction method and device and revenue management system |
| CN115421465A (en) * | 2022-10-31 | 2022-12-02 | 北京聚新工程技术有限公司 | Optimized self-adaptive control method and system for textile equipment |
| CN115421465B (en) * | 2022-10-31 | 2023-01-17 | 北京聚新工程技术有限公司 | Optimized self-adaptive control method and system for textile equipment |
| CN116542495A (en) * | 2023-07-05 | 2023-08-04 | 浙江和达科技股份有限公司 | Intelligent water supply scheduling method and device based on data mining and electronic equipment |
| CN116542495B (en) * | 2023-07-05 | 2024-02-23 | 浙江和达科技股份有限公司 | Intelligent water supply scheduling method and device based on data mining and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112819178A (en) | Data prediction model training method, device and storage medium | |
| CN109784806B (en) | Supply chain control method, system and storage medium | |
| Trapero et al. | Analysis of judgmental adjustments in the presence of promotions | |
| Ghasemi et al. | Optimal condition based maintenance with imperfect information and the proportional hazards model | |
| US9047559B2 (en) | Computer-implemented systems and methods for testing large scale automatic forecast combinations | |
| CN109376971B (en) | Load curve prediction method and system for power consumers | |
| Farah et al. | Bayesian emulation and calibration of a dynamic epidemic model for A/H1N1 influenza | |
| US20130024167A1 (en) | Computer-Implemented Systems And Methods For Large Scale Automatic Forecast Combinations | |
| Yang et al. | The optimization of total laboratory automation by simulation of a pull-strategy | |
| Gautam et al. | A novel moving average forecasting approach using fuzzy time series data set | |
| US20150348066A1 (en) | Business forecasting using predictive metadata | |
| Jin et al. | Multimode variation modeling and process monitoring for serial-parallel multistage manufacturing processes | |
| CN108830417B (en) | ARMA (autoregressive moving average) and regression analysis based life energy consumption prediction method and system | |
| US20230409014A1 (en) | Industrial Bottleneck Detection and Management Method and System | |
| Alexopoulos et al. | Quantifying the flexibility of a manufacturing system by applying the transfer function | |
| Wenninger et al. | Benchmarking energy quantification methods to predict heating energy performance of residential buildings in Germany | |
| JP7393244B2 (en) | Time series data prediction device and time series data prediction method | |
| Kasa | Approximating innovation potential with neurofuzzy robust model | |
| Sharko et al. | Methodological basis of causal forecasting of the economic systems development management processes under the uncertainty | |
| Ferencek et al. | Data quality assessment in product failure prediction models | |
| US20110231336A1 (en) | Forecasting product/service realization profiles | |
| Özcan et al. | Healthcare expenditure prediction in Turkey by using genetic algorithm based grey forecasting models | |
| Almomani et al. | Selecting a good stochastic system for the large number of alternatives | |
| JP4777941B2 (en) | Demand forecasting method and apparatus | |
| CN116820767A (en) | Cloud resource management method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210518 |
|
| RJ01 | Rejection of invention patent application after publication |