CN113537540B - Heat supply gas consumption prediction model based on automatic characteristic engineering - Google Patents

Abstract

The invention relates to a DFS-LSTM prediction model, which is mainly technically characterized in that: the model regards every 10-minute heat supply data and every minute heat supply data as two entity tables, uses a depth feature synthesis algorithm to realize automatic construction of features, and solves the problems that the traditional feature engineering depends on manpower, is tedious, time-consuming, and is easy to make mistakes when extracting the features. In order to further improve the prediction precision of the model, a plurality of classical prediction models are compared to find that the LSTM model has good performance on time sequence prediction, so that a DFS-LSTM heat and gas consumption prediction model introducing a deep synthesis algorithm is provided, and the accuracy of the model is verified through test set data. The invention has reasonable design, and adopts heat supply data every 10 minutes and every minute as a data set of an experiment in order to verify the accuracy of the model. And evaluating the prediction effects of the DFS-LSTM model, the RNN model and the SVM model through two evaluation criteria of MSE and MAPE. The results show that the model provided by the invention has good prediction effect in terms of MSE (mean square error) and MAPE (mapping image) results. The invention has good applicability to the prediction of gas consumption in the field of heat supply, and can more effectively predict the gas consumption, thereby effectively utilizing energy.

Description

Heat supply gas consumption prediction model based on automatic characteristic engineering

Technical Field

The invention belongs to time series data prediction in the field of heat supply, and relates to a time series prediction model based on automatic feature engineering.

Background

The heating data are sampled according to a certain sampling period to obtain a time sequence. A time series is a set of variables that are arranged in ascending or descending order of time, usually data that is chosen for a variable that is continuous over a period of time. Time series is very important and complex data, and widely exists in the fields of social life and engineering, such as total production value, price index of consumer goods, commodity sales, stock price and the like in the economic field.

Prediction is the conjecture of people on the behavior of a research object in a period of time in the future according to historical data, and accurate and reasonable prediction is the basis for reasonable decision making of people. In scientific research and engineering practice, rapid and accurate prediction of time series is always a subject of close attention of many scholars. The time sequence stores much information and contains operation rules of a plurality of systems, so that the time sequence data are analyzed and researched to find the evolution rule in the data, thereby predicting before the system operates, and having important significance and value in practical application.

In recent years, researchers have developed many time series prediction models. Support Vector Machine (SVM) methods are used in a variety of contexts, including the prediction of financial stock market, due to its many attractive features and superior performance in dealing with a wide range of problems. The Recurrent Neural Network (RNN) forms a feedback system by taking the output of the neuron at the previous moment as the input of the neuron at the current moment, effectively utilizes the dependency relationship among data and greatly improves the time series prediction effect. But most conventional RNNs employ a time-dependent Back Propagation (BPTT) algorithm. The algorithm has the disadvantage that the gradient disappears or the gradient explodes and the like due to the increase of the number of network layers along with the lapse of time. The long-short term memory network (LSTM) is a special RNN model, and solves the problem that the RNN cannot learn the long-term dependence of time sequence data. The method is widely applied to the fields of emotion analysis, voice recognition, handwritten character recognition, lexical analysis and the like, and obtains satisfactory results.

In machine learning, if an effective prediction model is obtained, additional work is required to ensure that the extracted features are effective as much as possible, which is called feature engineering. The feature engineering is a process of processing original data by using professional background knowledge and skills to generate some information capable of describing sample features to replace the original data as the input of a model, so that the features can play a better role in a machine learning algorithm. The feature engineering process comprises the steps of feature extraction, feature construction, feature selection and the like, wherein the feature construction refers to the step of combining some attributes in original data through some calculation methods to generate some new features with more expression capacity. This process is typically done manually, and often takes time to review the raw data, think about the underlying form and data structure of the problem, and knowledge of the sensitivity of the data and related fields can help construct the features.

Researchers have developed many automated feature construction methods.

AdaBoost is used for automatic feature construction, feature combination can be automatically carried out in the process of training a model by using the idea of an AdaBoost algorithm, and the characteristics of the AdaBoost algorithm ensure that an optimal combination mode can be found when the features are combined, so that blind exhaustive feature generation is avoided. However, the method is to linearly combine all the original features together, the combination mode is single, and the new features may have a problem of insufficient robustness when used.

The gradient lifting tree (GBDT) and Logistic Regression (LR) algorithm is based on Boosting idea and uses the gradient lifting tree (GBDT) to carry out nonlinear combination on the features, and the Logistic Regression (LR) carries out importance scoring on the combined features. However, the main final purpose of the method is to train the model to predict data, and feature combination is only an intermediate step of the method and does not output a form after feature combination.

A Depth Feature Synthesis (DFS) algorithm that iteratively runs a feature construction function in a depth traversal fashion to generate a feature set by establishing a linkage between data tables.

Based on the analysis of the research method, the invention provides an LSTM heat supply gas consumption prediction model which introduces a deep synthesis algorithm to automatically construct features as input dimensions to realize prediction fitting of gas consumption.

Disclosure of Invention

The invention predicts the heat supply gas consumption data, firstly, the feature engineering is considered to improve the gas consumption prediction accuracy, while the traditional feature engineering extracts features from the relational entity completely depends on manpower, is tedious, time-consuming and easy to make mistakes, because the heat supply data is data acquired by taking every minute as a unit, and the predicted instantaneous gas consumption is every 10 minutes, every 10 minutes of heat supply data is regarded as a entity table E ^k Regarding the heating data per minute as another entity table E ^l Thus, there is an obvious anteroposterior relationship between the two entity tables, so it is consideredAutomatic construction of features is contemplated using a depth feature synthesis algorithm. And (3) excavating deep features through a depth feature synthesis algorithm, thereby enhancing the prediction accuracy of predicting the instantaneous gas consumption every 10 minutes.

Because the input historical data is time series data, and compared with an SVM (support vector machine), an RNN (neural network regression) model has a good prediction effect on time series data, the LSTM neural network is selected as the prediction model.

In conclusion, a deep synthesis algorithm is introduced on the basis of the LSTM neural network model to improve the prediction effect of the model.

A heat supply gas consumption prediction model based on automatic characteristic engineering comprises the following steps:

step 1: and performing null data processing and invalid data processing on the heat supply data, and dividing a training set and a test set.

Step 2: and selecting a depth feature synthesis algorithm to automatically construct features, and automatically synthesizing the preprocessed heat supply data into features with depth.

And step 3: and constructing a DFS-LSTM model, wherein the input dimension of the model is the sum of the automatically constructed features and the original features, and initializing model parameters, including network weight and bias for generating a final prediction model.

And 4, step 4: and storing the trained model, and inputting the test set data to obtain a final prediction result.

Drawings

FIG. 1 is a flow chart of a model for predicting the amount of gas used for heating based on automatic characteristic engineering

FIG. 2 automatic construction of feature maps for depth feature synthesis algorithms

FIG. 3 is a MSE result graph of different prediction models predicting gas consumption instantaneously in every 10 minutes

FIG. 4 is a MAPE result graph of prediction of instantaneous gas consumption every 10 minutes by different prediction models

Detailed Description

Step 1: and (4) acquiring and preprocessing experimental data, and dividing a training set and a testing set.

In the example, the data adopted is the gas consumption data from 2017, 11 and 1 days to 2017, 11 and 7 days in the heating season, and the total data is 10065 effective data. The original 8052 pieces of original data are selected as a training set for model training, the rest data of the original data are used as a test set for verifying the feasibility of the model, the heat and gas consumption is predicted, and the original data are input into an LSTM model.

Step 2: features are automatically constructed using a depth feature synthesis algorithm.

The input to the depth feature synthesis algorithm is the cross-connected entities (relational representation) and the data tables associated with it. Each instance of each entity has a unique identity. Each entity instance in the table has a unique identifier. Instances of related entities may be referenced between entities by using unique identifiers of the related entities. There are three types of features generated, efeat, dfeat, rfeat, respectively, by jointly analyzing two related entities E in generating the dfeat and rfeat types of features ^l And E ^k It follows that these two entities have one of two relationships:

(1) forward relationship: presence in entity E ^l Examples m and E of ^k The relationship between the individual attributes of another entity i.

(2) The backward relation is as follows: from E ^k Examples i to E of (1) ^l All instances M having a forward relationship with k ═ 1.. M } relationship.

The three types of feature calculation modes are respectively as follows:

(1) efeat: by calculating the value of each attribute, features are obtained which can be obtained by pairing x _i，j The calculation function is applied element by element. This calculation can be expressed as:

x _i，j ＝dfeat(x _i，j ，i) (1)

(2) dfeat: applied to the forward relations in the relation table. Related entity i ∈ E ^k The feature in (1) is directly transferred as m ∈ E ^k The characteristics of (1).

(3) rfeat: applied to the backward relation. By applying mathematical functions to

Coming guiderExit entity E ^k E by extracting all values of the features j in the entity ^k The extraction conditions are e ^k I. This conversion can be expressed as:

the number of features z that the algorithm synthesizes for a given entity is given by:

establishing the preprocessed heat supply data into two entity tables: data E every 10 minutes ^k (ii) a Data per minute E ^l . And (3) mining the relationship characteristics (default, reset) of the two entity tables through Std, Mean and Max elementary operations, wherein the obtained characteristics comprise the average value of the return water temperature of the boiler room in 10 minutes per minute, the average difference of the local environment temperature in 10 minutes per minute, the maximum value of the total gas consumption in 10 minutes per minute and the like.

And 3, step 3: and constructing a DFS-LSTM model, wherein the input dimension of the model is the sum of the automatically constructed features and the original features, training the model and storing the model.

The DFS-LSTM model is designed into a two-layer LSTM network, preprocessed data are used as input of an LSTM neural network for training, and parameters of a forgetting gate, an input gate and an output gate of the LSTM neural network are continuously adjusted in a large amount of training for learning. The first layer of LSTM hidden layer inputs the output of each LSTM neuron of the layer to the LSTM neuron corresponding to the next layer of LSTM hidden layer for calculation. After two layers of LSTM calculation, a full connection layer is passed to finally obtain the value to be predicted.

The DFS-LSTM model comprises 2 layers of LSTM units, the number of the LSTM units on the first layer is finally set to be 128, the number of the LSTM units on the 2 nd layer is set to be 64, then the full-connection layer connected with a single nerve unit is used as an output layer of the gas consumption prediction model, and the result obtained from the output layer is subjected to inverse normalization to obtain a final prediction value. The activation function for the model was chosen to be the Relu function, the batch _ size was 32, and the number of training (epoch) was 100.

Because the size difference of each dimension data is large, the data needs to be normalized to reduce the error and improve the prediction precision. The normalization method employed herein is min-max minimum maximum normalization, mapping the data values between [0, 1], and the formula is as follows:

wherein x is _min Is the minimum value, x, of the original sample data component _max Is the maximum value, x, of the original sample data component ^* Is the data to be input to the neural network.

For the whole training process, the model adopts a back propagation algorithm, in the aspect of Gradient optimization, the Adam algorithm is adopted as the Gradient optimization algorithm of the model, and compared with a random Gradient descent (SGD) (stochastic Gradient decision) method, the learning step length of parameters of each iteration of Adam has a certain range, larger Gradient fluctuation cannot be generated, and the parameters are relatively stable.

And 4, step 4: test data is input to detect the model, and the prediction and curve fitting effects of the model are verified

The invention adopts the Mean Square Error of MSE (Mean-Square Error) and the Mean Absolute Percentage Error of MAPE (Mean Absolute Percentage Error) as the evaluation standard of the model accuracy, and the formula expression of MSE and RMSE is as follows:

y in formulae (5) and (6) _i The real value of the gas consumption data at the moment i,

the predicted value obtained by the model at the moment i, and n is the total amount of data in the test set. For a predictive model, smaller values of MSE and RMSE represent better predictions.

Claims (1)

1. A heat supply gas consumption prediction method based on automatic characteristic engineering is characterized by comprising the following steps:

step 1: performing null value data processing and invalid data processing on the heat supply data to generate data of a specified prediction step length, and dividing a training set and a test set; the adopted data is the gas consumption data of a certain heating season, 10065 effective data are selected, the original data of the previous 8052 data are selected as a training set for model training, the other data are used as a test set for verifying the feasibility of the model, the gas consumption for heating is predicted, and the original data are input into an LSTM model;

step 2: selecting a depth feature synthesis algorithm to automatically construct features, and automatically synthesizing the preprocessed heat supply data into features with depth; the input of the depth feature synthesis algorithm is cross-connected entities and a data table related to the cross-connected entities, and each instance of each entity has a unique identifier; each entity instance in the table has a unique identifier; instances of related entities may be referenced between entities by using unique identifiers of the related entities; there are three types of features generated, efeat, dfeat, rfeat, respectively, by jointly analyzing two related entities E in generating the dfeat and rfeat types of features ^l And E ^k It follows that these two entities have one of two relationships: (1) forward relationship: presence in entity E ^l Examples m and E of ^k The relationship between individual attributes of another entity i; (2) the backward relation is as follows: from E ^k Examples i to E in ^l A relationship of M ═ {1.. M } for all instances having a forward relationship with k;

the three types of feature calculation modes are respectively as follows:

(1) efeat: by calculating each attribute value, features are obtained which can be obtained by pairing x _i，j Applying a calculation function element by element;this calculation can be expressed as:

x _i，j ＝efeat(x _{i ，j} ，i) (1)

(2) dfeat: applying to the forward relation in the relation table; related entity i ∈ E ^k Is directly transferred as m ∈ E ^k The features of (1);

(3) rfeat: apply to the backward relationship; by applying mathematical functions to

To derive entity E ^k Example i of (1) which combines E by extracting all values of the features j in the entity ^k The extraction conditions are e ^k I; this conversion can be expressed as:

the number of features z that the algorithm synthesizes for a given entity is given by:

establishing the preprocessed heat supply data into two entity tables: data E every 10 minutes ^k (ii) a Data per minute E ^l (ii) a Digging out the relation characteristics of the two entity tables through Std, Mean and Max elementary operations, wherein the relation characteristics are defaat and refeat, and the obtained characteristics comprise the average value of the return water temperature of the boiler room in each minute within 10 minutes, the average difference of the local environment temperature in each minute within 10 minutes and the maximum value of the total gas consumption in each minute within 10 minutes;

and step 3: constructing a DFS-LSTM model, wherein the input dimension of the model is the sum of the automatically constructed features and the original features, and initializing model parameters, including generating network weight and bias of a final prediction model;

the DFS-LSTM model is designed into a two-layer LSTM network, and a predicted value is finally obtained through calculation of the two LSTM networks and a full connection layer; setting the number of LSTM units of a first layer to be 128 and the number of LSTM units of a layer 2 to be 64 by the DFS-LSTM model, then connecting a full-connection layer of a single nerve unit to serve as an output layer of the gas consumption prediction model, and performing inverse normalization on a result obtained from the output layer to obtain a final predicted value; the activation function of the model is selected as Relu function, the batch _ size is 32, and the training time epoch is 100; because the size difference of each dimension data is large, the data needs to be normalized to reduce errors and improve the prediction precision; the normalization method is min-max minimum and maximum normalization, and the data values are mapped between [0, 1], and the formula is as follows:

wherein xmin is a minimum value of the original sample data component, xmax is a maximum value of the original sample data component, and x is data to be input to the neural network;

and 4, step 4: storing the trained model, and inputting test set data to obtain a final prediction result: the MSE mean square error and MAPE mean absolute percentage error are used as evaluation criteria of model accuracy, and the formula of MSE and RMSE is expressed as follows:

y in formulae (5) and (6) _i The real value of the gas consumption data at the moment i,

the predicted value is obtained by the model at the moment i, and n is the total amount of data in the test set; for a predictive model, smaller values of MSE and RMSE represent better predictions.

Images