CN106886846A - A kind of bank outlets' excess reserve Forecasting Methodology that Recognition with Recurrent Neural Network is remembered based on shot and long term - Google Patents

Abstract

The invention discloses a method for forecasting reserve funds of bank outlets based on long-short-term memory cyclic neural network, which includes three stages of data preprocessing, model training and prediction. The data preprocessing stage takes the day as the unit, counts the total daily deposits and daily withdrawals of cash transactions at bank outlets, and the date attribute of the day, and constructs a feature vector; calculates the daily net amount based on the daily cash transaction records. In the model training phase, the LSTM model is trained based on historical feature vectors and daily net data. In the prediction stage, the eigenvectors of several days before the forecast date of the bank outlets are counted, and the range of the daily net amount predicted by the LSTM model is input, and the random value in this range is taken as the reserve fund requirement of the day. The invention makes full use of the historical data and the advantages of the long-short-term memory cyclic neural network in the analysis of time series data, and effectively improves the accuracy rate of reserve fund prediction at bank outlets.

Description

A Prediction of Bank Branch Reserve Fund Based on Long Short-term Memory Recurrent Neural Network method

technical field

The invention relates to a method for predicting reserve funds at bank outlets based on long-short-term memory cyclic neural networks.

Background technique

In recent years, with the rise and development of third-party payment institutions, the cash business of commercial banks has been impacted to a certain extent, and the intermediary business of commercial banks bears the brunt. The intermediary business of commercial banks mainly includes payment and settlement, guarantee, commitment, transaction, inquiry, etc., and the payment and settlement business as a traditional medium is the most important part. The financial performance system of commercial banks mainly measures the three indicators of liquidity, safety and profitability, among which the reserve ratio is the key factor affecting the three indicators.

For bank outlets, the reserve fund represents the cash business activity within its radiating range, which is affected by the date attribute. Usually, bank outlets have more frequent transactions on weekdays, especially when there is a strong demand for corporate business, while during holidays, bank outlets usually only provide self-service deposit and withdrawal services, and there are fewer large-value transactions. Therefore, when predicting the reserve fund demand of bank outlets, the date attribute needs to be considered first.

On the one hand, the reserve fund forecast of bank outlets is based on the cash transactions of bank outlets in the past period of time, mainly based on the daily net cash business of bank outlets, to forecast the demand for reserve funds. In a period of time, the net business days of bank outlets are a set of time series, so the forecasting of bank outlet reserve funds can be transformed into a time series forecasting problem. Researchers at home and abroad have carried out related research in the direction of time series. Traditional methods of constructing time series models, such as autoregressive models and linear dynamic models, usually simulate that time series are linearly related, and are not suitable for the prediction of non-linear structural data such as bank reserves.

On the other hand, with the rapid development of deep learning, neural network technology is also constantly updated. Recurrent Neural Networks (RNNs, Recurrent Neural Networks) are particularly suitable for time series forecasting due to their self-learning characteristics and their expansion model is a time-dependent model. RNNs are used in Natural Language Processing (NLP) to predict words. In theory, RNNs can remember long-term information to predict time series, but in practice, RNNs have insufficient ability to process long-term dependent information.

It can be seen that, whether in practical application or scientific research, time series has important research value. However, the existing research results cannot meet the new requirements for time series forecasting in more and more complex application scenarios. Most of the current reserve forecasting methods use statistical methods, which are difficult to efficiently and accurately meet the demand for reserve forecasting. Therefore, it is of great research significance to design a method that can efficiently and accurately predict the reserve requirements of bank outlets.

Contents of the invention

The technical problem that the present invention solves is, aiming at the deficiencies in the prior art, propose a kind of bank outlet reserve fund forecasting system based on long short-term memory cyclic neural network (Long Short Term Memory, LSTM), use bank outlet daily net amount as reserve The reference standard of payment demand, combined with the date attribute of the forecast date, establishes an LSTM forecasting model for the bank outlets, effectively digs out the reserve payment requirements of the bank outlets on the specified date from the past cash transaction records of the bank outlets, and can effectively improve the bank outlets. Reserve forecast accuracy.

In order to solve the problems of the technologies described above, the technical solution adopted in the present invention is:

A method for predicting reserve funds at bank outlets based on long-short-term memory recurrent neural networks, characterized in that it includes a data preprocessing stage, a model training stage, and a prediction stage;

In the data preprocessing stage, several days of cash transaction records of bank outlets are collected as a training set, and a transaction database is constructed. The database includes feature vectors constructed in units of days based on daily cash transaction records, and daily Record the calculated daily net amount;

In the model training stage, a long-short-term memory cycle neural network model with several hidden layers is set up, that is, an LSTM model, wherein each hidden layer contains several neurons, and the feature vector and daily net amount in the transaction database are used to pair The LSTM model is trained to obtain the optimal LSTM model weight parameters;

In the forecasting stage, determine the bank outlet to be predicted and the forecast date, collect the cash transaction records of the bank outlet in the past few days, and then convert them into feature vectors and input them into the LSTM forecasting model, and output the forecast result, that is, the predicted value of the reserve fund .

Further, in the data preprocessing stage, the daily total deposit amount, total withdrawal amount, and date attribute are counted in daily units, and a feature vector is built. The format is as follows: [month, date, total deposit amount, total withdrawal amount, date attribute], wherein, the date attribute is divided into weekdays, Saturdays, Sundays and holidays, which are marked as 0, 1, 2, 3 respectively, and the holiday attribute is superior to other date attributes;

Calculate the daily net amount according to the cash transaction records; calculate the daily net amount distribution of bank outlets within a certain period, and obtain the confidence interval; use the confidence interval as the total forecast interval, and divide it into N_CLASS sub-intervals; The sub-interval is marked as a one-hot vector of N_CLASS dimension. Each dimension in the one-hot vector corresponds to a sub-interval. Vector elements are marked with 0.

Further, the value of N_CLASS is 5.

According to the distribution chart of the daily net amount of the bank branch training set (Figure 2), it can be seen that the daily net amount of the bank branch roughly meets the characteristics of a normal distribution. Therefore, it is assumed that the data to be predicted obeys a normal distribution to determine the boundary of its value range. Next, the boundary can be divided into different intervals, so as to obtain the labels of different samples classified by intervals. Based on the assumption of normal distribution, calculate the mean (MEAN) and standard deviation (STD) of the daily net amount of the training set samples. Take the 99% confidence interval as the daily net distribution range of the entire sample [MEAN-2.576*STD, MEAN+2.576*STD]. This distribution range is equally divided into N_CLASS sub-intervals, and these sub-intervals are divided according to the standard of left closed and right open. After being divided into N_CLASS sub-intervals, number each sub-interval from 0 to N_CLASS-1, and determine the interval where the daily net amount of all samples is located. Data that is not within the range of the N_CLASS interval is divided into the 0th interval or the N_CLASS-1th interval nearby. Interval; convert the interval classification of the daily net amount of the sample into a one-hot vector form. Therefore, the reserve fund forecast of the bank outlets is to predict which specific sub-range the reserve fund demand on a specified date will fall in.

In the prediction stage, the feature vectors of different dates are sequentially input into the LSTM prediction model, and the one-hot vector of the prediction day is output after three matrix transformations of the input gate layer, the forgetting gate layer, and the output gate layer.

Further, in the forecasting stage, the subinterval in which the daily net amount on the forecasting day is obtained according to the one-hot vector of the forecasting day, and a random value in the subinterval is taken as the predicted value of reserve funds.

Further, in the model training stage, iteratively trains the LSTM model several times, and uses random small real numbers to initialize the weight parameters of the LSTM model (including input to hidden layer, hidden layer to output, hidden layer to hidden layer) Connect the weights U, V, and W), apply random deactivation for regularization, use the tanh nonlinear function as the activation function to determine the output, and select cross-entropy as the loss function, and correspond the predicted one-hot vector to the real daily net amount Compare the one-hot vectors, calculate the loss, and use the combination of Stochastic Gradient Descent (SGD) and Back Propagation (BP) to find the optimal weight parameter that minimizes the loss.

Further, in the model training stage, the number of hidden layers is 5, each hidden layer contains 50 neurons, and the LSTM model is trained for 100,000 iterations.

Each step in the model training phase is described in detail below.

Step 1: Weight initialization; during the LSTM model process, the weight parameters U, V and W need to be continuously updated to obtain optimal values. If each neuron in the network calculates the same output, the same gradient will be obtained during the backpropagation process, so that the parameters U, V, and W will be updated consistently, and the neurons will always remain symmetrical, which will affect the training effect. Therefore, in order to obtain suitable LSTM model learning parameters, the present invention chooses small random number initialization, that is, the initial values of weight parameters U, V and W are close to 0 and not equal to 0. Initialize with a very small value, so that the initial state of the neuron is random and unequal, and then calculate different updates, and in the gradient descent process, transform itself into a different part of the entire network, breaking the symmetry. The calculation formula for small random number weight initialization is as follows:

W=0.01*np.random.randn(n)*sqrt(2.0/n)

Among them, np.random is the method of generating random numbers in the Numpy library in Python, and the randn function is based on the standard deviation of Gaussian distribution to generate random numbers, where n is the number of neurons in the input layer. According to this calculation formula, the weight vector of each neuron will be initialized as a random vector, and these random vectors obey a multivariate Gaussian distribution. Therefore, in the input space, the orientation of all neurons is random.

Step 2: Regularization; in order to control the learning ability of the neural network and prevent overfitting, regularization is required during the forward propagation process. The present invention selects random inactivation (Dropout) as the regularization method. During the training process, a part of the complete neural network is sampled, and the parameters of the sub-network are updated based on the input data, so that the neurons are activated with a probability of p; the default setting of p in the present invention is 0.7.

Step 3: Activation function, the LSTM model needs to use a nonlinear function f to determine the output of the neuron. Commonly used activation functions are tanh and ReLU. The present invention adopts tanh nonlinear function for initialization, and tanh compresses the real value between [-1, 1], and its output is zero-centered. Calculated as follows:

tanh(x)=2δ(2x)-1

Among them, x represents the function input variable, δ is the Sigmoid function, and its calculation formula is as follows:

Step 4: SGD and backpropagation. During the model training process, the long-short-term memory cycle neural network will use the loss function to calculate the loss value based on the error between the predicted result and the real value after running to the last layer, and the back-propagation (BP) algorithm will reverse the loss value Passed to each neuron, and then each neuron will use stochastic gradient descent (SGD) to modify the LSTM weight parameters U, V and W. During the model training process, the LSTM weight parameters U, V and W are constantly updated to meet the minimum loss condition.

SGD is a method to iteratively solve the loss minimization. In order to find the local minimum of a function, SGD will iteratively search for the specified step distance point in the opposite direction of the gradient corresponding to the current point on the function. In SGD, each iteration does not have to traverse all samples, it will use a sample to update the parameters, and the parameters will be updated along the direction of the negative gradient.

The backpropagation algorithm uses a batch update method to update the weights and biases of the neural network, and its processing steps are shown in Table 1.

The loss function is represented by L. This method uses cross-entropy as the loss function to calculate the loss to measure the effect of model training. The calculation formula of cross-entropy is as follows:

Among them, N is the number of training samples, o _n is the predicted value of the daily net amount of the nth training sample output by the LSTM model, and y _n is the actual value of the daily net amount of the nth training sample. If the gap between o _n and y _n is larger, the loss L(y,o) is larger.

Beneficial effect:

Considering the high-dimensional and high-noise characteristics of the reserve fund sequence, the present invention converts the reserve fund sequence forecasting problem into a time series forecasting problem. In addition, considering that the long-short-term memory cyclic neural network model, that is, the LSTM model has a good performance in the classification of time series, and predicting a reasonable interval is more practical than predicting an accurate value, the present invention predicts the reserve The problem is converted to classification forecasting of time series. First, convert the daily net amount data corresponding to the sample, classify it according to the value range, and mark it with a category label; then use the real bank branch cash transaction data to train the LSTM model, and finally input the feature vectors of several days before the forecast date into The LSTM model is used to predict the reserve requirements of bank outlets. Compared with the existing ARIMA prediction method, the present invention makes full use of the dependence of different date attributes and time series, and the proposed prediction algorithm can more fully dig out the bank outlet reserve Dependence between payments, so as to solve the increasingly complex problem of reserve fund forecasting in bank outlets, improve the accuracy and precision of reserve fund forecasting, and avoid situations where the forecast is too high or too low. The model prediction results at 453 bank outlets show that compared with the existing ARIMA prediction method, the method proposed by the present invention has better performance in prediction evaluation standards MAD (mean absolute error), RMSE (root mean square error), MAPE (mean absolute Percentage error) comparison, significantly better than the ARIMA prediction method.

Description of drawings

Fig. 1 is the process flow of the method for predicting reserve funds of bank outlets based on the long-term and short-term memory cyclic network;

Figure 2 is the daily net distribution of a bank branch training set;

Fig. 3 is the training process of the bank outlet reserve fund prediction model based on the long-short-term memory recurrent neural network;

Figure 4 is a comparison chart of the prediction range of bank branch reserves and the daily net amount based on the long-term and short-term memory recurrent neural network;

Figure 5 is a comparison chart of the predicted value of bank branch reserves, the real daily net amount, and the ARIMA predicted value from January 1, 2015 to June 9, 2015;

Figure 6 is a comparison chart of the predicted value of bank branch reserves, the real daily net amount, and the ARIMA predicted value from April 1, 2015 to April 30, 2015;

Figure 7 is a comparison chart of the MAD standard comparison between the prediction results of bank branch reserves and the ARIMA prediction value from January 1, 2015 to June 9, 2015;

Figure 8 is a comparison chart of the RMSE standard between the prediction results of bank branch reserves and the ARIMA prediction value from January 1, 2015 to June 9, 2015;

Figure 9 is a comparison chart of the MAPE standard between the prediction results of bank branch reserves and the ARIMA prediction value from January 1, 2015 to June 9, 2015.

detailed description

The system architecture of the present invention is shown in Fig. 1, including three parts: data preprocessing, model training and reserve fund prediction of bank outlets. In the data preprocessing stage, the cash transaction records of the bank outlets are collected, and the transaction records are counted to obtain the daily net amount sequence of the bank outlets. The present invention will construct a feature vector of [month, date, total deposit, total withdrawal, date attribute]. Calculate the mean and standard deviation according to the daily net amount distribution of outlets, use the 99% confidence interval as the prediction interval, and divide the prediction interval into N_CLASS classes, and mark each record as a one-hot vector.

In the model training phase, the default parameter configuration used is shown in Table 1. As well as weight initialization through small random numbers, activation of tanh nonlinear functions, comparison of prediction results with one-hot vectors, calculation of cross entropy, and faster convergence of parameters during backpropagation.

Table 1 LSTM prediction model default configuration parameter values

parameter name default allocation number of hidden layers 5 LSTM model training sample days 10 The number of neurons in the hidden layer 50 activation function Tanh loss function cross entropy learning rate 0.001 LSTM model batch size 200 Initialize weight method small random number iterations 100000

The backpropagation algorithm is:

1. For all hidden layers (2≤l≤L), set ΔW ^(l) = 0, Δb ^(l) = 0, that is, ΔW ^(l) and Δb ^(l) are all-zero matrix and all-zero vector respectively ;

2. Fori=1:m

(1) Use the backpropagation algorithm to calculate the gradient matrix of the weights and biases of neurons in each layer with

(2) calculation

(3) calculation

3. Update weights and biases:

(1) calculation

(2) calculation

Among them, m is the number of training samples.

In the model prediction stage, select the eigenvectors of the daily net amount of bank cash transactions in the first ten days, input them into the LSTM model in turn, and take a random value in the prediction interval output by the prediction model as the predicted value of reserve demand.

This method uses real bank branch cash transaction data for experiments. In the forecasting method, because only the date attribute and the impact of the reserve fund as a time series are considered, a forecasting model is established for bank outlets, and its configuration parameters can be adjusted. The default configuration parameter values of the LSTM prediction model in this embodiment are shown in Table 2.

As shown in Figure 4, under the condition of a bank branch forecasting from January 1, 2015 to June 9, 2015, the proportion of the interval given by the LSTM-based forecasting model containing the real daily net amount is 85.0679%.

As shown in Figure 5, by taking a random value within the forecast interval as the final reserve demand forecast, compared with the ARIMA forecast method, the forecast result of the LSTM model is closer to the real daily net amount.

As shown in Figure 6, the prediction results of the LSTM model simulate the trend of the real daily net amount more stably, while the ARIMA prediction results are relatively stable and cannot well reflect the trend of the real daily net amount.

In addition, according to the existing time series evaluation standards, this method compares the LSTM forecasting method with the ARIMA (Autoregressive Integrated Moving Average Model, autoregressive moving average model) forecasting method in terms of MAD (mean absolute error), RMSE (root mean square error) , MAPE (mean absolute percentage error) on the three indicators of the value.

As shown in Figure 7, the MAD value of the LSTM prediction method is significantly lower than that of the ARIMA method. In addition, as the number of classification intervals increases, the MAD value of the LSTM prediction method will gradually decrease.

As shown in Figure 8, the RMSE value of the LSTM prediction method is basically equal to that of the ARIMA prediction method, but as the number of classification intervals increases, the RMSE value of the LSTM prediction method decreases slightly.

As shown in Figure 9, the MAPE value of the LSTM prediction method is significantly lower than the MAD value of the ARIMA method. In addition, as the number of classification intervals increases, the MAPE value of the LSTM prediction method will gradually decrease, basically approaching 0.

Claims (7)

1. A method for predicting reserve funds of bank outlets based on long-short-term memory recurrent neural network, is characterized in that, comprises data preprocessing stage, model training stage and prediction stage; In the data preprocessing stage, several days of cash transaction records of bank outlets are collected as a training set, and a transaction database is constructed. The database includes feature vectors constructed in units of days based on daily cash transaction records, and daily Record the calculated daily net amount; Described model training phase, set up the long-short-term memory cycle neural network model with several hidden layers, namely LSTM model, wherein each hidden layer contains several neurons, utilizes the feature vector in the transaction database and the daily net amount pair The LSTM model is trained to obtain the optimal LSTM model weight parameters; In the forecasting stage, determine the bank outlet to be predicted and the forecast date, collect the cash transaction records of the bank outlet in the past few days, and then convert them into feature vectors and input them into the LSTM forecasting model, and output the forecast result, that is, the predicted value of the reserve fund . 2. according to claim 1, based on the bank branch reserve fund forecasting method of long short-term memory recurrent neural network, it is characterized in that, in the data preprocessing stage, take day as unit, statistics daily total deposit amount, total withdrawal Amount and date attribute, build a feature vector, the format is as follows: [month, date, total deposit amount, total withdrawal amount, date attribute], where the date attribute is divided into weekdays, Saturdays, Sundays and holidays, which are marked as 0 , 1, 2, 3, and holiday attributes are superior to other date attributes; Calculate the daily net amount according to the cash transaction records; calculate the daily net amount distribution of bank outlets within a certain period, and obtain the confidence interval; use the confidence interval as the total forecast interval, and divide it into N_CLASS sub-intervals; The sub-interval is marked as a one-hot vector of N_CLASS dimension. Each dimension in the one-hot vector corresponds to a sub-interval. Vector elements are marked with 0. 3. the method for predicting reserve funds of bank outlets based on long-short-term memory recurrent neural network according to claim 2, characterized in that, the value of said N_CLASS is 5. 4. according to claim 3, based on the method for forecasting bank outlet reserve funds of long-term short-term memory cyclic neural network, it is characterized in that, in the forecast stage, the feature vectors of different dates are input into the LSTM predictive model successively, after inputting The three matrix transformations of gate layer, forget gate layer, and output gate layer output the one-hot vector of the forecast date. 5. according to claim 4, based on the method for forecasting bank outlet reserve fund of long-short-term memory recurrent neural network, it is characterized in that, described forecasting stage obtains the daily net amount of forecasted day according to the one-hot vector of forecasted day. The subinterval at , and take the random value in this subinterval as the predicted value of the reserve fund. 6. according to claim 3, based on the method for predicting bank site reserves of long-short-term memory cyclic neural network, it is characterized in that, described model training stage, iterative several times trains LSTM model, uses random small real number initialization LSTM The weight parameters of the model are regularized by random deactivation, the tanh nonlinear function is used as the activation function to determine the output, and the cross entropy is selected as the loss function, and the one-hot corresponding to the predicted one-hot vector and the real daily net amount is calculated. Vectors are compared, the loss is calculated, and the optimal weight parameters that minimize the loss are found using a combination of stochastic gradient descent and backpropagation methods. 7. the method for predicting bank outlet reserve funds based on long short-term memory recurrent neural network according to claim 6, is characterized in that, described model training stage, hidden layer number is 5, and each hidden layer comprises 50 neurons, and iterate 100,000 times to train the LSTM model.