CN115829150A - Accumulated water prediction system - Google Patents

Abstract

The invention provides a water accumulation prediction system, which is used for realizing the prediction of water accumulation by combining a water accumulation monitoring hardware platform based on the Internet of things, utilizing an advanced information means and an artificial intelligence method and adopting an LSTM network with better performance on the basis of fully considering the characteristics of different urban regions. The ponding prediction system requires the utilization of artificial intelligence algorithms, such as improved RNN, CNN and other algorithms, combined with meteorological data and real-time ponding monitoring data, and considering the geographic information of the area to which the monitoring point belongs and the difference of the ponding situation predicted by the future weather conditions, three types of monitoring point prediction models are respectively trained according to the rainfall level to predict the ponding of the ponding monitoring point 2 minutes, 5 minutes, 30 minutes and 2 hours later. The input data can enhance the robustness of the prediction model, the accuracy of the prediction result can be improved, and the requirements of flood disaster early warning function and personnel property safety protection near urban arterial roads, residential areas and river banks are fully met.

Description

Accumulated water prediction system

Technical Field

The invention belongs to the field of accumulated water detection, and particularly relates to an accumulated water prediction system.

Background

As some municipal drainage pipe networks in China are aged, the drainage standard is lower. Some local drainage facilities are not perfect and perfect, and the construction delay of a drainage system is an important reason for waterlogging. In addition, a large amount of hard pavement such as asphalt road and cement road in cities has poor water permeability and is not easy to infiltrate in rainfall and accumulated water in the road is easy to form. The requirements of citizens on living environment after the living standard is improved continuously, and the citizens are used to a fast-paced life style, so that the citizens cannot tolerate the sudden occurrence of ponding, traffic paralysis and the like. Once a city is attacked by a strong rainstorm, suddenly multiplied flood can go everywhere and certainly can rush in the city, so that the road becomes a river instantly, and a square becomes a lake immediately, so that residential areas, factories and the like built in low-lying areas such as a river channel, a lakeside and the like are threatened, and immeasurable serious loss is caused. The traditional waterlogging prediction system based on the RNN cannot comprehensively analyze data such as weather conditions such as radar, precipitation and remote sensing and the like and then use the data as input of the prediction system, so that the accuracy rate of prediction results is low.

The theoretical basis of deep learning is an artificial neural network, the essence of the neural network is reserved, the abstract concept is learned by utilizing a multilayer network, self-learning, self-feedback, understanding and summarization are added, and finally decision and judgment can be made, so that the deep learning algorithm has the capability of automatically learning characteristics and finding rules from a large amount of data, and the intelligent characteristic of the deep learning algorithm is extremely outstanding. In the ponding prediction subsystem, the ponding amount can be regarded as a time series, for the time series, the ponding depth of each time interval is closely related on the time relation, the ponding depth has complex historical dependence, the state at the moment has a certain degree of relation with the historical state at the last moment, and the change at the next moment can be caused. The information time span that can be used in practical use is very limited for standard RNN networks. When the task at the current moment is solved by using the information at a closer time point, the RNN can effectively learn the information at the historical moment. When the historical information which is different from the current time information for a long time needs to be used, the RNN capacity of learning information is weakened, and the problem of gradient disappearance of the RNN is solved. The problem is mainly caused by a given input on the hidden layer, which can cause the output of the network to fade or explode exponentially. The gradient return is the basis of RNN network training, when the time span is longer, the propagation of the gradient information generates an attenuation phenomenon, if the information is attenuated faster, the amount of gradient return information is smaller, and the return effect is worse. Theoretically, RNNs can handle information over a long time span, but in practice it is generally not possible to preserve information for all time segments due to fading. According to the method, long Short-Term Memory (LSTM) algorithm in a deep learning algorithm is adopted to predict the water accumulation data, and the LSTM algorithm can accurately finish Long-Term dependence relative to a standard recurrent neural network. In consideration of the characteristics and advantages, the subsystem can realize the maximum utilization of input data by adopting an LSTM network, efficiently and accurately predict the accumulated water depth values of all accumulated water monitoring points in cities after 2 minutes, 5 minutes and 30 minutes in the future through the input historical data, and provide an accumulated water change trend curve within 2 hours in the future. In addition, the detailed input data can enhance the robustness of the prediction model, and the accuracy of the prediction result can be improved.

Disclosure of Invention

In order to solve the problems, the invention provides a ponding prediction system based on the ponding prediction system by fully considering the regional characteristics of different cities, combining a ponding monitoring hardware platform based on the Internet of things, utilizing advanced information means and an artificial intelligence method and adopting an LSTM network with better performance, wherein the input data can enhance the robustness of a prediction model, and the accuracy of a prediction result can be improved, so that the requirements of flood disaster early warning function and personnel property safety protection near arterial roads, residential areas and river banks of urban areas are fully met.

The ponding prediction system requires the utilization of an artificial intelligence algorithm (algorithms such as improved RNN and CNN), combines meteorological data and real-time ponding monitoring data, considers the geographic information of the area where the monitoring points belong and the difference of the ponding situation predicted by the future weather conditions, respectively trains three types of monitoring point prediction models according to the rainfall levels, and predicts the ponding numerical values of the ponding monitoring points in the future 2 minutes, 5 minutes, 30 minutes and 2 hours later, wherein the rainfall levels are totally divided into six levels including heavy rainstorm, heavy rain, medium rain, light rain, sunny day and winter. The system needs to provide an input interface selection function, required input data can be selected to be used as input of the prediction system, and the training set and the test set can select a division ratio. Data needs to be regularly sorted, and user-defined screening rules are supported. The block diagram of the ponding prediction system is shown in fig. 1.

The specific technical scheme is as follows:

a ponding prediction system comprises a monitoring data acquisition and storage module, a preprocessing module, a model training and evaluation module and a data processing and display release system module;

the acquisition and storage module of the monitoring data is used for acquiring and storing the monitoring data, the monitoring data are respectively stored in 4 core data tables, and the 4 core data tables are respectively named as a real-time water level table waterLevel, a meteorological data table prediction, a model parameter table modelParemeter and a prediction data table waterLevelprediction; the real-time water level meter stores water level information measured in real time, and specifically comprises a device number used for measuring the water level information, GPS longitude and latitude information corresponding to the device, water level information acquired by the device and an error range; the weather data table stores the predicted precipitation information of the area where the equipment for measuring the water level information is located, which is obtained through an internal weather data API, and the predicted precipitation information comprises the area where the equipment is located, the precipitation collector number, the current precipitation per minute and the future precipitation; the prediction data table stores the prediction result of the accumulated water, and in order to reflect the expandability of the system, the stored content comprises the number of the precipitation collector and the accumulated water prediction results of 2 minutes, 5 minutes, 30 minutes and 2 hours in the future of the area where the precipitation collector is located; the model parameter table stores parameters of the ponding prediction model, including a training model position, namely model id (model _ id), and training learning parameters of each training model;

the preprocessing module is used for calling data from the acquisition and storage module of the monitoring data and carrying out preprocessing;

the model training and evaluating module comprises a model training part and a model evaluating part, wherein the model training part is used for training various accumulated water prediction models under the conditions of the same rainfall level and the same monitoring point category; the model evaluation is used for selecting a water accumulation prediction model with optimal performance from multiple water accumulation prediction models under the same rainfall level and monitoring point category conditions, and further obtaining the optimal water accumulation prediction model under the conditions corresponding to different rainfall levels and monitoring point categories;

calling an optimal water accumulation prediction model corresponding to rainfall level and monitoring point category conditions by a data processing and display issuing system to predict water accumulation, predicting water accumulation depth of a certain water accumulation hidden danger point at a future time point, storing a prediction result into a created database by using MyBatis after the water accumulation data is predicted, converting the data into a JSON format after the water accumulation data is stored into the database, outputting the JSON format and packaging the JSON format into an api interface for large-screen data calling, finally realizing graphic display, data query and waterlogging situation prediction simulation graphic display of water accumulation depth distribution, and giving early warning to areas and places with higher risks in advance;

selecting equipment according to data requirements, calling hyper-parameters of the prediction model according to the rainfall level of the monitoring point where the equipment is located and the type of the monitoring point, and inputting real-time water level data and meteorological data measured by the preprocessed equipment as a network model.

The training learning parameters include monitoring point type id, time interval, training data set proportion, testing data set proportion, input tensor dimension, hidden layer output node number, full connection layer output node number, hidden layer number, full connection layer number, iteration round number, training batch size, loss function option, optimizer option, learning rate, attenuation value of learning rate in each round, whether Nestterov momentum is used or not, and attenuation factor (i.e. optimizer (adadelta, sprop, adam, adamax, nadam)).

The data in the meteorological data table is obtained from an internal meteorological data interface, the data in the real-time water level table is obtained from the Internet of things ponding monitoring platform, and the data in the prediction data table and the model parameter table are obtained from the database.

The method comprises the steps that data are obtained from an Internet of things cloud platform and a weather meteorological interface and stored in a database, after parameters are configured, an equipment terminal SDK is used for being connected with the Internet of things cloud platform, before millimeter wave radar water level monitoring equipment is connected into the Internet of things platform through a protocol, information of the millimeter wave radar water level monitoring equipment is reported according to different authentication methods, and the millimeter wave radar water level monitoring equipment can be connected into the Internet of things platform after authentication; after the internet of things is successfully accessed, required equipment measurement data are downloaded according to requirements and stored into different data tables in a database according to data classification, and a weather meteorological interface API is requested to read required weather data and store the weather data into the database so as to perform corresponding data processing.

The preprocessing comprises the step of carrying out abnormal value processing on meteorological data; and carrying out mutation value processing on the real-time accumulated water data.

The accumulated water prediction model adopts an LSTM network model.

The model training and evaluating module divides the preprocessed data into a training set and a verification set according to a preset proportion, and the set proportion of the training set and the verification set can be changed according to requirements, wherein the training set is used for training model parameters, and the verification set is used for testing model prediction results; selecting millimeter wave radar water level monitoring equipment according to data requirements, calling hyper-parameters of the prediction model according to the rainfall level of a monitoring point where the equipment is located and the category of the monitoring point, constructing an LSTM network structure based on a keras framework, inputting training set data into the model, predicting the depth of ponding in two hours in the future according to real-time ponding data and meteorological data measured by the equipment in each training process, representing the error between a prediction result and the real-time ponding data measured by the equipment by using a loss function, and reversely propagating and adjusting parameters in the network through an optimizer according to the loss value. Repeating the steps in the above way, when the training is iterated to a certain number of rounds, the loss value is not reduced any more, the model is converged, and the training is finished; and repeatedly training for multiple times to obtain multiple models under the conditions of the same rainfall level and the same monitoring point category, and selecting and storing the model with the optimal performance according to different evaluation indexes, thereby obtaining multiple ponding prediction models corresponding to different rainfall levels and different monitoring point categories.

The model prediction is divided into manual prediction and automatic prediction, wherein the manual prediction is that a user manually inputs the serial number and the future time point of the millimeter wave radar monitoring equipment, calls the real-time ponding data currently monitored by the millimeter wave radar monitoring equipment and the meteorological data of the corresponding time of the real-time ponding data as a test data set, loads a stored ponding prediction model according to the rainfall level and the type of the monitoring point, inputs the test data set into the ponding prediction model, and outputs a ponding prediction depth result corresponding to the manually input future time point through the ponding prediction model;

automatic prediction refers to traversing all devices, calling their test data sets and corresponding prediction models for real-time prediction.

Further, a user defines a three-level threshold value for representing three different water accumulation degrees, primary, secondary and tertiary early warning is carried out, and early warning data are displayed through a large screen.

Compared with the prior art, the invention has the following advantages:

according to the scheme provided by the invention, the ponding prediction system is provided, different and standard RNN networks are adopted, and an LSTM network capable of accurately completing long-time dependence is adopted, compared with the traditional ponding prediction system, the system can comprehensively analyze real-time ponding data and meteorological data and then use the real-time ponding data and the meteorological data as the input of the prediction system, the detailed input data enables the robustness of the prediction model to be enhanced, and the accuracy of the prediction result is higher, so that the highly-accurate automatic acquisition of urban ponding monitoring data, the intelligent water situation prediction of urban waterlogging, the automatic distribution of urban waterlogging data transmission and early warning information and the intelligentization of emergency rescue command are further realized, and the flow chart of the ponding prediction system is shown in figure 2.

Drawings

FIG. 1 general functional block diagram of a water accumulation prediction system

FIG. 2 flow diagram of a water accumulation prediction system

FIG. 3 is a detailed flow chart of the module for acquiring and storing monitoring data

FIG. 4 Pre-processing Module Block diagram

FIG. 5 is a block diagram of model training and evaluation

FIG. 6 prediction result output flowchart

FIG. 7 model test block diagram

FIG. 8 is an expanded view of RNN network architecture

FIG. 9LSTM prediction model structure

FIG. 10 (a) prediction accuracy of lstm network model 1

FIG. 10 (b) prediction accuracy of lstm network model 2

FIG. 10 (c) prediction accuracy of lstm network model 3

FIG. 10 (d) prediction accuracy of lstm network model 4

FIG. 10 (e) lstm network model 5 prediction accuracy

FIG. 10 (f) cpu utilization

FIG. 11 shows predicted and actual stock values after 1-3 days

Detailed Description

The invention relates to 4 modules, each as follows:

the acquisition and storage module of the monitoring data: the method specifically comprises two processes of data transmission and module storage:

firstly, according to user functions and performance requirements, 4 core data tables are designed and named as real-time water level table waterLevel, weather data table weather prediction, model parameter table modelpameter and prediction data table waterLevelprediction respectively. The real-time water level meter mainly stores water level information measured in real time, and the water level information comprises measuring equipment (collector) numbers, GPS longitude information, GPS latitude information, water level information and error ranges. The meteorological data table mainly stores predicted precipitation information obtained through an internal meteorological data API, and the predicted precipitation information comprises the area, the collector number, the current precipitation per minute and the future precipitation. The prediction data table mainly stores the prediction result of the accumulated water, and in order to show the expandability of the system, the stored contents comprise collector numbers, accumulated water in the future of 2 minutes, 5 minutes, 30 minutes and 2 hours. The model parameter table mainly stores parameters of deep learning, including training model positions and training learning parameters of each point. A flow chart of the monitoring data acquisition and storage module is shown in fig. 3.

And then data are acquired from the Alicloud Internet of things cloud platform and the weather and weather interface and stored in a database. After the parameters are configured, the device side SDK is connected with the Aliskian, before the device is connected to the Internet of things platform through a protocol, the information of the millimeter wave radar monitoring device is reported according to different authentication methods, and the device can be connected to the Internet of things platform after the authentication. After the internet of things is successfully accessed, the required equipment measurement data can be downloaded according to the requirements and stored in different data tables in the database according to the data classification, and the weather meteorological interface API is requested to read the required weather station data and store the weather station data in the database, so that the next step of data analysis and prediction can be conveniently carried out.

A data preprocessing module: and after data are acquired from the database, corresponding data processing is carried out for different characteristic inputs. Abnormal value processing is carried out on weather data; and carrying out mutation value processing on the real-time accumulated water monitoring data. The specific implementation block diagram is shown in fig. 4.

The model training and evaluating module: selecting millimeter wave radar monitoring equipment according to data requirements, calling hyper-parameters of the prediction model according to the rainfall level of a monitoring point where the equipment is located and the category of the monitoring point, constructing an LSTM network structure based on a keras framework, inputting training set data into the model, predicting the depth of the accumulated water in two hours in the future according to meteorological data and real-time accumulated water monitoring data in each training process, representing the error between a prediction result and the actually monitored accumulated water depth by using a loss function, and reversely transmitting and adjusting parameters in the network through an optimizer according to the loss value. And repeating the steps, when the training is iterated to a certain number of rounds, the loss value is not reduced any more, and the model is converged. And automatically and circularly training various models according to different rainfall levels and monitoring point types. The model training and evaluation module diagram is shown in fig. 5, and the specific implementation steps are as follows:

(a) Selecting millimeter wave radar equipment according to data requirements, calling hyper-parameters of the prediction model according to the rainfall level of the monitoring point where the equipment is located and the category of the monitoring point

(b) Defining a model, namely creating an lstm model, adding a configuration layer, and constructing a forward propagation network structure;

(c) Compiling a model- -building a model structure according to the model hyper-parameters to complete model compiling; in the process, the average absolute error (MAE) is used as a loss function of the evaluation weight, the MAE is the average value of the absolute errors and can better reflect the actual situation of the predicted value error, and the formula is expressed as follows:

wherein x _i For the pre-processed current real-time water accumulation monitoring data, h (x) _i ) For predicting the water volume in future time periods from real-time data, y _i The water accumulation for the actual future time period.

(d) Training model- -input training data into the network for forward propagation, and train the model using an efficient optimization algorithm Adam as an optimizer for searching different weights of the network

(e) Saving model- -after training is completed, the trained model is saved to the specified path for test model invocation.

(f) And repeating the process, and automatically and circularly training various models.

In the process of evaluating the model, the prediction result of the training model is compared with the waterlogging real-time monitoring data obtained by the database, and the model performance is judged by respectively adopting Mean Square Error (MSE), root Mean Square Error (RMSE), mean Absolute Error (MAE) and R square (R square).

MSE is an expectation value of the square of the difference between the estimated value of the parameter and the true value of the parameter, the change degree of data can be evaluated, and the smaller the value of MSE is, the better accuracy of the prediction model in describing experimental data is shown. The formula is expressed as:

wherein f is _t Representing the observed value, y _t Indicating the predicted value.

RMSE is the arithmetic square root of MSE, which measures the deviation between observed and true values. Is often used as a measure of the prediction outcome of the machine learning model. Also, the smaller the value thereof, the higher the prediction accuracy is proved. The formula is expressed as:

MAE is the average absolute error, and can better reflect the actual situation of the error of the predicted value, and the formula is expressed as formula 4. The square R is the ratio of the regression sum of squares to the sum of squares of total deviations in linear regression, with the value equal to the square of the correlation coefficient. Which is a measure of the goodness of fit of the estimated regression equation. The closer the statistic is to 1, the higher the goodness of fit of the model. The formula is expressed as:

wherein the content of the first and second substances,

indicates the predicted value, y _i The actual value is represented by the value of,

the average value is shown.

And (3) customizing a threshold (three levels) by a user, comparing the predicted evaluation index with the threshold, if the prediction result exceeds the threshold, indicating that the accumulated water is serious, carrying out early warning of one level, two levels and three levels, and pushing the data to a display subsystem through a JSON format for large-screen display.

The data processing and display release system comprises: after the accumulated water data is predicted, storing a prediction result into a created database by using MyBatis, wherein the MyBatis is an excellent persistent layer frame, and a data logic library is generated from an XML configuration file by business logic factory production, wherein the business logic factory production, the business factory and the life cycle of a business are consistent, and JDBC and JTA transaction processing is supported. It supports customized SQL, stored procedures, and advanced mapping, which has the advantage that it avoids most JDBC codes and manual setting of parameters and acquisition of result sets. After the data are stored in a database, the data are converted into a JSON format, finally the JSON format is output and packaged into an api interface for large-screen calling of the data, finally, graph display of water accumulation depth distribution, data query and waterlogging situation prediction simulation graph display are achieved, and early warning is sent to areas and places with high risks. The prediction result output flowchart is shown in fig. 6.

The model prediction is divided into manual prediction and automatic prediction, wherein the manual prediction is that a user manually inputs the serial number and the future time point of the millimeter wave radar monitoring equipment, calls the real-time water level data monitored by the millimeter wave radar monitoring equipment at present, weather data at the corresponding time and water seepage data calculated by an urban waterlogging model as a test data set, loads a stored LSTM model according to the rainfall level and the type of the monitoring point, inputs the test data set into the model, and outputs the manually input depth prediction result of the water logging at the future time point through a neural network model. And automatic prediction refers to traversing all devices, and calling a test data set and a corresponding prediction model of the devices for real-time prediction. The implementation block diagram is shown in fig. 7:

the method comprises the following specific steps:

step 1: the system acquires real-time ponding data measured by a plurality of ponding points from a ponding monitoring hardware platform based on the Internet of things, acquires meteorological data such as radar, precipitation and remote sensing from an internal meteorological data interface, and uses the meteorological data as complete input of the ponding prediction system.

Step 2: and preprocessing the data, optimizing a preprocessing module according to user functions and technical indexes, completing comprehensive analysis of the data and meeting the input requirements of system data.

And step 3: the preprocessed data is used as network model input, according to different monitoring point types, the LSTM network model improved on the RNN network is used for predicting the depth condition of accumulated water at a certain time point or a certain monitoring point in a certain period of time in a city, the real-time accumulated water monitoring data is called to train and predict network parameters, algorithm self-correction is completed, and the model prediction accuracy is improved.

And 4, step 4: and after the training is finished, evaluating the test prediction result and the training model through different evaluation indexes. The optimal training network model is selected to carry out water accumulation depth prediction on a certain water accumulation hidden danger point at a future time point, graph display and data query of urban water accumulation depth distribution and simulation graph display of water accumulation prediction are realized by combining the data processing and display issuing subsystem, and early warning is sent out in advance for areas and places with higher risks.

Introduction of RNN and LSTM networks:

rnn is one of the very potential tools for time series modeling. The purpose of constructing the RNN by the system is to process time series data, for the time series, the water accumulation depth of each time interval is closely related in a time relation, the water accumulation depth has complex historical dependence, the state at the moment has a certain degree of relation with the historical state at the last moment, and the change at the next moment can be caused, so that the adoption of the recurrent neural network is very suitable.

There are four basic forms of RNN recurrent neural networks. The first is a one-to-many network, the input layer of which has only one input value and the output layer has a plurality of output values, i.e. a single input sequence is converted into a plurality of outputs; the second is a many-to-one network, as opposed to a one-to-many network, where the input layer has multiple input values and the output layer has only one output value, i.e., multiple input sequences are converted into a single output; the third is that there are many times of difference, there are many sequences in its input layer and output layer, there is a certain time difference in the input layer and output layer sequence, this model is used for predicting the future value of the time sequence, such as weather prediction, tourist amount prediction, stock prediction and traffic flow prediction, etc.; the fourth is no time difference many-to-many, and the input layer and the output layer are also composed of a plurality of sequences, but the output layer and the output layer have no time difference and are mostly used for voice recognition, digital recognition and the like. Four basic forms of RNN networks are shown in fig. 7.

A key advantage of RNN networks is that they can handle inputs of indefinite length, ultimately obtaining an output of a specified length. The water accumulation prediction to be realized is to predict the water accumulation depth of the next time period.

For the present system, the water accumulation can be viewed as a time series, where each circle can be viewed as a unit, which can be folded as shown in FIG. 8 since each unit will handle the same event. Wherein x (t) represents the preprocessed real-time water level data at the moment t, W, U and V are shared weight matrixes, s (t) represents the hidden state corresponding to the moment t, and the hidden state can be obtained by the hidden state of the previous step and the input of the current moment:

s(t)＝f(Ux _t +Ws _t-1 ) (5)

where f is a non-linear function. o (t) is the output at time V, and is expressed as:

o(t)＝sigmoid(Vs(t)) (6)

however, RNNs also have some drawbacks. The information time span that can be used in practical use is very limited for standard RNN networks. When the task at the current moment is solved by using the information at a closer time point, the RNN can effectively learn the information at the historical moment. When the historical information which is different from the current time information for a long time needs to be used, the RNN capacity of learning the information is weakened, and the problem of gradient disappearance of the RNN is solved. The problem is mainly caused by a given input on the hidden layer, which can cause the output of the network to fade or explode exponentially. The gradient return is the basis of RNN network training, when the time span is longer, the propagation of the gradient information generates an attenuation phenomenon, if the information is attenuated faster, the amount of gradient return information is smaller, and the return effect is worse. In theory RNNs can handle information over a long time span, but in practice it is generally not possible to retain information for all time periods due to fading. Therefore, the improved RNN adopts the LSTM network with better performance.

2. Compared with a standard RNN (radio network node), the Long Short-Term Memory (LSTM) algorithm can accurately finish Long-Term dependence. At the same time, the LSTM network has no major changes in structure relative to the standard RNN network, but some improvements are made to the hidden layer. In long and short term memory networks, memory cells (cells), each having an input gate, a forgetting gate and an output gate, are used instead of conventional neurons. Each gate of the memory cells is a selective way to pass information, using a sigmoid activation function to control whether they are triggered, selectively making state changes, and controlling the flow of information through the memory cells. The LSTM internal processor and triple gate are shown in fig. 9.

The forgetting gate conditionally determines which information is discarded from the unit, forgetting part of information is realized through the sigmoid layer, each element output by the sigmoid layer is a real number between 0 and 1, and the real number represents that corresponding information x is allowed to be corresponding to _t The ratio of the passage. For example, 0 means that no information is passed, and 1 means that all information is passed. The input of the forgetting gate is the output h of the last cell _t-1 And the current input x _t Output f of forgetting gate _t Comprises the following steps:

f _t ＝σ(W _f h _t-1 +W _f x _t +b _f ) t＝1,2,......,T (7)

where σ represents the sigmoid function. It is expressed as:

the entry gate conditionally decides which information to store in the cell. How much new information is added to the cell state is calculated. It comprises two parts: and a sigmoid layer and a tanh layer of the input gate. Output of the Sigmoid layer:

i _t ＝σ(W _i h _t-1 +W _i x _t +b _i ) t＝1,2,......,T (9)

output of tanh layer:

the tanh function is expressed as:

the new cell states are:

the output gate conditionally decides which information needs to be output and outputs the information. The Sigmoid layer determines which information of the cell state is to be output:

o _t ＝σ(W _o h _t-1 +W _o x _t +b ₀ ) t＝1,2,......,T (13)

the tanh layer processes the cell state and multiplies it with the output of the sigmoid gate, and finally outputs that portion which determines the output:

h _t ＝o _t tanh(C _t ) t＝1,2,……,T (14)

in order to verify the effectiveness and the practicability of the LSTM network on the prediction and classification tasks of time sequence data, the LSTM network is used for analyzing the short-term prediction which is the most difficult to attack in the Chinese stock market, and the data is used for obtaining a relatively accurate conclusion in the short-term prediction of the Chinese stock market.

After stock prediction is proved to have predictability in a theoretical level by visiting stockholders and communicating with financial experts, stock data is searched and converged with the Shanghai index K data to be extracted and integrated so that data can be based in the next operation. In data integration, 619737244 bytes of data are obtained in total, including data of 2827 stocks from 1/4/2015 to 1/14/2019. After the data is obtained, the sorted data is stored in a database. Before storage, various databases and storage methods are deeply compared, and MySQL is selected as a relational database to store all stock data.

The feasibility of stock prediction analysis is verified by using a feedforward neural network, model prediction accuracy is checked under different network models, the CPU utilization rate reaches over 95% by adopting CPU operation, and the experimental result is shown in figure 10.

After the data preparation work is completed, an LSTM neural network is built and the data is analyzed as an demonstration, so that the stock market in China is proved to be predictable, and a more accurate prediction model is obtained: the data after 1-3 days can be predicted, the predicted data is stable and effective, and the accuracy is stable to 86% -93%. The prediction result is shown in fig. 11, where the blue line is the true value of the stock and the red line is the predicted value of the stock, which proves that the LSTM network has a relatively ideal prediction accuracy for the prediction task of the time series data.

Claims (9)

1. A water logging prediction system, comprising: the system comprises a monitoring data acquisition and storage module, a preprocessing module, a model training and evaluation module and a data processing and display release system;

the acquisition and storage module of the monitoring data is used for acquiring and storing the monitoring data, the monitoring data are respectively stored in 4 core data tables, and the 4 core data tables are respectively named as a real-time water level table waterLevel, a meteorological data table prediction, a model parameter table modelParemeter and a prediction data table waterLevelprediction; the real-time water level meter stores water level information measured in real time, and specifically comprises a device number used for measuring the water level information, GPS longitude and latitude information corresponding to the device, water level information collected by the device (the water level information collected by the device is real-time ponding data) and an error range; the weather data table stores the predicted precipitation information of the area where the equipment for measuring the water level information is located, which is obtained through an internal weather data API, and the predicted precipitation information comprises the area where the equipment is located, the precipitation collector number, the current precipitation per minute and the future precipitation; the prediction data table stores the prediction result of the ponding, and the stored content comprises the number of the precipitation collector and the future ponding prediction result of the area where the precipitation collector is located; the model parameter table stores parameters of the ponding prediction model, including a training model position, namely model id (model _ id), and training learning parameters of each training model;

the preprocessing module is used for calling data from the acquisition and storage module of the monitoring data and carrying out preprocessing;

the model training and evaluating module comprises a model training part and a model evaluating part, wherein the model training part is used for training various accumulated water prediction models under the conditions of the same rainfall level and the same monitoring point category; the model evaluation is used for selecting a water accumulation prediction model with optimal performance from multiple water accumulation prediction models under the same rainfall level and monitoring point category conditions, and further obtaining the optimal water accumulation prediction model under the conditions corresponding to different rainfall levels and monitoring point categories;

calling an optimal water accumulation prediction model corresponding to rainfall level and monitoring point category conditions by a data processing and display issuing system to predict water accumulation, predicting water accumulation depth of a certain water accumulation hidden danger point at a future time point, storing a prediction result into a created database by using MyBatis after the water accumulation data is predicted, converting the data into a JSON format after the water accumulation data is stored into the database, outputting the JSON format and packaging the JSON format into an api interface for large-screen data calling, finally realizing graphic display, data query and waterlogging situation prediction simulation graphic display of water accumulation depth distribution, and giving early warning to areas and places with higher risks in advance;

selecting equipment according to data requirements, calling hyper-parameters of the prediction model according to the rainfall level of a monitoring point where the equipment is located and the category of the monitoring point, and inputting the real-time water level data and meteorological data measured by the preprocessed equipment as a network model.

2. The water accumulation prediction system as defined in claim 1, wherein:

the training learning parameters include monitoring point type id, time interval, training data set proportion, testing data set proportion, input tensor dimension, hidden layer output node number, full connection layer output node number, hidden layer number, full connection layer number, iteration round number, training batch size, loss function option, optimizer option, learning rate, attenuation value of learning rate in each round, whether Nestterov momentum is used or not, and attenuation factor (i.e. optimizer (adadelta, sprop, adam, adamax, nadam)).

3. The water accumulation prediction system as defined in claim 1, wherein:

the data in the meteorological data table is obtained from an internal meteorological data interface, the data of the real-time water level table is obtained from the Internet of things ponding monitoring platform, and the data in the prediction data table and the model parameter table are obtained from the database.

4. The water accumulation prediction system as defined in claim 1, wherein:

acquiring data from the Internet of things cloud platform and a weather meteorological interface, storing the data into a database, after parameters are configured, connecting the Internet of things cloud platform with an equipment terminal SDK, reporting millimeter wave radar water level monitoring equipment information according to different authentication methods before millimeter wave radar water level monitoring equipment is accessed into the Internet of things platform through a protocol, and accessing the Internet of things platform after the authentication is passed; after the internet of things is successfully accessed, required equipment measurement data are downloaded according to requirements and stored in different data tables in a database according to data classification, and a weather meteorological interface API is requested to read required weather data and store the weather data in the database so as to perform corresponding data processing.

5. The water accumulation prediction system as defined in claim 1, wherein:

the preprocessing comprises the step of carrying out abnormal value processing on the meteorological data; and carrying out mutation value processing on the real-time accumulated water data.

6. The water accumulation prediction system as defined in claim 1, wherein: the accumulated water prediction model adopts an LSTM network model.

7. The water accumulation prediction system as defined in claim 1, wherein:

the model training and evaluating module divides the preprocessed data into a training set and a verification set according to a preset proportion, and the set proportion of the training set and the verification set can be changed according to requirements, wherein the training set is used for training model parameters, and the verification set is used for testing model prediction results; selecting millimeter wave radar water level monitoring equipment according to data requirements, calling hyper-parameters of a prediction model according to the rainfall level of a monitoring point where the equipment is located and the category of the monitoring point, constructing an LSTM network structure based on a keras framework, inputting training set data into the model, predicting the depth of future ponding according to real-time ponding data and meteorological data measured by the equipment in each training process, expressing errors between a prediction result and the real-time ponding data measured by the equipment by using a loss function, and reversely propagating and adjusting parameters in the network through an optimizer according to the loss value; and repeatedly training for many times to obtain a plurality of models under the conditions of the same rainfall level and the same monitoring point category, and selecting and storing the model with the optimal performance through different evaluation indexes, thereby obtaining a plurality of accumulated water prediction models corresponding to different rainfall levels and different monitoring point categories.

8. The water accumulation prediction system as defined in claim 1, wherein:

the model prediction is divided into manual prediction and automatic prediction, wherein the manual prediction is that a user manually inputs the serial number and the future time point of the millimeter wave radar monitoring equipment, calls the real-time ponding data currently monitored by the millimeter wave radar monitoring equipment and the meteorological data of the corresponding time of the real-time ponding data as a test data set, loads a stored ponding prediction model according to the rainfall level and the type of the monitoring point, inputs the test data set into the ponding prediction model, and outputs a ponding prediction depth result corresponding to the manually input future time point through the ponding prediction model;

automatic prediction refers to traversing all devices, calling their test data sets and corresponding prediction models for real-time prediction.

9. The water accumulation prediction system as defined in claim 1, wherein:

further, a user defines a three-level threshold value for representing three different water accumulation degrees, primary, secondary and tertiary early warning is carried out, and early warning data are displayed through a large screen.

Images