CN117131991A - Urban rainfall prediction method and platform based on hybrid neural network - Google Patents
Urban rainfall prediction method and platform based on hybrid neural network Download PDFInfo
- Publication number
- CN117131991A CN117131991A CN202311145680.1A CN202311145680A CN117131991A CN 117131991 A CN117131991 A CN 117131991A CN 202311145680 A CN202311145680 A CN 202311145680A CN 117131991 A CN117131991 A CN 117131991A
- Authority
- CN
- China
- Prior art keywords
- neural network
- feature
- data
- weather
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 64
- 239000011159 matrix material Substances 0.000 claims abstract description 69
- 238000003062 neural network model Methods 0.000 claims abstract description 22
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 21
- 125000004122 cyclic group Chemical group 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 14
- 238000010219 correlation analysis Methods 0.000 claims abstract description 12
- 230000002093 peripheral effect Effects 0.000 claims abstract description 11
- 238000011176 pooling Methods 0.000 claims description 34
- 238000010606 normalization Methods 0.000 claims description 30
- 230000000007 visual effect Effects 0.000 claims description 21
- 230000007246 mechanism Effects 0.000 claims description 20
- 238000010586 diagram Methods 0.000 claims description 13
- 210000002569 neuron Anatomy 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 11
- 230000004913 activation Effects 0.000 claims description 10
- 238000004140 cleaning Methods 0.000 claims description 6
- 230000002452 interceptive effect Effects 0.000 claims description 5
- 230000009467 reduction Effects 0.000 claims description 5
- 230000002776 aggregation Effects 0.000 claims description 3
- 238000004220 aggregation Methods 0.000 claims description 3
- 238000013523 data management Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 6
- 238000010801 machine learning Methods 0.000 abstract description 5
- 230000008569 process Effects 0.000 description 11
- 238000012549 training Methods 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 7
- 238000012800 visualization Methods 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000010354 integration Effects 0.000 description 4
- 230000035945 sensitivity Effects 0.000 description 4
- 238000007405 data analysis Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000002779 inactivation Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000010220 Pearson correlation analysis Methods 0.000 description 1
- 241000728173 Sarima Species 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- YHXISWVBGDMDLQ-UHFFFAOYSA-N moclobemide Chemical compound C1=CC(Cl)=CC=C1C(=O)NCCN1CCOCC1 YHXISWVBGDMDLQ-UHFFFAOYSA-N 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000007794 visualization technique Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Tourism & Hospitality (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- General Business, Economics & Management (AREA)
- Software Systems (AREA)
- Marketing (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Development Economics (AREA)
- Data Mining & Analysis (AREA)
- Primary Health Care (AREA)
- Quality & Reliability (AREA)
- Game Theory and Decision Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Educational Administration (AREA)
- Operations Research (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
A city rainfall prediction method and platform based on a hybrid neural network belong to the field of machine learning, and in order to solve the problem of city rainfall prediction capable of expressing space, the main points are to acquire weather observation historical data of a target position and weather stations at the peripheral positions of the target position; carrying out correlation analysis on rainfall and historical data; normalizing the meteorological characteristic data; the method comprises the steps of processing weather feature data randomly distributed in space into an equidistant matrix structure, outputting feature matrixes of corresponding features, inputting the feature matrixes into a convolutional neural network, inputting the serialized feature matrixes arranged according to time into a cyclic neural network model, inputting the output of the cyclic neural network model into a full-connection layer, and outputting a prediction result by the full-connection layer, wherein the effect is that accurate rainfall prediction can be provided for a target position.
Description
Technical Field
The invention belongs to the field of machine learning, and relates to a city rainfall prediction method and a city rainfall prediction platform based on a hybrid neural network.
Background
The extreme weather event may aggravate the relevant effects of urban areas, and accurate and timely rainfall prediction becomes more and more important. However, conventional weather data analysis methods often fail to effectively process and handle massive amounts of weather data. Furthermore, the complexity and dynamics of weather patterns require the use of advanced machine learning techniques to make accurate predictions.
Extreme rainfall, especially in large and medium cities, may cause urban inland inundation problems. The threat to the production and living and property safety of people is brought. Therefore, accurate prediction of short-time heavy rainfall is particularly important. The existing rainfall prediction early warning is often carried out by combining relevant weather models such as satellite cloud pictures, weather dynamics and the like, but the weather problem is often a very complex model, the weather problem can have better accuracy on a large trend based on the weather dynamics, but the accuracy can be influenced by weather, geographical factors and multi-factor conditions in a small range and in a short period; the accuracy is often not very high, and the meteorological features of different positions and different cities are difficult to predict by using a complex meteorological model.
With the improvement of computer performance and the rising of big data technology in recent years, the application of advanced neural network models to classify data and conduct regression prediction has become popular. In the rainfall prediction field, there are many neural network algorithms based on time series models, for example, traditional time series models based on ARIMA, SARIMA and the like, and network structures based on RNN cyclic neural network structures and developed LSTM, GRU and the like also show good prediction accuracy.
The patent application CN111815037A proposes a short-term extreme rainfall prediction method based on an attention mechanism, which uses a random forest algorithm to calculate the correlation between meteorological site observation data and rainfall, uses a long-term and short-term memory neural network to train the data after important meteorological observation factors are selected, and still completes a prediction function based on the local historical data of a target position without paying attention to the change of continuous peripheral weather in space. The weather forecast is slightly weaker for spatially continuous extreme changes.
Patent application CN114325880a proposes a rainfall prediction method based on a radar echo map, which uses a convolutional neural network to process the radar echo map to construct a neural network structure to complete the model building and training process. Compared with the observation data of a weather station, the model requires the use of real-time radar echo for prediction, the prediction cost is high, the weather phenomenon is often very complex, and the training of a complex model cannot be completed for a target building with the missing radar echo diagram in the historical data.
Therefore, the existing rainfall prediction method based on the meteorological dynamics model has high requirements on data, a real-time satellite cloud image is often required to be acquired by means of a meteorological satellite system, the calculation process is limited by the model and the calculation complexity, and the prediction cost in specific cities and specific positions is high. The rainfall is regarded as a sequence with periodicity and predictability rules in the time dimension based on a traditional time sequence model and an RNN-based neural network structure, the continuity of the weather problem in space is ignored, and the calculation thinking is that the relation between the historical characteristics and the rainfall at the future moment is found in data. The accuracy of the method is higher in cities with vivid climates and single meteorological changes, but the method is very severe in meteorological changes, especially short-time heavy rainfall, and the rapid climate changes often lack good prediction capability.
In particular, as in patent application CN114325880a, weather information in the surrounding space cannot be obtained using single target position history data, however, the occurrence of rainfall has a very high correlation with the surrounding space, and weather stations in the surrounding space (city) have a predicted correlation with respect to the target position, and can influence the rainfall in the target position, so that weather station data in the surrounding space (city) participate in prediction, and the accuracy of prediction can be effectively improved. The method can be reflected by rainfall characteristics, rainfall has correlation with time and space, and single time model and space model are independently applicable to comprehensively extract implicit information in data, so that prediction related information may be lost, and the prediction performance is poor.
The conventional technology has not always been a method for expressing spatial city rainfall prediction because it is usually performed without taking into consideration the historical data of weather stations in the surrounding space (surrounding city) of the target location (city), usually using continuous data, and taking into consideration the historical data of weather stations in the surrounding space (surrounding city), actually discrete data in the processing space, which cannot be directly input into the convolutional neural network, because it is necessary to consider the historical data of weather stations in the surrounding space (surrounding city) together to express the correlation and comprehensiveness of rainfall and space.
Disclosure of Invention
In order to solve the problem of urban rainfall prediction capable of expressing space, the urban rainfall prediction method based on the hybrid neural network according to some embodiments of the present application includes:
s10, acquiring weather observation historical data of weather stations at the target position and the peripheral positions thereof;
s20, carrying out correlation analysis on the rainfall and the historical data, determining the rainfall, the air temperature, the humidity, the wind speed, the wind direction, the cloud layer visual thickness and the cloud layer percentage in unit time as weather features according to the correlation analysis, and representing the weather features in a two-dimensional plane determined by coordinate axes to obtain weather feature data of corresponding coordinates of the weather features;
s30, normalizing the meteorological feature data, wherein the air temperature, the humidity, the wind speed, the wind direction, the cloud layer visual thickness and the cloud layer percentage in unit time are normalized by a Max-Min normalization method, the rainfall is normalized by a logarithmic normalization method, and the normalized cloud layer visual thickness and the meteorological feature data of the cloud layer percentage in unit time output feature matrixes of corresponding features;
s40, processing weather characteristic data of normalized rainfall, air temperature, humidity, wind speed and wind direction into an equidistant matrix structure by a Kriging interpolation method, and outputting a characteristic matrix of corresponding characteristics;
S50, inputting the characteristic matrix of the meteorological characteristic data into a convolutional neural network to obtain a time-arranged serialized characteristic matrix, inputting the time-arranged serialized characteristic matrix into a cyclic neural network model, inputting the output of the cyclic neural network model into a full-connection layer, and outputting a prediction result by the full-connection layer.
According to some embodiments of the present application, the convolutional neural network in step S50 includes
A first convolution layer, a first CBAM attention mechanism, a first pooling layer, a second convolution layer, a second CBAM attention mechanism, a second pooling layer, and a flame layer, the first CBAM attention mechanism comprising a first channel attention module and a first spatial attention module, the second CBAM attention mechanism comprising a second channel attention module and a second spatial attention module;
the characteristic matrix of the weather characteristic data is input into a first convolution layer, the weather characteristic data is expressed as a four-dimensional tensor, and the output of the first convolution layer is a characteristic diagram of the weather characteristic data;
inputting the feature map of each meteorological feature data output by a first convolution layer into a first channel attention module of a first CBAM attention mechanism, wherein the feature map is H multiplied by W multiplied by C, H is long, W is wide, C is a channel, in the first channel attention module, two feature matrixes of 1 multiplied by C are obtained by the respective channels through global maximum pooling and global average pooling based on width and height respectively, the two feature matrixes are respectively input into two layers of neural network MLP, the number of neurons of the first layer of neural network MLP is C/radio, C is the channel, radio is the reduction rate, the activation function is Relu, the number of neurons of the second layer of neural network MLP is C, the features output by the two layers of neural network MLP are subjected to element-based addition operation, then the M_c weight matrix is generated through Sigmoid activation operation, and the M_c weight matrix and the feature matrixes are subjected to element-wise multiplication operation to obtain weighted features;
The weighted feature is used as an input feature map to be input into a first space attention module of a first CBAM attention mechanism, global average pooling and global maximum pooling are respectively carried out on Channel dimensions of a feature matrix to obtain two H multiplied by W multiplied by 1 feature maps, 2 feature maps are spliced by channels, then the dimension is reduced to 1 Channel through convolution operation to be the H multiplied by W multiplied by 1 feature map, a weight matrix M_s is generated through sigmoid operation, and the weight matrix M_s and the weighted feature are subjected to matrix multiplication to obtain a finally generated feature matrix;
inputting the finally generated feature matrix into a first pooling layer, and inputting the output of the first pooling layer into a second convolution layer;
the output of the second convolution layer is a characteristic diagram of weather characteristic data, the characteristic diagram of the weather characteristic data output by the second convolution layer is input into a second channel attention module of a second CBAM attention mechanism, the characteristic diagram is H multiplied by W multiplied by C, H is long, W is wide, C is a channel, in the second channel attention module, the characteristics output by the two layers of the neural network MLP are subjected to global maximum pooling and global average pooling based on width and height respectively by the channels to obtain two characteristic matrixes of 1 multiplied by C, the two characteristic matrixes are respectively input into two layers of the neural network MLP, the number of neurons of the first layer of the neural network MLP is C/radio, C is the channel, the radio is the reduction rate, the activation function is Relu, the number of neurons of the second layer of the neural network MLP is C, the characteristics output by the two layers of the neural network MLP are subjected to addition and operation based on element, and then the element activation operation is performed to generate M_c weight matrixes, and the element weighting operation is performed to obtain the characteristic weight matrixes;
The weighted feature is used as an input feature map to be input into a second space attention module of a second CBAM attention mechanism, global average pooling and global maximum pooling are respectively carried out on Channel dimensions of a feature matrix to obtain two H multiplied by W multiplied by 1 feature maps, 2 feature maps are spliced by channels, then the dimension is reduced to 1 Channel through convolution operation to be the H multiplied by W multiplied by 1 feature map, a weight matrix M_s is generated through sigmoid operation, and the weight matrix M_s and the weighted feature are subjected to matrix multiplication to obtain a finally generated feature matrix;
inputting the finally generated feature matrix into a second pooling layer, and inputting the output of the second pooling layer into a flat layer;
the output of the flat layer sequences the feature matrix in time.
According to some embodiments of the application, the urban rainfall prediction method based on the hybrid neural network is characterized in that the spatial downsampling range of the convolutional neural network is 16×16 latitude and longitude, the input of the convolutional neural network is a four-dimensional tensor of meteorological information of 10 hours in the sampling field range, shape of the four-dimensional tensor is n×long×lati×c, wherein n is the time length of historical data input by the convolutional neural network, long is the longitude range, the lati is the latitude range, and C is the number of features.
According to some embodiments of the present application, the urban rainfall prediction method based on the hybrid neural network calculates the correlation through pearson correlation coefficient in step S20.
According to some embodiments of the application, the urban rainfall prediction method based on the hybrid neural network normalizes meteorological feature data, wherein the air temperature, the humidity, the wind speed, the wind direction, the cloud layer visual thickness and the cloud layer percentage in unit time are normalized by a Max-Min normalization method, and the method is expressed as follows:
in X, X max 、X min X is the original meteorological feature value, the maximum value in the original meteorological feature and the minimum value in the original meteorological feature respectively scaled For the weather feature values after normalization, the weather features with different scales are between 0 and 1 after normalization treatment.
The rainfall is normalized by a logarithmic normalization method, and the normalized formula is as follows:
X scaled ＝log 10 (X i +1)
wherein X is i Is the original meteorological characteristic value.
According to some embodiments of the present application, the method for predicting urban rainfall based on the hybrid neural network, in step S40, the kriging interpolation method includes
Obtaining a difference result by estimating the attribute value of any point (X, Y) in space under the condition that the feature value zi=z (Xi, yi) after normalization of a certain meteorological attribute of a plurality of discrete points (Xi, yi) in space is known;
Is expressed by the following formula:
in the middle ofIs the point (x) o ,y 0 ) Estimated value of z o ＝z(x o ,y 0 )，λ i Is a weight coefficient, n represents the number of target positions and surrounding weather stations, n is a weighted sum of data of all known points in space to estimate the value of the unknown point, but the weight coefficient is not the inverse of the distance, is a value capable of satisfying the point (x o ,y 0 ) Estimated value +.>And the true value z o The minimum difference of the optimal coefficients is calculated by the following formula
At the same time meet the condition of unbiased estimationVar denotes variance and E denotes expectations.
According to some embodiments of the present application, the urban rainfall prediction method based on the hybrid neural network further includes cleaning the weather feature data of the obtained weather feature corresponding coordinates in step S20.
Urban rainfall prediction platform based on Hadoop and neural network according to some embodiments of the application comprises
The Hadoop distributed file system is used for storing and managing meteorological data;
the correlation analysis module comprises a step of performing correlation analysis on rainfall and historical data;
the normalization module is used for normalizing the weather characteristic data determined by the correlation analysis;
the interpolation module is used for processing weather characteristic data of normalized rainfall, air temperature, humidity, wind speed and wind direction into an equidistant matrix structure through a Kriging interpolation method, and outputting a characteristic matrix of corresponding characteristics;
The model weight module comprises a convolutional neural network, a cyclic neural network and a full-connection layer, wherein the characteristic matrix of meteorological characteristic data is input into the convolutional neural network, a time-arranged serialized characteristic matrix is obtained by the convolutional neural network, the time-arranged serialized characteristic matrix is input into the cyclic neural network model, the output of the cyclic neural network model is input into the full-connection layer, and the full-connection layer is used for outputting a prediction result;
springboot and Vue tools for visualizing the prediction results in an intuitive and interactive way.
According to some embodiments of the application, the urban rainfall prediction platform based on Hadoop and a neural network further comprises a Hive module, wherein the Hive module is used for creating a data warehouse to perform comprehensive data management, and comprises data query, data aggregation, data cleaning and filtering, and the Hive module cleans weather characteristic data of rainfall, air temperature, humidity, wind speed and wind direction.
According to some embodiments of the application, the urban rainfall prediction platform based on Hadoop and a neural network further comprises a Flink module for cleaning weather feature picture data of cloud layer visual thickness and cloud layer percentage per unit time.
The application has the beneficial effects that:
In the first aspect, in the prior art, weather information of a surrounding space cannot be acquired by using single target position historical data, but the occurrence of rainfall has very high correlation with the surrounding space, and weather stations of the surrounding space (city) have predicted correlation relative to the target position, so that the rainfall of the target position can be influenced, and therefore, the prediction accuracy can be effectively improved by using weather station data of the surrounding space (city) to participate in prediction.
However, the historical data of peripheral weather stations of the target position are considered in the rainfall prediction of the target position through the neural network model, so that the correlation and the comprehensiveness of rainfall and time and space can be expressed, and the accuracy of rainfall prediction is improved.
In order to adaptively solve the problem that historical data of weather stations around an introduced target position can still be predicted by using a neural network model, the method expresses weather characteristics in a two-dimensional plane determined by coordinate axes to obtain weather characteristic data of corresponding coordinates of the weather characteristics, and the weather characteristic data of normalized rainfall, air temperature, humidity, wind speed and wind direction are processed into an equidistant matrix structure by a Kriging interpolation method, and the characteristic matrix of corresponding characteristics is output, so that the introduced spatially discrete data can be obtained to obtain a spatially continuous characteristic matrix, and the weather data can be processed by a machine learning mode to realize the prediction of the rainfall. Therefore, the data interpolation processing of the invention has the function of introducing the historical data of weather stations around the target position in the rainfall prediction, expressing the correlation and comprehensiveness of rainfall and time and space and improving the accuracy of rainfall prediction.
In a second aspect, the relevant weather features of prediction are rainfall, air temperature, humidity, wind speed, wind direction, cloud layer visual thickness and cloud layer percentage per unit time, wherein the rainfall is particularly large in fluctuation range compared with other features, and the intermediate value is close to 0, so that the general normalization method can predict rainfall only aiming at the rainstorm and the rainless weather, and subsequent back propagation is insensitive, and therefore the rainfall in the weather features is subjected to logarithmic normalization method in a nonlinear normalization method, the difference between larger rainfall and medium and small rainfall is reduced, the problem of neuron inactivation in the back propagation is avoided, and the prediction accuracy of the model is improved. While other meteorological features use the conventional Max-Min normalization method.
In the third aspect, since the image characteristic cloud layer visualization thickness and the cloud layer percentage per unit time are continuous data, interpolation processing is not performed, and other characteristics with discrete characteristics due to the data of weather stations introduced into the peripheral space are subjected to interpolation processing, so that the problem that historical data of the weather stations at the periphery of the target position are introduced into rainfall prediction is solved, the correlation and comprehensiveness of rainfall and time and space can be expressed, the rainfall prediction accuracy is improved, the attribute of the characteristics is screened, and the processing complexity of the data can be reduced.
In the fourth aspect, based on the complexity and strong space-time correlation of climate data, rainfall, time and space have correlation, single time model and space model are independently applicable to comprehensively extract hidden information in the data, so that prediction related information may have deficiency, and the prediction performance is poor.
Drawings
Fig. 1 is a model acclimation flowchart.
Fig. 2 is a schematic diagram of an LSTM cell.
Fig. 3 is a neural network basic structure.
Fig. 4 is an infrastructure of a hybrid neural network model.
Fig. 5 is a prediction effect diagram.
Fig. 6 is a CBAM attention mechanism.
Fig. 7 is a CAM module.
Fig. 8 is a SAM module.
Fig. 9 is a platform overall architecture.
Fig. 10 is a flowchart of the overall architecture of the platform.
FIG. 11 is a visual interface.
Detailed Description
The rainfall prediction method based on the hybrid neural network is suitable for urban rainfall prediction, and comprises the following specific steps:
(1) And after the relevant data is obtained, firstly, carrying out correlation analysis on the rainfall and the historical data, wherein the number of the peripheral position weather stations depends on the predicted future time length, and the longer the predicted time length is, the number of the peripheral position weather stations is required to be increased. And taking the longitude and latitude of the peripheral position weather stations as an xy coordinate axis, so that the target positions and the peripheral position weather stations are distributed in a two-dimensional plane determined by the xy coordinate axis.
Generally, rainfall phenomenon is closely related to movement of rainfall cloud, short-time heavy rainfall often accompanies strong convection of cloud layers and cold air, weather data of peripheral weather stations which are theoretically required by a model are more and more along with longer prediction time, and enough information can be obtained based on the weather data to achieve more accurate prediction accuracy.
(2) And carrying out Pearson correlation analysis on the acquired surrounding meteorological features and rainfall features, and selecting meteorological features with larger rainfall correlation as model features. The pearson correlation coefficient calculating method comprises the following steps:
The larger the absolute value of the result of the pearson correlation coefficient calculation is, the stronger the correlation between the pearson correlation coefficient and the absolute value is, and the pearson coefficient is positive and represents positive correlation, otherwise, represents negative correlation.
After being calculated by the pearson method, the rainfall, the air temperature, the humidity, the wind speed, the wind direction and the cloud layer visual thickness are selected, and the cloud layer percentage in unit time is used as the input characteristic of the prediction model. And expressing the meteorological features in a two-dimensional plane determined by the xy coordinate axes to obtain coordinates (xi, yi) of the corresponding meteorological features. The meteorological feature data may preferably be cleaned.
(3) And normalizing the air temperature, the humidity, the wind speed, the wind direction, the cloud layer visual thickness and the cloud layer percentage in unit time in the obtained meteorological characteristic data, and constructing a training set and a testing set. Wherein the sample time step is t 1 The predicted time length is t 2 A sample dataset is constructed.
The invention selects a normalization method with a normalization mode of Max-Min, and the specific formula is as follows:
x, X in the above max 、X min Respectively an original characteristic value, a maximum value in the original characteristic and a minimum value in the original characteristic of the meteorological characteristic data, X scaled Is the characteristic value of the weather characteristic data after being standardized. The purpose of the normalization is to make the features have the same weight in the model, i.e. features of different scales will all lie between 0 and 1 after normalization.
For the rainfall in the obtained weather characteristic data, the rainfall is usually distributed between 0 and hundreds, the fluctuation range is particularly large, the intermediate value is close to 0, so that rainfall prediction can only be performed on heavy rain and rainless weather, the subsequent back propagation is insensitive, namely, as the rainfall characteristic is distributed in a scattered way, most of the rainfall is distributed near 0, the extreme rainfall weather is less common and the characteristic value is relatively large, if a linear normalization method is also used, the model is insensitive to smaller rainfall and thus model failure is caused, and in order to solve the problems, the logarithmic normalization method in a nonlinear normalization method is selected:
X scaled ＝log 10 (X i +1)
compared with a Max-Min normalization method, the logarithmic normalization reduces the gap between larger rainfall and medium and small rainfall, avoids the problem of neuron inactivation in counter propagation, and improves the prediction accuracy of the model.
(4) The normalized weather feature data is changed into an equidistant matrix structure by using a Kriging method (Kriging) which is a regression algorithm for spatially modeling and predicting (interpolating) a random process and a random field according to a covariance function, and is commonly used in the weather. The method is based on a random interpolation technology of a general least square algorithm, and a variance diagram is used as a weight function; this technique can be applied to any phenomenon that requires the use of point data to estimate its distribution over the surface.
Interpolation converts the above problem into a problem of estimating an attribute value of an arbitrary point (X, Y) in space under the condition that the observed value (characteristic value after normalization) of a certain weather characteristic attribute (such as air temperature) of a plurality of discrete points (Xi, yi) in space is known. Geographic attributes have spatial correlation, and similar things are more similar. The inverse distance interpolation is thus invented, and for the attribute z=z (X, Y) of any point (X, Y), (X, Y) in space, an inverse distance interpolation formula estimator is defined as formula
Where α is typically 1 or 2. The value of an unknown point is estimated by a weighted sum of the data of all known points in space, the weights depending on the inverse (or square of the inverse) of the distance. Then, the point with the closest distance has a large weight; the far points are weighted less. But in general the value of α is often uncertain and the inverse is used to describe the degree of spatial correlation to be inaccurate, thus a kriging interpolation is proposed.
The Kriging method considers the spatial correlation property of the description object in the process of data gridding, so that the interpolation result is more scientific and is closer to the actual situation; can give the interpolation error (Keli variance) to make the interpolation reliability clear, and its calculation is as follows
Wherein the method comprises the steps ofIs the point (x) o ,y 0 ) Of (a), i.e. z o ＝z(x o ,y 0 ). Lambda here i Is a weight coefficient. n represents the number of target locations and surrounding weather stations, which is again a weighted sum of the data from all known points in space to estimate the value of the unknown point. However, the weight coefficient is not the inverse of the distance, but can satisfy the point (x o ,y 0 ) Estimated value +.>And the true value z o A set of optimal coefficients with the smallest difference, i.e. as follows
At the same time meet the condition of unbiased estimation
Var denotes variance and E denotes expectations.
The characteristic data used by the platform established in the embodiment is used for removing cloud layer data (cloud layer visual thickness and cloud layer percentage in unit time), the image information can be directly obtained from a meteorological satellite, and other 5 characteristic rainfall, air temperature, humidity, wind speed and wind direction are all discrete data obtained from a ground observation station, so that the characteristic matrix is constructed by using a kriging interpolation method for five characteristics of rainfall, air temperature, humidity, wind speed and wind direction. And normalizing the cloud layer visual thickness and the cloud layer percentage in unit time to obtain the feature matrix.
(5) Because of the complexity and strong space-time correlation of the climate data, a single time sequence model and a convolution neural network model can lose the information implicit in part of the characteristics, so the invention adopts a hybrid neural network model, and the analysis and the extraction of the characteristic information in space are completed by combining a kriging interpolation method with the convolution neural network;
Seven feature matrices are input into a convolutional neural network to obtain seven serialized feature matrices which are arranged according to time, seven serialized feature matrices are input into a cyclic neural network model (GRU), the output of the cyclic neural network model (GRU) is connected with a full-connection layer, preferably two full-connection layers, prediction accuracy is improved, and the full-connection layer outputs a prediction structure.
And in the time dimension, further training the convolved serialized data by using a recurrent neural network model (GRU) to obtain a final mixed model. The invention tests the accuracy of rainfall prediction after 3 hours, 5 hours and 8 hours respectively for the same target city and compares the prediction result with a single model. The rainfall prediction with multiple time scales is also beneficial to the fact that the mixed model can predict and prevent disasters and losses caused by extreme rainfall earlier in practical application.
The spatial downsampling range of the convolution layer in the hybrid network is 40 longitude (x) x 35 (y) latitude, and considering that interpolation errors exist in the downsampling boundary range part, namely the boundary position errors of interpolation results are larger, and the influence on the final prediction result is smaller as the position of the target city is far away from the center, besides, the prediction performance of the model tends to be greatly reduced as the prediction time is increased, and the time length of rainfall prediction is limited. In order to accelerate training speed and reduce invalid calculation, the invention sets the space downsampling range of a convolution layer to be 16 multiplied by 16 longitude and latitude (set to be twice of the theoretical side length to prevent characteristic information loss), and inputs weather information (historical data) of 10 hours of the sampling field range into the mixed neural network. In connection with the above analysis, the invention results in a hybrid neural network model with a basic structure as shown in FIG. 4.
In the design network architecture, the input of a model (a Feature matrix after interpolation) is a four-dimensional tensor, shape of the four-dimensional tensor is n×long×lati×C, wherein n is the time length of historical data input by a cyclic neural network (GRU), long is the length of a convolution receptive field, namely a longitude range, lati is a latitude range, and C is the number of features. In this embodiment, the model evaluation index selects the MAE index.
(6) The resulting prediction effect of the base model is shown in fig. 5, where a of fig. 5 represents 3 hours rainfall prediction, b represents 5 hours rainfall prediction, and c represents 8 hours rainfall prediction.
Because the weather forecast of different cities has different sensitivity to information of different positions in surrounding space due to the influence of special conditions such as geographic factors, the invention introduces a CBAM Attention mechanism (matrix weighting, enhancing the sensitivity of important features) in the model structure, as shown in fig. 6, the CBAM totally comprises 2 independent sub-modules, a channel Attention module (Channel Attention Module, CAM) and a space Attention module (Spartial Attention Module, SAM), and the attribute weight calculation and multiplication of the channel and the space in two dimensions are respectively carried out, and the final obtained model output is still the same dimension as the input. This not only saves parameters and computational power, but also ensures that it can be integrated into existing network architecture as a plug and play module.
The CAM module uses the input characteristic diagram F (H (length) x W (width) x C (channel)), 7 characteristics of the invention and 7 channels, respectively, through width and height based Global max pooling (global maximum pooling) and Global average pooling (global average pooling) to obtain two characteristic matrixes of 1 x C, and then sends the characteristic matrixes into a two-layer neural network (MLP), wherein the number of neurons of the first layer is C/radio (r is the reduction rate), and a Relu activation function is selected, and the number of neurons of the second layer is C. Then, the feature output by the MLP is subjected to addition operation based on the element, and then is subjected to Sigmoid activation operation to generate a final channel attention feature, namely an M_c weight matrix. And finally, performing element-wise multiplication operation on the M_c and the input feature map F to generate input features required by the Spatial attention module. The structure of which is shown in fig. 7.
Compared with the CAM module SAM module, the CAM module SAM module is simpler, and as shown in FIG. 8, the weighted features output by the CAM module are used as the input feature map. Global max pooling and global average pooling are respectively carried out on the Channel dimension of the feature matrix to obtain two H multiplied by W multiplied by 1 feature graphs, and then the 2 feature graphs are spliced based on channels. Then the dimension is reduced to 1 channel through convolution operation, namely H multiplied by W multiplied by 1. And then generates spatial attention feature, i.e., m_s, via sigmoid. And finally, performing matrix multiplication on the feature and the input feature of the module to obtain a finally generated feature matrix.
According to the invention, the feature matrix after interpolation is overlapped to finally form the feature matrix with a similar picture structure of longitude×Latitude×Featuresize, wherein Featuresize is the Channel dimension in the picture, and generally the picture is an RGB three-Channel color picture, the feature matrix constructed in the invention has 7 features, so 7 channels are provided, and the CAM module can enable the sensitivity degree of the model to different features through the adjustment of the Channel weights, increase the weight proportion of the effective features, and further improve the performance of the whole model.
The performance of the model after adding the CBAM attention module is as follows:
it can be seen that the overall prediction accuracy is improved compared to the one without the CBAM attention mechanism model, especially in some extreme rainfall periods.
(7) And constructing a big data real-time computing platform based on Hadoop and Hive and a front-end and back-end separation WEB project based on SpringBoot, vue to realize real-time data acquisition, ETL, interpolation and real-time prediction display. Meanwhile, historical data inquiry, space and time dimension rainfall display functions, model accuracy curves and model performance monitoring are provided so as to find out an abnormal model and process the abnormal model in time.
The overall platform architecture is shown in fig. 9 and 10.
The final developed WEB application visualization interface is shown in fig. 11.
The invention has the following effects:
(1) Compared with the more complex weather prediction method using weather dynamics, satellite cloud pictures and the like, the method only uses the historical data of ground observation to complete model training and prediction, the model is lighter, can be used in various geographic positions and under the condition of weather, and has wider application range.
(2) Compared with other neural network rainfall prediction models built based on historical data, the method has the advantages that the historical rainfall data of the target position is added into the training of the model, meanwhile, the meteorological data observed by other meteorological stations around the target position is also added, a feature matrix is built in space by using a two-dimensional interpolation method, so that the model learns information in a time dimension through a cyclic neural network, meanwhile, the spatial continuity of meteorological problems is utilized, the spatial meteorological features are processed by using the convolutional neural network model, and the model can be trained in a limited historical data set to obtain good sensitivity and prediction accuracy.
(3) Besides the theoretical model, the invention also provides a big data computing platform based on Hadoop and a visual display technology, and realizes the application process from theory to practice.
Big data technology has become a powerful tool that can manage and process large amounts of data at high speed, in a variety and accurately. In particular Hadoop, is widely accepted due to its distributed storage and processing capabilities, making it an ideal choice for processing meteorological data. However, hadoop has still limited application in urban rainfall forecasting, with opportunities for further development and application.
Hive is a data warehouse software built on top of Hadoop that facilitates reading, writing and managing large data sets in distributed storage. The ease of use of the SQL-like interface and the ability to run complex analytical queries on large data sets make it a powerful tool in the field of data analysis. However, their potential in the analysis and prediction of meteorological data has not been fully exploited. On the other hand, neural networks, as a subset of artificial intelligence, have achieved a promising outcome in terms of prediction because of the ability to learn and adapt from large amounts of data. The ability of neural networks to capture nonlinear relationships makes them well suited for predicting complex phenomena such as weather patterns. However, the integration of neural networks with Hadoop and Hive for urban rainfall forecasting remains a largely unknown area. In addition, visualization of the prediction results is also an important aspect, which helps to explain and understand the predictions. Spring and Vue techniques have been widely used to create visual and interactive visualizations. Nevertheless, seamless integration of these techniques with big data and machine learning based prediction systems remains a challenge.
In view of these circumstances, there is an urgent need for an innovative system capable of enhancing urban rainfall forecast using the powerful functions of Hadoop, hive, neural networks and visualization techniques. The introduction of such a system would represent an important step forward in the analysis and prediction of meteorological data, providing a more reliable and efficient tool for managing the effects of urban rainfall.
The prediction method of the invention can be realized on a city rainfall prediction platform based on Hadoop and a neural network, wherein the platform comprises
Data collection and preprocessing: initially, meteorological data from various sources will be collected. The Hadoop Distributed File System (HDFS) will be used to store and manage these massive data. Data preprocessing techniques will be applied to clean and normalize the data, ready for further analysis.
Data analysis was performed using Hive: hive will be used to create a data warehouse for comprehensive data management (data querying, data aggregation, data cleansing, filtering). Hive provides an SQL-like interface that will be used to run complex analytical queries on data. Hive cleans 5 data of rainfall, air temperature, humidity, wind speed and wind direction in the prediction method
The Flink is to clean the other 2 image data cloud layer visual thickness and cloud layer percentage per unit time.
Model weight module: for neural network training and prediction: the preprocessed data are used for training a neural network model to conduct rainfall prediction, and the prediction method comprises the steps. The model learns complex nonlinear relations in the data, so that rainfall modes can be accurately predicted. After training is completed, the model will be used to predict future rainfall events.
Visualization using Springboot and Vue tools: by means of Spring and Vue, the prediction results are visualized in an intuitive and interactive way. This will facilitate interpretation of the forecast data and facilitate decision making processes associated with urban rainfall management.
Evaluation and fine tuning module: the performance of the prediction platform will be evaluated periodically using appropriate metrics. Based on the evaluation, the neural network model will be trimmed to improve its prediction accuracy.
Expansion and deployment: finally, the platform will be extended to handle larger data sets and deployed for real-time urban rainfall forecasting.
The invention also utilizes the capabilities of Hadoop and Hive to effectively manage, process and analyze massive complex meteorological data, thereby overcoming the limitation of the traditional data processing technology. Enhanced prediction accuracy: by taking advantage of the ability of neural networks to capture nonlinear relationships and learn from large amounts of data, the present invention aims to significantly improve the accuracy of rainfall predictions, particularly in predicting extreme weather events. Technology integration and application: the invention aims to create a comprehensive rainfall forecast platform by first creating the integration of visualization technologies such as Hadoop, hive, neural network, spring, vue and the like. Effective visualization: by means of Spring and Vue technologies, the platform aims at providing visual and interactive visual display of prediction results, so that interpretation and understanding of prediction are enhanced. Real-time response: the present invention aims to achieve real-time data processing and prediction, which is critical for effective urban rainfall management and minimizing the impact of heavy rainfall events on urban areas. In general, the present invention aims to advance the field of urban rainfall forecasting, providing a more reliable, accurate and efficient tool to better serve urban planning, disaster management and other related departments.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (10)
1. The urban rainfall prediction method based on the hybrid neural network is characterized by comprising the following steps of:
s10, acquiring weather observation historical data of weather stations at the target position and the peripheral positions thereof;
s20, carrying out correlation analysis on the rainfall and the historical data, determining the rainfall, the air temperature, the humidity, the wind speed, the wind direction, the cloud layer visual thickness and the cloud layer percentage in unit time as weather features according to the correlation analysis, and representing the weather features in a two-dimensional plane determined by coordinate axes to obtain weather feature data of corresponding coordinates of the weather features;
s30, normalizing the meteorological feature data, wherein the air temperature, the humidity, the wind speed, the wind direction, the cloud layer visual thickness and the cloud layer percentage in unit time are normalized by a Max-Min normalization method, the rainfall is normalized by a logarithmic normalization method, and the normalized cloud layer visual thickness and the meteorological feature data of the cloud layer percentage in unit time output feature matrixes of corresponding features;
S40, processing weather characteristic data of normalized rainfall, air temperature, humidity, wind speed and wind direction into an equidistant matrix structure by a Kriging interpolation method, and outputting a characteristic matrix of corresponding characteristics;
s50, inputting the characteristic matrix of the meteorological characteristic data into a convolutional neural network to obtain a time-arranged serialized characteristic matrix, inputting the time-arranged serialized characteristic matrix into a cyclic neural network model, inputting the output of the cyclic neural network model into a full-connection layer, and outputting a prediction result by the full-connection layer.
2. The urban rainfall prediction method based on the hybrid neural network according to claim 1, wherein the convolutional neural network in the step S50 comprises
A first convolution layer, a first CBAM attention mechanism, a first pooling layer, a second convolution layer, a second CBAM attention mechanism, a second pooling layer, and a flame layer, the first CBAM attention mechanism comprising a first channel attention module and a first spatial attention module, the second CBAM attention mechanism comprising a second channel attention module and a second spatial attention module;
the characteristic matrix of the weather characteristic data is input into a first convolution layer, the weather characteristic data is expressed as a four-dimensional tensor, and the output of the first convolution layer is a characteristic diagram of the weather characteristic data;
Inputting the feature map of each meteorological feature data output by a first convolution layer into a first channel attention module of a first CBAM attention mechanism, wherein the feature map is H multiplied by W multiplied by C, H is long, W is wide, C is a channel, in the first channel attention module, two feature matrixes of 1 multiplied by C are obtained by the respective channels through global maximum pooling and global average pooling based on width and height respectively, the two feature matrixes are respectively input into two layers of neural network MLP, the number of neurons of the first layer of neural network MLP is C/radio, C is the channel, radio is the reduction rate, the activation function is Relu, the number of neurons of the second layer of neural network MLP is C, the features output by the two layers of neural network MLP are subjected to element-based addition operation, then the M_c weight matrix is generated through Sigmoid activation operation, and the M_c weight matrix and the feature matrixes are subjected to element-wise multiplication operation to obtain weighted features;
the weighted feature is used as an input feature map to be input into a first space attention module of a first CBAM attention mechanism, global average pooling and global maximum pooling are respectively carried out on Channel dimensions of a feature matrix to obtain two H multiplied by W multiplied by 1 feature maps, 2 feature maps are spliced by channels, then the dimension is reduced to 1 Channel through convolution operation to be the H multiplied by W multiplied by 1 feature map, a weight matrix M_s is generated through sigmoid operation, and the weight matrix M_s and the weighted feature are subjected to matrix multiplication to obtain a finally generated feature matrix;
Inputting the finally generated feature matrix into a first pooling layer, and inputting the output of the first pooling layer into a second convolution layer;
the output of the second convolution layer is a characteristic diagram of weather characteristic data, the characteristic diagram of the weather characteristic data output by the second convolution layer is input into a second channel attention module of a second CBAM attention mechanism, the characteristic diagram is H multiplied by W multiplied by C, H is long, W is wide, C is a channel, in the second channel attention module, the characteristics output by the two layers of the neural network MLP are subjected to global maximum pooling and global average pooling based on width and height respectively by the channels to obtain two characteristic matrixes of 1 multiplied by C, the two characteristic matrixes are respectively input into two layers of the neural network MLP, the number of neurons of the first layer of the neural network MLP is C/radio, C is the channel, the radio is the reduction rate, the activation function is Relu, the number of neurons of the second layer of the neural network MLP is C, the characteristics output by the two layers of the neural network MLP are subjected to addition and operation based on element, and then the element activation operation is performed to generate M_c weight matrixes, and the element weighting operation is performed to obtain the characteristic weight matrixes;
the weighted feature is used as an input feature map to be input into a second space attention module of a second CBAM attention mechanism, global average pooling and global maximum pooling are respectively carried out on Channel dimensions of a feature matrix to obtain two H multiplied by W multiplied by 1 feature maps, 2 feature maps are spliced by channels, then the dimension is reduced to 1 Channel through convolution operation to be the H multiplied by W multiplied by 1 feature map, a weight matrix M_s is generated through sigmoid operation, and the weight matrix M_s and the weighted feature are subjected to matrix multiplication to obtain a finally generated feature matrix;
Inputting the finally generated feature matrix into a second pooling layer, and inputting the output of the second pooling layer into a flat layer;
the output of the flat layer sequences the feature matrix in time.
3. The urban rainfall prediction method based on the hybrid neural network according to claim 2, wherein the spatial downsampling range of the convolutional neural network is 16×16 latitude and longitude, the input of the convolutional neural network is a four-dimensional tensor of meteorological information of 10 hours of the history of the sampling field range, shape of the four-dimensional tensor is n×long×lati×c, where n is the time length of the history data input by the convolutional neural network, long is the longitude range, lati is the latitude range, and C is the number of features.
4. The urban rainfall prediction method based on hybrid neural network according to claim 2, wherein the correlation is calculated by pearson correlation coefficient in step S20.
5. The urban rainfall prediction method based on the hybrid neural network according to claim 2, wherein,
s30, normalizing meteorological characteristic data, wherein the air temperature, the humidity, the wind speed, the wind direction, the cloud layer visual thickness and the cloud layer percentage in unit time are normalized by a Max-Min normalization method, and are expressed as follows by a formula:
In X, X max 、X min X is the original meteorological feature value, the maximum value in the original meteorological feature and the minimum value in the original meteorological feature respectively scaled The weather characteristic values after normalization are between 0 and 1 after normalization treatment;
the rainfall is normalized by a logarithmic normalization method, and the normalized formula is as follows:
X scaled ＝log 10 (X i +1)
wherein X is i Is the original meteorological characteristic value.
6. The urban rainfall prediction method based on the hybrid neural network according to claim 2, wherein in step S40, the kriging interpolation method comprises
Obtaining a difference result by estimating the attribute value of any point (X, Y) in space under the condition that the feature value zi=z (Xi, yi) after normalization of a certain meteorological attribute of a plurality of discrete points (Xi, yi) in space is known;
is expressed by the following formula:
in the middle ofIs the point (x) o ,y 0 ) Estimated value of z o ＝z(x o ,y 0 )，λ i Is a weight coefficient, n represents the number of target positions and surrounding weather stations, n is a weighted sum of data of all known points in space to estimate the value of the unknown point, but the weight coefficient is not the inverse of the distance, is a value capable of satisfying the point (x o ,y 0 ) Estimated value +.>And the true value z o The minimum difference of the optimal coefficients is calculated by the following formula
At the same time meet the condition of unbiased estimationVar denotes variance and E denotes expectations.
7. The urban rainfall prediction method based on the hybrid neural network according to claim 2, further comprising the step of cleaning the obtained weather feature data of the corresponding coordinates of the weather feature in step S20.
8. Urban rainfall prediction platform based on Hadoop and neural network, which is characterized by comprising
The Hadoop distributed file system is used for storing and managing meteorological data;
the correlation analysis module comprises a step of performing correlation analysis on rainfall and historical data;
the normalization module is used for normalizing the weather characteristic data determined by the correlation analysis;
the interpolation module is used for processing weather characteristic data of normalized rainfall, air temperature, humidity, wind speed and wind direction into an equidistant matrix structure through a Kriging interpolation method, and outputting a characteristic matrix of corresponding characteristics;
the model weight module comprises a convolutional neural network, a cyclic neural network and a full-connection layer, wherein the characteristic matrix of meteorological characteristic data is input into the convolutional neural network, a time-arranged serialized characteristic matrix is obtained by the convolutional neural network, the time-arranged serialized characteristic matrix is input into the cyclic neural network model, the output of the cyclic neural network model is input into the full-connection layer, and the full-connection layer is used for outputting a prediction result;
Springboot and Vue tools for visualizing the prediction results in an intuitive and interactive way.
9. The urban rainfall prediction platform based on Hadoop and neural network according to claim 8, further comprising a Hive module for creating a data warehouse for comprehensive data management including data query, data aggregation, data cleaning and filtering, wherein the Hive module cleans meteorological characteristic data of rainfall, air temperature, humidity, wind speed and wind direction.
10. The urban rainfall prediction platform based on Hadoop and a neural network according to claim 9, further comprising a Flink module for cleaning weather feature picture data of cloud layer visual thickness and cloud layer percentage per unit time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311145680.1A CN117131991A (en) | 2023-09-06 | 2023-09-06 | Urban rainfall prediction method and platform based on hybrid neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311145680.1A CN117131991A (en) | 2023-09-06 | 2023-09-06 | Urban rainfall prediction method and platform based on hybrid neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117131991A true CN117131991A (en) | 2023-11-28 |
Family
ID=88861099
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311145680.1A Pending CN117131991A (en) | 2023-09-06 | 2023-09-06 | Urban rainfall prediction method and platform based on hybrid neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117131991A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117388953A (en) * | 2023-12-13 | 2024-01-12 | 南京信息工程大学 | Weather forecast method for improving MIM-rwkv by SADBO based on big data frame |
-
2023
- 2023-09-06 CN CN202311145680.1A patent/CN117131991A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117388953A (en) * | 2023-12-13 | 2024-01-12 | 南京信息工程大学 | Weather forecast method for improving MIM-rwkv by SADBO based on big data frame |
| CN117388953B (en) * | 2023-12-13 | 2024-03-08 | 南京信息工程大学 | Weather forecast method for improving MIM-rwkv by SADBO based on big data frame |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110675623B (en) | Short-term traffic flow prediction method, system and device based on hybrid deep learning | |
| Ma et al. | Transfer learning for long-interval consecutive missing values imputation without external features in air pollution time series | |
| CN109508360B (en) | Geographical multivariate stream data space-time autocorrelation analysis method based on cellular automaton | |
| CN114220271A (en) | Traffic flow prediction method, equipment and storage medium based on dynamic space-time graph convolution cycle network | |
| CN114565124B (en) | Ship traffic flow prediction method based on improved graph convolution neural network | |
| Lian et al. | A novel data-driven tropical cyclone track prediction model based on CNN and GRU with multi-dimensional feature selection | |
| CN113591380A (en) | Traffic flow prediction method, medium and equipment based on graph Gaussian process | |
| CN115376317B (en) | Traffic flow prediction method based on dynamic graph convolution and time sequence convolution network | |
| CN117494034A (en) | Air quality prediction method based on traffic congestion index and multi-source data fusion | |
| CN114936691A (en) | Temperature forecasting method integrating relevance weighting and space-time attention | |
| CN117131991A (en) | Urban rainfall prediction method and platform based on hybrid neural network | |
| CN110689183A (en) | Cluster photovoltaic power probability prediction method, system, medium and electronic device | |
| Zhao et al. | Mastgn: Multi-attention spatio-temporal graph networks for air pollution prediction | |
| CN114417693A (en) | Ocean three-dimensional temperature field inversion method based on deep learning | |
| Xu et al. | Monthly NDVI prediction using spatial autocorrelation and nonlocal attention networks | |
| CN112668711B (en) | Flood flow prediction method and device based on deep learning and electronic equipment | |
| Tavassoli et al. | Comparison of Kriging and artificial neural network models for the prediction of spatial data | |
| CN117194926A (en) | Method and system for predicting hoisting window period of land wind power base | |
| Yalçın | Weather parameters forecasting with time series using deep hybrid neural networks | |
| Fang et al. | Zonation and scaling of tropical cyclone hazards based on spatial clustering for coastal China | |
| Mozaffari et al. | A hierarchical selective ensemble randomized neural network hybridized with heuristic feature selection for estimation of sea-ice thickness | |
| Wei | Real-time extreme rainfall evaluation system for the construction industry using deep convolutional neural networks | |
| Wen et al. | AdaNAS: Adaptively Post-processing with Self-supervised Neural Architecture Search for Ensemble Rainfall Forecasts | |
| CN118072873B (en) | Air quality prediction method | |
| CN117456738B (en) | Expressway traffic volume prediction method based on ETC portal data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |