CN114385611A - Precipitation prediction method and system based on artificial intelligence algorithm and knowledge graph - Google Patents
Precipitation prediction method and system based on artificial intelligence algorithm and knowledge graph Download PDFInfo
- Publication number
- CN114385611A CN114385611A CN202111625218.2A CN202111625218A CN114385611A CN 114385611 A CN114385611 A CN 114385611A CN 202111625218 A CN202111625218 A CN 202111625218A CN 114385611 A CN114385611 A CN 114385611A
- Authority
- CN
- China
- Prior art keywords
- data
- precipitation
- radar
- prediction
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001556 precipitation Methods 0.000 title claims abstract description 190
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 43
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 32
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000004140 cleaning Methods 0.000 claims abstract description 7
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 230000011218 segmentation Effects 0.000 claims abstract description 7
- 238000006243 chemical reaction Methods 0.000 claims description 30
- 230000006870 function Effects 0.000 claims description 16
- 238000010586 diagram Methods 0.000 claims description 15
- 238000011156 evaluation Methods 0.000 claims description 14
- 238000013527 convolutional neural network Methods 0.000 claims description 13
- 238000012549 training Methods 0.000 claims description 11
- 238000013528 artificial neural network Methods 0.000 claims description 10
- 238000012937 correction Methods 0.000 claims description 9
- 230000004927 fusion Effects 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 7
- 238000003062 neural network model Methods 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 5
- 238000012800 visualization Methods 0.000 claims description 5
- 239000008186 active pharmaceutical agent Substances 0.000 claims description 4
- 238000010219 correlation analysis Methods 0.000 claims description 4
- 230000010354 integration Effects 0.000 claims description 4
- 238000010801 machine learning Methods 0.000 claims description 4
- 230000007246 mechanism Effects 0.000 claims description 4
- 238000011160 research Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 claims description 3
- 238000007499 fusion processing Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 description 8
- 238000000605 extraction Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000010304 firing Methods 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000001035 drying Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000004451 qualitative analysis Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01W—METEOROLOGY
- G01W1/00—Meteorology
- G01W1/14—Rainfall or precipitation gauges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A50/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE in human health protection, e.g. against extreme weather
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Environmental & Geological Engineering (AREA)
- Computational Linguistics (AREA)
- Fuzzy Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Animal Behavior & Ethology (AREA)
- Hydrology & Water Resources (AREA)
- Software Systems (AREA)
- Atmospheric Sciences (AREA)
- Biodiversity & Conservation Biology (AREA)
- Ecology (AREA)
- Environmental Sciences (AREA)
- Quality & Reliability (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a precipitation prediction method and system based on an artificial intelligence algorithm and a knowledge graph, belonging to the technical field of weather forecast and comprising the following steps: constructing a multi-mode data container, inputting different types of meteorological data into the multi-mode data container according to the structural characteristics of the multi-mode data container to obtain multi-mode data, and performing space-time alignment, data cleaning and preprocessing and data sequence segmentation on the multi-mode data; constructing a knowledge graph about precipitation prediction; and (3) creating a multi-modal precipitation prediction model based on an artificial intelligence algorithm, and intelligently correcting the predicted precipitation data. The method not only makes full use of different types of multi-modal meteorological data, improves the precision of precipitation prediction through a multi-modal precipitation prediction method, but also constructs a precipitation prediction knowledge map, and can intelligently correct the result of model prediction so as to reduce the uncertainty of an artificial intelligence algorithm and improve the accuracy and reliability of model prediction.
Description
Technical Field
The invention belongs to the technical field of weather forecast, and particularly relates to a precipitation prediction method and system based on an artificial intelligence algorithm and a knowledge graph.
Background
In recent years, precipitation is greatly increased in various parts of the world, and precipitation prediction plays an important role in urban traffic, agricultural systems, and even water resource planning and flood control early warning. At present, the prediction of the adjacent precipitation by using meteorological big data is a hotspot and a difficulty of domestic disaster prevention and reduction, and has important value and social significance for the prediction of the adjacent precipitation with high space-time resolution.
From the whole rainfall prediction development point of view, weather workers mainly use a method of a weather map, and the method has long history and mature theory, but is subjective and is qualitative analysis. With the continuous development of meteorology and computer technology, a numerical prediction method is gradually developed, and mainly is a method for predicting precipitation change by solving an atmospheric dynamics equation set. This method is also a relatively large method used in practice in recent years, but has a certain limitation in application. First, the accuracy of predictions in the short term (especially within 1-2 hours) of this method tends to be low. Mainly, the result of short-term prediction depends on the initial value of the equation set of atmospheric dynamics, but the currently acquired atmospheric information is limited, so that the initial value of the equation is difficult to estimate accurately. Secondly, the cooperation of a super computer is required, the size of the super computer is large, and expensive calculation cost and maintenance cost are required. In addition, the time spent in the numerical model is often long in terms of time, and the calculation results often take several hours from the acquisition of data to the assimilation to the forecasting, and the forecasting may lose the time effectiveness for a specific field.
In recent years, with the doubling of the types and the number of meteorological data, the combination of the artificial intelligence technology and precipitation prediction becomes a necessary trend, useful information can be mined from massive multi-modal meteorological data, climate characteristics and atmospheric motion can be found, and then precipitation can be predicted accurately. Meanwhile, the artificial intelligence algorithm has certain uncertainty and may have certain errors for some rare scene predictions.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a precipitation prediction method and system based on an artificial intelligence algorithm and a knowledge graph, so as to solve the problems in the background technology.
The invention is realized in such a way that a precipitation prediction method based on an artificial intelligence algorithm and a knowledge graph comprises the following steps:
constructing a multi-mode data container, inputting different types of meteorological data into the multi-mode data container according to the structural characteristics of the multi-mode data container to obtain multi-mode data, and performing space-time alignment, data cleaning and preprocessing and data sequence segmentation on the multi-mode data;
constructing a knowledge graph about precipitation prediction;
establishing a multi-modal precipitation prediction model based on an artificial intelligence algorithm, specifically: constructing a multi-modal radar echo prediction model, performing space-time fusion on various historical meteorological data, and predicting a future radar sequence diagram by using a deep neural network model; constructing a radar precipitation intelligent conversion model, combining geographical position data, utilizing a confrontation generation network to carry out mutual conversion on radar data and precipitation, and converting a predicted radar sequence diagram into precipitation data; based on the constructed rainfall prediction knowledge graph, carrying out knowledge reasoning by using a neural network algorithm, and intelligently correcting predicted rainfall data;
and carrying out classification visualization on the corrected precipitation data.
As a further scheme of the invention: the step of performing spatiotemporal alignment on the multi-modal data specifically includes:
merging and aligning historical radar echo sequence data with a time error of less than 2 minutes with satellite cloud map data; carrying out bilinear interpolation on data of a meteorological station, fusing data obtained after processing the bilinear interpolation with radar data and satellite data, and carrying out space matching and alignment on the radar data, the satellite data, the data of the meteorological station and altitude data according to longitude and latitude;
and performing space-time alignment on the radar sequence data and the geographical position big data and the precipitation data, wherein the geographical position big data comprises longitude and latitude data and altitude data.
As a further scheme of the invention: the step of constructing the knowledge graph about precipitation prediction specifically comprises the following steps:
acquiring data related to precipitation, including unstructured precipitation data and semi-structured precipitation data obtained from open websites and research, and structured precipitation data obtained from radar and satellites;
aiming at unstructured precipitation data and semi-structured precipitation data, extracting entities, relations and attributes of precipitation prediction by using a convolutional neural network based on an attention mechanism to obtain extracted precipitation information, wherein the entities comprise longitude and latitude, altitude, time, temperature, wind direction, humidity, precipitation and the like;
fusing the structured precipitation data and the extracted precipitation information, wherein the fusion process comprises precipitation entity disambiguation and alignment;
further processing the extracted precipitation information, and analyzing the relation between longitude and latitude, altitude, time, temperature, wind direction, humidity and precipitation by using a machine learning correlation analysis algorithm, namely giving specific values of meteorological factors to obtain the maximum precipitation and the minimum precipitation of the area;
and constructing a precipitation prediction knowledge graph according to the extracted precipitation information, and storing the precipitation prediction knowledge graph into a graph database.
As a further scheme of the invention: the step of constructing the multi-modal radar echo prediction model specifically comprises the following steps:
creating a multi-modal radar echo prediction model consisting of a Data Integration module, a CNN + LSTM module, a DS + LSTM module, an US + LSTM module, a CNN + LSTM module and a Data Generation module, wherein the multi-modal radar echo prediction model utilizes meteorological Data of a 512km area and radar images of a prediction center area 256km for better predicting precipitation of the center area; the multi-modal radar echo prediction model is used for fusing historical multi-modal meteorological big data, the historical multi-modal meteorological big data comprises radar data, satellite cloud pictures, longitude and latitude data, altitude data, temperature, wind direction and humidity, and a plurality of neural network modules are used for training and learning so as to accurately predict future high-resolution radar echo pictures; the loss function used by the multi-modal radar echo prediction model is:
wherein,
radar map data, y, representing model predictionskRepresenting true radar map data, ωkAnd mukRepresenting a weighting value for radar maps of different intensities;
and evaluating the multi-modal radar echo prediction model by utilizing an evaluation function, wherein the evaluation function is as follows:
the Value evaluation result Value is between-1 and 1, the effect is better when the Value evaluation result Value is larger, wherein M is the predicted radar chart sequence length, and Value isiScoring for a single radar chart, i.e.:
where L is the predicted class, N is the total number of samples, ωjIs the weight of the jth class, p (R)i，Tj) To representPredicting the total number of pixels with the real category j in the pixels with the category i, p (R)i) Representing the total number of pixels predicted to be of class i, p (T)j) Representing the total number of pixels of the real category j.
As a further scheme of the invention: the step of constructing the radar precipitation intelligent conversion model specifically comprises the following steps: constructing a conversion model comprising an R2P generator, a P2R generator, a P discriminator and an R discriminator, and training and optimizing the conversion model on a gpu cluster by combining longitude and latitude and altitude geographical position data and utilizing historical radar data and precipitation data to obtain an intelligent radar precipitation conversion model; the loss function used by the radar precipitation intelligent conversion model is as follows:
Loss＝Loss1+Loss2+Loss3
wherein:
another object of the present invention is to provide a precipitation prediction system based on artificial intelligence algorithm and knowledge-graph, the system comprising:
the multi-model data container is used for inputting different types of meteorological data into the multi-model data container according to the structural characteristics of the multi-model data container to obtain multi-model data, and performing space-time alignment, data cleaning and preprocessing and data sequence segmentation on the multi-model data;
the rainfall prediction knowledge map building module is used for building a knowledge map about rainfall prediction;
the multi-modal precipitation prediction and correction module based on the artificial intelligence algorithm is used for creating a multi-modal precipitation prediction model based on the artificial intelligence algorithm, and specifically comprises the following steps: the radar echo prediction model construction unit is used for constructing a multi-mode radar echo prediction model, performing space-time fusion on various historical meteorological data, and predicting a future radar sequence diagram by using a deep neural network model; the intelligent conversion model building unit is used for building a radar precipitation intelligent conversion model, combining the geographical position data and utilizing a countermeasure generation network to carry out mutual conversion on radar data and precipitation, and then converting the predicted radar sequence diagram into precipitation data; the intelligent correction unit is used for carrying out knowledge reasoning by utilizing a neural network algorithm based on the constructed rainfall prediction knowledge map and carrying out intelligent correction on the predicted rainfall data; and
and the regional precipitation visualization module is used for classifying and visualizing the corrected precipitation data.
Has the advantages that: the invention provides a precipitation prediction method and system based on an artificial intelligence algorithm and a knowledge graph, which not only fully utilize different types of multi-modal meteorological data, improve the precision of precipitation prediction and solve the problems in the prior art through the multi-modal precipitation prediction method, but also construct a precipitation prediction knowledge graph, and can intelligently correct the result of model prediction so as to reduce the uncertainty of the artificial intelligence algorithm and further improve the accuracy and reliability of the model prediction.
Drawings
In order to more clearly illustrate the technical solutions and embodiments of the present invention, the drawings used in the technical solutions and embodiments will be briefly described below.
Fig. 1 is a flow chart of a precipitation prediction method based on an artificial intelligence algorithm and a knowledge graph.
FIG. 2 is a flow chart of construction of a knowledge graph of precipitation predictions in an embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a multi-modal radar echo prediction model in an embodiment of the present invention.
Fig. 4 is a schematic structural diagram of an intelligent conversion model of radar precipitation in the embodiment of the invention.
Fig. 5 is an architecture diagram of a precipitation prediction system based on artificial intelligence algorithms and knowledge maps.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Specific implementations of the present invention are described in detail below with reference to specific embodiments.
As shown in fig. 1, an embodiment of the present invention provides a precipitation prediction method based on an artificial intelligence algorithm and a knowledge graph, where the method includes the following steps:
s100, constructing a multi-mode data container, inputting different types of meteorological data into the multi-mode data container according to the structural characteristics of the multi-mode data container to obtain multi-mode data, and performing space-time alignment, data cleaning and preprocessing and data sequence segmentation on the multi-mode data;
s200, constructing a knowledge map about precipitation prediction, extracting precipitation related information from public websites and precipitation related knowledge in research by using a natural language processing technology, fusing the precipitation related information with precipitation data acquired by a radar, a satellite and a meteorological station, and constructing a precipitation prediction knowledge map by using a deep learning algorithm;
s300, creating a multi-modal precipitation prediction model based on an artificial intelligence algorithm, specifically:
s301, constructing a multi-modal radar echo prediction model, performing space-time fusion on various historical meteorological data, and predicting a future radar sequence diagram by using a deep neural network model;
s302, constructing a radar precipitation intelligent conversion model, combining with geographical position data, utilizing a confrontation generation network to carry out mutual conversion on radar data and precipitation, and converting a predicted radar sequence diagram into precipitation data;
s303, based on the constructed rainfall prediction knowledge graph, carrying out knowledge reasoning by using a neural network algorithm, and intelligently correcting the predicted rainfall data;
s400, classifying and visualizing the corrected precipitation data, facilitating observation and contrastive analysis of users, classifying the precipitation into 8 classes (0), (0, 2.5), (2.5, 5), (5, 10), (10, 25), (25, 50), (50, 100), (100, and +∞) (unit: mm/h) for more intuitive observation, and then displaying the precipitation data on a map based on longitude and latitude data.
In the embodiment of the invention, before acquiring the multi-modal meteorological data related to precipitation, meteorological data such as historical radar echo sequence data, satellite cloud chart data, meteorological station data (temperature, humidity, wind direction and the like), geographic position big data (including longitude and latitude data, altitude data and the like), precipitation data and the like in a period of time in the same region need to be acquired, and the data are stored in a multi-modal data container according to categories. Then, the obtained multi-modal data are required to be aligned in a space-time mode, so that the processed data can be used for constructing a multi-modal radar echo prediction model and a radar precipitation intelligent conversion model; in addition, all data are cleaned and preprocessed, a smoothing factor is introduced to remove ground features from historical data, then two-dimensional wavelet transformation is used for drying, default data are filled by bilinear interpolation, in addition, some abnormal sequences or sequences with excessive missing values are removed, the noise is prevented from interfering the model, and finally, all preprocessed data are stored in a data container for subsequent modeling.
In an embodiment of the present invention, the step of performing spatio-temporal alignment on the multimodal data specifically includes:
s101, merging and aligning historical radar echo sequence data with time error less than 2 minutes with satellite cloud picture data; carrying out bilinear interpolation on data of a meteorological station, fusing data obtained after processing the bilinear interpolation with radar data and satellite data, and carrying out space matching and alignment on the radar data, the satellite data, the data of the meteorological station and altitude data according to longitude and latitude; it should be noted that, in terms of time, the time interval of historical radar echo sequence data is generally 6 minutes, the time interval of satellite cloud map data is generally 5 minutes, and data with the time closest to the radar map time (with an error smaller than 2 minutes) are integrated with the radar map time as a reference; secondly, the time interval of the meteorological station data (temperature, humidity, wind direction and the like) is generally 1 hour, and bilinear interpolation is carried out on the meteorological station data, so that fusion can be carried out on the meteorological station data and radar and satellite data; in space, radar data, satellite data, meteorological station data (temperature, humidity, wind direction and the like) and altitude data are spatially aligned according to longitude and latitude, and the fused data can be used for constructing a multi-modal radar echo prediction model.
S102, performing space-time alignment on radar sequence data, geographical position big data and precipitation data, wherein the geographical position big data comprises longitude and latitude data and altitude data. Similarly, the time interval of the acquired precipitation data is generally 1 hour, radar sequence data, geographical position big data (including longitude and latitude data, altitude data and the like) and precipitation data are aligned in space and time on the basis of the data, and the fused data can be used for constructing a radar precipitation intelligent conversion model.
As shown in fig. 2, in the embodiment of the present invention, the step of constructing a knowledge graph about precipitation prediction specifically includes:
s201, acquiring data related to precipitation, wherein the data comprises unstructured precipitation data and semi-structured precipitation data acquired from open websites and researches, and structured precipitation data acquired from radar and satellites;
s202, extracting entities, relations and attributes of rainfall prediction by using a convolutional neural network based on an attention mechanism aiming at unstructured rainfall data and semi-structured rainfall data to obtain extracted rainfall information, wherein the entities comprise longitude and latitude, altitude, time, temperature, wind direction, humidity, rainfall amount and the like;
and S203, fusing the structured precipitation data and the extracted precipitation information, wherein the fusion process comprises precipitation entity disambiguation and alignment.
S204, further processing the extracted precipitation information, and analyzing the relation between longitude and latitude, altitude, time, temperature, wind direction, humidity and precipitation by using a machine learning correlation analysis algorithm, namely giving specific values of meteorological factors to obtain the maximum precipitation and the minimum precipitation of the area;
s205, constructing a precipitation prediction knowledge graph according to the extracted precipitation information, and storing the precipitation prediction knowledge graph into a graph database.
In the embodiment of the present invention, first, from the public web page and the paper, the unstructured data and the semi-structured data are obtained by analyzing the page and the data through technologies such as a crawler, for example: "20 days 7 and 20 months in 2021, 4 to 5 pm, one hour of cumulative precipitation at Zhengzhou station reaches 201.9 mm, creates single hour precipitation record of large, medium and small cities in the world", and obtains the structured data related to precipitation from products such as radar satellites. Knowledge extraction is carried out on unstructured and semi-structured precipitation data, and entity extraction is firstly carried out, such as entities related to precipitation, such as longitude, latitude, temperature, altitude, wind direction, humidity, precipitation amount and the like. Because the acquired data is limited, the entity extraction is mainly finished by fine adjustment based on a pre-training migration model in natural language processing and utilizing a field with rich training expectation to help the entity extraction. Secondly, extracting the relationship and the attribute of the entities, wherein the convolutional neural network based on the attention mechanism is mainly used for extracting the relationship and the attribute by taking the combined sentence vector as a unit. For example, the knowledge extracted for an area is longitude: 109.809000, latitude: 40.657000, altitude: 1069m, time: 6/1/2021, temperature: 10 degrees centigrade, wind direction: grade 3, precipitation: 20mm/h (note that this is only for a simple illustration of the extraction results, and the data may have some deviation).
It is also necessary to use structured data obtained from radar satellites and other products, in combination with previously extracted knowledge and weather expert experience, to fuse knowledge, including entity disambiguation and alignment, such as "precipitation", "rainfall amount", "precipitation amount", and the like, all unified as "precipitation amount". The biLSTM neural network is mainly used, the entity alignment is regarded as similarity contrast between character strings, and finally the similarity of the rest strings is calculated to be the matching probability of the entity. And finally, further processing the extracted knowledge. And analyzing the relation between other meteorological factors such as longitude and latitude, altitude, time, temperature, wind direction, humidity and the like and the precipitation by utilizing a machine learning correlation analysis algorithm, namely giving specific values of the meteorological factors to obtain the maximum precipitation and the minimum precipitation of the area. And constructing a knowledge graph of rainfall prediction, constructing the extracted data into an ontology in an RDF format by using an inference engine based on an RDF model, and importing the ontology into a graph database to complete construction and storage of the knowledge graph.
As shown in fig. 3, in the embodiment of the present invention, the step of constructing the multi-modal radar echo prediction model specifically includes:
s3011, a multi-mode radar echo prediction model formed by a Data Integration module, a CNN + LSTM module, a DS + LSTM module, an US + LSTM module, a CNN + LSTM module and a Data Generation module is created, and as atmospheric clouds are constantly moving, the model enlarges the receptive field of observation Data for better predicting precipitation in a central area, namely, meteorological Data of a 512km by 512km area and a radar image of a 256km by 256km prediction central area are utilized; the multi-modal radar echo prediction model is used for fusing historical multi-modal meteorological big data, the historical multi-modal meteorological big data comprises radar data, satellite cloud pictures, longitude and latitude data, altitude data, temperature, wind direction and humidity, and a plurality of neural network modules are used for training and learning so as to predict future high-resolution radar echo pictures more accurately; the loss function used by the multi-modal radar echo prediction model is:
wherein,
radar map data, y, representing model predictionskRepresenting true radar map data, ωkAnd mukRepresenting the weighting values for radar maps of different strengths.
S3012, evaluating the multi-mode radar echo prediction model by utilizing an evaluation function, wherein the evaluation function is as follows:
the Value evaluation result Value is between-1 and 1, the effect is better when the Value evaluation result Value is larger, wherein M is the predicted radar chart sequence length, and Value isiScoring for a single radar chart, i.e.:
where L is the predicted class, N is the total number of samples, ωjIs the weight of the jth class, p (R)i，Tj) The total number of pixels with the real category j in the pixels with the prediction category i, p (R)i) Representing the total number of pixels predicted to be of class i, p (T)j) Representing the total number of pixel points with the real category of j;
in the embodiment of the invention, the sequence data processed in the data container is segmented according to requirements, namely, the radar sequence data of 3 hours in the future is predicted according to the meteorological data of the previous 3 hours, and the data is divided into a training set, a verification set and a test set according to the proportion of 7:1: 2. Firstly, inputting multimodality meteorological big data of the previous 3 hours with the time interval of 6 minutes, namely-180 min, -174min, … and 0min, wherein the multimodality meteorological big data comprises radar data, satellite data, temperature, wind direction, longitude and latitude and altitude, and inputting the data into each module. The Data Integration module mainly adopts a Data fusion technology in deep learning to fuse radar Data, satellite cloud pictures, longitude and latitude Data, altitude Data, temperature, wind direction and humidity. Firstly, in order to prevent the subsequent gradient explosion or gradient disappearance of the model, normalizing the dimensional data by using a hyperbolic tangent function:
then, data fusion is performed by using techniques such as Space-to-depth, Mean-firing, Center-crop, Conv (convolution), and Max-firing. The CNN + LSTM module is an encoder-decoder framework on the whole, time and space are mainly fused, convolution calculation is introduced into calculation of the LSTM in the network, and space-time correlation of meteorological data is extracted. If the sample size is sufficient for more sufficient feature extraction, the CNN + LSTM module may superpose several layers of networks, but since the samples we obtain are limited, the number of layers M is set to 1 to prevent over-fitting of the samples. The DS + LSTM module mainly uses a down-sampling technology to fully extract deep features of historical meteorological data of all dimensions. The US + LSTM module mainly uses an upsampling technology to predict future radar sequence data characteristics by using the extracted atmospheric motion characteristics. The CNN + LSTM module + Data Generation module generates radar sequence Data of an intended area as accurately as possible according to the extracted radar sequence Data characteristics. Finally, a radar sequence chart of the future 3 hours is output through a Data Generation module, the time interval is 30min, and half an hour is predicted once.
In the embodiment of the present invention, the loss function used by the whole model is:
wherein,
radar map data, y, representing model predictionskRepresenting true radar map data, ωkAnd mukRepresenting the weighting values for radar maps of different strengths. And training the improved model on a gpu cluster by utilizing the multi-modal meteorological big data of the previous 3 hours until the model is optimal. The optimal model is evaluated using the following evaluation function:
the Value evaluation results ranged between-1 and 1, with larger values being more effective.
As shown in fig. 4, the invention provides an artificial intelligence conversion model, which can regard a radar map and a precipitation map as two image domains, comprises four modules of an R2P generator, a P2R generator, a P discriminator and an R discriminator, combines geographical position data such as longitude and latitude, altitude and the like, and trains and optimizes the model on a gpu cluster by using historical radar data and precipitation data to obtain the model capable of intelligently converting radar and precipitation.
In the embodiment of the invention, firstly, radar data, geographic position big data (including longitude and latitude data, altitude data and the like) and precipitation data are acquired and set as { (X)1，X1，Y1)，(X2，Z2，Y2)，...，(Xn，Zn，Yn) }) the data was divided into a training set, a validation set, and a test set at a 7:1:2 ratio. Then, a deep generation network is constructed, which can be regarded as a whole consisting of two countermeasure networks, including four modules: R2P generator: an encoder-decoder framework is mainly adopted, real radar data are utilized, geographical position data such as longitude, latitude, altitude and the like are combined, characteristics of the radar data are encoded through an encoder of the encoder, and then a decoder is utilized to generate corresponding precipitation data. A P discriminator: mainly for distinguishing between false precipitation data and true precipitation data of the R2P generator. P2R generator: the radar generator is similar to the R2P generator in structure, but the main function of the radar generator is to encode the characteristics of real precipitation data by an encoder in combination with geographical position data such as longitude, latitude, altitude and the like, and then generate corresponding radar data by a decoder. An R discriminator: mainly to distinguish between false radar data or real radar data of the P2R generator. The model first uses real radar data in combination with longitude and latitude altitude data, i.e. (X)i，Zi) 1, 2.. and n, which are input into an R2P generator to generate corresponding precipitation data Yi', i-1, 2. Then, the P discriminator is used to compare the generated Yi' and true YiAnd (6) judging. Similarly, real precipitation data is utilized and longitude and latitude altitude data is combined to obtain (Y)i，Zi) I 1, 2.. and n, which are input to a P2R generator to generate corresponding radar data Xi', i-1, 2. Then, the R discriminator is used to compare the generated XiAnd true XiAnd (6) judging.
In the embodiment of the present invention, the loss function used by the whole model is:
Loss＝Loss1+Loss2+Loss3
wherein:
and (3) performing countermeasure training optimization on the gpu cluster, finally obtaining a group of optimal parameters by the R2P generator and the P2R generator, and flexibly converting radar data and precipitation data by combining longitude and latitude altitudes. The obtained R2P generator is an adjacent precipitation conversion model and can convert radar data into precipitation data.
In the embodiment of the invention, based on the constructed rainfall prediction knowledge map, a deep learning algorithm is utilized to carry out knowledge reasoning, and the obtained rainfall data is intelligently corrected. By using the data and the relation acquired from the knowledge map obtained in the S200, the maximum value and the minimum value of the regional precipitation can be predicted through a deep learning prediction model according to meteorological factors such as longitude and latitude, altitude, time, temperature, wind direction, humidity and the like. And then intelligently correcting the precipitation data obtained in the step S32 according to the precipitation threshold predicted by the algorithm, and further improving the accuracy and reliability of precipitation prediction.
In the embodiment of the invention, the precipitation data visualization is to more intuitively observe, classify the predicted and corrected precipitation into 8 types, namely 0, (0, 2.5), (2.5, 5), (5, 10), (10, 25), (25, 50), (50,100, (+ ∞) (unit: mm/h) and then display the precipitation data on a map based on longitude and latitude data for observation and analysis.
As shown in fig. 5, an embodiment of the present invention further discloses a precipitation prediction system based on an artificial intelligence algorithm and a knowledge graph, where the system includes:
the multi-model data container is used for inputting different types of meteorological data into the multi-model data container according to the structural characteristics of the multi-model data container to obtain multi-model data, and performing space-time alignment, data cleaning and preprocessing and data sequence segmentation on the multi-model data;
and the rainfall prediction knowledge map building module is used for building a knowledge map about rainfall prediction.
The multi-modal precipitation prediction and correction module based on the artificial intelligence algorithm is used for creating a multi-modal precipitation prediction model based on the artificial intelligence algorithm, and specifically comprises the following steps: the radar echo prediction model construction unit is used for constructing a multi-mode radar echo prediction model, performing space-time fusion on various historical meteorological data, and predicting a future radar sequence diagram by using a deep neural network model; the intelligent conversion model building unit is used for building a radar precipitation intelligent conversion model, combining the geographical position data and utilizing a countermeasure generation network to carry out mutual conversion on radar data and precipitation, and then converting the predicted radar sequence diagram into precipitation data; the intelligent correction unit is used for carrying out knowledge reasoning by utilizing a neural network algorithm based on the constructed rainfall prediction knowledge map and carrying out intelligent correction on the predicted rainfall data; and
and the regional precipitation visualization module is used for classifying and visualizing the corrected precipitation data.
In the using process of the system, real-time data of dimensions of a radar, a satellite, a meteorological station and the like of a certain area A are acquired firstly, then the area is divided into n small areas of 500 x 500km, then an intelligent rainfall prediction system is used for predicting future rainfall of the n small areas through modules, and finally the small areas are combined and visualized, so that the future rainfall condition can be observed and analyzed.
The present invention has been described in detail with reference to the preferred embodiments thereof, and it should be understood that the invention is not limited thereto, but is intended to cover modifications, equivalents, and improvements within the spirit and scope of the present invention.
It should be understood that, although the steps in the flowcharts of the embodiments of the present invention are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in various embodiments may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
Claims (6)
1. A precipitation prediction method based on an artificial intelligence algorithm and a knowledge graph is characterized by comprising the following steps:
constructing a multi-mode data container, inputting different types of meteorological data into the multi-mode data container according to the structural characteristics of the multi-mode data container to obtain multi-mode data, and performing space-time alignment, data cleaning and preprocessing and data sequence segmentation on the multi-mode data;
constructing a knowledge graph about precipitation prediction;
establishing a multi-modal precipitation prediction model based on an artificial intelligence algorithm, specifically: constructing a multi-modal radar echo prediction model, performing space-time fusion on various historical meteorological data, and predicting a future radar sequence diagram by using a deep neural network model; constructing a radar precipitation intelligent conversion model, combining geographical position data, utilizing a confrontation generation network to carry out mutual conversion on radar data and precipitation, and converting a predicted radar sequence diagram into precipitation data; based on the constructed rainfall prediction knowledge graph, carrying out knowledge reasoning by using a neural network algorithm, and intelligently correcting predicted rainfall data;
and carrying out classification visualization on the corrected precipitation data.
2. The precipitation prediction method based on artificial intelligence algorithm and knowledge graph according to claim 1, wherein the step of performing space-time alignment on the multi-modal data specifically comprises:
merging and aligning historical radar echo sequence data with a time error of less than 2 minutes with satellite cloud map data; carrying out bilinear interpolation on data of a meteorological station, fusing data obtained after processing the bilinear interpolation with radar data and satellite data, and carrying out space matching and alignment on the radar data, the satellite data, the data of the meteorological station and altitude data according to longitude and latitude;
and performing space-time alignment on the radar sequence data and the geographical position big data and the precipitation data, wherein the geographical position big data comprises longitude and latitude data and altitude data.
3. The method for predicting precipitation according to claim 1, wherein the step of constructing the knowledge graph about precipitation prediction specifically comprises:
acquiring data related to precipitation, including unstructured precipitation data and semi-structured precipitation data obtained from open websites and research, and structured precipitation data obtained from radar and satellites;
aiming at unstructured precipitation data and semi-structured precipitation data, extracting entities, relations and attributes of precipitation prediction by using a convolutional neural network based on an attention mechanism to obtain extracted precipitation information, wherein the entities comprise longitude and latitude, altitude, time, temperature, wind direction, humidity, precipitation and the like;
fusing the structured precipitation data and the extracted precipitation information, wherein the fusion process comprises precipitation entity disambiguation and alignment;
further processing the extracted precipitation information, and analyzing the relation between longitude and latitude, altitude, time, temperature, wind direction, humidity and precipitation by using a machine learning correlation analysis algorithm, namely giving specific values of meteorological factors to obtain the maximum precipitation and the minimum precipitation of the area;
and constructing a precipitation prediction knowledge graph according to the extracted precipitation information, and storing the precipitation prediction knowledge graph into a graph database.
4. The precipitation prediction method based on the artificial intelligence algorithm and the knowledge graph as claimed in claim 1, wherein the step of constructing the multi-modal radar echo prediction model specifically comprises:
creating a multi-modal radar echo prediction model consisting of a Data Integration module, a CNN + LSTM module, a DS + LSTM module, an US + LSTM module, a CNN + LSTM module and a Data Generation module, wherein the multi-modal radar echo prediction model utilizes meteorological Data of a 512km area and radar images of a prediction center area 256km for better predicting precipitation of the center area; the multi-modal radar echo prediction model is used for fusing historical multi-modal meteorological big data, the historical multi-modal meteorological big data comprises radar data, satellite cloud pictures, longitude and latitude data, altitude data, temperature, wind direction and humidity, and a plurality of neural network modules are used for training and learning so as to accurately predict future high-resolution radar echo pictures; the loss function used by the multi-modal radar echo prediction model is:
wherein,
radar map data, y, representing model predictionskRepresenting true radar map data, ωkAnd mukRepresenting a weighting value for radar maps of different intensities;
and evaluating the multi-modal radar echo prediction model by utilizing an evaluation function, wherein the evaluation function is as follows:
the Value evaluation result Value is between-1 and 1, the effect is better when the Value evaluation result Value is larger, wherein M is the predicted radar chart sequence length, and Value isiScoring for a single radar chart, i.e.:
where L is the predicted class, N is the total number of samples, ωjIs the weight of the jth class, p (R)i，Tj) The total number of pixels with the real category j in the pixels with the prediction category i, p (R)i) Representing the total number of pixels predicted to be of class i, p (T)j) Representing the total number of pixels of the real category j.
5. The precipitation prediction method based on the artificial intelligence algorithm and the knowledge graph as claimed in claim 4, wherein the step of constructing the radar precipitation intelligent conversion model specifically comprises: constructing a conversion model comprising an R2P generator, a P2R generator, a P discriminator and an R discriminator, and training and optimizing the conversion model on a gpu cluster by combining longitude and latitude and altitude geographical position data and utilizing historical radar data and precipitation data to obtain an intelligent radar precipitation conversion model; the loss function used by the radar precipitation intelligent conversion model is as follows:
Loss＝＝Loss1+Loss2+Loss3
wherein:
6. a precipitation prediction system based on artificial intelligence algorithms and knowledge maps, the system comprising:
the multi-model data container is used for inputting different types of meteorological data into the multi-model data container according to the structural characteristics of the multi-model data container to obtain multi-model data, and performing space-time alignment, data cleaning and preprocessing and data sequence segmentation on the multi-model data;
the rainfall prediction knowledge map building module is used for building a knowledge map about rainfall prediction;
the multi-modal precipitation prediction and correction module based on the artificial intelligence algorithm is used for creating a multi-modal precipitation prediction model based on the artificial intelligence algorithm, and specifically comprises the following steps: the radar echo prediction model construction unit is used for constructing a multi-mode radar echo prediction model, performing space-time fusion on various historical meteorological data, and predicting a future radar sequence diagram by using a deep neural network model; the intelligent conversion model building unit is used for building a radar precipitation intelligent conversion model, combining the geographical position data and utilizing a countermeasure generation network to carry out mutual conversion on radar data and precipitation, and then converting the predicted radar sequence diagram into precipitation data; the intelligent correction unit is used for carrying out knowledge reasoning by utilizing a neural network algorithm based on the constructed rainfall prediction knowledge map and carrying out intelligent correction on the predicted rainfall data; and
and the regional precipitation visualization module is used for classifying and visualizing the corrected precipitation data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111625218.2A CN114385611A (en) | 2021-12-28 | 2021-12-28 | Precipitation prediction method and system based on artificial intelligence algorithm and knowledge graph |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111625218.2A CN114385611A (en) | 2021-12-28 | 2021-12-28 | Precipitation prediction method and system based on artificial intelligence algorithm and knowledge graph |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114385611A true CN114385611A (en) | 2022-04-22 |
Family
ID=81197843
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111625218.2A Pending CN114385611A (en) | 2021-12-28 | 2021-12-28 | Precipitation prediction method and system based on artificial intelligence algorithm and knowledge graph |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114385611A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114610799A (en) * | 2022-05-11 | 2022-06-10 | 未名环境分子诊断（常熟）有限公司 | Data processing method and device based on environment monitoring and storage medium |
| CN115877345A (en) * | 2023-02-28 | 2023-03-31 | 航天宏图信息技术股份有限公司 | Method and device for supplementing missing data of wind profile radar |
-
2021
- 2021-12-28 CN CN202111625218.2A patent/CN114385611A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114610799A (en) * | 2022-05-11 | 2022-06-10 | 未名环境分子诊断（常熟）有限公司 | Data processing method and device based on environment monitoring and storage medium |
| CN114610799B (en) * | 2022-05-11 | 2022-07-22 | 未名环境分子诊断（常熟）有限公司 | Data processing method and device based on environmental monitoring and storage medium |
| CN115877345A (en) * | 2023-02-28 | 2023-03-31 | 航天宏图信息技术股份有限公司 | Method and device for supplementing missing data of wind profile radar |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ali et al. | Exploiting dynamic spatio-temporal graph convolutional neural networks for citywide traffic flows prediction | |
| CN107529651B (en) | Urban traffic passenger flow prediction method and equipment based on deep learning | |
| Qin et al. | A novel combined prediction scheme based on CNN and LSTM for urban PM 2.5 concentration | |
| Ali et al. | Leveraging spatio-temporal patterns for predicting citywide traffic crowd flows using deep hybrid neural networks | |
| CN114385611A (en) | Precipitation prediction method and system based on artificial intelligence algorithm and knowledge graph | |
| Qi et al. | A deep learning approach for long-term traffic flow prediction with multifactor fusion using spatiotemporal graph convolutional network | |
| Jonnalagadda et al. | Forecasting atmospheric visibility using auto regressive recurrent neural network | |
| CN117556197B (en) | Typhoon vortex initialization method based on artificial intelligence | |
| CN109214716A (en) | Mountain fire risk profile modeling method based on stacking algorithm | |
| CN117494034A (en) | Air quality prediction method based on traffic congestion index and multi-source data fusion | |
| CN113496310A (en) | Atmospheric pollutant prediction method and system based on deep learning model | |
| CN116663742B (en) | Regional capacity prediction method based on multi-factor and model fusion | |
| Ganji et al. | Traffic volume prediction using aerial imagery and sparse data from road counts | |
| CN116662468A (en) | Urban functional area identification method and system based on geographic object space mode characteristics | |
| CN117670527B (en) | Method and system for determining peasant household loan credit limit based on land parcel data | |
| CN117575873B (en) | Flood warning method and system for comprehensive meteorological hydrologic sensitivity | |
| CN117233869B (en) | Site short-term wind speed prediction method based on GRU-BiTCN | |
| CN109829583B (en) | Mountain fire risk prediction method based on probability programming technology | |
| CN114118511B (en) | Large-area multi-star combined coverage effectiveness evaluation method based on cloud cover prediction information | |
| Porto et al. | Machine learning approaches to extreme weather events forecast in urban areas: Challenges and initial results | |
| CN117706660A (en) | Strong convection weather forecast method based on CNN-ViT technology | |
| Zhang | Remote sensing data processing of urban land using based on artificial neural network | |
| Kaparakis et al. | WF-UNet: Weather data fusion using 3d-unet for precipitation nowcasting | |
| Ning | Prediction and detection of urban trajectory using data mining and deep neural network | |
| Li et al. | WSPTGAN for Global Ocean Surface Wind Speed Generation with High Temporal Resolution and Spatial Coverage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |