CN110019595B - Multi-source meteorological data integration method and system - Google Patents
Multi-source meteorological data integration method and system Download PDFInfo
- Publication number
- CN110019595B CN110019595B CN201710903730.6A CN201710903730A CN110019595B CN 110019595 B CN110019595 B CN 110019595B CN 201710903730 A CN201710903730 A CN 201710903730A CN 110019595 B CN110019595 B CN 110019595B
- Authority
- CN
- China
- Prior art keywords
- data
- meteorological
- weather
- format
- source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Radar Systems Or Details Thereof (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application relates to a multisource meteorological data integration method and system, comprising the following steps: collecting weather meteorological data and electric power meteorological data to form original data; unifying the original data format into a preset standard format; and integrating all the meteorological data unified into a preset standard format to form a meteorological monitoring data set. The scheme is used for improving the accuracy of numerical weather forecast and realizing the establishment and application of customized numerical modes in the power industry.
Description
Technical Field
The application belongs to the field of new energy power generation, and particularly relates to a multi-source meteorological data integration method and system.
Background
At present, all grid-connected new energy stations and transmission lines are built with special weather monitoring equipment along the way, and the weather observation data which can serve the operation of a power grid are combined with radars, satellites and conventional weather observation networks of a weather department to show typical characteristics of 'various sources and huge body volumes'. However, due to different formats and structures, unified integration application has not been developed effectively, and integration degree of meteorological data is improved effectively through research application of a multi-source meteorological observation data integration technology, and new energy resource forecasting and simulation evaluation are supported comprehensively.
Meanwhile, a certain result is achieved in the aspect of multisource meteorological observation data integration, and a relatively mature multisource observation data integration system is established. The world meteorological organization WMO (World Weather Organization) integrates meteorological observation data of various sources of members in the world approaching two hundred countries and regions, integrates the observation data through methods such as quality control, space analysis and the like and shares the observation data with member units; the system is characterized in that a national weather data center is established in the United states, global and United states weather data are collected by using a weather resource transmission network in the United states, the data are identified, quality control, integration, archiving and sharing processing are carried out, the MICAPS comprehensive weather data retrieval system is developed by the China weather bureau, quality control technologies and error analysis correction methods of various detection systems and different types of data are researched, and high-quality data and analysis products are obtained.
With the large-scale access of clean energy represented by wind power and photovoltaic to a power grid, and the acceleration of the construction of an extra-high voltage power grid and a smart power grid, the relationship between meteorological conditions and the operation of the power grid is becoming more and more intimate. From the perspective of resource space distribution, satellite data cannot meet the continuous requirement of new energy power generation on weather data time sequences such as wind, irradiance and the like; from the point of generating meteorological elements from the new energy station, the meteorological elements monitored by the new energy station are closely related to the height of the power generation device, mainly, wind speeds, irradiance, humidity, temperature and the like at different heights, and are more customized than conventional meteorological data. In addition, the data recorded by the monitoring device needs to be analyzed by using professional software and programs, and the intermediate processing process is more complex; from the analysis of a power transmission channel, a power transmission line of a power grid is generally erected in mountain areas, gobi desert and the like with few people and complicated terrains, more weather departments are facing public service, and deployed observation sites are mainly concentrated in non-power transmission channel areas with dense population and frequent activities. The monitoring devices such as microclimate, icing and the like arranged along the transmission line can more intensively and intensively monitor the meteorological conditions around the transmission line while compensating for the conventional meteorological data loss of the transmission channel. In general, the power industry has larger meteorological data volume, denser time interval and more complex format, more means and methods are needed for processing, the data quality is uneven, the calibration is needed to be checked and corrected from different space-time dimensions, and the integration and fusion difficulty is higher.
The integration of multi-source meteorological data, although having a certain basis, still has the following problems: firstly, the data integration is affected by local climate, and has a certain regional property, so that a meteorological data integration system cannot access electric power meteorological observation data such as a new energy station and the like. Secondly, the meteorological department observes the electric power meteorological data source diversification such as data and new energy stations, and the structure is very different, and the research of multisource data fusion technology is to be further developed.
Disclosure of Invention
In order to solve the problems, the application provides a multisource weather data integration method and a multisource weather data integration system, which effectively improve the accuracy of numerical weather forecast and realize the establishment and application of customized numerical modes in the power industry.
The application aims at adopting the following technical scheme:
a method of multi-source weather data integration, the method comprising:
collecting weather meteorological data and electric power meteorological data to form original data;
unifying the original data format into a preset standard format;
and integrating all the meteorological data unified into a preset standard format to form a meteorological monitoring data set.
Preferably, the unifying the original data format to a preset standard format further includes:
storing all meteorological data unified into a preset standard format into a multi-source meteorological database;
the integrating all the meteorological data unified into a preset standard format to form a meteorological monitoring data set comprises the following steps:
and integrating all the weather data stored in a unified way to form a weather monitoring data set.
Preferably, the unifying the original data format to a preset standard format includes: converting the data into a standard format through a characteristic element extraction method and a data inversion method; the standard format is ASCII text format.
Preferably, the weather meteorological data includes: satellite weather data, weather station weather data; the power weather data includes: new energy station observation data, radar data and transmission line data.
Further, the power transmission line data comprise monitoring time, line names, tower numbers, longitude and latitude information and meteorological element observation values;
the new energy station observation data comprises wind power plant numbers, wind tower numbers, hub heights, longitude and latitude information and meteorological element observation values;
the radar data comprises site numbers, names, acquisition time, radar wavelength, echo shapes, body scanning layers and longitude and latitude information;
the satellite weather data includes acquisition time, wave band, frequency spectrum range, resolution and meteorological element inversion matrix data.
Further, the integrating all the weather data stored in the unified way to form a weather monitoring data set includes:
firstly judging whether abnormal data exist in a multi-source weather database, and if so, eliminating the abnormal data;
and then adopting an interpolation analysis method, a buffer analysis method and a superposition analysis method to generate a meteorological monitoring data set for the data set after the abnormality is removed.
Further, the exception processing of the meteorological data in the multi-source meteorological database includes:
judging whether the meteorological data in the multi-source meteorological database has abnormal data or not, and if so, eliminating the abnormal data;
the judging whether the meteorological data in the multi-source meteorological database has abnormal data or not comprises the following steps:
performing a logical consistency check, comprising: defining meteorological data below a preset threshold as abnormal data;
performing time continuity test, including judging the change of regularity of the weather data to be predicted in a continuous time range; if the data change is obvious or no change in unit time, defining the data at the current moment as suspicious data, and carrying out consistency comparison with historical abnormal data, and if the suspicious data is consistent with the historical abnormal data, determining the suspicious data as abnormal data;
performing a spatial consistency check, comprising: dividing areas with different radiuses by taking a target position as a circle center based on space geographic information and combining a four-azimuth dividing method, checking weather data relativity of four azimuth of east, west, south and north based on the space geographic information, and judging weather data as abnormal data if fluctuation of the weather data is obvious; the space geographic information comprises underlying data information and transmission line tower coordinate information;
performing integrity and correctness checks, including: checking the integrity and correctness of the meteorological data, and defining the meteorological data which does not contain all the attributes as abnormal data; the attributes of the meteorological data comprise file naming, data files, data layering, element expression, data formats, data organization, data storage media and original data;
an associative matching check based on data mining, comprising: generating a frequent item set by using an Apriori algorithm, acquiring an association rule with the confidence coefficient of more than 0.6 generated by the frequent item set, associating the changes of different meteorological elements, and establishing a meteorological data rule base; and matching the meteorological elements at each moment in the multi-source meteorological database with each rule in the meteorological data rule base according to the association rule, checking whether the changes of different meteorological elements are matched with the association rule every moment by moment, and defining the data which cannot be matched as abnormal data.
Further, the generating the frequent item set using the Apriori algorithm includes:
selecting meteorological elements from a multi-source meteorological database according to actual working conditions, and acquiring a ground observation station database which contains monitoring station numbers, names, time, longitude and latitude information, altitude and meteorological element observation values according to instrument specifications and derived data formats;
discretizing the observed values of the meteorological elements, and classifying the observed values according to the categories and grades of the meteorological elements;
and according to a preset support degree threshold value and a preset confidence degree threshold value, calculating and obtaining a frequent item set formed by association rules of meteorological element observation values within the allowable range of the support degree and the confidence degree threshold value by adopting an Apriori algorithm.
Preferably, the collecting weather meteorological data includes:
weather data of a weather station are obtained through a weather department;
satellite weather data is received by a network download or satellite receiving station.
Preferably, the collecting the power weather data includes:
radar data are obtained through a meteorological department;
acquiring new energy station observation data in RWD format by using a meteorological device installed in the new energy station;
and acquiring the data of the transmission line covering the transmission channel in the terrain complex region by using a microclimate on-line monitoring device arranged on the transmission line.
Preferably, a multisource meteorological data integration system facing power industry is characterized by comprising:
the acquisition module is used for acquiring weather meteorological data and electric power meteorological data to form original data;
the storage module is used for unifying the original data format into a preset standard format;
and the integration module is used for integrating all the weather data unified into a preset standard format to form a weather monitoring data set.
Compared with the closest prior art, the application has the following beneficial effects:
the application provides a multisource meteorological data integration method and a multisource meteorological data integration system, which are mainly aimed at integration and application of (micro) meteorological data and traditional meteorological data of a power grid such as a new energy station and a transmission line channel in the power industry, and acquire the weather meteorological data and the power meteorological data to form original data; unifying the original data format into a preset standard format; and integrating all the meteorological data unified into a preset standard format to form a meteorological monitoring data set. The method can integrate, format and integrate traditional meteorological data with meteorological data of the power industry. The availability and the accuracy of the data are greatly improved; the fusion of cross-domain multi-source data is realized, resources are fully integrated, and the data availability is improved; the method combines the geographical information of the power transmission line with the meteorological data, performs quality control on the meteorological data of different sources by combining the dimensions of time, space, logic and the like with the geographical positions of the new energy station and the power transmission line, perfects missing data, establishes different source meteorological databases and greatly reduces errors of the meteorological data caused by the geographical information; in the process of deleting the abnormal data, the data mining idea is applied to relevance matching of the meteorological data, so that the meteorological data is analyzed more comprehensively and deeply, and the quality of the multi-source meteorological data is improved. In the long term, the numerical forecasting mode forecasting precision can be improved, and further the disaster prevention and reduction capacity of the power grid is improved.
Drawings
FIG. 1 is a general flow chart of a multi-source weather data integration method provided in an embodiment of the present application;
FIG. 2 is a block diagram of a multi-source weather data integration method provided in an embodiment of the present application;
FIG. 3 is a flow chart of a method of constructing a multisource weather database provided in an embodiment of the present application.
Detailed Description
The following describes the embodiments of the present application in further detail with reference to the drawings.
The method is based on new energy station multisource meteorological observation data, analyzes distribution characteristics, time sequence and space association characteristics of the meteorological observation data, combines new energy station regional meteorological element characteristic analysis under different geographic and climatic characteristic conditions, evaluates data applicability, accuracy, reliability, continuity, integrity, availability, objectivity and the like by utilizing fuzzy recognition, bayesian discrimination theory and the like, and performs redundancy deletion, bad data restoration and the like on the data, so that the data has higher credibility. The method is characterized in that the method comprises the steps of respectively applying image processing, pattern recognition technology, heterogeneous text data integration, text preprocessing and the like to unstructured and semi-structured meteorological data to achieve information extraction of data of different structure types. And through the structured processing of the data, the space-time positioning and matching of the data, the mutual verification of different meteorological observation data is carried out, and the integration of multi-source heterogeneous meteorological observation data is realized by adopting a meteorological data fusion algorithm.
According to the multi-source meteorological data integration method, traditional meteorological data (conventional meteorological stations, satellite data and the like) and electric power industry meteorological data (wind measurement data, photometry data and transmission line microclimate data) are obtained.
The format, continuity, etc. of the weather data are identified, and the raw data are converted into ASCII text data.
And performing quality control on meteorological data of different sources from the dimensions of time, space, logic and the like in combination with the geographical positions of new energy stations and transmission lines, removing abnormal data, perfecting missing data and establishing different source meteorological databases.
And carrying out integrated processing according to the spatial interpolation, the buffer area analysis and the superposition analysis to generate a high space-time resolution meteorological monitoring database, so as to realize the integration of multi-source meteorological data.
As shown in fig. 1 and 2, the method specifically includes the following steps:
s1, acquiring weather meteorological data and electric power meteorological data to form original data;
weather meteorological data includes: satellite weather data, weather station weather data;
wherein, gather weather meteorological data includes:
weather data of a weather station are obtained through a weather department;
satellite weather data is received by a network download or satellite receiving station.
The power weather data includes: new energy station observation data, radar data and transmission line data.
Collecting power weather data includes:
radar data are obtained through a meteorological department;
acquiring new energy station observation data in RWD format by using a meteorological device installed in the new energy station;
and acquiring the data of the transmission line covering the transmission channel in the terrain complex region by using a microclimate on-line monitoring device arranged on the transmission line.
The power transmission line data comprises monitoring time, line name, tower number, longitude and latitude information and meteorological element observation values;
the new energy station observation data comprises wind power plant numbers, wind tower numbers, hub heights, longitude and latitude information and meteorological element observation values;
radar data including site number, name, acquisition time, radar wavelength, echo shape, number of layers of body scanning and longitude and latitude information;
satellite weather data, including acquisition time, band, spectral range, resolution, and meteorological element inversion matrix data.
S2, unifying the original data format into a preset standard format;
identifying whether the data format in the original data is a preset standard format or not; if yes, directly storing the data into a multi-source weather database; if not, storing all the meteorological data unified into a preset standard format into a multi-source meteorological database:
converting the data into a standard format through a characteristic element extraction method and a data inversion method; structuring unstructured meteorological data is a precondition for implementing the method, and a specific logic diagram is shown in fig. 3. The standard format is ASCII text format.
The design aims to change all format data into ASCII text format data recognizable by a computer to form a multi-source weather database:
(1) and carrying out format recognition on different data in the multi-source weather database, wherein the default data format which can be put in storage is ASCII format. If the data format is ASCII text format data, the data can be directly imported into a database for quality control, otherwise, a processing method (feature element extraction, data inversion and the like) is required to be selected for preprocessing.
(2) The data are converted into ASCII text data through preprocessing, and the ASCII text data are respectively imported into a new energy station database, a satellite database, a radar database, a transmission line database and the like according to different sources to form a uniformly formatted multi-source weather database.
(3) Because the time resolutions of the data from different sources are different, automatic warehousing is realized according to the generation frequency of the multi-source meteorological data, the preprocessing program carries out parallel calculation, and the processing efficiency of the data is improved.
And S3, integrating all the meteorological data unified into a preset standard format to form a meteorological monitoring data set.
Firstly judging whether abnormal data exist in a multi-source weather database, and if so, eliminating the abnormal data;
and then adopting an interpolation analysis method, a buffer analysis method and a superposition analysis method to generate a meteorological monitoring data set for the data set after the abnormality is removed.
Performing exception handling on meteorological data in a multi-source meteorological database comprises:
judging whether the meteorological data in the multi-source meteorological database has abnormal data or not, and if so, eliminating the abnormal data;
the method for judging whether the meteorological data in the multi-source meteorological database has abnormal data comprises the following steps:
performing a logical consistency check, comprising: defining meteorological data below a preset threshold as abnormal data; such as dew point temperature < air temperature, etc.
Performing time continuity test, including judging the change of regularity of the weather data to be predicted in a continuous time range; if the data change is obvious or no change in unit time, the change is obvious data with the data change amount more than or equal to 0.5, defining the data at the current moment as suspicious data, carrying out consistency comparison with historical abnormal data, and if the suspicious data is consistent with the historical abnormal data, determining the suspicious data as abnormal data;
performing a spatial consistency check, comprising: dividing areas with different radiuses by taking a target position as a circle center based on space geographic information and combining a four-azimuth dividing method, checking weather data relativity of four azimuth of east, west, south and north based on the space geographic information, and judging weather data as abnormal data if fluctuation of the weather data is obvious; the space geographic information comprises underlying data information and transmission line tower coordinate information;
performing integrity and correctness checks, including: checking the integrity and correctness of the meteorological data, and defining the meteorological data which does not contain all the attributes as abnormal data; the attributes of the meteorological data comprise file naming, data files, data layering, element expression, data formats, data organization, data storage media and original data;
an associative matching check based on data mining, comprising: generating a frequent item set by using an Apriori algorithm, acquiring an association rule with the confidence coefficient of more than 0.6 generated by the frequent item set, associating the changes of different meteorological elements, and establishing a meteorological data rule base; and matching the meteorological elements at each moment in the multi-source meteorological database with each rule in the meteorological data rule base according to the association rule, checking whether the changes of different meteorological elements are matched with the association rule every moment by moment, and defining the data which cannot be matched as abnormal data.
Generating the frequent item set using the Apriori algorithm includes:
selecting meteorological elements from a multi-source meteorological database according to actual working conditions, and acquiring a ground observation station database which contains monitoring station numbers, names, time, longitude and latitude information, altitude and meteorological element observation values according to instrument specifications and derived data formats;
discretizing the observed values of the meteorological elements, and classifying the observed values according to the categories and grades of the meteorological elements; meteorological elements are elements that indicate the physical state and physical phenomena of the atmosphere. The main steps are as follows: air temperature, air pressure, wind, humidity, clouds, precipitation and various weather phenomena. For example, the wind is classified into 0 to 17 levels of wind or strong wind and weak wind according to the level and grade of wind.
And according to a preset support degree threshold value and a preset confidence degree threshold value, calculating and obtaining a frequent item set formed by association rules of meteorological element observation values within the allowable range of the support degree and the confidence degree threshold value by adopting an Apriori algorithm.
And integrating and warehousing the data after the abnormality rejection by adopting an interpolation analysis method, a buffer area analysis method and a superposition analysis method.
The integration of the multi-source meteorological data is mainly based on spatial data analysis, and data with uniform structures are formed. Methods used in this patent are interpolation, buffer and stacking.
(1) Interpolation analysis method
The method adopts the inverse distance proportional weight interpolation, spline interpolation, gram Lv Jin interpolation, radial basis function interpolation and point density interpolation to verify and integrate different meteorological data sets. These methods give some assumptions on how to obtain the best estimate, and different results are deduced based on different interpolation methods. Whichever interpolation method is selected, on the premise of fusing different data sets, the more the known points are, the wider the distribution is, and the closer the interpolation result is to the actual situation.
(2) Buffer analysis
The buffer area analysis adopted by the method is to establish a buffer area polygon for a group or a class of elements according to the buffer distance condition, and then to carry out superposition analysis on the layer and the layer needing buffer area analysis to obtain the required result. So in practice the buffer analysis involves a two-step operation, the first step being to build up a buffer layer and the second step being to perform a superposition analysis.
(3) Stacked analysis method
Stacking analysis stacks two or more geographic element layers, enabling new attribute features to be generated, or establishing spatial correspondence between geographic objects. The method integrates weather data with different attributes by adopting a superposition algorithm to form a weather data set with all attributes.
Based on the same inventive concept, the application also provides a multisource meteorological data integration system, which comprises:
the acquisition module is used for acquiring weather meteorological data and electric power meteorological data to form original data;
the storage module is used for unifying the original data format into a preset standard format;
and the integration module is used for integrating all the weather data unified into a preset standard format to form a weather monitoring data set.
Wherein, the integrated module includes: the logic consistency checking unit is used for defining weather data lower than a preset threshold value as abnormal data;
the time continuity testing unit is used for performing time continuity testing and comprises judging the change of regularity of the weather data to be predicted in a continuous time range; if the data change is obvious or no change in unit time, defining the data at the current moment as suspicious data, and carrying out consistency comparison with historical abnormal data, and if the suspicious data is consistent with the historical abnormal data, determining the suspicious data as abnormal data;
the space consistency checking unit is used for dividing areas with different radiuses by taking a target position as a circle center based on space geographic information and combining a four-azimuth dividing method, checking weather data relevance of four azimuth of east, west, south and north based on the space geographic information, and judging weather data as abnormal data if certain weather data fluctuation is obvious; the space geographic information comprises underlying data information and transmission line tower coordinate information;
the integrity and correctness checking unit is used for checking the integrity and correctness of the meteorological data and defining the meteorological data which does not contain all the attributes as abnormal data; the attributes of the meteorological data comprise file naming, data files, data layering, element expression, data formats, data organization, data storage media and original data;
the data mining-based association matching detection unit is used for generating a frequent item set by using an Apriori algorithm, acquiring an association rule with the confidence coefficient of more than 0.6 generated by the frequent item set, associating the changes of different meteorological elements, and establishing a meteorological data rule base; and matching the meteorological elements at each moment in the multi-source meteorological database with each rule in the meteorological data rule base according to the association rule, checking whether the changes of different meteorological elements are matched with the association rule every moment by moment, and defining the data which cannot be matched as abnormal data.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application and not for limiting the scope of protection thereof, and although the present application has been described in detail with reference to the above embodiments, it should be understood by those of ordinary skill in the art that: various alterations, modifications, and equivalents may occur to others skilled in the art upon reading the present disclosure, and are within the scope of the appended claims.
Claims (6)
1. A method of multi-source weather data integration, the method comprising:
collecting weather meteorological data and electric power meteorological data to form original data;
unifying the original data format into a preset standard format;
integrating all the meteorological data unified into a preset standard format to form a meteorological monitoring data set;
the unifying the original data format to a preset standard format further comprises:
storing all meteorological data unified into a preset standard format into a multi-source meteorological database;
the integrating all the meteorological data unified into a preset standard format to form a meteorological monitoring data set comprises the following steps:
integrating all the uniformly stored meteorological data to form a meteorological monitoring data set;
the integrating all the weather data stored in the unified way to form a weather monitoring data set comprises the following steps:
performing exception processing on meteorological data in a multi-source meteorological database;
then adopting an interpolation analysis method, a buffer analysis method and a superposition analysis method to generate a meteorological monitoring data set for the data set after the exception processing;
the exception processing of the meteorological data in the multi-source meteorological database comprises the following steps:
judging whether the meteorological data in the multi-source meteorological database has abnormal data or not, and if so, eliminating the abnormal data;
the judging whether the meteorological data in the multi-source meteorological database has abnormal data or not comprises the following steps:
performing a logical consistency check, comprising: defining meteorological data below a preset threshold as abnormal data;
performing time continuity test, including judging the change of regularity of the weather data to be predicted in a continuous time range; if the data change is obvious or no change in unit time, defining the data at the current moment as suspicious data, and carrying out consistency comparison with historical abnormal data, and if the suspicious data is consistent with the historical abnormal data, determining the suspicious data as abnormal data;
performing a spatial consistency check, comprising: dividing areas with different radiuses by taking a target position as a circle center based on space geographic information and combining a four-azimuth dividing method, checking weather data relativity of four azimuth of east, west, south and north based on the space geographic information, and judging weather data as abnormal data if fluctuation of the weather data is obvious; the space geographic information comprises underlying data information and transmission line tower coordinate information;
performing integrity and correctness checks, including: checking the integrity and correctness of the meteorological data, and defining the meteorological data which does not contain all the attributes as abnormal data; the attributes of the meteorological data comprise file naming, data files, data layering, element expression, data formats, data organization, data storage media and original data;
an associative matching check based on data mining, comprising: generating a frequent item set by using an Apriori algorithm, acquiring an association rule with the confidence coefficient of more than 0.6 generated by the frequent item set, associating the changes of different meteorological elements, and establishing a meteorological data rule base; matching the meteorological elements at each moment in the multi-source meteorological database with each rule in the meteorological data rule base according to the association rule, checking whether the changes of different meteorological elements are matched with the association rule every moment by moment, and defining the data which cannot be matched as abnormal data;
the generating the frequent item set using the Apriori algorithm comprises:
selecting meteorological elements from a multi-source meteorological database according to actual working conditions, and acquiring a ground observation station database which contains monitoring station numbers, names, time, longitude and latitude information, altitude and meteorological element observation values according to instrument specifications and derived data formats;
discretizing the observed values of the meteorological elements, and classifying the observed values according to the categories and grades of the meteorological elements;
and according to a preset support degree threshold value and a preset confidence degree threshold value, calculating and obtaining a frequent item set formed by association rules of meteorological element observation values within the allowable range of the support degree and the confidence degree threshold value by adopting an Apriori algorithm.
2. The method of claim 1, wherein unifying the original data format to a preset standard format comprises: converting the data into a standard format through a characteristic element extraction method and a data inversion method; the standard format is ASCII text format.
3. The method of claim 1, wherein the weather meteorological data comprises: satellite weather data, weather station weather data; the power weather data includes: new energy station observation data, radar data and transmission line data.
4. A method according to claim 3, wherein the transmission line data includes monitoring time, line name, tower number, latitude and longitude information, and meteorological element observations;
the new energy station observation data comprises wind power plant numbers, wind tower numbers, hub heights, longitude and latitude information and meteorological element observation values;
the radar data comprises site numbers, names, acquisition time, radar wavelength, echo shapes, body scanning layers and longitude and latitude information;
the satellite weather data includes acquisition time, wave band, frequency spectrum range, resolution and meteorological element inversion matrix data.
5. The method of claim 3, wherein the collecting weather meteorological data comprises:
weather data of a weather station are obtained through a weather department;
satellite weather data is received by a network download or satellite receiving station.
6. The method of claim 3, wherein the collecting power weather data comprises:
radar data are obtained through a meteorological department;
acquiring new energy station observation data in RWD format by using a meteorological device installed in the new energy station;
and acquiring the data of the transmission line covering the transmission channel in the terrain complex region by using a microclimate on-line monitoring device arranged on the transmission line.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710903730.6A CN110019595B (en) | 2017-09-29 | 2017-09-29 | Multi-source meteorological data integration method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710903730.6A CN110019595B (en) | 2017-09-29 | 2017-09-29 | Multi-source meteorological data integration method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110019595A CN110019595A (en) | 2019-07-16 |
| CN110019595B true CN110019595B (en) | 2023-09-22 |
Family
ID=67186461
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710903730.6A Active CN110019595B (en) | 2017-09-29 | 2017-09-29 | Multi-source meteorological data integration method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110019595B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110781230B (en) * | 2019-09-12 | 2024-04-12 | 腾讯大地通途（北京）科技有限公司 | Data access method, device and equipment |
| CN110826231B (en) * | 2019-11-08 | 2022-07-22 | 南京南瑞继保电气有限公司 | Multi-source data integration method for meteorological site |
| CN111125585A (en) * | 2019-12-24 | 2020-05-08 | 武汉珞珈灵智科技有限公司 | Visualization method and system for Web-end three-dimensional model |
| CN111062633A (en) * | 2019-12-24 | 2020-04-24 | 广东电网有限责任公司 | Power transmission and transformation line and equipment state evaluation system based on multi-source heterogeneous data |
| CN112364029B (en) * | 2021-01-12 | 2021-04-02 | 中国气象局气象探测中心 | Satellite-ground raindrop spectrum parameter matching and evaluating method and system |
| CN112732696B (en) * | 2021-01-21 | 2022-02-08 | 中科三清科技有限公司 | Data cleaning method and device applied to atmospheric environment monitoring and storage medium |
| CN113095690B (en) * | 2021-04-16 | 2023-07-25 | 中国气象局气象探测中心 | Quality rolling evaluation method and device for basic elements of ground observation station |
| CN113986911B (en) * | 2021-12-29 | 2022-04-01 | 中国气象局气象探测中心 | Information acquisition method and system for monitoring racing field operation |
| CN114564629B (en) * | 2022-04-22 | 2022-08-16 | 国家海洋技术中心 | Abnormal data processing method and device, computer equipment and storage medium |
| CN115798177B (en) * | 2023-02-09 | 2023-04-25 | 浙江省气象台 | Multi-source data dynamic threshold approach alarm method and system |
| CN116881241B (en) * | 2023-09-06 | 2023-11-07 | 深圳市银河系科技有限公司 | Safety management method and system applied to meteorological data |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002156467A (en) * | 2000-11-15 | 2002-05-31 | Toshiba Corp | Local weather observation system |
| CN103106321A (en) * | 2011-11-15 | 2013-05-15 | 无锡南理工科技发展有限公司 | Meteorological disaster intelligent sensing method based on Apriori method |
| CN104809665A (en) * | 2015-05-14 | 2015-07-29 | 国家电网公司 | Distribution network early-warning method based on improved Apriori algorithm |
| CN104951992A (en) * | 2014-09-04 | 2015-09-30 | 国网山东省电力公司应急管理中心 | GIS-based power-meteorology longitudinal association method and system |
| CN104951489A (en) * | 2014-09-04 | 2015-09-30 | 国网山东省电力公司应急管理中心 | Meteorological data analyzing and processing method applied to power system |
| CN104951857A (en) * | 2014-11-27 | 2015-09-30 | 国网山东省电力公司应急管理中心 | Power-grid-GIS-based meteorological data invoking and pretreatment method and system |
| CN105095589A (en) * | 2015-08-10 | 2015-11-25 | 贵州电网有限责任公司电力科学研究院 | Drawing method of power network wind zone distribution map in mountainous area |
-
2017
- 2017-09-29 CN CN201710903730.6A patent/CN110019595B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002156467A (en) * | 2000-11-15 | 2002-05-31 | Toshiba Corp | Local weather observation system |
| CN103106321A (en) * | 2011-11-15 | 2013-05-15 | 无锡南理工科技发展有限公司 | Meteorological disaster intelligent sensing method based on Apriori method |
| CN104951992A (en) * | 2014-09-04 | 2015-09-30 | 国网山东省电力公司应急管理中心 | GIS-based power-meteorology longitudinal association method and system |
| CN104951489A (en) * | 2014-09-04 | 2015-09-30 | 国网山东省电力公司应急管理中心 | Meteorological data analyzing and processing method applied to power system |
| CN104951857A (en) * | 2014-11-27 | 2015-09-30 | 国网山东省电力公司应急管理中心 | Power-grid-GIS-based meteorological data invoking and pretreatment method and system |
| CN104809665A (en) * | 2015-05-14 | 2015-07-29 | 国家电网公司 | Distribution network early-warning method based on improved Apriori algorithm |
| CN105095589A (en) * | 2015-08-10 | 2015-11-25 | 贵州电网有限责任公司电力科学研究院 | Drawing method of power network wind zone distribution map in mountainous area |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110019595A (en) | 2019-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110019595B (en) | Multi-source meteorological data integration method and system | |
| US20220275788A1 (en) | Forecasting output power of wind turbine in wind farm | |
| CN105095589B (en) | A kind of mountain area power grid wind area is distributed drawing drawing method | |
| EP2856339B1 (en) | Method and apparatus for forecasting solar radiation and solar power production using synthetic irradiance imaging | |
| Haupt et al. | Big data and machine learning for applied weather forecasts: Forecasting solar power for utility operations | |
| CN107356926A (en) | Difference cloud cluster extrapolation precipitation predicting algorithm based on Hu squares | |
| CN109407177B (en) | Machine learning and conventional meteorological observation-based fog identification system and application method | |
| CN110134907B (en) | Rainfall missing data filling method and system and electronic equipment | |
| CN108632832B (en) | Network coverage analysis method and system | |
| CN114091756B (en) | Township tsunami risk assessment method based on Thiessen polygon | |
| CN111709644B (en) | Wind power plant wind resource calculation method utilizing unit SCADA data | |
| CN104200082A (en) | Typhoon landing prediction method | |
| Kezunovic et al. | Systematic framework for integration of weather data into prediction models for the electric grid outage and asset management applications | |
| CN111678880A (en) | Satellite remote sensing identification method and system for stratospheric ozone invading lower layer in troposphere | |
| CN109583467A (en) | A kind of electric power meteorological data fusion method and system | |
| CN114048944A (en) | Estimation method for people to be evacuated and houses to be damaged under rainstorm induced geological disaster | |
| CN115688032A (en) | Power grid risk early warning method and system under typhoon disaster based on multi-source data fusion | |
| CN112015786B (en) | Extreme weather monitoring and early warning information processing system for outer race field | |
| CN103093044A (en) | Electric transmission line icing galloping distribution diagram surveying and mapping method | |
| CN115100819A (en) | Landslide hazard early warning method and device based on big data analysis and electronic equipment | |
| Miller et al. | A preliminary assessment of using spatiotemporal lightning patterns for a binary classification of thunderstorm mode | |
| US20170102482A1 (en) | Forensic weather system | |
| CN115456463A (en) | Risk grade classification method and system for mountain torrent disaster dangerous area | |
| Torre‐Tojal et al. | Estimation of forest biomass from light detection and ranging data by using machine learning | |
| CN114722909A (en) | Solar flare time sequence classification method based on low-dimensional convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |