CN113742327A - Automatic rainfall station abnormal value screening method based on rain-measuring radar data - Google Patents
Automatic rainfall station abnormal value screening method based on rain-measuring radar data Download PDFInfo
- Publication number
- CN113742327A CN113742327A CN202111026172.2A CN202111026172A CN113742327A CN 113742327 A CN113742327 A CN 113742327A CN 202111026172 A CN202111026172 A CN 202111026172A CN 113742327 A CN113742327 A CN 113742327A
- Authority
- CN
- China
- Prior art keywords
- station
- data
- abnormal
- time sequence
- rainfall
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000012216 screening Methods 0.000 title claims abstract description 41
- 206010010947 Coordination abnormal Diseases 0.000 title claims abstract description 13
- 230000002159 abnormal effect Effects 0.000 claims abstract description 136
- 238000002310 reflectometry Methods 0.000 claims abstract description 64
- 230000002093 peripheral effect Effects 0.000 claims abstract description 52
- 230000008859 change Effects 0.000 claims abstract description 46
- 238000012544 monitoring process Methods 0.000 claims abstract description 23
- 238000005065 mining Methods 0.000 claims abstract description 12
- 238000001556 precipitation Methods 0.000 claims description 63
- 230000001360 synchronised effect Effects 0.000 claims description 13
- 239000006185 dispersion Substances 0.000 claims description 12
- 230000005856 abnormality Effects 0.000 claims description 8
- 238000004458 analytical method Methods 0.000 claims description 8
- 239000013598 vector Substances 0.000 claims description 8
- 230000011218 segmentation Effects 0.000 claims description 7
- 238000011156 evaluation Methods 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims 1
- 230000008569 process Effects 0.000 description 9
- 238000005259 measurement Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 230000000875 corresponding effect Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000005484 gravity Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002265 prevention Effects 0.000 description 2
- 238000010187 selection method Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 241000220479 Acacia Species 0.000 description 1
- IAZDPXIOMUYVGZ-UHFFFAOYSA-N Dimethylsulphoxide Chemical compound CS(C)=O IAZDPXIOMUYVGZ-UHFFFAOYSA-N 0.000 description 1
- 238000006424 Flood reaction Methods 0.000 description 1
- 235000010643 Leucaena leucocephala Nutrition 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000010835 comparative analysis Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000036962 time dependent Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/95—Radar or analogous systems specially adapted for specific applications for meteorological use
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01W—METEOROLOGY
- G01W1/00—Meteorology
- G01W1/14—Rainfall or precipitation gauges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A10/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE at coastal zones; at river basins
- Y02A10/40—Controlling or monitoring, e.g. of flood or hurricane; Forecasting, e.g. risk assessment or mapping
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A50/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE in human health protection, e.g. against extreme weather
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Remote Sensing (AREA)
- Business, Economics & Management (AREA)
- Data Mining & Analysis (AREA)
- Radar, Positioning & Navigation (AREA)
- General Engineering & Computer Science (AREA)
- Tourism & Hospitality (AREA)
- Environmental & Geological Engineering (AREA)
- Life Sciences & Earth Sciences (AREA)
- Quality & Reliability (AREA)
- Strategic Management (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Atmospheric Sciences (AREA)
- Environmental Sciences (AREA)
- Hydrology & Water Resources (AREA)
- Ecology (AREA)
- Radar Systems Or Details Thereof (AREA)
- Fuzzy Systems (AREA)
- Electromagnetism (AREA)
- Educational Administration (AREA)
- Development Economics (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Business, Economics & Management (AREA)
- Biodiversity & Conservation Biology (AREA)
Abstract
The invention discloses an automatic rainfall station abnormal value screening method based on rain-measuring radar data, which is used for judging abnormal values and screening the quality of ground automatic rainfall stations based on national Doppler radar body scanning reflectivity and jigsaw products. The method comprises the steps of monitoring and screening obvious fault points based on long-sequence time sequence station data, obtaining three-dimensional change curved surfaces of the reflectivity of different height layers based on a multi-layer Doppler radar reflectivity product, obtaining singular change values in ground rainfall meter time sequence monitoring data by using a curved surface similarity matching algorithm in combination with a rainfall distribution stereogram generated by an automatic rainfall station, and extracting suspected abnormal values of the automatic rainfall station. Establishing an extreme value judgment set, a peripheral station judgment set and a radar reflectivity judgment set of an abnormal value station, acquiring the similarity of normalized time sequence data of the rainfall station and time sequence data of peripheral stations and radar reflectivity grids based on a time sequence data similarity mining algorithm, automatically judging abnormal values of suspected stations according to a similarity standard, thereby acquiring a large amount of screening results of the abnormal values of the automatic rainfall station, and giving a primary judgment result of the station fault type according to different characteristics of the abnormal station time sequence.
Description
Technical Field
The invention relates to the field of rain-measuring radar data application and ground automatic rainfall station quality control, in particular to a method for automatically screening the rainfall station abnormity based on rain-measuring radar data.
Background
Flood disasters caused by rainstorm are one of the main natural disasters in China, and are difficult problems faced by disaster prevention and reduction in China for a long time. The central content of the protection against rainstorm and flood disasters is accurate rainfall monitoring. Rainfall information is the most active factor influencing the water circulation of the drainage basin, and accurate rainfall information is one of the most main factors in order to realize accurate and efficient early warning and forecasting of mountain torrent disasters.
Precipitation has a large amount of uncertain phenomena and strong space-time heterogeneity, precipitation data is generally obtained through observation data of rainfall stations in the area, at present, 4 thousands of multi-element rainfall automatic observation stations and more than 30 thousands of encrypted torrential flood rainfall monitoring stations are built in China, abundant data are provided for flood control commands, the rainfall automatic observation stations are important facilities for preventing torrential floods and geological disasters and basically cover a torrential flood disaster prevention and control area in China, but due to the fact that about 92% of the regional automatic stations and the torrential flood stations are unattended, the environment is complex, rainfall data faults occur frequently, rainstorm flood risk calculation is wrong due to wrong data, government decision is influenced, and great social and economic losses are caused. The misjudgment is mainly divided into the following seven aspects: (1) under the condition of no precipitation in the live scene, large-range false weak precipitation sites appear; (2) under the condition that precipitation exists in the scene, large-range real precipitation information is filtered; (3) when the whole precipitation in the live area is weak, the real strong precipitation center is filtered; (4) when the precipitation in the live area does not have strong precipitation, false strong precipitation is judged; (5) the daily rainfall accumulation is small; (6) missing data of the ground station; (7) the time-dependent influence of the data of the ground station.
The automatic observation stations are generally built in villages and towns, are densely distributed, and can detect weather systems of small and medium sizes, but the rainfall data of the automatic stations are analyzed abnormally, so that the rainfall data of the automatic stations are difficult to analyze abnormally, the rainfall is high in sudden and uneven in spatial and temporal distribution, and whether the rainfall data are abnormal or not is difficult to find in the process of one-time rainfall. At present, China still mainly adopts a space-time difference and extreme value checking mode of rainfall data in an analysis process, on one hand, automatic discrimination of large-scale mass automatic stations is difficult to realize, on the other hand, the automatic stations are compared with self monitoring data and are not correlated with peripheral rainfall monitoring conditions, rainfall data has correlation in two dimensions of space and time, and can not be analyzed in an isolated mode, so that misjudgment and misjudgment can often occur. Therefore, for a nationwide rainstorm flood risk monitoring and early warning platform, the existing conventional automatic station rainfall abnormality analysis method cannot provide a high-timeliness and accurate station abnormality screening result, and therefore high-precision rainstorm flood risk prediction and early warning in a nationwide range cannot be realized.
Disclosure of Invention
In order to overcome the technical defects, the invention designs an automatic rainfall station abnormity screening method based on the rain-measuring radar data, and abnormal value judgment and quality screening are carried out on the ground automatic rainfall station based on national Doppler radar volume-scanning reflectivity and jigsaw products. The method comprises the steps of monitoring and screening obvious fault points based on long-sequence time sequence station data, obtaining three-dimensional change curved surfaces of the reflectivity of different height layers based on a multi-layer Doppler radar reflectivity product, obtaining singular change values in ground rainfall meter time sequence monitoring data by using a curved surface similarity matching algorithm in combination with a rainfall distribution stereogram generated by an automatic rainfall station, and extracting suspected abnormal values of the automatic rainfall station. Establishing an extreme value judgment set, a peripheral station judgment set and a radar reflectivity judgment set of an abnormal value station, acquiring the similarity of normalized time sequence data of the rainfall station and time sequence data of peripheral stations and radar reflectivity grids based on a time sequence data similarity mining algorithm, automatically judging abnormal values of suspected stations according to a similarity standard, thereby acquiring a large amount of screening results of the abnormal values of the automatic rainfall station, and giving a primary judgment result of the station fault type according to different characteristics of the abnormal station time sequence.
In order to solve the technical problems, the invention adopts the following scheme:
an automatic rainfall station anomaly screening method based on rain-measuring radar data comprises the following steps:
step 1, a Hampel method and a Grabbs criterion are adopted to realize long sequence monitoring of station abnormal data, abnormal years are selected by the Hampel method, the annual accumulated rainfall of the station and the annual accumulated rainfall of adjacent stations are subjected to Grabbs criterion analysis, station data with fewer abnormal years and singular points (less than 20%) are compiled, month, day, hour and minute extreme values are set according to the compiling quality, and an extreme value judgment set of station long sequence in the region is constructed;
step 2, establishing an optimal association relationship between the sites and peripheral sites based on the high-precision topographic data, establishing a synchronous time sequence data judgment set of the peripheral sites, and establishing a radar grid time sequence data judgment set based on national 6min rain measuring radar volume sweep data;
step 3, extracting three-dimensional change curved surfaces of the reflectivity of the rain measuring radar of different height layers in the rain area range in real time, combining a rainfall distribution triangulation network curved surface generated by an automatic rainfall station, acquiring singular points in monitoring data of the ground rainfall meter by using a free curved surface similarity evaluation algorithm based on region segmentation, extracting suspected station abnormal values and acquiring a primary screening result;
step 4, for the abnormal values of the stations, based on sequence values of a radar grid time sequence data judgment set and a peripheral station synchronous time sequence data judgment set, utilizing a time sequence data similarity mining algorithm based on distance to obtain the similarity of singular station normalized time sequence data and peripheral stations and radar reflectivity grid time sequence data, automatically studying and judging suspected station abnormal values, and determining the degree of reasonability of the singular station time sequence data;
and 5, providing an abnormal value screening result of the rainfall station based on an extreme value judgment result, a curved surface similarity screening result and the time sequence data reasonability of the singular station data, extracting abnormal value characteristics, and providing a station fault type judgment result according to different abnormal characteristics of the abnormal station time sequence.
Further, the specific operation in step 1 is as follows: calculating the annual accumulated rainfall of a single station, acquiring a data dispersion degree statistic value in a long sequence observation time sequence through a Hampel identifier, extracting abnormal stations, performing Grabas criterion analysis by comparing annual rainfall total amounts of the abnormal stations and peripheral stations within 50KM, eliminating annual abnormal values until no abnormal value exists in the station sequence values, integrally compiling an abnormal-free data sequence, extracting month, day and hour extreme values of the stations in the region, and constructing a long sequence extreme value judgment set of the stations in the region.
Further, the specific operation in step 2 is as follows: extracting factors such as altitude difference, shelter elevation, station distance, slope included angle and the like of stations within a range of 10km around the stations based on high-precision topographic and geomorphic data of a small watershed, calculating influence weights of the factors based on an optimal weight selection method, automatically matching and establishing an optimal association relationship between the stations and the surrounding stations, and establishing a synchronous time sequence data judgment set of the surrounding stations; and (4) extracting reflectivity data of the radar grid where the station is located every 6min, and constructing a radar grid time sequence data judgment set.
Further, the specific operation in step 3 is as follows: extracting real-time national radar reflectivity jigsaw puzzle, segmenting according to a precipitation range, automatically matching all rainfall sites in a precipitation segment area, obtaining the reflectivity of 6 conventional elevation height layers of the radar, and generating 6 three-dimensional curved surfaces with different height layers by taking the reflectivity as height change; meanwhile, a Delaunay triangulation network is generated based on the station positions in the precipitation zone range, and a real-time station precipitation value is used as a height dimension to generate a real-time station precipitation distribution three-dimensional change curved surface; according to the Gaussian curvature feature classification criterion, a region growing algorithm is adopted to segment the free-form surface, the geometric shape and topological features of the surface are extracted, and 7-dimensional generalized vector descriptors of the sub-region are extracted to form shape description of the free-form surface.
Further, in step 3, the curved surface similarity of the real-time rainfall distribution three-dimensional change curved surface and the 6 radar reflectivity change curved surfaces is sequentially compared, the 7-dimensional vector descriptor of the region is used as a node attribute, matching similarity between the site and the radar reflectivity curved surfaces is calculated by adopting a Kuhn-Munkres method, singular points of ground site real-time monitoring data are obtained according to the similarity coefficient, suspected site abnormal values are extracted, and a primary screening result is obtained.
Further, the specific operation in step 4 is as follows: extracting and normalizing hour precipitation time sequence data of suspected abnormal sites, comparing the hour precipitation time sequence data with a long sequence extreme value judgment set, determining abnormality if the extreme value is exceeded, extracting precipitation time sequence normalized data of the peripheral sites and radar reflectivity time sequence normalized data of the grids where the sites are located in the peripheral site synchronous time sequence data judgment set and the radar grid time sequence data judgment set of the peripheral sites of the same period respectively for the points which do not exceed the extreme value, and acquiring the similarity of the hour precipitation time sequence data of the suspected abnormal sites by using a distance-based time sequence data similarity mining algorithm so as to judge the reasonability of the precipitation change trend of the suspected abnormal sites.
Further, the specific operation in step 5 is as follows: whether an abnormal site exceeds an extreme value range or not is known based on a site extreme value judgment set, the dispersion degree of a singular value is extracted based on a curved surface similarity screening result, the abnormal condition of the site is comprehensively judged based on a time sequence data reasonability comparison result, and the possible fault type and the possible fault time of the abnormal site are judged based on the dispersion degree of the abnormal value, the duration time of the abnormal value, the change degree of the abnormal value, the change trend of the abnormal value and the range characteristic of the abnormal site.
The invention principle is as follows:
in order to obtain the high-efficiency large-range encryption automatic station abnormity screening result, a point + surface fusion analysis method is adopted, grid rain-measuring radar reflectivity data is introduced, and large-range automatic rainfall station abnormity screening is realized by starting from two dimensions of a point-surface change trend and an automatic station-grid time sequence change trend. Although the automatic rainfall station is a single-point measuring device, precipitation data has spatial continuity and time continuity, has relevance in both space-time dimensions, and cannot be analyzed in an isolated mode. In the aspect of space, radar reflectivity data is one-surface data and can be combined with peripheral stations to realize the identification of station space abnormality at a certain moment; in time, since radar reflectivity data is a moving variable, both space and time are changing. The counts of the autonomous stations also vary over time and space, and therefore need to be compared over a long sequence of time periods. Therefore, the method is based on national Doppler radar body-scanning reflectivity and jigsaw products, the abnormal value of the site is judged by respectively adopting a three-dimensional free-form surface similarity evaluation algorithm and a similarity mining algorithm of time sequence data in two dimensions of time and space, and a primary judgment result of the site fault type is given according to different characteristics of the abnormal site time sequence.
The invention has the beneficial effects that:
(1) the invention can realize the automatic anomaly monitoring and screening of the anomaly of the automatic encryption station in a high-efficiency and large-range manner, realize the strong association between the station and radar reflectivity data and peripheral station data, and realize the association of the station anomaly monitoring in two dimensions of space and time. In the spatial dimension, the method not only performs comparative analysis on single-point precipitation data in a single process, but also is combined with radar data and station data, so that observation on abnormal point data and spatial gradient in space is realized, and identification on station spatial value abnormality is realized. In the time dimension, precipitation data are moving variables, and station measurement data also change at any time in space.
(2) The method extracts the abnormal value characteristics of the abnormal value station in the space-time dimension, associates the abnormal conditions possibly caused by the existing station fault types, and automatically judges the possible fault types of the station, so that the method can automatically and efficiently monitor the station abnormality and research and judge the fault in a large range nationwide, and provides help for the operation management of the operation condition of the precipitation station.
The invention is described in further detail below with reference to the accompanying drawings and the detailed description.
Drawings
FIG. 1 is a schematic diagram of an automatic rainfall station anomaly screening process based on rain radar data;
FIG. 2 is a site-peripheral site association;
fig. 3 is a flow of evaluation of similarity of curved surfaces.
Detailed Description
Example 1
The technical scheme adopted by the invention is an automatic rainfall station abnormity screening method based on rain-measuring radar data, which mainly comprises two main parts: the method comprises the steps of firstly, constructing an extreme value judgment set, a peripheral station judgment set and a radar reflectivity judgment set of an abnormal station, using the extreme value judgment set, the peripheral station judgment set and the radar reflectivity judgment set as a space-time sequence two-dimensional criterion of the abnormal station in an actual measurement precipitation process, secondly, obtaining a singular variation value in ground rain gauge time sequence monitoring data by utilizing a curved surface similarity matching algorithm, extracting a suspected abnormal station in a space dimension, obtaining the similarity of the rainfall station normalized time sequence data, the peripheral station and radar reflectivity grid time sequence data based on a time sequence data similarity mining algorithm, judging the reasonability of the station monitoring data in a time dimension, and judging the possible fault type of the station according to the abnormal characteristics of the station. The method comprises the following steps:
step 1, a Hampel method and a Grabbs criterion are adopted to realize long sequence monitoring of station abnormal data, abnormal years are selected by the Hampel method, the annual accumulated rainfall of a station and the annual accumulated rainfall of adjacent stations are subjected to Grabbs criterion analysis, station data with less abnormal years and singular points are compiled, month, day and hour extreme values are set according to the compiling quality, and an extreme value judgment set of station long sequence in an area is constructed;
step 2, establishing an optimal association relationship between the sites and peripheral sites based on the high-precision topographic data, establishing a synchronous time sequence data judgment set of the peripheral sites, and establishing a radar grid time sequence data judgment set based on national 6min rain measuring radar volume sweep data;
step 3, extracting three-dimensional change curved surfaces of the reflectivity of the rain-measuring radar of layers with different heights in the area range in real time, combining a rainfall distribution triangulation network curved surface generated by an automatic rainfall station, acquiring singular points in monitoring data of the ground rainfall meter by using a free-form surface similarity evaluation algorithm based on area segmentation, extracting suspected station abnormal values and acquiring a primary screening result;
step 4, for the abnormal values of the stations, based on sequence values of a radar grid time sequence data judgment set and a peripheral station synchronous time sequence data judgment set, utilizing a time sequence data similarity mining algorithm based on distance to obtain the similarity of singular station normalized time sequence data and peripheral stations and radar reflectivity grid time sequence data, automatically studying and judging suspected station abnormal values, and determining the degree of reasonability of the singular station time sequence data;
and 5, providing an abnormal value screening result of the rainfall station based on an extreme value judgment result, a curved surface similarity screening result and the time sequence data reasonability of the singular station data, and providing a judgment result of the station fault type according to different abnormal characteristics of the abnormal station time sequence.
In step 1 of this embodiment, the ground observation stations are observation data of 4 ten thousand multi-factor observation stations across the country, more than 30 ten thousand torrential rainfall monitoring stations across the country and 2170 standard meteorological stations, the rainfall acquisition time is 1min, and due to the existence of a delay condition, standardized processing is performed according to 6min and 1h, the standardized processing is performed into a 6min time interval sequence, and the 6min time interval sequence and radar reflectivity data are kept in a unified time interval unit.
In the step 1, the accumulated rainfall of each year is calculated for a single station, the annual accumulated rainfall of the stations is arranged in sequence, the abnormal year is distinguished by using a Hampel identifier criterion, and the abnormal year is judged if the numerical value exceeds the statistical dispersion degree.
In the step 1, all the site areas are partitioned according to the radius of 50km, the annual rainfall sum of the numerical points exceeding the statistical dispersion degree and the sites within the peripheral 50km is compared, and Grabbs criterion iterative calculation is carried out on the abnormal sites and the annual rainfall sum of the peripheral sites within the peripheral 50km until no abnormal value exists. And (3) editing the data sequence after eliminating the abnormality, eliminating the month, day and hour data sequence by adopting the same algorithm, extracting the month, day and hour extreme values of the stations in the region, constructing a long sequence extreme value judgment set of the stations in the region, and taking the long sequence extreme value judgment set as a judgment basis if the real-time observed value of one station exceeds the extreme value range.
In the step 2 of the embodiment, based on high-precision topographic and geomorphic data of 1:50000, correlation factors of all stations within a range of 10km around a station are extracted, including separation distance, altitude difference, shelter elevation, gradient, slope included angle and the like, the correlation degrees of the peripheral stations and the station in rainfall monitoring are influenced by the factors, influence weights of the factors are calculated, so that the comprehensive correlation degree of the peripheral stations is calculated, the peripheral stations are sorted according to the correlation degree by using an intelligent matching tool, the peripheral stations with the correlation degrees meeting requirements are automatically matched based on an optimal weight selection method, a station-peripheral station fixed correlation relation data set is established, a peripheral station synchronous time sequence data judgment set is established, when the rationality judgment of the station time sequence data is needed, rainfall time sequence data of the corresponding peripheral stations in the same time period are automatically acquired, and the correlation relation is shown in fig. 2.
In the step 2, based on the station-peripheral station association relationship, the longitude and latitude of the station and the associated station are recorded, the radar reflectivity data grid position is located, a radar grid time sequence data judgment set is constructed, and when the grid time sequence rationality is required to be judged, the reflectivity grid where the station is located and the reflectivity grid where the associated station is located can be automatically obtained.
In step 3 of this embodiment, the real-time national radar reflectivity jigsaw is used to divide precipitation areas, extract areas with reflectivity greater than the background value, and determine precipitation range areas, and all precipitation stations within the coverage area of the areas belong to the same discrimination area. Because the radar elevation heights are different and reflect different rainfall conditions, the data of the conventional 6 altitude layers are adopted for simultaneous comparison. And extracting radar reflectivity grid data in the same time period, normalizing, and generating a 6-layer reflectivity grid curved surface by taking each grid reflectivity value as a height dimension, wherein the curvature change characteristic and the gradient change size of the curved surface reflect the continuous spatial change trend of precipitation.
In the step 3, all stations in the coverage area in the segment are used as vertexes, a Delaunay triangulation network is constructed by adopting a merging network construction method, actual-measured precipitation of the stations is used as the height dimension of the triangulation network, a three-dimensional change curved surface of the precipitation distribution of the stations is generated, and the precipitation surface triangulation network reflects the continuous change condition of the precipitation of the stations on the space.
And 3, extracting feature description vectors of the radar reflectivity changing curved surface and the station triangulation network changing curved surface. Curvature is an important attribute for describing a three-dimensional curved surface, convex features, concave features, hyperbolic point region features and parabolic point region features of the curved surface are classified by taking surface gravity center Gaussian curvature as a criterion, automatic segmentation of the regions is realized, 7-dimensional generalized vectors of sub-regions of the segmented sub-regions are extracted for describing topological features of the segmented regions, and feature descriptors are as follows: the similarity of the change trend of two unknown curved surfaces can be compared by comparing the similarity of the feature descriptors, wherein the region type, the relative area of the regions, the total curvature of the regions and the similarity of the topological connection relationship of the regions (including the relative boundary length of 4 types of connection with each region).
In the step 3, the similarity measurement of the curved surfaces can be realized by comparing the similarity of the local areas, the similarity of the three-dimensional change curved surfaces of the station precipitation distribution at each moment and the 6 radar reflectivity change curved surfaces is sequentially calculated, and the geometric shape, the topological relation and the curvature change conditions of the local areas of the 6 groups of curved surfaces are compared. The similarity of the curved surface is formed by the synthesis of the similarity of each region, each region of the curved surface is regarded as a node, a descriptor is a node attribute, the optimal matching of the feature descriptor is completed by adopting an empowerment bipartite graph optimal matching Kuhn-Munkres method, as shown in figure 3, each sub-region can obtain a group of curved surface similarity coefficients Si after matching, and a similarity coefficient threshold is set. In the three-dimensional curved surface of the station precipitation distribution, singular points lower than the threshold value are suspected precipitation station fault points and are stored in a temporary file as a primary screening result.
In this embodiment, in step 4, the link site long sequence extremum determining set compares the abnormal site time sequence data, determines whether there is extremum exceeding data, determines that the extremum exceeding data is abnormal, records the time length of the extremum exceeding data, the occurrence frequency, and the time when the extremum exceeding data starts to end, and stores the extremum exceeding data in the site fault feature library for site fault determination.
In step 4, for the points which do not exceed the extreme value but are suspected to be abnormal, extracting and normalizing hourly precipitation time sequence data of abnormal stations (30d/15d/7d), linking a synchronous time sequence data judgment set of peripheral stations, respectively extracting normalized instantaneous-period peripheral station precipitation time sequence normalized data and radar reflectivity time sequence normalized data of grids where the stations are located on the basis of the radar grid time sequence data judgment set, obtaining the hourly precipitation time sequence data similarity of the suspected abnormal stations by using a time sequence data similarity mining algorithm based on distance, and determining whether the change condition of the suspected station data in the time dimension is consistent with the change conditions of the peripheral and radar reflectivity. Performing Discrete Fourier Transform (DFT) on time sequence data to transform the time sequence data from a time domain space to a frequency domain space, mapping the time sequence data to points of a multidimensional space, reducing the dimensionality of the time sequence data and the computational complexity, taking a distance function between two groups of time sequence data as a similarity discrimination function, solving the distance between the kth nearest neighbor and the point of each point (abnormal site time sequence point) of a data set and the distance between the kth nearest neighbor and the point of a reference point r point (radar reflectivity grid and peripheral site time sequence), mining the distance difference of a similar data point set, and acquiring the similarity measurement of the site-reference point data;
in step 4, if the station-peripheral station and station-radar reflectivity grid time sequence data are compared in sequence, the goodness of fit of the station precipitation measurement result on the variation trend and the variation conditions of the peripheral station and radar reflectivity can be known, and therefore whether an abnormal fault occurs on the time dimension can be known. And if the similarity of the variation trend is higher, the precipitation variation of the station is considered to be reasonable, the station is not abnormal, otherwise, the station is considered to be abnormal, the time length of abnormal value data, the occurrence frequency and the starting and ending time of the over-extreme value data are recorded and are stored in the station fault feature library for judging the station fault.
In this embodiment, in step 5, the possible failure type and the possible failure time of the station are determined. The method comprises the steps of obtaining a suspected abnormal site based on a curved surface similarity screening result, judging whether an abnormal point exceeds an extreme value range based on a site extreme value judgment set, extracting abnormal data features exceeding the extreme value, extracting the deviation degree of abnormal value change, the abnormal value duration, the occurrence frequency and other features based on the curved surface similarity judgment set, extracting the comparison result based on the time sequence data reasonability, integrating the abnormal features of three stages, and judging the possible fault type of the abnormal site and the fault occurrence time and time of the site based on the characteristics of the deviation degree of the abnormal value, the abnormal value duration, the abnormal value change degree, the abnormal value change trend, the abnormal site range and the like.
TABLE 1 possible reasons for failure
The invention is further illustrated with reference to fig. 1:
in the step 1), 1) the difference value of the annual data of a single site is checked, and whether an abnormal value exists is judged. For the well-processed 6min interval precipitation observation sequence, the annual accumulated precipitation of the sites is arranged in sequence, and the Hampel identifier criterion is used for distinguishing abnormal years. Taking the rainfall sequence as a variable X, arranging the annual rainfall total data of the sites in a descending order, selecting a median Med, calculating the absolute value of the annual rainfall total and the median card difference value, namely firstly calculating the median X of the XmidAnd then calculating the deviation R ═ X-X of each variable from the medianmidL. The median absolute value (MAD) of all the above absolute distances is also found, and the median reflects the degree of data dispersion, as shown in the following equation:
wherein MAD is Median | X-XmidIf the data Z of the site in a certain year is more than or equal to 2.24, the extreme value is judged to be abnormal, and the next step of judgment is carried out. .
2) And partitioning all the site areas according to the radius of 50KM, comparing the annual rainfall sum of the numerical points exceeding the statistical dispersion degree with the peripheral sites within 50KM, and performing Grabbs criterion iterative calculation by comparing the annual rainfall sum of the abnormal sites with the peripheral sites within 50 KM. The method can effectively eliminate the shielding effect of the abnormal values on the same side by solving a plurality of abnormal values according to the Grabbs criterion, and the judging process is as follows:
firstly, in the first subarea, the annual rainfall amount of the station judged to be abnormal in the last step and the peripheral stations within 50km is compared, if data of a certain station in 2016 are abnormal, the annual rainfall amount data of the station and the peripheral stations in 2016 are selected, and the data are arranged from small to large to obtain a median XmidMinimum value of x1Maximum value of xn。
Ordering samples from small to large into a new series X ═ (X)1,x2,x3) Statistical value G of the critical coefficient G (a, n)0(obtained by looking up a critical value table), and then calculating Gi，Gn：
Wherein a is significance level, n is measurement times, and XmidIs the median of the samples, σ is the standard deviation.
If G isi≥GnAnd Gi≥G0Then should xiIs removed if Gn≥GiAnd Gn≥G0Then xnShould be removed, if Gi<G0And Gn<G0Then does not existAn outlier. If the abnormal value exists, repeating the steps after removing the abnormal value until the abnormal value does not exist; if no abnormal value exists, the selection is directly finished. And if the abnormal value exists, removing the abnormal value, and repeating the steps according to the new sequence until the abnormal value does not exist.
Step 2: calculating correlation factors including separation distance R and altitude difference H for all stations within 10km around the station△Shelter height HSSlope G, slope included angle A and wind direction slope included angle PWEI, calculating different area factor impression weights based on early precipitation long sequence data, and expressing the correlation of correlation factors to precipitation measured values between two stations as a normalized data set
In different areas, the action degree of each influence factor is different, the influence on the distance in a plain area is small, the influence on the slope and the gradient of a mountain area is large, and the correlation of the altitude difference and the included angle of the slope and the gradient of the wind direction in some areas on precipitation is greatly influenced, so that the relation function of the association factor data set and the association factor of the area is determined by adopting weight regression calculation based on machine learning. For fixed stations in the same area, the comprehensive association degree of station-peripheral stations is matched pairwise by using intelligent matching tools, and automatic association of the stations-peripheral precipitation intensity related stations is realized.
And storing the association relationship as static data in an association relationship file, and calling the site time sequence data-associated site time sequence data through an interface to realize real-time linkage.
And step 3: 1) for the stations with the radar basic reflectivity larger than or equal to 10db, extracting radar reflectivity coverage areas meeting the reflectivity requirements, extracting reflectivity data of 6 elevation angle layers, and simultaneously delimiting all torrential flood stations in the coverage areas of the areas. The project of the invention needs to process rainfall data of 10 ten thousand magnitude, and aims at the high efficiency requirement of processing mass data, so the Delaunay triangulation network is constructed in parallel by adopting a merging and network construction method, which comprises the following specific steps:
c1, n of original networked datasetsThe data points are sorted from big to small according to x and y, and the sorting result is stored in a data point set V [0.. n ]]In (1). If k computing nodes exist in the distributed environment, the data point set V [0.. n ] is divided into a plurality of data points according to the memory and the computing capacity of each node and the data quantity of the blocks]The data points in (A) are divided into m corresponding length segments Vs0,s1…sm-1]. Opening up array T [0.. m-1]]The initially generated child triangulation is recorded.
C2, taking l segments at a time as a unit, sequentially adding siAnd allocating the nodes to corresponding nodes, and calling a Delaunay triangulation network generation program sub-triangulation network. Set of points s consisting of sub-observation pointsiAdditionally setting three points P for the basic data setiPjPkSo that the triangle of the connecting line stroke can cover the whole point set siWhile ensuring that these three points are not at siIn any circumscribed circle. From siExtracting any point P, analyzing the position relation between the point P and the current triangle, and if P is positioned on the triangle delta PiPjPkIn (1), then P and Δ P areiPjPkConnecting lines to form new sides and triangles if P is exactly located at delta PiPjPkAnd connecting the edge with the two triangle vertexes corresponding to the edge, p and the two end points of the edge to form a new triangle. After obtaining a new triangulation, continuously turning the illegal edge until the Delaunay condition is met, wherein the criterion is as follows:
repeating the steps until the Delaunay subdivision of all the points is completed, and finally deleting the initially added three points PiPjPk。
C3, once storing the constructed sub-triangulation network into T [ i.. i + l-1], changing out to an external memory, circularly calling a sub-triangulation network generation program, and finally forming m initial merging sections T [0.. m-1 ]. And recording the next round of triangulation by using a linked list G, calling a plurality of adjacent sub-triangulation networks for each node from T [0.. m-1] in sequence, and merging the sub-triangulation networks in sequence. When the left and right adjacent Delaunay sub-triangulation networks are combined, the upper and lower baselines connecting the convex shells of the two sub-triangulation networks are found first, and then the two sub-triangulation networks are combined in sequence from the lower baselines to the upper baselines according to the empty circle criterion. And inserting the triangulation networks obtained by combining all the nodes into the tail part of the linked list in sequence according to the distribution sequence, changing out the triangulation networks to an external memory, merging the adjacent sub triangulation networks in sequence, and repeating the process until the merging of the m triangulation networks is finished.
C4, repeating the C3 process for chain table G, and performing the next round of merging on the adjacent sub-triangulation networks until the final Delaunay triangulation network is formed.
2) Defining the station triangulation network change curved surface as SundelThe radar reflectivity change curved surface is
Quantifying the Gaussian curvature of each triangle on the curved surface, and taking the Gaussian curvature of the center of the triangle
Calculation of where KiThe value of the Gaussian curvature of the vertex of the triangle and the mean curvature of the gravity center of the triangle are
Wherein HiIs the mean curvature of the triangle vertices. The surface gravity center Gaussian curvature is taken as a criterion to classify convex features, concave features, hyperbolic point region features and parabolic point region features of the curved surface, points with Gaussian curvature larger than 0 are elliptic points, points smaller than 0 are hyperbolic points, and points equal to 0 are parabolic points, so that the curved surface region is divided into 4 types:
the automatic segmentation of the region is realized through the automatic growth of the region, the 7-dimensional generalized vectors of the sub-region after segmentation are extracted to carry out the topological feature description of the segmented region, and the feature descriptors are respectively as follows: region type, region relative area, region total curvature, and region topological connection relationship similarity.
The region types are divided, if the types of the 2 regions are different, the two regions act similarly to each other as 0;
the integral of the total curvature of the region, Gaussian curvature, over the area of the region is
Wherein m is the number of meshes/triangles in the area, AiIs the mesh/triangle absolute area;
relative area of the region is
SiIs the area of the segmentation subarea, and S is the area of the total curved surface;
quantitatively expressing the connected regions around the segmented regions, wherein the topological characteristics of the I-IV class segmented regions are described as
Wherein liRepresenting the side length of the adjacent edge region, wherein n is the total number of the connecting regions around the region, then:
the area type comparison is as follows:
where T denotes the region type.
The relative area similarity of the regions is expressed as:
the similarity of the regional topological connection relation and the similarity of the geometric shape are as follows:
wherein a represents the corresponding weight value,
representing the overall curvature similarity of the region.
The total curvature of the two regions is respectively
Taking a region 7-dimensional vector descriptor as a node attribute, calculating the matching similarity of the site and the radar reflectivity curved surface by adopting a Kuhn-Munkres method, wherein the similarity between 2 curved surfaces is as follows:
wherein deltan(j)jIs a curved surface S1M (j) th area and curved surface S2M (j) represents the number of best matching rows of the similarity matrix. A. thejRepresenting a curved surface S2Relative area of jth region, Am(j)Representing a curved surface S1Relative area of the (m), (j) th region.
And sequentially calculating the similarity of the three-dimensional change curved surface of the station precipitation distribution at each moment and the 6 radar reflectivity change curved surfaces, obtaining a group of curved surface similarity coefficients Si after each subarea is matched, and setting a similarity coefficient threshold value (S is less than or equal to 0.22-0.34). In the three-dimensional curved surface of the station precipitation distribution, singular points lower than the threshold value are suspected precipitation station fault points and are stored in a temporary file as a primary screening result.
And 4, step 4: extracting and normalizing hourly precipitation time sequence data of abnormal sites (30d/15d/7d), linking a synchronous time sequence data judgment set of peripheral sites and a radar grid based time sequence data judgment setAnd respectively extracting normalized precipitation time sequence normalized data of peripheral stations at the same time period and normalized data of the reflectivity time sequence of the grid radar of the station. The time sequence data is subjected to Discrete Fourier Transform (DFT) to be transformed from a time domain space to a frequency domain space, the time sequence data is mapped to a point of a multi-dimensional space, and k coefficients before transformation are selected as time sequence data frequency domain features, so that the time sequence data dimension is reduced, and the calculation complexity is reduced. For time sequence points mapped to k-dimensional space, use Dk(p) represents the distance of the kth nearest neighbor of point p from p as a function of data distance.
First, the distance D of the k-th nearest point of each point in the data set is obtainedk(p), distance D of the kth nearest point of reference point r from rk(r), the data set is a suspected site time sequence data set P, the reference point set is a peripheral site and a reflectivity normalized grid time sequence data set, and pairwise comparison of suspected sites to peripheral sites and suspected sites to reflectivity grids is respectively carried out:
1) setting a hollow missing data point set C by time series data DFT;
2) computing D for point p in time series datasetk(p) and D of reference point rk(r) interpolation, Diff ═ Dk(p)-Dk(r)
3) Arranging Diffs from small to large, and inserting points p with the first n Diffs into an acacia data point set C;
4) calculating D for point p and reference point rkSelecting point p with the first n Diffs as similar data points, calculating the sum of point p and reference point Dk(p) and D of reference point rkAnd (r) mining the distance difference of the distance similar data point set to obtain the similarity measurement of the site-reference point data.
The distance dispersion degree of the time sequence data k domain point set represents the data change trend similarity, and distance dispersion threshold values are set in different regions. And if the similarity of the variation trend is higher, the precipitation variation of the station is considered to be reasonable, the station is not abnormal, otherwise, the station is considered to be abnormal, the time length of abnormal value data, the occurrence frequency and the starting and ending time of the over-extreme value data are recorded and are stored in the station fault feature library for judging the station fault.
And 5: the method comprises the steps of obtaining a suspected abnormal site based on a curved surface similarity screening result, judging whether an abnormal point exceeds an extreme value range based on a site extreme value judgment set, extracting abnormal data features exceeding the extreme value, extracting the deviation degree of abnormal value change, the abnormal value duration, the occurrence frequency and other features based on the curved surface similarity judgment set, extracting the comparison result based on the time sequence data reasonability, integrating the abnormal features of three stages, and judging the possible fault type of the abnormal site and the fault occurrence time and time of the site based on the characteristics of the deviation degree of the abnormal value, the abnormal value duration, the abnormal value change degree, the abnormal value change trend, the abnormal site range and the like.
The invention is described above with reference to the accompanying drawings, it is obvious that the implementation of the invention is not limited in the above manner, and it is within the scope of the invention to adopt various modifications of the inventive method concept and solution, or to apply the inventive concept and solution directly to other applications without modification.
Claims (7)
1. An automatic rainfall station abnormal value screening method based on rain-measuring radar data is characterized in that: the method comprises the following steps:
step 1, a Hampel method and a Grabbs criterion are adopted to realize long sequence monitoring of station abnormal data, abnormal years are selected by the Hampel method, the annual accumulated rainfall of a station and the annual accumulated rainfall of adjacent stations are subjected to Grabbs criterion analysis, station data with the abnormal years and less than 20% of singular points are compiled, month, day and hour extreme values are set according to the compiling quality, and an extreme value judgment set of the station long sequence in the region is constructed;
step 2, establishing an optimal association relationship between the sites and peripheral sites based on the high-precision topographic data, establishing a synchronous time sequence data judgment set of the peripheral sites, and establishing a radar grid time sequence data judgment set based on national 6min rain measuring radar volume sweep data;
step 3, extracting three-dimensional change curved surfaces of the reflectivity of the rain measuring radar of different height layers in the rain area range in real time, combining a rainfall distribution triangulation network curved surface generated by an automatic rainfall station, acquiring singular points in monitoring data of the ground rainfall meter by using a free curved surface similarity evaluation algorithm based on region segmentation, extracting suspected station abnormal values and acquiring a primary screening result;
step 4, based on sequence values of the radar grid time sequence data judgment set and the peripheral station synchronous time sequence data judgment set, utilizing a distance-based time sequence data similarity mining algorithm to obtain the similarity of singular station normalized time sequence data and peripheral station and radar reflectivity grid time sequence data, automatically studying and judging suspected station abnormal values, and determining the degree of reasonability of the singular station time sequence data;
and 5, providing an abnormal value screening result of the rainfall station based on an extreme value judgment result, a curved surface similarity screening result and the time sequence data reasonability of the singular station data, and providing a judgment result of the station fault type according to different abnormal characteristics of the abnormal station time sequence.
2. The method for automatically screening the abnormal value of the rainfall station based on the rain-measuring radar data as claimed in claim 1, wherein: the concrete operation in the step 1 is as follows: calculating the annual accumulated rainfall of a single station, acquiring a data dispersion degree statistic value in a long sequence observation time sequence through a Hampelidentifier, extracting abnormal stations, performing Grabas criterion analysis by comparing annual rainfall total amount of the abnormal stations with surrounding stations within 50km, eliminating annual abnormal values until no abnormal value exists in the station sequence values, integrally editing screened abnormal data station sequences in an area, extracting month, day and hour extreme values of the stations in the area, and constructing an extreme value judgment set of the long sequence of the stations in the area.
3. The method for automatically screening the abnormal value of the rainfall station based on the rain-measuring radar data as claimed in claim 1, wherein: in the step 2, the specific operation is as follows: extracting factors of altitude difference, shelter elevation, site distance and slope included angle of a site within a range of 10km around the site based on high-precision topographic and geomorphic data of a small watershed, calculating influence weight of each factor based on a weight optimization principle, automatically matching and establishing a site-peripheral site optimal association relation, and establishing a synchronous time sequence data judgment set of a peripheral site; and (4) extracting reflectivity data of the radar grid where the station is located every 6min, and constructing a radar grid time sequence data judgment set.
4. The method for automatically screening the abnormal value of the rainfall station based on the rain-measuring radar data as claimed in claim 1, wherein: in step 3, the specific operation is as follows: extracting real-time national radar reflectivity jigsaw puzzle, segmenting according to a precipitation range, automatically matching all rainfall sites in a precipitation segment area, obtaining the reflectivity of 6 conventional elevation height layers of the radar, and generating 6 three-dimensional curved surfaces with different height layers by taking the reflectivity as height change; meanwhile, a Delaunay triangulation network is generated based on the station positions in the precipitation zone range, and a real-time station precipitation value is used as a height dimension to generate a real-time station precipitation distribution three-dimensional change curved surface; according to the Gaussian curvature feature classification criterion, a region growing algorithm is adopted to segment the free-form surface, the geometric shape and topological features of the surface are extracted, and 7-dimensional generalized vector descriptors of the sub-region are extracted to form shape description of the free-form surface.
5. The method of claim 4, wherein the method comprises the following steps: and step 3, sequentially comparing the curved surface similarity of the real-time rainfall distribution three-dimensional change curved surface and the 6 radar reflectivity change curved surfaces, taking the region 7-dimensional vector descriptor as the node attribute, calculating the matching similarity of the site and the radar reflectivity curved surfaces by adopting a Kuhn-Munkres method, obtaining singular points of the real-time monitoring data of the ground site according to the similarity coefficient, extracting suspected site abnormal values, and obtaining a primary screening result.
6. The method for automatically screening the abnormal value of the rainfall station based on the rain-measuring radar data as claimed in claim 1, wherein: in step 4, the specific operation is as follows: extracting and normalizing hour precipitation time sequence data of suspected abnormal sites, comparing the hour precipitation time sequence data with a long sequence extreme value judgment set, determining abnormality if the extreme value is exceeded, extracting precipitation time sequence normalized data of the peripheral sites and radar reflectivity time sequence normalized data of the grids where the sites are located in the peripheral site synchronous time sequence data judgment set and the radar grid time sequence data judgment set of the peripheral sites of the same period respectively for the points which do not exceed the extreme value, and acquiring the similarity of the hour precipitation time sequence data of the suspected abnormal sites by using a distance-based time sequence data similarity mining algorithm so as to judge the reasonability of the precipitation change trend of the suspected abnormal sites.
7. The method for automatically screening the abnormal value of the rainfall station based on the rain-measuring radar data as claimed in claim 1, wherein: in step 5, the specific operation is as follows: whether an abnormal site exceeds an extreme value range or not is known based on a site extreme value judgment set, the dispersion degree of a singular value is extracted based on a curved surface similarity screening result, the abnormal condition of the site is comprehensively judged based on a time sequence data reasonability comparison result, and the possible fault type and the possible fault time of the abnormal site are judged based on the dispersion degree of the abnormal value, the duration time of the abnormal value, the change degree of the abnormal value, the change trend of the abnormal value and the range characteristic of the abnormal site.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111026172.2A CN113742327B (en) | 2021-09-02 | 2021-09-02 | Automatic rainfall station abnormal value screening method based on rain-measuring radar data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111026172.2A CN113742327B (en) | 2021-09-02 | 2021-09-02 | Automatic rainfall station abnormal value screening method based on rain-measuring radar data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113742327A true CN113742327A (en) | 2021-12-03 |
| CN113742327B CN113742327B (en) | 2022-03-25 |
Family
ID=78735021
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111026172.2A Active CN113742327B (en) | 2021-09-02 | 2021-09-02 | Automatic rainfall station abnormal value screening method based on rain-measuring radar data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113742327B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114817228A (en) * | 2022-06-01 | 2022-07-29 | 武汉大学 | Abnormal value removing method for long-time sequence monitoring data of lake water level |
| CN114898810A (en) * | 2022-05-20 | 2022-08-12 | 厦门大学 | Method for detecting growth condition of microorganism |
| CN114997534A (en) * | 2022-07-29 | 2022-09-02 | 长江水利委员会水文局 | Similar rainfall forecasting method and equipment based on visual features |
| CN115271003A (en) * | 2022-09-30 | 2022-11-01 | 江苏云天新材料制造有限公司 | Abnormal data analysis method and system for automatic environment monitoring equipment |
| CN117111181A (en) * | 2023-09-05 | 2023-11-24 | 浙江省气象台 | Short-time strong precipitation probability prediction method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6850184B1 (en) * | 2003-05-05 | 2005-02-01 | Wsi Corporation | Forecasted radar mosaics |
| WO2013181463A1 (en) * | 2012-05-30 | 2013-12-05 | Saint Louis University | Apparatus and method for provoding environmental predictive indicators to emergency response managers |
| CN104483719A (en) * | 2014-12-31 | 2015-04-01 | 贵州东方世纪科技股份有限公司 | Method for diagnosing data exception of telemetering precipitation stations by means of radar echoes |
| CN108761576A (en) * | 2018-05-28 | 2018-11-06 | 国网山西省电力公司电力科学研究院 | A kind of X-band weather radar and precipitation station data fusion method and system |
| CN109840260A (en) * | 2019-02-02 | 2019-06-04 | 中国水利水电科学研究院 | A kind of extensive real-time rainfall automatic Observation station ranked data processing method based on dynamic interpolation |
-
2021
- 2021-09-02 CN CN202111026172.2A patent/CN113742327B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6850184B1 (en) * | 2003-05-05 | 2005-02-01 | Wsi Corporation | Forecasted radar mosaics |
| WO2013181463A1 (en) * | 2012-05-30 | 2013-12-05 | Saint Louis University | Apparatus and method for provoding environmental predictive indicators to emergency response managers |
| CN104483719A (en) * | 2014-12-31 | 2015-04-01 | 贵州东方世纪科技股份有限公司 | Method for diagnosing data exception of telemetering precipitation stations by means of radar echoes |
| CN108761576A (en) * | 2018-05-28 | 2018-11-06 | 国网山西省电力公司电力科学研究院 | A kind of X-band weather radar and precipitation station data fusion method and system |
| CN109840260A (en) * | 2019-02-02 | 2019-06-04 | 中国水利水电科学研究院 | A kind of extensive real-time rainfall automatic Observation station ranked data processing method based on dynamic interpolation |
Non-Patent Citations (2)
| Title |
|---|
| 丛芳等: "新一代天气雷达与地面雨量资料的综合分析", 《气象》 * |
| 李哲: "多源降雨观测与融合及其在长江流域的水文应用", 《中国优秀博硕士学位论文全文数据库(博士) 基础科学辑》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114898810A (en) * | 2022-05-20 | 2022-08-12 | 厦门大学 | Method for detecting growth condition of microorganism |
| CN114817228A (en) * | 2022-06-01 | 2022-07-29 | 武汉大学 | Abnormal value removing method for long-time sequence monitoring data of lake water level |
| CN114997534A (en) * | 2022-07-29 | 2022-09-02 | 长江水利委员会水文局 | Similar rainfall forecasting method and equipment based on visual features |
| CN115271003A (en) * | 2022-09-30 | 2022-11-01 | 江苏云天新材料制造有限公司 | Abnormal data analysis method and system for automatic environment monitoring equipment |
| CN117111181A (en) * | 2023-09-05 | 2023-11-24 | 浙江省气象台 | Short-time strong precipitation probability prediction method and system |
| CN117111181B (en) * | 2023-09-05 | 2024-04-09 | 浙江省气象台 | Short-time strong precipitation probability prediction method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113742327B (en) | 2022-03-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113742327B (en) | Automatic rainfall station abnormal value screening method based on rain-measuring radar data | |
| CN110163472B (en) | Large-range extreme drought emergency monitoring and influence evaluation method and system | |
| Fathizad et al. | Evaluating desertification using remote sensing technique and object-oriented classification algorithm in the Iranian central desert | |
| Lakshmanan et al. | Data mining storm attributes from spatial grids | |
| CN111625993A (en) | Small watershed surface rainfall interpolation method based on mountainous terrain and rainfall characteristic prediction | |
| CN110363299B (en) | Spatial case reasoning method for outcrop rock stratum layering | |
| CN114004137A (en) | Multi-source meteorological data fusion and pretreatment method | |
| Shokri et al. | A robust and efficient method for power lines extraction from mobile LiDAR point clouds | |
| Laverde-Barajas et al. | Decomposing satellite-based rainfall errors in flood estimation: Hydrological responses using a spatiotemporal object-based verification method | |
| CN115410036A (en) | Automatic classification method for key element laser point clouds of high-voltage overhead transmission line | |
| CN116597199A (en) | Point cloud tree classification method and system based on airborne LiDAR | |
| Che et al. | Vo-SmoG: A versatile, smooth segment-based ground filter for point clouds via multi-scale voxelization | |
| Haddjeri et al. | Exploring the sensitivity to precipitation, blowing snow, and horizontal resolution of the spatial distribution of simulated snow cover | |
| CN116912582A (en) | Strong robustness hyperspectral target detection method based on characterization model | |
| CN116543223A (en) | Neural network optimized lower tree detection method | |
| Derouiche et al. | Characterisation of rainfall events in northern Tunisia using self-organising maps | |
| Rettelbach et al. | From images to hydrologic networks-understanding the arctic landscape with graphs | |
| Senthilnath et al. | Multi-temporal satellite imagery for flood damage assessment | |
| CN117851778B (en) | Apparent recognition method and system based on bridge exposure degree | |
| CN118379649B (en) | Method for evaluating activity of debris flow in arid region based on deposition feature recognition of stacking fan | |
| Asthana et al. | Comparative analysis of pixel and object based classification approach for rapid landslide delineation with the aid of open source tools in Garhwal Himalaya | |
| Wu et al. | Channel head extraction based on fuzzy unsupervised machine learning method | |
| Yuan et al. | Estimation of water storage capacity of Chinese reservoirs by statistical and machine learning models | |
| Liu et al. | Terrain-adaptive ground filtering of airborne LIDAR data based on saliency-aware thin plate spline | |
| Rutzinger et al. | Object detection in airborne laser scanning data-an integrative approach on object-based image and point cloud analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |