CN109614744B - Big data-based precipitation detection method and system - Google Patents
Big data-based precipitation detection method and system Download PDFInfo
- Publication number
- CN109614744B CN109614744B CN201811617914.7A CN201811617914A CN109614744B CN 109614744 B CN109614744 B CN 109614744B CN 201811617914 A CN201811617914 A CN 201811617914A CN 109614744 B CN109614744 B CN 109614744B
- Authority
- CN
- China
- Prior art keywords
- detection
- optimization
- space
- model
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01W—METEOROLOGY
- G01W1/00—Meteorology
- G01W1/14—Rainfall or precipitation gauges
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Environmental & Geological Engineering (AREA)
- Life Sciences & Earth Sciences (AREA)
- Atmospheric Sciences (AREA)
- Ecology (AREA)
- Environmental Sciences (AREA)
- Biodiversity & Conservation Biology (AREA)
- Computer Hardware Design (AREA)
- Evolutionary Computation (AREA)
- Geometry (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Hydrology & Water Resources (AREA)
- Radar Systems Or Details Thereof (AREA)
- Testing Or Calibration Of Command Recording Devices (AREA)
Abstract
The invention provides a big data-based precipitation detection method and a big data-based precipitation detection system, wherein the control method comprises the following steps: acquiring detection data, and establishing a space-time detection model according to the detection data; judging whether the detection parameters in the space-time detection model have singular values or not, and correcting the detection parameters when the detection parameters have the singular values; performing data optimization on the space-time detection model according to a preset optimization rule to obtain a space-time optimization model; and performing model loss calculation aiming at the space-time optimization model to obtain a loss function, and outputting the space-time optimization model to obtain a detection result when the loss function is judged to be smaller than a loss threshold value. According to the invention, through establishing the design of the space-time detection model, the detection data is converted into specific model parameters, so that the subsequent processing and calculation of the data are facilitated, and through designing the space-time detection model in a data optimization manner, the influence of interference factors on the space-time detection model is prevented, so that the accuracy of the detection result is improved.
Description
Technical Field
The invention relates to the technical field of precipitation detection, in particular to a precipitation detection method and system based on big data.
Background
The abnormal spatial and temporal distribution of rainfall is an important factor for causing natural disasters such as flood disasters, landslides, debris flows and the like, the direct economic loss caused by the flood disasters in China is more than billions of yuan every year, and the average disaster population is more than 1.2 billion people. Although rainfall gauges, weather radars, weather satellites and other precipitation measurement means are widely applied, precipitation has extremely complex space-time changes in key areas such as cities and mountainous areas, sites of the rainfall gauges are not uniformly distributed, and even in areas with relatively dense sites such as cities, the fine space-time changes of the precipitation are still difficult to monitor; the weather radar can only measure partial falling water bodies or clouds under the condition of high elevation angle and is easily influenced by ground object echoes under the condition of low elevation angle, so that the measuring effect in cities and mountainous areas is limited; the rain measuring satellite can only measure the cloud top or penetrate the cloud top from top to bottom, and has a large difference with actual precipitation falling to the vicinity of the ground surface, so that accurate precipitation distribution is difficult to invert and detect according to echoes.
The existing method for detecting the precipitation is radar-based precipitation detection, and the existing radar-based precipitation detection method is low in detection accuracy and can effectively detect the precipitation in use.
Disclosure of Invention
Based on this, an object of the embodiments of the present invention is to solve the problem in the prior art that the detection accuracy for rainfall is low.
In a first aspect, the present invention provides a big data-based precipitation detection method, including:
acquiring detection data, and establishing a space-time detection model according to the detection data;
judging whether a detection parameter in the space-time detection model has a singular value or not, and correcting the detection parameter when the detection parameter is judged to have the singular value;
performing data optimization on the space-time detection model according to a local preset optimization rule to obtain a space-time optimization model;
and performing model loss calculation aiming at the space-time optimization model to obtain a loss function, and outputting the space-time optimization model to obtain a detection result when the loss function is judged to be smaller than a loss threshold value.
Further, in a preferred embodiment of the present invention, the step of building a spatio-temporal detection model according to the detection data comprises:
acquiring radar reflectivity factor combinations stored in the detection data, and establishing a precipitation space dependence mapping relation by taking MSE as an optimization target;
and acquiring the correlation of the precipitation process stored in the detection data, and establishing a precipitation time series mapping relation by taking the correlation of the precipitation process as a target.
Further, in a preferred embodiment of the present invention, the step of determining whether there is a singular value in the detection parameters in the spatio-temporal detection model includes:
respectively calculating parameter difference values between adjacent detection parameters, and judging whether the parameter difference values are larger than a difference threshold value;
and if so, judging that the detection parameters corresponding to the parameter difference values are singular values.
Further, in a preferred embodiment of the present invention, the step of correcting the detection parameter includes:
deleting the detection parameters and acquiring the parameter sum between the adjacent parameters of the detection parameters;
and calculating the average value of the parameter sum to obtain a replacement value, and replacing the detection parameter by the replacement value.
Further, in a preferred embodiment of the present invention, the step of optimizing data of the spatio-temporal detection model according to a local preset optimization rule includes:
carrying out variance calculation on the space-time detection model to obtain a variance value, and matching the variance value with an optimization table prestored locally to obtain first optimization fluctuation;
performing fluctuation optimization on the space-time detection model according to the first optimization fluctuation;
carrying out filtering calculation on the space-time detection model to obtain a filtering value, and matching the filtering value with the optimization table to obtain second optimization fluctuation;
performing fluctuation optimization on the space-time detection model according to the second optimization fluctuation;
performing filtering variance calculation on the space-time detection model to obtain a filtering variance value, and matching the filtering variance value with the optimization table to obtain a third optimization fluctuation;
and performing fluctuation optimization on the space-time detection model according to the third optimization fluctuation.
Further, in a preferred embodiment of the present invention, the calculation formula for performing model loss calculation on the spatio-temporal optimization model is as follows:
according to the rainfall detection method based on big data, the detection data are converted into specific model parameters by establishing the design of the space-time detection model, subsequent data processing and calculation are effectively facilitated, abnormal data are effectively and timely corrected by judging whether the singular values exist in the detection parameters, the accuracy of a detection result is improved, the influence of interference factors on the space-time detection model is effectively prevented by designing the space-time detection model in a data optimization mode, the accuracy of the space-time optimization model is judged, and when the space-time optimization model is judged to be correct, the result is output.
In a second aspect, the present invention provides a big data based precipitation detection system, including:
the modeling module is used for acquiring detection data and establishing a space-time detection model according to the detection data;
the judging module is used for judging whether singular values exist in detection parameters in the space-time detection model or not and correcting the detection parameters when the singular values exist in the detection parameters;
the optimization module is used for carrying out data optimization on the space-time detection model according to a local preset optimization rule so as to obtain a space-time optimization model;
and the calculation module is used for performing model loss calculation on the space-time optimization model to obtain a loss function, and outputting the space-time optimization model to obtain a detection result when the loss function is judged to be smaller than a loss threshold value.
Further, in a preferred embodiment of the present invention, the modeling module includes:
the first modeling unit is used for acquiring radar reflectivity factor combinations stored in the detection data and establishing a precipitation space dependence mapping relation by taking MSE as an optimization target;
and the second modeling unit is used for acquiring the correlation of the precipitation process stored in the detection data and establishing a precipitation time series mapping relation by taking the correlation of the precipitation process as a target.
Further, in a preferred embodiment of the present invention, the determining module includes:
the first calculating unit is used for respectively calculating parameter difference values between the adjacent detection parameters;
the judging unit is used for judging whether the parameter difference value is larger than a difference value threshold value or not; and if so, judging that the detection parameters corresponding to the parameter difference values are singular values.
Further, in a preferred embodiment of the present invention, the determining module further includes:
the acquisition unit is used for deleting the detection parameters and acquiring the parameter sum between the adjacent parameters of the detection parameters;
and the second calculating unit is used for calculating the average value of the parameter sum to obtain a replacement value and replacing the detection parameter by the replacement value.
According to the rainfall detection system based on big data, the design of the space-time detection model is established through the modeling module, so that the detection data are converted into specific model parameters, subsequent data processing and calculation are effectively facilitated, whether the singular value design exists in the detection parameters or not is judged through the judging module, abnormal data are effectively and timely corrected, the accuracy of a detection result is improved, the space-time detection model is subjected to data optimization through the optimizing module, the influence of interference factors on the space-time detection model is effectively prevented, the precision of the detection result is further improved, the space-time optimization model is subjected to model loss calculation through the calculating module, so that the accuracy of the space-time optimization model is judged, and when the space-time optimization model is judged to be correct, the result is output.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a flowchart of a big data-based precipitation detection method according to a first embodiment of the present invention;
FIG. 2 is a flowchart of a big data based precipitation detection method according to a second embodiment of the present invention;
FIG. 3 is a flowchart illustrating an embodiment of step S61 shown in FIG. 2;
fig. 4 is a schematic structural diagram of a big data based precipitation amount detection system according to a third embodiment of the present invention:
Detailed Description
In order to facilitate a better understanding of the invention, the invention will be further explained below with reference to the accompanying drawings of embodiments. Embodiments of the present invention are shown in the drawings, but the present invention is not limited to the preferred embodiments described above. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
The logic and/or steps represented in the flowcharts or otherwise described herein, such as an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Please refer to fig. 1, which is a flowchart of a method for detecting precipitation based on big data according to a first embodiment of the present invention, including steps S10 to S60.
Step S10, acquiring detection data, and establishing a space-time detection model according to the detection data;
the method for monitoring and acquiring the detection data can be ground meteorological station monitoring, satellite monitoring or radar monitoring, the satellite monitoring is mainly detected in a satellite cloud picture mode and mainly reflects cloud top information, and the radar monitoring can complete airspace scanning within minutes. The spatial resolution is about 1 kilometer, each radar can generate millions of groups of spatial monitoring data every hour, and radar reflection has a direct correlation with precipitation, so that detection data are acquired by adopting a radar monitoring mode in the step, preferably, the detection data are converted into specific model parameters by establishing the design of a space-time detection model in the step, and further, the subsequent processing and calculation of the data are effectively facilitated;
step S20, judging whether the detection parameters in the space-time detection model have singular values or not;
whether the singular value exists in the detection parameters or not is judged, so that abnormal data can be effectively and timely corrected, and the accuracy of a detection result is improved;
when the singular value exists in the detection parameters, the step S20 is executed, and the step S30 is executed;
step S30, correcting the detection parameters;
the correction method adopted in the step can be used for correcting in a mode of parameter replacement, parameter fluctuation modification or parameter deletion, so that detection errors caused by wrong parameters are effectively prevented, and the accuracy of the big data-based precipitation detection method is further improved;
s40, performing data optimization on the spatio-temporal detection model according to a local preset optimization rule to obtain a spatio-temporal optimization model;
the preset optimization rule stores a plurality of optimization conditions, the optimization conditions can be set according to the requirements of users, the optimization conditions can be data variance optimization, data fluctuation optimization and the like, and by designing the data optimization on the space-time detection model, the influence of interference factors on the space-time detection model is effectively prevented, and the accuracy of the detection result is further improved;
s50, calculating model loss aiming at the space-time optimization model to obtain a loss function;
wherein the accuracy of the spatio-temporal optimization model is determined by performing model loss calculation on the spatio-temporal optimization model;
step S60, when the loss function is judged to be smaller than the loss threshold, outputting the space-time optimization model to obtain a detection result;
the detection result stores precipitation information corresponding to each area, and a user can predict precipitation based on the precipitation information;
in this embodiment, the design of the spatiotemporal detection model is established to convert the detection data into specific model parameters, so as to effectively facilitate subsequent processing and calculation of data, the design of judging whether the singular values exist in the detection parameters is used to effectively and timely correct abnormal data, so as to improve the accuracy of the detection result, the design of data optimization is performed on the spatiotemporal detection model, so as to effectively prevent interference factors from affecting the spatiotemporal detection model, further improve the accuracy of the detection result, the model loss calculation is performed on the spatiotemporal optimization model, so as to judge the accuracy of the spatiotemporal optimization model, and when the spatiotemporal optimization model is judged to be correct, a result is output.
Referring to fig. 2, a flowchart of a big data based precipitation detection method according to a second embodiment of the present invention is shown, where the method includes steps S11 to S81.
S11, acquiring detection data, acquiring a radar reflectivity factor combination stored in the detection data, and establishing a precipitation space dependence mapping relation by taking MSE as an optimization target;
in order to prevent the spatial phase relationship from affecting the detection result, in this embodiment, a spatial difference method is introduced to establish the precipitation spatial dependence mapping relationship, so that the effect of the spatial phase relationship on the detection result is effectively reduced, and the accuracy of the precipitation detection method based on big data is improved;
s21, acquiring the correlation of the precipitation process stored in the detection data, and establishing a precipitation time series mapping relation by taking the correlation of the precipitation process as a target to obtain a space-time detection model;
in order to prevent the time sequence from affecting the detection result, the method establishes the precipitation space dependence mapping relationship by introducing a conditional random field, so that the influence of the time sequence on the detection result is effectively reduced, and the accuracy of the precipitation detection method based on big data is improved;
step S31, respectively calculating parameter difference values between adjacent detection parameters in the space-time detection model, and judging whether the parameter difference values are larger than a difference threshold value;
preferably, in the step, a comparator mode can be adopted to perform size calculation between the parameter difference and the difference threshold, and in this embodiment, the difference threshold can be set autonomously according to the requirements of users, so that the requirement of diversity of user data is effectively met;
when the parameter difference is larger than the difference threshold value in the step S31, performing a step S41;
step S41, judging that the detection parameters corresponding to the parameter difference values are singular values, deleting the detection parameters, and acquiring parameter sums between adjacent parameters of the detection parameters;
in the step, a deleting mode is adopted to prevent detection errors caused by error parameters, so that the accuracy of the detection method is improved;
s51, calculating the average value of the parameter sum to obtain a replacement value, and replacing the detection parameter with the replacement value;
in the step, data is supplemented in a mean value replacement mode, so that the data is complete, the integrity of the data is improved, and the user experience is improved;
s61, performing data optimization on the spatio-temporal detection model according to a local preset optimization rule to obtain a spatio-temporal optimization model;
the space-time detection model is subjected to data optimization design, so that the influence of interference factors on the space-time detection model is effectively prevented, and the accuracy of the detection result is further improved;
please refer to fig. 3, which is a flowchart illustrating an embodiment of step S61 in fig. 2:
step S610, calculating variance of the space-time detection model to obtain a variance value, and matching the variance value with an optimization table prestored locally to obtain first optimization fluctuation;
step S611, performing fluctuation optimization on the space-time detection model according to the first optimization fluctuation;
step S612, filtering calculation is carried out on the space-time detection model to obtain a filtering value, and the filtering value is matched with the optimization table to obtain second optimization fluctuation;
step S613, performing fluctuation optimization on the space-time detection model according to the second optimization fluctuation;
step S614, filtering variance calculation is carried out on the space-time detection model to obtain a filtering variance value, and the filtering variance value is matched with the optimization table to obtain a third optimization fluctuation;
step S615, carrying out fluctuation optimization on the space-time detection model according to the third optimization fluctuation;
continuing to refer to fig. 2, in step S71, performing model loss calculation on the spatio-temporal optimization model to obtain a loss function;
performing model loss calculation on the space-time optimization model to judge the accuracy of the space-time optimization model;
s81, when the loss function is judged to be smaller than a loss threshold value, outputting the space-time optimization model to obtain a detection result;
the detection result stores precipitation information corresponding to each area, and a user can predict precipitation based on the precipitation information;
preferably, in this embodiment, the calculation formula for calculating the model loss for the spatio-temporal optimization model is as follows:
in this embodiment, the design of the space-time detection model is established to convert the detection data into specific model parameters, so as to effectively facilitate subsequent processing and calculation of the data, the design of judging whether the singular value exists in the detection parameters is used to effectively and timely correct abnormal data so as to improve the accuracy of the detection result, the design of data optimization is performed on the space-time detection model so as to effectively prevent the influence of interference factors on the space-time detection model, so as to further improve the accuracy of the detection result, the model loss calculation is performed on the space-time optimization model so as to judge the accuracy of the space-time optimization model, and when the space-time optimization model is judged to be correct, a result is output.
Referring to fig. 4, a schematic structural diagram of a big data based precipitation detection system 100 according to a third embodiment of the present invention includes:
the modeling module 10 is used for acquiring detection data and establishing a space-time detection model according to the detection data, wherein the monitoring acquisition method of the detection data can be ground meteorological station monitoring, satellite monitoring or radar monitoring, the satellite monitoring is mainly detected in a satellite cloud picture mode and mainly reflects cloud top information, and the radar monitoring can complete airspace scanning within several minutes. The spatial resolution is about 1 kilometer, millions of groups of spatial monitoring data can be generated by each radar every time, radar reflection has a direct correlation with precipitation, and therefore detection data are acquired in the module in a radar monitoring mode.
The judging module 20 is configured to judge whether a singular value exists in a detection parameter in the spatio-temporal detection model, and correct the detection parameter when the singular value exists in the detection parameter, where the singular value is judged to exist in the detection parameter, so that abnormal data is effectively and timely corrected to improve accuracy of a detection result.
The optimization module 30 is configured to perform data optimization on the spatio-temporal detection model according to a local preset optimization rule to obtain a spatio-temporal optimization model, where the preset optimization rule stores a plurality of optimization conditions, the optimization conditions may be set according to user requirements, the optimization conditions may be data variance optimization, data fluctuation optimization, and the like, and by designing the spatio-temporal detection model through data optimization, the influence of interference factors on the spatio-temporal detection model is effectively prevented, and the accuracy of the detection result is further improved.
And the calculation module 40 is configured to perform model loss calculation on the spatio-temporal optimization model to obtain a loss function, and output the spatio-temporal optimization model to obtain a detection result when it is determined that the loss function is smaller than a loss threshold, where the accuracy of the spatio-temporal optimization model is determined by performing model loss calculation on the spatio-temporal optimization model, and the detection result stores precipitation information corresponding to each region, and a user can predict precipitation based on the precipitation information.
Preferably, the calculation formula adopted by the calculation module 40 for performing model loss calculation on the spatio-temporal optimization model is as follows:
specifically, in this embodiment, the modeling module 10 includes:
the first modeling unit 11 is configured to obtain a radar reflectivity factor combination stored in the detection data, and use MSE as an optimization target to establish a precipitation space dependency mapping relationship, where in this embodiment, in order to prevent the influence of the space phase relation on the detection result, the unit establishes the precipitation space dependency mapping relationship by introducing a space difference method, so as to effectively reduce the influence of the space phase relation on the detection result, and improve the accuracy of the precipitation detection method based on the big data.
The second modeling unit 12 is configured to obtain a correlation of the precipitation process stored in the detection data, and use the correlation of the precipitation process as a target to establish a precipitation time series mapping relationship, where in this embodiment, in order to prevent a time series from affecting a detection result, the unit establishes the precipitation space dependency mapping relationship by introducing a conditional random field, so as to effectively reduce an impact of the time series on the detection result, and improve the accuracy of the big-data-based precipitation amount detection method.
Further, the determining module 20 includes:
a first calculating unit 21 for calculating parameter differences between adjacent detection parameters, respectively;
a judging unit 22, configured to judge whether the parameter difference is greater than a difference threshold; if yes, the detection parameter corresponding to the parameter difference is judged to be a singular value, wherein the unit judges the accuracy of the corresponding detection parameter by calculating the parameter difference, and then effectively judges the data accuracy of the detection parameter in the space-time detection model, so that the accuracy of the detection method is improved.
Preferably, the judging module 20 further includes:
the obtaining unit 23 is configured to delete the detection parameter and obtain a sum of parameters between adjacent parameters of the detection parameter, where the deleting unit is used to prevent a detection error caused by an incorrect parameter, so as to improve accuracy of the detection method.
And the second calculating unit 24 is configured to calculate an average value of the parameter sum to obtain a replacement value, and replace the detection parameter with the replacement value, where data is supplemented in the unit by using an average value replacement method, so that the data is complete, the integrity of the data is further improved, and the user experience is improved.
Specifically, the optimization module 30 includes:
a first optimization unit 31, configured to perform variance calculation on the spatio-temporal detection model to obtain a variance value, match the variance value with a locally pre-stored optimization table to obtain a first optimization fluctuation, and perform fluctuation optimization on the spatio-temporal detection model according to the first optimization fluctuation;
a second optimization unit 32, configured to perform filtering calculation on the spatio-temporal detection model to obtain a filtered value, match the filtered value with the optimization table to obtain a second optimization fluctuation, and perform fluctuation optimization on the spatio-temporal detection model according to the second optimization fluctuation;
a third optimization unit 33, configured to perform filtering variance calculation on the spatio-temporal detection model to obtain a filtering variance value, match the filtering variance value with the optimization table to obtain a third optimization fluctuation, and perform fluctuation optimization on the spatio-temporal detection model according to the third optimization fluctuation.
In the above-mentioned rainfall detection system 100 based on big data, the modeling module 10 is used to establish the design of the space-time detection model, so as to convert the detection data into specific model parameters, and further to effectively facilitate the subsequent processing and calculation of data, the judging module 20 is used to judge whether the singular value design exists in the detection parameters, so as to effectively and timely correct abnormal data, so as to improve the accuracy of the detection result, the optimizing module 30 is used to design the data optimization of the space-time detection model, so as to effectively prevent the influence of interference factors on the space-time detection model, further improve the accuracy of the detection result, and the calculating module 40 is used to perform model loss calculation on the space-time optimization model, so as to judge the accuracy of the space-time optimization model, and output the result when the space-time optimization model is judged to be correct.
The embodiment also provides a mobile terminal, which includes a storage device and a processor, where the storage device is used to store a computer program, and the processor runs the computer program to make the mobile terminal execute the above method for detecting precipitation based on big data.
The present embodiment also provides a storage medium on which a computer program used in the above-mentioned mobile terminal is stored, which when executed, includes the steps of:
acquiring detection data, and establishing a space-time detection model according to the detection data;
judging whether a detection parameter in the space-time detection model has a singular value or not, and correcting the detection parameter when the detection parameter is judged to have the singular value;
performing data optimization on the space-time detection model according to a local preset optimization rule to obtain a space-time optimization model;
and performing model loss calculation aiming at the space-time optimization model to obtain a loss function, and outputting the space-time optimization model to obtain a detection result when the loss function is judged to be smaller than a loss threshold value. The storage medium, such as: ROM/RAM, magnetic disk, optical disk, etc.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is used as an example, in practical applications, the above-mentioned function distribution may be performed by different functional units or modules according to needs, that is, the internal structure of the storage device is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit, and the integrated unit may be implemented in a form of hardware, or may be implemented in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application.
Those skilled in the art will appreciate that the component configuration shown in fig. 4 does not constitute a limitation of the big data based precipitation detection system of the present invention and may include more or less components than those shown, or some components in combination, or a different arrangement of components, and that the big data based precipitation detection method of fig. 1-3 may also be implemented using more or less components than those shown in fig. 4, or some components in combination, or a different arrangement of components. The units, modules, etc. referred to in this disclosure are a series of computer programs that can be executed by a processor (not shown) in the big data based precipitation detection system and that can perform specific functions, and all of them can be stored in a storage device (not shown) of the big data based precipitation detection system.
The above-described embodiments describe the technical principles of the present invention, and these descriptions are only for the purpose of explaining the principles of the present invention and are not to be construed as limiting the scope of the present invention in any way. Based on the explanations herein, those skilled in the art will be able to conceive of other embodiments of the present invention without inventive effort, which would fall within the scope of the present invention.
Claims (5)
1. A big data-based precipitation detection method is characterized by comprising the following steps:
acquiring detection data, and establishing a space-time detection model according to the detection data;
judging whether a detection parameter in the space-time detection model has a singular value or not, and correcting the detection parameter when the detection parameter is judged to have the singular value;
performing data optimization on the space-time detection model according to a local preset optimization rule to obtain a space-time optimization model;
performing model loss calculation on the space-time optimization model to obtain a loss function, and outputting the space-time optimization model to obtain a detection result when the loss function is judged to be smaller than a loss threshold value;
the step of establishing a spatio-temporal detection model according to the detection data comprises:
acquiring radar reflectivity factor combinations stored in the detection data, and establishing a precipitation space dependence mapping relation by taking MSE as an optimization target;
acquiring the correlation of the precipitation process stored in the detection data, and establishing a precipitation time series mapping relation by taking the correlation of the precipitation process as a target;
the step of judging whether the detection parameters in the space-time detection model have singular values comprises the following steps:
respectively calculating parameter difference values between adjacent detection parameters, and judging whether the parameter difference values are larger than a difference threshold value;
and if so, judging that the detection parameters corresponding to the parameter difference values are singular values.
2. The big data based precipitation detection method according to claim 1, wherein said step of correcting said detection parameter comprises:
deleting the detection parameters and acquiring the parameter sum between the adjacent parameters of the detection parameters;
and calculating the average value of the parameter sum to obtain a replacement value, and replacing the detection parameter by the replacement value.
3. The big data based precipitation detection method according to claim 1, wherein said step of performing data optimization on said spatio-temporal detection model according to a local preset optimization rule comprises:
performing variance calculation on the space-time detection model to obtain a variance value, and matching the variance value with an optimization table prestored locally to obtain first optimization fluctuation;
performing fluctuation optimization on the space-time detection model according to the first optimization fluctuation;
carrying out filtering calculation on the space-time detection model to obtain a filtering value, and matching the filtering value with the optimization table to obtain second optimization fluctuation;
performing fluctuation optimization on the space-time detection model according to the second optimization fluctuation;
performing filtering variance calculation on the space-time detection model to obtain a filtering variance value, and matching the filtering variance value with the optimization table to obtain a third optimization fluctuation;
and carrying out fluctuation optimization on the space-time detection model according to the third optimization fluctuation.
4. A big data based precipitation detection system, comprising:
the modeling module is used for acquiring detection data and establishing a space-time detection model according to the detection data;
the judging module is used for judging whether the detection parameters in the space-time detection model have singular values or not and correcting the detection parameters when the singular values in the detection parameters are judged to exist;
the optimization module is used for carrying out data optimization on the space-time detection model according to a local preset optimization rule to obtain a space-time optimization model;
the calculation module is used for performing model loss calculation on the space-time optimization model to obtain a loss function, and outputting the space-time optimization model to obtain a detection result when the loss function is judged to be smaller than a loss threshold value;
the modeling module comprises:
the first modeling unit is used for acquiring radar reflectivity factor combinations stored in the detection data and establishing a precipitation space dependence mapping relation by taking MSE as an optimization target;
the second modeling unit is used for acquiring the correlation of the precipitation process stored in the detection data and establishing a precipitation time series mapping relation by taking the correlation of the precipitation process as a target;
the judging module comprises:
the first calculating unit is used for respectively calculating parameter difference values between the adjacent detection parameters;
the judging unit is used for judging whether the parameter difference value is larger than a difference value threshold value or not; and if so, judging that the detection parameters corresponding to the parameter difference values are singular values.
5. The big data-based precipitation detection system according to claim 4, wherein said determining module further comprises:
the acquisition unit is used for deleting the detection parameters and acquiring the parameter sum between the adjacent parameters of the detection parameters;
and the second calculating unit is used for calculating the average value of the parameter sum to obtain a replacement value and replacing the detection parameter with the replacement value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811617914.7A CN109614744B (en) | 2018-12-28 | 2018-12-28 | Big data-based precipitation detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811617914.7A CN109614744B (en) | 2018-12-28 | 2018-12-28 | Big data-based precipitation detection method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109614744A CN109614744A (en) | 2019-04-12 |
| CN109614744B true CN109614744B (en) | 2022-11-22 |
Family
ID=66011595
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811617914.7A Active CN109614744B (en) | 2018-12-28 | 2018-12-28 | Big data-based precipitation detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109614744B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111123410A (en) * | 2019-12-26 | 2020-05-08 | 国网北京市电力公司 | Precipitation monitoring system and method, storage medium and processor |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101647215A (en) * | 2007-03-29 | 2010-02-10 | 索尼株式会社 | Method of and apparatus for analyzing noise in a signal processing system |
| CN103948393A (en) * | 2014-05-21 | 2014-07-30 | 华侨大学 | Near-infrared noninvasive detection method and device for contents of blood components |
| CN104656163A (en) * | 2015-02-04 | 2015-05-27 | 中国人民解放军理工大学 | Rainfall distribution and dynamic measurement method based on big-data mobile communication network |
| CN104751580A (en) * | 2015-04-08 | 2015-07-01 | 武汉理工光科股份有限公司 | Distributed optical fiber sensing signal mode identifying method and system |
| CN105425319A (en) * | 2015-09-16 | 2016-03-23 | 河海大学 | Rainfall satellite rainstorm assimilation method based on ground measuring data correction |
| CN106127242A (en) * | 2016-06-21 | 2016-11-16 | 河海大学 | Year of based on integrated study Extreme Precipitation prognoses system and Forecasting Methodology thereof |
| CN107423190A (en) * | 2017-04-19 | 2017-12-01 | 国家电网公司 | A kind of daily record data points to recognition methods and device extremely |
| CN107703564A (en) * | 2017-10-13 | 2018-02-16 | 中国科学院深圳先进技术研究院 | A kind of precipitation predicting method, system and electronic equipment |
| CN108009632A (en) * | 2017-12-14 | 2018-05-08 | 清华大学 | Confrontation type space-time big data Forecasting Methodology |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103792524A (en) * | 2014-03-06 | 2014-05-14 | 兰州大学 | Radar quantitative precipitation estimation method based on cloud classification |
| CN104398271A (en) * | 2014-11-14 | 2015-03-11 | 西安交通大学 | Method using three-dimensional mechanics and tissue specific imaging of blood vessels and plaques for detection |
| CN104820222B (en) * | 2014-12-25 | 2017-08-29 | 邓勇 | The real-time dynamic attenuation of rainfall radar is corrected and precipitation computational methods |
| US10509868B2 (en) * | 2016-01-29 | 2019-12-17 | Fujitsu Limited | Operating a solar power generating system |
-
2018
- 2018-12-28 CN CN201811617914.7A patent/CN109614744B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101647215A (en) * | 2007-03-29 | 2010-02-10 | 索尼株式会社 | Method of and apparatus for analyzing noise in a signal processing system |
| CN103948393A (en) * | 2014-05-21 | 2014-07-30 | 华侨大学 | Near-infrared noninvasive detection method and device for contents of blood components |
| CN104656163A (en) * | 2015-02-04 | 2015-05-27 | 中国人民解放军理工大学 | Rainfall distribution and dynamic measurement method based on big-data mobile communication network |
| CN104751580A (en) * | 2015-04-08 | 2015-07-01 | 武汉理工光科股份有限公司 | Distributed optical fiber sensing signal mode identifying method and system |
| CN105425319A (en) * | 2015-09-16 | 2016-03-23 | 河海大学 | Rainfall satellite rainstorm assimilation method based on ground measuring data correction |
| CN106127242A (en) * | 2016-06-21 | 2016-11-16 | 河海大学 | Year of based on integrated study Extreme Precipitation prognoses system and Forecasting Methodology thereof |
| CN107423190A (en) * | 2017-04-19 | 2017-12-01 | 国家电网公司 | A kind of daily record data points to recognition methods and device extremely |
| CN107703564A (en) * | 2017-10-13 | 2018-02-16 | 中国科学院深圳先进技术研究院 | A kind of precipitation predicting method, system and electronic equipment |
| CN108009632A (en) * | 2017-12-14 | 2018-05-08 | 清华大学 | Confrontation type space-time big data Forecasting Methodology |
Non-Patent Citations (1)
| Title |
|---|
| "基于"3σ法则"的显著误差检测";李九龙等;《计算机与现代化》;20120131(第1期);第10-13页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109614744A (en) | 2019-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108761576B (en) | Data fusion method and system for X-band meteorological radar and rainfall station | |
| CN112965146B (en) | Quantitative precipitation estimation method combining meteorological radar and rainfall barrel observation data | |
| CN109029368B (en) | Image space compensation remote sensing image/SAR image high-precision geometric positioning post-processing method | |
| Sun et al. | Evaluation and correction of GPM IMERG precipitation products over the capital circle in Northeast China at multiple spatiotemporal scales | |
| CN108230375B (en) | Registration method of visible light image and SAR image based on structural similarity rapid robustness | |
| CN112197749B (en) | Cross calibration method and device for effective wave height of wave buoy | |
| CN113610729B (en) | Method, system and storage medium for correcting hyperspectral remote sensing image satellite-ground cooperative atmosphere | |
| CN112762888A (en) | Bridge space displacement monitoring method and system and readable storage medium | |
| CN109614744B (en) | Big data-based precipitation detection method and system | |
| CN115656989A (en) | External parameter calibration method and device, electronic equipment and storage medium | |
| Saponaro et al. | Evaluation of aerosol and cloud properties in three climate models using MODIS observations and its corresponding COSP simulator, as well as their application in aerosol–cloud interactions | |
| Zhong et al. | Cross-evaluation of reflectivity from the space-borne precipitation radar and multi-type ground-based weather radar network in China | |
| CN113742927B (en) | Meteorological forecast data quality detection method | |
| Crombaghs et al. | Assessing height precision of laser altimetry DEMs | |
| CN116029162B (en) | Flood disaster inundation range monitoring method and system by using satellite-borne GNSS-R data | |
| CN110532969B (en) | Slope unit dividing method based on multi-scale image segmentation | |
| JP2005017266A (en) | Method and device for radar pluviometry | |
| CN108919264B (en) | InSAR interferometric phase truth value determination and differential interferometry method | |
| CN114091562A (en) | Multi-sensing data fusion method, device, system, equipment and storage medium | |
| CN111915570A (en) | Atmospheric delay estimation method based on back propagation neural network | |
| CN116819561A (en) | Point cloud data matching method, system, electronic equipment and storage medium | |
| CN115685127A (en) | Method and device for analyzing settlement risk of target object based on point cloud data | |
| CN114646933B (en) | 4D millimeter wave radar pitching installation error static measurement method | |
| CN110907984A (en) | Method for detecting earthquake front infrared long-wave radiation abnormal information based on autoregressive moving average model | |
| CN113558536B (en) | Intelligent calibration method, device and system of intelligent sweeping robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200918 Address after: Hangzhou City, Zhejiang province 310018 Xiasha Higher Education Park No. 2 Street No. 928 Applicant after: ZHEJIANG SCI-TECH University Address before: 330013 Jiangxi Province, Nanchang Qingshan Lake East Shuanggang Street No. 808 South Campus of East China Jiaotong University Applicant before: East China Jiaotong University |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |