CN114564629B - Abnormal data processing method and device, computer equipment and storage medium - Google Patents
Abnormal data processing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114564629B CN114564629B CN202210428018.6A CN202210428018A CN114564629B CN 114564629 B CN114564629 B CN 114564629B CN 202210428018 A CN202210428018 A CN 202210428018A CN 114564629 B CN114564629 B CN 114564629B
- Authority
- CN
- China
- Prior art keywords
- data
- observation data
- meteorological
- abnormal data
- observation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/9035—Filtering based on additional data, e.g. user or group profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Abstract
The application relates to an abnormal data processing method, an abnormal data processing device, computer equipment and a storage medium. The method comprises the steps of obtaining a meteorological parameter time sequence, wherein the meteorological parameter time sequence comprises a plurality of meteorological observation data, carrying out filtering processing on the meteorological observation data, obtaining primary screening abnormal data, determining target abnormal data from the primary screening abnormal data according to a difference value between the primary screening abnormal data and reference observation data corresponding to the primary screening abnormal data, achieving the purpose of primarily screening the abnormal data from the meteorological observation data through filtering processing in a mathematical statistical sense, then determining the target abnormal data from the initially obtained abnormal data in an actual physical sense, guaranteeing the quality control effect of the meteorological observation data in both the mathematical statistical sense and the actual physical sense, and removing error values and abnormal values in the meteorological parameter time sequence, thereby improving the accuracy and reliability of the meteorological observation data.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to an abnormal data processing method and apparatus, a computer device, and a storage medium.
Background
In the oceanographic science, the collection of key meteorological parameters of a sea-air interface, such as the temperature, the air pressure, the air speed, the wind direction, the relative humidity and the like at the height of 3 meters at sea, provides observation data support for the deep exploration of the marine dynamic process, and has important significance in the research fields of sea-gas interaction research, disaster forecast early warning, marine environment guarantee and the like. In the actual meteorological parameter observation process, manual operation errors, environmental influences, unstable communication transmission and the like can interfere with meteorological observation results to cause data abnormity, the quality of field observation data is mostly problematic, the field observation data cannot be directly put into application, and the quality control needs to be performed on the field observation data to remove error values and abnormal values. However, the atmospheric system is active and changes rapidly, and the change range of key meteorological elements such as air temperature, air pressure, wind speed and relative humidity is large, so that the situation that correct meteorological observation data are judged to be abnormal often occurs in the traditional technology, and the abnormal data are difficult to accurately identify.
Disclosure of Invention
Therefore, in order to solve the above technical problems, it is necessary to provide an abnormal data processing method, an abnormal data processing apparatus, a computer device, and a storage medium, so as to accurately identify abnormal data and improve the accuracy and reliability of observation data.
In a first aspect, the present application provides an abnormal data processing method, including:
acquiring a meteorological parameter time sequence, wherein the meteorological parameter time sequence comprises a plurality of meteorological observation data;
filtering the meteorological observation data to obtain primary screening abnormal data;
determining target abnormal data from the primary screening abnormal data according to the difference value between the primary screening abnormal data and the reference observation data corresponding to the primary screening abnormal data; and the reference observation data and the preliminary screening abnormal data are meteorological observation data at adjacent moments.
In some embodiments of the present application, filtering the meteorological observation data to obtain prescreening abnormal data includes:
dividing meteorological observation data in a meteorological parameter time sequence into short sequence observation data with preset length by using a time window;
acquiring a median and a median absolute deviation corresponding to short sequence observation data in a time window;
determining the value range of the observation data in the time window according to the median and the median absolute deviation;
and determining the meteorological observation data exceeding the observation data value range in the short-sequence observation data as primary screening abnormal data.
In some embodiments of the present application, determining target abnormal data from the preliminary screening abnormal data according to a difference value between the preliminary screening abnormal data and reference observation data corresponding to the preliminary screening abnormal data includes:
sequentially determining any initially screened abnormal data as target observation data, and performing differential operation on the target observation data and reference observation data corresponding to the target observation data to obtain a differential value;
and if the difference value is greater than the preset difference threshold value, determining the target observation data as target abnormal data.
In some embodiments of the present application, the reference observation data includes first reference observation data and second reference observation data, where the first reference observation data is meteorological observation data at a time before the target observation data, and the second reference observation data is meteorological observation data at a time after the target observation data;
carrying out differential operation on the target observation data and the reference observation data corresponding to the target observation data to obtain a differential value, wherein the differential operation comprises the following steps:
carrying out differential operation on the target observation data and the first reference observation data to obtain a first differential value;
carrying out differential operation on the target observation data and the second reference observation data to obtain a second differential value;
if the difference value is greater than the preset difference threshold value, determining the target observation data as target abnormal data, including:
and if the first difference value and the second difference value are both larger than a preset difference threshold value, determining the target observation data as target abnormal data.
In some embodiments of the present application, obtaining a weather parameter time series includes:
acquiring meteorological initial data through a drift-type sea-air interface buoy;
preprocessing the meteorological initial data to obtain a meteorological parameter time sequence; among these, pre-processing includes, but is not limited to, repetitive tests and time-incremental tests.
In some embodiments of the present application, after determining the target abnormal data from the preliminary screening abnormal data, the method further includes:
dividing the target abnormal data into continuous target abnormal data and single target abnormal data according to the time sequence information of the target abnormal data;
if the target abnormal data is the single type of target abnormal data, performing interpolation processing on the target abnormal data;
and if the target abnormal data are continuous target abnormal data, rejecting the target abnormal data.
In some embodiments of the present application, the meteorological observations include, but are not limited to, air temperature observations, air pressure observations, wind speed observations, or relative humidity observations.
In a second aspect, the present application provides an exception data processing apparatus, comprising:
the observation data acquisition module is used for acquiring a meteorological parameter time sequence, and the meteorological parameter time sequence comprises a plurality of meteorological observation data;
the filtering processing module is used for filtering the meteorological observation data to obtain primary screening abnormal data;
the abnormal data determining module is used for determining target abnormal data from the primary screening abnormal data according to the difference value between the primary screening abnormal data and the reference observation data corresponding to the primary screening abnormal data; and the reference observation data and the primary screening abnormal data are meteorological observation data at adjacent moments.
In a third aspect, the present application further provides a computer device, comprising:
one or more processors;
a memory; and
one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the exception data handling method.
In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps in the method for exception data handling.
In a fifth aspect, embodiments of the present application provide a computer program product or a computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided by the first aspect.
The abnormal data processing method, the abnormal data processing device, the computer equipment and the storage medium have the advantages that by acquiring the meteorological parameter time sequence which comprises a plurality of meteorological observation data, filtering the meteorological observation data to obtain primary screening abnormal data, determining target abnormal data from the primary screening abnormal data according to a difference value between the primary screening abnormal data and reference observation data corresponding to the primary screening abnormal data, primarily screening abnormal data from the meteorological observation data by filtering in a mathematical statistic sense, then determining target abnormal data from the initially obtained abnormal data in a practical physical sense, the quality control effect of the meteorological observation data is guaranteed in two aspects of mathematical statistic significance and actual physical process, the correct meteorological observation data are prevented from being judged as abnormal data, the accuracy of identifying the abnormal data is improved, and the accuracy and the reliability of the meteorological observation data are improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic diagram of a scenario of an abnormal data processing method in an embodiment of the present application;
FIG. 2 is a flow chart of an abnormal data processing method in the embodiment of the present application;
FIG. 3 is a schematic flow chart of the step of obtaining the preliminary screening abnormal data in the embodiment of the present application;
FIG. 4 is a schematic flow chart diagram illustrating another abnormal data processing method according to an embodiment of the present application;
FIG. 5 is a schematic structural diagram of an abnormal data processing apparatus according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a computer device in an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the description of the present application, it is to be understood that the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.
In the description of the present application, the word "for example" is used to mean "serving as an example, instance, or illustration". Any embodiment described herein as "for example" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the invention. In the following description, details are set forth for the purpose of explanation. It will be apparent to one of ordinary skill in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and processes are not shown in detail to avoid obscuring the description of the invention with unnecessary detail. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
In the embodiment of the present application, it should be further described that the abnormal data processing method provided in the embodiment of the present application may be applied to an abnormal data processing system as shown in fig. 1. The abnormal data processing system includes a terminal 100 and a server 200, wherein the terminal 100 may be a weather detecting device, such as an automatic weather station, a buoy provided with a weather detecting device, or the like. The server 200 may be an independent server, or may be a server network or a server cluster composed of servers, which includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets, or a cloud server composed of a plurality of servers. Among them, the Cloud server is constituted by a large number of computers or web servers based on Cloud Computing (Cloud Computing).
Those skilled in the art will appreciate that the application environment shown in fig. 1 is only one application scenario related to the present application, and does not constitute a limitation on the application scenario of the present application, and that other application environments may further include more or less computer devices than those shown in fig. 1, for example, only 1 server 200 is shown in fig. 1, and it is understood that the exception data processing system may further include one or more other servers, which are not limited herein. In addition, as shown in FIG. 1, the exception data handling system may also include a memory for storing data.
It should be further noted that the scenario diagram of the abnormal data processing system shown in fig. 1 is only an example, and the abnormal data processing system and the scenario described in the embodiment of the present invention are for more clearly illustrating the technical solution of the embodiment of the present invention, and do not form a limitation on the technical solution provided in the embodiment of the present invention.
Referring to fig. 2, an embodiment of the present application provides an abnormal data processing method, which is mainly illustrated by applying the method to the server 200 in fig. 1, and the method includes steps S210 to S230, which are specifically as follows:
s210, acquiring a meteorological parameter time sequence, wherein the meteorological parameter time sequence comprises a plurality of meteorological observation data.
The meteorological observation data refers to data related to air hydrogeology dynamic or static meteorological, such as observation data of meteorological elements such as air temperature, air pressure, wind speed and relative humidity; the meteorological parameter time series refers to a series formed by arranging meteorological observation data according to the sequence of the acquisition time.
Specifically, the weather parameter time series or the weather observation data may be acquired by a weather detection device, such as an automatic weather station, a buoy provided with the weather detection device, or the like. For another example, for a sea-Air Interface scene, meteorological observation data of the sea-Air Interface can be collected by a Drifting Air-sea Interface Buoy (DrIB).
Further, in an embodiment, the acquiring of the weather parameter time series may specifically be acquiring of weather initial data by a drift-type sea-air interface buoy; preprocessing the meteorological initial data to obtain a meteorological parameter time sequence; among these, pre-processing includes, but is not limited to, repetitive tests and time-incremental tests.
In the actual meteorological observation data acquisition process, manual operation errors, environmental factor influences, communication transmission instability and the like can interfere with observation results to cause data abnormity, and the acquired meteorological initial data often has quality problems and cannot be directly put into application. Therefore, after the meteorological initial data are acquired by the drift type sea-air interface buoy, the meteorological initial data can be preprocessed to remove abnormal data with obvious errors.
For example, the weather observation data should be arranged in a time sequence, and in order to prevent the occurrence of the time sequence error, the time increment inspection may be performed on the weather initial data, that is, the time sequence information of the weather initial data is subjected to the increment inspection to check whether the time sequence information (such as year, month, day, time, minute, second, and the like) of the weather initial data is monotonically increased all the time, so as to ensure that the data time sequence is normal.
For another example, if the communication transmission process is unstable or the data storage module fails during the operation of the drifting sea-air interface buoy, a problem of storing two or more pieces of observation data at the same time may occur, which may cause data repetition errors. Therefore, repeated inspection can be carried out on the meteorological initial data, repeated observation data are removed through repeated inspection, and the observation time and observation results are guaranteed to be in one-to-one correspondence.
For another example, in consideration of observation indexes of instrument design, observation data acquired by the drift type sea-air interface buoy is often within a certain measurement range; therefore, the range of the initial meteorological data can be checked, if the initial meteorological data are not in the measuring range corresponding to the drifting ocean air interface buoy, the initial meteorological data can be marked as abnormal data, and the initial meteorological data can be deleted or adjusted. For example, the measurement range of the drift type sea air interface buoy to the air temperature is [ -40, 60], and if the initial meteorological data records that the air temperature value is 70 degrees, the initial meteorological data can be marked as abnormal data and deleted.
And S220, filtering the meteorological observation data to obtain primary screening abnormal data.
The meteorological observation data which are continuously observed always obey a certain probability distribution, so that the meteorological observation data can be further inspected by using a statistical inspection method, the meteorological observation data can be subjected to filtering processing, an arithmetic mean value and a standard deviation can be used as judgment standards, and the meteorological observation data of which the difference value with the arithmetic mean value is greater than a preset multiple (such as 3 times) of the standard deviation are marked as primary screening abnormal data; specifically, the arithmetic mean and the standard deviation of the meteorological parameter time series may be calculated, then the difference between the arithmetic mean and any one of the meteorological observation data in the meteorological parameter time series may be calculated, and if the difference is greater than the standard deviation of a preset multiple (e.g., 3 times), the meteorological observation data may be marked as primary screening abnormal data.
Further, considering that the atmospheric system is active and changes rapidly, and the meteorological observation data of meteorological elements such as air temperature, air pressure, wind speed, and relative humidity change significantly, in an embodiment, as shown in fig. 3, the step S220 may specifically include:
s310, dividing meteorological observation data in a meteorological parameter time sequence into short sequence observation data with preset length by using a time window;
s320, acquiring a median and a median absolute deviation corresponding to the short sequence observation data in the time window;
s330, determining the value range of the observation data in the time window according to the median and the median absolute deviation;
and S340, determining the meteorological observation data which exceed the observation data value range in the short-sequence observation data as primary screening abnormal data.
The length of the time window can be set according to actual conditions; for example, if the data sampling interval of the meteorological observation data in the time series of meteorological parameters is 1 hour, the length of the time window may be set to 24 hours, and the resulting short series of observation data within the time window includes 24 meteorological observation data. Specifically, the time window is moved according to a certain data step length, for example, the time window can be moved sequentially according to the step length of one meteorological observation data at a time until the last meteorological observation data of the meteorological parameter time sequence, so that the meteorological observation data in the meteorological parameter time sequence is divided into a plurality of short sequence observation data, and the deviation caused by the atmospheric environment change of different sea areas is effectively reduced under the conditions of rapid atmospheric environment change and large meteorological observation data fluctuation degree. In addition, in a scene of a meteorological parameter time sequence obtained through a drift type ocean air interface buoy, the meteorological parameter time sequence is segmented by using a time window, so that drift observation time and consideration of continuity and regularity of space change are added into identification abnormal data, the deviation caused by space position dynamic change and atmospheric environment difference of different sea areas is reduced, and the accuracy of identification abnormal data is ensured.
The median absolute deviation refers to the median of the difference between each meteorological observation data and the median in the short-sequence observation data. Specifically, for any short sequence observation data, a median and a median absolute deviation corresponding to the short sequence observation data can be calculated, and then the median and the median absolute deviation are used as judgment standards to determine the observation data value range of the short sequence observation data in the time window according to the median and the median absolute deviation.
For example, assume that there is a time series of meteorological parameters
The data sampling interval is 1h, the time window is set to one day, and the data can be obtained
Is a set of short-sequence observations,
and obtaining a plurality of groups of short sequence observation data by analogy in sequence for a group of short sequence observation data.
To be provided with
Taking the group of short sequence observation data as an example, calculating the median and the median absolute deviation of the group of short sequence observation data; wherein the median is
The absolute deviation of the median is the MAD,
，
。
the length of the value can be determined according to the absolute deviation of the median
，
The value range of the observed data in the time window is
I.e. when the meteorological observation data is a median distance
Over 3 times
And if so, the meteorological observation data is abnormal observation data.
The method has the advantages that the meteorological observation data in the meteorological parameter time sequence are segmented by using the time window, the median and the median absolute deviation are used as the judgment standard of the abnormal observation data aiming at the segmented short sequence observation data, the influence of extreme outlier data in the meteorological parameter time sequence on data filtering is reduced, the condition that the correct meteorological observation data is judged to be abnormal is avoided, meanwhile, the drifting observation time and the consideration of the continuity and regularity of the spatial variation are added, the deviation caused by the dynamic change of the spatial position or the difference of the atmospheric environments of different sea areas is effectively reduced, and the accuracy of identifying the abnormal data is improved.
S230, determining target abnormal data from the primary screening abnormal data according to the difference value between the primary screening abnormal data and the reference observation data corresponding to the primary screening abnormal data; and the reference observation data and the primary screening abnormal data are meteorological observation data at adjacent moments.
The reference observation data refers to meteorological observation data which are adjacent to the primary screening abnormal data in time sequence dimension; for example, the prescreening abnormal data is meteorological observation data at the time t, and the reference observation data includes, but is not limited to, meteorological observation data at the time (t + 1) and/or meteorological observation data at the time (t-1).
The abnormal observation data obtained by filtering the meteorological observation data are abnormal data determined from the mathematical statistics significance level, and when the atmospheric environment is relatively stable, the fluctuation degree of the meteorological observation data is not large, so that the secondary abnormal inspection can be performed on the primarily screened abnormal data in order to reduce the misjudgment caused by filtering. Specifically, local anomaly detection can be performed on the primary screening anomaly data, and target anomaly data can be determined from the primary screening anomaly data according to a difference value between the primary screening anomaly data and the adjacent meteorological observation data.
Further, in an embodiment, determining the target abnormal data from the primary screening abnormal data according to a difference value between the primary screening abnormal data and the reference observation data corresponding to the primary screening abnormal data includes: sequentially determining any initially screened abnormal data as target observation data, and performing differential operation on the target observation data and reference observation data corresponding to the target observation data to obtain a differential value; and if the difference value is greater than a preset difference threshold value, determining the target observation data as target abnormal data.
The abnormal observation data obtained by filtering the meteorological observation data are often multiple, and for any one of the primary screening abnormal data, the difference operation can be performed on the primary screening abnormal data and the reference observation data corresponding to the target observation data, so as to detect whether the primary screening abnormal data is the target abnormal data or not according to the difference value.
Wherein, predetermine the difference threshold value and can set up according to actual conditions, furtherly, for avoiding meteorological detection equipment self observation error to disturb the judgement, predetermine the difference threshold value and should be greater than meteorological detection equipment's observation error.
In one embodiment, the reference observation data includes first reference observation data and second reference observation data, wherein the first reference observation data is meteorological observation data at a previous moment of the target observation data, and the second reference observation data is meteorological observation data at a later moment of the target observation data; the method for carrying out differential operation on the target observation data and the reference observation data corresponding to the target observation data to obtain a differential value comprises the following steps: carrying out differential operation on the target observation data and the first reference observation data to obtain a first differential value; carrying out differential operation on the target observation data and the second reference observation data to obtain a second differential value; if the difference value is greater than the preset difference threshold value, determining the target observation data as target abnormal data, including: and if the first difference value and the second difference value are both larger than a preset difference threshold value, determining the target observation data as target abnormal data.
Specifically, as described above, the prescreening abnormal data is assumed to be meteorological observation data at time t, and the reference observation data includes, but is not limited to, first reference observation data at time (t-1) and second reference observation data at time (t + 1). After the primary screening abnormal data are obtained, sequentially determining any primary screening abnormal data as target observation data, and obtaining first reference observation data and first reference observation data corresponding to the target observation data; calculating a difference value between the target observation data and the first reference observation data to obtain a first difference value, and calculating a difference value between the target observation data and the second reference observation data to obtain a second difference value; and then comparing the first difference value and the second difference value with a preset difference threshold value, and determining the target observation data as target abnormal data when the first difference value and the second difference value are both greater than the preset difference threshold value.
The abnormal data processing method comprises the steps of obtaining a meteorological parameter time sequence, wherein the meteorological parameter time sequence comprises a plurality of meteorological observation data, filtering the meteorological observation data to obtain primary screened abnormal data, determining target abnormal data from the primary screened abnormal data according to a difference value between the primary screened abnormal data and reference observation data corresponding to the primary screened abnormal data, primarily screening abnormal data from the meteorological observation data through filtering in a mathematical statistic sense, then determining target abnormal data from the initially obtained abnormal data in a practical physical sense, the quality control effect of the meteorological observation data is guaranteed in two aspects of mathematical statistic significance and actual physical process, the correct meteorological observation data are prevented from being judged as abnormal data, the accuracy of identifying the abnormal data is improved, and the accuracy and the reliability of the meteorological observation data are improved.
In one embodiment, as shown in fig. 4, after determining the target abnormal data from the preliminary screening abnormal data, the method further includes:
s204, dividing the target abnormal data into continuous target abnormal data and single target abnormal data according to the time sequence information of the target abnormal data;
s205, if the target abnormal data is the single type target abnormal data, performing interpolation processing on the target abnormal data;
and S206, if the target abnormal data is continuous target abnormal data, rejecting the target abnormal data.
The time sequence information of the target abnormal data refers to the acquisition time of the target abnormal data; the continuous target anomaly data is a plurality of meteorological observation data collected for a continuous time on the time series information, and the single target anomaly data is a single or two meteorological observation data on the time series information.
Specifically, in order to not destroy the statistical characteristics and the variation trend of the data and keep the continuity of the meteorological observation data as much as possible, for single type target abnormal data, linear interpolation can be carried out on the meteorological observation data at the front and back adjacent moments based on the single type target abnormal data, and the target abnormal data is replaced by the result obtained by the linear interpolation; and the continuous target abnormal data is deleted.
In the following, the above abnormal data processing method is further explained by taking DrIB as an example of a terminal for an air interface scene.
DrIB has rich observation variables, complete records of buoy operation conditions and marine environment changes, instrument design observation indexes and operation postures are considered in the quality control process on the basis of observation data, advantages of platform observation records are played, and abnormal data can be found out more quickly and accurately.
The method comprises the following steps of firstly preprocessing initial meteorological data aiming at initial meteorological data such as air temperature, air pressure, wind speed and relative humidity, wherein the preprocessing comprises but is not limited to the following basic quality control steps:
(1) and (4) land position inspection. Unlike marine fixed-point observation, due to ship overtaking and the like, DrIB may start to operate before entering water, send land observation information, and directly cause data abnormality of meteorological elements such as air temperature, air pressure and the like, and such errors usually exist only in the initial stage of a continuous time sequence. Therefore, land position inspection needs to be performed firstly, invalid data of the buoy at the stage of not entering water are eliminated, and observation is guaranteed to reflect the real marine environment. Specifically, the working environment can be accurately judged by judging whether the swing angle amplitude of the buoy conforms to the underwater motion attitude, the sea surface temperature and the stable working period numerical value have fault difference, land observation information is removed, and meanwhile, an independent land observation information file is generated, so that later inspection and checking are facilitated.
(2) And (5) repeating the test. In the DrIB operation process, if the communication transmission process is unstable or the data storage module fails, a problem of storing two or more pieces of observation data at the same time may occur, resulting in data repetition errors. And (3) removing repeated observation data by applying repeated inspection, and ensuring the one-to-one relation between observation time and observation results. The repeated data files are generated at the same time, so that the duplicate disk check is facilitated.
(3) And (5) carrying out a time increment test. Under normal conditions, the DrIB observation data are arranged according to a time sequence, and in order to prevent the occurrence of the 'time backflow', the time sequence information such as year, month, day, time, minute, second and the like is subjected to incremental inspection to check whether the time sequence information is increased monotonously all the time, so that the time sequence information of the observation data is ensured to be normal. If the observation data with disordered time sequence information is detected, the observation data is deleted, the time sequence is adjusted if necessary, and the deletion and adjustment actions of the time sequence related logs are recorded in real time.
(4) And (6) checking the range. The range check is based on basic cognition of geographical knowledge and general rules of oceanographic elements and the observation capability of the buoy, whether the observation data are reasonable or not is effectively judged, and if the observation data are invalid, the observation data are marked as abnormal data. E.g., relative humidity up to 100%, etc. The DrIB platform design observation indexes are as shown in table 1, and the data beyond the observation range in the meteorological initial data is set as a set W.
TABLE 1 drift type design deviation index for buoy observation of sea-air interface
After the basic quality control of the data is completed, aiming at meteorological elements, obvious errors in a meteorological parameter time sequence are eliminated.
The continuity and the gradient of a data sequence are determined by the working characteristics of DrIB 'wave-following flow-by-flow', then the meteorological observation data in the meteorological parameter time sequence are subjected to one-step targeted quality control, specifically, problem data violating the continuous trend of the sequence is found out based on the continuity judgment criterion, and the specific steps are as follows:
the atmospheric system is more active and changes rapidly, and the development rule characteristics of key meteorological elements such as air temperature, air pressure, wind speed, relative humidity and the like are more obvious. When the peak inspection method is applied to detecting meteorological observation data, the situation that a large amount of correct data is judged to be abnormal due to the fact that the difference between the meteorological observation data and the judgment standard data is larger and larger is found, and the method cannot be directly applied.
Statistically, it is theorized that meteorological variables observed continuously at fixed points tend to follow a certain probability distribution, and therefore data can be examined using statistical tests, such as the rhineda method (3 δ criterion), considering data with a residual error from an arithmetic mean value exceeding three times the standard deviation as an abnormal value. However, when the arithmetic mean value and the standard deviation are used as the judgment standard, the result deviation is easily caused by extreme outlier data; in addition, the spatial position of the DrIB is dynamically changed, and the atmospheric environments of different sea areas are greatly different, so that the significance of the statistical characteristic research of a long-time observation sequence is small. Therefore, based on the continuity and regularity of the drift observation time and space change, the filtering processing can be carried out on the meteorological observation data by taking the day as a time window unit and utilizing the Hampel filter to judge the degree of the meteorological observation data departing from the daily variability, the initially screened abnormal data is obtained, and the interference of extreme values in a meteorological parameter time sequence on the identification of the abnormal data is avoided.
In particular, it is assumed that the meteorological parameter time series is
The data sampling interval is 1h, the time window is set to one day, and the data can be obtained
Is a set of short-sequence observations,
and obtaining a plurality of groups of short sequence observation data by analogy in sequence for a group of short sequence observation data.
To be provided with
Taking the group of short sequence observation data as an example, calculating the median and the median absolute deviation of the group of short sequence observation data; wherein the median is
The absolute deviation of the median is the MAD,
，
。
from the absolute deviation of the median
The value range of the observed data in the time window is
I.e. when the meteorological observation data is a median distance
Over 3 times
And if so, the meteorological observation data is abnormal observation data.
And filtering the meteorological observation data to obtain primary screening abnormal data which are recorded as a set Q1.
It can be understood that the initially screened abnormal data is abnormal data determined from the aspect of mathematical statistics, if the atmospheric environment is relatively stable, the fluctuation degree of the observation parameters is balanced, and the local conversion MAD value is small, so that the condition that the correct data is mistakenly judged as abnormal is easily caused.
In order to avoid the misjudgment risk, local anomaly detection is further introduced, and any meteorological observation data in the set Q1 is subjected to
If the difference operation result between the observation data and any adjacent moment does not exceed the preset difference threshold value, the method comprises the following steps
The meteorological observation data which is misjudged as abnormal data is judged, if the difference operation result of the meteorological observation data and observation data at any adjacent moment exceeds a preset difference threshold value, the meteorological observation data is judged to be abnormal data, and the difference operation result of the meteorological observation data is judged to be abnormal data
Abnormal meteorological observation data. Let the meteorological observation data set misjudged as anomalous data be Q2. In order to avoid the interference judgment of the observation error of the instrument, the preset difference threshold value is set to be a value larger than the buoy design observation error.
Therefore, the set of abnormal meteorological observation data includes the data left after eliminating the data of the set Q2, among all the data of the set W plus the set Q1. In order to keep the continuity of observation as much as possible on the basis of not damaging the statistical characteristics and the variation trend of the data, for single abnormal data or two abnormal data, the result of linear interpolation of observation data at the front and back adjacent moments is used for substitution; and deleting more than two continuous abnormal data.
In order to better implement the abnormal data processing method provided in the embodiment of the present application, on the basis of the abnormal data processing method provided in the embodiment of the present application, an abnormal data processing apparatus is further provided in the embodiment of the present application, as shown in fig. 5, the abnormal data processing apparatus 500 includes:
an observation data obtaining module 510, configured to obtain a weather parameter time series, where the weather parameter time series includes a plurality of weather observation data;
the filtering processing module 520 is used for filtering the meteorological observation data to obtain primary screening abnormal data;
an abnormal data determining module 530, configured to determine target abnormal data from the preliminary screening abnormal data according to a difference value between the preliminary screening abnormal data and reference observation data corresponding to the preliminary screening abnormal data; and the reference observation data and the preliminary screening abnormal data are meteorological observation data at adjacent moments.
In some embodiments of the present application, the filtering processing module is specifically configured to segment the meteorological observation data in the meteorological parameter time series into short-series observation data with a preset length by using a time window; acquiring a median and a median absolute deviation corresponding to short sequence observation data in a time window; determining the value range of the observation data in the time window according to the median and the median absolute deviation; and determining the meteorological observation data exceeding the observation data value range in the short-sequence observation data as primary screening abnormal data.
In some embodiments of the application, the abnormal data determining module is specifically configured to sequentially determine any preliminarily screened abnormal data as target observation data, and perform differential operation on the target observation data and reference observation data corresponding to the target observation data to obtain a differential value;
and if the difference value is greater than a preset difference threshold value, determining the target observation data as target abnormal data.
In some embodiments of the present application, the reference observation data includes first reference observation data and second reference observation data, where the first reference observation data is meteorological observation data at a time before the target observation data, and the second reference observation data is meteorological observation data at a time after the target observation data; the abnormal data determining module is specifically used for carrying out differential operation on the target observation data and the first reference observation data to obtain a first differential value; carrying out differential operation on the target observation data and the second reference observation data to obtain a second differential value; and if the first difference value and the second difference value are both larger than a preset difference threshold value, determining the target observation data as target abnormal data.
In some embodiments of the present application, the observation data obtaining module is specifically configured to collect initial meteorological data through a drift-type ocean air interface buoy; preprocessing the meteorological initial data to obtain a meteorological parameter time sequence; among these, pre-processing includes, but is not limited to, repetitive tests and time-incremental tests.
In some embodiments of the present application, the exception data processing apparatus further includes an exception data processing module, configured to divide the target exception data into continuous type target exception data and single type target exception data according to timing information of the target exception data; if the target abnormal data is single target abnormal data, performing interpolation processing on the target abnormal data; and if the target abnormal data are continuous target abnormal data, rejecting the target abnormal data.
The abnormal data processing device obtains the meteorological parameter time sequence, the meteorological parameter time sequence comprises a plurality of meteorological observation data, the meteorological observation data are subjected to filtering processing, primary screening abnormal data are obtained, target abnormal data are determined from the primary screening abnormal data according to the difference value between the primary screening abnormal data and reference observation data corresponding to the primary screening abnormal data, the abnormal data are primarily screened from the meteorological observation data through filtering processing in the mathematical statistical sense, then the target abnormal data are determined from the initially obtained abnormal data in the actual physical sense, the quality control effect of the meteorological observation data is guaranteed in the mathematical statistical sense and the actual physical process, the error values and the abnormal values in the meteorological parameter time sequence are removed, and therefore the accuracy and the reliability of the meteorological observation data are improved.
In some embodiments of the present application, the exception data handling apparatus 500 may be implemented in the form of a computer program which may be run on a computer device as shown in fig. 6. The memory of the computer device may store various program modules constituting the abnormal data processing apparatus 500, such as an observation data acquiring module 510, a filtering processing module 520, and an abnormal data determining module 530 shown in fig. 5. The computer program constituted by the respective program modules causes the processor to execute the steps in the abnormal data processing method of the embodiments of the present application described in the present specification.
For example, the computer device shown in fig. 6 may execute step S210 by the observation data acquiring module 510 in the abnormal data processing apparatus 500 shown in fig. 5. The computer device may perform step S220 through the filter processing module 520. The computer device may perform step S530 through the abnormal data determining module 530. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external computer device through a network connection. The computer program is executed by a processor to implement an exception data handling method.
Those skilled in the art will appreciate that the architecture shown in fig. 6 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In some embodiments of the present application, there is provided a computer device comprising one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to perform the steps of the above-described exception data handling method. Here, the steps of the abnormal data processing method may be steps in the abnormal data processing method of each of the above embodiments.
In some embodiments of the present application, a computer-readable storage medium is provided, which stores a computer program, and the computer program is loaded by a processor, so that the processor executes the steps of the above-mentioned abnormal data processing method. Here, the steps of the abnormal data processing method may be steps in the abnormal data processing method of each of the above embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the computer program is executed. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM may take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), for example.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The foregoing describes in detail an abnormal data processing method, an abnormal data processing apparatus, a computer device, and a storage medium provided in the embodiments of the present application, and a specific example is applied in the present application to explain the principle and the implementation of the present invention, and the description of the foregoing embodiments is only used to help understanding the method and the core idea of the present invention; meanwhile, for those skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (7)
1. An exception data processing method, comprising:
acquiring meteorological initial data, and swing angle amplitude data and sea surface temperature corresponding to each initial meteorological observation data in the meteorological initial data through a drift type ocean air interface buoy; if the amplitude data of the swing angle corresponding to the initial meteorological observation data do not accord with the motion attitude data in water and the sea surface temperature corresponding to the initial meteorological observation data is different from other sea surface temperatures in a fault manner, determining the initial meteorological observation data as the meteorological observation data with the data observation position as the land position, and removing the meteorological observation data with the data observation position as the land position to obtain a meteorological parameter time sequence, wherein the meteorological parameter time sequence comprises a plurality of meteorological observation data;
filtering the meteorological observation data to obtain primary screening abnormal data;
determining target abnormal data from the primary screening abnormal data according to the difference value between the primary screening abnormal data and reference observation data corresponding to the primary screening abnormal data; the reference observation data and the preliminary screening abnormal data are meteorological observation data at adjacent moments;
wherein, it handles to said meteorological observation data carries out filtering, obtains the preliminary screening abnormal data, includes:
dividing meteorological observation data in the meteorological parameter time sequence into short sequence observation data with preset length by using a time window;
acquiring a median and a median absolute deviation corresponding to the short sequence observation data in the time window;
determining the value range of the observation data in the time window according to the median and the median absolute deviation; wherein the observation data has a value range of [ q ] me -3d m ,q me +3d m ]Wherein d is m =1.4826MAD，q me Representing the median, and MAD representing the absolute deviation of the median;
determining meteorological observation data which exceed the observation data value range in the short-sequence observation data as primary screening abnormal data;
the determining target abnormal data from the preliminary screening abnormal data according to the difference value between the preliminary screening abnormal data and the reference observation data corresponding to the preliminary screening abnormal data includes:
sequentially determining any initially screened abnormal data as target observation data, and performing differential operation on the target observation data and first reference observation data to obtain a first differential value;
carrying out differential operation on the target observation data and second reference observation data to obtain a second differential value;
if the first differential value and the second differential value are both greater than a preset differential threshold value, determining the target observation data as target abnormal data; the first reference observation data is meteorological observation data at a previous moment of the target observation data, and the second reference observation data is meteorological observation data at a later moment of the target observation data.
2. The method of claim 1, wherein after collecting the initial meteorological data via the drift-type ocean air interface buoy, further comprising:
preprocessing the meteorological initial data to obtain a meteorological parameter time sequence; wherein the pre-processing includes, but is not limited to, a repeat test and a time-increment test.
3. The method according to any one of claims 1 to 2, wherein after determining target abnormal data from the preliminary screening abnormal data, the method further comprises:
dividing the target abnormal data into continuous target abnormal data and single target abnormal data according to the time sequence information of the target abnormal data;
if the target abnormal data is single target abnormal data, performing interpolation processing on the target abnormal data;
and if the target abnormal data are continuous target abnormal data, rejecting the target abnormal data.
4. The method of any one of claims 1 to 2, wherein the meteorological observations comprise, but are not limited to, air temperature observations, air pressure observations, wind speed observations or relative humidity observations.
5. An exception data handling apparatus, said apparatus comprising:
the observation data acquisition module is used for acquiring meteorological initial data, and swing angle amplitude data and sea surface temperature corresponding to each initial meteorological observation data in the meteorological initial data through a drift type ocean air interface buoy; if the swing angle amplitude data corresponding to the initial meteorological observation data do not conform to the underwater motion attitude data and the sea surface temperature corresponding to the initial meteorological observation data is different from the other sea surface temperatures in a fault mode, determining the initial meteorological observation data as the meteorological observation data with the data observation position as the land position, and removing the meteorological observation data with the data observation position as the land position to obtain a meteorological parameter time sequence, wherein the meteorological parameter time sequence comprises a plurality of meteorological observation data;
the filtering processing module is used for filtering the meteorological observation data to obtain primary screening abnormal data;
the abnormal data determining module is used for determining target abnormal data from the primary screening abnormal data according to a difference value between the primary screening abnormal data and reference observation data corresponding to the primary screening abnormal data; the reference observation data and the preliminary screening abnormal data are meteorological observation data at adjacent moments;
the filtering processing module is used for segmenting meteorological observation data in the meteorological parameter time sequence into short sequence observation data with preset length by using a time window; acquiring a median and a median absolute deviation corresponding to the short sequence observation data in the time window; determining the value range of the observation data in the time window according to the median and the median absolute deviation; wherein the observation data has a value range of [ q ] me -3d m ,q me +3d m ]Wherein d is m =1.4826MAD，q me Representing the median, and MAD representing the absolute deviation of the median; determining meteorological observation data which exceed the observation data value range in the short-sequence observation data as primary screening abnormal data;
the abnormal data determining module is used for sequentially determining any preliminarily screened abnormal data as target observation data, and carrying out differential operation on the target observation data and first reference observation data to obtain a first differential value; carrying out differential operation on the target observation data and second reference observation data to obtain a second differential value; if the first differential value and the second differential value are both greater than a preset differential threshold value, determining the target observation data as target abnormal data; the first reference observation data is meteorological observation data at a previous moment of the target observation data, and the second reference observation data is meteorological observation data at a later moment of the target observation data.
6. A computer device, characterized in that the computer device comprises:
one or more processors;
a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the exception data handling method of any of claims 1 to 4.
7. A computer-readable storage medium, having stored thereon a computer program which is loaded by a processor to perform the steps of the method of exception data handling of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210428018.6A CN114564629B (en) | 2022-04-22 | 2022-04-22 | Abnormal data processing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210428018.6A CN114564629B (en) | 2022-04-22 | 2022-04-22 | Abnormal data processing method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114564629A CN114564629A (en) | 2022-05-31 |
| CN114564629B true CN114564629B (en) | 2022-08-16 |
Family
ID=81721014
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210428018.6A Active CN114564629B (en) | 2022-04-22 | 2022-04-22 | Abnormal data processing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114564629B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115454981A (en) * | 2022-08-30 | 2022-12-09 | 山东师范大学 | Data cleaning and marking method and system for meteorological data of Antarctic device |
| CN116028809A (en) * | 2022-12-27 | 2023-04-28 | 北京津发科技股份有限公司 | Continuous blood pressure measurement model training and measuring method and device |
| CN117131455B (en) * | 2023-10-23 | 2024-01-19 | 华清安泰能源股份有限公司 | Method, system, device and storage medium for processing operation parameters of ground temperature field |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202177419U (en) * | 2011-07-25 | 2012-03-28 | 国家海洋局第二海洋研究所 | Global position system-real time kinematic (GPS-RTK) real-time sea level measuring system based on ship posture correction |
| CN104821064A (en) * | 2014-11-27 | 2015-08-05 | 王忠亮 | Bathing beach data acquisition and alarming method |
| CN110362724A (en) * | 2019-07-23 | 2019-10-22 | 国家卫星海洋应用中心 | A kind of data filtering method, device, electronic equipment and readable storage medium storing program for executing |
| CN112305572A (en) * | 2020-10-14 | 2021-02-02 | 自然资源部第一海洋研究所 | Satellite microwave radiometer wet delay evaluation method for space decorrelation |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002098775A (en) * | 2000-09-26 | 2002-04-05 | Mitsubishi Heavy Ind Ltd | Observation data inspecting device |
| CN102938021B (en) * | 2012-11-02 | 2015-10-28 | 云南大学 | A kind of powerline ice-covering load quantitative is estimated and Forecasting Methodology |
| CN110019595B (en) * | 2017-09-29 | 2023-09-22 | 中国电力科学研究院 | Multi-source meteorological data integration method and system |
| CN109668635B (en) * | 2019-01-16 | 2020-08-07 | 中国人民解放军61741部队 | Sea surface temperature fusion method and system |
| CN110188810A (en) * | 2019-05-23 | 2019-08-30 | 西北工业大学 | A kind of detection of online outlier, identification and modification method based on Density Clustering |
| CN111275307B (en) * | 2020-01-16 | 2023-09-05 | 生态环境部华南环境科学研究所 | Quality control method for high-frequency continuous observation data of automatic online station for water quality |
| CN113704693B (en) * | 2021-08-19 | 2024-02-02 | 华北电力大学 | High-precision effective wave height data estimation method |
-
2022
- 2022-04-22 CN CN202210428018.6A patent/CN114564629B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202177419U (en) * | 2011-07-25 | 2012-03-28 | 国家海洋局第二海洋研究所 | Global position system-real time kinematic (GPS-RTK) real-time sea level measuring system based on ship posture correction |
| CN104821064A (en) * | 2014-11-27 | 2015-08-05 | 王忠亮 | Bathing beach data acquisition and alarming method |
| CN110362724A (en) * | 2019-07-23 | 2019-10-22 | 国家卫星海洋应用中心 | A kind of data filtering method, device, electronic equipment and readable storage medium storing program for executing |
| CN112305572A (en) * | 2020-10-14 | 2021-02-02 | 自然资源部第一海洋研究所 | Satellite microwave radiometer wet delay evaluation method for space decorrelation |
Non-Patent Citations (2)
| Title |
|---|
| FZF3-1型海洋资料浮标系统;卜照蓬等;《海洋技术》;20030615(第02期);全文 * |
| 我国无缝隙精细化网格天气预报技术进展与挑战;金荣花等;《气象》;20190421(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114564629A (en) | 2022-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114564629B (en) | Abnormal data processing method and device, computer equipment and storage medium | |
| CN114492680B (en) | Buoy data quality control method and device, computer equipment and storage medium | |
| CN112735094A (en) | Geological disaster prediction method and device based on machine learning and electronic equipment | |
| CN111080976B (en) | Method and device for monitoring natural gas leakage in real time under temperature change scene | |
| CN112395382A (en) | Ship abnormal track data detection method and device based on variational self-encoder | |
| CN111767957A (en) | Method and device for detecting log abnormity, storage medium and electronic equipment | |
| KR102575917B1 (en) | IoT sensor abnormality diagnosing method and system using cloud-based virtual sensor | |
| CN114780644A (en) | Ship navigation data processing method, device, equipment and storage medium | |
| CN111914424B (en) | Design wind speed value taking method and system based on short-term wind measurement data | |
| CN112949735A (en) | Liquid hazardous chemical substance volatile concentration abnormity discovery method based on outlier data mining | |
| CN110751201B (en) | SAR equipment task failure cause reasoning method based on textural feature transformation | |
| CN112202630A (en) | Network quality abnormity detection method and device based on unsupervised model | |
| CN115063337A (en) | Intelligent maintenance decision-making method and device for buried pipeline | |
| CN112926636A (en) | Method and device for detecting abnormal temperature of traction converter cabinet body | |
| KR102158100B1 (en) | Auto monitoring method and apparatus by using anomaly detection | |
| CN115659271A (en) | Sensor abnormality detection method, model training method, system, device, and medium | |
| CN115982591A (en) | Method and device for detecting health state of transformer, equipment and storage medium | |
| CN109870404A (en) | A kind of awning Structural Damage Identification, device and terminal device | |
| CN112749035A (en) | Anomaly detection method, device and computer readable medium | |
| CN112597425B (en) | Method, device, equipment and medium for detecting abnormal data in flow velocity measurement data | |
| CN116878728B (en) | Pressure sensor fault detection analysis processing system | |
| CN117686682B (en) | Indoor gas fault monitoring method and system | |
| CN117668684B (en) | Power grid electric energy data anomaly detection method based on big data analysis | |
| CN117114500B (en) | Behavior baseline establishment method, system and medium based on data decomposition | |
| CN116448062B (en) | Bridge settlement deformation detection method, device, computer and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |