CN111752936A - Data detection management method, device, server and readable storage medium - Google Patents
Data detection management method, device, server and readable storage medium Download PDFInfo
- Publication number
- CN111752936A CN111752936A CN202010609050.5A CN202010609050A CN111752936A CN 111752936 A CN111752936 A CN 111752936A CN 202010609050 A CN202010609050 A CN 202010609050A CN 111752936 A CN111752936 A CN 111752936A
- Authority
- CN
- China
- Prior art keywords
- data
- processed
- detection
- server
- rule
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 196
- 238000007726 management method Methods 0.000 title abstract description 45
- 230000002159 abnormal effect Effects 0.000 claims abstract description 42
- 238000000034 method Methods 0.000 claims abstract description 39
- 238000010606 normalization Methods 0.000 claims description 17
- 230000005856 abnormality Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012508 change request Methods 0.000 claims description 9
- 230000007613 environmental effect Effects 0.000 description 29
- VNWKTOKETHGBQD-UHFFFAOYSA-N methane Chemical compound C VNWKTOKETHGBQD-UHFFFAOYSA-N 0.000 description 16
- 238000012545 processing Methods 0.000 description 16
- 230000004048 modification Effects 0.000 description 11
- 238000012986 modification Methods 0.000 description 11
- CURLTUGMZLYLDI-UHFFFAOYSA-N Carbon dioxide Chemical compound O=C=O CURLTUGMZLYLDI-UHFFFAOYSA-N 0.000 description 10
- 230000009286 beneficial effect Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- 239000002689 soil Substances 0.000 description 8
- 229910002092 carbon dioxide Inorganic materials 0.000 description 5
- 239000001569 carbon dioxide Substances 0.000 description 5
- 230000008859 change Effects 0.000 description 5
- 238000012423 maintenance Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000005855 radiation Effects 0.000 description 4
- 238000001556 precipitation Methods 0.000 description 3
- RAHZWNYVWXNFOC-UHFFFAOYSA-N Sulphur dioxide Chemical compound O=S=O RAHZWNYVWXNFOC-UHFFFAOYSA-N 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- MGWGWNFMUOTEHG-UHFFFAOYSA-N 4-(3,5-dimethylphenyl)-1,3-thiazol-2-amine Chemical compound CC1=CC(C)=CC(C=2N=C(N)SC=2)=C1 MGWGWNFMUOTEHG-UHFFFAOYSA-N 0.000 description 1
- 239000002028 Biomass Substances 0.000 description 1
- 208000032368 Device malfunction Diseases 0.000 description 1
- CBENFWSGALASAD-UHFFFAOYSA-N Ozone Chemical compound [O-][O+]=O CBENFWSGALASAD-UHFFFAOYSA-N 0.000 description 1
- 239000000443 aerosol Substances 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 239000007900 aqueous suspension Substances 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- JCXJVPUVTGWSNB-UHFFFAOYSA-N nitrogen dioxide Inorganic materials O=[N]=O JCXJVPUVTGWSNB-UHFFFAOYSA-N 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 239000004576 sand Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Chemical compound O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/31—User authentication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
Abstract
The application provides a data detection management method, a data detection management device, a server and a readable storage medium. The method comprises the following steps: acquiring a current data set collected by site equipment; determining a detection rule corresponding to the data type of the data to be processed in the current data set as a target detection rule based on the corresponding relation between the data type and the detection rule; detecting the data to be processed through a target detection rule to obtain a detection result of the data to be processed; and when the detection result represents that the data to be processed is normal, taking the data to be processed as target data and recording the target data into the database. In the scheme, the data to be processed is detected through the detection rules corresponding to the data types, the data to be processed of different data types can be detected in a differentiated mode, in addition, the data to be processed which is detected normally is put into a warehouse, the abnormal data can be prevented from being put into the warehouse by the server, and the effectiveness and the reliability of the data which are put into the warehouse are improved.
Description
Technical Field
The invention relates to the technical field of computer data processing, in particular to a data detection management method, a data detection management device, a server and a readable storage medium.
Background
In field environmental observation, it is usually necessary to detect environmental data with corresponding sensors. The detected environmental data includes, but is not limited to, meteorological data, soil data, light radiation levels, and the like. During the observation process, the detected data is usually stored in the server for subsequent analysis processing. At present, the server is limited by the existing detection mode, the received environment data cannot be identified and managed, abnormal environment data are easy to store, and subsequent analysis work is influenced.
Disclosure of Invention
The application provides a data detection management method, a data detection management device, a server and a readable storage medium, which can solve the problem that abnormal data is easy to store due to the fact that received data cannot be identified and managed.
In order to achieve the above purpose, the technical solutions provided in the embodiments of the present application are as follows:
in a first aspect, an embodiment of the present application provides a data detection management method, where the method includes:
acquiring a current data set collected by site equipment;
determining a detection rule corresponding to the data type of the data to be processed in the current data set as a target detection rule based on the corresponding relation between the data type and the detection rule;
detecting the data to be processed through the target detection rule to obtain a detection result of the data to be processed;
and when the detection result represents that the data to be processed is normal, taking the data to be processed as target data and recording the target data into a database.
In the embodiment, the data to be processed is detected through the detection rules corresponding to the data types, so that the data to be processed of different data types can be detected in a differentiated mode.
With reference to the first aspect, in some optional embodiments, the method further comprises:
and when the detection result represents that the data to be processed is abnormal, sending prompt information to the site equipment, wherein the prompt information is used for the site equipment to send new data to be processed to a server based on the prompt information.
In the above embodiment, when the detection result is abnormal, the station device sends a prompt by feeding back the detection result, which is beneficial for the station device to send new data to be processed to the server based on the prompt information, so as to avoid the repeated occurrence of the same type of abnormal data.
With reference to the first aspect, in some optional embodiments, the method further comprises:
in a detection period, when the number of times of repeatedly sending the data to be processed by the site equipment is greater than or equal to a specified number of times, and a detection result of the data to be processed sent each time represents that the data to be processed is abnormal, and the data type of the data to be processed is a specified type, acquiring a management authority from the site equipment;
and recording the data to be processed into the database based on the management authority.
In the above embodiment, when the data to be processed is of the designated type, it usually indicates that the data to be processed is special data, and at this time, if the detection of the data to be processed cannot pass all the time, an administrator needs to be contacted to obtain a corresponding authority, and then the data to be processed passes the detection and is stored in a storage.
With reference to the first aspect, in some optional embodiments, the target detection rule includes a sub-rule for detecting at least one of data integrity, data normalization, data accuracy, data consistency, and data submission timeliness, and the target detection rule is used to detect the data to be processed to obtain a detection result of the data to be processed, including:
detecting at least one of data integrity, data normalization, data accuracy, data consistency and data submission timeliness of the data to be processed through the target detection rule;
when the at least one type of the data to be processed is detected normally, obtaining a detection result representing that the data to be processed is normal;
and when any one of the at least one class of the data to be processed is abnormal, obtaining a detection result representing the abnormality of the data to be processed.
In the above embodiment, data integrity, data normalization, data accuracy, data consistency, and data submission timeliness can be used as a basis for determining whether the data to be processed is abnormal, and whether the data is abnormal can be determined quickly and accurately by detecting at least one item.
With reference to the first aspect, in some optional embodiments, the method further comprises:
based on the received change request, the corresponding relation between the specified data category and the corresponding detection rule is changed.
In the above embodiment, the user may change the correspondence between the data category and the detection rule, which is beneficial to improving the flexibility of detection.
With reference to the first aspect, in some optional embodiments, the method further comprises:
based on a viewing request received from a terminal device, judging whether the user permission corresponding to the viewing request is greater than or equal to the permission of the viewing content corresponding to the viewing request;
and when the user permission is larger than or equal to the permission of the viewing content corresponding to the viewing request, sending the viewing content corresponding to the viewing request to the terminal equipment.
In the above embodiment, by authenticating the user authority, it is beneficial to improve the security and confidentiality of the viewing action and improve the information security level of the data.
With reference to the first aspect, in some optional embodiments, the obtaining the current data set collected by the site device includes:
and acquiring a current data set collected by at least one site device from at least one site device every preset time interval.
In a second aspect, an embodiment of the present application further provides a data detection management apparatus, where the apparatus includes:
the data acquisition unit is used for acquiring a current data set collected by the site equipment;
the determining unit is used for determining a detection rule corresponding to the data type of the data to be processed in the current data set as a target detection rule based on the corresponding relation between the data type and the detection rule;
the detection unit is used for detecting the data to be processed through the target detection rule to obtain a detection result of the data to be processed;
and the storage unit is used for taking the data to be processed as target data and recording the target data into a database when the detection result represents that the data to be processed is normal.
In a third aspect, an embodiment of the present application further provides a server, where the server includes a memory and a processor coupled to each other, where the memory stores a computer program, and when the computer program is executed by the processor, the server is caused to perform the method described above.
In a fourth aspect, the present application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program runs on a computer, the computer is caused to execute the above method.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below. It is appreciated that the following drawings depict only certain embodiments of the application and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.
Fig. 1 is a schematic communication connection diagram of a data detection management system according to an embodiment of the present application.
Fig. 2 is a schematic flow chart of a data detection management method according to an embodiment of the present application.
Fig. 3 is a schematic diagram of a detection rule provided in the embodiment of the present application.
Fig. 4 is a functional block diagram of a data detection management apparatus according to an embodiment of the present application.
Icon: 10-a data detection management system; 20-a server; 30-station equipment; 100-data detection management means; 110-a data acquisition unit; 120-a determination unit; 130-a detection unit; 140-warehousing unit.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. It should be noted that the terms "first," "second," and the like are used merely to distinguish one description from another, and are not intended to indicate or imply relative importance.
The embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the features of the embodiments can be combined with each other without conflict.
Referring to fig. 1, a data detection management system 10 provided in the present application may include a server 20 and a site device 30. Wherein, the server 20 can establish a communication connection with at least one site device 30 through the network for data interaction. The site device 30 may be used to collect environmental data and upload the environmental data to the server 20, among other things. The number of the station devices 30 communicatively connected to the server 20 may be set according to actual circumstances, and may be one or more, and is not particularly limited herein.
In this embodiment, the environmental data includes, but is not limited to, methane concentration, carbon dioxide concentration, soil humiture, precipitation, light radiation amount, vegetation coverage, surface reflectivity, aerosol optical thickness, atmospheric water vapor content, surface temperature, soil heat flux, forest aboveground biomass, water suspension concentration, near-surface nitrogen dioxide concentration, tropospheric ozone concentration, tropospheric and near-surface sulfur dioxide concentration, and the like. The site device 30 may include sensors for collecting one or more types of environmental data. The number and types of sensors included in the station device 30 may be set according to actual conditions. For example, the environmental data detected by the station device 30 includes carbon dioxide concentration and soil temperature and humidity, and the station device 30 may include a sensor for detecting carbon dioxide concentration and a sensor for detecting soil temperature and humidity.
The types of the environmental data collected by the different site devices 30 may be the same or different, and may be determined according to actual situations. Generally, when it is required to detect the environment of a region (the region may be divided according to actual conditions, and may be a village, a county, a forest, a mountain, or the like), one or more site devices 30 may be provided in the detection region to detect the environment data of the region. In addition, the station apparatuses 30 of different detection regions may upload the detected environment data to the server 20.
In this embodiment, each site device 30 may aggregate the detected environmental data and then periodically transmit the aggregated environmental data to the server 20. The time for periodically transmitting the environment data can be set according to actual conditions. For example, the period may be 24 hours, a week, a month, etc. Alternatively, the station device 30 may directly transmit the detected environment data to the server 20 without periodically aggregating to transmit the environment data to the server 20. It is understood that the manner in which the station device 30 transmits the environment data to the server 20 may be determined according to practice and is not particularly limited herein.
The server 20 may store the data in a specified format after acquiring the environment data transmitted by the site device 30. The specified format can be determined according to actual conditions. For example, the server 20 may store the environment data in a CSV (Comma-separated values) file format.
In this embodiment, the server 20 may include a storage module and a processing module coupled to each other, the storage module stores a computer program, and when the computer program is executed by the processing module, the server 20 may execute each step in the data detection management method described below.
It is understood that the server 20 may also include other modules. For example, the server 20 may further include a communication module for establishing a communication connection with the station device 30. The processing module, the communication module and the storage module are electrically connected directly or indirectly to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines.
Referring to fig. 2, an embodiment of the present application further provides a data detection management method, which can be applied to the data detection management system 10, and the server 20 in the data detection management system 10 executes each step in the method. The method may include steps S210 to S240 as follows:
step S210, acquiring a current data set collected by the station device 30;
step S220, determining a detection rule corresponding to the data type of the data to be processed in the current data set as a target detection rule based on the corresponding relation between the data type and the detection rule;
step S230, detecting the data to be processed through the target detection rule to obtain a detection result of the data to be processed;
and S240, when the detection result represents that the data to be processed is normal, taking the data to be processed as target data and recording the target data into a database.
In the scheme, the data to be processed is detected through the detection rules corresponding to the data types, the data to be processed of different data types can be detected in a differentiated mode, in addition, the data to be processed which is detected normally is put into a warehouse, the abnormal data can be prevented from being put into the warehouse by the server 20, the problem that the abnormal data are easily stored due to the fact that the received data cannot be identified and managed can be solved, and effectiveness and reliability of the data which are put into the warehouse are improved.
The individual steps in the process are explained in detail below, as follows:
in step S210, the server 20 may acquire the current data set collected by the site device 30. The current data set may be understood as a set of all environmental data detected by the station device 30 at the current time or within a specified time period from the current time. The specified time can be determined according to actual conditions, and can be 24 hours, one month and the like. Understandably, the data category of the environmental data included in the current data set may be determined according to actual conditions, and may include, but is not limited to, the above-mentioned methane concentration, carbon dioxide concentration, soil temperature and humidity, precipitation, light radiation amount, vegetation coverage, and the like. The site device 30 may automatically take the currently collected/detected environmental data as the current data set and send it to the server 20, so that the server 20 acquires the current data set.
In this embodiment, the data format of the current data set stored by the server 20 is usually a specified file format, for example, each type of environment data in the current data set is a CSV file format. Understandably, if the formats of various types of environment data in the data set transmitted by the site device 30 are not specified file formats, the server 20 may convert the data formats of the environment data into the specified formats. The server 20 is beneficial to subsequent analysis processing by uniformly standardizing the acquired data format.
As an alternative implementation, step S210 may include: and acquiring a current data set collected by at least one station device 30 from at least one station device 30 every a preset time interval.
In this embodiment, the preset time duration may be the specified time duration, and may be determined according to an actual situation. For example, the preset time period may be a 24-hour, one-week, etc. time period. The server 20 may obtain the current data set collected by the site device 30 every preset time period.
Understandably, the site device 30 may send the environmental data detected/collected by the site device 30 within a preset time period to the server 20 at every preset time period as the current data set of the site device 30. Alternatively, the server 20 may obtain, from the site device 30, the environmental data detected/collected by the site device 30 within a preset time period at every preset time period.
After the server 20 acquires the current data set of the site device 30, when the server 20 acquires the data set from the site device 30 next time, the acquired data of the server 20 is a set of environmental data collected/detected by the site device 30 within a preset time period after the current time. Based on this, the server 20 can acquire the environmental data in the current period from the station device 30 in time.
In step S220, the data category of the environmental data included in the current data set may be determined according to actual conditions. The electronic device stores the corresponding relation between the corresponding data type and the detection rule in advance. The detection rules of the environmental data of different data types can be the same or different, and can be flexibly set by a user according to actual conditions. The administrator can subsequently change the detection rule corresponding to the data type according to the actual situation.
After obtaining the current data set, the server 20 may determine the data type of each piece of data to be processed in the current data set. Then, based on the corresponding relationship between the data type and the detection rule and the data type of the data to be processed, the corresponding detection rule of the data to be processed can be determined. And the detection rule corresponding to the data type of the data to be processed is the target detection rule of the data to be processed.
In step S230, after determining the target detection rule of the data to be processed, the server 20 may perform targeted detection on the data to be processed based on the target detection rule, and then obtain a detection result of the data to be processed.
Referring to fig. 3, the detection rule may be determined according to actual conditions. For example, the detection rules may include, but are not limited to, sub-rules for detecting at least one of integrity, normalization, accuracy, consistency, and timeliness of submission of the data to be processed. Understandably, each detection rule may include sub-rules that detect one or more of integrity, normalization, accuracy, consistency, and timeliness of submission of the data to be processed. For example, the detection rules may include sub-rules that simultaneously detect integrity, normalization, accuracy, consistency, and timeliness of submission of the data to be processed. Alternatively, the detection rule may only include sub-rules that simultaneously detect the integrity, normalization, and timeliness of submission of the data to be processed.
The target detection rule includes sub-rules for detecting at least one of data integrity, data normalization, data accuracy, data consistency, and data submission timeliness, and step S230 may include:
detecting at least one of data integrity, data normalization, data accuracy, data consistency and data submission timeliness of the data to be processed through the target detection rule;
when the at least one type of the data to be processed is detected normally, obtaining a detection result representing that the data to be processed is normal;
when any one of the at least one class of the data to be processed is abnormal, obtaining a detection result representing the abnormality of the data to be processed;
or when any one of the data integrity, the data normalization and the data submission timeliness of the data to be processed is abnormal, obtaining a detection result representing the abnormality of the data to be processed.
For each sub-rule in the target detection rule, for convenience of distinguishing, the sub-rule for detecting the integrity of the data may be referred to as a first type of sub-rule; the sub-rule for detecting the data normalization can be called a second type of sub-rule; the sub-rule for detecting the data accuracy can be called a third type of sub-rule; the sub-rule for detecting data consistency may be referred to as a fourth class of sub-rule; the rules that detect timeliness of data submission may be referred to as a fifth category of sub-rules.
One detection cycle per station device 30 can be understood as one of the above-mentioned predetermined interval durations, for example, 24 hours.
The first type of sub-rule may include a rule for detecting integrity of the data to be processed, for example, whether the number of data of the data to be processed reaches a corresponding specified number is detected, when the number of data of the data to be processed reaches the specified number, it is determined that the integrity of the data to be processed is normal, and when the number of data of the data to be detected is less than the specified number, it is determined that the integrity of the data to be processed is abnormal. The specified number can be set according to actual conditions, and can be 10, 100, and the like, and is not particularly limited herein.
For each detection period, the first-class sub-rule may further include a rule for detecting the integrity of the current data set, and is configured to determine whether the current data set includes the environmental data of the specified data class, and determine that the integrity of the current data set is abnormal if the current data set lacks the environmental data of the specified data class. And if the current data set comprises the environmental data of the specified data type, determining that the integrity of the current data set is normal. The specified data category may be determined according to actual conditions, and is not particularly limited herein.
The second type of sub-rule may include a fine rule for detecting a data format of the environment data, a name of the data variable. The names of the variable elements need to be refined to layers, such as 1 meter wind speed, and 2 meter wind speeds are respectively WS _1 and WS _ 2. The second type of sub-rule may detect whether the data format and the data variable name of the environment data correspond to the specified data format and variable name. If the current data format does not correspond to the specified data format or the current variable name does not correspond to the specified variable name, determining that the data normalization is abnormal; and the current data format corresponds to the specified data format, and the current variable name corresponds to the specified variable name, so that the data normalization is determined to be normal. The specified data format and the variable name may be determined according to actual conditions, and are not limited herein. In addition, the detection method is well known to those skilled in the art and will not be described herein. When the data to be processed detects a normative abnormality of the data, the server 20 may return the data to be processed, so that the station device 30 uploads the data to be processed again, or after returning the data to be processed, the station device 30 uploads the data to be processed after maintenance and modification are performed by a manager for the abnormality.
The third type of sub-rules may include fine rules for detecting whether the data to be processed is null, whether a numerical value is out of bounds, and whether data redundancy occurs. When detecting that the data to be processed is null, or the numerical value is out of range, or the data redundancy occurs, determining that the data accuracy of the data to be processed is abnormal, and sending a prompt to the station device 30. And when detecting that the data to be processed is not null, and numerical value out-of-range does not occur, and data redundancy does not occur, determining that the data accuracy of the data to be processed is normal.
If the value of the data to be processed does not exist or is 0, determining that the data to be processed is a null value; and if the value of the data to be processed exists or is not 0, determining that the data to be processed is not a null value. Null values are derived from calculating null values, transmitting null values, and instrument failure null values (e.g., "NAN", "-999"). If the value of the data to be processed is in the corresponding preset range, determining that the numerical value is not out of range; if the value of the data to be processed is not within the corresponding preset range, the numerical value is determined to be out of range, the preset range can be set according to the data type of the data to be processed, the data to be processed can be determined according to the actual situation, and the data to be processed is not specifically limited. And if the data redundancy is that the time stamp, the variable name and the observed value are completely the same, determining that the data redundancy exists in the data to be processed, otherwise, determining that the data redundancy does not exist in the data to be processed.
The fourth type of sub-rule may include a fine rule for detecting data consistency such as acquisition frequency consistency, time consistency and the like of the data to be processed. For each type of data to be processed, when each type of data consistency is detected to be normal, determining that the data consistency of the type of data to be processed is normal; otherwise, determining that the data consistency of the data to be processed in the class is abnormal.
The detection mode of the acquisition frequency consistency can be as follows: calculating the acquisition frequency of the current data to be processed according to the timestamp of each data to be processed acquired by the station equipment 30, comparing the acquisition frequency with the calibrated acquisition frequency, judging whether the acquisition frequency is consistent with the calibrated acquisition frequency, and if the acquisition frequency of the current data to be processed is not consistent with the calibrated acquisition frequency, determining that the consistency of the acquisition frequency is abnormal; and if the acquisition frequency of the current data to be processed is consistent with the calibration acquisition frequency, determining that the consistency of the acquisition frequency is normal, wherein the calibration acquisition frequency can be set according to the actual situation, and is not particularly limited. In addition, for the data to be processed without setting the calibration acquisition frequency, the calibration frequency may be replaced by a statistical mode of the actual acquisition frequency of the data within a period of time (the time length may be set according to actual conditions).
The detection mode of the time consistency can be as follows: for example, for meteorological data, if the checking of whether the meteorological data meets the rules within a certain time exceeds a certain range, the meteorological data is determined to be suspicious. For example, for the ground temperature and the soil/sand temperature at different depths of the ground, the maximum temperature variation allowed in 1 minute is the corresponding temperature threshold. For example, the maximum temperature change allowed within one minute of the ground temperature is 5 ℃; the maximum temperature variation allowed within one minute of the ground temperature with the ground depth of 5 cm is 1 ℃; the maximum temperature variation allowed within one minute of the ground temperature with the ground depth of 50 cm is 0.1 ℃, and if the maximum temperature variation within one minute of the ground temperature with the corresponding depth exceeds the corresponding temperature threshold, the time consistency is determined to be abnormal; and if the maximum temperature variation within one minute does not exceed the corresponding temperature threshold, determining that the time consistency is normal.
A fifth category of sub-rules may include rules for detecting whether the data to be processed is submitted on time. For example, it is determined whether the time at which the data to be processed is submitted to the server 20 is within a specified time range. If the data to be processed and the current data set are submitted to the server 20 within a specified time range, or the server 20 acquires the current data set or the data to be processed within a specified time range, determining that the timeliness of data submission is normal; otherwise, it is determined that the timeliness of data submission of the data to be processed is abnormal, and at this time, the site device 30 may be notified to automatically submit the current data set to the server 20, or a manager may be notified to submit the current data set to the server 20 as soon as possible. The specified time range may be set according to the actual situation, specifying a morning submission every day, or specifying a month end submission every month.
It should be noted that the detection rule and the corresponding detection manner shown in fig. 3 are for facilitating understanding of an exemplary description of a method implementation flow, and in other embodiments, a sub-rule included in the detection rule may be different from that shown in fig. 3, and is not limited herein.
In step S240, when the detection result indicates that the data to be processed is normal, the data to be processed is valid and normative, and may be recorded into the database for subsequent analysis and processing. Understandably, in the subsequent analysis processing process, the environment data adopted by the user is the target data input in the database. Based on this, because the environmental data in the database is preprocessed, abnormal data is filtered out, and the effectiveness of the environmental data is improved, thereby being beneficial to improving the accuracy and the reliability of subsequent analysis and processing. The subsequent analysis processing may be determined according to actual conditions, for example, based on various types of historical environmental data, analyzing future changes of various types of environmental data in the detection area, or evaluating environmental quality, and the like.
As an optional implementation, the method may further include: when the detection result represents that the data to be processed is abnormal, sending a prompt message to the site device 30, so that the site device 30 sends new data to be processed to the server 20 based on the prompt message.
Understandably, when detecting that the data to be processed is abnormal, the server 20 may send prompt information to the station device 30 or the management terminal, the prompt information including details of the abnormal data, for example, a data type of the abnormal data, a reason of the abnormality (such as detection that does not satisfy the first-class detection rule, for example, the number of data is not qualified), and the like. The administrator can find the details of the abnormal data in time through the prompt information of the site device 30 or the management terminal. Then, maintenance or modification is carried out according to the abnormal reason so as to avoid the same abnormal condition again. After maintenance by the administrator, the station device 30 may retransmit the maintenance-modified data to be processed, and then transmit the data to the server 20 as new data to be processed.
Alternatively, when abnormality of the data to be processed is detected, the station device 30 retransmits new data to be processed to the server 20. When the detection result is abnormal, the station device 30 is fed back to send a prompt, which is beneficial for the station device 30 to send new data to be processed to the server 20 based on the prompt information, so as to avoid the repeated occurrence of the same type of abnormal data.
In this implementation, the method may further include: and generating a detection report combined with the current data aiming at the detection result of each to-be-processed data in the current data set. The generated detection report may be stored in the server 20. In addition, the detection report may also be sent by the server 20 to the site device 30 for an administrator of the site device 30 to view. The detection report may include the detection result of each piece of data to be processed. The administrator can perform detection maintenance on the station device 30 based on the detection result of the abnormality to see whether or not the corresponding sensor of the detection device malfunctions.
In the above embodiment, by feeding back a detection report to the station device 30, or sending a prompt message to the station device 30 or the management terminal when the data to be processed is detected to be abnormal, it is beneficial for a manager to investigate the reason of the abnormality based on the prompt message, and further beneficial for fundamentally solving the problem of data abnormality/quality.
As an optional implementation, the method may further include: in a detection period, when the number of times of repeatedly sending the data to be processed by the site device 30 is greater than or equal to a specified number of times, and a detection result of the data to be processed sent each time represents that the data to be processed is abnormal, and a data type of the data to be processed is a specified type, acquiring a management authority from the site device 30; and recording the data to be processed into the database based on the management authority.
In this embodiment, the specified type may be determined according to actual conditions. The specified times can be set according to actual conditions. For example, the specified type may be, but is not limited to, one or more of methane concentration, carbon dioxide concentration, soil humiture, precipitation amount, light radiation amount, vegetation coverage. The number of designation may be 3 times, 4 times, etc.
Understandably, when the data to be processed is of a specified type, the data to be processed is usually indicated as special data, which indicates that the data needs to be put in storage even if the data is abnormal. At this time, if the data to be processed is retransmitted for many times and each detection cannot pass, the administrator needs to be contacted to obtain the corresponding authority, and then the data to be processed passes the detection and is put into a warehouse. Therefore, the flexibility of data storage is improved, and the situation that special data cannot be recorded into the database due to abnormity is avoided.
As an optional implementation, the method may further include: based on the received change request, the corresponding relation between the specified data category and the corresponding detection rule is changed.
Understandably, the manager can send a change request to the server 20 through the station device 30 or the management terminal. The server 20 may parse the change request after receiving the change request. For example, if the modification request is for modifying the corresponding relationship between the specified data type and the corresponding detection rule, the server 20 responds to the modification request to modify the corresponding relationship between the specified data type and the corresponding detection rule. The management terminal may be, but is not limited to, a personal computer, a smart phone, and the like.
For example, the data to be processed is the methane concentration, and the target detection rule of the data category of the methane concentration includes the first category sub-rule, the second category sub-rule, the third category sub-rule, and the fifth category sub-rule. If the administrator needs to add the fourth type sub-rule to the target detection rule of the data category of methane concentration, the administrator may send a change request for changing the detection rule of methane concentration to the server 20, and then add the fourth type sub-rule, so that the target detection rule of the data category of methane concentration includes the first type sub-rule, the second type sub-rule, the third type sub-rule, the fourth type sub-rule, and the fifth type sub-rule. Of course, the operation corresponding to the change request may also be other operations, such as deleting part of the sub-rules in the detection rules corresponding to the data category, or adding new other detailed rules in the sub-rules.
The manager can change the corresponding relation between the data category and the detection rule, so that the flexibility of detection can be improved.
It should be noted that, during the modification operation, the server 20 is usually required to authenticate the authority of the user account in the current modification operation, and if the user account corresponding to the modification operation has the authority, the modification request is responded, so as to improve the security and reliability of the modification operation.
As an optional implementation, the method may further include: based on a viewing request received from a terminal device, judging whether the user permission corresponding to the viewing request is greater than or equal to the permission of the viewing content corresponding to the viewing request; and when the user permission is larger than or equal to the permission of the viewing content corresponding to the viewing request, sending the viewing content corresponding to the viewing request to the terminal equipment.
In this embodiment, the viewing content corresponding to the viewing request may be determined according to an actual situation. For example, the test results of the corresponding environment data may be viewed, or the latest test report may be viewed. The administrator can check the detection results of each type of environmental data, including the current latest detection result and the historical detection result, through the management terminal or the site device 30, and can also check the detection report obtained each time.
Each time the administrator performs the viewing, the management terminal or the site device 30 may generate a corresponding viewing request based on the viewing operation of the administrator, and send the viewing request to the server 20. After receiving the viewing request, the server 20 may determine whether the user right corresponding to the viewing request is greater than or equal to the right of the viewing content corresponding to the viewing request. The user right corresponding to the viewing request can be understood as the user right of the user account sending the viewing request. The right to view the content may be understood as a preset right. The server 20 responds to the viewing request when the level of the user authority is greater than or equal to the level of the authority to view the content, and then transmits the viewing content corresponding to the viewing request to the management terminal or site device 30.
In the above embodiment, by authenticating the user authority, it is beneficial to improve the security and confidentiality of the viewing action and improve the information security level of the data.
Referring to fig. 4, an embodiment of the present application further provides a data detection management apparatus 100, which can be applied to the server 20. The data detection management device 100 includes at least one software functional module which can be stored in a storage module in the form of software or Firmware (Firmware) or solidified in an Operating System (OS) of the server 20. The processing module is used for executing executable modules stored in the storage module, such as software functional modules and computer programs included in the data detection management device 100.
The data detection management apparatus 100 may include a data acquisition unit 110, a determination unit 120, a detection unit 130, and a warehousing unit 140.
A data obtaining unit 110, configured to obtain a current data set collected by the station device 30.
A determining unit 120, configured to determine, based on a corresponding relationship between a data category and a detection rule, that the detection rule corresponding to the data category of the to-be-processed data in the current data set is a target detection rule.
The detecting unit 130 is configured to detect the data to be processed according to the target detection rule, so as to obtain a detection result of the data to be processed.
And the storage unit 140 is configured to, when the detection result indicates that the data to be processed is normal, take the data to be processed as target data and record the target data into a database.
Optionally, the data detection management apparatus 100 may further include a prompting unit, configured to send prompting information to the station device 30 when the detection result indicates that the data to be processed is abnormal, so that the station device 30 sends new data to be processed to the server 20 based on the prompting information.
Optionally, the data detection management device 100 may further include a rights management unit. The authority management unit is configured to, in a detection period, obtain a management authority from the site device 30 when a number of times that the site device 30 repeatedly sends the to-be-processed data is greater than or equal to a specified number of times, a detection result of the to-be-processed data sent each time represents that the to-be-processed data is abnormal, and a data type of the to-be-processed data is a specified type; the storage unit 140 is further configured to record the to-be-processed data into the database based on the management authority.
Optionally, the target detection rule includes a sub-rule for detecting at least one of data integrity, data normalization, data accuracy, data consistency, and data submission timeliness. The detection unit 130 may be configured to: detecting at least one of data integrity, data normalization, data accuracy, data consistency and data submission timeliness of the data to be processed through the target detection rule; when the at least one type of the data to be processed is detected normally, obtaining a detection result representing that the data to be processed is normal; and when any one of the at least one class of the data to be processed is abnormal, obtaining a detection result representing the abnormality of the data to be processed.
Optionally, the data detection management apparatus 100 may further include a changing unit configured to change the correspondence relationship between the specified data category and the corresponding detection rule based on the received change request.
Optionally, the data detection management device 100 may further include a right management unit and a sending unit. The permission management unit is used for judging whether the user permission corresponding to the viewing request is larger than or equal to the permission of the viewing content corresponding to the viewing request or not based on the viewing request received from the terminal equipment; and the sending unit is used for sending the viewing content corresponding to the viewing request to the terminal equipment when the user permission is greater than or equal to the permission of the viewing content corresponding to the viewing request.
Optionally, the data obtaining unit 110 is configured to obtain, from at least one station device 30, a current data set collected by the at least one station device 30 every preset time interval.
It should be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the server 20 and the data detection management apparatus 100 described above may refer to the corresponding processes of the steps in the foregoing method, and are not described in detail herein.
In this embodiment, the processing module of the server 20 may be an integrated circuit chip having signal processing capability. The processing module may be a general purpose processor. For example, the Processor may be a Central Processing Unit (CPU), a Network Processor (NP), or the like; the method, the steps and the logic block diagram disclosed in the embodiments of the present Application may also be implemented or executed by a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
The communication module is used for establishing communication connection between the server 20 and the station device 30 through a network and transmitting and receiving data through the network.
The memory module may be, but is not limited to, a random access memory, a read only memory, a programmable read only memory, an erasable programmable read only memory, an electrically erasable programmable read only memory, and the like. In this embodiment, the storage module may be used to store data to be processed, detection rules, and the like. Of course, the storage module may also be used to store a program, and the processing module executes the program after receiving the execution instruction.
It should be noted that, in this embodiment, there is low coupling between the server 20 and the site device 30, that is, the site device 30 transmits the data to be processed, and the server 20 performs quality detection on the data to be processed, which is separated from the logical and physical levels. The data detection management apparatus 100 may be a software system for implementing the data detection management method, and may be migrated or deployed on another server, not limited to the server 20. Based on the method, the software system is convenient to maintain, the flexibility of system operation, deployment and maintenance is improved, and the software system is convenient to be butted with other systems and equipment (such as newly added station equipment).
The embodiment of the application also provides a computer readable storage medium. The readable storage medium has stored therein a computer program that, when run on a computer, causes the computer to execute the data detection management method as described in the above embodiments.
From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by hardware, or by software plus a necessary general hardware platform, and based on such understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.), and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments of the present application.
In summary, the present application provides a data detection management method, an apparatus, a server and a readable storage medium. The method comprises the following steps: acquiring a current data set collected by site equipment; determining a detection rule corresponding to the data type of the data to be processed in the current data set as a target detection rule based on the corresponding relation between the data type and the detection rule; detecting the data to be processed through a target detection rule to obtain a detection result of the data to be processed; and when the detection result represents that the data to be processed is normal, taking the data to be processed as target data and recording the target data into the database. In the scheme, the data to be processed is detected through the detection rules corresponding to the data types, the data to be processed of different data types can be detected in a differentiated mode, in addition, the data to be processed which is detected normally is put into a warehouse, the abnormal data can be prevented from being put into the warehouse by the server, and the effectiveness and the reliability of the data which are put into the warehouse are improved.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus, system, and method may be implemented in other ways. The apparatus, system, and method embodiments described above are illustrative only, as the flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A method for data detection management, the method comprising:
acquiring a current data set collected by site equipment;
determining a detection rule corresponding to the data type of the data to be processed in the current data set as a target detection rule based on the corresponding relation between the data type and the detection rule;
detecting the data to be processed through the target detection rule to obtain a detection result of the data to be processed;
and when the detection result represents that the data to be processed is normal, taking the data to be processed as target data and recording the target data into a database.
2. The method of claim 1, further comprising:
and when the detection result represents that the data to be processed is abnormal, sending prompt information to the site equipment, wherein the prompt information is used for the site equipment to send new data to be processed to a server based on the prompt information.
3. The method of claim 2, further comprising:
in a detection period, when the number of times of repeatedly sending the data to be processed by the site equipment is greater than or equal to a specified number of times, and a detection result of the data to be processed sent each time represents that the data to be processed is abnormal, and the data type of the data to be processed is a specified type, acquiring a management authority from the site equipment;
and recording the data to be processed into the database based on the management authority.
4. The method according to claim 1, wherein the target detection rule includes sub-rules for detecting at least one of data integrity, data normalization, data accuracy, data consistency, and data submission timeliness, and the detecting of the data to be processed by the target detection rule to obtain the detection result of the data to be processed includes:
detecting at least one of data integrity, data normalization, data accuracy, data consistency and data submission timeliness of the data to be processed through the target detection rule;
when the at least one type of the data to be processed is detected normally, obtaining a detection result representing that the data to be processed is normal;
and when any one of the at least one class of the data to be processed is abnormal, obtaining a detection result representing the abnormality of the data to be processed.
5. The method of claim 1, further comprising:
based on the received change request, the corresponding relation between the specified data category and the corresponding detection rule is changed.
6. The method of claim 1, further comprising:
based on a viewing request received from a terminal device, judging whether the user permission corresponding to the viewing request is greater than or equal to the permission of the viewing content corresponding to the viewing request;
and when the user permission is larger than or equal to the permission of the viewing content corresponding to the viewing request, sending the viewing content corresponding to the viewing request to the terminal equipment.
7. The method of claim 1, wherein obtaining the current data set collected by the site device comprises:
and acquiring a current data set collected by at least one site device from at least one site device every preset time interval.
8. A data detection management apparatus, characterized in that the apparatus comprises:
the data acquisition unit is used for acquiring a current data set collected by the site equipment;
the determining unit is used for determining a detection rule corresponding to the data type of the data to be processed in the current data set as a target detection rule based on the corresponding relation between the data type and the detection rule;
the detection unit is used for detecting the data to be processed through the target detection rule to obtain a detection result of the data to be processed;
and the storage unit is used for taking the data to be processed as target data and recording the target data into a database when the detection result represents that the data to be processed is normal.
9. A server, characterized in that the server comprises a memory, a processor, coupled to each other, in which memory a computer program is stored which, when executed by the processor, causes the server to carry out the method according to any one of claims 1-7.
10. A computer-readable storage medium, in which a computer program is stored which, when run on a computer, causes the computer to carry out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010609050.5A CN111752936B (en) | 2020-06-30 | Data detection management method, device, server and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010609050.5A CN111752936B (en) | 2020-06-30 | Data detection management method, device, server and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111752936A true CN111752936A (en) | 2020-10-09 |
| CN111752936B CN111752936B (en) | 2024-04-26 |
Family
ID=
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112468516A (en) * | 2020-12-17 | 2021-03-09 | 全球能源互联网研究院有限公司 | Security defense method and device, electronic equipment and storage medium |
| CN113469857A (en) * | 2021-07-19 | 2021-10-01 | 神彩科技股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113489773A (en) * | 2021-06-30 | 2021-10-08 | 未鲲(上海)科技服务有限公司 | Data access method, device, equipment and medium |
| WO2023092954A1 (en) * | 2021-11-26 | 2023-06-01 | 华为云计算技术有限公司 | Data governance method and device and storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008085819A (en) * | 2006-09-28 | 2008-04-10 | Oki Electric Ind Co Ltd | Network abnormality detection system, network abnormality detection method, and network abnormality detection program |
| KR20100068733A (en) * | 2008-12-15 | 2010-06-24 | 한국과학기술정보연구원 | Data cleansing system and method for developing integrated database, and recording medium therefor |
| CN105119783A (en) * | 2015-09-30 | 2015-12-02 | 北京奇艺世纪科技有限公司 | Network request data detection method and device |
| US20160261482A1 (en) * | 2015-03-04 | 2016-09-08 | Fisher-Rosemount Systems, Inc. | Anomaly detection in industrial communications networks |
| US20180173744A1 (en) * | 2016-12-20 | 2018-06-21 | International Business Machines Corporation | Determining integrity of database workload transactions |
| CN108446362A (en) * | 2018-03-13 | 2018-08-24 | 平安普惠企业管理有限公司 | Data cleansing processing method, device, computer equipment and storage medium |
| WO2019019493A1 (en) * | 2017-07-28 | 2019-01-31 | 平安科技（深圳）有限公司 | Data sharing method and device, and computer readable storage medium |
| US20190087738A1 (en) * | 2017-09-20 | 2019-03-21 | Siemens Aktiengesellschaft | Method, apparatus and device for detecting abnormal data, and machine-readable medium |
| CN109656917A (en) * | 2018-12-18 | 2019-04-19 | 深圳前海微众银行股份有限公司 | Data detection method, device, equipment and the readable storage medium storing program for executing of multi-data source |
| CN109756368A (en) * | 2018-12-24 | 2019-05-14 | 广州市百果园网络科技有限公司 | Detection method, device, computer readable storage medium and the terminal of unit exception change |
| CN110505196A (en) * | 2019-07-02 | 2019-11-26 | 中国联合网络通信集团有限公司 | Internet of Things network interface card method for detecting abnormality and device |
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008085819A (en) * | 2006-09-28 | 2008-04-10 | Oki Electric Ind Co Ltd | Network abnormality detection system, network abnormality detection method, and network abnormality detection program |
| KR20100068733A (en) * | 2008-12-15 | 2010-06-24 | 한국과학기술정보연구원 | Data cleansing system and method for developing integrated database, and recording medium therefor |
| US20160261482A1 (en) * | 2015-03-04 | 2016-09-08 | Fisher-Rosemount Systems, Inc. | Anomaly detection in industrial communications networks |
| CN105119783A (en) * | 2015-09-30 | 2015-12-02 | 北京奇艺世纪科技有限公司 | Network request data detection method and device |
| US20180173744A1 (en) * | 2016-12-20 | 2018-06-21 | International Business Machines Corporation | Determining integrity of database workload transactions |
| WO2019019493A1 (en) * | 2017-07-28 | 2019-01-31 | 平安科技（深圳）有限公司 | Data sharing method and device, and computer readable storage medium |
| US20190087738A1 (en) * | 2017-09-20 | 2019-03-21 | Siemens Aktiengesellschaft | Method, apparatus and device for detecting abnormal data, and machine-readable medium |
| CN108446362A (en) * | 2018-03-13 | 2018-08-24 | 平安普惠企业管理有限公司 | Data cleansing processing method, device, computer equipment and storage medium |
| CN109656917A (en) * | 2018-12-18 | 2019-04-19 | 深圳前海微众银行股份有限公司 | Data detection method, device, equipment and the readable storage medium storing program for executing of multi-data source |
| CN109756368A (en) * | 2018-12-24 | 2019-05-14 | 广州市百果园网络科技有限公司 | Detection method, device, computer readable storage medium and the terminal of unit exception change |
| CN110505196A (en) * | 2019-07-02 | 2019-11-26 | 中国联合网络通信集团有限公司 | Internet of Things network interface card method for detecting abnormality and device |
Non-Patent Citations (2)
| Title |
|---|
| 曾昭文;: "数据质量检测方法及应用", 电脑编程技巧与维护, no. 12 * |
| 蒋梦丹;林宏刚;曹鹤鸣;: "基于业务逻辑思想的异常检测研究", 成都信息工程大学学报, no. 02 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112468516A (en) * | 2020-12-17 | 2021-03-09 | 全球能源互联网研究院有限公司 | Security defense method and device, electronic equipment and storage medium |
| CN113489773A (en) * | 2021-06-30 | 2021-10-08 | 未鲲(上海)科技服务有限公司 | Data access method, device, equipment and medium |
| CN113469857A (en) * | 2021-07-19 | 2021-10-01 | 神彩科技股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| WO2023092954A1 (en) * | 2021-11-26 | 2023-06-01 | 华为云计算技术有限公司 | Data governance method and device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10924949B2 (en) | Abnormality detecting method and abnormality detecting device | |
| CN105049291B (en) | A method of detection exception of network traffic | |
| US8774023B2 (en) | Method and system for detecting changes in network performance | |
| CN102812441A (en) | Automated malware detection and remediation | |
| EP3667952B1 (en) | Method, device, and storage medium for locating failure cause | |
| EP3482528A1 (en) | A system and method for providing a secure data monitoring system implemented within factory or plant | |
| CN109995555B (en) | Monitoring method, device, equipment and medium | |
| CN110958161B (en) | Block chain link point monitoring method and device and storage medium | |
| CN111713123A (en) | Control unit and method for the manipulation-protected detection of operating-safety-relevant integrity monitoring data | |
| JPWO2020075801A1 (en) | Information processing equipment, anomaly analysis method and program | |
| CN111752936A (en) | Data detection management method, device, server and readable storage medium | |
| CN111752936B (en) | Data detection management method, device, server and readable storage medium | |
| US11079400B2 (en) | Monitoring a product build process via a smart tray | |
| CN117111100B (en) | Electric bicycle anti-theft tracking system based on Internet of things | |
| CN114238036A (en) | Method and device for monitoring abnormity of SAAS (software as a service) platform in real time | |
| US20210152587A1 (en) | Method and system to detect abnormal message transactions on a network | |
| US10560365B1 (en) | Detection of multiple signal anomalies using zone-based value determination | |
| Ediriweera et al. | Monitoring water distribution systems: understanding and managing sensor networks | |
| KR20190128420A (en) | IoT sensor abnormality diagnosing method and system using cloud-based virtual sensor | |
| CN109379439B (en) | Information positioning system based on Internet of things | |
| US8214907B1 (en) | Collection of confidential information dissemination statistics | |
| CN115543665A (en) | Memory reliability evaluation method and device and storage medium | |
| US10996076B2 (en) | Sensor device management method and sensor device management system | |
| CN114691443A (en) | Cross section data sending method and device, electronic equipment and storage medium | |
| CN115967570A (en) | Network communication environment safety supervision system based on big data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |