CN107484196B - Data quality assurance method for sensor network and computer readable medium - Google Patents
Data quality assurance method for sensor network and computer readable medium Download PDFInfo
- Publication number
- CN107484196B CN107484196B CN201710693830.0A CN201710693830A CN107484196B CN 107484196 B CN107484196 B CN 107484196B CN 201710693830 A CN201710693830 A CN 201710693830A CN 107484196 B CN107484196 B CN 107484196B
- Authority
- CN
- China
- Prior art keywords
- detection data
- data
- sequence
- differential
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/04—Arrangements for maintaining operational condition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W84/00—Network topologies
- H04W84/18—Self-organising networks, e.g. ad-hoc networks or sensor networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
A data quality assurance method and a computer-readable medium for a sensor network are disclosed. The method comprises the steps of detecting a detection data sequence obtained by periodically detecting nodes of a sensor network, judging whether defect detection data exist or not based on the change characteristics of the detection data sequence, and correcting the defect detection data based on historical detection data. Therefore, error data which are sporadically generated by the sensor network nodes can be effectively discovered and corrected. And then, comparing detection data sequences of different sensor network nodes through the main road branch relation so as to locate time periods possibly with errors and nodes possibly with faults. Therefore, data correction and fault node positioning can be automatically carried out in a sensor network with a large scale, and data quality is guaranteed.
Description
Technical Field
The present invention relates to a network data diagnosis technology, and more particularly, to a data quality assurance method and a computer-readable medium for a sensor network.
Background
With the development of computer and network technologies, the technology of internet of things for intelligently monitoring and controlling equipment data and environmental parameters in large buildings is widely applied. The prior art can monitor all the electrical equipment and environmental parameter changes in the whole building through a sensor network. However, it is possible for a node of the sensor network to report erroneous detection data due to a fault or environmental change. Therefore, in order to ensure the data quality acquired by the sensor network, a data quality assurance method for the sensor network, which can search for a faulty node and error data, is urgently needed.
Disclosure of Invention
In view of the above, the present invention provides a data quality assurance method and a computer readable medium for a sensor network, so as to automatically search for erroneous data recorded by nodes of the sensor network and nodes that may fail, and ensure the data quality of the sensor network.
According to a first aspect of the present application, there is provided a data quality assurance method for a sensor network, comprising:
acquiring a detection data sequence of each node in the sensor network, wherein the detection data sequence comprises detection data periodically acquired by corresponding nodes;
detecting the detection data sequence of each node to obtain defect detection data;
correcting the defect detection data according to historical detection data to obtain a corrected detection data sequence; and
and detecting a fault node according to the corrected detection data sequence and a preset main road branch relation between the detection data of each node, wherein the fault node is a node with defects in the detection data all the time.
Preferably, the acquiring of the detection data sequence of each node in the sensor network includes:
acquiring an original detection data sequence of each node in the sensor network; and
and acquiring the detection data sequence with the same detection period according to the original detection data sequence.
Preferably, the detecting the detection data sequence of each node, and the acquiring the defect detection data includes:
acquiring a differential sequence of the detection data sequence;
when the quantity of the historical detection data meets the requirement, acquiring a difference threshold range of a preset time period or a preset time point according to at least part of the historical detection data;
marking detection data corresponding to the difference data exceeding the difference threshold range as defect detection data;
when the detection data is an accumulated quantity, the differential sequence is a second-order differential sequence of the detection data sequence, and when the detection data is an instantaneous quantity, the differential sequence is a first-order differential sequence of the detection data sequence.
Preferably, the differential threshold range includes a first threshold range and a second threshold range, where the first threshold range is obtained according to historical detection data statistics of a previous predetermined time period, and the second threshold range is obtained according to historical detection data statistics of a corresponding time period in a same type day.
Preferably, the detecting the detection data sequence of each node, and the acquiring the defect detection data further includes:
when two continuous differential data with opposite directions are detected in the differential sequence, searching differential data with the same mode in a detection data sequence with a first length before the current time point;
when the differential data with the same mode is found, marking the corresponding detection data as defect detection data;
here, the differential data having the same pattern refers to consecutive differential data having the same sign change order.
Preferably, when differential data with an opposite mode is found, prompting detection data corresponding to time and acquiring a marking instruction corresponding to the detection data;
here, the differential data having the opposite pattern refers to continuous differential data having the opposite sign change order.
Preferably, the detecting the detection data sequence of each node, and the acquiring the defect detection data further includes:
and when the length of the difference sequence which is continuously zero is greater than the second length and no corresponding detection data sequence of which the difference data continuously has the second length is present in the historical detection data, prompting the detection data at the corresponding time and acquiring a marking instruction for the detection data.
Preferably, the detecting the detection data sequence of each node, and the acquiring the defect detection data further includes:
acquiring the variation trend of the differential sequence in a time period with a third length;
and when the change trend is continuously increased or decreased, prompting the detection data corresponding to the time and acquiring a marking instruction for the detection data.
Preferably, the modifying the defect detection data according to the historical detection data to obtain a modified detection data sequence includes:
replacing the defect detection data with detection data corresponding to a previous time period;
and correcting the replaced detection data according to the detection data in a preset time period before and after the defect detection data.
Preferably, the detecting data is an instantaneous value, and the modifying the replaced detecting data according to the detecting data in a predetermined time period before and after the defect detecting data includes:
linearly adjusting the mean value and the amplitude of the replaced detection data according to the detection data in a preset time period before and after the defect detection data;
alternatively, the first and second electrodes may be,
the detection data is an accumulated value, and the correction of the replaced detection data according to the detection data in a preset time period before and after the defect detection data comprises the following steps:
adjusting the instantaneous value corresponding to the replaced detection data according to the detection data in the preset time period before and after the defect detection data; and
the substituted detection data is corrected based on the adjusted instantaneous value.
Preferably, the detecting the fault node according to the predetermined total road branch relationship between the corrected detection data sequence and the detection data of each node includes:
calculating the sum sequence of the detection data of all branches corresponding to the main road;
calculating the relative error and correlation coefficient of the detection data sequence of the main road and the sum sequence;
when the relative error and the correlation coefficient meet a preset condition, prompting that the main branch is unbalanced, and searching for an error time period;
and positioning the fault node according to the detection data sequence in the error time period.
Preferably, the lookup error time period includes:
performing sliding detection on the detection data sequence of the main road and the sum sequence through a time window with a preset window length;
setting the starting time of a time window with the number of deviated data points in the window larger than a first threshold value as the starting time of an error time period;
taking the end time of the time window when the number of the deviated data points in the window is reduced to a second threshold value as the end time of the error time period;
and the number of the deviated data points is that the detection data of the total way is more than the number of the data points of the corresponding data in the sum sequence or the detection data of the total way is less than the number of the data points of the corresponding data in the sum sequence.
In a second aspect, there is provided a computer readable medium for storing computer program instructions, characterized in that the computer program instructions, when executed, implement the method according to the first aspect.
The method comprises the steps of detecting a detection data sequence obtained by periodically detecting nodes of a sensor network, judging whether defect detection data exist or not based on the change characteristics of a historical detection data sequence, and correcting the defect detection data based on the historical detection data. Error data sporadically appearing in the nodes of the sensor network can be effectively discovered and corrected. And then, comparing detection data sequences of different sensor network nodes through the main road branch relation so as to locate time periods possibly with errors and nodes possibly with faults. Therefore, data correction and fault node positioning can be automatically carried out in a sensor network with a large scale, and data quality is guaranteed.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent from the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
fig. 1 is a flowchart of a data quality assurance method of a sensor network according to an embodiment of the present invention;
FIG. 2 is a flow chart of detection data sequence acquisition performed by an embodiment of the present invention;
FIG. 3 is a flow chart of detecting defect detection data according to an embodiment of the present invention;
FIG. 4 is a flow diagram of another implementation of detecting defect detection data according to an embodiment of the invention;
FIG. 5 is a flow diagram of yet another implementation of detecting defect detection data according to an embodiment of the invention;
FIG. 6 is a flow chart of a method for performing defect detection data correction according to an embodiment of the present invention;
FIG. 7 is a flow chart of locating a failed node according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a computer system for performing the method of an embodiment of the invention.
Detailed Description
The present invention will be described below based on examples, but the present invention is not limited to only these examples. In the following detailed description of the present invention, certain specific details are set forth. It will be apparent to one skilled in the art that the present invention may be practiced without these specific details. Well-known methods, procedures, components and circuits have not been described in detail so as not to obscure the present invention.
Further, those of ordinary skill in the art will appreciate that the drawings provided herein are for illustrative purposes and are not necessarily drawn to scale.
Unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise", "comprising", and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is, what is meant is "including, but not limited to".
In the description of the present invention, it is to be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In addition, in the description of the present invention, "a plurality" means two or more unless otherwise specified.
Fig. 1 is a flowchart of a data quality assurance method for a sensor network according to an embodiment of the present invention. As shown in fig. 1, the method includes:
and S100, acquiring a detection data sequence of each node in the sensor network.
Each detection data sequence comprises detection data periodically acquired by the corresponding node. Since the nodes in the sensor network may be individually configured step by step. The detection period of each node may be the same or different according to different configured node types. For example, a portion of the nodes may collect data in a 3 minute period. And another part of nodes can collect data in a period of 5 minutes. Thus, the raw test data sequence acquired needs to be normalized. Specifically, as shown in fig. 2, step S100 may include the steps of:
and step S110, acquiring an original detection data sequence of each node in the sensor network.
And step S120, acquiring detection data sequences with the same detection period according to the original detection data sequences.
For example, when the intervals of the original detection data sequence are 3 minutes and 5 minutes, respectively, sampling may be performed from the original detection data sequence at a period of 15 minutes, so as to obtain detection data sequences each having a detection period of 15 minutes.
Thus, further processing can be facilitated subsequently.
And S200, detecting the detection data sequence of each node to acquire defect detection data.
In this step, a number of different methods may be employed to obtain defect detection data.
In an optional implementation manner of this embodiment, as shown in fig. 3, step S200 includes:
and step S210, acquiring a differential sequence of the detection data sequence.
When the detection data is an accumulated quantity, the differential sequence is a second-order differential sequence of the detection data sequence, and when the detection data is an instantaneous quantity, the differential sequence is a first-order differential sequence of the detection data sequence. The variation trend of the corresponding instantaneous variable can be obtained through the second order difference or the first order difference, and therefore whether sudden change exists in the data is judged through monitoring the variation trend.
And step S220, when the quantity of the historical detection data meets the requirement, acquiring a difference threshold range of a preset time period or a preset time point according to at least part of the historical detection data.
Specifically, if the historical detection data is small, the accumulated data is maintained.
Specifically, the differential threshold range may include a first threshold range and a second threshold range, where the first threshold range is obtained according to historical detection data statistics of a previous predetermined time period, and the second threshold range is obtained according to historical detection data statistics of a corresponding time period in a same type day. That is, the first threshold range is a threshold range obtained by continuous time trend statistics in a short period. For example, for a difference sequence of 10 am on monday, a first threshold range is statistically derived from historical detection data of previous 2 hours. The second threshold range is a threshold range obtained by data change trend statistics of the same time on the same date. For example, for a differential sequence of 10 am on Monday, the second threshold range is statistically derived from historical test data of 10 am on every Monday of the previous 10 weeks.
Step S230, marking the detection data corresponding to the difference data exceeding the difference threshold range as defect detection data.
For example, the mean and variance of the difference sequence at a particular point in time may be determined, and a difference threshold range may be set based on the mean and variance. And when the detection data exceed the difference threshold range, determining that the detection data are abnormal and marking the detection data as defect detection data.
Under the premise of setting the difference threshold range, the detection data of some sensor network nodes can jump in the difference threshold range, and the jump can also be used as a clue for detecting error data.
In another optional implementation manner of this embodiment, as shown in fig. 4, step S200 further includes:
step S240, when two continuous differences with opposite signs are detected in the difference sequence, searching for a detection data transition in the history detection data sequence with the first length before the current time point.
In this embodiment, the transition of the detection data is detected by monitoring the differential sequence. When the detected data is instantaneous value, whether the data has jump can be judged by calculating the first order difference sequence. The first order difference sequence can represent the variation trend of the detected data, and the data is jumped when the difference sequence value is larger than a positive preset value or smaller than a negative preset value. The sign of the first order difference value may characterize the direction of the transition. When the detected data is an accumulated value, the second order difference is obtained to determine whether the corresponding instantaneous value has jump. The second-order difference sequence can represent the variation trend of the instantaneous variable corresponding to the accumulated value. Therefore, the transition of the detection data can be detected as well. It should be understood that other ways to detect a transition may be used.
In this step, if two consecutive differential data with opposite signs are detected, a transition of the detected data occurs at the corresponding time. The detected data transition is looked up from the time when the transition occurred back to the differential sequence of the first length L1 (i.e., it is looked up whether there are two consecutive differential data symbols that are different) to determine whether the same transition occurred before.
Step S250, when the differential data with the same pattern is found, marking the corresponding detection data as defect detection data.
Here, the differential data having the same pattern refers to consecutive differential data having the same sign change order. For example, if two consecutive differential data symbols are detected as + and-, respectively, it is searched for whether there are two consecutive differential data symbols different from each other, which is also + of the previous one, from the preceding differential sequence of length L1, and if so, it is determined that differential data having the same pattern is found.
In this step, if differential data with the same pattern is found, that is, the same jump is detected in the history detection data sequence with the first length before the jump is detected (the jumping directions are opposite to each other and are respectively the same as the current jump), the detection data corresponding to the current jump is marked as defect detection data.
If differential data with an opposite mode is found, namely a jump is detected in a historical detection data sequence with a first length before the jump is detected, but the mode of the jump is different from the current jump mode (namely the direction of the current jump is different), the detection data of the time corresponding to the differential data is prompted to a manager through a man-machine interaction interface, and a marking instruction of the manager for the detection data is obtained. That is, only defect inspection data can be suspected for such inspection data, and thus, further manual judgment and marking are required.
Alternatively, if no jump is detected, it can be determined that the jump is due to a replacement sensor, which generally does not result in an error in the detected data.
For example, taking the active power of the monitoring meter as an example, if in the detection data sequence, the instantaneous value of the active power jumps first downwards and then upwards (correspondingly, the differential sequence is embodied as consecutive differential values with opposite signs). It is looked up in the detected data sequence starting from the current time back to length L1 whether there is the same case of first a down transition and then an up transition (correspondingly, the differential sequence is embodied as consecutive differential values of opposite sign). If so, marking the current detection data as the defect detection data. If the jump is detected, but the jump mode is that the jump is carried out after the jump is carried out upwards, the manual intervention is needed to judge whether the jump is the defect data.
Meanwhile, whether errors occur or not can be judged according to the trend of the differential sequence within a period of time.
In yet another optional implementation manner of this embodiment, step S200 further includes:
and step S260, when the length of the difference sequence with the continuous zero is larger than the second length L2 and the difference sequence corresponding to the historical detection data does not have the continuous difference sequence with the second length zero, prompting the detection data and acquiring a marking instruction for the detection data.
That is, if the difference sequence is found to be continuously zero and long, but never before, it indicates that the detected data may be problematic, and needs to be prompted to the user and obtain further instructions.
In yet another alternative implementation of this embodiment, as shown in fig. 5, step S200 further includes:
and step S270, acquiring the variation trend of the differential sequence in a time period with the third length.
And step S280, when the change trend is continuously increased or decreased, prompting the detection data corresponding to the differential data and acquiring a marking instruction corresponding to the detection data.
The differential data sequence corresponding to the detection data sequence usually fluctuates up and down according to the actual operation condition of the device, and is not always increased or is always decreased. Therefore, when a substantially continuous increase or decrease in the differential sequence is detected, it can be submitted to the user for further confirmation as suspicious detection data.
Therefore, the defect data appearing in most short time ranges can be detected by detecting the characteristics of the differential sequence corresponding to the data sequence and the historical detection data sequence.
And step S300, correcting the defect detection data according to historical detection data to obtain a corrected detection data sequence.
In this step, the detection data at the time point or the time period corresponding to the defect detection data may be predicted by using the historical detection data, so that the defect detection data may be corrected.
In an alternative implementation manner of the embodiment, as shown in fig. 6, step S300 is modified by:
and step S310, replacing the defect detection data with the detection data corresponding to the previous cycle.
Step S320, modifying the replaced detection data according to the detection data in the predetermined time period before and after the defect detection data.
In fact, the detection data at the time point needing to be corrected are predicted by utilizing the characteristic that the detection data sequence changes basically show correlation. The data correction in the above manner does not need to store a large amount of historical detection data, and as long as no defect detection data is detected in a period (for example, one week or one month), all detection data of the period can be saved and used for subsequent correction.
Meanwhile, the detection data may be instantaneous values or accumulated values. Taking the active power measured by the electric meter as an example, the detection data may be the instantaneous active power or the accumulated power amount. Different correction modes are required for different types of detection data.
When the detected data is instantaneous values, step S320 includes:
step S321, linearly adjusting the mean value and the amplitude of the replaced detection data according to the detection data in the preset time period before and after the defect detection data. The instantaneous value is usually linearly changed, and therefore, after the replacement with the historical detection data, the linear adjustment is further performed by the data before and after the time point, so that the corrected detection data is continuous with the detection data recorded before and after, and thus the actual value is more approximated.
When the detected data is the accumulated value, step S320 includes:
step S322, adjusting an instantaneous value corresponding to the replaced detection data according to the detection data in a predetermined time period before and after the defect detection data.
The principle of adjustment of the instantaneous value is similar to step S321 described above.
Step S323 corrects the replaced detection data based on the adjusted instantaneous value.
Thus, the accumulated value can be corrected based on the instantaneous value that is adjusted to approach the actual value.
Through steps S100-S300, the defect data can be detected and corrected according to the detection data sequence recorded by each sensor network node itself, and the accuracy of the data is ensured. However, when a node fails, the detection data sequence of its corresponding branch is always problematic, and thus such an error cannot be discovered and corrected only by the detection data sequence recorded by itself. Therefore, further diagnosis needs to be performed based on the relationship between different sensor network nodes detecting objects, particularly the trunk-branch relationship.
And S400, detecting a fault node according to the corrected detection data sequence and the preset total path branch relation between the detection data of each node, wherein the fault node is a node with defects in the detection data all the time.
In this embodiment, one main road corresponds to a plurality of branches, and the sum of the detection data of the plurality of branches should be theoretically equal to the detection data of the main road. The application scenarios of the main road and the branch road comprise various application scenarios such as water quantity detection, air flow detection or active power detection of power supply. By configuring corresponding main branch road relations according to detection objects of different sensor network nodes, the fault node can be detected according to the corrected detection data sequence and the preset main branch road relation between the detection data of each node.
Specifically, as shown in fig. 7, step S400 may include:
and step S410, calculating the sum obtaining and sequence of the detection data of all the branches corresponding to the main road.
In this embodiment, the corrected detection data is further processed into detection data in units of days, and then the subsequent processing is performed.
However, as will be readily appreciated by those skilled in the art, the interval or period of the detection data may be selected according to the application scenario or period.
And step S420, calculating the relative error and the correlation coefficient of the detection data sequence of the main road and the sum sequence.
Wherein, the relative error refers to the ratio of the absolute error caused by measurement to the agreed true value. In this embodiment, the sum sequence is taken as the agreed true value. The correlation coefficient is used to measure the linear relationship between two variables, which can be calculated by the following formula:
where r (x, y) is the correlation coefficient of x and y, cov (x, y) is the covariance of the variables x and y, var (x) is the variance of the variable x, and var (y) is the variance of the variable y.
The relative error can represent the deviation degree of the detection data sequence of the general road and the sum sequence, and the correlation coefficient can represent the correlation degree of the variation trend of the detection data sequence and the sum sequence. Therefore, it is possible to determine whether or not there is an abnormality in the amounts by which these two should be changed in synchronization.
And step S430, when the relative error and the correlation coefficient meet a preset condition, prompting that the main branch is unbalanced, and searching for an error time period.
In this embodiment, when the relative error is less than 5% and the correlation coefficient is greater than 0.9, the detected data sequence and sum sequence of the main road are assumed to change substantially synchronously, and the main road branch relationship is balanced. Otherwise, the two are determined not to be synchronously changed, and the main branch is unbalanced. Thus, the time period causing the imbalance, i.e., the error time period, is searched for in the entire detection data sequence.
In this embodiment, the sliding detection is performed by a time window having a predetermined window length, and if the number of deviated data points within the time window is greater than the first threshold value, the start time of the current time window is set as the start time of the error period. The number of the deviated data points is that the detection data of the total path is larger than the number of the data points of the corresponding data in the sum sequence or the detection data of the total path is smaller than the number of the data points of the corresponding data in the sum sequence.
As the time window slides forward, the end time of the current time window is set to the end time of the error period when the number of deviated data points falls to the second threshold.
In this embodiment, the length of the time window is set to 30 days, the first threshold is set to 25 days, and the second threshold is set to 20 days.
Therefore, the time period of deviation of a large amount of data can be effectively found, and the error time period can be positioned.
And step S440, positioning the fault node according to the detection data in the error time period.
In this step, the failed node may be located in a variety of ways.
On one hand, whether the nodes with the missing number exist can be judged through the detection data sequences of all the main ways or the branch ways in the error time period.
On the other hand, whether the branch circuit changes can be judged through the detection data sequences of all branch circuits in the error time period.
Therefore, the detection data sequence obtained by periodically detecting the nodes of the sensor network is detected, whether the defect detection data exist is judged based on the change characteristics of the historical detection data sequence, and the defect detection data is corrected based on the historical detection data. Error data sporadically appearing in the nodes of the sensor network can be effectively discovered and corrected. And then, comparing detection data sequences of different sensor network nodes through the main road branch relation so as to locate time periods possibly with errors and nodes possibly with faults. Therefore, data correction and fault node positioning can be automatically carried out in a sensor network with a large scale.
The data structures and code described in the foregoing detailed description are typically stored on a computer-readable storage medium, which may be any device or medium that can store code and/or data for use by a computer system. Computer-readable storage media include, but are not limited to, volatile memory, non-volatile memory, magnetic and optical storage devices such as disk drives, magnetic tape, CDs (compact discs), DVDs (digital versatile discs or digital video discs), or other media capable of storing code and/or data now known or later developed.
The methods and processes described in the detailed description section can be embodied as code and/or data, which can be stored in a computer-readable storage medium as described above. When a computer system reads and executes the code and/or data stored on the computer-readable storage medium, the computer system performs the methods and processes embodied as data structures and code and stored within the computer-readable storage medium.
Further, the methods and processes described herein may be included in a hardware module or device. These modules or means may include, but are not limited to, an Application Specific Integrated Circuit (ASIC) chip, a Field Programmable Gate Array (FPGA), a dedicated or shared processor that executes a particular software module or piece of code at a particular time, and/or other now known or later developed programmable logic devices. When the hardware modules or devices are activated, they perform the methods and processes included therein.
FIG. 8 is a schematic diagram of a computer system for performing the method of an embodiment of the invention. As shown in fig. 8, the server 8 comprises a general-purpose computer hardware structure including at least a processor 81 and a memory 82. The processor 81 and the memory 82 are connected by a bus 83. The memory 82 is adapted to store instructions or programs executable by the processor 81. Processor 81 may be a stand-alone microprocessor or a collection of one or more microprocessors. Thus, the processor 81 implements the rendering of the web page by executing the instructions stored by the memory 82 to perform the method flows of the embodiments of the present invention as described above. The bus 88 connects the above components together, as well as to the display controller 84 and the display device and input/output (I/O) device 85. Input/output (I/O) devices 85 may be a mouse, keyboard, modem, network interface, touch input device, motion sensing input device, printer, and other devices known in the art. Typically, the input/output devices 85 are coupled to the system through an input/output (I/O) controller 86.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (11)
1. A data quality assurance method for a sensor network is characterized by comprising the following steps:
acquiring a detection data sequence of each node in the sensor network, wherein the detection data sequence comprises detection data periodically acquired by corresponding nodes;
detecting the detection data sequence of each node to obtain defect detection data;
correcting the defect detection data according to historical detection data to obtain a corrected detection data sequence; and
detecting a fault node according to the corrected detection data sequence and a preset main road branch relation between the detection data of each node, wherein the fault node is a node with defects in the detection data all the time;
wherein, detecting the detection data sequence of each node, and acquiring the defect detection data comprises:
acquiring a differential sequence of the detection data sequence;
when the quantity of the historical detection data meets the requirement, acquiring a difference threshold range of a preset time period or a preset time point according to at least part of the historical detection data;
marking detection data corresponding to the difference data exceeding the difference threshold range as defect detection data;
when the detection data is an accumulated quantity, the differential sequence is a second-order differential sequence of the detection data sequence, and when the detection data is an instantaneous quantity, the differential sequence is a first-order differential sequence of the detection data sequence;
wherein, detecting the fault node according to the corrected detection data sequence and the preset total path branch relation between the detection data of each node comprises:
calculating the sum sequence of the detection data of all branches corresponding to the main road;
calculating the relative error and correlation coefficient of the detection data sequence of the main road and the sum sequence;
when the relative error and the correlation coefficient meet a preset condition, prompting that the main branch is unbalanced, and searching for an error time period; and
and positioning the fault node according to the detection data sequence in the error time period.
2. The method of claim 1, wherein obtaining a sequence of sensed data for each node in the sensor network comprises:
acquiring an original detection data sequence of each node in the sensor network; and
and acquiring the detection data sequence with the same detection period according to the original detection data sequence.
3. The method of claim 2, wherein the differential threshold range comprises a first threshold range and a second threshold range, wherein the first threshold range is obtained from historical detection data statistics for a previous predetermined time period, and the second threshold range is obtained from historical detection data statistics for a corresponding time period in a same type of day;
wherein, the week is taken as a period, and the same type day is the same type day in the historical week;
or, taking a month as a period, and taking the same type of day as the same date in the historical month.
4. The method of claim 2, wherein the detecting the sequence of inspection data for each node, and the obtaining the defect inspection data further comprises:
when two continuous differential data with opposite signs are detected in the differential sequence, searching differential data with the same pattern in a detection data sequence with a first length before the current time point;
when the differential data with the same mode is found, marking the corresponding detection data as defect detection data;
here, the differential data having the same pattern refers to consecutive differential data having the same sign change order.
5. The method according to claim 3, wherein when differential data with opposite modes are found, prompting detection data corresponding to time and acquiring a marking instruction corresponding to the detection data;
here, the differential data having the opposite pattern refers to continuous differential data having the opposite sign change order.
6. The method of claim 1, wherein the detecting the sequence of inspection data for each node, and the obtaining the defect inspection data further comprises:
and when the length of the difference sequence which is continuously zero is greater than the second length and no corresponding detection data sequence of which the difference data continuously has the second length is present in the historical detection data, prompting the detection data at the corresponding time and acquiring a marking instruction for the detection data.
7. The method of claim 1, wherein the detecting the sequence of inspection data for each node, and the obtaining the defect inspection data further comprises:
acquiring the variation trend of the differential sequence in a time period with a third length;
and when the change trend is continuously increased or decreased, prompting the detection data corresponding to the time and acquiring a marking instruction for the detection data.
8. The method of claim 1, wherein modifying the defect detection data based on historical detection data to obtain a modified detection data sequence comprises:
replacing the defect detection data with detection data corresponding to a previous time period;
and correcting the replaced detection data according to the detection data in a preset time period before and after the defect detection data.
9. The method of claim 8, wherein the inspection data is instantaneous values, and wherein modifying the replaced inspection data based on inspection data within a predetermined time period before and after the defect inspection data comprises:
linearly adjusting the mean value and the amplitude of the replaced detection data according to the detection data in a preset time period before and after the defect detection data;
alternatively, the first and second electrodes may be,
the detection data is an accumulated value, wherein the correction of the replaced detection data according to the detection data in a preset time period before and after the defect detection data comprises the following steps:
adjusting the instantaneous value corresponding to the replaced detection data according to the detection data in the preset time period before and after the defect detection data; and
the substituted detection data is corrected based on the adjusted instantaneous value.
10. The method of claim 1, wherein finding an error time period comprises:
performing sliding detection on the detection data sequence of the main road and the sum sequence through a time window with a preset window length;
setting the starting time of a time window with the number of deviated data points in the window larger than a first threshold value as the starting time of an error time period;
taking the end time of the time window when the number of the deviated data points in the window is reduced to a second threshold value as the end time of the error time period;
and the number of the deviated data points is that the detection data of the total way is more than the number of the data points of the corresponding data in the sum sequence or the detection data of the total way is less than the number of the data points of the corresponding data in the sum sequence.
11. A computer readable medium storing computer program instructions, wherein the computer program instructions, when executed, implement the method of any of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710693830.0A CN107484196B (en) | 2017-08-14 | 2017-08-14 | Data quality assurance method for sensor network and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710693830.0A CN107484196B (en) | 2017-08-14 | 2017-08-14 | Data quality assurance method for sensor network and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107484196A CN107484196A (en) | 2017-12-15 |
| CN107484196B true CN107484196B (en) | 2020-10-09 |
Family
ID=60600339
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710693830.0A Active CN107484196B (en) | 2017-08-14 | 2017-08-14 | Data quality assurance method for sensor network and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107484196B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102011612A (en) * | 2010-11-20 | 2011-04-13 | 无锡泰立特科技有限公司 | Multifunctional tube bundle monitoring and emergency extinguishing prevention system |
| CN103439758A (en) * | 2013-02-08 | 2013-12-11 | 北京佳讯飞鸿电气股份有限公司 | Data processing method for rain sensor |
| CN104096303A (en) * | 2013-04-15 | 2014-10-15 | 深圳迈瑞生物医疗电子股份有限公司 | Electronic flow monitor, monitoring method and anesthesia machine |
| CN104202765A (en) * | 2014-08-26 | 2014-12-10 | 北京邮电大学 | Method and device for detecting sensor faults based on distributed network |
| CN105184394A (en) * | 2015-08-26 | 2015-12-23 | 中国电力科学研究院 | On-line data mining optimized control method based on cyber physical system (CPS) of power distribution network |
| CN105627528A (en) * | 2016-03-30 | 2016-06-01 | 北京博锐尚格节能技术股份有限公司 | Alarm method for cold station group control system |
| CN106371939A (en) * | 2016-09-12 | 2017-02-01 | 山东大学 | Time-series data exception detection method and system thereof |
| CN106709250A (en) * | 2016-12-22 | 2017-05-24 | 河海大学 | Data flow abnormality detection method based on parallel Kalman algorithm |
| US9693223B2 (en) * | 2012-02-28 | 2017-06-27 | Samsung Electronics Co., Ltd | Apparatus and method of providing medical data based on a handover of a medical sensor |
| CN106909664A (en) * | 2017-02-28 | 2017-06-30 | 国网福建省电力有限公司 | A kind of power equipment data stream failure recognition methods |
| CN107038338A (en) * | 2017-03-22 | 2017-08-11 | 西安交通大学 | A kind of intelligent monitoring device and method based on Rankine cycle |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5035389B2 (en) * | 2010-05-20 | 2012-09-26 | トヨタ自動車株式会社 | Oxygen concentration sensor response acquisition device |
| US10839307B2 (en) * | 2015-10-09 | 2020-11-17 | International Business Machines Corporation | Data collection for predictive maintenance of networked assets leveraging mobile measurement devices |
| US10504036B2 (en) * | 2016-01-06 | 2019-12-10 | International Business Machines Corporation | Optimizing performance of event detection by sensor data analytics |
-
2017
- 2017-08-14 CN CN201710693830.0A patent/CN107484196B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102011612A (en) * | 2010-11-20 | 2011-04-13 | 无锡泰立特科技有限公司 | Multifunctional tube bundle monitoring and emergency extinguishing prevention system |
| US9693223B2 (en) * | 2012-02-28 | 2017-06-27 | Samsung Electronics Co., Ltd | Apparatus and method of providing medical data based on a handover of a medical sensor |
| CN103439758A (en) * | 2013-02-08 | 2013-12-11 | 北京佳讯飞鸿电气股份有限公司 | Data processing method for rain sensor |
| CN104096303A (en) * | 2013-04-15 | 2014-10-15 | 深圳迈瑞生物医疗电子股份有限公司 | Electronic flow monitor, monitoring method and anesthesia machine |
| CN104202765A (en) * | 2014-08-26 | 2014-12-10 | 北京邮电大学 | Method and device for detecting sensor faults based on distributed network |
| CN105184394A (en) * | 2015-08-26 | 2015-12-23 | 中国电力科学研究院 | On-line data mining optimized control method based on cyber physical system (CPS) of power distribution network |
| CN105627528A (en) * | 2016-03-30 | 2016-06-01 | 北京博锐尚格节能技术股份有限公司 | Alarm method for cold station group control system |
| CN106371939A (en) * | 2016-09-12 | 2017-02-01 | 山东大学 | Time-series data exception detection method and system thereof |
| CN106709250A (en) * | 2016-12-22 | 2017-05-24 | 河海大学 | Data flow abnormality detection method based on parallel Kalman algorithm |
| CN106909664A (en) * | 2017-02-28 | 2017-06-30 | 国网福建省电力有限公司 | A kind of power equipment data stream failure recognition methods |
| CN107038338A (en) * | 2017-03-22 | 2017-08-11 | 西安交通大学 | A kind of intelligent monitoring device and method based on Rankine cycle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107484196A (en) | 2017-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107871190B (en) | Service index monitoring method and device | |
| US9354968B2 (en) | Systems and methods for data quality control and cleansing | |
| JP6467186B2 (en) | System and method for detecting, correcting, and inspecting bad data in a data stream | |
| US10852357B2 (en) | System and method for UPS battery monitoring and data analysis | |
| WO2013136739A1 (en) | Operation administration device, operation administration method, and program | |
| US8130112B2 (en) | Method of alarm mask generation and condition monitoring of wind turbines | |
| JP2004150439A (en) | Method for determining failure phenomenon | |
| JP5965406B2 (en) | System, method and apparatus for detecting irregular sensor signal noise | |
| JP2016211838A (en) | Combustion optimization system and method | |
| JP5933386B2 (en) | Data management apparatus and program | |
| JP2020068025A (en) | System and method for anomaly characterization based on joint historical and time-series analysis | |
| CN107484196B (en) | Data quality assurance method for sensor network and computer readable medium | |
| CN116106465B (en) | Chromatograph fault diagnosis method and device based on parameter analysis | |
| CN111259338B (en) | Component failure rate correction method and device, computer equipment and storage medium | |
| JP2002214185A (en) | Detecting method and detecting device for sensor abnormality | |
| JP5025524B2 (en) | Test apparatus, test system, and test method | |
| JP2016144244A (en) | Power generation situation diagnosis method for photovoltaic power generation system, and device therefor | |
| CN111025223A (en) | Clock correction method, device, equipment and storage medium of electric energy meter | |
| CN113608953B (en) | Test data generation method and device, electronic equipment and readable storage medium | |
| CN106053978B (en) | A kind of satellite aperiodicity telemetering analog quantity interpretation method based on window | |
| KR20180081437A (en) | Outlier detecting method, device and system using the method | |
| CN113267264B (en) | Temperature detection method, system, equipment and medium of temperature sensor | |
| KR102172840B1 (en) | System for controlling and monitoring object | |
| JP6381282B2 (en) | Abnormality detection apparatus and program | |
| KR102173655B1 (en) | A method for determinating the disturbance direction in power plant |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200907 Address after: Room 221, Building 5, 11 Deshengmen Avenue, Xicheng District, Beijing, 100120 Applicant after: Borui Shangge Technology Co.,Ltd. Address before: Office b-221-a046, 2nd floor, building 2, yard 1, Nongda South Road, Haidian District, Beijing 100084 Applicant before: BEIJING SAGA CLOUD TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |