CN110286656B - False alarm filtering method and device for tolerance of error data - Google Patents
False alarm filtering method and device for tolerance of error data Download PDFInfo
- Publication number
- CN110286656B CN110286656B CN201910374187.4A CN201910374187A CN110286656B CN 110286656 B CN110286656 B CN 110286656B CN 201910374187 A CN201910374187 A CN 201910374187A CN 110286656 B CN110286656 B CN 110286656B
- Authority
- CN
- China
- Prior art keywords
- time sequence
- similarity
- suspected
- data
- alarm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000001914 filtration Methods 0.000 title claims abstract description 61
- 230000005856 abnormality Effects 0.000 claims abstract description 61
- 230000002159 abnormal effect Effects 0.000 claims abstract description 30
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 25
- 238000001514 detection method Methods 0.000 claims abstract description 20
- 238000012545 processing Methods 0.000 claims description 24
- 230000008859 change Effects 0.000 claims description 22
- 230000001133 acceleration Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 9
- 230000009191 jumping Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000000691 measurement method Methods 0.000 description 5
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000013450 outlier detection Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
- G05B19/4184—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM] characterised by fault tolerance, reliability of production system
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/31—From computer integrated manufacturing till monitoring
- G05B2219/31088—Network communication between supervisor and cell, machine group
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Manufacturing & Machinery (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Alarm Systems (AREA)
Abstract
The embodiment of the invention provides a false alarm filtering method and device for tolerance of error data. The method comprises the steps of obtaining time sequence data corresponding to equipment measured by a sensor, and determining a suspected time sequence included in the time sequence data according to a state parameter value corresponding to normal operation of the equipment; according to a preset similarity matching method, calculating the similarity between each suspected time sequence and an alarm time sequence corresponding to a historical true alarm, determining the suspected time sequence with the corresponding similarity higher than a similarity threshold as the true alarm, and calculating the abnormality of each suspected time sequence with the corresponding similarity not higher than the similarity threshold according to a preset abnormality factor detection algorithm; and judging whether the suspected time sequence is a true alarm or not according to the abnormal degree of each corresponding suspected time sequence of which the similarity is not higher than the similarity threshold. The embodiment of the invention can filter out most false alarms caused by error data and improve the alarm accuracy.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a false alarm filtering method and device for tolerance of error data.
Background
In the industrial field, in order to monitor the operating state of equipment, enterprises may install a plurality of sensors on the equipment, and determine whether the equipment is working normally according to data measured by the sensors. And the data measured by the sensors in time sequence is the so-called time series data. The enterprise judges the running state of the equipment according to certain logic by utilizing time sequence data obtained by measuring through the sensor and combining with state parameters when the equipment works normally.
However, due to factors of the sensor itself, such as the sensor may be damaged, or the sensor itself may measure inaccurately, or the sensor data may be contaminated during the process of receiving and storing the sensor data, etc., it may occur that a part of the measured data is erroneous data, and some "false alarm" may be obtained by determining the operation state of the device based on the erroneous data. By "false alarm" is meant that the device is operating properly, but the error data is not within the parameters of the state in which the device is operating properly, and therefore an alarm is given that the device is not operating properly at that time. While the sensors continuously generate data, false data in the data can continuously generate false alarms, and the false alarms bring great interference to production monitoring of enterprises.
Therefore, how to reduce the false alarm caused by the error data is an urgent technical problem to be solved in the industry.
Disclosure of Invention
To solve the problems in the prior art, embodiments of the present invention provide a false alarm filtering method and apparatus for tolerance of error data.
In a first aspect, an embodiment of the present invention provides a false alarm filtering method for tolerance of error data, including:
acquiring time sequence data corresponding to equipment measured by a sensor, and determining a suspected time sequence included in the time sequence data according to a state parameter value corresponding to normal operation of the equipment;
according to a preset similarity matching method, calculating the similarity between each suspected time sequence and an alarm time sequence corresponding to a historical true alarm, determining the suspected time sequence with the corresponding similarity higher than a similarity threshold as the true alarm, and calculating the abnormality of each suspected time sequence with the corresponding similarity not higher than the similarity threshold according to a preset abnormality factor detection algorithm;
and judging whether the suspected time sequence is a true alarm or not according to the abnormal degree of each corresponding suspected time sequence of which the similarity is not higher than the similarity threshold.
In a second aspect, an embodiment of the present invention provides a false alarm filtering apparatus for tolerance of error data, including:
the device comprises a first processing module, a second processing module and a third processing module, wherein the first processing module is used for acquiring time sequence data corresponding to equipment measured by a sensor and determining a suspected time sequence included in the time sequence data according to a state parameter value corresponding to normal operation of the equipment;
the second processing module is used for calculating the similarity between each suspected time sequence and an alarm time sequence corresponding to a historical true alarm according to a preset similarity matching method, determining the suspected time sequence with the corresponding similarity higher than a similarity threshold as the true alarm, and calculating the abnormality of each suspected time sequence with the corresponding similarity not higher than the similarity threshold according to a preset abnormality factor detection algorithm;
and the third processing module is used for judging whether the suspected time sequence is a true alarm or not according to the abnormal degree of each corresponding suspected time sequence, wherein the similarity of each suspected time sequence is not higher than the similarity threshold.
In a third aspect, an embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the error data tolerant false alarm filtering method according to the first aspect when executing the program.
In a fourth aspect, embodiments of the present invention provide a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the false alarm filtering method for fault data tolerance as described in the first aspect.
The false alarm filtering method and the false alarm filtering device for tolerance of the error data provided by the embodiment of the invention can filter most false alarms caused by the error data by matching the similarity between the suspected time sequence which can possibly generate the alarm and the historical true alarms and calculating the abnormality degree of the suspected time sequence to filter the false alarms caused by the error data, thereby improving the accuracy of the alarm.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a flow chart illustrating a method for false alarm filtering for tolerance of error data according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating a method for false alarm filtering for tolerance of error data according to another embodiment of the present invention;
FIG. 3 is a flowchart illustrating a method for false alarm filtering for tolerance to erroneous data according to another embodiment of the present invention;
FIG. 4 is a schematic diagram of a false alarm filtering apparatus for tolerance of error data according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic flow diagram of a false alarm filtering method for tolerance of error data according to an embodiment of the present invention, and as shown in fig. 1, an embodiment of the present invention provides a following solution for a situation that a "false alarm" occurs in an operating state of an error data determination device, and specifically includes the following steps:
in order to monitor the operating state of the equipment, enterprises may install various types of sensors on the equipment, and detect various operating parameter values of the equipment in real time according to the various sensors. The sensors collect device data in a time sequence, the collected data forming time series data.
The execution subject of the flow of the embodiment of the method may be a device, such as a CPU or a DSP, for executing the false alarm filtering method for tolerance of error data, and the device acquires the timing data corresponding to the device measured by the sensor from the sensor. Then, the device compares the time sequence data with a state parameter value corresponding to the normal operation of the equipment, and determines a suspected time sequence included in the time sequence data. The state parameter value corresponding to the normal operation of the equipment is the data detected by the equipment under the condition that no alarm occurs.
In the embodiment of the method, the value of the state parameter when the equipment normally runs is taken as the threshold value, and whether the data measured by the sensor falls within the normal numerical range or not is judged. The logic of the determination may be: for example, if the status data of the device obtained for one minute is below the threshold, the device may be bad and an alarm may need to be generated. This time series data for one continuous minute is a suspected time series that may generate an alarm. The method comprises the steps of taking a state parameter value of each device in normal operation as a threshold value, filtering time sequence data obtained by measurement of a sensor, filtering data which do not accord with alarm logic, and obtaining suspected time sequences which can possibly generate alarms.
after the suspected time sequences are obtained through threshold filtering, the similarity between each suspected time sequence and the alarm time sequence corresponding to the historical true alarm can be calculated according to a preset similarity matching method. The alarm time sequence corresponding to the historical true alarm is stored in a historical true alarm database, and each alarm time sequence stored in the true alarm database is data causing real alarm of equipment, but not false alarm. In the embodiment of the method, the similarity between the suspected Time sequence and the alarm Time sequence is calculated according to a similarity matching method, for example, by using a Dynamic Time Warping (DTW).
And determining the suspected time sequence with the similarity higher than the similarity threshold as a true alarm by calculating the similarity, namely determining that the suspected time sequence is regarded as the true alarm if the similarity between a certain suspected time sequence and the alarm time sequence causing the true alarm is higher than a preset similarity threshold. Specifically, the suspected time series obtained by threshold filtering and the alarm series which actually appears in history are subjected to similarity calculation, the similarity is high, the suspected time series is similar to the real alarm in history, the device which is in the state in history alarms, and the suspected time series at the moment is the real alarm with a high probability. The similarity measurement method can use DTW or other time sequence data measurement methods, the reason for calculating the similarity of two time sequence data sequences by using DTW is that the time sequence data sequences may have deletion, so that the dimensions of the two sequences are inconsistent, and the DTW can tolerate the condition of data deletion.
The degree OF abnormality OF each suspected time sequence with the corresponding degree OF similarity not higher than the similarity threshold can be calculated according to a preset abnormal Factor detection algorithm, such as a local abnormal Factor detection algorithm (L oclilier Factor; L OF). The suspected time sequence with the high degree OF similarity is considered to be a true alarm, and the suspected time sequence with the low degree OF similarity is not necessarily considered to be a true alarm, because the suspected time sequence does not appear in history, the suspected time sequence cannot be judged to be a true alarm, and the degree OF abnormality needs to be further calculated for judgment.
And 102, judging whether the suspected time sequence is a true alarm or not according to the abnormal degree of each corresponding suspected time sequence of which the similarity is not higher than the similarity threshold.
And calculating the corresponding abnormality degree OF the suspected time sequence with the similarity not higher than the similarity threshold according to the L OF algorithm, and judging whether the suspected time sequence is a true alarm or not according to the abnormality degree.
According to the false alarm filtering method for tolerance of error data provided by the embodiment of the invention, the false alarms caused by the error data are filtered by matching the similarity between the suspected time sequence which can possibly generate the alarm and the historical true alarms and calculating the abnormality degree of the suspected time sequence, so that most of the false alarms caused by the error data can be filtered, and the accuracy of the alarm is improved.
The false alarm filtering method for tolerance OF the error data provided by the embodiment OF the invention is used for filtering false alarms generated by the error data based on a time sequence data similarity matching method and an abnormal factor detection algorithm, and can realize filtering OF most OF false alarms generated by the error data.
Fig. 2 is a schematic flow chart of a false alarm filtering method for tolerance of error data according to another embodiment of the present invention, as shown in fig. 2, the method includes the following steps:
200, acquiring time sequence data corresponding to equipment measured by a sensor, and determining a suspected time sequence included in the time sequence data according to a state parameter value corresponding to normal operation of the equipment;
Specifically, when the state value of the device is continuously lower than a certain value, the device may be in an abnormal state, and an alarm needs to be generated at this time, and this value is a threshold value for determining whether the device is operating normally. And if the continuous time is less than the threshold value, generating the alarm is alarm judgment logic, and the alarm judgment logic and the threshold value are used for filtering the time sequence data to find a suspected time sequence which is possible to generate the alarm.
The obtained suspected time sequence is matched with a historical real alarm time sequence, the matching method can be a DTW or other time sequence data similarity measurement method, the suspected time sequence with high similarity is judged as a true alarm, the time sequence with low similarity needs further judgment, the suspected time sequence with low similarity to the historical true alarm obtained by filtering can be a false alarm or a true alarm, and needs further judgment, an outlier detection algorithm L OF or other abnormal detection algorithms can be adopted, the suspected time sequence with low abnormality is judged as a false alarm by calculating the abnormality OF the suspected time sequence, the false alarms are filtered, the obtained suspected time sequence with the abnormality higher than a certain threshold is fed back to service personnel, the service personnel judges whether the suspected time sequence is the true alarm, the suspected time sequence corresponding to the true alarm is determined and is added into a historical true alarm database as the historical true alarm, and if the suspected time sequence is not the true alarm, the filtering is carried out.
In the method provided by the embodiment of the method, the abnormal degree of the suspected time sequences which are obtained by filtering and have low similarity with historical true alarms is calculated, the suspected time sequences which have high abnormal degree have high probability of being true alarms, the sequences which have low abnormal degree are shown to be historically appeared, and no alarm is generated, so the sequences can be determined to be false alarms, and filtering can be carried out. When the degree of abnormality is calculated, a pseudo time series is taken as a calculation target, and it is assumed that a certain pseudo time series is represented by S _ t { (t _1, v _1), (t _2, v _2), …, (t _ n, v _ n) }, which is a value measured by the sensor at a time point. In the calculation process, the time point is omitted, and each subsequence is represented as (v _1, v _2, …, v _ n) to calculate the degree of abnormality.
Further, the embodiment of the method further includes a step of performing jump clearing after acquiring time series data corresponding to the device measured by the sensor, where the step includes: and after time sequence data corresponding to the equipment measured by the sensor is obtained, automatically jumping and clearing the time sequence data according to a 3 sigma criterion so as to delete the data jumping in the time sequence data.
Specifically, some jump data may appear in the time series data, and the jump data may interfere with the determination of the false alarm, so that the jump data needs to be deleted. The jump data is a data point in which an abnormality in speed change or an abnormality in acceleration change occurs according to the 3 σ criterion. Specifically, through a 3 σ criterion, a single point, namely a jump point, which causes speed change abnormality and acceleration change abnormality is found, and the found jump point is likely to be some wrong data, which may affect the judgment of a suspected time sequence which may cause an alarm, and therefore needs to be cleared.
The false alarm filtering method for tolerance of error data provided by the embodiment of the invention filters out the trip point in the data through a 3 sigma criterion; according to the running state of the equipment, judging logic to find a suspected time sequence which is possible to generate an alarm from the data after the trip points are filtered; calculating the similarity of the suspected time sequence which is possible to generate alarm and the historical true alarm time sequence data, judging the suspected time sequence as the true alarm if the similarity is high, and filtering if the similarity is low; calculating the sequence abnormality degree of the suspected time sequence with low similarity obtained by filtering; and feeding back the time sequence with the abnormality degree exceeding a certain threshold value to service personnel to judge whether the time sequence is a true alarm.
Fig. 3 is a flowchart illustrating a false alarm filtering method for tolerance to error data according to another embodiment of the present invention, as shown in fig. 3, the method includes:
various operating parameter values of the equipment are collected based on sensors installed on the equipment. The sensors collect device data in a time sequence, the collected data forming time series data.
the step of clearing the self-jump of the time series data according to the 3 σ criterion may specifically include: the time series data is used as input, the change speed and the change acceleration are calculated based on the state parameter values corresponding to the actual operation of the equipment, and the mean value u and the standard deviation sigma of the change speed and the mean value u and the standard deviation sigma of the change acceleration are sequentially obtained. And determining data of which the variation speed and the variation acceleration fall outside an interval (u-3 sigma, u +3 sigma) as data of which the jump occurs in the time series data according to a 3 sigma criterion, and deleting the abnormal point.
And determining the position of the original data of the equipment state through the positions of the abnormal points of the variable speed and the variable acceleration. Assuming that the position of the abnormal single point of the change speed is on the m-th row, the change speed of this position is calculated from the data measured at the m-th time point and the data measured at the m + 1-th time point of the device state data, and therefore the m-th time point data and the m + 1-th time point data in the device state data are marked as data to be deleted. Assuming that the position of the abnormal single point of the variation acceleration is on the nth row, the variation acceleration of the position is calculated from the data measured at the nth time point, the (n + 1) th time point and the (n + 2) th time point of the device state data, and therefore the (n + 1) th time point data in the device state data is marked as the data to be deleted. And taking a union set of the positions of the data points to be deleted, and clearing the jumping data points at the corresponding positions of the state data of the original equipment.
according to a preset threshold value of abnormal work of the equipment and alarm judgment logic, on the basis of filtering out the trip point, a suspected time sequence with continuous time lower than the threshold value is searched. Assuming that the water temperature pressure measured by the engine for one minute is lower than the threshold s, the device is considered to be in failure, at which time an alarm should be generated. The logic of threshold filtering is to segment the time series data measured by the sensor at one minute intervals, each segment of data is the state data of the equipment measured in the same minute, judge whether the water temperature and pressure measured in the consecutive minute is lower than the threshold s, if yes, regard the time series data as the suspected time series data which may generate alarm, if not, filter. And searching a suspected time sequence which can generate an alarm on the basis of clearing the jump data in the previous step, and outputting the suspected time sequence.
and taking each section of suspected time sequence obtained in the last step as a calculation object, and respectively calculating the similarity with the historical time sequence data of real alarm. The similarity measurement method can be DTW, and the algorithm is based on the idea of dynamic programming and solves the problem of template matching with different sequence lengths. One important reason for using DTW is that data may be missing in the time series, that is, data at some time points are not collected, and if the similarity calculation method is used in the conventional similarity calculation method, the similarity error of the time series with inconsistent length may be very large, and the matching effect with the true alarm may be poor. The similarity calculation method based on the DTW algorithm can find a sequence highly similar to the historical true alarm in the suspected time sequence, and the sequence is determined as the true alarm, i.e. step 304 is executed. The suspected time series with low similarity will be used as the input of the next step, i.e. step 305.
the suspected time series with low matching degree with the historical true alarm is used as the input OF the step, and the abnormality degree OF each series is calculated by an abnormality detection algorithm, taking a local abnormality factor detection algorithm L OF as an example, the local reachable density OF each point is calculated by setting a parameter k and the distance OF the kth neighbor OF each point, so that the local outlier factor OF the point is calculated, and the abnormality degree OF each point is obtained.
setting a threshold value of the abnormal degree according to the abnormal list of the suspected time series obtained in the step 305, filtering the suspected time series lower than the threshold value, returning the suspected time series which is more than or equal to the threshold value, feeding back to service personnel, and judging whether the suspected time series is a true alarm. If yes, adding the historical true alarm library, and otherwise, filtering.
The false alarm filtering method for tolerance of error data provided by the embodiment of the invention filters out the trip point in the data through the 3 sigma criterion, matches the true alarm based on the similarity measurement method, and searches the time sequence with high abnormality degree through the abnormality detection algorithm to judge whether the time sequence is the true alarm.
Fig. 4 is a schematic structural diagram of an error data tolerant false alarm filtering apparatus according to an embodiment of the present invention, as shown in fig. 4, the apparatus includes a first processing module 401, a second processing module 402, and a third processing module 403, where:
the first processing module 401 is configured to acquire time series data corresponding to a device measured by a sensor, and determine a suspected time sequence included in the time series data according to a state parameter value corresponding to normal operation of the device; the second processing module 402 is configured to calculate, according to a preset similarity matching method, a similarity between each suspected time sequence and an alarm time sequence corresponding to a historical true alarm, determine a suspected time sequence having a corresponding similarity higher than a similarity threshold as a true alarm, and calculate, according to a preset anomaly factor detection algorithm, an anomaly of each suspected time sequence having a corresponding similarity not higher than the similarity threshold; the third processing module 403 is configured to determine whether each suspected time sequence is a true alarm according to the abnormality degree of each suspected time sequence whose similarity degree is not higher than the similarity degree threshold.
Specifically, the first processing module 401 searches for a suspected time sequence which may cause an alarm in the data after abnormal single-point filtering through an alarm logic, the second processing module 402 calculates the similarity between the suspected time sequence found by the suspected time sequence discovery module and the historical true alarm time sequence data, determines that the suspected time sequence with high similarity is a true alarm, the third processing module 403 calculates the abnormality degree of the obtained time sequence with low similarity by using an abnormality detection algorithm, sets a threshold to filter the time sequence with low abnormality degree, returns the suspected time sequence with high abnormality degree, and feeds back the time sequences to the service staff.
Further, the third processing module 403 includes: the first unit is used for filtering the suspected time sequence if the corresponding similarity is not higher than the abnormality degree of the suspected time sequence of the similarity threshold value and is lower than the abnormality degree threshold value; and the second unit is used for outputting the suspected time sequence for manual judgment of whether the suspected time sequence is a true alarm or not if the corresponding similarity of the suspected time sequence is not higher than the similarity threshold and is not lower than the abnormality threshold.
Still further, the false alarm filtering apparatus for tolerance of error data provided in this embodiment may further include a fourth processing module, configured to calculate a mean value and a standard deviation of a speed of change of the data, calculate a mean value and a standard deviation of an acceleration of change of the data, find, based on a 3 σ criterion, a single point of the data that causes an anomaly in the speed change and an anomaly in the acceleration change, and filter out data points that jump.
The false alarm filtering apparatus for tolerance of error data provided in the embodiments of the present invention is specifically configured to execute the false alarm filtering method for tolerance of error data provided in the above-mentioned method embodiments, and specific functions and procedures thereof may be referred to in the above-mentioned method embodiments, and are not described herein again.
The false alarm filtering device for tolerance of error data provided by the embodiment of the invention filters single points causing data change speed abnormity and data change acceleration abnormity according to a 3 sigma criterion, searches a suspected time sequence possibly causing alarm according to alarm judgment logic on the basis, matches the searched suspected time sequence with historical true alarms in similarity, calculates the abnormality degree of the suspected time sequence with low similarity degree, returns the suspected time sequence with high abnormality degree, filters all suspected time sequences with low abnormality degree, realizes filtering of false alarms generated by error data, and can realize filtering of false alarms generated by equipment in any type of scenes.
Fig. 5 is a schematic structural diagram of an entity of an electronic device according to an embodiment of the present invention, and as shown in fig. 5, the server may include: a processor (processor)510, a communication Interface (Communications Interface)520, a memory (memory)530 and a communication bus 540, wherein the processor 510, the communication Interface 520 and the memory 530 communicate with each other via the communication bus 540. Processor 510 may invoke computer programs in memory 530 to perform the methods provided by the various embodiments described above, including, for example: acquiring time sequence data corresponding to equipment measured by a sensor, and determining a suspected time sequence included in the time sequence data according to a state parameter value corresponding to normal operation of the equipment; according to a preset similarity matching method, calculating the similarity between each suspected time sequence and an alarm time sequence corresponding to a historical true alarm, determining the suspected time sequence with the corresponding similarity higher than a similarity threshold as the true alarm, and calculating the abnormality of each suspected time sequence with the corresponding similarity not higher than the similarity threshold according to a preset abnormality factor detection algorithm; and judging whether the suspected time sequence is a true alarm or not according to the abnormal degree of each corresponding suspected time sequence of which the similarity is not higher than the similarity threshold.
In addition, the computer program stored in the memory 530 may be implemented in the form of a software functional unit and may be stored in a computer readable storage medium when the computer program is sold or used as a separate product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Embodiments of the present invention further provide a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the methods provided in the foregoing embodiments, and the methods include: acquiring time sequence data corresponding to equipment measured by a sensor, and determining a suspected time sequence included in the time sequence data according to a state parameter value corresponding to normal operation of the equipment; according to a preset similarity matching method, calculating the similarity between each suspected time sequence and an alarm time sequence corresponding to a historical true alarm, determining the suspected time sequence with the corresponding similarity higher than a similarity threshold as the true alarm, and calculating the abnormality of each suspected time sequence with the corresponding similarity not higher than the similarity threshold according to a preset abnormality factor detection algorithm; and judging whether the suspected time sequence is a true alarm or not according to the abnormal degree of each corresponding suspected time sequence of which the similarity is not higher than the similarity threshold.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for false alarm filtering for tolerance of erroneous data, comprising:
acquiring time sequence data corresponding to equipment measured by a sensor, and determining a suspected time sequence included in the time sequence data according to a state parameter value corresponding to normal operation of the equipment;
according to a preset similarity matching method, calculating the similarity between each suspected time sequence and an alarm time sequence corresponding to a historical true alarm, determining the suspected time sequence with the corresponding similarity higher than a similarity threshold as the true alarm, and calculating the abnormality of each suspected time sequence with the corresponding similarity not higher than the similarity threshold according to a preset abnormality factor detection algorithm;
and judging whether the suspected time sequence is a true alarm or not according to the abnormal degree of each corresponding suspected time sequence of which the similarity is not higher than the similarity threshold.
2. The method of claim 1, wherein the determining whether the suspected time series is true according to the abnormality degree of each suspected time series whose similarity degree is not higher than the similarity degree threshold comprises:
if the corresponding similarity is not higher than the degree of abnormality of the suspected time sequence of the similarity threshold value and is lower than the degree of abnormality threshold value, filtering the suspected time sequence;
and if the corresponding similarity is not higher than the abnormality degree of the suspected time sequence of the similarity threshold and is not lower than the abnormality degree threshold, outputting the suspected time sequence for manual judgment of whether the suspected time sequence is a true alarm.
3. The method of claim 1 or 2, wherein the obtaining of the timing data corresponding to the device measured by the sensor comprises:
and after time sequence data corresponding to the equipment measured by the sensor is obtained, automatically jumping and clearing the time sequence data according to a 3 sigma criterion so as to delete the data jumping in the time sequence data.
4. The method of claim 3, wherein the step of self-jump clearing the time-series data according to 3 σ criterion comprises:
taking the time sequence data as input, calculating a change speed and a change acceleration based on a state parameter value corresponding to actual operation of the equipment, and calculating to obtain a mean value u and a standard deviation sigma of the change speed, and a mean value u and a standard deviation sigma of the change acceleration;
and determining data of which the change speed and the change acceleration fall outside an interval (u-3 sigma, u +3 sigma) as data of which the jump occurs in the time series data according to a 3 sigma criterion, and deleting a time point of the data of which the jump occurs.
5. The method of claim 1, wherein the similarity matching method is a dynamic time warping algorithm, and the anomaly detection algorithm is a local anomaly detection algorithm.
6. The method of false data tolerant false alarm filtering according to claim 1 or 2, wherein the method further comprises:
and adding the suspected time sequence determined to correspond to the true alarm as a historical true alarm into a historical true alarm database.
7. A false alarm filtering device for fault data tolerance, comprising:
the device comprises a first processing module, a second processing module and a third processing module, wherein the first processing module is used for acquiring time sequence data corresponding to equipment measured by a sensor and determining a suspected time sequence included in the time sequence data according to a state parameter value corresponding to normal operation of the equipment;
the second processing module is used for calculating the similarity between each suspected time sequence and an alarm time sequence corresponding to a historical true alarm according to a preset similarity matching method, determining the suspected time sequence with the corresponding similarity higher than a similarity threshold as the true alarm, and calculating the abnormality of each suspected time sequence with the corresponding similarity not higher than the similarity threshold according to a preset abnormality factor detection algorithm;
and the third processing module is used for judging whether the suspected time sequence is a true alarm or not according to the abnormal degree of each corresponding suspected time sequence, wherein the similarity of each suspected time sequence is not higher than the similarity threshold.
8. The erroneous-data-tolerant false alarm filtering device of claim 7, wherein the third processing module comprises:
the first unit is used for filtering the suspected time sequence if the corresponding similarity is not higher than the abnormality degree of the suspected time sequence of the similarity threshold value and is lower than the abnormality degree threshold value;
and the second unit is used for outputting the suspected time sequence for manual judgment of whether the suspected time sequence is a true alarm or not if the corresponding similarity of the suspected time sequence is not higher than the similarity threshold and is not lower than the abnormality threshold.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the error data tolerant false alarm filtering method according to any of claims 1 to 6 when executing the program.
10. A non-transitory computer readable storage medium, having stored thereon a computer program, which, when being executed by a processor, carries out the steps of the false alarm filtering method for fault data tolerance of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910374187.4A CN110286656B (en) | 2019-05-07 | 2019-05-07 | False alarm filtering method and device for tolerance of error data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910374187.4A CN110286656B (en) | 2019-05-07 | 2019-05-07 | False alarm filtering method and device for tolerance of error data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110286656A CN110286656A (en) | 2019-09-27 |
| CN110286656B true CN110286656B (en) | 2020-07-17 |
Family
ID=68001831
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910374187.4A Active CN110286656B (en) | 2019-05-07 | 2019-05-07 | False alarm filtering method and device for tolerance of error data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110286656B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111338892B (en) * | 2020-02-17 | 2022-02-15 | 北京理工大学 | Time sequence rise abnormity identification method under extreme operation condition |
| CN113381874B (en) * | 2020-03-10 | 2023-06-02 | 上海杰之能软件科技有限公司 | Fault signal processing method, storage medium and terminal |
| CN111726341B (en) * | 2020-06-02 | 2022-10-14 | 五八有限公司 | Data detection method and device, electronic equipment and storage medium |
| CN111897695B (en) * | 2020-07-31 | 2022-06-17 | 平安科技（深圳）有限公司 | Method and device for acquiring KPI abnormal data sample and computer equipment |
| CN112115184A (en) * | 2020-09-18 | 2020-12-22 | 平安科技（深圳）有限公司 | Time series data detection method and device, computer equipment and storage medium |
| CN113077613A (en) * | 2021-04-07 | 2021-07-06 | 浙江佳伯尔电子科技有限公司 | Alarm for preventing sensor misjudgment and judgment method thereof |
| CN116821833B (en) * | 2023-08-29 | 2023-11-14 | 好林(威海)新材料有限公司 | Data anomaly detection method for machine filter cloth adhesion |
| CN117193164B (en) * | 2023-11-08 | 2024-01-30 | 东莞市博思特数控机械有限公司 | Fault monitoring method and system of numerical control machine tool |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104075749A (en) * | 2014-06-30 | 2014-10-01 | 通号通信信息集团有限公司 | Abnormal state detecting method and system for equipment in internet of things |
| CN104700532A (en) * | 2013-12-11 | 2015-06-10 | 杭州海康威视数字技术股份有限公司 | Video alarm method and video alarm device |
| CN105764162A (en) * | 2016-05-10 | 2016-07-13 | 江苏大学 | Wireless sensor network abnormal event detecting method based on multi-attribute correlation |
| CN107798272A (en) * | 2016-08-30 | 2018-03-13 | 佳能株式会社 | Fast multi-target detects and tracking system |
| CN108319981A (en) * | 2018-02-05 | 2018-07-24 | 清华大学 | A kind of time series data method for detecting abnormality and device based on density |
| CN108595300A (en) * | 2018-03-21 | 2018-09-28 | 北京奇艺世纪科技有限公司 | A kind of method and device of configurable monitoring and alarm |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2897112B1 (en) * | 2014-01-17 | 2019-03-06 | Wincor Nixdorf International GmbH | Method and apparatus for the prevention of false alarms in monitoring systems |
| US10726540B2 (en) * | 2017-10-17 | 2020-07-28 | International Business Machines Corporation | Self-similarity analysis for defect detection on patterned industrial objects |
-
2019
- 2019-05-07 CN CN201910374187.4A patent/CN110286656B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104700532A (en) * | 2013-12-11 | 2015-06-10 | 杭州海康威视数字技术股份有限公司 | Video alarm method and video alarm device |
| CN104075749A (en) * | 2014-06-30 | 2014-10-01 | 通号通信信息集团有限公司 | Abnormal state detecting method and system for equipment in internet of things |
| CN105764162A (en) * | 2016-05-10 | 2016-07-13 | 江苏大学 | Wireless sensor network abnormal event detecting method based on multi-attribute correlation |
| CN107798272A (en) * | 2016-08-30 | 2018-03-13 | 佳能株式会社 | Fast multi-target detects and tracking system |
| CN108319981A (en) * | 2018-02-05 | 2018-07-24 | 清华大学 | A kind of time series data method for detecting abnormality and device based on density |
| CN108595300A (en) * | 2018-03-21 | 2018-09-28 | 北京奇艺世纪科技有限公司 | A kind of method and device of configurable monitoring and alarm |
Non-Patent Citations (3)
| Title |
|---|
| False alarms in episodic recognition : an examination of base-rate, similarity-based and comprehensive theories;Angela M. Maguire.;《memory》;20050101;全文 * |
| 搜索网络中虚警干扰内容隔离技术研究仿真;马一路;《计算机仿真》;20150123;全文 * |
| 无线传感器网络环境下的一种轻量级事件容错检测算法;董传明, 刘克中, 罗广,等;《传感技术学报》;20140315;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110286656A (en) | 2019-09-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110286656B (en) | False alarm filtering method and device for tolerance of error data | |
| CN105718715B (en) | Method for detecting abnormality and equipment | |
| WO2021179574A1 (en) | Root cause localization method, device, computer apparatus, and storage medium | |
| CA2634328C (en) | Method and system for trend detection and analysis | |
| CN112348237B (en) | Abnormal trend detection method for dynamic drilling data | |
| KR102440335B1 (en) | A method and apparatus for detecting and managing a fault | |
| CN112149860A (en) | Automatic anomaly detection method and system | |
| WO2015121176A1 (en) | Method of identifying anomalies | |
| US11733688B2 (en) | System and method for recognizing and forecasting anomalous sensory behavioral patterns of a machine | |
| JP7127305B2 (en) | Information processing device, information processing method, program | |
| CN107391335B (en) | Method and equipment for checking health state of cluster | |
| KR20210017651A (en) | Method for Fault Detection and Fault Diagnosis in Semiconductor Manufacturing Process | |
| Folmer et al. | Detection of temporal dependencies in alarm time series of industrial plants | |
| CN115311829B (en) | Accurate alarm method and system based on mass data | |
| CN118174788B (en) | Fault detection method, device and equipment of optical fiber wiring cabinet and storage medium | |
| JP2022084435A5 (en) | ||
| US20230153428A1 (en) | Information processing device, information processing method, recording medium, information processing system | |
| CN114285612B (en) | Method, system, device, equipment and medium for detecting abnormal data | |
| CN110489260B (en) | Fault identification method and device and BMC | |
| CN106652393B (en) | False alarm determination method and device | |
| CN111371647A (en) | Data center monitoring data preprocessing method and device | |
| JP2008146157A (en) | Network abnormality decision device | |
| KR102108975B1 (en) | Apparatus and method for condition based maintenance support of naval ship equipment | |
| CA3114157C (en) | Process management device, process management method, and process management program storage medium | |
| US10295965B2 (en) | Apparatus and method for model adaptation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |