CN112699946A - Anomaly detection method and device for sensing monitoring data and related equipment - Google Patents
Anomaly detection method and device for sensing monitoring data and related equipment Download PDFInfo
- Publication number
- CN112699946A CN112699946A CN202011640085.1A CN202011640085A CN112699946A CN 112699946 A CN112699946 A CN 112699946A CN 202011640085 A CN202011640085 A CN 202011640085A CN 112699946 A CN112699946 A CN 112699946A
- Authority
- CN
- China
- Prior art keywords
- data
- training
- current measured
- training set
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Abstract
The invention discloses an anomaly detection method aiming at sensing monitoring data, which is applied to the technical field of sensor anomaly data processing and is used for solving the technical problem of small application range caused by large limitation of the existing anomaly detection method. The method provided by the invention comprises the following steps: selecting corresponding training data from the sensing monitoring data according to a preset rule for the current measured data in the sensing monitoring data to construct a training set; constructing a detection model based on the training set, and acquiring corresponding model error characteristics according to the detection model; calculating the fitting degree of the detection model according to the model error characteristics; obtaining an abnormal threshold interval according to the fitting degree; and determining whether the current measured data is abnormal or not according to the abnormal threshold interval.
Description
Technical Field
The invention relates to the technical field of sensor abnormal data processing, in particular to an abnormality detection method and device for sensor monitoring data, computer equipment and storage medium
Background
The industries of manufacturing, communication, transportation and the like rely heavily on the detection and monitoring of related equipment facilities by sensors to judge whether the related equipment facilities operate normally. If the sensor fails, the control of the state of the equipment facility can be affected, and even serious safety accidents can be caused.
The existing sensor abnormality detection methods mainly include the following methods:
(1) and constructing a neural network, and detecting abnormal data points by using a preset threshold interval or a prediction model.
(2) And carrying out model construction and training by using the historical offline data and the real-time data.
(3) And performing model prediction and detecting abnormal data points by using the autocorrelation model.
The method (1) adapts to data in a mode of manually controlling the threshold value aiming at different scenes, and overfitting is easy to occur when different data sets are subjected to hyper-parameter adjustment, so that the abnormal data is inaccurately detected; the training of the historical data by the method (2) takes a lot of time and cannot be suitable for the continuous data with timeliness; the method (3) cannot accurately detect the time series data of the equal interval generally due to the equipment of the sensor.
In summary, the conventional anomaly detection method for sensor data has a high requirement on the construction of the early detection environment, and cannot detect anomalous data points existing in time series data at equal intervals. Therefore, the application range of the existing anomaly detection method is small due to the large limitation, and the existing method needs to be improved so as to detect the anomaly data points in the time series data with equal intervals, so that the method is suitable for more industrial demands.
Disclosure of Invention
The invention provides an anomaly detection method, an anomaly detection device, computer equipment and a storage medium for sensing monitoring data, and aims to solve the technical problem that the application range is small due to large limitation of the existing anomaly detection method.
An anomaly detection method for sensing monitoring data, comprising:
selecting corresponding training data from the sensing monitoring data according to a preset rule for the current measured data in the sensing monitoring data to construct a training set;
constructing a detection model based on the training set, and acquiring corresponding model error characteristics according to the detection model;
calculating the fitting degree of the detection model according to the model error characteristics;
obtaining an abnormal threshold interval according to the fitting degree;
and determining whether the current measured data is abnormal or not according to the abnormal threshold interval.
An anomaly detection apparatus for sensing monitored data, comprising:
the training set construction module is used for selecting corresponding training data from the sensing monitoring data according to a preset rule to construct a training set for the current measured data in the sensing monitoring data;
the model error characteristic acquisition module is used for constructing a detection model based on the training set and acquiring corresponding model error characteristics according to the detection model;
the fitting degree calculation module is used for calculating the fitting degree of the detection model according to the model error characteristics;
an abnormal threshold interval obtaining module, configured to obtain an abnormal threshold interval according to the fitting degree;
and the abnormal data determining module is used for determining whether the current measured data is abnormal or not according to the abnormal threshold interval.
A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the above-described anomaly detection method for sensing monitored data when executing the computer program.
A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the above-mentioned steps of the anomaly detection method for sensing monitored data.
The invention provides an anomaly detection method, an anomaly detection device, computer equipment and a storage medium for sensing monitoring data, which aim at current detected data in the sensing monitoring data, select corresponding training data from the sensing monitoring data according to preset rules to construct a training set, construct a detection model based on the training set, obtain corresponding model error characteristics according to the detection model, calculate the fitting degree of the detection model according to the model error characteristics, obtain an anomaly threshold interval according to the fitting degree, determine whether the current detected data is abnormal according to the anomaly threshold interval, construct the detection model according to the time sequence correlation of the sensing monitoring data, and can repair the detected abnormal data, improve the applicability of the anomaly detection method for the sensing monitoring data and better meet industrial requirements.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
FIG. 1 is a schematic diagram of an application environment of an anomaly detection method for sensing monitored data according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method for anomaly detection for sensing monitored data according to an embodiment of the present invention;
FIG. 3 is a flow chart of a method for anomaly detection for sensing monitored data according to another embodiment of the present invention;
FIG. 4 is a schematic structural diagram of an anomaly detection apparatus for sensing monitored data according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a computer device according to an embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The anomaly detection method for the sensing monitoring data provided by the application can be applied to the application environment shown in fig. 1, wherein the computer equipment is communicated with the server through a network. The computer device may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, among others. The server may be implemented as a stand-alone server or as a server cluster consisting of a plurality of servers.
In an embodiment, as shown in fig. 2, an anomaly detection method for sensing monitoring data is provided, which is described by taking the method as an example applied to the server in fig. 1, and includes steps S101 to S105.
S101, aiming at the current measured data in the sensing monitoring data, selecting corresponding training data from the sensing monitoring data according to a preset rule to construct a training set.
Further, in step S101, the step of selecting corresponding training data from the sensing monitoring data according to a preset rule for the current measured data in the sensing monitoring data to construct a training set includes the following steps S1011 to S1013:
and S1011, selecting a plurality of training data from the sensing monitoring data according to a preset length window based on the current measured data.
The sensing monitoring data is a string of time sequence data, the preorder training data before the moment of the current measured data is selected by a time window with a preset length, a plurality of training data are arranged in the preorder training data, and the training data are distributed discretely. To ensure the timing of the subsequent detection model, the length of the time window is set to Wmax.
S1012, calculating the similarity between the training data and the current measured data.
Further, the step S1012 includes the following steps a to b:
a. the training data or the current measured data are normalized according to the following formula:
wherein Z isnorm,iNormalized value, Z, for training data or current measured dataiFor the i-th dimension, Z, of the training data or current measured datamax,iIs the maximum of the i-th dimension, Z, of all training datamin,iIs the minimum of the ith dimension of all training data. Wherein, the dimension refers to some characteristics (such as temperature) of the thing that the training data or the current measured data refers to.
b. Calculating the similarity degree of the training data and the current measured data according to the following formula:
wherein x is the current measured data, y is the training data, D (x, y) is the Euclidean distance between the current measured data and the training data for representing the similarity between the training data and the current measured data, xiFor the i-th dimension, y, of the current data under testiAnd m is the dimension number of the training data or the current measured data.
In this embodiment, before calculating the similarity between each of the plurality of training data and the current data to be tested, the training data and the current data to be tested are normalized in step a, so that the influence of the characteristics of the training data and the current data to be tested on the calculation similarity can be kept consistent.
And S1013, selecting corresponding training data to construct a training set based on the degree of similarity.
After similarity calculation is carried out on all training data selected by a preset length window, sorting is carried out according to the similarity, and the first k training data are selected to construct a training set. Wherein the value range of k is [ k ]min,Wmax]，kminIs a predetermined minimum value, Wmax isThe preset length of the preset length window.
Further, on the basis of step S101, step S1013 includes the following steps S111 to S112:
and S111, constructing a plurality of training sets according to the selected training data, wherein the number of the training data included in each training set is different.
Wherein, in the plurality of constructed training sets, the number of training data included in each training set is from kminThe iteration is started. For example, k is included in the first training setminTraining data, including k in the second training setmin+1 training data.
And S112, calculating the characteristic value of the discrete degree of each training set, and selecting the training set for constructing the detection model according to the characteristic value of the discrete degree.
Further, the step of calculating the characteristic value of the degree of dispersion of each training set in step S112 specifically includes:
calculating the characteristic value of the discrete degree of each training set according to the following formula:
wherein, δ is the characteristic value of the discrete degree of each training set, y is the training data in the training set, μ is the average value of all the training data in the training set, and k is the number of the training data in the training set.
Further, the step S112 of selecting a training set for constructing the detection model according to the characteristic value of the degree of dispersion specifically includes:
selecting a training set for constructing a detection model according to the following formula:
wherein S isa-1And a-1 training set, wherein the number of the training data is k. DeltaaThe characteristic value of the discrete degree of the training set with the training data number a,δa-1the discrete degree characteristic value of the training set with the training data number of a-1 is shown, and C is a discrete degree threshold value.
Is the difference between the discrete degree eigenvalues of the two training sets, if
A training set with a training data number a-1 is selected for subsequent construction of the detection model.
S102, constructing a detection model based on the training set, and obtaining corresponding model error characteristics according to the detection model.
Wherein, step S102 includes the following steps S1021 to S1022:
s1021, constructing a detection model based on the training set according to the following formula:
hθ(y)＝θ0+θ1y1+…+θmym＝θTx
wherein h isθ(y) is a predicted value of training data after training of the detection model, theta0、θ1、θmFor each weight coefficient of the training data, thetaT＝(θ0,θ1,…,θm)TIs the weight coefficient of each training data.
S1022, according to the following formula, obtaining corresponding model error characteristics according to the detection model:
wherein, Ji(theta) is the error characteristic of the ith training set through linear fitting of the detection model to the training set, hθ(y(i)) Predicted value y of the ith training set passing through the detection model(i)The true value of the ith training set.
S103, calculating the fitting degree of the detection model according to the model error characteristics.
Specifically, step S103 includes the following steps S1031 to S1032:
and S1031, calculating the fitting degree weight characteristic of the detection model according to the model error characteristic.
Further, step S1031 specifically includes:
calculating the fitting degree weight characteristic of the detection model according to the model error characteristic according to the following formula:
wherein, JsumIs the sum of the model error characteristics of all training sets, Ji(theta) is the model error characteristic of the ith training set,
is the fitness weight feature of the ith training set. k is the number of training sets.
S1032, acquiring corresponding fitting degree based on the fitting degree weight characteristics.
The fitting degree and the fitting degree weight characteristic have a corresponding relation, and the fitting degree weight characteristic is solved aiming at the same training set, so that the corresponding fitting degree of the same training set can be solved.
And S104, acquiring an abnormal threshold interval according to the fitting degree.
Specifically, the step S104 includes the following steps S1041 to S1042:
s1041, obtaining a predicted value of the training data based on the detection model, and calculating a confidence interval according to an error between the predicted value and an actual value of the training data.
S1042, obtaining an abnormal threshold interval according to the fitting degree and the confidence interval.
For the above step S1041, in a training set, an error between the predicted value and the actual value of the training data represents an error between the predicted value and the actual value of the training set.
The error between the predicted value and the actual value of the training set can be obtained according to the following formula:
εi＝hθ(y(i))-y(i)
where ε is the error of the training set, hθ(y(i)) Is the predicted value of the ith training set, y(i)Is the actual value of the ith training set.
Further, the error of the training set satisfies the gaussian distribution, and the gaussian model parameters are estimated by using the maximum likelihood theory, wherein the likelihood function is as follows:
the partial derivatives are calculated by taking the logarithm, and the partial derivatives are 0, so that:
the confidence interval after solving is:
wherein k is the number of training data of the training set, z is a threshold value of the confidence level, and s is the variance of the model error characteristics of the training set. In this embodiment, z is a threshold with a confidence level of 80%, μ is the mathematical expectation of the training data, σ2Is the variance of the training data.
For the step S1042, the abnormal threshold interval may be obtained according to the following interval expression:
where θ is the fitness of the training set and y is the training set. k is the number of training data of the training set, z is a threshold value of the confidence level, and s is the variance of the model error characteristics of the training set. In this embodiment, z is a threshold with a confidence level of 80%, μ is the mathematical expectation of the training data, σ2Is the variance of the training data.
In this embodiment, after calculating a corresponding abnormal threshold interval for each training set and performing cumulative calculation on the abnormal threshold intervals, an abnormal threshold interval corresponding to a training set with a fitting degree greater than 0.5 is selected to perform abnormal detection on the current data to be measured.
And S105, determining whether the current measured data is abnormal or not according to the abnormal threshold interval.
Specifically, step S105 includes the following steps S1051 to S1052.
S1051, judging whether the current measured data is in the abnormal threshold interval.
S1052, if the current measured data is not in the abnormal threshold interval, determining that the current measured data is abnormal.
In step S1052, it is determined that the current measured data is abnormal if the current measured data is outside the abnormal threshold interval.
In another embodiment of the present invention, as shown in fig. 3, on the basis of step S105, the abnormality detection method for sensing the monitoring data further includes the following steps S100 to S110.
And S100, if the current measured data is determined to be abnormal, calculating a data repair value corresponding to the current measured data based on the fitting degree.
S110, the current measured data is replaced by the data restoration value.
The step of calculating the data restoration value corresponding to the current measured data based on the degree of fitting in step S100 specifically includes:
calculating a data restoration value corresponding to the current measured data according to the following formula:
d＝θx
wherein d is a data restoration value, theta is a fitting degree, and x is the current measured data.
Therefore, if it is determined that the current data to be tested is abnormal based on step S105, the data restoration value corresponding to the current data to be tested can be obtained by selecting the maximum fitting degree through steps S100 to S110, and the data restoration value is replaced with the data restoration value, thereby ensuring that the data is restored to normal.
The anomaly detection method, the anomaly detection device, the computer equipment and the storage medium for the sensing monitoring data provided by the embodiment of the invention are oriented to the current detected data in the sensing monitoring data, corresponding training data are selected from the sensing monitoring data according to a preset rule to construct a training set, a detection model is constructed based on the training set, corresponding model error characteristics are obtained according to the detection model, the fitting degree of the detection model is calculated according to the model error characteristics, an anomaly threshold interval is obtained according to the fitting degree, whether the current detected data are abnormal or not is determined according to the anomaly threshold interval, the detection model is constructed according to the time sequence correlation of the sensing monitoring data, the detected abnormal data can be repaired, the applicability of the sensing data anomaly detection method is improved, and the industrial requirements can be met better.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
In an embodiment, an anomaly detection device for sensing monitoring data is provided, and the anomaly detection device for sensing monitoring data corresponds to the anomaly detection method for sensing monitoring data in the above embodiment one to one. As shown in fig. 4, the abnormality detection device for sensing monitoring data includes:
and a training set constructing module 41, configured to select, for a current measured data in the sensing monitoring data, a corresponding training data from the sensing monitoring data according to a preset rule to construct a training set.
And a model error feature obtaining module 42, configured to construct a detection model based on the training set, and obtain a corresponding model error feature according to the detection model.
And a fitting degree calculation module 43, configured to calculate a fitting degree of the detection model according to the model error feature.
And an abnormal threshold interval obtaining module 44, configured to obtain an abnormal threshold interval according to the fitting degree.
And a data anomaly determination module 45, configured to determine whether the current measured data is anomalous according to the anomaly threshold interval.
In this embodiment, the training set constructing module 41 includes the following units:
and the training data selection unit is used for selecting a plurality of training data from the sensing monitoring data according to a preset length window based on the current measured data.
And the similarity calculation unit is used for calculating the similarity between the training data and the current measured data respectively.
And the training set constructing unit is used for selecting corresponding training data to construct a training set based on the degree of similarity.
Further, the training set constructing unit comprises the following sub-units:
and the training set constructing subunit is used for constructing a plurality of training sets according to the selected training data, and the number of the training data included in each training set is different.
And the operator unit of the discrete degree characteristic value is used for calculating the discrete degree characteristic value of each training set and selecting the training set for constructing the detection model according to the discrete degree characteristic value.
In the present embodiment, the fitting degree calculation module 43 includes the following units:
and the fitting degree weight characteristic calculation unit is used for calculating the fitting degree weight characteristic of the detection model according to the model error characteristic.
And the fitting degree obtaining unit is used for obtaining corresponding fitting degree based on the fitting degree weight characteristics.
The abnormality threshold section acquisition module 44 includes the following units:
and the confidence interval calculation unit is used for acquiring a predicted value of the training data based on the detection model and calculating a confidence interval according to an error between the predicted value and an actual value of the training data.
And the abnormal threshold interval acquisition unit is used for acquiring an abnormal threshold interval according to the fitting degree and the confidence interval.
The module 45 for determining the abnormality of the measured data includes the following units:
and the abnormal judgment unit is used for judging whether the current measured data is within the abnormal threshold interval.
And the abnormality determining unit is used for determining that the current measured data is abnormal if the current measured data is not in the abnormal threshold interval.
In another embodiment, the anomaly detection device for sensing monitoring data further comprises the following modules:
and a data restoration value calculating module 46, configured to calculate a data restoration value corresponding to the current measured data based on the degree of fitting if it is determined that the current measured data is abnormal.
A data repair value replacing module 47, configured to replace the current measured data with the data repair value.
Wherein the meaning of "first" and "second" in the above modules/units is only to distinguish different modules/units, and is not used to define which module/unit has higher priority or other defining meaning. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to such process, method, article, or apparatus, and such that a division of modules presented in this application is merely a logical division and may be implemented in a practical application in a further manner.
For specific limitations of the anomaly detection device for the sensing monitoring data, reference may be made to the above limitations of the anomaly detection method for the sensing monitoring data, and details are not repeated here. The modules in the anomaly detection device for sensing monitoring data can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 5. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing data involved in the anomaly detection method for the sensing monitoring data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of anomaly detection for sensing monitored data.
In one embodiment, a computer device is provided, which includes a memory, a processor and a computer program stored on the memory and executable on the processor, and the processor executes the computer program to implement the steps of the abnormality detection method for sensing monitoring data in the above embodiments, such as the steps S101 to S105 shown in fig. 2 and other extensions of the method and related steps. Alternatively, the processor, when executing the computer program, implements the functions of the modules/units of the abnormality detection apparatus for sensing monitoring data in the above-described embodiment, for example, the functions of the modules 41 to 45 shown in fig. 4. To avoid repetition, further description is omitted here.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like which is the control center for the computer device and which connects the various parts of the overall computer device using various interfaces and lines.
The memory may be used to store the computer programs and/or modules, and the processor may implement various functions of the computer device by running or executing the computer programs and/or modules stored in the memory and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, video data, etc.) created according to the use of the cellular phone, etc.
The memory may be integrated in the processor or may be provided separately from the processor.
In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements the steps of the anomaly detection method for sensing monitoring data in the above-described embodiments, such as the steps S101 to S105 shown in fig. 2 and extensions of other extensions and related steps of the method. Alternatively, the computer program, when executed by the processor, implements the functions of the modules/units of the abnormality detection apparatus for sensing monitoring data in the above-described embodiment, for example, the functions of the modules 41 to 45 shown in fig. 4. To avoid repetition, further description is omitted here.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer readable storage medium, and can include the processes of the embodiments of the anomaly detection method for sensing monitoring data described above when executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (10)
1. An anomaly detection method for sensing monitoring data, comprising:
selecting corresponding training data from the sensing monitoring data according to a preset rule for the current measured data in the sensing monitoring data to construct a training set;
constructing a detection model based on the training set, and acquiring corresponding model error characteristics according to the detection model;
calculating the fitting degree of the detection model according to the model error characteristics;
obtaining an abnormal threshold interval according to the fitting degree;
and determining whether the current measured data is abnormal or not according to the abnormal threshold interval.
2. The anomaly detection method for the sensing monitoring data according to claim 1, wherein the step of selecting corresponding training data from the sensing monitoring data according to a preset rule to construct a training set for the current measured data in the sensing monitoring data comprises:
selecting a plurality of training data from the sensing monitoring data according to a preset length window based on the current measured data;
calculating the similarity between the training data and the current measured data;
and selecting corresponding training data to construct a training set based on the magnitude ordering of the similarity degrees.
3. The method of claim 2, wherein the step of selecting the corresponding training data to construct the training set based on the magnitude ordering of the similarity degrees comprises:
constructing a plurality of training sets according to the selected training data, wherein the number of the training data in each training set is different;
and calculating the characteristic value of the discrete degree of each training set, and selecting the training set for constructing the detection model according to the characteristic value of the discrete degree.
4. The anomaly detection method for sensing monitored data according to claim 1, characterized in that said step of calculating the degree of fit of said detection model from said model error signature comprises:
calculating the fitting degree weight characteristic of the detection model according to the model error characteristic;
and acquiring corresponding fitting degree based on the fitting degree weight characteristics.
5. The method of claim 1, wherein the step of obtaining an anomaly threshold interval according to the fitness comprises:
obtaining a predicted value of the training data based on the detection model, and calculating a confidence interval according to an error between the predicted value and an actual value of the training data;
and acquiring an abnormal threshold interval according to the fitting degree and the confidence interval.
6. The abnormality detection method for the sensing monitored data according to claim 1, characterized in that the step of determining whether the current data under test is abnormal or not according to the abnormality threshold interval comprises:
judging whether the current measured data is within the abnormal threshold interval or not;
and if the current measured data is not within the abnormal threshold interval, determining that the current measured data is abnormal.
7. The anomaly detection method for sensing surveillance data according to any one of claims 1-6, characterized in that the method further comprises:
if the current measured data is determined to be abnormal, calculating a data restoration value corresponding to the current measured data based on the fitting degree;
and replacing the current measured data with the data restoration value.
8. An anomaly detection device for sensing monitored data, comprising:
the training set construction module is used for selecting corresponding training data from the sensing monitoring data according to a preset rule to construct a training set for the current measured data in the sensing monitoring data;
the model error characteristic acquisition module is used for constructing a detection model based on the training set and acquiring corresponding model error characteristics according to the detection model;
the fitting degree calculation module is used for calculating the fitting degree of the detection model according to the model error characteristics;
an abnormal threshold interval obtaining module, configured to obtain an abnormal threshold interval according to the fitting degree;
and the abnormal data determining module is used for determining whether the current measured data is abnormal according to the abnormal threshold interval.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the anomaly detection method for sensing surveillance data according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method for detecting anomalies in sensor-monitored data according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011640085.1A CN112699946B (en) | 2020-12-31 | 2020-12-31 | Anomaly detection method and device for sensing monitoring data and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011640085.1A CN112699946B (en) | 2020-12-31 | 2020-12-31 | Anomaly detection method and device for sensing monitoring data and related equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112699946A true CN112699946A (en) | 2021-04-23 |
| CN112699946B CN112699946B (en) | 2022-06-21 |
Family
ID=75513992
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011640085.1A Active CN112699946B (en) | 2020-12-31 | 2020-12-31 | Anomaly detection method and device for sensing monitoring data and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112699946B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113093985A (en) * | 2021-06-09 | 2021-07-09 | 中国南方电网有限责任公司超高压输电公司广州局 | Sensor data link abnormity detection method and device and computer equipment |
| CN113218537A (en) * | 2021-05-25 | 2021-08-06 | 中国南方电网有限责任公司超高压输电公司广州局 | Training method, device, equipment and storage medium of temperature anomaly detection model |
| CN115388931A (en) * | 2022-10-27 | 2022-11-25 | 河北省科学院应用数学研究所 | Credible monitoring method, monitoring terminal and storage medium for sensor abnormal data |
| CN115565379A (en) * | 2022-12-06 | 2023-01-03 | 成都智元汇信息技术股份有限公司 | Method and system for replacing abnormal passenger flow data in historical passenger flow data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108089962A (en) * | 2017-11-13 | 2018-05-29 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality, device and electronic equipment |
| CN109740661A (en) * | 2018-12-28 | 2019-05-10 | 沈阳工业大学 | A kind of well fluid level adaptive forecasting method based on fuzzy evaluation |
| US20200117177A1 (en) * | 2018-10-15 | 2020-04-16 | Uptake Technologies, Inc. | Computer System and Method of Defining a Set of Anomaly Thresholds for an Anomaly Detection Model |
| CN111814821A (en) * | 2020-05-21 | 2020-10-23 | 北京迈格威科技有限公司 | Deep learning model establishing method, sample processing method and device |
-
2020
- 2020-12-31 CN CN202011640085.1A patent/CN112699946B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108089962A (en) * | 2017-11-13 | 2018-05-29 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality, device and electronic equipment |
| US20200117177A1 (en) * | 2018-10-15 | 2020-04-16 | Uptake Technologies, Inc. | Computer System and Method of Defining a Set of Anomaly Thresholds for an Anomaly Detection Model |
| CN109740661A (en) * | 2018-12-28 | 2019-05-10 | 沈阳工业大学 | A kind of well fluid level adaptive forecasting method based on fuzzy evaluation |
| CN111814821A (en) * | 2020-05-21 | 2020-10-23 | 北京迈格威科技有限公司 | Deep learning model establishing method, sample processing method and device |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113218537A (en) * | 2021-05-25 | 2021-08-06 | 中国南方电网有限责任公司超高压输电公司广州局 | Training method, device, equipment and storage medium of temperature anomaly detection model |
| CN113218537B (en) * | 2021-05-25 | 2024-04-05 | 中国南方电网有限责任公司超高压输电公司广州局 | Training method, training device, training equipment and training storage medium for temperature anomaly detection model |
| CN113093985A (en) * | 2021-06-09 | 2021-07-09 | 中国南方电网有限责任公司超高压输电公司广州局 | Sensor data link abnormity detection method and device and computer equipment |
| CN113093985B (en) * | 2021-06-09 | 2021-09-10 | 中国南方电网有限责任公司超高压输电公司广州局 | Sensor data link abnormity detection method and device and computer equipment |
| CN115388931A (en) * | 2022-10-27 | 2022-11-25 | 河北省科学院应用数学研究所 | Credible monitoring method, monitoring terminal and storage medium for sensor abnormal data |
| CN115388931B (en) * | 2022-10-27 | 2023-02-03 | 河北省科学院应用数学研究所 | Credible monitoring method, monitoring terminal and storage medium for sensor abnormal data |
| CN115565379A (en) * | 2022-12-06 | 2023-01-03 | 成都智元汇信息技术股份有限公司 | Method and system for replacing abnormal passenger flow data in historical passenger flow data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112699946B (en) | 2022-06-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112699946B (en) | Anomaly detection method and device for sensing monitoring data and related equipment | |
| US11120127B2 (en) | Reconstruction-based anomaly detection | |
| CN108959934B (en) | Security risk assessment method, security risk assessment device, computer equipment and storage medium | |
| CN109587001B (en) | Performance index abnormality detection method and device | |
| CN110912867A (en) | Intrusion detection method, device, equipment and storage medium for industrial control system | |
| CN112231174A (en) | Abnormity warning method, device, equipment and storage medium | |
| CN113518011A (en) | Abnormality detection method and apparatus, electronic device, and computer-readable storage medium | |
| CN112257755A (en) | Method and device for analyzing operating state of spacecraft | |
| CN111325159A (en) | Fault diagnosis method and device, computer equipment and storage medium | |
| US20210320931A1 (en) | Dynamic monitoring and securing of factory processes, equipment and automated systems | |
| CN114547145B (en) | Time sequence data anomaly detection method, system, storage medium and equipment | |
| CN112884199A (en) | Method and device for predicting faults of hydropower station equipment, computer equipment and storage medium | |
| CN112444748A (en) | Battery abnormality detection method, battery abnormality detection device, electronic apparatus, and storage medium | |
| CN110672326B (en) | Bearing fault detection method and computer readable storage medium | |
| JP2020071845A (en) | Abnormality detection device, abnormality detection method, and abnormality detection program | |
| CN110866682B (en) | Underground cable early warning method and device based on historical data | |
| CN114553681A (en) | Equipment state abnormity detection method and device and computer equipment | |
| US11592808B2 (en) | Methods, systems, articles of manufacture and apparatus to improve boundary excursion detection | |
| CN115659826A (en) | Server failure rate detection method and device, electronic equipment and storage medium | |
| US20230384770A1 (en) | Calculation device and method for extracting feature of measurement data by using same | |
| US11544620B2 (en) | System and method for context-based training of a machine learning model | |
| CN113377564A (en) | Fault diagnosis method and device, computer equipment and storage medium | |
| CN112162993A (en) | Data updating method and device of blacklist and computer equipment | |
| CN117289685B (en) | Production line fault prediction and self-healing method and system based on artificial intelligence | |
| JP7389518B2 (en) | Protecting industrial production from advanced attacks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |