CN109034180B - Abnormality detection method, abnormality detection device, computer-readable storage medium, and electronic apparatus - Google Patents

Abnormality detection method, abnormality detection device, computer-readable storage medium, and electronic apparatus Download PDF

Info

Publication number
CN109034180B
CN109034180B CN201810552461.8A CN201810552461A CN109034180B CN 109034180 B CN109034180 B CN 109034180B CN 201810552461 A CN201810552461 A CN 201810552461A CN 109034180 B CN109034180 B CN 109034180B
Authority
CN
China
Prior art keywords
period
detected
abnormal
vector corresponding
bucket
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810552461.8A
Other languages
Chinese (zh)
Other versions
CN109034180A (en
Inventor
石子凡
纪勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Neusoft Corp
Original Assignee
Neusoft Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Neusoft Corp filed Critical Neusoft Corp
Priority to CN201810552461.8A priority Critical patent/CN109034180B/en
Publication of CN109034180A publication Critical patent/CN109034180A/en
Application granted granted Critical
Publication of CN109034180B publication Critical patent/CN109034180B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures

Abstract

The present disclosure relates to an abnormality detection method, apparatus, computer-readable storage medium, and electronic device, the method comprising: sampling data in a period to be detected to obtain target sampling point data; performing barrel dividing processing on the target sampling point data according to a preset barrel dividing number, and determining a barrel dividing vector corresponding to a period to be detected; determining a target parameter between a period to be detected and an abnormal period according to a bucket dividing vector corresponding to the period to be detected and a bucket dividing vector corresponding to a known abnormal period, wherein the target parameter is used for representing the similarity between the period to be detected and the abnormal period, the bucket dividing vector corresponding to the abnormal period is determined by carrying out bucket dividing processing on sampling point data in the abnormal period according to a preset bucket dividing number, and the time length of the period to be detected is the same as that of the abnormal period; and obtaining an abnormal detection result aiming at the period to be detected according to the target parameters. Therefore, the efficiency and accuracy of anomaly detection can be effectively improved, and the user experience is improved.

Description

Abnormality detection method, abnormality detection device, computer-readable storage medium, and electronic apparatus
Technical Field
The present disclosure relates to the field of anomaly detection, and in particular, to an anomaly detection method and apparatus, a computer-readable storage medium, and an electronic device.
Background
The development of information technology enables electronic information management to enter more and more industries, however, in the process of electronic information management, abnormal conditions are easy to occur, and inconvenience is brought to users. Therefore, it is important to find the abnormal situation in time. In the prior art, the following method is generally adopted for abnormality detection:
1. based on the way the threshold is defined manually. For example, by setting an abnormality threshold value, when the data of the index is higher than the abnormality threshold value, the data of the index is determined to be abnormal. However, in this method, the accuracy and robustness of the threshold setting are low.
2. Based on outlier detection. In this approach, when outlier features are not significant, the efficiency and accuracy of anomaly detection is low.
3. The method is an unsupervised mode based on a time series prediction model, namely, a model is established through historical data, and abnormality detection is carried out in a prediction mode. However, in this method, the data to be detected cannot be detected using a known abnormality.
Disclosure of Invention
In order to solve the above problems, the present disclosure provides an abnormality detection method, apparatus, computer-readable storage medium, and electronic device.
In order to achieve the above object, according to a first aspect of the present disclosure, there is provided an abnormality detection method including:
sampling data in a period to be detected to obtain target sampling point data;
performing barrel dividing processing on the target sampling point data according to a preset barrel dividing number, and determining a barrel dividing vector corresponding to the period to be detected;
determining a target parameter between the period to be detected and the abnormal period according to the bucket dividing vector corresponding to the period to be detected and the bucket dividing vector corresponding to the known abnormal period, wherein the target parameter is used for representing the similarity between the period to be detected and the abnormal period, the bucket dividing vector corresponding to the abnormal period is determined by carrying out bucket dividing processing on sampling point data in the abnormal period according to the preset bucket dividing number, and the time length of the period to be detected is the same as that of the abnormal period;
and obtaining an abnormal detection result aiming at the period to be detected according to the target parameters.
Optionally, the determining, according to the sub-bucket vector corresponding to the period to be detected and the sub-bucket vector corresponding to the known abnormal period, a target parameter between the period to be detected and the abnormal period includes:
mapping the bucket-divided vectors corresponding to the period to be detected into binary vectors according to a preset mapping rule so as to obtain the characteristic vectors corresponding to the period to be detected;
and determining a target parameter between the period to be detected and the abnormal period according to the feature vector corresponding to the period to be detected and the feature vector corresponding to the abnormal period, wherein the feature vector corresponding to the abnormal period is obtained by mapping the sub-bucket vector corresponding to the abnormal period into a binary vector according to the preset mapping rule.
Optionally, the preset mapping rule is an ITQ algorithm.
Optionally, the determining, according to the feature vector corresponding to the period to be detected and the feature vector corresponding to the abnormal period, a target parameter between the period to be detected and the abnormal period includes:
determining the Hamming distance between the characteristic vector corresponding to the period to be detected and the characteristic vector corresponding to the abnormal period;
and mapping the Hamming distance to a preset numerical value interval, and determining a numerical value obtained by mapping as the target parameter, wherein the smaller the target parameter is, the more similar the period to be detected and the abnormal period is.
Optionally, the obtaining, according to the target parameter, an abnormality detection result for the period to be detected includes at least one of:
when the target parameter is determined that the period to be detected is similar to the abnormal period, determining that the period to be detected is abnormal;
when the target parameter is determined that the period to be detected is similar to the abnormal period, the abnormal event corresponding to the abnormal period is included in the abnormal detection result of the period to be detected;
and when the target parameter is determined to be similar to the abnormal period, the target parameter is included in the abnormal detection result of the period to be detected.
Optionally, the method further comprises:
and outputting an abnormal solution corresponding to the abnormal period when the period to be detected is determined to be similar to the abnormal period according to the target parameters.
According to a second aspect of the present disclosure, there is provided an abnormality detection apparatus, the apparatus including:
the sampling module is used for sampling data in a period to be detected to obtain target sampling point data;
the first processing module is used for carrying out barrel dividing processing on the target sampling point data according to a preset barrel dividing number and determining a barrel dividing vector corresponding to the period to be detected;
a determining module, configured to determine a target parameter between the period to be detected and the abnormal period according to the bucket dividing vector corresponding to the period to be detected and the bucket dividing vector corresponding to the known abnormal period, where the target parameter is used to represent a similarity between the period to be detected and the abnormal period, the bucket dividing vector corresponding to the abnormal period is determined by performing bucket dividing processing on sampling point data in the abnormal period according to the preset number of buckets, and the time length of the period to be detected is the same as that of the abnormal period;
and the second processing module is used for obtaining an abnormal detection result aiming at the period to be detected according to the target parameter.
Optionally, the determining module includes:
the mapping submodule is used for mapping the sub-bucket vector corresponding to the period to be detected into a binary vector according to a preset mapping rule so as to obtain a characteristic vector corresponding to the period to be detected;
and the first determining submodule is used for determining a target parameter between the period to be detected and the abnormal period according to the feature vector corresponding to the period to be detected and the feature vector corresponding to the abnormal period, wherein the feature vector corresponding to the abnormal period is obtained by mapping the sub-bucket vector corresponding to the abnormal period into a binary vector according to the preset mapping rule.
Optionally, the preset mapping rule is an ITQ algorithm.
Optionally, the first determining sub-module includes:
the second determining submodule is used for determining the Hamming distance between the characteristic vector corresponding to the period to be detected and the characteristic vector corresponding to the abnormal period;
and the third determining submodule is used for mapping the Hamming distance to a preset numerical value interval and determining a numerical value obtained by mapping as the target parameter, wherein the smaller the target parameter is, the more similar the period to be detected and the abnormal period is.
Optionally, the second processing module comprises at least one of:
the fourth determining submodule is used for determining that the period to be detected is abnormal when the period to be detected is determined to be similar to the abnormal period according to the target parameters;
a fifth determining submodule, configured to, when it is determined that the period to be detected is similar to the abnormal period according to the target parameter, include an abnormal event corresponding to the abnormal period in an abnormal detection result of the period to be detected;
and the sixth determining submodule is used for including the target parameter in the abnormal detection result of the period to be detected when the period to be detected is determined to be similar to the abnormal period according to the target parameter.
Optionally, the apparatus further comprises:
and the output module is used for outputting an abnormal solution corresponding to the abnormal period when the target parameter is determined that the period to be detected is similar to the abnormal period.
According to a third aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of any of the methods of the first aspect described above.
According to a fourth aspect of the present disclosure, there is provided an electronic device comprising:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to implement the steps of the method of any of the first aspects above.
In the technical scheme, the data in the period to be detected is sampled and subjected to barrel separation processing, so that the data volume in the abnormal detection process can be effectively reduced. Meanwhile, according to the bucket dividing vector corresponding to the period to be detected and the bucket dividing vector corresponding to the known abnormal period, the abnormal detection result of the period to be detected is determined, on one hand, the abnormal detection can be carried out on the period to be detected by utilizing the data characteristics corresponding to the known abnormal period, the efficiency and the accuracy of the abnormal detection are effectively improved, and the influence on the abnormal detection result caused by the fact that an abnormal threshold value deviation or an outlier is set is not obvious is avoided. On the other hand, the data in the period to be detected and the abnormal period are compared to determine the abnormal detection result of the period to be detected, and the abnormal detection can be performed by taking the period as a unit, so that the obvious data characteristics can be ensured, the influence of different data quantity on the detection result can be avoided, the accuracy of the abnormal detection is further ensured, and the user experience is improved.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:
FIG. 1 is a flow chart of an anomaly detection method provided in accordance with one embodiment of the present disclosure;
FIG. 2 is a flow diagram of an exemplary implementation of determining a target parameter between a period to be detected and an abnormal period based on a bucket vector corresponding to the period to be detected and a bucket vector corresponding to a known abnormal period;
FIG. 3 is a flow diagram of an exemplary implementation of determining a target parameter between a cycle to be detected and an abnormal cycle based on a feature vector corresponding to the cycle to be detected and a feature vector corresponding to the abnormal cycle;
FIG. 4 is a block diagram of an anomaly detection apparatus provided in accordance with one embodiment of the present disclosure;
FIG. 5 is a block diagram of a determination module of an anomaly detection apparatus provided in accordance with another embodiment of the present disclosure;
FIG. 6 is a block diagram of a first determination submodule of an anomaly detection apparatus provided in accordance with another embodiment of the present disclosure;
FIG. 7 is a block diagram illustrating an electronic device in accordance with an exemplary embodiment;
FIG. 8 is a block diagram illustrating an electronic device in accordance with an example embodiment.
Detailed Description
The following detailed description of specific embodiments of the present disclosure is provided in connection with the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present disclosure, are given by way of illustration and explanation only, not limitation.
Fig. 1 is a flowchart illustrating an abnormality detection method according to an embodiment of the present disclosure. As shown in fig. 1, the method includes:
in S11, data in the period to be detected is sampled, and target sampling point data is obtained.
For example, the period to be detected may be 1 hour, and the preset time interval is 1 minute, and then 60 target sampling point data may be obtained in the period to be detected.
In S12, the target sample point data is subjected to sub-bucket processing according to a preset sub-bucket number, and a sub-bucket vector corresponding to the period to be detected is determined.
For example, the preset number of buckets is 12, and an embodiment of performing the bucket dividing processing on the target sampling point data according to the preset number of buckets is as follows:
and grouping the target sampling point data according to a group of 5, wherein the 5 target sampling point data are continuous sampling point data. One for each sub-bucket, illustratively target sample point data a0、a1、a2、a3And a4Is divided into sub-barrels A0By way of example, a may be0、a1、a2、a3And a4Average value of (2) as bucket A0The values of (a) are obtained in the same way for the other sub-buckets. Therefore, the target sampling point data can be converted into 12 sub-buckets in the above mode, and the 12 sub-buckets form vectors according to the time sequenceAnd determining the sub-bucket vector corresponding to the period to be detected.
In S13, determining a target parameter between the period to be detected and the abnormal period according to the bucket dividing vector corresponding to the period to be detected and the known bucket dividing vector corresponding to the abnormal period, where the target parameter is used to represent the similarity between the period to be detected and the abnormal period, the bucket dividing vector corresponding to the abnormal period is determined by performing bucket dividing processing on the sampling point data in the abnormal period according to the preset number of buckets, and the period of the period to be detected and the time of the abnormal period are the same.
Wherein the abnormal period may be determined by:
(1) and sampling data in a historical period to obtain historical sampling point data, wherein the historical period has the same duration as the period to be detected.
(2) And carrying out barrel dividing processing on the historical sampling point data according to a preset barrel dividing number so as to obtain a barrel dividing vector corresponding to the historical period.
(3) And respectively determining whether each sub-bucket data in the sub-bucket vector corresponding to the historical period is abnormal.
Wherein, whether each sub-bucket data is abnormal or not can be determined in a manual marking mode. For example, a technician may perform exception labeling on the data of the bucket, and for example, may label 1 on the abnormal data of the bucket and label 0 on the non-abnormal data of the bucket.
For another example, whether the data in the sub-bucket is abnormal may also be determined according to the number of log errors in the time period corresponding to the data in the sub-bucket. Illustratively, the history period is 1 hour, and in the history period, 60 pieces of history sampling point data can be obtained by collecting one piece of history sampling point data every 1 minute. If the preset number of buckets is 12, the time period corresponding to the first data of the buckets is 0-5 minutes, and therefore, whether the data of the buckets are abnormal can be determined by determining the number of log errors recorded in the previous 5 minutes. For example, a log error number threshold may be preset, and when the log error number in the time period corresponding to the sub-bucket data exceeds the log error number threshold, it is determined that the sub-bucket data is abnormal. In addition, the number of access errors in the period corresponding to the sub-bucket data or a combination of the number of access errors and the number of access errors in the period may be considered comprehensively, and will not be described herein again. The time period corresponding to the second sub-bucket data is 5-10 minutes, the time periods corresponding to other sub-buckets are analogized in sequence, and the sub-bucket data can be used for determining whether the sub-bucket data is abnormal or not in the above manner, which is not described herein again.
(4) And when each sub-bucket data in the sub-bucket vector corresponding to the history period meets an abnormal condition, determining the history period as an abnormal period.
In an embodiment, the exception condition is that the number of abnormal bucket data in the bucket vector corresponding to the history period exceeds a first threshold. For example, if the first threshold is 8, the history period may be determined as an abnormal period when the number of abnormal bucket data in the bucket vector corresponding to the history period is 9.
In another embodiment, the exception condition is that the number of consecutive exceptions in the bucket vector corresponding to the history period exceeds a second threshold. For example, if the first threshold is 4, when there are 5 consecutive abnormal sub-bucket data in the sub-bucket vector corresponding to the history period, the history period may be determined as the abnormal period.
Optionally, in the above manner, a plurality of abnormal periods may be determined. And the bucket dividing vector corresponding to the abnormal period and the bucket dividing vector corresponding to the period to be detected are subjected to bucket dividing treatment in the same mode, so that the consistency of data bucket dividing can be ensured. In addition, when the known abnormal period is stored, historical sampling point data in the abnormal period can be stored, alternatively or additionally, a sub-bucket vector corresponding to the abnormal period can also be stored, so that repeated sub-bucket processing steps on the abnormal period are avoided, and the processing efficiency of abnormal detection is improved.
Therefore, in an embodiment, when determining the target parameter between the period to be detected and the abnormal period according to the sub-bucket vector corresponding to the period to be detected and the sub-bucket vector corresponding to the known abnormal period, the sub-bucket vector corresponding to the period to be detected and the sub-bucket vector corresponding to the known abnormal period may be directly calculated, for example, the distance between the sub-bucket vector corresponding to the period to be detected and the sub-bucket vector corresponding to the known abnormal period, such as the euclidean distance, may be directly determined according to the sub-bucket vector corresponding to the period to be detected and the sub-bucket vector corresponding to the known abnormal period, and the distance may be determined as the target parameter between the period to be detected and the abnormal period.
Optionally, an exemplary implementation manner of determining the target parameter between the period to be detected and the abnormal period according to the sub-bucket vector corresponding to the period to be detected and the sub-bucket vector corresponding to the known abnormal period is as follows, as shown in fig. 2, and includes:
in S21, mapping the sub-bucket vector corresponding to the period to be detected into a binary vector according to a preset mapping rule, so as to obtain a feature vector corresponding to the period to be detected.
Optionally, the preset mapping rule is an ITQ (Iterative Quantization) algorithm. The length of the mapped binary vector may be determined according to the length of the bucket vector, for example, when the length of the bucket vector is long, the length of the mapped binary vector may be determined to be 64 bits for convenience of calculation, and when the length of the bucket vector is short, the length of the mapped binary vector may be determined to be 32 bits. The split-bucket vector can be mapped into a binary vector through an ITQ algorithm, corresponding quantization errors in the mapping process are effectively reduced, the consistency of the split-bucket vector and the feature vector converted into the binary vector is effectively guaranteed, the accuracy of the mapped binary vector is improved, the accuracy of the subsequently determined target parameters is guaranteed, and the accuracy of abnormal detection is effectively guaranteed.
In S22, determining a target parameter between the period to be detected and the abnormal period according to the feature vector corresponding to the period to be detected and the feature vector corresponding to the abnormal period, where the feature vector corresponding to the abnormal period is obtained by mapping the sub-bucket vector corresponding to the abnormal period into a binary vector according to the preset mapping rule, so as to ensure consistency between the feature vectors and avoid the influence of the converted feature vectors on the abnormal detection result.
When the known abnormal period is stored, the feature vector corresponding to the abnormal period can be directly stored, so that when the target parameter between the period to be detected and the abnormal period is determined according to the feature vector corresponding to the period to be detected and the feature vector corresponding to the abnormal period, the feature vector corresponding to the abnormal period can be directly obtained, and the efficiency of abnormal detection is improved.
Optionally, an exemplary implementation manner of determining the target parameter between the period to be detected and the abnormal period according to the feature vector corresponding to the period to be detected and the feature vector corresponding to the abnormal period is as follows, as shown in fig. 3, and includes:
in S31, a hamming distance between the feature vector corresponding to the cycle to be detected and the feature vector corresponding to the abnormal cycle is determined. The determination of the hamming distance between two vectors is prior art and is not described herein again.
In S32, the hamming distance is mapped to a preset value interval, and the mapped value is determined as a target parameter, where a smaller target parameter indicates a more similar period to be detected and the abnormal period.
For example, the preset value interval may be [0,1], and thus, mapping the hamming distance to the preset value interval may be normalizing the hamming distance, so that it may be mapped to the range of [0,1 ]. Therefore, the value obtained by normalizing the hamming distance can be determined as the target parameter. The closer the target parameter is to 0, the more similar the period to be detected and the abnormal period are, that is, the greater the similarity between the period to be detected and the abnormal period is represented, the closer the target parameter is to 1, the less the similarity between the period to be detected and the abnormal period is represented.
Therefore, in the technical scheme, the process of calculating the Hamming distance is simple, the target parameter is determined by determining the Hamming distance between the characteristic vector corresponding to the period to be detected and the characteristic vector corresponding to the abnormal period, and the complexity of determining the target parameter can be effectively reduced, so that the data processing efficiency and the calculation efficiency can be improved, and the efficiency of abnormal detection can be improved.
In another embodiment, the euclidean distance between the feature vector corresponding to the period to be detected and the feature vector corresponding to the abnormal period may be determined, and the euclidean distance may be mapped to a preset numerical range, for example, [0,1 ]. Therefore, the value obtained by mapping the euclidean distance can be determined as the target parameter. The closer the target parameter is to 0, the more similar the period to be detected and the abnormal period are, that is, the greater the similarity between the period to be detected and the abnormal period is represented, the closer the target parameter is to 1, the less the similarity between the period to be detected and the abnormal period is represented.
In another embodiment, the cosine similarity between the feature vector corresponding to the period to be detected and the feature vector corresponding to the abnormal period may also be directly determined, and the cosine similarity is determined as the target parameter. The value range of the cosine similarity is [ -1,1], and the closer the cosine similarity is to 1, the more similar the period to be detected and the abnormal period is, that is, the greater the similarity between the period to be detected and the abnormal period is represented, the closer the cosine similarity is to-1, and the smaller the similarity between the period to be detected and the abnormal period is represented.
It should be noted that the foregoing is only an exemplary implementation manner of determining the target parameter between the period to be detected and the abnormal period, and the disclosure is not limited thereto. Other numerical values capable of representing the similarity between the period to be detected and the abnormal period can also be used as target parameters in the disclosure for calculation, and are not described herein again.
In the technical scheme, the bucket vectors are converted into the binary characteristic vectors, so that the complexity of data can be reduced, and then when the calculation is carried out according to the characteristic vectors of the period to be detected and the abnormal period, the calculation amount of the data can be effectively reduced, and meanwhile, the calculation efficiency of the data can be improved, so that the efficiency and accuracy of abnormal detection are improved, and the use experience of a user is improved.
Turning back to fig. 1, in S14, an abnormality detection result for the cycle to be detected is obtained based on the target parameter.
Optionally, the obtaining, according to the target parameter, an abnormality detection result for the period to be detected includes at least one of:
when the target parameter is determined that the period to be detected is similar to the abnormal period, determining that the period to be detected is abnormal;
when the target parameter is determined that the period to be detected is similar to the abnormal period, the abnormal event corresponding to the abnormal period is included in the abnormal detection result of the period to be detected;
and when the target parameter is determined to be similar to the abnormal period, the target parameter is included in the abnormal detection result of the period to be detected.
Illustratively, a parameter threshold may be set for the target parameter. For example, when the target parameter is a value obtained by mapping a hamming distance between the feature vector corresponding to the period to be detected and the feature vector corresponding to the abnormal period to a preset value interval [0,1], the parameter threshold may be 0.3. When the target parameter is less than 0.3, it may be determined that the period to be detected is similar to the abnormal period, and it is determined that the period to be detected is abnormal. For another example, the target parameter is cosine similarity between the period to be detected and the abnormal period, at this time, the target parameter may be set to 0.8, and when the target parameter is greater than 0.8, it may be determined that the period to be detected is similar to the abnormal period, and it is determined that the period to be detected is abnormal.
Hereinafter, a numerical value obtained by mapping a hamming distance between a feature vector corresponding to a period to be detected and a feature vector corresponding to an abnormal period to a preset numerical value interval [0,1], and a parameter threshold value of 0.3 will be described as an example.
In an embodiment, when a known exception period is stored, an exception event corresponding to the exception period may be stored in association. Therefore, when the target parameter is less than 0.3, it may be determined that the period to be detected is similar to the abnormal period, and at this time, the abnormal event corresponding to the abnormal period may be included in the abnormal detection result of the period to be detected. Meanwhile, the abnormality detection result can be output to prompt the user.
In another embodiment, when the target parameter is less than 0.3, it may be determined that the period to be detected is similar to the abnormal period, and at this time, the target parameter may be included in the abnormal detection result of the period to be detected. Meanwhile, the abnormality detection result can be output to prompt the user.
The anomaly detection result of the period to be detected may include one or more of the above embodiments, which is not described herein again.
In summary, in the above technical solution, by sampling and barrel-dividing the data in the period to be detected, the data amount in the anomaly detection process can be effectively reduced. Meanwhile, according to the bucket dividing vector corresponding to the period to be detected and the bucket dividing vector corresponding to the known abnormal period, the abnormal detection result of the period to be detected is determined, on one hand, the abnormal detection can be carried out on the period to be detected by utilizing the data characteristics corresponding to the known abnormal period, the efficiency and the accuracy of the abnormal detection are effectively improved, and the influence on the abnormal detection result caused by the fact that an abnormal threshold value deviation or an outlier is set is not obvious is avoided. On the other hand, the data in the period to be detected and the abnormal period are compared to determine the abnormal detection result of the period to be detected, and the abnormal detection can be performed by taking the period as a unit, so that the obvious data characteristics can be ensured, the influence of different data quantity on the detection result can be avoided, the accuracy of the abnormal detection is further ensured, and the user experience is improved.
Optionally, the method further comprises:
and outputting an abnormal solution corresponding to the abnormal period when the period to be detected is determined to be similar to the abnormal period according to the target parameters.
For example, when determining the exception event corresponding to each exception period, the exception solution for the exception event may be associated with the exception period. Therefore, when the target parameter is determined that the period to be detected is similar to the abnormal period, the probability that the abnormal event corresponding to the abnormal period occurs in the period to be detected is high, and the abnormal solution corresponding to the abnormal period can be output at the moment, so that a user can conveniently make a solution aiming at the abnormal condition in time, the smooth operation of work is ensured, and the use experience of the user is further improved.
The present disclosure also provides an anomaly detection device. Fig. 4 is a block diagram illustrating an abnormality detection apparatus according to an embodiment of the present disclosure. As shown in fig. 4, the apparatus 10 includes:
the sampling module 100 is configured to sample data in a period to be detected to obtain target sampling point data;
the first processing module 200 is configured to perform barrel dividing processing on the target sampling point data according to a preset barrel dividing number, and determine a barrel dividing vector corresponding to the period to be detected;
a determining module 300, configured to determine a target parameter between the period to be detected and the abnormal period according to the bucket dividing vector corresponding to the period to be detected and the bucket dividing vector corresponding to the known abnormal period, where the target parameter is used to represent a similarity between the period to be detected and the abnormal period, the bucket dividing vector corresponding to the abnormal period is determined by performing bucket dividing processing on sampling point data in the abnormal period according to the preset number of buckets, and the time length of the period to be detected is the same as that of the abnormal period;
the second processing module 400 is configured to obtain an anomaly detection result for the period to be detected according to the target parameter.
Optionally, as shown in fig. 5, the determining module 300 includes:
the mapping submodule 301 is configured to map the bucket-divided vector corresponding to the period to be detected into a binary vector according to a preset mapping rule, so as to obtain a feature vector corresponding to the period to be detected;
the first determining submodule 302 is configured to determine a target parameter between the period to be detected and the abnormal period according to the feature vector corresponding to the period to be detected and the feature vector corresponding to the abnormal period, where the feature vector corresponding to the abnormal period is obtained by mapping the sub-bucket vector corresponding to the abnormal period into a binary vector according to the preset mapping rule.
Optionally, the preset mapping rule is an ITQ algorithm.
Optionally, as shown in fig. 6, the first determining sub-module 302 includes:
a second determining submodule 3021, configured to determine a hamming distance between the feature vector corresponding to the period to be detected and the feature vector corresponding to the abnormal period;
a third determining submodule 3022, configured to map the hamming distance to a preset value interval, and determine a value obtained through mapping as the target parameter, where a smaller target parameter indicates that the period to be detected is more similar to the abnormal period.
Optionally, the second processing module 400 comprises at least one of:
the fourth determining submodule is used for determining that the period to be detected is abnormal when the period to be detected is determined to be similar to the abnormal period according to the target parameters;
a fifth determining submodule, configured to, when it is determined that the period to be detected is similar to the abnormal period according to the target parameter, include an abnormal event corresponding to the abnormal period in an abnormal detection result of the period to be detected;
and the sixth determining submodule is used for including the target parameter in the abnormal detection result of the period to be detected when the period to be detected is determined to be similar to the abnormal period according to the target parameter.
Optionally, the apparatus 10 further comprises:
and the output module is used for outputting an abnormal solution corresponding to the abnormal period when the target parameter is determined that the period to be detected is similar to the abnormal period.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Fig. 7 is a block diagram illustrating an electronic device 700 in accordance with an example embodiment. As shown in fig. 7, the electronic device 700 may include: a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input/output (I/O) interface 704, and a communication component 705.
The processor 701 is configured to control the overall operation of the electronic device 700, so as to complete all or part of the steps in the above-mentioned abnormality detection method. The memory 702 is used to store various types of data to support operation at the electronic device 700, such as instructions for any application or method operating on the electronic device 700 and application-related data, such as contact data, transmitted and received messages, pictures, audio, video, and the like. The Memory 702 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia components 703 may include screen and audio components. Wherein the screen may be, for example, a touch screen and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may further be stored in the memory 702 or transmitted through the communication component 705. The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 704 provides an interface between the processor 701 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of one or more of them, so that the corresponding communication component 705 may include: Wi-Fi module, bluetooth module, NFC module.
In an exemplary embodiment, the electronic Device 700 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described abnormality detection method.
In another exemplary embodiment, a computer readable storage medium comprising program instructions which, when executed by a processor, implement the steps of the above-described anomaly detection method is also provided. For example, the computer readable storage medium may be the memory 702 described above including program instructions that are executable by the processor 701 of the electronic device 700 to perform the anomaly detection method described above.
Fig. 8 is a block diagram illustrating an electronic device 1900 in accordance with an example embodiment. For example, the electronic device 1900 may be provided as a server. Referring to fig. 8, an electronic device 1900 includes a processor 1922, which may be one or more in number, and a memory 1932 for storing computer programs executable by the processor 1922. The computer program stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processor 1922 may be configured to execute the computer program to perform the above-described abnormality detection method.
Additionally, electronic device 1900 may also include a power component 1926 and a communication component 1950, the power component 1926 may be configured to perform power management of the electronic device 1900, and the communication component 1950 may be configured to enable communication, e.g., wired or wireless communication, of the electronic device 1900. In addition, the electronic device 1900 may also include input/output (I/O) interfaces 1958. The electronic device 1900 may operate based on an operating system, such as Windows Server, Mac OS XTM, UnixTM, Linux, etc., stored in memory 1932.
In another exemplary embodiment, a computer readable storage medium comprising program instructions which, when executed by a processor, implement the steps of the above-described anomaly detection method is also provided. For example, the computer readable storage medium may be the memory 1932 described above that includes program instructions that are executable by the processor 1922 of the electronic device 1900 to perform the anomaly detection method described above.
The preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.
It should be noted that the various features described in the above embodiments may be combined in any suitable manner without departing from the scope of the invention. In order to avoid unnecessary repetition, various possible combinations will not be separately described in this disclosure.
In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure, as long as it does not depart from the spirit of the present disclosure.

Claims (8)

1. An anomaly detection method, characterized in that it comprises:
sampling data in a period to be detected to obtain target sampling point data;
performing barrel dividing processing on the target sampling point data according to a preset barrel dividing number, and determining a barrel dividing vector corresponding to the period to be detected;
determining a target parameter between the period to be detected and the abnormal period according to the bucket dividing vector corresponding to the period to be detected and the bucket dividing vector corresponding to the known abnormal period, wherein the target parameter is used for representing the similarity between the period to be detected and the abnormal period, the bucket dividing vector corresponding to the abnormal period is determined by carrying out bucket dividing processing on sampling point data in the abnormal period according to the preset bucket dividing number, and the time length of the period to be detected is the same as that of the abnormal period;
obtaining an abnormal detection result aiming at the period to be detected according to the target parameter;
determining a target parameter between the period to be detected and the abnormal period according to the bucket dividing vector corresponding to the period to be detected and the bucket dividing vector corresponding to the known abnormal period, wherein the determining comprises the following steps:
mapping the bucket-divided vectors corresponding to the period to be detected into binary vectors according to a preset mapping rule so as to obtain the characteristic vectors corresponding to the period to be detected;
and determining a target parameter between the period to be detected and the abnormal period according to the feature vector corresponding to the period to be detected and the feature vector corresponding to the abnormal period, wherein the feature vector corresponding to the abnormal period is obtained by mapping the sub-bucket vector corresponding to the abnormal period into a binary vector according to the preset mapping rule.
2. The method according to claim 1, wherein the preset mapping rule is an ITQ algorithm.
3. The method according to claim 1, wherein the determining the target parameter between the period to be detected and the abnormal period according to the eigenvector corresponding to the period to be detected and the eigenvector corresponding to the abnormal period comprises:
determining the Hamming distance between the characteristic vector corresponding to the period to be detected and the characteristic vector corresponding to the abnormal period;
and mapping the Hamming distance to a preset numerical value interval, and determining a numerical value obtained by mapping as the target parameter, wherein the smaller the target parameter is, the more similar the period to be detected and the abnormal period is.
4. The method according to claim 1, wherein the obtaining of the anomaly detection result for the period to be detected according to the target parameter includes at least one of:
when the target parameter is determined that the period to be detected is similar to the abnormal period, determining that the period to be detected is abnormal;
when the target parameter is determined that the period to be detected is similar to the abnormal period, the abnormal event corresponding to the abnormal period is included in the abnormal detection result of the period to be detected;
and when the target parameter is determined to be similar to the abnormal period, the target parameter is included in the abnormal detection result of the period to be detected.
5. The method according to any one of claims 1-4, further comprising:
and outputting an abnormal solution corresponding to the abnormal period when the period to be detected is determined to be similar to the abnormal period according to the target parameters.
6. An abnormality detection apparatus, characterized in that the apparatus comprises:
the sampling module is used for sampling data in a period to be detected to obtain target sampling point data;
the first processing module is used for carrying out barrel dividing processing on the target sampling point data according to a preset barrel dividing number and determining a barrel dividing vector corresponding to the period to be detected;
a determining module, configured to determine a target parameter between the period to be detected and the abnormal period according to the bucket dividing vector corresponding to the period to be detected and the bucket dividing vector corresponding to the known abnormal period, where the target parameter is used to represent a similarity between the period to be detected and the abnormal period, the bucket dividing vector corresponding to the abnormal period is determined by performing bucket dividing processing on sampling point data in the abnormal period according to the preset number of buckets, and the time length of the period to be detected is the same as that of the abnormal period;
the second processing module is used for obtaining an abnormal detection result aiming at the period to be detected according to the target parameter;
wherein the determining module comprises:
the mapping submodule is used for mapping the sub-bucket vector corresponding to the period to be detected into a binary vector according to a preset mapping rule so as to obtain a characteristic vector corresponding to the period to be detected;
and the first determining submodule is used for determining a target parameter between the period to be detected and the abnormal period according to the feature vector corresponding to the period to be detected and the feature vector corresponding to the abnormal period, wherein the feature vector corresponding to the abnormal period is obtained by mapping the sub-bucket vector corresponding to the abnormal period into a binary vector according to the preset mapping rule.
7. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5.
8. An electronic device, comprising:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to carry out the steps of the method of any one of claims 1 to 5.
CN201810552461.8A 2018-05-31 2018-05-31 Abnormality detection method, abnormality detection device, computer-readable storage medium, and electronic apparatus Active CN109034180B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810552461.8A CN109034180B (en) 2018-05-31 2018-05-31 Abnormality detection method, abnormality detection device, computer-readable storage medium, and electronic apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810552461.8A CN109034180B (en) 2018-05-31 2018-05-31 Abnormality detection method, abnormality detection device, computer-readable storage medium, and electronic apparatus

Publications (2)

Publication Number Publication Date
CN109034180A CN109034180A (en) 2018-12-18
CN109034180B true CN109034180B (en) 2020-11-03

Family

ID=64611832

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810552461.8A Active CN109034180B (en) 2018-05-31 2018-05-31 Abnormality detection method, abnormality detection device, computer-readable storage medium, and electronic apparatus

Country Status (1)

Country Link
CN (1) CN109034180B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111291131A (en) * 2019-12-25 2020-06-16 东软集团股份有限公司 Data processing method, data processing device, storage medium and electronic equipment
CN111310697B (en) * 2020-02-17 2023-03-24 硕橙（厦门）科技有限公司 Equipment operation period detection and health degree analysis method and device and storage medium
CN113280265B (en) * 2020-02-20 2022-08-05 中国石油天然气股份有限公司 Working condition identification method and device, computer equipment and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101441107B1 (en) * 2013-04-29 2014-09-23 주식회사 에스원 Method and apparatus for determining abnormal behavior
CN105262647A (en) * 2015-11-27 2016-01-20 广州神马移动信息科技有限公司 Abnormal index detection method and device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107085548A (en) * 2016-02-16 2017-08-22 阿里巴巴集团控股有限公司 A kind of method, device and electronic equipment for monitoring application program internal memory

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101441107B1 (en) * 2013-04-29 2014-09-23 주식회사 에스원 Method and apparatus for determining abnormal behavior
CN105262647A (en) * 2015-11-27 2016-01-20 广州神马移动信息科技有限公司 Abnormal index detection method and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于用户行为周期的移动设备异常检测方法;吴志忠等;《计算机系统应用》;20150415;摘要、第3-4节 *

Also Published As

Publication number Publication date
CN109034180A (en) 2018-12-18

Similar Documents

Publication Publication Date Title
CN109034180B (en) Abnormality detection method, abnormality detection device, computer-readable storage medium, and electronic apparatus
CN107861915B (en) Method and device for acquiring early warning threshold value and storage medium
CN109213655B (en) Solution determination method, device, storage medium and equipment for alarm
CN108885787B (en) Method for training image restoration model, image restoration method, device, medium, and apparatus
US10678914B2 (en) Virus program detection method, terminal, and computer readable storage medium
CN109656923B (en) Data processing method and device, electronic equipment and storage medium
CN109800858B (en) Application system abnormality detection method and device, readable storage medium and electronic equipment
CN109492531B (en) Face image key point extraction method and device, storage medium and electronic equipment
CN109658346B (en) Image restoration method and device, computer-readable storage medium and electronic equipment
CN111016908B (en) Vehicle driving position determining method and device, storage medium and electronic equipment
US20160314141A1 (en) Compression-based filtering for deduplication
KR20120110035A (en) Method and system for comparing documents based on different document-similarity calculation methods using adaptive weighting
CN112631888A (en) Fault prediction method and device of distributed system, storage medium and electronic equipment
CN111090582A (en) Error code positioning method and device, storage medium and electronic equipment
CN108804574B (en) Alarm prompting method and device, computer readable storage medium and electronic equipment
CN114567396A (en) Wireless communication method, fitting method of nonlinear function, terminal and equipment
CN110930110B (en) Distributed flow monitoring method and device, storage medium and electronic equipment
CN111209746B (en) Natural language processing method and device, storage medium and electronic equipment
CN111080233B (en) Method, device and storage medium for generating subscription information
CN109581324B (en) Abnormal frame data processing method and device
CN115019150A (en) Target detection fixed point model establishing method and device and readable storage medium
CN108173608B (en) Method, apparatus and storage medium for obtaining power estimation value and electronic device
CN114358581A (en) Method and device for determining abnormal threshold of performance index, equipment and storage medium
CN110413706B (en) Data processing method, data processing device, storage medium and electronic equipment
CN113010571A (en) Data detection method, data detection device, electronic equipment, storage medium and program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant