CN112445832B - Data anomaly detection method and device, electronic equipment and storage medium - Google Patents
Data anomaly detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112445832B CN112445832B CN201910804714.0A CN201910804714A CN112445832B CN 112445832 B CN112445832 B CN 112445832B CN 201910804714 A CN201910804714 A CN 201910804714A CN 112445832 B CN112445832 B CN 112445832B
- Authority
- CN
- China
- Prior art keywords
- trend
- time
- trend change
- time point
- point set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 77
- 230000008859 change Effects 0.000 claims abstract description 408
- 230000002159 abnormal effect Effects 0.000 claims abstract description 54
- 238000000034 method Methods 0.000 claims abstract description 28
- 238000001914 filtration Methods 0.000 claims description 23
- 238000012417 linear regression Methods 0.000 claims description 19
- 230000005856 abnormality Effects 0.000 claims description 17
- 230000004044 response Effects 0.000 claims description 16
- 238000012790 confirmation Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims description 4
- 238000012163 sequencing technique Methods 0.000 claims description 4
- 230000002547 anomalous effect Effects 0.000 claims 1
- 238000010200 validation analysis Methods 0.000 claims 1
- 230000009286 beneficial effect Effects 0.000 abstract description 5
- 238000012545 processing Methods 0.000 description 13
- 238000004891 communication Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 230000007774 longterm Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 230000000630 rising effect Effects 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 101150117538 Set2 gene Proteins 0.000 description 3
- 230000001174 ascending effect Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 101150055297 SET1 gene Proteins 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000012300 Sequence Analysis Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000010183 spectrum analysis Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Abstract
The disclosure relates to a data anomaly detection method, a device, an electronic device and a storage medium, wherein the method comprises the following steps: acquiring target service data to be detected; acquiring a first crossing time point set of an average value curve of the target service data in different first-class time periods and a second crossing time point set of an average value curve of the target service data in different second-class time periods according to the time sequence relation of the target service data; the second class time period is greater than the first class time period; acquiring a trend change time point set of the target service data according to the first crossing time point set and the second crossing time point set; acquiring trend change parameters between every two adjacent trend change time points in the trend change time point set; and responding to the trend change parameters to meet preset abnormal conditions, and confirming that the business data corresponding to the trend change parameters are abnormal. The method has the beneficial effects of reducing the abnormal detection cost and improving the accuracy of the detection result.
Description
Technical Field
The disclosure relates to the technical field of data processing, and in particular relates to a data anomaly detection method, a data anomaly detection device, electronic equipment and a storage medium.
Background
In the scene of analysis of the time sequence data of the business, the abnormal detection is a common scene, and the trend of the index can be obtained initially by analyzing the fluctuation rate and the slope of the time sequence business index. However, in general, the traffic fluctuation has random factors, and under the interference of the random factors, the index can have a plurality of short-term fluctuations, and an erroneous increase conclusion can be obtained if no interference rejection is performed.
The traditional data anomaly detection focuses on short-term fluctuation anomaly, a certain degree of prediction is carried out by fitting a time sequence, and if an actual value exceeds a prediction confidence interval, the data is considered to be abnormal; or performing derivative calculation on the original time sequence waveform, such as a singular spectrum analysis method, and performing abnormal scanning in a mode of obtaining a set threshold value of the derivative waveform. The long-term trend anomaly detection is different from the anomaly detection method of short-term fluctuation, and the long-term trend anomaly detection is characterized in that fluctuation is eliminated to a certain extent, a stable and smooth curve is obtained, and corresponding business problems are locked through change of the slope of the curve.
Short term anomaly detection, however, is susceptible to noise, such as heavy snow in the south, which affects the number of short video productions, and local base tower faults, which affect the network behavior in this area, but these are not a business anomaly, more like noise caused by a random phenomenon. And long-term abnormal detection is easy to cause lag of detection results, and cannot accurately position the time point of trend change, so that the accuracy of the abnormal detection results is low, and the business risk is increased.
Disclosure of Invention
The disclosure provides a data anomaly detection method, a data anomaly detection device, electronic equipment and a storage medium, so as to at least solve the problems that detection results in related technologies are easily affected by noise and the accuracy of anomaly detection results is not high. The technical scheme of the present disclosure is as follows:
according to a first aspect of an embodiment of the present disclosure, there is provided a data anomaly detection method, including:
acquiring target service data to be detected;
acquiring a first crossing time point set of an average value curve of the target service data in different first class time periods and a second crossing time point set of an average value curve of the target service data in different second class time periods according to the time sequence relation of the target service data; the second class time period is greater than the first class time period;
Acquiring a trend change time point set of the target service data according to the first crossing time point set and the second crossing time point set;
acquiring trend change parameters between every two adjacent trend change time points in the trend change time point set;
and responding to the trend change parameters to meet preset abnormal conditions, and confirming that the business data corresponding to the trend change parameters are abnormal.
Optionally, the step of acquiring the trend change time point set of the target service data according to the first crossing time point set and the second crossing time point set includes:
for each second crossing time point in the second crossing time point set, acquiring an N-th first crossing time point before the second crossing time point from the first crossing time point set as a trend change time point, wherein N is a positive integer;
and sequencing the trend change time points according to a time sequence to obtain the trend change time point set.
Optionally, the step of arranging the trend change time points according to the time sequence to obtain the trend change time point set includes:
Sorting the trend change time points according to a time sequence;
filtering the trend change time points according to the time distance between every two adjacent trend change time points until the time distance between every two adjacent trend change time points meets a preset time threshold, and constructing the trend change time point set according to the reserved trend change time points;
wherein each filtering process comprises:
taking every two adjacent trend change time points as a group, and acquiring the time distance between each group of trend change time points;
and deleting at least one trend change time point in the trend change time point combination corresponding to the time distance in response to the time distance exceeding a preset time threshold.
Optionally, the trend change parameter includes at least one of a linear regression slope, a change speed, and a change amplitude.
Optionally, the step of acquiring the trend change parameter between every two adjacent trend change time points in the trend change time point set includes:
acquiring a trend time period between every two adjacent trend change time points in the trend change time point set;
For each trend time period, carrying out linear regression on the business data in the trend time period, and acquiring the linear slope of the trend time period;
and acquiring the change speed and/or the change amplitude of the trend time period according to the start time, the end time, the starting point data value and the end point data value of the trend time period.
Optionally, the step of determining that the service data corresponding to the trend change parameter is abnormal in response to the trend change parameter meeting a preset abnormal condition includes:
acquiring the current trend of the target service data according to the trend change parameter of the current trend time period and the trend change parameter of the previous trend time period;
and responding to the current trend to meet a preset abnormal condition, confirming that the service data corresponding to the current trend time period is abnormal, and carrying out service alarm.
According to a second aspect of the embodiments of the present disclosure, there is provided a data anomaly detection apparatus including:
the target service data acquisition module is configured to acquire target service data to be detected;
a cross time point acquisition module configured to perform acquiring a first cross time point set of an average value curve of the target service data in different first category time periods and a second cross time point set of an average value curve of the target service data in different second category time periods according to a time sequence relation of the target service data; the second class time period is greater than the first class time period;
A trend change time point acquisition module configured to perform acquisition of a trend change time point set of the target service data according to the first intersection time point set and the second intersection time point set;
a trend change parameter acquisition module configured to perform acquisition of a trend change parameter between every two adjacent trend change time points in the trend change time point set;
and the data abnormality confirmation module is configured to execute the confirmation of abnormal business data corresponding to the trend change parameter in response to the trend change parameter meeting a preset abnormal condition.
Optionally, the trend change time point obtaining module includes:
a trend change time point acquisition sub-module configured to perform acquisition of an nth first intersection time point preceding the second intersection time point from the first intersection time point set as a trend change time point for each of the second intersection time points in the second intersection time point set, N being a positive integer;
and the trend change time point set construction submodule is configured to execute the sequence of the trend change time points according to the time sequence to obtain the trend change time point set.
Optionally, the trend change time point set building sub-module includes:
a trend change time point ranking unit configured to perform ranking of the trend change time points in time order;
a trend change time point filtering unit configured to perform filtering on the trend change time points according to a time distance between every two adjacent trend change time points until the time distance between every two adjacent trend change time points reserved meets a preset time threshold, and construct the trend change time point set with the reserved trend change time points;
wherein each filtering process comprises:
taking every two adjacent trend change time points as a group, and acquiring the time distance between each group of trend change time points;
and deleting at least one trend change time point in the trend change time point combination corresponding to the time distance in response to the time distance exceeding a preset time threshold.
Optionally, the trend change parameter includes at least one of a linear regression slope, a change speed, and a change amplitude.
Optionally, the trend change parameter obtaining module includes:
a trend time period acquisition sub-module configured to perform acquisition of a trend time period between every two adjacent trend change time points in the trend change time point set;
A linear slope obtaining sub-module configured to perform linear regression on the service data in the trend time period for each trend time period, and obtain a linear slope of the trend time period;
an actual change parameter obtaining sub-module configured to obtain a change speed and/or a change amplitude of the trend time period according to the start time, the end time, the start point data value and the end point data value of the trend time period.
Optionally, the data anomaly confirmation module includes:
the current trend acquisition sub-module is configured to execute trend change parameters according to a current trend time period and trend change parameters of a previous trend time period and acquire the current trend of the target business data;
and the data abnormality confirmation sub-module is configured to execute the steps of responding to the current trend to meet a preset abnormality condition, confirming that the service data corresponding to the current trend time period is abnormal, and carrying out service alarm.
According to a third aspect of embodiments of the present disclosure, there is provided an electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement any of the data anomaly detection methods as previously described.
According to a fourth aspect of embodiments of the present disclosure, there is provided a storage medium, which when executed by a processor of an electronic device, enables the electronic device to perform any one of the data anomaly detection methods as described previously.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product which, when executed by a processor of an electronic device, enables the electronic device to perform any one of the data anomaly detection methods as previously described.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:
in the embodiment of the disclosure, target service data to be detected is obtained; acquiring a first crossing time point set of an average value curve of the target service data in different first class time periods and a second crossing time point set of an average value curve of the target service data in different second class time periods according to the time sequence relation of the target service data; the second class time period is greater than the first class time period; acquiring a trend change time point set of the target service data according to the first crossing time point set and the second crossing time point set; acquiring trend change parameters between every two adjacent trend change time points in the trend change time point set; and responding to the trend change parameters to meet preset abnormal conditions, and confirming that the business data corresponding to the trend change parameters are abnormal. The method has the beneficial effects of reducing the cost of anomaly detection and improving the accuracy of the detection result.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure and do not constitute an undue limitation on the disclosure.
Fig. 1 is a flowchart illustrating a data anomaly detection method according to an example embodiment.
Fig. 2 is a flow chart illustrating another data anomaly detection method according to an example embodiment.
Fig. 3 is a block diagram illustrating a data anomaly detection apparatus according to an example embodiment.
Fig. 4 is a block diagram illustrating another data anomaly detection apparatus according to an example embodiment.
Fig. 5 is a block diagram of an apparatus according to an example embodiment.
Fig. 6 is a block diagram of another apparatus according to an example embodiment.
Detailed Description
In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the foregoing figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the disclosure described herein may be capable of operation in sequences other than those illustrated or described herein. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.
Fig. 1 is a flowchart of a data anomaly detection method according to an exemplary embodiment, and as shown in fig. 1, the data anomaly detection method may be used in a mobile terminal such as a mobile phone, a computer, etc., and includes the following steps:
in step S11, target service data to be detected is acquired.
In step S12, according to the time sequence relationship of the target service data, a first crossing time point set of the average value curves of the target service data in different first category time periods and a second crossing time point set of the average value curves of the target service data in different second category time periods are obtained; the second class time period is greater than the first class time period.
In the embodiment of the present disclosure, in order to perform data anomaly detection, first, target service data to be detected needs to be acquired, specifically, the target service data may be acquired by any available method, which is not limited to the embodiment of the present disclosure. Moreover, according to the generation time of different target service data, the time sequence relation of the target service data can be obtained, and further, the abnormality detection is sequentially carried out according to the time sequence relation of the target service data.
The average value curve is a time sequence analysis method commonly used in statistics, is commonly used in time sequence data, and is used for trowelling short-term fluctuation, reflecting a longer-term change trend or period, specifically, the average value curve can be formed by averaging data in a certain time period and connecting the average values of different time periods. Moreover, the point where the average curves of two different time periods intersect is typically the location where the trend of the data changes.
As described above, the average curves under different time periods intersect near the inflection point of the service, and the location where the inflection point occurs is usually the moment when the abnormal trend of the service changes. In general, two average value curves with shorter time periods intersect at multiple points, and a certain hysteresis exists at the intersection point of two average value curves with longer time periods, so in the embodiment of the present disclosure, in order to reduce hysteresis of long-term anomaly detection while eliminating short-term fluctuation, average value curves with different types of time periods may be combined.
Wherein, the categories of the time periods can be divided according to the lengths of the time periods. For example, two time period categories may be set according to the length of the time period, and the time length of the first category time period is smaller than the time length of the second category time period, where the first category time period is a short time period with respect to the second category time period, and the second category time period is a long time period with respect to the first category time period.
Specifically, a first crossing time point set of an average value curve of the target service data in different first category time periods and a second crossing time point set of an average value curve of the target service data in different second category time periods can be obtained according to the time sequence relation of the target service data; the second class time period is greater than the first class time period.
For example, two different first class time periods w1 and w2 may be selected according to the service requirement, and w1< w2, two average curves with time period lengths w1 and w2 may be taken for the target service data respectively. At this time, since w2 is greater than w1, the average curve under w2 has a longer delay than the average curve under w1, and the average curve under w2 always changes slower than the average curve under w1, regardless of whether the rising or falling speed is slow, so that when the trend of the target traffic data changes, the two average curves with different speeds must intersect. Therefore, the intersection time point set of the average value curves under the first category time period, namely the first intersection time point set, can be obtained according to the positions of the intersection points of the average value curves corresponding to w1 and w 2.
Correspondingly, different second category time periods such as w3 and w4 can be selected according to service requirements, and w3 is smaller than w4, so that a second crossing time point set of the average value curve of the target service data under w3 and w4 can be obtained. The method for acquiring the first cross time point set of the average value curve in the first class time period may be referred to specifically, and will not be described herein.
The specific number of the first class time period and the second class time period may be preset according to requirements, and the crossing time points of the average value curves in the plurality of different time periods may include at least any two crossing time points of the average value curves. The specific value ranges of the first class time period and the second class time period may be preset according to specific service requirements, which is not limited in the embodiments of the present disclosure.
In addition, in the embodiment of the disclosure, the change condition of the average value curves before and after the intersection point can also represent the change trend of the current data. For example, for the above-described time periods w1 and w2, when the state changes from the value of the average curve corresponding to w1 < the value of the average curve corresponding to w2 to the value of the average curve corresponding to w1 > the value of the average curve corresponding to w2, the trend of the target traffic data changes from a downward or gentle trend to an upward trend; otherwise, when the state changes from the value of the average value curve corresponding to w1 to the value of the average value curve corresponding to w2 to the value of the average value curve corresponding to w1 < the value of the average value curve corresponding to w2, the trend of the target service data changes from an ascending trend to a descending or stable trend.
In step S13, a trend change time point set of the target service data is acquired according to the first crossing time point set and the second crossing time point set.
As mentioned above, the existing trend inflection points detected in a shorter time period, that is, the crossing time points are numerous, so that the complexity of service analysis and investigation is increased, the method is easily affected by short-term outliers, the effect fluctuation is relatively large, and the long-term trend change of more focused service cannot be accurately detected; the longer time period is used for detection, the serious problem of inflection point hysteresis occurs, namely the found trend inflection point is obviously delayed from the real service inflection point, and the larger the time period length is, the more obvious the hysteresis is, and the accurate position of the trend inflection point cannot be accurately positioned.
Therefore, in the embodiment of the present disclosure, in order to improve the accuracy of the detected trend inflection point, the final trend inflection point may be determined by combining the intersection time points in the long and short time periods, that is, the trend change time point set of the target service data is obtained according to the first intersection time point set corresponding to the first category time period and the second intersection time point set corresponding to the second category time period. Specifically, the trend change time point set of the target service data can be obtained by correcting the second crossing time point in the second category time period according to the first crossing time point in the first category time period, and/or filtering the first crossing time point in the first category time period according to the second crossing time point in the second category time period. The correction policy and/or the filtering policy may be preset according to requirements, which is not limited to the embodiments of the present disclosure.
In step S14, a trend change parameter between every two adjacent trend change time points in the trend change time point set is acquired.
After the final trend change time point set of the target business data is obtained, the data anomaly detection can be further performed according to each trend change time point in the trend change time point set. At this time, the data anomaly detection can be performed by taking the service data between every two adjacent trend change time points in the front-back direction as a unit, so as to obtain the trend change parameters between every two adjacent trend change time points in the trend change time point set. Wherein, the two adjacent trend change time points refer to two trend change time points adjacent in time sequence.
The content specifically included in the trend change parameter may be preset according to the requirement, and the embodiment of the disclosure is not limited. For example, trend change parameters may be set including, but not limited to, data change speed, data change amplitude, and the like of traffic data between the front-to-back adjacent trend change time points. Moreover, in the embodiments of the present disclosure, the trend change parameter may also be obtained in any available manner, nor is the embodiment of the present disclosure limited thereto.
For example, if the trend change parameter includes a data change amplitude, the data values at two adjacent trend change time points can be further obtained, so that the difference between the data value at the next trend change time point and the data value at the previous trend change time point is further obtained, and the data change amplitude between the two adjacent trend change time points is obtained accordingly.
In step S15, in response to the trend change parameter meeting a preset abnormal condition, confirming that the service data corresponding to the trend change parameter is abnormal.
After the trend change parameters between every two adjacent trend change time points in the trend change time point set are obtained, whether the data between every two adjacent trend change time points is abnormal or not can be detected according to the trend change parameters. Specifically, a corresponding abnormal condition may be preset according to the service requirement for the trend change parameter, and if the trend change parameter between two adjacent trend change time points meets the abnormal condition, it may be determined that the service data corresponding to the corresponding trend change parameter is abnormal, that is, it is determined that the service data between two adjacent trend change time points is abnormal, and/or it is determined that the service data at the previous trend change time point in the two adjacent trend change time points is abnormal, and so on. Otherwise, it can be confirmed that the service data corresponding to the corresponding trend change parameters are not abnormal.
For example, assuming that the trend change parameter includes a change amplitude, the preset prediction condition is that the change amplitude is a negative value, if the change amplitude between certain two adjacent trend change time points is a negative value, it is possible to confirm that the corresponding business data is abnormal.
In the embodiment of the present disclosure, after each trend change parameter between two adjacent trend change time points is obtained, that is, the data anomaly detection is performed for the current trend change parameter; after the trend change parameters between every two adjacent trend change time points in the trend change time point set are obtained, carrying out anomaly detection on service data corresponding to the corresponding trend change parameters according to the trend change parameters between every two adjacent trend change time points in sequence; or the method can take a certain time period as a period, and carry out abnormality detection on the service data in one time period at a time; etc. The specific configuration may be preset according to the requirement, and the embodiments of the present disclosure are not limited.
In the embodiment of the disclosure, target service data to be detected is obtained; acquiring a first crossing time point set of an average value curve of the target service data in different first class time periods and a second crossing time point set of an average value curve of the target service data in different second class time periods according to the time sequence relation of the target service data; the second class time period is greater than the first class time period; acquiring a trend change time point set of the target service data according to the first crossing time point set and the second crossing time point set; acquiring trend change parameters between every two adjacent trend change time points in the trend change time point set; and responding to the trend change parameters to meet preset abnormal conditions, and confirming that the business data corresponding to the trend change parameters are abnormal. The method has the beneficial effects of reducing the cost of anomaly detection and improving the accuracy of the detection result.
Referring to fig. 2, in an embodiment of the present disclosure, the step S13 may further include:
step S131, for each second intersection time point in the second intersection time point set, acquiring an nth first intersection time point before the second intersection time point from the first intersection time point set, where N is a positive integer as a trend change time point.
And step S132, sorting the trend change time points according to a time sequence to obtain the trend change time point set.
Because the time periods corresponding to the first crossing time point sets are smaller than the time periods corresponding to the second crossing time point sets, that is, the number of first crossing time points contained in the first crossing time point sets is generally greater than the number of second crossing time points contained in the second crossing time point sets, and the hysteresis of the second crossing time points is higher than that of the first crossing time points. Therefore, in the embodiment of the disclosure, in order to improve the accuracy of each trend change time point obtained by screening, according to each second intersection time point in the second intersection time point set, a first intersection time point which is matched with the first intersection time point set and has low hysteresis is obtained from the first intersection time point set as a trend change time point, so that the accuracy of each trend change time point is improved while the number of trend change time points is reduced.
Specifically, for each second intersection time point in the second intersection time point set, an nth first intersection time point before the second intersection time point is obtained from the first intersection time point set and is used as a trend change time point, and further the trend change time points can be ordered according to a time sequence to obtain the trend change time point set. The value of N is a positive integer, and the specific value of N may be preset according to the service requirement, which is not limited in this embodiment of the disclosure, for example, N may be 1, and so on.
For example, for the first set of intersecting time points set_l and the second set of intersecting time points set_2, both sets of time points set_1 and set_2 may be arranged in reverse order in terms of time, then each second intersecting time point t_i in set_2 is traversed, the nth first intersecting time point t_s preceding t_i is found from set_1 as a trend change time point, and the trend change time point corresponding to each second intersecting time point may be acquired by analogy. And then, the trend change time points can be sequenced according to the time sequence, and the current trend change time point set of the target service data is constructed and obtained.
Optionally, in an embodiment of the disclosure, the step S132 may further include:
a1, sorting the trend change time points according to a time sequence;
step A2, filtering the trend change time points according to the time distance between every two adjacent trend change time points until the time distance between every two adjacent trend change time points meets a preset time threshold, and constructing the trend change time point set according to the reserved trend change time points;
wherein each filtering process comprises:
step A21, taking every two adjacent trend change time points as a group, and obtaining the time distance between each group of trend change time points;
and step A22, deleting at least one trend change time point in the trend change time point combination corresponding to the time distance in response to the time distance exceeding a preset time threshold.
In addition, in practical application, if the two trend change time points are closer, the two trend change time points are still easily affected by short-term abnormal values during data abnormal detection, and the complexity of business data abnormal detection is increased.
Therefore, in the embodiment of the present disclosure, in order to further reduce the influence of the short-term outlier and the complexity of the data anomaly detection, after the trend change time points are sorted according to the time sequence, the trend change time points may be further filtered according to the time distance between every two adjacent trend change time points, so as to avoid that the trend change time points are too close. Specifically, the trend change time points may be filtered according to the time distance between every two adjacent trend change time points until the time distance between every two adjacent trend change time points that are reserved meets a preset time threshold, and the trend change time point set is constructed according to the reserved trend change time points. The specific value of the preset time threshold may be preset according to the service requirement, which is not limited in the embodiments of the present disclosure.
In addition, in each filtering process, every two adjacent trend change time points can be grouped to obtain the time distance between each group of trend change time points; and deleting at least one trend change time point in the trend change time point combination corresponding to the time distance in response to the time distance exceeding a preset time threshold.
Specifically, when the time distance between two trend change time points in the trend change time point combination exceeds the preset time threshold, at least one trend change time point in the corresponding trend change time point combination may be deleted accordingly, and when only one trend change time point is deleted, the former trend change time point may be deleted, the latter trend change time point may also be deleted, and specifically, the time distance between two trend change time points may be preset according to the requirement.
Moreover, in order to ensure that the time distance between every two adjacent trend change time points in the finally generated trend change time point set meets the preset time threshold, at least one filtering may be performed on the initially acquired trend change time points.
Optionally, in an embodiment of the present disclosure, the trend change parameter may include, but is not limited to, at least one of a linear regression slope, a change speed, and a change amplitude.
The linear regression slope may include a slope obtained by linearly regressing the business data between two adjacent trend change time points; the actual change speed may include an average change speed, a real-time change speed, etc. of the service data between two adjacent trend change time points; the actual change amplitude may include the change amplitude of the service data of the latter trend change time point with respect to the former trend change time point, and the like.
Referring to fig. 2, in an embodiment of the present disclosure, the step S14 may further include:
step S141, obtaining a trend time period between every two adjacent trend change time points in the trend change time point set;
step S142, for each trend time period, carrying out linear regression on the business data in the trend time period, and obtaining the linear slope of the trend time period;
step S143, obtaining a change speed and/or a change amplitude of the trend time period according to the start time, the end time, the start data value and the end data value of the trend time period.
In the embodiment of the disclosure, in order to conveniently obtain the trend change parameters between every two adjacent trend change time points, the target service data may be first divided by the trend change time points, so as to obtain trend time periods between every two adjacent trend change time points in the trend change time point set, and further, for each trend time period, linear regression may be performed on the service data in one trend time period, so as to obtain the linear slope of the corresponding trend time period. Wherein linear regression may be performed in any useful manner, and embodiments of the present disclosure are not limited in this regard.
In addition, the change speed and/or the change amplitude of each trend time period can be obtained according to the start time, the end time, the starting point data value and the end point data value of each trend time period.
For example, assuming that the start time of the current trend period is t0, the end time is t1, the data value at the start is y0 and the data value at the end is yt, the change speed of the corresponding trend period may be obtained to include (yt-y 0)/(t 1-t 0), and the change amplitude may include yt-y0.
Referring to fig. 2, in an embodiment of the present disclosure, the step S15 may further include:
step S151, obtaining the current trend of the target business data according to the trend change parameter of the current trend time period and the trend change parameter of the previous trend time period;
and step S152, responding to the current trend to meet a preset abnormal condition, confirming that the service data corresponding to the current trend time period is abnormal, and carrying out service alarm.
In practical applications, the expected trend of data change will also be different according to different service requirements. For example, data trends tend to be relatively smooth for some conventional traffic, data trends tend to be continually downward for some problem-elimination class traffic, and data trends tend to be continually upward for some data-production class traffic. The exception conditions set by different services will also be correspondingly different. Or, some data services require that their data change trend is relatively stable, for example, in a rising or steady state all the time, or in a falling state all the time, etc.
In addition, in the embodiment of the present disclosure, in order to timely perform data anomaly alarm, real-time data anomaly detection may be performed on a current trend time period including current latest data, and specifically, the current trend of the target service data may be obtained according to a trend change parameter of the current trend time period and a trend change parameter of a previous trend time period.
For example, according to the trend change parameter of the current trend time period, it may be known that the data of the current trend time period is in an ascending trend, and according to the trend change parameter of the previous trend time period, it is known that the data of the previous trend time period is in a descending trend, and then it may be obtained that the current trend of the target service data is changed from ascending to descending. If the preset abnormal condition comprises that the current trend is changed from rising or rising at a high speed to falling or being gentle, the abnormal service data corresponding to the current trend time period can be confirmed, and service alarm is carried out on the target service data. The specific service alarm mode can be set in a self-defined manner according to the requirement, and the embodiment of the disclosure is not limited.
In addition, in order to facilitate the related staff to know the position of the data abnormality in time, the trend time period of the data abnormality and/or the previous trend change time point in the trend time period can be simultaneously prompted when the service alarms, and the like. The specific prompting mode can also be set in a self-defined manner according to the requirement, and the embodiment of the disclosure is not limited.
In the embodiment of the present disclosure, by acquiring, for each second intersection time point in the second intersection time point set, an nth first intersection time point preceding the second intersection time point from the first intersection time point set as a trend change time point, N is a positive integer; and sequencing the trend change time points according to a time sequence to obtain the trend change time point set. And sorting the trend change time points according to a time sequence; filtering the trend change time points according to the time distance between every two adjacent trend change time points until the time distance between every two adjacent trend change time points meets a preset time threshold, and constructing the trend change time point set according to the reserved trend change time points; wherein each filtering process comprises: taking every two adjacent trend change time points as a group, and acquiring the time distance between each group of trend change time points; and deleting at least one trend change time point in the trend change time point combination corresponding to the time distance in response to the time distance exceeding a preset time threshold. The accuracy of each trend change time point can be improved, so that the workload of abnormality detection can be reduced, and the accuracy of an abnormality detection result is improved.
Moreover, in an embodiment of the present disclosure, the trend change parameter includes at least one of a linear regression slope, a change speed, and a change amplitude. Moreover, a trend time period between every two adjacent trend change time points in the trend change time point set can also be acquired; for each trend time period, carrying out linear regression on the business data in the trend time period, and acquiring the linear slope of the trend time period; and acquiring the change speed and/or the change amplitude of the trend time period according to the start time, the end time, the starting point data value and the end point data value of the trend time period. The accuracy of the acquired trend change parameters can be improved.
In addition, the current trend of the target service data can be obtained according to the trend change parameter of the current trend time period and the trend change parameter of the previous trend time period; and responding to the current trend to meet a preset abnormal condition, confirming that the service data corresponding to the current trend time period is abnormal, and carrying out service alarm. Therefore, the method and the device can timely detect the abnormality of the current service data, timely alarm the service under the condition of abnormal data, and improve the accuracy of the data detection result and the timeliness of the alarm.
Fig. 3 is a block diagram of a data anomaly detection apparatus according to an example embodiment. Referring to fig. 3, the apparatus includes: the system comprises a target service data acquisition module 21, a crossing time point acquisition module 22, a trend change time point acquisition module 23, a trend change parameter acquisition module 24 and a data abnormality confirmation module 25.
A target service data acquisition module 21 configured to perform acquisition of target service data to be detected;
a cross time point acquisition module 22 configured to perform acquiring a first cross time point set of an average value curve of the target service data in different first category time periods and a second cross time point set of an average value curve of the target service data in different second category time periods according to a timing relationship of the target service data; the second class time period is greater than the first class time period;
a trend change time point acquisition module 23 configured to perform acquisition of a trend change time point set of the target service data from the first crossing time point set and the second crossing time point set;
a trend change parameter acquisition module 24 configured to perform acquisition of a trend change parameter between every two adjacent trend change time points in the trend change time point set;
A data anomaly confirmation module 25 configured to perform confirmation of anomaly of the service data corresponding to the trend change parameter in response to the trend change parameter satisfying a preset anomaly condition.
In the embodiment of the disclosure, target service data to be detected is obtained; acquiring a first crossing time point set of an average value curve of the target service data in different first class time periods and a second crossing time point set of an average value curve of the target service data in different second class time periods according to the time sequence relation of the target service data; the second class time period is greater than the first class time period; acquiring a trend change time point set of the target service data according to the first crossing time point set and the second crossing time point set; acquiring trend change parameters between every two adjacent trend change time points in the trend change time point set; and responding to the trend change parameters to meet preset abnormal conditions, and confirming that the business data corresponding to the trend change parameters are abnormal. The method has the beneficial effects of reducing the cost of anomaly detection and improving the accuracy of the detection result.
Referring to fig. 4, in an embodiment of the present disclosure, the trend change time point obtaining module 23 may further include:
a trend change time point acquisition sub-module 231 configured to perform acquisition of an nth first intersection time point preceding the second intersection time point from the first intersection time point set as a trend change time point for each of the second intersection time points in the second intersection time point set, N being a positive integer;
the trend change time point set construction submodule 232 is configured to execute ranking of the trend change time points according to time sequence to obtain the trend change time point set.
Optionally, in an embodiment of the disclosure, the trend change time point set constructing submodule 232 may further include:
a trend change time point ranking unit configured to perform ranking of the trend change time points in time order;
a trend change time point filtering unit configured to perform filtering on the trend change time points according to a time distance between every two adjacent trend change time points until the time distance between every two adjacent trend change time points reserved meets a preset time threshold, and construct the trend change time point set with the reserved trend change time points;
Wherein each filtering process comprises:
taking every two adjacent trend change time points as a group, and acquiring the time distance between each group of trend change time points;
and deleting at least one trend change time point in the trend change time point combination corresponding to the time distance in response to the time distance exceeding a preset time threshold.
Optionally, in an embodiment of the disclosure, the trend change parameter includes at least one of a linear regression slope, a change speed, and a change amplitude.
Referring to fig. 4, in an embodiment of the disclosure, the trend change parameter obtaining module 24 may further include:
a trend time period acquiring sub-module 241 configured to perform acquisition of a trend time period between every two adjacent trend change time points in the trend change time point set;
a linear slope obtaining sub-module 242 configured to perform linear regression on the service data in the trend time period for each trend time period, and obtain a linear slope of the trend time period;
the actual change parameter obtaining sub-module 243 is configured to obtain a change speed and/or a change amplitude of the trend time period according to the start time, the end time, the start data value, and the end data value of the trend time period.
Referring to fig. 4, in an embodiment of the present disclosure, the data anomaly confirmation module 25 may further include:
a current trend obtaining sub-module 251 configured to perform trend change parameters according to a current trend time period and trend change parameters of a previous trend time period, and obtain a current trend of the target service data;
the data anomaly confirmation sub-module 252 is configured to perform confirmation of anomalies of the service data corresponding to the current trend time period in response to the current trend meeting a preset anomaly condition, and perform service alerting.
In the embodiment of the present disclosure, by acquiring, for each second intersection time point in the second intersection time point set, an nth first intersection time point preceding the second intersection time point from the first intersection time point set as a trend change time point, N is a positive integer; and sequencing the trend change time points according to a time sequence to obtain the trend change time point set. And sorting the trend change time points according to a time sequence; filtering the trend change time points according to the time distance between every two adjacent trend change time points until the time distance between every two adjacent trend change time points meets a preset time threshold, and constructing the trend change time point set according to the reserved trend change time points; wherein each filtering process comprises: taking every two adjacent trend change time points as a group, and acquiring the time distance between each group of trend change time points; and deleting at least one trend change time point in the trend change time point combination corresponding to the time distance in response to the time distance exceeding a preset time threshold. The accuracy of each trend change time point can be improved, so that the workload of abnormality detection can be reduced, and the accuracy of an abnormality detection result is improved.
Moreover, in an embodiment of the present disclosure, the trend change parameter includes at least one of a linear regression slope, a change speed, and a change amplitude. Moreover, a trend time period between every two adjacent trend change time points in the trend change time point set can also be acquired; for each trend time period, carrying out linear regression on the business data in the trend time period, and acquiring the linear slope of the trend time period; and acquiring the change speed and/or the change amplitude of the trend time period according to the start time, the end time, the starting point data value and the end point data value of the trend time period. The accuracy of the acquired trend change parameters can be improved.
In addition, the current trend of the target service data can be obtained according to the trend change parameter of the current trend time period and the trend change parameter of the previous trend time period; and responding to the current trend to meet a preset abnormal condition, confirming that the service data corresponding to the current trend time period is abnormal, and carrying out service alarm. Therefore, the method and the device can timely detect the abnormality of the current service data, timely alarm the service under the condition of abnormal data, and improve the accuracy of the data detection result and the timeliness of the alarm.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
In addition, in an embodiment of the present disclosure, there is also provided an electronic device including:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement any of the data anomaly detection methods as previously described.
The disclosed embodiments also provide a storage medium that, when executed by a processor of an electronic device, enables the electronic device to perform any one of the data anomaly detection methods as described previously.
The disclosed embodiments also provide a computer program product that, when executed by a processor of an electronic device, enables the electronic device to perform any one of the data anomaly detection methods as previously described.
FIG. 5 is a block diagram illustrating an apparatus 300 for data anomaly detection, according to an example embodiment. For example, apparatus 300 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, exercise device, personal digital assistant, or the like.
Referring to fig. 5, the apparatus 300 may include one or more of the following components: a processing component 302, a memory 304, a power component 306, a multimedia component 308, an audio component 310, an input/output (I/O) interface 312, a sensor component 314, and a communication component 316.
The processing component 302 generally controls overall operation of the apparatus 300, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 302 may include one or more processors 320 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 302 can include one or more modules that facilitate interactions between the processing component 302 and other components. For example, the processing component 302 may include a multimedia module to facilitate interaction between the multimedia component 308 and the processing component 302.
Memory 304 is configured to store various types of data to support operations at device 300. Examples of such data include instructions for any application or method operating on the device 300, contact data, phonebook data, messages, pictures, videos, and the like. The memory 304 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
The power supply component 306 provides power to the various components of the device 300. The power supply components 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 300.
The multimedia component 308 includes a screen between the device 300 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 308 includes a front-facing camera and/or a rear-facing camera. The front-facing camera and/or the rear-facing camera may receive external multimedia data when the device 300 is in an operational mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
The audio component 310 is configured to output and/or input audio signals. For example, the audio component 310 includes a Microphone (MIC) configured to receive external audio signals when the device 300 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 304 or transmitted via the communication component 316. In some embodiments, audio component 310 further comprises a speaker for outputting audio signals.
The I/O interface 312 provides an interface between the processing component 302 and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.
The sensor assembly 314 includes one or more sensors for providing status assessment of various aspects of the apparatus 300. For example, the sensor assembly 314 may detect the on/off state of the device 300, the relative positioning of the components, such as the display and keypad of the apparatus 300, the sensor assembly 314 may also detect a change in position of the apparatus 300 or one component of the apparatus 300, the presence or absence of user contact with the apparatus 300, the orientation or acceleration/deceleration of the apparatus 300, and a change in temperature of the apparatus 300. The sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. The sensor assembly 314 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 314 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 316 is configured to facilitate communication between the apparatus 300 and other devices, either wired or wireless. The device 300 may access a wireless network based on a communication standard, such as WiFi, an operator network (e.g., 2G, 3G, 4G, or 5G), or a combination thereof. In one exemplary embodiment, the communication component 316 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 316 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the apparatus 300 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.
In an exemplary embodiment, a storage medium is also provided, such as a memory 304, including instructions executable by the processor 320 of the apparatus 300 to perform the above-described method. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.
FIG. 6 is a block diagram illustrating an apparatus 400 for data anomaly detection, according to an example embodiment. For example, the apparatus 400 may be provided as a server. Referring to fig. 6, the apparatus 400 includes a processing component 422 that further includes one or more processors, and memory resources represented by memory 432, for storing instructions, such as applications, executable by the processing component 422. The application program stored in memory 432 may include one or more modules each corresponding to a set of instructions. Further, the processing component 422 is configured to execute instructions to perform any of the data anomaly detection methods described above.
The apparatus 400 may also include a power component 426 configured to perform power management of the apparatus 400, a wired or wireless network interface 450 configured to connect the apparatus 400 to a network, and an input output (I/O) interface 458. The apparatus 400 may operate based on an operating system stored in memory 432, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (14)
1. A data anomaly detection method, comprising:
acquiring target service data to be detected;
acquiring a first crossing time point set of an average value curve of the target service data in different first class time periods and a second crossing time point set of an average value curve of the target service data in different second class time periods according to the time sequence relation of the target service data; the second class time period is greater than the first class time period;
acquiring a trend change time point set of the target service data according to the first crossing time point set and the second crossing time point set;
acquiring trend change parameters between every two adjacent trend change time points in the trend change time point set;
and responding to the trend change parameters to meet preset abnormal conditions, and confirming that the business data corresponding to the trend change parameters are abnormal.
2. The data anomaly detection method according to claim 1, wherein the step of acquiring the trend change time point set of the target business data from the first crossing time point set and the second crossing time point set comprises:
for each second crossing time point in the second crossing time point set, acquiring an Nth first crossing time point before the second crossing time point from the first crossing time point set as the trend change time point, wherein N is a positive integer;
and sequencing the trend change time points according to a time sequence to obtain the trend change time point set.
3. The data anomaly detection method according to claim 2, wherein the step of arranging the trend change time points in time order to obtain the trend change time point set includes:
sorting the trend change time points according to a time sequence;
filtering the trend change time points according to the time distance between every two adjacent trend change time points until the time distance between every two adjacent trend change time points meets a preset time threshold, and constructing the trend change time point set according to the reserved trend change time points;
Wherein each filtering process comprises:
taking every two adjacent trend change time points as a group, and acquiring the time distance between each group of trend change time points;
and deleting at least one trend change time point in the trend change time point combination corresponding to the time distance in response to the time distance exceeding a preset time threshold.
4. A data anomaly detection method according to any one of claims 1 to 3, wherein the trend change parameter comprises at least one of a linear regression slope, a change speed, a change amplitude.
5. The data anomaly detection method according to claim 4, wherein the step of acquiring the trend change parameter between every two adjacent trend change time points in the trend change time point set includes:
acquiring a trend time period between every two adjacent trend change time points in the trend change time point set;
for each trend time period, carrying out linear regression on the business data in the trend time period, and acquiring the linear slope of the trend time period;
and acquiring the change speed and/or the change amplitude of the trend time period according to the start time, the end time, the starting point data value and the end point data value of the trend time period.
6. The data anomaly detection method according to claim 5, wherein the step of confirming that the traffic data corresponding to the trend change parameter is anomalous in response to the trend change parameter satisfying a preset anomaly condition comprises:
acquiring the current trend of the target service data according to the trend change parameter of the current trend time period and the trend change parameter of the previous trend time period;
and responding to the current trend to meet a preset abnormal condition, confirming that the service data corresponding to the current trend time period is abnormal, and carrying out service alarm.
7. A data anomaly detection device, comprising:
the target service data acquisition module is configured to acquire target service data to be detected;
a cross time point acquisition module configured to perform acquiring a first cross time point set of an average value curve of the target service data in different first category time periods and a second cross time point set of an average value curve of the target service data in different second category time periods according to a time sequence relation of the target service data; the second class time period is greater than the first class time period;
A trend change time point acquisition module configured to perform acquisition of a trend change time point set of the target service data according to the first intersection time point set and the second intersection time point set;
a trend change parameter acquisition module configured to perform acquisition of a trend change parameter between every two adjacent trend change time points in the trend change time point set;
and the data abnormality confirmation module is configured to execute the confirmation of abnormal business data corresponding to the trend change parameter in response to the trend change parameter meeting a preset abnormal condition.
8. The data anomaly detection apparatus according to claim 7, wherein the trend change time point acquisition module includes:
a trend change time point acquisition sub-module configured to perform acquisition of an nth first intersection time point preceding the second intersection time point from the first intersection time point set as the trend change time point for each of the second intersection time points in the second intersection time point set, N being a positive integer;
and the trend change time point set construction submodule is configured to execute the sequence of the trend change time points according to the time sequence to obtain the trend change time point set.
9. The data anomaly detection device of claim 8, wherein the trend change time point set construction sub-module comprises:
a trend change time point ranking unit configured to perform ranking of the trend change time points in time order;
a trend change time point filtering unit configured to perform filtering on the trend change time points according to a time distance between every two adjacent trend change time points until the time distance between every two adjacent trend change time points reserved meets a preset time threshold, and construct the trend change time point set with the reserved trend change time points;
wherein each filtering process comprises:
taking every two adjacent trend change time points as a group, and acquiring the time distance between each group of trend change time points;
and deleting at least one trend change time point in the trend change time point combination corresponding to the time distance in response to the time distance exceeding a preset time threshold.
10. The data anomaly detection apparatus according to any one of claims 7 to 9, wherein the trend change parameter includes at least one of a linear regression slope, a change speed, and a change amplitude.
11. The data anomaly detection device of claim 10, wherein the trend parameter acquisition module comprises:
a trend time period acquisition sub-module configured to perform acquisition of a trend time period between every two adjacent trend change time points in the trend change time point set;
a linear slope obtaining sub-module configured to perform linear regression on the service data in the trend time period for each trend time period, and obtain a linear slope of the trend time period;
an actual change parameter obtaining sub-module configured to obtain a change speed and/or a change amplitude of the trend time period according to the start time, the end time, the start point data value and the end point data value of the trend time period.
12. The data anomaly detection device of claim 11, wherein the data anomaly validation module comprises:
the current trend acquisition sub-module is configured to execute trend change parameters according to a current trend time period and trend change parameters of a previous trend time period and acquire the current trend of the target business data;
and the data abnormality confirmation sub-module is configured to execute the steps of responding to the current trend to meet a preset abnormality condition, confirming that the service data corresponding to the current trend time period is abnormal, and carrying out service alarm.
13. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the data anomaly detection method of any one of claims 1 to 6.
14. A storage medium, which when executed by a processor of an electronic device, enables the electronic device to perform the data anomaly detection method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910804714.0A CN112445832B (en) | 2019-08-28 | 2019-08-28 | Data anomaly detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910804714.0A CN112445832B (en) | 2019-08-28 | 2019-08-28 | Data anomaly detection method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112445832A CN112445832A (en) | 2021-03-05 |
| CN112445832B true CN112445832B (en) | 2024-02-23 |
Family
ID=74741401
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910804714.0A Active CN112445832B (en) | 2019-08-28 | 2019-08-28 | Data anomaly detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112445832B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115684918B (en) * | 2023-01-04 | 2023-04-07 | 北京志翔科技股份有限公司 | Switch state identification method and device |
| CN115994137B (en) * | 2023-03-23 | 2023-06-23 | 无锡弘鼎软件科技有限公司 | Data management method based on application service system of Internet of things |
| CN117391125B (en) * | 2023-12-08 | 2024-03-01 | 成都星幔长庚科技有限公司 | Data processing method and system based on neural network |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107566665A (en) * | 2017-08-15 | 2018-01-09 | 携程旅游信息技术（上海）有限公司 | Traffic method for detecting abnormality and its equipment |
| CN109146236A (en) * | 2018-07-06 | 2019-01-04 | 东软集团股份有限公司 | Indexes Abnormality detection method, device, readable storage medium storing program for executing and electronic equipment |
| CN109587008A (en) * | 2018-12-28 | 2019-04-05 | 华为技术服务有限公司 | Detect the method, apparatus and storage medium of abnormal flow data |
| CN109697207A (en) * | 2018-12-25 | 2019-04-30 | 苏州思必驰信息科技有限公司 | The abnormality monitoring method and system of time series data |
| CN110032670A (en) * | 2019-04-17 | 2019-07-19 | 腾讯科技（深圳）有限公司 | Method for detecting abnormality, device, equipment and the storage medium of time series data |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160164721A1 (en) * | 2013-03-14 | 2016-06-09 | Google Inc. | Anomaly detection in time series data using post-processing |
-
2019
- 2019-08-28 CN CN201910804714.0A patent/CN112445832B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107566665A (en) * | 2017-08-15 | 2018-01-09 | 携程旅游信息技术（上海）有限公司 | Traffic method for detecting abnormality and its equipment |
| CN109146236A (en) * | 2018-07-06 | 2019-01-04 | 东软集团股份有限公司 | Indexes Abnormality detection method, device, readable storage medium storing program for executing and electronic equipment |
| CN109697207A (en) * | 2018-12-25 | 2019-04-30 | 苏州思必驰信息科技有限公司 | The abnormality monitoring method and system of time series data |
| CN109587008A (en) * | 2018-12-28 | 2019-04-05 | 华为技术服务有限公司 | Detect the method, apparatus and storage medium of abnormal flow data |
| CN110032670A (en) * | 2019-04-17 | 2019-07-19 | 腾讯科技（深圳）有限公司 | Method for detecting abnormality, device, equipment and the storage medium of time series data |
Non-Patent Citations (1)
| Title |
|---|
| 基于关联度分析的生产异常模式挖掘;李春生，宋佳;计算机技术与发展;第27卷(第9期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112445832A (en) | 2021-03-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112445832B (en) | Data anomaly detection method and device, electronic equipment and storage medium | |
| CN111431727B (en) | Group display method, device, terminal, server and system | |
| EP3125187A1 (en) | Method and apparatus for recommending contact information | |
| CN104834559B (en) | Notification message display method and device | |
| US20210279473A1 (en) | Video processing method and apparatus, electronic device, and storage medium | |
| CN106919629B (en) | Method and device for realizing information screening in group chat | |
| CN109447125B (en) | Processing method and device of classification model, electronic equipment and storage medium | |
| CN104462296B (en) | File management method and device and terminal | |
| CN107508573B (en) | Crystal oscillator oscillation frequency correction method and device | |
| CN110502444B (en) | Testing method and testing device for image processing algorithm | |
| CN110312300B (en) | Control method, control device and storage medium | |
| EP3261324A1 (en) | Method and device for application switching | |
| CN109214175B (en) | Method, device and storage medium for training classifier based on sample characteristics | |
| KR101744696B1 (en) | Method, device, program and recording medium for filtering information | |
| CN111125388B (en) | Method, device and equipment for detecting multimedia resources and storage medium | |
| US11570693B2 (en) | Method and apparatus for sending and receiving system information, and user equipment and base station | |
| US11218990B2 (en) | Method and apparatus for receiving and sending system information, user equipment and base station | |
| CN113032627A (en) | Video classification method and device, storage medium and terminal equipment | |
| CN112019948B (en) | Intercommunication method for intercom equipment, intercom equipment and storage medium | |
| CN105260088B (en) | Information classification display processing method and device | |
| CN112256563A (en) | Android application stability testing method and device, electronic equipment and storage medium | |
| US20200293884A1 (en) | Image processing method and device and terminal | |
| CN112333233B (en) | Event information reporting method and device, electronic equipment and storage medium | |
| CN115314426A (en) | Data acquisition method, system, electronic device and storage medium | |
| CN114124866A (en) | Session processing method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |