CN108804703A

CN108804703A - A kind of data exception detection method and device

Info

Publication number: CN108804703A
Application number: CN201810631649.1A
Authority: CN
Inventors: 郑琳琳; 杨劲锋
Original assignee: Beijing Focus Shinkansen Information Technology Co Ltd
Current assignee: Beijing Bodian Zhihe Technology Co ltd
Priority date: 2018-06-19
Filing date: 2018-06-19
Publication date: 2018-11-13
Anticipated expiration: 2038-06-19
Also published as: CN108804703B

Abstract

The present invention provides a kind of data exception detection method and device, in the case where the current service data got is periodic data, first kind adjustment at least once can be carried out to current service data and Second Type adjusts at least once, and at least one of period abnormality detection and inflection point abnormality detection are carried out to the current service data after adjustment, being deposited with current service data after the adjustment can determine that Exception Type is that period exception or inflection point are abnormal when abnormal, and it is realized by first kind adjustment and Second Type adjustment for periodic data and the dynamic multi of periodic data is adjusted, improve the accuracy of periodic data abnormality detection.In the case where the current service data got is not periodic data; can at least one of deviation abnormality detection and inflection point abnormality detection directly be carried out to current service data, determine that Exception Type is that deviation exception or inflection point are abnormal when abnormal to be deposited in current service data.

Description

A kind of data exception detection method and device

Technical field

The invention belongs to technical field of data processing, more specifically more particularly to a kind of data exception detection method and Device.

Background technology

Over time, the corresponding business datum of any appliance such as by the collected data of sensor or executes certain There may be problems for the data of a service generation, therefore are one in data analysis important for the abnormality detection of business datum Branch plays an important role in fields such as data centers, such as event detection can be carried out to data center, invasion is examined Survey, fraud detection, error detection etc..

It is to the method that business datum carries out abnormality detection at present：To current service data (i.e. acquired business datum) It is for statistical analysis, the rule of development of current service data is obtained, and data prediction is carried out according to the rule of development, is predicted Business datum obtains deviation of the actual traffic data relative to prediction business datum, according to reality after obtaining actual traffic data Deviation of the border business datum relative to prediction business datum, determines whether actual traffic data has exception, but data are different at present Often detection is only capable of whether detection actual traffic data has exception, and Exception Type can not be determined.

Invention content

In view of this, the purpose of the present invention is to provide a kind of data exception detection method and device, in current industry Exception Type is determined when data exception of being engaged in.Technical solution is as follows：

The present invention provides a kind of data exception detection method, the method includes：

Obtain current service data；

In the case where it is periodic data to determine the current service data, the current service data is carried out at least First kind adjustment and Second Type adjustment at least once, the current service data after being adjusted, wherein described first The adjusting parameter that type is adjusted with Second Type adjustment uses is different, and adjustment is to the current service data for the first time It is adjusted, other secondary adjustment in addition to first time adjusts are that the data obtained to last adjustment are adjusted；

At least one of period abnormality detection and inflection point abnormality detection are carried out to the current service data after the adjustment, It is at least one of abnormal with the presence or absence of period exception and inflection point with the current service data after the determination adjustment；

In the case where it is periodic data to determine the current service data not, the current service data is carried out inclined At least one of poor abnormality detection and inflection point abnormality detection, with the determination current service data with the presence or absence of deviation it is abnormal and At least one of inflection point exception.

Preferably, described in the case where it is periodic data to determine the current service data, to the current business Data carry out first kind adjustment at least once and the adjustment of Second Type at least once：

If ith is adjusted to first kind adjustment, the normal distribution of this time adjustment corresponding data is obtained, described will be somebody's turn to do The data point reuse that value is more than the first predetermined threshold value in the normal distribution of secondary adjustment corresponding data is that this time adjustment corresponds to The median of data, wherein i be equal to 1 when this time adjustment corresponding data be the current service data, i be more than 1 certainly So this time adjustment corresponding data adjusts obtained data for (i-1)-th time when number；

If ith is adjusted to Second Type adjustment, this time is obtained by Time Series algorithm and adjusts corresponding data Residual component, and obtain the normal distribution of the residual component of this time adjustment corresponding data, determine described in the secondary adjustment Value is more than data corresponding time point of the second predetermined threshold value in the normal distribution of the residual component of corresponding data, right Any value is more than the data of second predetermined threshold value：It is that this time adjusts corresponding data in the data by the data point reuse Desired value on corresponding time point, wherein this time adjustment corresponding data is in i for the current service data when i is equal to 1 This time adjustment corresponding data is to adjust obtained data (i-1)-th time when natural number more than 1, and second predetermined threshold value is different from First predetermined threshold value.

Preferably, the method further includes：If the current service data is the current service data of main business, and described The current service data of main business is corresponding with the current service data of at least one subservice, then working as after determining the adjustment Preceding business datum is deposited when abnormal, by the current service data of the subservice to the current service data of the main business Exception is verified, this is at least one of the period is abnormal and inflection point is abnormal extremely.

Preferably, the current service data by the subservice is to the different of the current service data of the main business Often carrying out verification includes：

Data deviation direction of the current service data of the main business in corresponding each time point is determined, for belonging to In the current service data of any subservice of the main business：Determine the current service data of the subservice in each time Data deviation direction on point, wherein it is number that the data deviation direction, which is used to indicate the data development trend at the time point, According to any one increased in being reduced with data；

To any time point：If each subservice at the time point on data deviation direction be different from main business at this Data deviation direction on time point, it is determined that the exception of the current service data of the main business is pseudo- abnormal, and will be described The current service data of main business is determined as regular traffic data.

Preferably, the method further includes：If the current service data is the current service data of main business, and described The current service data of main business is corresponding with the current service data of at least one subservice, then is determining the current industry after adjusting There are when period exception, calculate expectation of the current service data of each subservice in corresponding each time point for data of being engaged in Value, to any time point：Determine the actual value of the current service data of the time point upper each subservice relative to the time point On desired value deviation；

The deviation for meeting the first preset condition is determined from all deviations, and according to the inclined of the first preset condition of satisfaction Difference carries out anomaly analysis to the current service data of the main business；

If the current service data is the current service data of main business, and the current service data pair of the main business There should be the current service data of at least one subservice, then be obtained there are when inflection point exception in the current service data of main business The differential data of the current service data of each subservice；

The data for meeting the second preset condition are determined from all differential datas, and according to the number for meeting the second preset condition Anomaly analysis is carried out according to the current service data to the main business.

Preferably, described in the case where it is periodic data to determine the current service data not, to the current industry Business data carry out at least one of deviation abnormality detection and inflection point abnormality detection：

In the case where it is periodic data to determine the current service data not, the current service data is being obtained just State distribution map, and the deviation point in the normal distribution of the current service data is obtained, according to the current service data Deviation point in normal distribution determines that the current service data is abnormal with the presence or absence of deviation；

In the case where it is periodic data to determine the current service data not, the difference of the current service data is obtained The normal distribution of divided data, and the deviation point in the normal distribution of the differential data is obtained, according to the differential data Normal distribution in deviation point, determine that the current service data is abnormal with the presence or absence of inflection point.

The present invention also provides a kind of data exception detection device, described device includes：

Acquiring unit, for obtaining current service data；

Adjustment unit, in the case where it is periodic data to determine the current service data, to the current industry Data of being engaged in carry out first kind adjustment at least once and Second Type adjusts at least once, the current business number after being adjusted According to wherein first kind adjustment is different with the adjusting parameter that Second Type adjustment uses, and adjustment is pair for the first time The current service data is adjusted, other secondary adjustment in addition to first time adjusts are the numbers obtained to last adjustment According to being adjusted；

First abnormality detecting unit, for carrying out period abnormality detection and inflection point to the current service data after the adjustment At least one of abnormality detection is abnormal abnormal with inflection point with the presence or absence of the period with the current service data after the determination adjustment At least one of；

Second abnormality detecting unit, it is right in the case where it is periodic data to determine the current service data not The current service data carries out at least one of deviation abnormality detection and inflection point abnormality detection, with the determination current business Data are at least one of abnormal with the presence or absence of deviation exception and inflection point.

Preferably, the adjustment unit obtains the secondary adjustment if being adjusted to first kind adjustment specifically for ith This time is adjusted value in the normal distribution of corresponding data and is more than the first predetermined threshold value by the normal distribution of corresponding data Data point reuse be median that this time adjusts corresponding data, wherein it is described that when i is equal to 1, this time, which adjusts corresponding data, Current service data, it is the data that (i-1)-th adjustment obtains that when i is the natural number more than 1, this time, which adjusts corresponding data,；

Preferably, described device further includes：Abnormal authentication unit, if being working as main business for the current service data Preceding business datum, and the current service data of the main business is corresponding with the current service data of at least one subservice, then exists Determine that the current service data after the adjustment is deposited when abnormal, by the current service data of the subservice to the main business The exception of the current service data of business is verified, this is at least one of the period is abnormal and inflection point is abnormal extremely.

Preferably, described device further includes：Deviation determination unit, if being main business for the current service data Current service data, and the current service data of the main business is corresponding with the current service data of at least one subservice, then Current service data after determining adjustment calculates the current service data of each subservice corresponding there are when period exception Desired value in each time point, to any time point：Determine the reality of the current service data of the time point upper each subservice Deviation of the actual value relative to the desired value on the time point；

First anomaly analysis unit, for determining the deviation for meeting the first preset condition, and root from all deviations Anomaly analysis is carried out to the current service data of the main business according to the deviation for meeting the first preset condition；

Differential data acquiring unit, if for the current service data that the current service data is main business, and it is described The current service data of main business is corresponding with the current service data of at least one subservice, then in the current business number of main business According to there are when inflection point exception, the differential data of the current service data of each subservice is obtained；

Second anomaly analysis unit, for determining the data for meeting the second preset condition, and root from all differential datas Anomaly analysis is carried out to the current service data of the main business according to the data for meeting the second preset condition.

Compared with prior art, above-mentioned technical proposal provided by the invention has the following advantages that：

It, can be right from above-mentioned technical proposal it is found that in the case where the current service data got is periodic data Current service data carries out first kind adjustment at least once and Second Type adjusts at least once, and to the current industry after adjustment Data of being engaged in carry out at least one of period abnormality detection and inflection point abnormality detection, so that it is determined that the current service data after adjustment Abnormal and at least one of inflection point is abnormal with the presence or absence of the period, being deposited with current service data after the adjustment when abnormal can be with Determining Exception Type is that period exception or inflection point are abnormal, and passes through first kind adjustment and for periodic data The adjustment of two types, which is realized, adjusts the dynamic multi of periodic data, improves the accuracy of periodic data abnormality detection.It is obtaining In the case that the current service data got not is periodic data, can deviation directly be carried out to current service data and examined extremely At least one of survey and inflection point abnormality detection, so that it is determined that current service data is with the presence or absence of in deviation exception and inflection point exception At least one, determine that Exception Type is that deviation exception or inflection point are abnormal when abnormal to be deposited in current service data.

Description of the drawings

In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is the present invention Some embodiments for those of ordinary skill in the art without creative efforts, can also basis These attached drawings obtain other attached drawings.

Fig. 1 is a kind of flow chart of data exception detection method provided in an embodiment of the present invention；

Fig. 2 is the schematic diagram of inflection point exception provided in an embodiment of the present invention；

Fig. 3 is another flow chart of data exception detection method provided in an embodiment of the present invention；

Fig. 4 is another flow chart of data exception detection method provided in an embodiment of the present invention；

Fig. 5 is a kind of structural schematic diagram of data exception detection device provided in an embodiment of the present invention；

Fig. 6 is another structural schematic diagram of data exception detection device provided in an embodiment of the present invention；

Fig. 7 is the yet another construction schematic diagram of data exception detection device provided in an embodiment of the present invention.

Specific implementation mode

In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art The every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.

Referring to Fig. 1, it illustrates a kind of flow chart of data exception detection method provided in an embodiment of the present invention, it is used for Exception Type is determined in current service data exception, may comprise steps of：

101：Obtain current service data.It can be understood that：Current service data is corresponding with some business and The data of acquisition, the data such as generated when executing some business, naturally it is also possible to when corresponding for business datum distribution Between, a detection cycle is such as distributed, then current service data is in the business datum in detection cycle, such as detection cycle It can be daily 8 points to 24 points, then current service data is the business datum in daily 8 points to 24 points.In practical application Under environment, business may include main business and subservice, and indicate which sub- industry is main business correspond to by service allocation list Business, if therefore determine that main business is corresponding at least one subservice by service allocation list when obtaining current service data, Also need to obtain the current service data of subservice when obtaining the current service data of main business.

102：In the case where it is periodic data to determine current service data, current service data is carried out at least once The first kind adjusts and Second Type adjusts at least once, the current service data after being adjusted.Wherein determine current business Whether data are the feasible patterns of periodic data：Pass through FFT (Fast Fourier Transform, in quick Fu Leaf transformation) the current service data corresponding period is calculated, illustrate if it can be derived that the current service data corresponding period current Business datum is periodic data, and it is periodic data otherwise to illustrate current service data not.

For periodic data, it is repeatedly adjusted by first kind adjustment and Second Type adjustment, The adjusting parameter that the adjustment of the middle first kind and Second Type adjustment use is different, and first time adjust be to current service data into Row adjustment, other numbers adjustment in addition to first time adjusts are that the data obtained to last adjustment are adjusted.Also It is to say repeatedly to be adjusted using different adjusting parameters, and it is also different to adjust corresponding data every time, specific tune for the first time Whole corresponding data are current service data (data that i.e. step 101 is got), and it is upper that other numbers, which adjust corresponding data, Obtained data are once adjusted, by taking jth time adjustment (j is the natural number more than 1) as an example, the corresponding data of jth time adjustment are the The data that j-1 adjustment obtains.The feasible pattern of first kind adjustment and Second Type adjustment is illustrated below：

If ith is adjusted to first kind adjustment, the normal distribution of this time adjustment corresponding data is obtained, by the secondary tune The data point reuse that value is more than the first predetermined threshold value in the normal distribution of whole corresponding data is that this time adjusts in corresponding data Digit, wherein this time adjustment corresponding data is current service data, the secondary tune when i is more than 1 natural number when i is equal to 1 Whole corresponding data is the data that (i-1)-th adjustment obtains.

Such as first predetermined threshold value can be this time adjustment corresponding data standard deviation the first preset multiple, such as but it is unlimited In six times, value in the normal distribution of corresponding data is adjusted for this time and is more than six times of standard deviations that this time adjusts corresponding data Data for, by the data point reuse be this time adjust corresponding data median, i.e., this time adjust corresponding data mediant, The data length of such as this time adjustment corresponding data is N, and when N is odd number, median is the data of (N+1)/2, is even in N Median is the average of the data and N/2 data of (N+1)/2 when number.

If ith is adjusted to Second Type adjustment, pass through STL (Seasonaland Trend decomposition Using Loess, Time Series) algorithm obtains the residual component of this time adjustment corresponding data, and obtains this time adjustment pair The normal distribution for answering the residual component of data determines value in the normal distribution of the residual component of this time adjustment corresponding data More than data corresponding time point of the second predetermined threshold value, the data of the second predetermined threshold value are more than to any value：It should Data point reuse is that this time adjusts desired value of the corresponding data on the data corresponding time point, wherein the secondary tune when i is equal to 1 Whole corresponding data is current service data, and when i is natural number more than 1, this time adjustment corresponding data is to be adjusted so as to for (i-1)-th time The data arrived, and the second predetermined threshold value is different from the first predetermined threshold value.

Such as second predetermined threshold value can be this time adjustment corresponding data standard deviation the second preset multiple, such as but it is unlimited In five times, value in the normal distribution of the residual component of corresponding data is adjusted for this time and is more than this time adjustment corresponding data For any data of five times of standard deviations：The data corresponding time point is determined first, then obtains the expectation on the time point The data point reuse is finally the desired value on the data corresponding time point by value, and expected value is the corresponding number of this time adjustment According to upper the sum of periodic quantity and Trend value at the time point, and periodic quantity and Trend value can be by STL algorithms to this time adjustment pair Data are answered to be decomposed to obtain periodic component and trend component, and from the periodic quantity determined in periodic component on the time point, From the Trend value determined in trend component on the time point.

And in practical application scene, random factor can have an impact current service data, can will pass through thus STL algorithms to the trend component that is decomposed of this time adjustment corresponding data as initial trend component, then by movement Digit algorithm calculates initial trend component, obtains the trend component of this time adjustment corresponding data, thus reduces current industry Data of being engaged in are interfered by random factor and the random variations due of generation, wherein moving the feasible pattern of median algorithm is： Assuming that initial trend component is X, moving window size is 7, and the data length of initial trend component is n, then moves median calculation The corresponding formula of method is as follows：

X_t'=median (X_t-3,X_t-2,X_t-1,X_t,X_t+1,X_t+2,X_t+3)(t>=3 or t<=n-4)

Multiple median X are being obtained by the formula of above-mentioned mobile median algorithm_t' after, to multiple median X_t' carry out Fitting (such as linear fit) obtains the trend component of this time adjustment corresponding data.

In the present embodiment, the second predetermined threshold value is preferably smaller than the first predetermined threshold value, and by the first predetermined threshold value into The first kind adjusts and then carries out Second Type adjustment at least once by the second predetermined threshold value row at least once, in this way may be used To carry out coarse adjustment to data by first kind adjustment, fine tuning is carried out to data by Second Type adjustment, to reduce current industry Influence of the abnormal data to periodic component and trend component in data of being engaged in, and when the data volume of current service data is smaller Current service data is easy to happen larger fluctuation, can reduce fluctuation coverage by coarse adjustment and fine tuning, improve the standard of detection Exactness.

Herein it should be noted is that：Can be one for the first predetermined threshold value and the second predetermined threshold value A threshold value changed with data development trend, such as current service data, data development trend increases for data, The first predetermined threshold value and the second predetermined threshold value can then be increased, if data development trend is reduced for data, first can be reduced Predetermined threshold value and the second predetermined threshold value.Additionally such as it can be arranged different for different time points according to practical application scene One predetermined threshold value and the second predetermined threshold value are such as arranged for the time point at night and in the two periods on daytime different First predetermined threshold value and the second predetermined threshold value, are no longer described in detail this present embodiment.

And for the adjustment number of first kind adjustment and Second Type adjustment, the feelings for the data that can be obtained according to adjustment Depending on condition, such as after carrying out ith first kind adjustment, the normal distribution of data that ith adjusts is obtained, determines the Value is more than the data volume of the data of the first predetermined threshold value in the i normal distribution for adjusting obtained data, if data volume is small In the first preset data amount, then stop first kind adjustment, and starts Second Type adjustment.Similarly Second Type is adjusted For, after carrying out ith Second Type adjustment, the normal distribution of the residual component for the data that ith adjusts is obtained, Determine that value is more than the number of the data of the second predetermined threshold value in the normal distribution of the residual component for the data that ith adjusts According to amount, if data volume is less than the second preset data amount, stop Second Type adjustment, wherein the first preset data amount and second is in advance If data volume can not limit its value depending on practical application to this present embodiment.

103：At least one of period abnormality detection and inflection point abnormality detection are carried out to the current service data after adjustment, To determine that the current service data after adjusting is at least one of abnormal with the presence or absence of period exception and inflection point.

In the present embodiment, period abnormality detection can be determined by above-mentioned STL algorithms, and the process of inflection point abnormality detection Can be：The normal distribution of the differential data (such as first-order difference data) of the current service data after adjustment is obtained, and is obtained Deviation point in the normal distribution of differential data, according to the deviation point in the normal distribution of differential data, after determining adjustment Current service data it is abnormal with the presence or absence of inflection point, as judged whether the current service data after adjusting belongs to deviation point to determine Current service data after adjustment is abnormal with the presence or absence of inflection point, if judging, the current service data after adjustment belongs to deviation point, It then determines that there are inflection point exceptions for the current service data after adjustment, inflection point exception is otherwise not present, in this way in current service data The catastrophe point in current service data in the case of there are inflection point exception after automatic positioning adjustment, so-called catastrophe point are that value becomes Change amount is more than the point for the preset value variable quantity of deviation point, and value variable quantity is the corresponding data of deviation point and its previous number Value between is poor, wherein deviation point be differential data normal distribution in the first preset multiple standard deviation except from Group's point, i.e., larger and smaller than the data except the standard deviation of the first preset multiple in the normal distribution of differential data, for the One preset multiple can be depending on practical application, to this present embodiment without limiting.

From the above-mentioned introduction to catastrophe point it is found that the present embodiment determines that the current service data after adjustment whether there is inflection point A kind of abnormal feasible pattern can be：It is more than preset value variable quantity with the presence or absence of value variable quantity in determination deviation point Point, there are inflection point exceptions for the current service data after illustrating adjustment in the presence of if.

104：In the case where it is periodic data to determine current service data not, it is different that deviation is carried out to current service data Often at least one of detection and inflection point abnormality detection, to determine that current service data is abnormal with the presence or absence of deviation exception and inflection point At least one of.

In the present embodiment, the feasible pattern of deviation abnormality detection is：The normal distribution of current service data is obtained, and The deviation point in the normal distribution of current service data is obtained, according to the deviation in the normal distribution of current service data Point determines that current service data is abnormal with the presence or absence of deviation.Such as judge whether current service data belongs to deviation point and work as to determine Preceding business datum is abnormal with the presence or absence of deviation, if judging, current service data belongs to deviation point, it is determined that current service data There are deviation exceptions, are otherwise not present that deviation is abnormal, wherein deviation point be current service data normal distribution in it is second pre- If the outlier except the standard deviation of multiple, i.e., it is larger and smaller than the second preset multiple in the normal distribution of current service data Standard deviation except data, for the second preset multiple can depending on practical application, to this present embodiment without limit It is fixed.

The feasible pattern of inflection point abnormality detection is：The normal distribution of the differential data of current service data is obtained, and is obtained The deviation point in the normal distribution of differential data is taken, according to the deviation point in the normal distribution of differential data, is determined current Business datum is abnormal with the presence or absence of inflection point, and detailed process please refers to the explanation in above-mentioned steps 103, no longer explains this present embodiment It states.

Herein it should be noted is that：In the current industry for determining the current service data after adjusting or not adjusting Data of being engaged in are there are when inflection point exception, if a catastrophe point causes, two inflection points are abnormal and the abnormal continuous but data of the two inflection points are sent out Exhibition trend is on the contrary, as shown in Fig. 2, the data development trend of two inflection point exceptions caused by a catastrophe point is data increase respectively It is reduced with data, then the latter inflection point regarded in the two inflection point exceptions is abnormal to be pseudo- abnormal, and determines that the latter inflection point is abnormal Corresponding data are normal data.

Referring to Fig. 3, it illustrates another flow chart of data exception detection method provided in an embodiment of the present invention, On the basis of above-mentioned Fig. 1, it can also include the following steps：

105：If current service data is the current service data of main business, and the current service data of main business is corresponding with The current service data of at least one subservice, then determining the current service data after adjusting, in the presence of exception, (the above-mentioned period is different Often and inflection point at least one of extremely) when, by the current service data of subservice to the current service data of main business Exception is verified, so that it is determined that going out whether the abnormal of the current service data of main business is pseudo- abnormal, reduces False Rate.

In the present embodiment, the process verified to the exception of the current service data of main business can be：Determine master Data deviation direction of the current service data of business in corresponding each time point, for belonging to any son of the main business The current service data of business：Data deviation direction on determining the current service data of the subservice at every point of time, It is arbitrary in data increase and data reduction that middle data deviation direction, which is used to indicate the data development trend at the time point, It is a kind of.

To any time point：If each subservice at the time point on data deviation direction be different from main business at this Data deviation direction on time point, it is determined that the exception of the current service data of main business is pseudo- abnormal, and by main business Current service data is determined as regular traffic data, realizes and is automatically corrected to pseudo- exception.

And the corresponding each time point of current service data can be generated time or the record of the current service data Time such as can record a business datum at interval of a period of time, can root for current service data corresponding time point Depending on practical application scene, the present embodiment does not limit this.

From above-mentioned technical proposal it is found that for the periodic data of main business, in the periodicity for determining the main business Data, can be by the periodic data of subservice corresponding with the main business to the period there are when period exception or inflection point exception Exception or inflection point are verified extremely, so as to reduce False Rate, and are pseudo- in determination to determine if being pseudo- abnormal The current service data of main business can be determined as to regular traffic data when abnormal, realize and pseudo- exception is automatically corrected.

Referring to Fig. 4, it illustrates another flow chart of data exception detection method provided in an embodiment of the present invention, it can To include the following steps：

401：Obtain current service data.

402：In the case where it is periodic data to determine current service data, current service data is carried out at least once The first kind adjusts and Second Type adjusts at least once, the adjustment of the current service data after being adjusted, the wherein first kind It is different with the adjusting parameter that Second Type adjustment uses, and adjustment is adjusted to current service data for the first time, removes first Other numbers adjustment except secondary adjustment is that the data obtained to last adjustment are adjusted.

403：At least one of period abnormality detection and inflection point abnormality detection are carried out to the current service data after adjustment, To determine that the current service data after adjusting is at least one of abnormal with the presence or absence of period exception and inflection point.

In the present embodiment, step 401 is to step 403：It is identical to step 103 as above-mentioned steps 101, to this present embodiment No longer step 401 to step 403 is described in detail.

404：If current service data is the current service data of main business, and the current service data of main business is corresponding with The current service data of at least one subservice is then determining that the current service data after adjusting there are when period exception, calculates Desired value of the current service data of each subservice in corresponding each time point, to any time point：Determine the time Deviation of the actual value of the current service data of each subservice relative to the desired value on the time point on point, the deviation For indicating the upper difference between actual value and desired value of point at the same time.The current service data of wherein each subservice exists Desired value in corresponding each time point is referred to the explanation of step 102 expected value, no longer illustrates this present embodiment.

405：The deviation for meeting the first preset condition is determined from all deviations, and according to meeting the first preset condition Deviation anomaly analysis is carried out to the current service data of main business, that is to say, that satisfaction the is selected from all deviations The deviation of one preset condition carries out anomaly analysis, for how to the current of main business to the current service data of main business Business datum carries out anomaly analysis, and the present embodiment is no longer described in detail.

Wherein the first preset condition can not be limited this present embodiment depending on practical application scene, If the first preset condition can be the deviation or the first default item of value maximum and/or value minimum in all deviations Part can be in all deviations value in the deviation of some range.

406：In the case where it is periodic data to determine current service data not, it is different that deviation is carried out to current service data Often at least one of detection and inflection point abnormality detection, to determine that current service data is abnormal with the presence or absence of deviation exception and inflection point At least one of, detailed process please refers to the related description of above-mentioned steps 104, this present embodiment is no longer described in detail.

407：If current service data is the current service data of main business, and the current service data of main business is corresponding with The current service data of at least one subservice, then it is each there are when inflection point exception, obtaining in the current service data of main business The differential data of the current service data of subservice.

Herein it should be noted is that：Step 407 and step 408 can be applied in periodic data and aperiodic Property data on, the periodic data or some main business for determining some main business aperiodicity data there are inflection point exception When obtain corresponding with main business each subservice current service data differential data, such as first-order difference data.

408：The data for meeting the second preset condition are determined from all differential datas, and according to meeting the second preset condition Data anomaly analysis is carried out to the current service data of main business, that is to say, that satisfaction the is selected from all differential datas The differential data of two preset conditions carries out anomaly analysis, for how to work as to main business to the current service data of main business Preceding business datum carries out anomaly analysis, and the present embodiment is no longer described in detail.

Wherein the second preset condition can not be limited this present embodiment depending on practical application scene, As the first preset condition can be in all differential datas value is maximum and/or the differential data or first of value minimum in advance If condition can be in all differential datas value in the differential data of some range.

It, can be by inclined from above-mentioned technical proposal it is found that for the period exception of the current service data of main business Difference carries out the period anomaly analysis extremely can pass through son for the inflection point exception of the current service data of main business The differential data of the current service data of business carries out anomaly analysis extremely to inflection point, so that it is determined that going out, the period is abnormal and inflection point is different Normal abnormal cause.

In addition in the present embodiment, determining current service data, in the presence of exception, (above-mentioned period exception, inflection point are abnormal and inclined Any one during difference is normal) when, the exception is exported, type of alarm such as may be used and export the exception, sound such as may be used At least one of type of alarm, word alarm mode and picture type of alarm export the exception, and are determining the different of exception The abnormal cause can also be exported when normal reason, in order to be checked to abnormal cause.

For each method embodiment above-mentioned, for simple description, therefore it is all expressed as a series of combination of actions, but Be those skilled in the art should understand that, the present invention is not limited by the described action sequence because according to the present invention, certain A little steps can be performed in other orders or simultaneously.Secondly, it those skilled in the art should also know that, is retouched in specification The embodiment stated belongs to preferred embodiment, and involved action and module are not necessarily essential to the invention.

Corresponding with above method embodiment, the embodiment of the present invention also provides a kind of data exception detection device, structure As shown in figure 5, may include：Acquiring unit 11, adjustment unit 12, the first abnormality detecting unit 13 and the second abnormality detecting unit 14。

Acquiring unit 11 please refers to method for the explanation of current service data and implements for obtaining current service data Example part, no longer illustrates this present embodiment.

Adjustment unit 12, in the case where it is periodic data to determine current service data, to current service data It carries out first kind adjustment at least once and Second Type adjusts at least once, the current service data after being adjusted.

Wherein determine whether current service data is the feasible pattern of periodic data and can be：It is calculated by FFT current The business datum corresponding period illustrates that current service data is periodically if it can be derived that the current service data corresponding period Data, it is periodic data otherwise to illustrate current service data not.

For periodic data, it is repeatedly adjusted by first kind adjustment and Second Type adjustment, The adjusting parameter that the adjustment of the middle first kind and Second Type adjustment use is different, and first time adjust be to current service data into Row adjustment, other numbers adjustment in addition to first time adjusts are that the data obtained to last adjustment are adjusted.Also It is to say repeatedly to be adjusted using different adjusting parameters, and it is also different to adjust corresponding data every time, specific tune for the first time Whole corresponding data are current service data, and it is the data that last adjustment obtains that other numbers, which adjust corresponding data, with jth For secondary adjustment (j is the natural number more than 1), the corresponding data of jth time adjustment adjust obtained data -1 time for jth.Below The feasible pattern of first kind adjustment and Second Type adjustment is illustrated：

If ith is adjusted to first kind adjustment, the normal distribution of this time adjustment corresponding data is obtained, by the secondary tune The data point reuse that value is more than the first predetermined threshold value in the normal distribution of whole corresponding data is that this time adjusts in corresponding data Digit, wherein this time adjustment corresponding data is current service data, the secondary tune when i is more than 1 natural number when i is equal to 1 Whole corresponding data is the data that (i-1)-th adjustment obtains；And if ith is adjusted to Second Type adjustment, passes through STL algorithms The residual component of this time adjustment corresponding data is obtained, and obtains the normal distribution of the residual component of this time adjustment corresponding data, Determine that value is respectively right more than the data of the second predetermined threshold value in the normal distribution of the residual component of this time adjustment corresponding data The time point answered is more than any value the data of the second predetermined threshold value：It is that this time adjustment corresponding data exists by the data point reuse Desired value on the data corresponding time point, wherein this time adjustment corresponding data is current service data when i is equal to 1, in i For natural number more than 1 when this time to adjust corresponding data be to adjust obtained data (i-1)-th time, and the second predetermined threshold value is different from First predetermined threshold value, illustrates and please refers to embodiment of the method, no longer illustrates this present embodiment.

First abnormality detecting unit 13 is used to carry out period abnormality detection to the current service data after adjustment and inflection point is different At least one of normal detection, to determine that the current service data after adjusting whether there is in period exception and inflection point exception extremely Few one kind.

In the present embodiment, period abnormality detection can be determined by above-mentioned STL algorithms, and the process of inflection point abnormality detection Can be：The normal distribution of the differential data (such as first-order difference data) of the current service data after adjustment is obtained, and is obtained Deviation point in the normal distribution of differential data, according to the deviation point in the normal distribution of differential data, after determining adjustment Current service data it is abnormal (illustrate and please refer to embodiment of the method part) with the presence or absence of inflection point, in this way in current business number According to the catastrophe point in the current service data after automatic positioning adjustment in the case of there are inflection point exception, so-called catastrophe point is value Variable quantity is more than the point for the preset value variable quantity of deviation point, and value variable quantity is that the corresponding data of deviation point are previous with it Value between data is poor, wherein deviation point be differential data normal distribution in the first preset multiple standard deviation except Outlier, i.e., the data being larger and smaller than in the normal distribution of differential data except the standard deviation of the first preset multiple, for First preset multiple can be depending on practical application, to this present embodiment without limiting.

Second abnormality detecting unit 14, in the case where it is periodic data to determine current service data not, to working as Preceding business datum carries out at least one of deviation abnormality detection and inflection point abnormality detection, to determine whether current service data is deposited It is at least one of abnormal in deviation exception and inflection point.

In the present embodiment, the feasible pattern of deviation abnormality detection is：The normal distribution of current service data is obtained, and The deviation point in the normal distribution of current service data is obtained, according to the deviation in the normal distribution of current service data Point determines that current service data is abnormal with the presence or absence of deviation, illustrates and please refer to embodiment of the method part.

The feasible pattern of inflection point abnormality detection is：The normal distribution of the differential data of current service data is obtained, and is obtained The deviation point in the normal distribution of differential data is taken, according to the deviation point in the normal distribution of differential data, is determined current Business datum is abnormal with the presence or absence of inflection point, and detailed process please refers to the explanation in above method embodiment, not to this present embodiment It illustrates again.

Referring to Fig. 6, it illustrates another structure of data exception detection device provided in an embodiment of the present invention, scheming Can also include on the basis of 5：Abnormal authentication unit 15, if for the current service data that current service data is main business, and The current service data of main business is corresponding with the current service data of at least one subservice, then is determining the current industry after adjusting Business data are deposited when abnormal, are tested the exception of the current service data of main business by the current service data of subservice Card, this is at least one of the period is abnormal and inflection point is abnormal extremely.

In the present embodiment, the process that abnormal authentication unit 15 verifies the exception of the current service data of main business Can be：Data deviation direction of the current service data of main business in corresponding each time point is determined, for belonging to this The current service data of any subservice of main business：Number on determining the current service data of the subservice at every point of time According to bias direction, it is data increase and data that wherein data deviation direction, which is used to indicate the data development trend at the time point, Any one in reduction.

Referring to Fig. 7, it illustrates the yet another construction of data exception detection device provided in an embodiment of the present invention, scheming Can also include on the basis of 5：Deviation determination unit 16, the first anomaly analysis unit 17, differential data acquiring unit 18 and Two anomaly analysis units 19.

Deviation determination unit 16, if for the current service data that current service data is main business, and main business Current service data is corresponding with the current service data of at least one subservice, then the current service data after determining adjustment is deposited In period exception, desired value of the current service data of each subservice in corresponding each time point is calculated, to any Time point：Determine the actual value of the current service data of the time point upper each subservice relative to the desired value on the time point Deviation.

First anomaly analysis unit 17, for determining the deviation for meeting the first preset condition from all deviations, and Anomaly analysis is carried out to the current service data of main business according to the deviation for meeting the first preset condition, that is to say, that from all The deviation for meeting the first preset condition is selected in deviation, and anomaly analysis is carried out to the current service data of main business, it is right In how to carry out anomaly analysis to the current service data of main business, the present embodiment is no longer described in detail.

Differential data acquiring unit 18, if for the current service data that current service data is main business, and main business Current service data be corresponding with the current service data of at least one subservice, then exist in the current service data of main business When inflection point exception, the differential data of the current service data of each subservice is obtained, the current business of each subservice is such as obtained The first-order difference data of data.

Second anomaly analysis unit 19, for determining the data for meeting the second preset condition from all differential datas, and Anomaly analysis is carried out to the current service data of main business according to the data for meeting the second preset condition, that is to say, that from all differences The differential data for meeting the second preset condition is selected in divided data, and anomaly analysis is carried out to the current service data of main business, For how to carry out anomaly analysis to the current service data of main business, the present embodiment is no longer described in detail.

In addition in the present embodiment, determining current service data, in the presence of exception, (above-mentioned period exception, inflection point are abnormal and inclined Any one during difference is normal) when, data exception detection device can also export the exception, and type of alarm output such as may be used The exception, the output of at least one of audible alarm mode, word alarm mode and picture type of alarm such as may be used, and this is different Often, and when determining abnormal abnormal cause the abnormal cause can also be exported, in order to be checked to abnormal cause.

It should be noted that each embodiment in this specification is described in a progressive manner, each embodiment weight Point explanation is all difference from other examples, and the same or similar parts between the embodiments can be referred to each other. For device class embodiment, since it is basically similar to the method embodiment, so fairly simple, the related place ginseng of description See the part explanation of embodiment of the method.

Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning Covering non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only that A little elements, but also include other elements that are not explicitly listed, or further include for this process, method, article or The intrinsic element of equipment.In the absence of more restrictions, the element limited by sentence "including a ...", is not arranged Except there is also other identical elements in the process, method, article or apparatus that includes the element.

The foregoing description of the disclosed embodiments enables those skilled in the art to realize or use the present invention.To this A variety of modifications of a little embodiments will be apparent for a person skilled in the art, and the general principles defined herein can Without departing from the spirit or scope of the present invention, to realize in other embodiments.Therefore, the present invention will not be limited It is formed on the embodiments shown herein, and is to fit to consistent with the principles and novel features disclosed in this article widest Range.

The above is only a preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also answered It is considered as protection scope of the present invention.

Claims

1. a kind of data exception detection method, which is characterized in that the method includes：

Obtain current service data；

In the case where it is periodic data to determine the current service data, the current service data is carried out at least once The first kind adjusts and Second Type adjusts at least once, the current service data after being adjusted, wherein the first kind It is different to adjust the adjusting parameter for adjusting and using with the Second Type, and adjustment for the first time is carried out to the current service data Adjustment, other secondary adjustment in addition to first time adjusts are that the data obtained to last adjustment are adjusted；

At least one of period abnormality detection and inflection point abnormality detection are carried out to the current service data after the adjustment, with true Current service data after the fixed adjustment is at least one of abnormal with the presence or absence of period exception and inflection point；

In the case where it is periodic data to determine the current service data not, it is different that deviation is carried out to the current service data Often at least one of detection and inflection point abnormality detection whether there is deviation exception and inflection point with the determination current service data It is at least one of abnormal.

2. according to the method described in claim 1, it is characterized in that, described determining that the current service data is periodical number In the case of, first kind adjustment and at least once Second Type adjustment package at least once are carried out to the current service data It includes：

If ith is adjusted to first kind adjustment, the normal distribution of this time adjustment corresponding data is obtained, it will the described secondary tune The data point reuse that value is more than the first predetermined threshold value in the normal distribution of whole corresponding data is that this time adjusts corresponding data Median wherein this time adjustment corresponding data is the current service data when i is equal to 1 be natural number more than 1 in i When this time adjustment corresponding data be to adjust obtained data (i-1)-th time；

If ith is adjusted to Second Type adjustment, this time is obtained by Time Series algorithm and adjusts the residual of corresponding data Difference component, and the normal distribution of the residual component of this time adjustment corresponding data is obtained, determine that this time adjustment corresponds to Value is more than data corresponding time point of the second predetermined threshold value in the normal distribution of the residual component of data, to any Value is more than the data of second predetermined threshold value：It is that this time adjusts corresponding data in data correspondence by the data point reuse Time point on desired value, wherein i be equal to 1 when this time adjustment corresponding data be the current service data, i for more than This time adjustment corresponding data is the data that (i-1)-th adjustment obtains when 1 natural number, and second predetermined threshold value is different from described First predetermined threshold value.

3. according to the method described in claim 1, it is characterized in that, the method further includes：If the current service data is The current service data of main business, and the current service data of the main business is corresponding with the current business of at least one subservice Data, then the current service data after determining the adjustment deposit when abnormal, pass through the current service data of the subservice The exception of the current service data of the main business is verified, this is at least one in period exception and inflection point exception extremely Kind.

4. according to the method described in claim 3, it is characterized in that, the current service data by the subservice is to institute State the current service data of main business exception carry out verification include：

Data deviation direction of the current service data of the main business in corresponding each time point is determined, for belonging to this The current service data of any subservice of main business：Determine the current service data of the subservice in each time point Data deviation direction, wherein the data deviation direction be used to indicate the data development trend at the time point for data increase Sum it up any one during data are reduced；

To any time point：If each subservice at the time point on data deviation direction be different from main business in the time Data deviation direction on point, it is determined that the exception of the current service data of the main business is pseudo- abnormal, and by the main business The current service data of business is determined as regular traffic data.

5. according to the method described in claim 1, it is characterized in that, the method further includes：If the current service data is The current service data of main business, and the current service data of the main business is corresponding with the current business of at least one subservice Data, then determine adjust after current service data there are when period exception, calculate the current service data of each subservice Desired value in corresponding each time point, to any time point：Determine the current business of the time point upper each subservice Deviation of the actual value of data relative to the desired value on the time point；

The deviation for meeting the first preset condition is determined from all deviations, and according to the deviation for meeting the first preset condition Anomaly analysis is carried out to the current service data of the main business；

If the current service data is the current service data of main business, and the current service data of the main business is corresponding with The current service data of at least one subservice, then it is each there are when inflection point exception, obtaining in the current service data of main business The differential data of the current service data of subservice；

The data for meeting the second preset condition are determined from all differential datas, and according to the data pair for meeting the second preset condition The current service data of the main business carries out anomaly analysis.

6. according to the method described in claim 1, it is characterized in that, described determining the current service data not and be periodically In the case of data, at least one of deviation abnormality detection and inflection point abnormality detection packet are carried out to the current service data It includes：

In the case where it is periodic data to determine the current service data not, the normal state point of the current service data is obtained Butut, and the deviation point in the normal distribution of the current service data is obtained, according to the normal state of the current service data Deviation point in distribution map determines that the current service data is abnormal with the presence or absence of deviation；

In the case where it is periodic data to determine the current service data not, the difference number of the current service data is obtained According to normal distribution, and obtain the deviation point in the normal distribution of the differential data, just according to the differential data Deviation point in state distribution map determines that the current service data is abnormal with the presence or absence of inflection point.

7. a kind of data exception detection device, which is characterized in that described device includes：

Acquiring unit, for obtaining current service data；

Adjustment unit, in the case where it is periodic data to determine the current service data, to the current business number It is adjusted according to first kind adjustment and Second Type at least once at least once is carried out, the current service data after being adjusted, Described in the first kind adjustment and the Second Type adjustment use adjusting parameter it is different, and first time adjust be to work as to described Preceding business datum is adjusted, other secondary adjustment in addition to first time adjusts are that the data obtained to last adjustment carry out Adjustment；

First abnormality detecting unit, for carrying out period abnormality detection and inflection point exception to the current service data after the adjustment At least one of detection, with the current service data after the determination adjustment with the presence or absence of in period exception and inflection point exception It is at least one；

Second abnormality detecting unit, in the case where it is periodic data to determine the current service data not, to described Current service data carries out at least one of deviation abnormality detection and inflection point abnormality detection, with the determination current service data It is at least one of abnormal with the presence or absence of deviation exception and inflection point.

8. device according to claim 7, which is characterized in that the adjustment unit, if being adjusted to specifically for ith One type adjusts, then obtains the normal distribution of this time adjustment corresponding data, this time is adjusted to the normal state point of corresponding data The data point reuse that value is more than the first predetermined threshold value in Butut is the median that this time adjusts corresponding data, wherein in i etc. This time adjustment corresponding data is the current service data when 1, this time adjustment corresponding data when i is the natural number more than 1 Obtained data are adjusted for (i-1)-th time；

9. device according to claim 8, which is characterized in that described device further includes：Abnormal authentication unit, if being used for institute State current service data be main business current service data, and the current service data of the main business be corresponding with it is at least one The current service data of subservice, then the current service data after determining the adjustment deposit when abnormal, pass through the sub- industry The current service data of business verifies the exception of the current service data of the main business, which is period exception and turns At least one of point exception.

10. device according to claim 7, which is characterized in that described device further includes：Deviation determination unit, is used for If the current service data is the current service data of main business, and the current service data of the main business is corresponding at least The current service data of one subservice is then determining that the current service data after adjusting is each there are when period exception, calculating Desired value of the current service data of subservice in corresponding each time point, to any time point：It determines on the time point Deviation of the actual value of the current service data of each subservice relative to the desired value on the time point；

First anomaly analysis unit, for determining the deviation for meeting the first preset condition from all deviations, and according to full The deviation of the first preset condition of foot carries out anomaly analysis to the current service data of the main business；

Differential data acquiring unit, if for the current service data that the current service data is main business, and the main business The current service data of business is corresponding with the current service data of at least one subservice, then is deposited in the current service data of main business In inflection point exception, the differential data of the current service data of each subservice is obtained；

Second anomaly analysis unit, for determining the data for meeting the second preset condition from all differential datas, and according to full The data of the second preset condition of foot carry out anomaly analysis to the current service data of the main business.