CN105279386B - Method and device for determining index abnormal data - Google Patents
Method and device for determining index abnormal data Download PDFInfo
- Publication number
- CN105279386B CN105279386B CN201510786205.1A CN201510786205A CN105279386B CN 105279386 B CN105279386 B CN 105279386B CN 201510786205 A CN201510786205 A CN 201510786205A CN 105279386 B CN105279386 B CN 105279386B
- Authority
- CN
- China
- Prior art keywords
- index
- evaluated
- data
- abnormal
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Telephone Function (AREA)
- Alarm Systems (AREA)
Abstract
Description
技术领域technical field
本发明涉及数据分析技术领域，尤其涉及一种指标异常数据确定的方法及装置。The invention relates to the technical field of data analysis, in particular to a method and device for determining abnormal index data.
背景技术Background technique
判断一个指标的异常数据，常用的方法是设定最高阈值和最低阈值，不在该最高阈值和最低阈值范围内的数据可以判定为异常数据。To judge the abnormal data of an indicator, the common method is to set the highest threshold and the lowest threshold, and the data that is not within the range of the highest threshold and the lowest threshold can be judged as abnormal data.
现有技术中不同指标的数据大小千差万别，这就需要对每个指标设定阈值，使得阈值判断的方式实施起来很困难；同时，即便是同一指标，在不同时间可能对应的指标设定阈值不同，进一步增加了阈值判断方式的难度，若设定阈值不合理也会使得异常数据的判断不准确。In the existing technology, the data sizes of different indicators vary widely, which requires setting thresholds for each indicator, making it difficult to implement the threshold judgment method; at the same time, even for the same indicator, the corresponding indicators may have different thresholds at different times , which further increases the difficulty of the threshold judgment method. If the threshold is set unreasonably, the judgment of abnormal data will also be inaccurate.
发明内容Contents of the invention
本发明实施例提供一种指标异常数据确定的方法及装置，用以确定当前待评估指标的异常程度。Embodiments of the present invention provide a method and device for determining index abnormal data, which are used to determine the abnormal degree of the current index to be evaluated.
本发明实施例提供的一种指标异常数据确定的方法，包括：A method for determining abnormal index data provided by an embodiment of the present invention includes:
确定待评估指标；Determine the indicators to be evaluated;
获取所述待评估指标的当前指标数据；Acquiring the current indicator data of the indicator to be evaluated;
根据所述当前指标数据相对应的统计时刻，获取所述待评估指标的历史数据；Acquiring the historical data of the indicators to be evaluated according to the statistical time corresponding to the current indicator data;
根据所述待评估指标的历史数据，依次确定所述待评估指标的平均水平和波动水平；According to the historical data of the indicators to be evaluated, sequentially determine the average level and fluctuation level of the indicators to be evaluated;
根据所述待评估指标的平均水平和所述待评估指标的波动水平，确定所述待评估指标的异常系数，所述异常系数用于指示所述待评估指标的当前指标数据是否为异常数据。An abnormality coefficient of the index to be evaluated is determined according to the average level of the index to be evaluated and the fluctuation level of the index to be evaluated, and the abnormality coefficient is used to indicate whether the current index data of the index to be evaluated is abnormal data.
较佳地，还包括：Preferably, it also includes:
统计连续m个当前指标数据的异常系数，m≥0；Count the abnormal coefficients of m continuous current indicator data, m≥0;
根据所述连续m个当前指标数据的异常系数，确定所述待评估指标的异常趋势；determining the abnormal trend of the index to be evaluated according to the abnormal coefficients of the m consecutive current index data;
根据所述异常趋势进行报警。An alarm is issued according to the abnormal trend.
较佳地，所述根据所述当前指标数据相对应的统计时刻，获取所述待评估指标的历史数据，包括：Preferably, the acquiring historical data of the indicators to be evaluated according to the statistical time corresponding to the current indicator data includes:
确定以所述当前指标数据相对应的统计时刻为中心的预设时间段，获取位于所述预设时间段内的所述待评估指标的历史数据；所述预设时间段是根据所述待评估指标的周期确定的。Determine a preset time period centered on the statistical moment corresponding to the current indicator data, and acquire historical data of the indicator to be evaluated within the preset time period; the preset time period is based on the to-be-evaluated The cycle of evaluating indicators is determined.
较佳地，根据公式(1)确定所述待评估指标的平均水平；根据公式(2)确定所述待评估指标的波动水平；Preferably, the average level of the index to be evaluated is determined according to formula (1); the fluctuation level of the index to be evaluated is determined according to formula (2);
所述公式(1)为：Described formula (1) is:
其中，μ为待评估指标的平均水平，xi为待评估指标的历史数据中的第i个指标数据，n≥0，0≤i≤n；Among them, μ is the average level of the index to be evaluated, x i is the i-th index data in the historical data of the index to be evaluated, n≥0, 0≤i≤n;
所述公式(2)为：Described formula (2) is:
其中，σ为待评估指标的波动水平，xi为待评估指标的历史数据中的第i个指标数据，μ为待评估指标的平均水平，n≥0，0≤i≤n。Among them, σ is the fluctuation level of the index to be evaluated, x i is the i-th index data in the historical data of the index to be evaluated, μ is the average level of the index to be evaluated, n≥0, 0≤i≤n.
较佳地，根据所述公式(3)确定所述待评估指标的异常系数；Preferably, the abnormality coefficient of the index to be evaluated is determined according to the formula (3);
所述公式(3)为：Described formula (3) is:
其中，m为待评估指标的异常系数，x为当前指标数据，σ为待评估指标的波动水平，μ为待评估指标的平均水平；Among them, m is the abnormal coefficient of the index to be evaluated, x is the current index data, σ is the fluctuation level of the index to be evaluated, and μ is the average level of the index to be evaluated;
m＞1或m＜-1表示待评估指标的当前指标数据为异常数据。m>1 or m<-1 indicates that the current index data of the index to be evaluated is abnormal data.
较佳地，根据所述公式(4)确定所述待评估指标的异常趋势；Preferably, the abnormal trend of the index to be evaluated is determined according to the formula (4);
所述公式(4)为：Described formula (4) is:
其中，t为待评估指标的异常趋势，c为常数，0＜α＜1，xj为第j个当前指标数据，m≥0，0≤j≤m。Among them, t is the abnormal trend of the index to be evaluated, c is a constant, 0<α<1, x j is the jth current index data, m≥0, 0≤j≤m.
相应地，本发明实施例还提供了一种确定异常数据的装置，包括：Correspondingly, an embodiment of the present invention also provides a device for determining abnormal data, including:
第一确定单元，用于确定待评估指标；a first determination unit, configured to determine the indicators to be evaluated;
第一获取单元，用于获取所述待评估指标的当前指标数据；a first obtaining unit, configured to obtain current index data of the index to be evaluated;
第二获取单元，用于根据所述当前指标数据相对应的统计时刻，获取所述待评估指标的历史数据；The second acquisition unit is configured to acquire the historical data of the indicator to be evaluated according to the statistical time corresponding to the current indicator data;
第二确定单元，用于根据所述待评估指标的历史数据，依次确定所述待评估指标的平均水平和波动水平；The second determining unit is configured to sequentially determine the average level and fluctuation level of the indicators to be evaluated according to the historical data of the indicators to be evaluated;
第三确定单元，用于根据所述待评估指标的平均水平和所述待评估指标的波动水平，确定所述待评估指标的异常系数，所述异常系数用于指示所述待评估指标的当前指标数据是否为异常数据。The third determining unit is configured to determine an abnormal coefficient of the index to be evaluated according to the average level of the index to be evaluated and the fluctuation level of the index to be evaluated, and the abnormal coefficient is used to indicate the current value of the index to be evaluated Whether the indicator data is abnormal data.
较佳地，还包括：报警单元；Preferably, it also includes: an alarm unit;
所述报警单元具体用于：The alarm unit is specifically used for:
统计连续m个指标数据的异常系数，m≥0；Count the abnormal coefficients of m consecutive index data, m≥0;
根据所述连续m个指标数据的异常系数，确定所述待评估指标的异常趋势；determining the abnormal trend of the index to be evaluated according to the abnormal coefficients of the m consecutive index data;
根据所述异常趋势进行报警。An alarm is issued according to the abnormal trend.
较佳地，所述第二获取单元具体用于：Preferably, the second acquisition unit is specifically used for:
确定以所述当前指标数据相对应的统计时刻为中心的预设时间段，获取位于所述预设时间段内的所述待评估指标的历史数据；所述预设时间段是根据所述待评估指标的周期确定的。Determine a preset time period centered on the statistical moment corresponding to the current indicator data, and acquire historical data of the indicator to be evaluated within the preset time period; the preset time period is based on the to-be-evaluated The cycle of evaluating indicators is determined.
较佳地，所述第二确定单元具体用于：Preferably, the second determining unit is specifically configured to:
根据公式(1)确定所述待评估指标的平均水平；根据公式(2)确定所述待评估指标的波动水平；Determine the average level of the index to be evaluated according to formula (1); determine the fluctuation level of the index to be evaluated according to formula (2);
所述公式(1)为：Described formula (1) is:
其中，μ为待评估指标的平均水平，xi为待评估指标的历史数据中的第i个指标数据，n≥0，0≤i≤n；Among them, μ is the average level of the index to be evaluated, x i is the i-th index data in the historical data of the index to be evaluated, n≥0, 0≤i≤n;
所述公式(2)为：Described formula (2) is:
其中，σ为待评估指标的波动水平，xi为待评估指标的历史数据中的第i个指标数据，μ为待评估指标的平均水平，n≥0，0≤i≤n。Among them, σ is the fluctuation level of the index to be evaluated, x i is the i-th index data in the historical data of the index to be evaluated, μ is the average level of the index to be evaluated, n≥0, 0≤i≤n.
较佳地，所述第三确定单元具体用于：Preferably, the third determining unit is specifically configured to:
根据所述公式(3)确定所述待评估指标的异常系数；Determine the abnormality coefficient of the index to be evaluated according to the formula (3);
所述公式(3)为：Described formula (3) is:
其中，m为待评估指标的异常系数，x为当前指标数据，σ为待评估指标的波动水平，μ为待评估指标的平均水平；Among them, m is the abnormal coefficient of the index to be evaluated, x is the current index data, σ is the fluctuation level of the index to be evaluated, and μ is the average level of the index to be evaluated;
m＞1或m＜-1表示待评估指标的当前指标数据为异常数据。m>1 or m<-1 indicates that the current index data of the index to be evaluated is abnormal data.
较佳地，所述报警单元具体用于：Preferably, the alarm unit is specifically used for:
根据所述公式(4)确定所述待评估指标的异常趋势；Determine the abnormal trend of the index to be evaluated according to the formula (4);
所述公式(4)为：Described formula (4) is:
其中，t为待评估指标的异常趋势，c为常数，0＜α＜1，xj为第j个当前指标数据，m≥0，0≤j≤m。Among them, t is the abnormal trend of the index to be evaluated, c is a constant, 0<α<1, x j is the jth current index data, m≥0, 0≤j≤m.
本发明实施例表明，通过确定待评估指标，获取待评估指标的当前指标数据，根据当前指标数据相对应的统计时刻，获取待评估指标的历史数据，根据待评估指标的历史数据，依次确定待评估指标的平均水平和波动水平，根据待评估指标的平均水平和待评估指标的波动水平，确定待评估指标的异常系数。通过按照周期提取指标的数据点到达的同时段的历史数据作为分析样本，可以消除数据的周期性影响，综合待评估指标的平均水平和波动水平动态的给出异常系数，工作人员只需要比较该待评估指标的异常系数与常数1/-1的大小，即可以知道当前待评估指标的异常程度。针对每个实时的指标数据，结合该指标数据对应的统计时刻的历史指标数据，实时评估每个指标数据的异常状况，既提高了评估异常的准确性又提高了评估的时效性；同时，结合历史数据的平均水平和波动水平，进一步提高了评估的准确性The embodiment of the present invention shows that by determining the indicators to be evaluated, the current index data of the indicators to be evaluated is obtained, the historical data of the indicators to be evaluated is obtained according to the statistical time corresponding to the current index data, and the historical data of the indicators to be evaluated are sequentially determined according to the historical data of the indicators to be evaluated. The average level and fluctuation level of the evaluation index, according to the average level of the index to be evaluated and the fluctuation level of the index to be evaluated, determine the abnormal coefficient of the index to be evaluated. By periodically extracting the historical data of the same period when the data point of the index arrives as the analysis sample, the periodic influence of the data can be eliminated, and the abnormal coefficient can be dynamically given based on the average level and fluctuation level of the index to be evaluated. The staff only need to compare the The abnormality coefficient of the index to be evaluated and the constant 1/-1 can know the degree of abnormality of the index to be evaluated. For each real-time indicator data, combined with the historical indicator data at the statistical time corresponding to the indicator data, the abnormal status of each indicator data is evaluated in real time, which not only improves the accuracy of the evaluation of abnormalities, but also improves the timeliness of the evaluation; at the same time, combined with The average level and fluctuation level of historical data further improve the accuracy of the assessment
附图说明Description of drawings
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简要介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For Those of ordinary skill in the art can also obtain other drawings based on these drawings without any creative effort.
图1为本发明实施例提供的一种指标异常数据确定的方法的流程示意图；FIG. 1 is a schematic flowchart of a method for determining abnormal index data provided by an embodiment of the present invention;
图2为现有技术提供的一种阈值设定的示意图；FIG. 2 is a schematic diagram of a threshold setting provided by the prior art;
图3为本发明实施例提供的一种阈值设定的示意图；FIG. 3 is a schematic diagram of a threshold setting provided by an embodiment of the present invention;
图4为本发明实施例提供的一种指标异常数据确定的装置的结构示意图。Fig. 4 is a schematic structural diagram of a device for determining abnormal index data provided by an embodiment of the present invention.
具体实施方式Detailed ways
为了使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请作进一步地详细描述，显然，所描述的实施例仅仅是本申请一部份实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本申请保护的范围。In order to make the purpose, technical solutions and advantages of the application clearer, the application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the application, not all of them. . Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.
图1示出了本发明实施例提供的一种指标异常数据确定的流程，该流程可以由指标异常数据确定的装置执行。Fig. 1 shows a process of determining abnormal index data provided by an embodiment of the present invention, and the process can be executed by an apparatus for determining abnormal index data.
如图1所示，该流程的具体步骤包括：As shown in Figure 1, the specific steps of the process include:
步骤101，确定待评估指标。Step 101, determining the indicators to be evaluated.
步骤102，获取所述待评估指标的当前指标数据。Step 102, acquiring current index data of the index to be evaluated.
步骤103，根据所述当前指标数据相对应的统计时刻，获取所述待评估指标的历史数据。Step 103, according to the statistical time corresponding to the current indicator data, acquire the historical data of the indicator to be evaluated.
步骤104，根据所述待评估指标的历史数据，依次确定所述待评估指标的平均水平和波动水平。Step 104, according to the historical data of the indicators to be evaluated, sequentially determine the average level and fluctuation level of the indicators to be evaluated.
步骤105，根据所述待评估指标的平均水平和所述待评估指标的波动水平，确定所述待评估指标的异常系数。Step 105: Determine the abnormality coefficient of the index to be evaluated according to the average level of the index to be evaluated and the fluctuation level of the index to be evaluated.
在步骤101中，需要在多个指标中确定待评估指标，该待评估指标可以是需要监控的指标。In step 101, an indicator to be evaluated needs to be determined among multiple indicators, and the indicator to be evaluated may be an indicator that needs to be monitored.
在步骤102中，在确定待评估指标之后，当前有一个该待评估指标的指标数据到达时，获取该待评估指标的当前指标数据，并记录该当前指标数据相对应的统计时刻。In step 102, after the indicator to be evaluated is determined, when an indicator data of the indicator to be evaluated arrives, the current indicator data of the indicator to be evaluated is obtained, and the statistical time corresponding to the current indicator data is recorded.
在步骤103中，首先确定以该当前指标数据相对应的统计时刻为中心的预设时间段，然后获取位于预设时间段内的该待评估指标的历史数据。该预设时间段时根据该待评估指标的周期确定的。该待评估指标的周期是可以根据经验设定的，该周期可以是一天、一周或一个月，实际应用时依据经验设定。In step 103, a preset time period centered at the statistical moment corresponding to the current indicator data is firstly determined, and then historical data of the indicator to be evaluated within the preset time period is acquired. The preset time period is determined according to the period of the indicator to be evaluated. The period of the indicator to be evaluated can be set according to experience, and the period can be one day, one week or one month, and it is set according to experience in practical application.
该预设时间段与该待评估指标的周期成一定比例，若待评估指标的周期是一天，则该预设时间段可以是20分钟，占该待评估指标的周期的1.4％。若待评估指标的当前指标数据相对应的统计时刻是12点，而预设时间段时20分钟，则以该12点为中心，11点50分至12点10分之间的时间段即为需要获取待评估指标的历史数据的时间段，从而获取在该预设时间段内该待评估指标的历史数据，即在11点50分至12点10分之间的所有关于该待评估指标的历史数据。The preset time period is proportional to the period of the indicator to be evaluated. If the period of the indicator to be evaluated is one day, the preset time period may be 20 minutes, accounting for 1.4% of the period of the indicator to be evaluated. If the statistical time corresponding to the current indicator data of the indicator to be evaluated is 12:00, and the preset time period is 20 minutes, the time period between 11:50 and 12:10 is centered at 12:00. The time period for obtaining the historical data of the indicator to be evaluated, so as to obtain the historical data of the indicator to be evaluated within the preset time period, that is, all the data about the indicator to be evaluated between 11:50 and 12:10 historical data.
通过按照周期提取待评估指标位于同时段的历史数据作为分析样本，可以消除数据的周期性影响。By extracting the historical data of the indicators to be evaluated in the same period according to the period as the analysis sample, the periodic influence of the data can be eliminated.
在步骤104中，当在步骤103中获取该待评估指标的历史数据后，根据公式(1)确定该待评估指标的平均水平。In step 104, after the historical data of the indicator to be evaluated is acquired in step 103, the average level of the indicator to be evaluated is determined according to formula (1).
上述公式(1)为：The above formula (1) is:
其中，μ为待评估指标的平均水平，xi为待评估指标的历史数据中的第i个指标数据，n≥0，0≤i≤n。Among them, μ is the average level of the index to be evaluated, x i is the i-th index data in the historical data of the index to be evaluated, n≥0, 0≤i≤n.
在确定出该待评估指标的平均水平后，根据该待评估指标的平均水平和该待评估指标的历史数据，以及公式(2)可以确定该待评估指标的波动水平。After the average level of the index to be evaluated is determined, the fluctuation level of the index to be evaluated can be determined according to the average level of the index to be evaluated, the historical data of the index to be evaluated, and the formula (2).
上述公式(2)为：The above formula (2) is:
其中，σ为待评估指标的波动水平，xi为待评估指标的历史数据中的第i个指标数据，μ为待评估指标的平均水平，n≥0，0≤i≤n。Among them, σ is the fluctuation level of the index to be evaluated, x i is the i-th index data in the historical data of the index to be evaluated, μ is the average level of the index to be evaluated, n≥0, 0≤i≤n.
在步骤105中，根据步骤104中确定的待评估指标的平均水平和该待评估指标的波动水平，可以确定该待评估指标的异常系数，该异常系数可以用于指示所述待评估指标的当前指标数据是否为异常数据。如异常系数大于1，则表示该待评估指标的当前指标数据异常大；异常系数小于-1，则表示该待评估指标的当前指标数据异常小，异常系数大于1或者异常系数小于-1都表示该待评估指标的当前指标数据为异常数据。工作人员可以根据异常系数判断出该待评估指标的当前指标数据是否为异常数据，若为异常数据，可以通过尾部平均的方法来削弱该异常。In step 105, according to the average level of the index to be evaluated determined in step 104 and the fluctuation level of the index to be evaluated, the abnormality coefficient of the index to be evaluated can be determined, and the abnormality coefficient can be used to indicate the current state of the index to be evaluated. Whether the indicator data is abnormal data. If the abnormal coefficient is greater than 1, it means that the current index data of the index to be evaluated is abnormally large; if the abnormal coefficient is less than -1, it means that the current index data of the index to be evaluated is abnormally small, and the abnormal coefficient is greater than 1 or the abnormal coefficient is less than -1. The current indicator data of the indicator to be evaluated is abnormal data. The staff can judge whether the current index data of the index to be evaluated is abnormal data according to the abnormal coefficient, and if it is abnormal data, the abnormal data can be weakened by the method of tail average.
上述待评估指标的异常系数可以通过公式(3)来确定。The abnormal coefficients of the above indicators to be evaluated can be determined by formula (3).
上述公式(3)为：The above formula (3) is:
其中，m为待评估指标的异常系数，x为当前指标数据，σ为待评估指标的波动水平，μ为待评估指标的平均水平。m＞1或m＜-1表示待评估指标的当前指标数据为异常数据。Among them, m is the abnormal coefficient of the index to be evaluated, x is the current index data, σ is the fluctuation level of the index to be evaluated, and μ is the average level of the index to be evaluated. m>1 or m<-1 indicates that the current index data of the index to be evaluated is abnormal data.
本发明实施例综合待评估指标的平均水平和波动水平动态的给出异常系数，工作人员只需要比较该待评估指标的异常系数与常数1/-1的大小，即可以知道当前待评估指标的异常程度。The embodiment of the present invention synthesizes the average level and fluctuation level of the index to be evaluated to dynamically give the abnormal coefficient, and the staff only need to compare the abnormal coefficient of the index to be evaluated with the constant 1/-1 to know the current value of the index to be evaluated abnormal degree.
当确定出连续m个当前指标数据的异常系数后，统计该连续m个当前指标数据的异常系数。根据该连续m个当前指标数据的异常系数，确定出该待评估指标的异常趋势。然后根据该异常趋势进行报警，当该异常趋势超过报警阈值后，即可以进行报警。本发明实施例的可以提供短信、Hipchat等报警方式，也可以是在网页中实时渲染指标的异常信息，如异常数据点会被标红。After the abnormal coefficients of the m consecutive current index data are determined, the abnormal coefficients of the m consecutive current index data are counted. According to the abnormal coefficients of the m continuous current index data, the abnormal trend of the index to be evaluated is determined. Then, an alarm is issued according to the abnormal trend, and when the abnormal trend exceeds the alarm threshold, an alarm can be issued. The embodiment of the present invention can provide SMS, Hipchat and other alarm methods, and can also render abnormal information of indicators in real time on the web page, such as abnormal data points will be marked red.
上述待评估指标的异常趋势可以根据公式(4)确定。The abnormal trend of the above indicators to be evaluated can be determined according to formula (4).
该公式(4)为：The formula (4) is:
其中，t为待评估指标的异常趋势，c为常数，0＜α＜1，xj为第j个当前指标数据，m≥0，0≤j≤m。Among them, t is the abnormal trend of the index to be evaluated, c is a constant, 0<α<1, x j is the jth current index data, m≥0, 0≤j≤m.
从公式(4)中可以看出，α越大时效性也会越好，越能反映最近数据点的异常趋势。It can be seen from formula (4) that the larger α is, the better the timeliness will be, and the more it can reflect the abnormal trend of recent data points.
本发明实施例通过对订餐的web主站，后端RPC服务等接口的调用量、调用时长等指标进行监控，对接口超时、刷帖等事故检测起到了较好的报警作用。The embodiment of the present invention monitors indicators such as the number of calls and call durations of the main web site for ordering meals, the back-end RPC service and other interfaces, and has a good alarm effect on the detection of accidents such as interface timeout and brushing.
现有技术中判断一个指标的异常数据，常用的方法是设定最高阈值和最低阈值，如图2所示，上下两条直线为设定的最高阈值和最低阈值，不在该最高阈值和最低阈值范围内的指标数据可以判定为异常数据。如图3所示，本发明实施例可以提供动态的指标数据的阈值，从而不必对每个指标都设定阈值。In the prior art, the common method for judging the abnormal data of an indicator is to set the highest threshold and the lowest threshold, as shown in Figure 2, the upper and lower two straight lines are the set highest threshold and the lowest threshold, not the highest threshold and the lowest threshold Indicator data within the range can be judged as abnormal data. As shown in FIG. 3 , the embodiment of the present invention can provide a dynamic threshold of index data, so that it is not necessary to set a threshold for each index.
本发明实施例表明，通过确定待评估指标，获取待评估指标的当前指标数据，根据当前指标数据相对应的统计时刻，获取待评估指标的历史数据，根据待评估指标的历史数据，依次确定待评估指标的平均水平和波动水平，根据待评估指标的平均水平和待评估指标的波动水平，确定待评估指标的异常系数。通过按照周期提取指标的数据点到达的同时段的历史数据作为分析样本，可以消除数据的周期性影响，根据正态分布原理，综合待评估指标的平均水平和波动水平动态的给出异常系数，只需要比较该待评估指标的异常系数与常数1/-1的大小，即可以知道当前待评估指标的异常程度。The embodiment of the present invention shows that by determining the indicators to be evaluated, the current index data of the indicators to be evaluated is obtained, the historical data of the indicators to be evaluated is obtained according to the statistical time corresponding to the current index data, and the historical data of the indicators to be evaluated are sequentially determined according to the historical data of the indicators to be evaluated. The average level and fluctuation level of the evaluation index, according to the average level of the index to be evaluated and the fluctuation level of the index to be evaluated, determine the abnormal coefficient of the index to be evaluated. By periodically extracting the historical data of the same period when the data point of the index arrives as the analysis sample, the periodic influence of the data can be eliminated. According to the principle of normal distribution, the abnormal coefficient is dynamically given based on the average level and fluctuation level of the index to be evaluated. It is only necessary to compare the abnormality coefficient of the index to be evaluated with the constant 1/-1 to know the degree of abnormality of the index to be evaluated.
基于相同的技术构思，图4示出了本发明实施例提供的一种指标异常数据确定的装置的结构，该装置可以执行指标异常数据确定的流程。Based on the same technical concept, FIG. 4 shows the structure of an apparatus for determining abnormal index data provided by an embodiment of the present invention, and the apparatus can execute a process for determining abnormal index data.
如图4所示，该装置具体包括：As shown in Figure 4, the device specifically includes:
第一确定单元401，用于确定待评估指标；The first determination unit 401 is configured to determine the indicators to be evaluated;
第一获取单元402，用于获取所述待评估指标的当前指标数据；A first obtaining unit 402, configured to obtain current index data of the index to be evaluated;
第二获取单元403，用于根据所述当前指标数据相对应的统计时刻，获取所述待评估指标的历史数据；The second acquiring unit 403 is configured to acquire the historical data of the indicator to be evaluated according to the statistical time corresponding to the current indicator data;
第二确定单元404，用于根据所述待评估指标的历史数据，依次确定所述待评估指标的平均水平和波动水平；The second determining unit 404 is configured to sequentially determine the average level and fluctuation level of the indicators to be evaluated according to the historical data of the indicators to be evaluated;
第三确定单元405，用于根据所述待评估指标的平均水平和所述待评估指标的波动水平，确定所述待评估指标的异常系数，所述异常系数用于指示所述待评估指标的当前指标数据是否为异常数据。The third determination unit 405 is configured to determine an abnormal coefficient of the index to be evaluated according to the average level of the index to be evaluated and the fluctuation level of the index to be evaluated, and the abnormal coefficient is used to indicate the index to be evaluated Whether the current indicator data is abnormal data.
优选地，还包括：报警单元；Preferably, it also includes: an alarm unit;
所述报警单元具体用于：The alarm unit is specifically used for:
统计连续m个指标数据的异常系数，m≥0；Count the abnormal coefficients of m consecutive index data, m≥0;
根据所述连续m个指标数据的异常系数，确定所述待评估指标的异常趋势；determining the abnormal trend of the index to be evaluated according to the abnormal coefficients of the m consecutive index data;
根据所述异常趋势进行报警。An alarm is issued according to the abnormal trend.
优选地，所述第二获取单元403具体用于：Preferably, the second acquiring unit 403 is specifically configured to:
确定以所述当前指标数据相对应的统计时刻为中心的预设时间段，获取位于所述预设时间段内的所述待评估指标的历史数据；所述预设时间段是根据所述待评估指标的周期确定的。Determine a preset time period centered on the statistical moment corresponding to the current indicator data, and acquire historical data of the indicator to be evaluated within the preset time period; the preset time period is based on the to-be-evaluated The cycle of evaluating indicators is determined.
优选地，所述第二确定单元404具体用于：Preferably, the second determining unit 404 is specifically configured to:
根据公式(1)确定所述待评估指标的平均水平；根据公式(2)确定所述待评估指标的波动水平；Determine the average level of the index to be evaluated according to formula (1); determine the fluctuation level of the index to be evaluated according to formula (2);
所述公式(1)为：Described formula (1) is:
其中，μ为待评估指标的平均水平，xi为待评估指标的历史数据中的第i个指标数据，n≥0，0≤i≤n；Among them, μ is the average level of the index to be evaluated, x i is the i-th index data in the historical data of the index to be evaluated, n≥0, 0≤i≤n;
所述公式(2)为：Described formula (2) is:
其中，σ为待评估指标的波动水平，xi为待评估指标的历史数据中的第i个指标数据，μ为待评估指标的平均水平，n≥0，0≤i≤n。Among them, σ is the fluctuation level of the index to be evaluated, x i is the i-th index data in the historical data of the index to be evaluated, μ is the average level of the index to be evaluated, n≥0, 0≤i≤n.
优选地，所述第三确定单元405具体用于：Preferably, the third determining unit 405 is specifically configured to:
根据所述公式(3)确定所述待评估指标的异常系数；Determine the abnormality coefficient of the index to be evaluated according to the formula (3);
所述公式(3)为：Described formula (3) is:
其中，m为待评估指标的异常系数，x为当前指标数据，σ为待评估指标的波动水平，μ为待评估指标的平均水平；Among them, m is the abnormal coefficient of the index to be evaluated, x is the current index data, σ is the fluctuation level of the index to be evaluated, and μ is the average level of the index to be evaluated;
m＞1或m＜-1表示待评估指标的当前指标数据为异常数据。m>1 or m<-1 indicates that the current index data of the index to be evaluated is abnormal data.
优选地，所述报警单元具体用于：Preferably, the alarm unit is specifically used for:
根据所述公式(4)确定所述待评估指标的异常趋势；Determine the abnormal trend of the index to be evaluated according to the formula (4);
所述公式(4)为：Described formula (4) is:
其中，t为待评估指标的异常趋势，c为常数，0＜α＜1，xj为第j个当前指标数据，m≥0，0≤j≤m。Among them, t is the abnormal trend of the index to be evaluated, c is a constant, 0<α<1, x j is the jth current index data, m≥0, 0≤j≤m.
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present application is described with reference to flowcharts and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.
尽管已描述了本申请的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本申请范围的所有变更和修改。While preferred embodiments of the present application have been described, additional changes and modifications to these embodiments can be made by those skilled in the art once the basic inventive concept is appreciated. Therefore, the appended claims are intended to be construed to cover the preferred embodiment and all changes and modifications which fall within the scope of the application.
显然，本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请的精神和范围。这样，倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。Obviously, those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. In this way, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510786205.1A CN105279386B (en) | 2015-11-16 | 2015-11-16 | Method and device for determining index abnormal data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510786205.1A CN105279386B (en) | 2015-11-16 | 2015-11-16 | Method and device for determining index abnormal data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105279386A CN105279386A (en) | 2016-01-27 |
| CN105279386B true CN105279386B (en) | 2019-08-16 |
Family
ID=55148389
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510786205.1A Active CN105279386B (en) | 2015-11-16 | 2015-11-16 | Method and device for determining index abnormal data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105279386B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105956935B (en) * | 2016-05-06 | 2017-08-25 | 泉州亿兴电力有限公司 | A kind of power distribution network transformer threshold value update method and system |
| CN106293976A (en) * | 2016-08-15 | 2017-01-04 | 东软集团股份有限公司 | Application performance Risk Forecast Method, device and system |
| CN107888435B (en) * | 2016-09-29 | 2020-11-17 | 华为技术有限公司 | Threshold setting method and device for alarm monitoring |
| CN107202604B (en) * | 2017-03-02 | 2019-09-03 | 湖南工业大学 | An alarm processing method and system |
| CN108334417B (en) * | 2018-01-26 | 2021-03-02 | 创新先进技术有限公司 | Method and device for determining data exception |
| CN109740099A (en) * | 2018-12-30 | 2019-05-10 | 北京奇艺世纪科技有限公司 | A kind of data exception localization method and device |
| CN113420816B (en) * | 2021-06-24 | 2024-09-06 | 北京市生态环境监测中心 | Data outlier determining method for full-spectrum water quality monitoring equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013254337A (en) * | 2012-06-06 | 2013-12-19 | Ntt Data Corp | Virtualization device, virtualization control method and virtualization device control program |
| CN103823980A (en) * | 2014-02-26 | 2014-05-28 | 浙江水利水电学院 | Method for detecting abnormal dam monitoring data based on entropy theory |
| CN104462794A (en) * | 2014-11-26 | 2015-03-25 | 北京金水永利科技有限公司 | Algorithm for finding abnormal data of environmental monitoring based on comparative statistic analysis |
| CN104536868A (en) * | 2014-11-26 | 2015-04-22 | 北京广通信达科技有限公司 | Dynamic threshold analysis method for operation index of IT system |
| CN105046075A (en) * | 2015-07-10 | 2015-11-11 | 中国农业大学 | Analyzing-processing method and device for dam quality monitoring data |
-
2015
- 2015-11-16 CN CN201510786205.1A patent/CN105279386B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013254337A (en) * | 2012-06-06 | 2013-12-19 | Ntt Data Corp | Virtualization device, virtualization control method and virtualization device control program |
| CN103823980A (en) * | 2014-02-26 | 2014-05-28 | 浙江水利水电学院 | Method for detecting abnormal dam monitoring data based on entropy theory |
| CN104462794A (en) * | 2014-11-26 | 2015-03-25 | 北京金水永利科技有限公司 | Algorithm for finding abnormal data of environmental monitoring based on comparative statistic analysis |
| CN104536868A (en) * | 2014-11-26 | 2015-04-22 | 北京广通信达科技有限公司 | Dynamic threshold analysis method for operation index of IT system |
| CN105046075A (en) * | 2015-07-10 | 2015-11-11 | 中国农业大学 | Analyzing-processing method and device for dam quality monitoring data |
Non-Patent Citations (1)
| Title |
|---|
| 数据流中异常模式的提取与趋势监测;宋国杰 等;《计算机研究与发展》;20041031;第41卷(第10期);第1754-1759页 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105279386A (en) | 2016-01-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105279386B (en) | Method and device for determining index abnormal data | |
| CN112735094B (en) | Geological disaster prediction method and device based on machine learning and electronic equipment | |
| CN104375912B (en) | The measuring method and device of mobile terminal interim card | |
| CN110894038B (en) | Method and device for predicting running state of elevator door system | |
| CN107301570B (en) | Traffic prediction method, abnormal traffic detection device and electronic equipment | |
| CN108763040B (en) | Data acquisition method and data acquisition device | |
| CN110795324B (en) | Data processing method and device | |
| CN110471821A (en) | Abnormal alteration detection method, server and computer readable storage medium | |
| CN107276779B (en) | Monitoring method, system and equipment | |
| CA3072890C (en) | Method and system for identifying location of a parked vehicle | |
| WO2002095633A3 (en) | Method and apparatus for determining the health of a component using condition indicators | |
| CN105069701A (en) | Monte Carlo method based risk evaluation method for power transmission system | |
| CN112527601A (en) | Monitoring early warning method and device | |
| CN103217590B (en) | A kind of automatic method for obtaining atmospheric electric field thunder and lightning early-warning characteristic parameter threshold | |
| JP2017522951A5 (en) | ||
| JP6131976B2 (en) | Personnel management system, information analysis apparatus, personnel management method, and personnel management program | |
| CN110988624A (en) | Detection method and system for intermittent partial discharge signal | |
| CN108535033A (en) | Virtual-sensor and its health monitoring intelligent decision system | |
| CN107247871A (en) | Item detection time checking method for early warning and device | |
| CN203837658U (en) | A visualized bridge health detection system | |
| CN107843811A (en) | A method and system for analyzing online monitoring data of power grid equipment | |
| CN106570073B (en) | Surface water quality data rough error screening method and device | |
| CN114938339B (en) | Data processing method and related device | |
| JP2014095586A (en) | Earthquake identification apparatus, and earthquake identification system and earthquake identification method using the same | |
| CN112488465B (en) | Method for estimating total fission times of triuranium octoxide in critical accident emergency |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |