CN112395120A - Abnormal point detection method, device, equipment and storage medium - Google Patents
Abnormal point detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112395120A CN112395120A CN201910749953.0A CN201910749953A CN112395120A CN 112395120 A CN112395120 A CN 112395120A CN 201910749953 A CN201910749953 A CN 201910749953A CN 112395120 A CN112395120 A CN 112395120A
- Authority
- CN
- China
- Prior art keywords
- data
- determining
- time sequence
- residual data
- measurement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 111
- 238000001514 detection method Methods 0.000 title claims abstract description 36
- 238000005259 measurement Methods 0.000 claims abstract description 103
- 238000000034 method Methods 0.000 claims abstract description 68
- 238000012545 processing Methods 0.000 claims description 18
- 238000006243 chemical reaction Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 9
- 238000013450 outlier detection Methods 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000004140 cleaning Methods 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 3
- 230000001932 seasonal effect Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 description 17
- 238000010586 diagram Methods 0.000 description 14
- 230000008859 change Effects 0.000 description 5
- 238000012937 correction Methods 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0751—Error or fault detection not based on redundancy
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
The embodiment of the invention provides an abnormal point detection method, an abnormal point detection device, abnormal point detection equipment and a storage medium, wherein the method comprises the following steps: firstly, acquiring measurement data of a service index time sequence of a monitored system; secondly, determining a mean baseline for the service index time sequence, and calculating measurement residual data of the measurement data and the mean baseline; and finally, determining abnormal points of the monitored system on the time sequence according to the predicted residual data and the measured residual data. Therefore, the false alarm rate is reduced under the condition of detecting the abnormal condition, and the system stability is maintained.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a method, an apparatus, a device, and a storage medium for detecting an abnormal point.
Background
With the enhancement of data dependency of service operation, the demand for timely grasping data dynamics and quickly and accurately acquiring data interpretation is increasing. The correct data indexes can help us to accurately grasp the user trend and the business situation of the website. Therefore, the situation that data are abnormal is found and solved in time, and the problem becomes the concern in business operation work.
Currently, the main way to detect the abnormal condition in the data is to compare the measured value of the data point with an index abnormal threshold value preset manually to determine whether the data point is abnormal. However, in this method, when the system has conventional data jitter or has timing increase or decrease due to service characteristics, the artificially set abnormal threshold is not flexible enough, and whether a data point is abnormal or not is determined only according to the abnormal threshold, which may cause a problem of high false alarm rate.
Disclosure of Invention
One or more embodiments of the present invention describe an abnormal point detection method, apparatus, device and storage medium to solve the problem of high false alarm rate in case of detecting abnormal situations.
According to a first aspect, there is provided an abnormal point detecting method, which may include:
acquiring measurement data of a service index time sequence of a monitored system;
determining a mean baseline for the service index time sequence, and calculating measurement residual data of the measurement data and the mean baseline;
residual prediction is carried out on the measured residual data to generate predicted residual data;
and determining abnormal points of the monitored system on the time sequence according to the prediction residual data and the measurement residual data.
According to a second aspect, there is provided an abnormal point detecting apparatus, which may include:
the acquisition module is used for acquiring the measurement data of the service index time sequence of the monitored system;
the processing module is used for determining a mean baseline for the service index time sequence and calculating measurement residual data of the measurement data and the mean baseline;
the prediction module is used for carrying out residual prediction on the measured residual data to generate predicted residual data;
and the generating module is used for determining abnormal points of the monitored system on the time sequence according to the prediction residual data and the measurement residual data.
According to a third aspect, there is provided a computing device comprising a transceiver for transceiving data, at least one processor for executing a program of the memory to control a server to implement the method of any of the first or second aspects, and a memory for storing computer program instructions.
According to a fourth aspect, there is provided a computer readable storage medium having stored thereon a computer program which, if executed in a computer, causes the computer to perform the method of any of the first or second aspects.
By using the scheme of the embodiment of the invention, the mean baseline determined by historical data is subtracted on the basis of the measured value, so that the abnormal false alarm caused by the increase and decrease of the real-time index period and the conventional data jitter of the system is eliminated. Then, the method of judging abnormal points of the monitored system on the time sequence by the measurement residual data determined by the measurement data and the mean baseline and the prediction residual data reduces or eliminates the possibility of false alarm or missing report in the abnormal point detection process so as to keep the system stable.
Drawings
The present invention will be better understood from the following description of specific embodiments thereof taken in conjunction with the accompanying drawings, in which like or similar reference characters designate like or similar features.
FIG. 1 illustrates a schematic structural diagram of a data center according to one embodiment;
FIG. 2 is a diagram illustrating an application scenario of the outlier detection method according to an embodiment;
FIG. 3 is a schematic diagram illustrating another application scenario of the outlier detection method according to an embodiment;
FIG. 4 illustrates a flow diagram of an outlier detection method according to one embodiment;
FIG. 5 shows a flow diagram of a method of anomaly detection according to another embodiment;
FIG. 6 illustrates a flow diagram for an anomaly detection system implementing anomaly detection for a data center, according to one embodiment;
FIG. 7 shows a block diagram of an outlier detection apparatus according to one embodiment;
FIG. 8 illustrates a schematic structural diagram of a computing device, according to one embodiment.
Detailed Description
Features and exemplary embodiments of various aspects of the present invention will be described in detail below, and in order to make objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention. It will be apparent to one skilled in the art that the present invention may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the present invention.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any such measured relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
In order to solve the problem in the prior art, embodiments of the present invention provide a method, an apparatus, a device, and a storage medium for detecting an abnormal point, which are described in the following.
FIG. 1 shows a schematic structural diagram of a data center according to one embodiment.
As shown in fig. 1, the data center includes a plurality of monitored systems such as a monitored system 1, a monitored system 2, and a monitored system 3, and an abnormal point detecting system 4.
The abnormal point detection system 4 determines the operation status of the data center by detecting the service index of the monitored system. The service index may come from a monitored system or other application program, or may come from a hardware device of a data center or other sources.
For example, in a scenario where a data center manages multiple customer service robots, as shown in fig. 2, the monitored system may be customer service robots under different service lines, and the service index of the monitored system may be an index system for measuring the operation performance and the question-and-answer performance of the robot. Further, monitored system 1 may be a customer service robot serving a customer, monitored system 2 may be a customer service robot serving a merchant, and monitored system 3 may be a customer service robot serving a business. The service index can be the visit amount, the man-machine conversation turn, the man-machine conversation amount, the manual conversion rate, the single-turn conversation solution rate and the like.
The following is a detailed description of the transfer rate of the customer service robot serving the customer, using two application scenarios as an example:
in one possible application scenario, as shown in fig. 2, the anomaly detection system 4 obtains measurement data of a service index time series of the labor conversion rate of a customer service robot serving a customer; then, determining a mean baseline for the service index time sequence, and calculating measurement residual data of the measurement data and the mean baseline; secondly, residual prediction is carried out on the measured residual data to generate predicted residual data; then, according to the predicted residual data and the measured residual data, the labor conversion rate of the customer service robot serving the consumer at 23:00 pm in Beijing hours is determined to be 89%, and the labor conversion rate data corresponding to the time point is determined to be an abnormal point. When the abnormal point information 'the labor rate of the customer service robot serving the consumer is 89% (23: 00 pm in beijing time') is detected, actively pushing the abnormal point information to the user through the data robot equipment, such as: "little man, you are good, the conversion rate of customer service robot serving customer at 23:00 PM of Beijing hours is 89%, unlike usual, please pay attention to! "
In another possible application scenario, as shown in fig. 3, the anomaly detection system 4 obtains measurement data of a service index time series of the labor conversion rate of the customer service robot serving the customer; then, determining a mean baseline for the service index time sequence, and calculating measurement residual data of the measurement data and the mean baseline; secondly, residual prediction is carried out on the measured residual data to generate predicted residual data; then, according to the predicted residual data and the measured residual data, the labor conversion rate of the customer service robot serving the consumer at 23:00 pm in Beijing hours is determined to be 89%, and the labor conversion rate data corresponding to the time point is used as an abnormal point.
Under the condition that the data robot equipment receives query information for inquiring whether index data is abnormal or not at a certain time or a certain moment by natural voice, abnormal point information corresponding to the query information is obtained, and the abnormal point is pushed to a user.
For example: the data robot receives user voice query information "data robot, you good, please help me query if an exception point appears at yesternight? Or ' data robot, you are good, please help me inquire whether the data center is abnormal at 23:00 yesternight ' or not ', acquire whether the index data is abnormal at a certain time or a certain moment in the inquiry information or not, and under the condition of acquiring the abnormal point information, push the abnormal point to the user, such as: "little man, you are good, the conversion rate of customer service robot serving customer at 23:00 PM of Beijing hours is 89%, unlike usual, please pay attention to! "
Therefore, in the embodiment of the invention, the abnormal point detection method executed on the abnormal point detection system 4 is improved and optimized, so that the monitoring operation is more accurate and sensitive, and the problem of high false alarm rate caused by the fact that the abnormal threshold is not flexible due to manual setting is solved. The data center is only an application object of the abnormal point detection method provided by the embodiment of the invention; in fact, the abnormal point detection method provided in the embodiment of the present invention may be applied to other electronic devices, structures, or systems besides the data center, and the embodiment of the present invention is not limited thereto.
In addition, the method provided by the embodiment of the invention can also push the abnormal point detection service related to the monitored system according to different attributes of the monitored system. For example: the monitored system is a customer service robot of a merchant, and can push abnormal point detection service corresponding to commodity visit quantity and/or abnormal point detection service corresponding to commodity consultation quantity to the merchant so that the merchant can select different types of abnormal point detection service according to push contents.
Based on the above scenario, the embodiment of the invention provides an abnormal point detection method. Two abnormal point detection methods provided by the embodiment of the present invention are described in detail below with reference to fig. 4 to 5.
Example 1:
FIG. 4 illustrates a flow diagram of an outlier detection method according to one embodiment.
As shown in fig. 4, the method flow includes steps 410 to 440: firstly, step 410, obtaining measurement data of a service index time sequence of a monitored system; secondly, step 420, determining a mean baseline for the service index time series, and calculating measurement residual data of the measurement data and the mean baseline; then, step 430, residual prediction is performed on the measured residual data to generate predicted residual data; then, in step 440, outliers of the monitored system in the time series are determined based on the predicted residual data and the measured residual data.
The above steps are described in detail below:
first, step 410 is involved: and acquiring the measurement data of the service index time sequence of the monitored system.
In one example, the traffic indicator time series includes a plurality of discrete points; the plurality of discrete points includes a first discrete point, and the measurement data of the first discrete point is used for representing the statistical data in a first time window to which the first discrete point belongs.
For example: the first discrete point corresponds to data of 10 am on 4 days of 5 months, and actually the measurement data of the first discrete point is 9 am: 55-10: 00 mean data for this time period, wherein the first time window is 9: 55-10: 00 for 5 minutes.
Secondly, step 420 is involved: and determining a mean baseline for the service index time series, and calculating measurement residual data of the measurement data and the mean baseline.
Wherein the step includes two processes, the first is a process of determining a mean baseline, and the second is a process of calculating measured residual data.
For the first process, in one example, historical subdata of a plurality of discrete points is obtained from historical data of the service index according to a time period; the measurement data of a preset number of discrete points before each discrete point are respectively selected for mean value calculation, and therefore a mean value baseline for the first discrete point is determined.
For example: assuming that the first discrete point is measurement data of 5 month, 4 days, 10 am, and the historical data of the service index is divided according to a period of 24 hours, and historical sub-data of 3 days before 5 month, 4 days, that is, historical sub-data from 5 month, 1 day to 5 month, 3 days, is determined, and each of the 3 pieces of historical sub-data includes historical data of a plurality of discrete points. The measurement data of the same time as that of the 3 historical subdata at 10 am on day 4 of month 5, namely the data of 10 am on day 3 of month 5, the data of 10 am on day 2 of month 5 and the data of 10 am on day 1 of month 5 are respectively selected. The measurement data of 75 discrete points before 10 am of 3 months of 5 months, 10 am of 2 months of 5 months and 10 am of 1 month of 5 months are respectively selected, so that the mean value data of each day of 3 days is determined, and then the mean value baseline aiming at 10 am of 3 days of 5 months is determined according to the mean value data of each day. It should be reminded that the example section only exemplifies the measurement data three days before the first discrete point, and actually includes, but is not limited to, statistics only for three days.
Here, the mean baseline in the embodiment of the present invention is adjusted at any time with the update of the data, and specifically, the mean baseline for the first discrete point may be updated according to a periodic change rule of the historical data.
Therefore, the mean baseline in the embodiment of the invention is a dynamic mean baseline, which is different from a static baseline in the related art, so that the mean baseline can sense changes caused by factors such as people, materials, machines and processing processes, and is suitable for sudden people or time-varying workflows. The dynamic mean baseline can identify the abnormal condition with non-constant fluctuation amplitude (for example, the condition that the system has conventional data jitter or the condition that the system is increased or decreased regularly due to the service characteristics), so that the dynamic mean baseline is well adapted to the data change, and the accuracy of determining the measurement residual data can be improved, so as to avoid causing false alarm or false missing report of abnormal points later.
For the second procedure, in one example, the measurement residual data is determined according to the difference between the measurement data of the service indicator time series obtained in step 410 and the mean baseline in the first procedure in step 420.
Thus, step 420 determines the measurement residual data by subtracting the average value of three days on the basis of the measured value, so as to eliminate the abnormal false alarm caused by the increase and decrease of the real-time index period and the conventional data jitter of the system.
Step 430: and performing residual prediction on the measured residual data to generate predicted residual data.
In one example, the measured residual data is residual predicted using at least one of: exponential smoothing model, autoregressive model, seasonal difference autoregressive moving average model. Wherein, the measured residual data of each discrete point in the service index time sequence has a predicted residual data corresponding to it.
Step 440: and determining abnormal points of the monitored system on the time sequence according to the prediction residual data and the measurement residual data.
In one example, difference data of the measured residual data and the predicted residual data is calculated; determining a confidence interval according to the difference data; and if the measured data is not in the confidence interval, determining that the discrete point corresponding to the measured data on the time sequence is an abnormal point.
Further, the determining the confidence interval according to the difference data may specifically include: and calculating the mean value and the standard deviation corresponding to the difference data, and determining the confidence interval by using an N-sigma judgment method.
In the embodiment of the invention, the dynamic baseline average value is calculated by using historical measurement data, manual judgment and setting of an empirical threshold are not needed, and the method is better suitable for data change. And subtracting the average value of nearly three days on the basis of the measured value to determine the measured residual data, so as to eliminate the abnormal false alarm caused by the increase and decrease of the real-time index period and the conventional data jitter of the system. Then, the measured residual data is predicted through a residual prediction model, and a confidence interval is calculated, so that the real-time response to the burst early warning can be ensured, and the lower false alarm rate can be kept.
Example 2:
based on the method shown in embodiment 1, an embodiment of the present invention provides another abnormal point detection method, and the difference between the method and embodiment 1 is mainly in step 440, that is, in addition to determining the abnormal point in the service index time sequence by using the prediction residual data and the measurement residual data, the abnormal point in the service index time sequence may be determined by using a ring ratio threshold, a same ratio threshold, the prediction residual data and the measurement residual data. Another anomaly detection method is described in detail below with reference to fig. 5.
Fig. 5 shows a flowchart of an outlier detection method according to another embodiment.
As shown in fig. 5, the method flow includes steps 510 to 540: firstly, step 510, judging whether the measurement data of the service index time sequence belongs to an incremental window type index; secondly, step 520, respectively calculating a ring ratio time sequence and a same ratio time sequence corresponding to the measured data; then, step 530, determining a ring ratio threshold and a same ratio threshold according to the ring ratio time sequence and the same ratio time sequence; then, in step 540, an abnormal point of the monitored system on the time series is determined according to the ring ratio threshold, the same ratio threshold, the prediction residual data and the measurement residual data.
The above steps are described in detail below:
first, step 510 is involved: and judging whether the measurement data of the service index time sequence belongs to the incremental window type index.
In one example, if the measurement data of the service indicator time series belongs to the incremental window type indicator, then the step 520 is continued. If the measured data of the service index time series does not belong to the incremental window type index, the measured data of the service index time series is converted into the incremental window type index, and then step 520 is executed.
The incremental window type index refers to accumulated data, and whether the measured data belongs to the incremental window type index or not can be judged in the following mode:
a difference between the measured data in the first time window and the measured data in the second time window is calculated. Determining that the measurement data belongs to the incremental window type index under the condition that the difference is larger than a preset threshold (if the preset threshold is zero); otherwise, when the difference is smaller than the preset threshold (for example, the preset threshold is zero), it is determined that the measured data does not belong to the incremental window type index.
And under the condition that the measured data does not belong to the incremental window type index, detecting the data source of the measured data, adjusting the data source to meet the condition of the window type index, and converting the data source into the incremental window type index again.
Thus, in this example, the traffic anomaly is actually reflected by a data anomaly, and the possibility of error in the data source itself is eliminated by step 510, so as to improve the ring ratio threshold and the same ratio threshold for obtaining higher accuracy.
In another example, before step 510, the method may further include: dividing the measurement data of the service index time sequence according to the data type;
and if the measured data are determined to be divided into proportion index data or magnitude index data according to the data type, the measured data are incremental window type indexes. The proportion index data refers to the fact that the measurement data are presented in the form of fractional values, and the magnitude index data refers to the fact that the measurement data are presented in the form of indexes.
Next, in step 520, a ring ratio time sequence and a same ratio time sequence corresponding to the measured data are calculated respectively.
In one example, a ring ratio value and a same ratio value of the measurement data are respectively calculated; and determining a ring ratio time sequence corresponding to the measured data according to the ring ratio value, and determining a same ratio time sequence corresponding to the measured data according to the same ratio value.
Next, in step 530, a ring ratio threshold and a comparand threshold are determined based on the ring ratio timing and the comparand timing.
In one example, the loop ratio time sequence is processed by a preset loop ratio threshold rule to determine a loop ratio threshold; and carrying out preset comparison threshold rule processing on the comparison time sequence to determine a comparison threshold.
In another example, before step 530, the method may further include: and performing data cleaning on the ring ratio time sequence and the same ratio time sequence, wherein the data cleaning is used for reexamining and checking the measured data, aims to delete repeated data, correct existing errors and provide the consistency of the measured data so as to improve the accuracy of the ring ratio threshold and the same ratio threshold.
Then, in step 540, an abnormal point of the monitored system on the time series is determined according to the ring ratio threshold, the same ratio threshold, the prediction residual data and the measurement residual data.
In one example, a confidence interval is determined from the prediction residual data and the measurement residual data; if the measured data is not in the confidence interval, taking the corresponding discrete point of the measured data on the time sequence as an assumed abnormal point; the assumed outlier is corrected using the ring ratio threshold and the isometry threshold to determine whether the assumed outlier is an outlier.
It should be noted that the manner of determining the prediction residual data and the measurement residual data is the same as that of steps 410 to 430 in embodiment 1, and is not described herein again.
Further, the hypothesis outliers are corrected by the ring ratio threshold and the equivalence threshold. Under the condition that the assumed abnormal point is still determined to be abnormal after correction, determining a discrete point corresponding to the measured data on the time series to be an abnormal point; on the contrary, in the case where the assumed abnormal point is determined to be normal after the correction, the discrete point corresponding to the measurement data in time series is determined to be a normal point.
In an example, when the discrete point corresponding to the measurement data on the time series is determined to be an abnormal point, the abnormal point needs to be subjected to data processing, and the specific data processing mode can determine the discrete degree of the abnormal point through an isolated forest model or a local abnormal factor algorithm.
For example, determining a first discrete point as an abnormal point, and outputting an abnormal score of the first discrete point through an isolated forest model or a local abnormal factor algorithm according to the measurement data of the service index time sequence where the first discrete point is located; and weighting and summing the abnormal score and the probability score after the obtained statistical distribution probability value is normalized to determine the abnormal score of the first discrete point, wherein the abnormal score is used for indicating the discrete degree of the first discrete point.
In the embodiment of the invention, the ring ratio threshold value and the same ratio threshold value are determined by measuring data of the service index time sequence, so that abnormal false alarm of service caused by errors of a data source is avoided. The confidence interval is corrected through the ring ratio threshold and the same ratio threshold, so that the monitoring accuracy of the service index is further improved and the false alarm or the false missing alarm is avoided compared with the embodiment 1.
For convenience of understanding, the anomaly detection method provided by the embodiment of the present invention is exemplified below by taking an anomaly detection system as an example for performing anomaly detection on a data center.
FIG. 6 illustrates a flow diagram for an anomaly detection system implementing anomaly detection for a data center, according to one embodiment.
As shown in fig. 6, the detection process may include steps 601 to 610, which are specifically as follows:
first, an anomaly detection task request is received.
The task request can be determined based on a query instruction sent by a user during man-machine interaction, or can be a task request sent by the data center.
Step 601: and responding to the abnormal point detection task request, and acquiring the measurement data of the service index time sequence of the monitored system in the data center.
For example: the measurement data of the service index time series are recorded as { X1, X2, X3 … … X10 }.
The method is divided into two processes, wherein one process is a process for determining a confidence interval, the other process is a process for determining a ring ratio threshold and a same ratio threshold, and then, an abnormal point in the service index time sequence is determined through the confidence interval, the ring ratio threshold and the same ratio threshold.
Step 602: and dividing the measurement data of the service index time sequence according to the data category.
And if the measured data are determined to be divided into proportion index data or magnitude index data according to the data type, the measured data are incremental window type indexes.
Step 603: and judging whether the measured data belongs to the incremental window type index.
In case the measurement data belongs to an incremental window-type indicator, step 604 is performed.
If the measured data does not belong to the incremental window indicator, the non-incremental window indicator is converted into the incremental window indicator, and step 604 is performed.
For example: calculating whether X2-X1 is greater than or equal to 0, and if so, the measured data are incremental window type indexes; if the difference value is equal to 0, calculating whether X3-X2 is larger than 0, and sequentially calculating until the difference value between the current value and the previous value is larger than 0, and determining the difference value as an incremental window type index; in the case of less than 0, it is necessary to determine whether the data source of X2 is correct itself, and if the data source itself is in error, X2 may be deleted or a new data source of X2 may be retrieved.
Step 604: and respectively calculating a ring ratio time sequence and a same ratio time sequence corresponding to the measurement data.
And calculating a ring ratio value and a same ratio value of the measured data, and determining a ring ratio time sequence and a same ratio time sequence according to the ring ratio value and the same ratio value.
For example: the loop ratio value for calculation of X2 is determined by equation (1):
(X2-X1)/X1*100％ (1)；
calculating the equivalence of X2 is determined by equation (2):
(X2-N)/N*100％ (2)；
where N is the measurement data of X2 at the same time as the previous cycle, that is, when X2 represents the measurement data of 10 am on day 4 of month 5, N is the measurement data of 10 am on day 4 of month 5.
According to the mode, the ring ratio and the same ratio corresponding to each measured data are respectively determined, then, the ring ratio time sequence corresponding to the measured data is determined according to the ring ratios, and the same ratio time sequence corresponding to the measured data is determined according to the same ratios.
Step 605: and performing data cleaning on the ring ratio time sequence and the same ratio time sequence.
Step 606: and determining a ring ratio threshold and a same ratio threshold according to the cleaned ring ratio time sequence and the same ratio time sequence.
For example: carrying out preset loop ratio threshold value rule processing on the loop ratio time sequence to determine a loop ratio threshold value; and carrying out preset comparison threshold rule processing on the comparison time sequence to determine a comparison threshold.
Step 607: and determining a mean baseline for the service index time series, and calculating measurement residual data of the measurement data and the mean baseline.
The step of calculating the mean baseline may refer to the method shown in step 420, and is not described herein again. The mean baseline is labeled { M1, M2, M3 … … M10}, where M1 is the mean baseline of X1, and so on.
Calculating measurement residual data of the measurement data and the mean baseline, for example: { X1-M1, X2-M2, X3-M3, … …, and X10-M10}, and the difference between the two, i.e., measurement residual data, is recorded as { Y1, Y2, and Y3 … … Y10 }.
Step 608: and performing residual prediction on the measured residual data to generate predicted residual data.
Residual prediction of the measured residual data by at least one of the following models: performing residual prediction on the measured residual data { Y1, Y2, Y3 … … Y10} to determine predicted residual data { Z1, Z2 and Z3 … … Z10}, wherein each measured value Y corresponds to one predicted value Z.
Step 609: and determining a confidence interval according to the prediction residual data and the measurement residual data.
Calculating difference data of the measured residual data and the predicted residual data; determining a confidence interval according to the difference data; and if the measured data is not in the confidence interval, taking the corresponding discrete point of the measured data on the time series as an assumed abnormal point.
For example: (Y1-Z1), (Y2-Z2), … … (Y10-Z10) are represented as { C1, C2, C3, … … C10}, mean and standard deviation are calculated for { C1, C2, C3, … … C10}, and confidence intervals are determined based on the mean and standard deviation in combination with N-sigma (i.e., mean + N standard deviation as upper line interval, mean-N standard deviation as lower line interval).
Step 610: and determining abnormal points of the monitored system on the time sequence according to the ring ratio threshold, the same ratio threshold, the predicted residual data and the measured residual data.
Wherein, the assumed outlier is corrected by the ring ratio threshold and the same ratio threshold determined in step 606, and a correction result is determined. If the correction result indicates that the assumed abnormal point remains abnormal, determining the assumed abnormal point as the abnormal point; on the contrary, when the correction result indicates that the assumed outlier is a false positive, the assumed outlier is marked as a normal.
It should be noted that, in the embodiment of the present invention, the steps 602 to 606 and the steps 607 to 609 may perform processing sequentially or may perform processing simultaneously, the order is merely a schematic representation, and the embodiment of the present invention does not limit the order in which the steps 602 to 606 and the steps 607 to 609 are performed.
Outputting the abnormal score of the abnormal point through the measured data of the service index time sequence where the abnormal point is located by an isolated forest model or a local abnormal factor algorithm; and weighting and summing the probability score obtained by normalizing the abnormal score and the obtained statistical distribution probability value to determine the abnormal score of the abnormal point, wherein the abnormal score is used for indicating the dispersion degree of the abnormal point.
In the embodiment of the invention, the dynamic baseline average value is calculated by using historical measurement data, manual judgment and setting of an empirical threshold are not needed, and the method is better suitable for data change. And subtracting the average value of nearly three days on the basis of the measured value to determine the measured residual data, so as to eliminate the abnormal false alarm caused by the increase and decrease of the real-time index period and the conventional data jitter of the system. Then, the measured residual data is predicted through a residual prediction model, and a confidence interval is calculated, so that the real-time response to the burst early warning can be ensured, and the lower false alarm rate can be kept. Then, a ring ratio threshold value and a same ratio threshold value are determined for the measurement data of the service index time sequence, so that abnormal false alarm of service caused by errors of a data source is avoided. And the signaling interval is corrected based on the ring ratio threshold and the same ratio threshold, so that the monitoring accuracy of the service index is further improved, and false alarm or missing report is avoided.
Fig. 7 shows a block diagram of an abnormal point detecting apparatus according to an embodiment.
As shown in fig. 7, the apparatus 70 may include:
an acquisition module 701, configured to acquire measurement data of a service index time sequence of a monitored system;
a processing module 702, configured to determine a mean baseline for the service indicator time series, and calculate measurement residual data between the measurement data and the mean baseline;
a prediction module 703, configured to perform residual prediction on the measured residual data to generate predicted residual data;
a generating module 704, configured to determine an abnormal point of the monitored system on the time series according to the prediction residual data and the measurement residual data.
Wherein the service index time series comprises a plurality of discrete points; the plurality of discrete points includes a first discrete point, and the measurement data of the first discrete point is used for representing the statistical data in a first time window to which the first discrete point belongs. The service indicator of the monitored system comprises at least one of the following: the access amount, the man-machine conversation turns, the man-machine conversation amount, the labor conversion rate and the single-turn conversation solution rate.
Here, the measurement residual data is residual predicted using at least one of: exponential smoothing model, autoregressive model, seasonal difference autoregressive moving average model.
In an example, the processing module 702 in the embodiment of the present invention may be specifically configured to obtain historical sub-data of a plurality of discrete points from historical data of a service indicator according to a time period; the measurement data of a preset number of discrete points before each discrete point are respectively selected, mean calculation is carried out, and a mean baseline for the first discrete point is determined.
In an example, the processing module 704 in the embodiment of the present invention may be specifically configured to calculate difference data between the measured residual data and the predicted residual data; determining a confidence interval according to the difference data; and if the measured data is not in the confidence interval, the corresponding discrete point of the measured data on the time series is an abnormal point.
In one example, the processing module 702 may be further configured to, if the measurement data of the service indicator time sequence belongs to an incremental window type indicator, respectively calculate a ring ratio time sequence and a same ratio time sequence corresponding to the measurement data; and determining a ring ratio threshold and a same ratio threshold according to the ring ratio time sequence and the same ratio time sequence. Further, if the measurement data of the service index time sequence does not belong to the incremental window type index, the measurement data of the service index time sequence is converted into the incremental window type index. The processing module 702 may also be configured to perform data cleansing on the ring ratio timing and the same ratio timing.
Here, the processing module 702 may be further configured to determine that the measurement data is classified into the proportion-type index data or the magnitude-type index data according to the data type, and then determine that the measurement data is an incremental window-type index.
In one example, the generating module 704 can be specifically configured to determine an outlier of the monitored system in the time series according to the ring ratio threshold, the same ratio threshold, the predicted residual data, and the measured residual data. Further, the generating module 704 determines a confidence interval according to the prediction residual data and the measurement residual data; if the measured data is not in the confidence interval, taking the corresponding discrete point of the measured data on the time sequence as an assumed abnormal point; and determining whether the assumed outlier is an outlier by using the ring ratio threshold and the same ratio threshold.
In one example, the generating module 704 may be further configured to determine the discrete degree of the outlier using an isolated forest model or a local outlier algorithm.
In the embodiment of the invention, the dynamic baseline average value is calculated by using historical measurement data, manual judgment and setting of an empirical threshold are not needed, and the method is better suitable for data change. And determining prediction residual data and calculating a confidence interval through a residual prediction model, so that the real-time reaction to the burst early warning can be ensured, and a lower false alarm rate can be kept. In addition, a ring ratio threshold value and a same ratio threshold value are determined for the measurement data of the service index time sequence, so that abnormal false alarm of service caused by errors of a data source is avoided. The signaling interval is corrected through the ring ratio threshold and the same ratio threshold, so that the monitoring accuracy of the service index is improved, and false alarm or missing report is avoided.
FIG. 8 illustrates a schematic structural diagram of a computing device, according to one embodiment.
As shown in fig. 8, a block diagram of an exemplary hardware architecture of a computing device capable of implementing the anomaly point detection method and apparatus according to the embodiments of the present invention.
The apparatus may include a processor 801 and a memory 802 storing computer program instructions.
Specifically, the processor 801 may include a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
The processor 801 reads and executes computer program instructions stored in the memory 802 to implement any one of the anomaly detection methods in the above-described embodiments.
The transceiver 803 is mainly used for implementing the apparatuses in the embodiments of the present invention or communicating with other devices.
In one example, the device may also include a bus 804. As shown in fig. 8, the processor 801, the memory 802, and the transceiver 803 are connected via a bus 804 to complete communication with each other.
Embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed in a computer, the computer is caused to execute the steps of the abnormal point detecting method according to the embodiments of the present invention.
It is to be understood that the invention is not limited to the particular arrangements and instrumentality described in the above embodiments and shown in the drawings. For convenience and brevity of description, detailed description of a known method is omitted here, and for the specific working processes of the system, the module and the unit described above, reference may be made to corresponding processes in the foregoing method embodiments, which are not described herein again.
It will be apparent to those skilled in the art that the method procedures of the present invention are not limited to the specific steps described and illustrated, and that various changes, modifications and additions, or equivalent substitutions and changes in the sequence of steps within the technical scope of the present invention are possible within the technical scope of the present invention as those skilled in the art can appreciate the spirit of the present invention.
Claims (16)
1. An abnormal point detection method, comprising:
acquiring measurement data of a service index time sequence of a monitored system;
determining a mean baseline for the service indicator time series, and calculating measurement residual data of the measurement data and the mean baseline;
residual prediction is carried out on the measured residual data to generate predicted residual data;
and determining abnormal points of the monitored system on the time sequence according to the prediction residual data and the measurement residual data.
2. The method of claim 1, wherein,
the service indicator time series comprises a plurality of discrete points; the plurality of discrete points includes a first discrete point, and the measurement data of the first discrete point is used for representing the statistical data in a first time window to which the first discrete point belongs.
3. The method of claim 2, wherein determining a mean baseline for the traffic indicator comprises:
acquiring historical subdata of a plurality of discrete points from the historical data of the service index according to a time period;
and respectively selecting the measurement data of a preset number of discrete points before each discrete point, carrying out mean value calculation, and determining a mean value baseline aiming at the first discrete point.
4. The method of claim 1, wherein determining an anomaly point of the monitored system over a time series from the prediction residual data and the measurement residual data comprises:
calculating difference data of the measurement residual data and the prediction residual data;
determining a confidence interval according to the difference data;
and if the measured data is not in the confidence interval, determining that the discrete point corresponding to the measured data on the time sequence is an abnormal point.
5. The method of claim 1, wherein the measured residual data is residual predicted using at least one of:
exponential smoothing model, autoregressive model, seasonal difference autoregressive moving average model.
6. The method of claim 1, further comprising:
if the measured data belong to the incremental window type indexes, respectively calculating a ring ratio time sequence and a same ratio time sequence corresponding to the measured data;
and determining a ring ratio threshold and a ratio threshold according to the ring ratio time sequence and the ratio time sequence.
7. The method of claim 6, further comprising:
and if the measured data does not belong to the increment window type index, converting the measured data into the increment window type index.
8. The method of claim 6, further comprising:
and if the measured data are determined to be divided into proportion index data or magnitude index data according to the data types, the measured data are incremental window type indexes.
9. The method of claim 6, wherein the step of determining the ring ratio threshold and the same ratio threshold is preceded by:
and performing data cleaning on the ring ratio time sequence and the same ratio time sequence.
10. The method of claim 9, wherein determining an anomaly point of the monitored system over a time series from the prediction residual data and the measurement residual data comprises:
and determining abnormal points of the monitored system on the time sequence according to the ring ratio threshold, the same ratio threshold, the prediction residual data and the measurement residual data.
11. The method of claim 10, wherein determining outliers of the monitored system in time series from the ring ratio threshold, the comparability threshold, the prediction residual data, and the measurement residual data comprises:
determining a confidence interval according to the prediction residual data and the measurement residual data;
if the measured data is not in the confidence interval, taking a discrete point corresponding to the measured data on a time sequence as an assumed abnormal point;
and determining whether the hypothesis outlier is an outlier by using the ring ratio threshold and the same ratio threshold.
12. The method of claim 1, further comprising:
and determining the discrete degree of the abnormal point by adopting an isolated forest model or a local abnormal factor algorithm.
13. The method of claim 1, wherein the traffic indicators of the monitored system include at least one of: the access amount, the man-machine conversation turns, the man-machine conversation amount, the labor conversion rate and the single-turn conversation solution rate.
14. An abnormal point detecting device, comprising:
the acquisition module is used for acquiring the measurement data of the service index time sequence of the monitored system;
the processing module is used for determining a mean baseline for the service index time sequence and calculating measurement residual data of the measurement data and the mean baseline;
the prediction module is used for carrying out residual prediction on the measured residual data to generate predicted residual data;
and the generating module is used for determining abnormal points of the monitored system on a time sequence according to the prediction residual data and the measurement residual data.
15. A computing device, wherein the device comprises a transceiver for transceiving data, at least one processor, and a memory for storing computer program instructions, the processor being configured to execute the program of the memory to control the computing device to implement the outlier detection method of any of claims 1-13.
16. A computer-readable storage medium on which a computer program is stored, wherein the computer program, if executed in a computer, causes the computer to execute the abnormal point detecting method according to any one of claims 1 to 13.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910749953.0A CN112395120A (en) | 2019-08-14 | 2019-08-14 | Abnormal point detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910749953.0A CN112395120A (en) | 2019-08-14 | 2019-08-14 | Abnormal point detection method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112395120A true CN112395120A (en) | 2021-02-23 |
Family
ID=74601408
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910749953.0A Pending CN112395120A (en) | 2019-08-14 | 2019-08-14 | Abnormal point detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112395120A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113127716A (en) * | 2021-04-29 | 2021-07-16 | 南京大学 | Sentiment time sequence anomaly detection method based on saliency map |
| CN113568950A (en) * | 2021-07-29 | 2021-10-29 | 北京字节跳动网络技术有限公司 | Index detection method, device, equipment and medium |
| CN115081969A (en) * | 2022-08-23 | 2022-09-20 | 中国中金财富证券有限公司 | Abnormal data determination method and related device |
| CN115168159A (en) * | 2022-09-06 | 2022-10-11 | 北京达佳互联信息技术有限公司 | Abnormality detection method, abnormality detection device, electronic apparatus, and storage medium |
| CN117454089A (en) * | 2023-11-17 | 2024-01-26 | 浙江预策科技有限公司 | Real-time analysis method and device for instrument panel, computer equipment and storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140358833A1 (en) * | 2013-05-29 | 2014-12-04 | International Business Machines Corporation | Determining an anomalous state of a system at a future point in time |
| CN104766175A (en) * | 2015-04-16 | 2015-07-08 | 东南大学 | Power system abnormal data identifying and correcting method based on time series analysis |
| CN106302487A (en) * | 2016-08-22 | 2017-01-04 | 中国农业大学 | Agricultural Internet of Things data flow anomaly detects processing method and processing device in real time |
| CN107086944A (en) * | 2017-06-22 | 2017-08-22 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality and device |
| CN107092582A (en) * | 2017-03-31 | 2017-08-25 | 江苏方天电力技术有限公司 | One kind is based on the posterior exceptional value on-line checking of residual error and method for evaluating confidence |
| CN107528722A (en) * | 2017-07-06 | 2017-12-29 | 阿里巴巴集团控股有限公司 | Abnormal point detecting method and device in a kind of time series |
| CN109542740A (en) * | 2017-09-22 | 2019-03-29 | 阿里巴巴集团控股有限公司 | Method for detecting abnormality and device |
| US20190147300A1 (en) * | 2017-11-16 | 2019-05-16 | International Business Machines Corporation | Anomaly detection in multidimensional time series data |
| CN109784042A (en) * | 2018-12-29 | 2019-05-21 | 北京奇安信科技有限公司 | The detection method of abnormal point, device, electronic equipment and storage medium in time series |
| CN110008080A (en) * | 2018-12-25 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Operational indicator method for detecting abnormality, device and electronic equipment based on time series |
| CN110032670A (en) * | 2019-04-17 | 2019-07-19 | 腾讯科技（深圳）有限公司 | Method for detecting abnormality, device, equipment and the storage medium of time series data |
| CN110083803A (en) * | 2019-04-22 | 2019-08-02 | 水利部信息中心 | Based on Time Series AR IMA model water intaking method for detecting abnormality and system |
-
2019
- 2019-08-14 CN CN201910749953.0A patent/CN112395120A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140358833A1 (en) * | 2013-05-29 | 2014-12-04 | International Business Machines Corporation | Determining an anomalous state of a system at a future point in time |
| CN104766175A (en) * | 2015-04-16 | 2015-07-08 | 东南大学 | Power system abnormal data identifying and correcting method based on time series analysis |
| CN106302487A (en) * | 2016-08-22 | 2017-01-04 | 中国农业大学 | Agricultural Internet of Things data flow anomaly detects processing method and processing device in real time |
| CN107092582A (en) * | 2017-03-31 | 2017-08-25 | 江苏方天电力技术有限公司 | One kind is based on the posterior exceptional value on-line checking of residual error and method for evaluating confidence |
| CN107086944A (en) * | 2017-06-22 | 2017-08-22 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality and device |
| CN107528722A (en) * | 2017-07-06 | 2017-12-29 | 阿里巴巴集团控股有限公司 | Abnormal point detecting method and device in a kind of time series |
| CN109542740A (en) * | 2017-09-22 | 2019-03-29 | 阿里巴巴集团控股有限公司 | Method for detecting abnormality and device |
| US20190147300A1 (en) * | 2017-11-16 | 2019-05-16 | International Business Machines Corporation | Anomaly detection in multidimensional time series data |
| CN110008080A (en) * | 2018-12-25 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Operational indicator method for detecting abnormality, device and electronic equipment based on time series |
| CN109784042A (en) * | 2018-12-29 | 2019-05-21 | 北京奇安信科技有限公司 | The detection method of abnormal point, device, electronic equipment and storage medium in time series |
| CN110032670A (en) * | 2019-04-17 | 2019-07-19 | 腾讯科技（深圳）有限公司 | Method for detecting abnormality, device, equipment and the storage medium of time series data |
| CN110083803A (en) * | 2019-04-22 | 2019-08-02 | 水利部信息中心 | Based on Time Series AR IMA model water intaking method for detecting abnormality and system |
Non-Patent Citations (3)
| Title |
|---|
| LI, H: "Research of K-MEANS Algorithm based on Information Entropy in Anomaly Detection", 《2012 2012 FOURTH INTERNATIONAL CONFERENCE ON MULTIMEDIA INFORMATION NETWORKING AND SECURITY (MINES 2012)》, 31 December 2012 (2012-12-31) * |
| 于艳华;宋美娜;张文婷;宋俊德;: "网络异常点检测中性能指标阈值的动态确定方法", 北京邮电大学学报, no. 02, 15 April 2011 (2011-04-15) * |
| 陈兴蜀;江天宇;曾雪梅;尹学渊;邵国林;: "基于多维时间序列分析的网络异常检测", 工程科学与技术, no. 01, 20 January 2017 (2017-01-20) * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113127716A (en) * | 2021-04-29 | 2021-07-16 | 南京大学 | Sentiment time sequence anomaly detection method based on saliency map |
| CN113127716B (en) * | 2021-04-29 | 2023-07-28 | 南京大学 | Emotion time sequence anomaly detection method based on saliency map |
| CN113568950A (en) * | 2021-07-29 | 2021-10-29 | 北京字节跳动网络技术有限公司 | Index detection method, device, equipment and medium |
| CN115081969A (en) * | 2022-08-23 | 2022-09-20 | 中国中金财富证券有限公司 | Abnormal data determination method and related device |
| CN115168159A (en) * | 2022-09-06 | 2022-10-11 | 北京达佳互联信息技术有限公司 | Abnormality detection method, abnormality detection device, electronic apparatus, and storage medium |
| CN117454089A (en) * | 2023-11-17 | 2024-01-26 | 浙江预策科技有限公司 | Real-time analysis method and device for instrument panel, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112395120A (en) | Abnormal point detection method, device, equipment and storage medium | |
| CN111984503B (en) | Method and device for identifying abnormal data of monitoring index data | |
| CN107871190B (en) | Service index monitoring method and device | |
| US8065098B2 (en) | Progressive humidity filter for load data forecasting | |
| US8886477B2 (en) | Reserve capacity calculating apparatus and method therefor, and computer readable medium | |
| CN107301570B (en) | Traffic prediction method, abnormal traffic detection device and electronic equipment | |
| CN112149860A (en) | Automatic anomaly detection method and system | |
| KR101941854B1 (en) | System and method of estimating load with null data correction | |
| CN113590429B (en) | Server fault diagnosis method and device and electronic equipment | |
| CN110909306B (en) | Business abnormality detection method and device, electronic equipment and storage equipment | |
| CN110874674A (en) | Anomaly detection method, device and equipment | |
| CN109376919B (en) | Prediction method for gas emission quantity of coal mine fully-mechanized coal mining face | |
| CN112148557B (en) | Method for predicting performance index in real time, computer equipment and storage medium | |
| US11567483B2 (en) | Computer-implemented determination of a quality indicator of a production batch-run that is ongoing | |
| JP2017130100A (en) | Information processing unit, information processing method and program | |
| US20130080125A1 (en) | Continuous prediction of expected chip performance throuhout the production lifecycle | |
| CN109376929B (en) | Distribution parameter determination method, distribution parameter determination device, storage medium, and electronic apparatus | |
| CN116775205A (en) | Resource capacity expansion and contraction method and device of cloud host | |
| CN113011664A (en) | Logistics timeliness monitoring method and device, electronic equipment and storage medium | |
| WO2010001966A1 (en) | Time-series data processing device and method and program thereof | |
| CN112182864A (en) | Method for selecting clock error prediction based on drift condition of hydrogen atomic clock | |
| JP2007164346A (en) | Decision tree changing method, abnormality determination method, and program | |
| CN112418534B (en) | Method and device for predicting quantity of collected parts, electronic equipment and computer readable storage medium | |
| CN115953032A (en) | Enterprise project execution risk assessment system based on data analysis | |
| US9699525B2 (en) | Method and system for automatic residual consumption |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |