CN113570396A

CN113570396A - Time series data abnormity detection method, device, equipment and storage medium

Info

Publication number: CN113570396A
Application number: CN202110103814.8A
Authority: CN
Inventors: 李昊容; 吴鸣; 李志颖; 崔艺华
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-01-26
Filing date: 2021-01-26
Publication date: 2021-10-29

Abstract

The disclosure provides a time series data anomaly detection method, a time series data anomaly detection device, time series data anomaly detection equipment and a storage medium. The method comprises the following steps: obtaining prediction data of the current time point according to the acquired historical time sequence data in the target length moving window before the current time point; determining the prediction data of the current time point and the exponential decay weight of the historical data of each historical time point according to the obtained weighted descending rate; obtaining a smooth value of the current time point according to the prediction data and the exponential decay weight thereof of the current time point, and the historical data and the exponential decay weight thereof of each historical time point; obtaining the discrete distance measurement of the current time point according to the prediction data of the current time point, historical time sequence data in a target length moving window, the weighted descending rate and the smooth value of the current time point; and determining a prediction threshold value of the next time point based on the smooth value of the current time point and the discrete degree quantity thereof. The method improves the accuracy of anomaly detection on the data at the next time point.

Description

Time series data abnormity detection method, device, equipment and storage medium

Technical Field

The present disclosure relates to the field of data mining technologies, and in particular, to a method and an apparatus for detecting time series data anomalies, an electronic device, and a computer-readable storage medium.

Background

In application scenarios of the internet, system operation and maintenance, financial indexes, industrial design and the like, time series data often need to be monitored for abnormalities. In an actual data monitoring application scenario, data has many situations which are difficult to process, such as complicated data characteristics, data missing, a large number of data topics, and the like. Some of the anomaly detection methods employed in the related art rely on time-series data to satisfy a specific distribution situation, and the accuracy of anomaly detection for data with complicated data characteristics is low. Other related techniques employ median quartile range (IQR) to detect outliers, which, while departing from the assumption of data distribution, tend to be too coarse to use and are not very sensitive to fluctuations in the data.

As described above, how to improve the accuracy of abnormality detection of time-series data in a complicated situation becomes an urgent problem to be solved.

The above information disclosed in this background section is only for enhancement of understanding of the background of the disclosure and therefore it may contain information that does not constitute prior art that is already known to a person of ordinary skill in the art.

Disclosure of Invention

The invention aims to provide a time series data abnormity detection method, a device, equipment and a readable storage medium, which can improve the abnormity detection accuracy of time series data under complex conditions.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.

The embodiment of the disclosure provides a time series data abnormity detection method, which comprises the following steps: acquiring historical time sequence data in a target length moving window before a current time point; obtaining the prediction data of the current time point according to the historical time sequence data in the target length moving window; acquiring a weighted descending rate; determining the prediction data of the current time point and the exponential decay weight of the historical data of each historical time point in the historical time sequence data according to the weighted descending rate; obtaining a smooth value of the current time point according to the prediction data and the exponential decay weight thereof of the current time point, and the historical data and the exponential decay weight thereof of each historical time point in the historical time sequence data; obtaining a discrete distance measurement of the current time point according to the prediction data of the current time point, historical time sequence data in the target length moving window, the weighted descending rate and the smooth value of the current time point; and determining a prediction threshold value of the next time point based on the smooth value of the current time point and the discrete degree quantity of the current time point, so as to be used for carrying out abnormity detection on the data of the next time point.

The embodiment of the present disclosure provides an abnormality detection apparatus, including: the historical data acquisition module is used for acquiring historical time series data in a target length moving window before the current time point; the current data prediction module is used for obtaining prediction data of the current time point according to historical time sequence data in the target length moving window; a weighted rate obtaining module, configured to obtain a weighted decreasing rate; the exponential weight acquisition module is used for determining the prediction data of the current time point and the exponential decay weight of the historical data of each historical time point in the historical time sequence data according to the weighted descending rate; a current smooth value obtaining module, configured to obtain a smooth value of the current time point according to the prediction data of the current time point and the exponential decay weight thereof, and historical data of each historical time point in the historical time series data and the exponential decay weight thereof; a current discrete magnitude obtaining module, configured to obtain a discrete magnitude of the current time point according to the prediction data of the current time point, the historical time series data in the target length moving window, the weighted descent rate, and the smoothed value of the current time point; and the threshold prediction module is used for determining a prediction threshold of the next time point based on the smooth value of the current time point and the discrete degree quantity of the current time point so as to detect the abnormality of the data of the next time point.

According to an embodiment of the present disclosure, the current smooth value obtaining module includes: the smooth value calculation module is used for adding the historical data of each historical time point in the historical time sequence data according to the exponential decay weight of the historical data to obtain a smooth value of the previous time point; and the weighted iteration module is used for weighting and adding the smooth value of the previous time point and the predicted data of the current time point according to the weighted descending rate to obtain the smooth value of the current time point.

According to an embodiment of the present disclosure, the weighted iteration module includes: a correction coefficient obtaining module, configured to obtain a deviation correction coefficient according to the weighted descent rate and the target length moving window; and the deviation correction module is used for weighting and adding the smooth value of the previous time point and the predicted data of the current time point according to the weighted descending rate and the deviation correction coefficient to obtain the smooth value of the current time point.

According to an embodiment of the present disclosure, the current data prediction module is further configured to: according to historical time sequence data in the target length moving window, adopting a moving average module, an exponential weighted average module or an interpolation module to obtain the prediction data of the current time point, wherein: the moving average module is used for selecting a target number of historical time sequence data from the historical time sequence data in the target length moving window; obtaining an average value of the target number of historical time-series data as prediction data of the current time point; the exponential weighted average module is used for acquiring historical time series data before a previous time point from the historical time series data in the target length moving window; obtaining a smooth value of the previous time point according to historical time sequence data before the previous time point; weighting and adding the smooth value of the last time point and the data of the last time point to obtain the predicted data of the current time point; the interpolation module is used for fitting the historical time sequence data in the target length moving window to obtain a target interpolation function; and obtaining the prediction data of the current time point according to the target interpolation function.

According to an embodiment of the present disclosure, the current data prediction module includes: the prediction method acquisition module is used for acquiring an input data prediction module to be called, and the data prediction module is at least one of the moving average module, the exponential weighted average module and the interpolation module; and the prediction data obtaining module is used for calling the input data prediction module to be called to obtain the prediction data of the current time point according to the historical time sequence data in the target length moving window.

According to an embodiment of the present disclosure, the weighted rate obtaining module includes: a half-life determination module for determining a target half-life, which is a time unit required by the exponential decay weights to be sequentially accumulated from back to front to half of the total weight; and the weighted rate calculation module is used for obtaining the weighted descending rate according to the target half-life.

According to an embodiment of the present disclosure, the discrete degree amount of the current time point includes a standard deviation of the current time point; the current discrete quantity acquisition module includes: the deviation calculation module is used for respectively obtaining the deviation between the time sequence data in the target length moving window and the smooth value of the current time point; and the standard deviation calculation module is used for weighting and adding the squared deviation according to the exponential decay weight of the corresponding time point, and then calculating a square root to obtain the standard deviation of the current time point.

According to an embodiment of the present disclosure, the prediction threshold comprises a prediction upper limit; the threshold prediction module comprises: the interval parameter acquisition module is used for acquiring interval prediction parameters according to the current time point; and the prediction upper limit calculation module is used for adding the smooth value of the current time point and the product of the interval prediction parameter and the standard deviation of the current time point to obtain the prediction upper limit of the next time point.

According to an embodiment of the present disclosure, the apparatus further comprises: the evaluation data acquisition module is used for acquiring time sequence data of an evaluation time window; an evaluation threshold obtaining module, configured to use each time point in the evaluation time window as the current time point, and obtain a prediction threshold of a next time point; an evaluation abnormal data obtaining module, configured to obtain predicted abnormal data from the time series data of the evaluation time window according to a prediction threshold of a next time point of each time point within the evaluation time window; a reference abnormal data obtaining module, configured to perform abnormal detection on data at each time point in the evaluation time window according to the time series data in the evaluation time window, so as to obtain reference abnormal data; and the prediction threshold covering index obtaining module is used for obtaining the prediction threshold covering index according to the abnormal prediction data and the abnormal reference data.

According to an embodiment of the present disclosure, the reference abnormal data obtaining module is further configured to: constructing a data space of time series data within the evaluation time window; dividing the data space into a plurality of subspaces through a random hyperplane, wherein each subspace of the plurality of subspaces only comprises data of one time point; and determining the reference abnormal data from the data of each subspace according to the times of dividing the data space when each subspace is obtained.

An embodiment of the present disclosure provides an apparatus, including: a memory, a processor and executable instructions stored in the memory and executable in the processor, the processor implementing any of the methods described above when executing the executable instructions.

Embodiments of the present disclosure provide a computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, implement any of the methods described above.

Embodiments of the present disclosure provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in the various alternative implementations described above.

According to the time series data abnormity detection method provided by the embodiment of the disclosure, prediction data of a current time point is obtained according to acquired historical time series data in a target length moving window before the current time point, then the prediction data of the current time point and the exponential decay weight of the historical data of each historical time point in the historical time series data are determined according to the acquired weighted descent rate, then the smooth value of the current time point is obtained according to the prediction data of the current time point and the exponential decay weight thereof as well as the historical data of each historical time point in the historical time series data and the exponential decay weight thereof, then the discrete distance measurement of the current time point is obtained according to the prediction data of the current time point, the historical time series data in the target length moving window, the weighted descent rate and the smooth value of the current time point, and then the threshold prediction threshold of the next time point is determined based on the smooth value of the current time point and the discrete distance measurement of the current time point The obtained prediction threshold value refers to some historical data which are closest to the current time point more, so that the prediction threshold value which is closer to the actual data fluctuation condition and is more accurate at the next time point can be set for carrying out abnormity detection on the data at the next time point, and the abnormity detection accuracy of the time series data is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

Fig. 1 shows a schematic diagram of a system architecture in an embodiment of the disclosure.

Fig. 2 shows a flowchart of a time series data anomaly detection method in an embodiment of the present disclosure.

Fig. 3 is a schematic diagram illustrating a processing procedure of step S204 shown in fig. 2 in an embodiment.

Fig. 4 is a schematic diagram illustrating a processing procedure of step S204 shown in fig. 2 in an embodiment.

Fig. 5 is a schematic diagram illustrating a processing procedure of step S204 shown in fig. 2 in an embodiment.

Fig. 6 is a schematic diagram illustrating a processing procedure of step S206 shown in fig. 2 in an embodiment.

Fig. 7 is a schematic diagram illustrating a processing procedure of step S210 shown in fig. 2 in an embodiment.

Fig. 8 is a schematic diagram illustrating a processing procedure of step S212 shown in fig. 2 in an embodiment.

FIG. 9 is a flowchart illustrating a method of threshold prediction model evaluation in accordance with an exemplary embodiment.

FIG. 10 is a diagram illustrating threshold predictive model test results in accordance with an exemplary embodiment.

FIG. 11 is a diagram illustrating threshold predictive model test results in accordance with an exemplary embodiment.

FIG. 12 is a diagram illustrating a threshold predictive model evaluation result according to an exemplary embodiment.

FIG. 13 is a diagram illustrating a threshold predictive model evaluation result according to an exemplary embodiment.

FIG. 14 is a diagram illustrating threshold predictive model test results in accordance with an exemplary embodiment.

FIG. 15 is a diagram illustrating threshold predictive model test results in accordance with an exemplary embodiment.

FIG. 16 is an illustration of an anomaly detection results interface, according to an example embodiment.

FIG. 17 is a diagram illustrating anomaly detection result data according to an exemplary embodiment.

Fig. 18 shows a block diagram of a time-series data abnormality detection apparatus in an embodiment of the present disclosure.

Fig. 19 shows a block diagram of another time-series data abnormality detection apparatus in the embodiment of the present disclosure.

Fig. 20 shows a schematic structural diagram of an electronic device in an embodiment of the present disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, apparatus, steps, etc. In other instances, well-known structures, methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.

Further, in the description of the present disclosure, unless otherwise explicitly specified or limited, terms such as "connected" and the like are to be construed broadly, e.g., may be electrically connected or may be in communication with each other; may be directly connected or indirectly connected through an intermediate. "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise. The specific meaning of the above terms in the present disclosure can be understood by those of ordinary skill in the art as appropriate.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

Machine Learning (ML) is a multi-domain cross discipline, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, formal learning, anomaly detection, and the like.

Anomaly Detection (AD) is an important branch of machine learning, and has a wide range of practical applications, and is more closely related to our lives. Anomaly detection is the discovery of objects that are different from most objects, i.e., the discovery of outliers. Data is generally specified to have a "normal" model, and anomalies are considered deviations from this normal model. The definition of anomalies in actual applications is also specific.

With the research and progress of artificial intelligence technology, the artificial intelligence technology is developed and applied in a plurality of fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, unmanned aerial vehicles, robots, smart medical care, smart customer service, and the like.

Fig. 1 illustrates an exemplary system architecture 10 to which the anomaly detection method, anomaly detection apparatus of the present disclosure may be applied.

As shown in fig. 1, system architecture 10 may include a terminal device 102, a network 104, a server 106, and a database 108. The terminal device 102 may be a variety of electronic devices having a display screen and supporting input, output, including but not limited to smart phones, tablets, laptop portable computers, desktop computers, wearable devices, virtual reality devices, smart homes, and the like. Network 104 is the medium used to provide communication links between terminal device 102 and server 106. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few. The server 106 may be a server or a cluster of servers, etc. that provide various services. The database 108 may be a large database software installed on a server or a small database software installed on a computer for storing data.

A user may use terminal device 102 to interact with server 106 and database 108 via network 104 to receive or transmit data and the like. For example, the user obtains the abnormal amount in the detected cross-border invoice data from the server 106 through the terminal device 102 via the network 104, and then marks the merchant corresponding to the abnormal amount through the invoice software on the terminal device 102. For another example, the user operates on the terminal device 102 to upload the device operation and maintenance index data to the database 108 through the network 104 for storage.

Data may also be received from database 108 or sent to database 108, etc. at server 106 via network 104. For example, server 106 may be a background processing server configured to perform anomaly detection on time series data obtained from database 108 via network 104. For another example, the server 106 may be configured to clean data obtained from the terminal device 102 via the network 104, and upload the cleaned data to the database 108 via the network 104 for storage.

It should be understood that the number of terminal devices, networks, servers, and databases in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, servers, and databases, as desired for implementation.

The technical scheme provided by the embodiment of the disclosure relates to aspects such as an abnormality detection technology, and the like, and is exemplified by a specific embodiment.

In a time series data processing scene, after data collection of each day is finished, the data is cleaned and stored in a storage system, and the data is required to predict a threshold value of the next day and output the threshold value according to the day. Theoretically, data can be reintegrated in each data record (for example, each order), and a new threshold value is output to each monitored object (for example, a merchant), so that the requirement on the computing power of the server is very high, and the data volume may be too large to support in a scenario with a large transaction volume and a large merchant.

As described above, in an actual data monitoring application scenario, there are many situations that are difficult to process, such as complicated data characteristics, missing data, and a large number of data topics. Therefore, the present disclosure provides a time series data anomaly detection method, which obtains prediction data of a current time point according to acquired historical time series data in a target length moving window before the current time point, then determines the prediction data of the current time point and an exponential decay weight of the historical data of each historical time point in the historical time series data according to the acquired weighted decrease rate, then obtains a smooth value of the current time point according to the prediction data of the current time point and the exponential decay weight thereof, and the historical data of each historical time point in the historical time series data and the exponential decay weight thereof, then obtains a discrete metric of the current time point according to the prediction data of the current time point, the historical time series data in the target length moving window, the weighted decrease rate and the smooth value of the current time point, and then determines a prediction metric of a next time point based on the smooth value of the current time point and the discrete metric of the current time point And the threshold value is used for carrying out abnormity detection on the data at the next time point, so that the accuracy of abnormity detection of the time series data can be improved by setting a more accurate prediction threshold value at the next time point.

Fig. 2 is a flowchart illustrating a time-series data anomaly detection method according to an exemplary embodiment. The method shown in fig. 2 may be applied to, for example, a server of the above system, and may also be applied to a terminal device of the above system.

Referring to fig. 2, a method 20 provided by an embodiment of the present disclosure may include the following steps.

In step S202, historical time-series data within a target-length moving window before the current time point is acquired. The time series data can be transaction amount of each merchant in the cross-border order-receiving project at continuous time points, check data of the equipment operation and maintenance index at the continuous time points, and the like. The current time point is a time point for threshold prediction, data before the current time point are historical time series data, and the latest data which can be obtained at the current time point is data at the previous time point. The mobile window is a time period of the selected time sequence data, and the length of the mobile window can be set.

In some embodiments, for example, in a project of detecting a transaction amount of a merchant, tens of thousands or hundreds of thousands of merchant information are processed every day, each merchant generates its own time series data, so that the calculation process may require hours or even hundreds of hours, if "day" is taken as a time unit, it is usually required to model the data processing beginning in the early morning of t days (the current time point, t is a positive integer greater than 1), at this time, the available data is data of 1,2, …, t-1 day, and then a prediction threshold value of t +1 day (the next time point) is output and pushed to the line, so that it is ensured that the transaction data can be detected abnormally using the prediction threshold value beginning in the early morning of t +1 day. The length of the moving window is t, and t can be set to 30 days, 60 days, 90 days and the like according to actual conditions.

In some embodiments, for example, in an overseas receipt project, the activity and data quality of merchants can be evaluated according to indexes such as total days of historical transactions, days of the latest transaction time, transaction activity, transaction stability, daily average transaction scale and the like, and then the merchants are grouped according to the transaction activity and data quality thereof, for example, the merchants can be classified as merchants with higher activity, better data quality (SEG _ a), moderate activity and the like, merchants with general data quality (SEG _ B), merchants with lower activity and poorer data quality (SEG _ C). When threshold prediction is performed on merchants for t +1 day, merchants in different groups output thresholds by using different models, for example, an accurate prediction scheme (prophet) is adopted by an SEG _ a merchant, a median IQR scheme is adopted by an SEG _ B merchant, an industry mean scheme is adopted by an SEG _ C merchant, and the like, only one threshold table is actually needed in online application, if the prediction step lengths of different merchant models are different, when a group skipping situation occurs in a merchant, a situation that no threshold is available or two thresholds are adaptive on the same day occurs in the merchant, and therefore the prediction step lengths of the models need to be unified. Therefore, a 2-step prediction can be achieved by making a threshold prediction of day t +1 on day t with day t-1 data.

In step S204, prediction data of the current time point is obtained from the historical time-series data within the target length moving window. The data at the current time point can be predicted according to historical data before the current time point, so that the prediction data can be used for predicting the threshold value at the next time point.

In some embodiments, for example, the prediction data of the current time point may be estimated from historical time series data within a moving window of the target length, including but not limited to obtaining the prediction data of the current time point by using a moving average method, an exponential weighted average method, and an interpolation method. The data prediction method at the current time point may refer to the embodiments of fig. 3 to 5.

In other embodiments, for example, the data prediction method at the current time point may be used as an input parameter, i.e., the input data prediction method may be one of a moving average method, an exponential weighted average method, and an interpolation method, and then the input data prediction method is used to obtain the prediction data at the current time point according to the historical time series data in the target length moving window.

In step S206, a weighted decreasing rate is acquired. The weighted descent rate used to obtain the exponential decay weight may be obtained as an input parameter prior to performing the smoothing value calculation.

In some embodiments, for example, the weighted decreasing rate may be set by setting the time required for the exponentially decaying weights to accumulate from back to front sequentially to half of the total weight, and specific embodiments may refer to fig. 6. The weighted rate of descent can be noted as α.

In other embodiments, the weighted decreasing rate may also be set by setting the time required for the exponentially decaying weights to accumulate from back to front sequentially to two thirds, or three quarters, etc. of the total weight, for example, in an embodiment similar to FIG. 6, i.e., let W be_HNThe weighted descent rate is obtained for two thirds, or three quarters, etc., and will not be described herein.

In step S208, an exponential decay weight of the prediction data at the current time point and the history data at each history time point in the history time-series data is determined according to the weighted decreasing rate. The meaning of the exponentially decaying weight is that the weight of the historical data at each historical time point decays exponentially in the order of time from back to front.

In some embodiments, for example, the current time point after obtaining the weighted rate of decline α

Predicted data and subsequent data y_t-1，y_t-2，y_t-3… … may correspond to an alpha/Λ, alpha²/Λ，α³/Λ，α⁴Λ, …, where Λ ═ α + α²+α³+α⁴+…。

Predicted data and subsequent data y_t-1，y_t-2，y_t-3… …, the exponential decay weight may correspond toIs alpha, alpha (1-alpha)²，α(1-α)³，…。

In step S210, a smoothed value at the current time point is obtained from the prediction data at the current time point and the exponential decay weight thereof, and the historical data at each historical time point in the historical time series data and the exponential decay weight thereof.

In some embodiments, for example, an exponentially weighted average of historical time points before the current time point may be obtained sequentially in chronological order by an iterative method, and a specific embodiment may refer to fig. 7.

In other embodiments, for example, the prediction data for the current time point (day t) may be directly based on

And its exponential decay weight alpha, and historical data y of each historical time point in the historical time series data₁，y₂，……y_t-1And its exponential decay weight alpha (1-alpha)^t，α(1-α)^t-1… α (1 α), the smoothed value S at the current time point can be obtained by calculation as follows_t：

In step S212, a discrete distance measure of the current time point is obtained according to the prediction data of the current time point, the historical time series data in the target length moving window, the weighted descending rate and the smooth value of the current time point. The discrete measure of the discrete data may be variance, standard deviation, etc., and the variance or standard deviation of these data may be obtained by taking the smoothed value of the current time point as the average value of the historical time series data within the target length moving window.

In some embodiments, for example, when the standard deviation is used as the discrete distance measure for predicting the threshold value of the next time point, the standard deviation of the current time point can be obtained by weighting the deviation of each historical time series data from the average value, and the specific implementation manner can refer to fig. 8,

in step S214, a prediction threshold value for the next time point is determined based on the smoothed value for the current time point and the amount of dispersion degree for the current time point, for use in abnormality detection of data for the next time point. The upper limit and the lower limit of the normal numerical value interval can be set for the next time point by the smooth value of the current time point and the discrete distance measurement of the current time point, and when the data of the next time point is obtained, if the data falls outside the interval, the data is judged to be abnormal data.

In some embodiments, for example, the interval prediction parameter may be obtained according to the current time point, and the smooth value of the current time point and the product of the interval prediction parameter and the standard deviation of the current time point are added to obtain the prediction upper limit of the next time point; the smoothed value of the current time point and the product of the interval prediction parameter and the standard deviation of the current time point can be subtracted to obtain the lower prediction limit of the next time point. If the interval prediction parameter is recorded as N_tThe standard deviation of the current time point is denoted as σ_tThen the predicted Upper limit Upper _ limit for the next time point can be obtained_t+1Comprises the following steps:

Lower_limit_t+1＝S_t-N_t*σ_t (2)

wherein N is_tAs an adjustable parameter with the current point in time, N_tFor example, it may take 3, 6 or 8, etc., which may be adjusted according to the actual situation, and take different values on different dates (e.g., weekday, holiday, special operating date). For distinguishing the special dates, a traditional method of festival + general festival + marketing festival + isolated forest identification can be used, after the threshold value is predicted, the set of the special dates is updated according to the identified abnormal points, the new special date set is used for predicting the new threshold value to find the abnormal points, and the special date set is generated in an iteration mode by combining the special dates identified in other models. For example, if different interval prediction parameters are set for a holiday and a working day, and the holday set is a set of holidays, it can be configured that:

wherein N is_{sigma_holiday}Predicting a parameter for a vacation interval, N_{sigma_normal}Parameters are predicted for the interval of the working day.

According to the time series data abnormity detection method provided by the embodiment of the disclosure, the prediction data of the current time point is obtained according to the acquired historical time series data in the target length moving window before the current time point, then the exponential decay weight of the prediction data of the current time point and the historical data of each historical time point in the historical time series data is determined according to the acquired weighted descent rate, then the smooth value of the current time point is obtained according to the prediction data of the current time point and the exponential decay weight thereof as well as the historical data of each historical time point in the historical time series data and the exponential decay weight thereof, then the discrete distance measurement of the current time point is obtained according to the prediction data of the current time point, the historical time series data in the target length moving window, the weighted descent rate and the smooth value of the current time point, and then the prediction degree measurement of the next time point is determined based on the smooth value of the current time point and the discrete distance measurement of the current time point And the threshold value is used for carrying out abnormity detection on the data at the next time point, so that the available prediction threshold value more references some historical data which are closest to the current time point, therefore, the prediction threshold value which is closer to the actual data fluctuation condition and is more accurate at the next time point can be set, and the abnormity detection accuracy of the time series data is further improved.

Fig. 3 is a schematic diagram illustrating a processing procedure of step S204 shown in fig. 2 in an embodiment. As shown in fig. 3, in the embodiment of the present disclosure, when the prediction data of the current time point is estimated from the historical time-series data by using the moving average method in step S204, step S204 may further include the following steps.

In step S20402, a target number of historical time-series data are selected from the historical time-series data within the target length shifting window.

In step S20404, the average of the target number of historical time-series data is obtained as the prediction data at the current point in time.

In some embodiments, historical time series data within the target length moving window may be represented as: y is₁，y₂，……y_t-1Where t (day) is the moving window target length. The data y can be obtained at the day of t-1_t-1Initially, a continuous target amount of data (e.g., m ∈ N × (positive integer), which is an adjustable parameter) is selected to be arithmetically averaged to be used as the prediction data of the current time point. The estimated t-th day data by the moving average method can be obtained by the following formula

According to the data prediction method of the current time point provided by the embodiment of the disclosure, the average value of the target number of historical time series data closest to the current time point is used as the prediction data of the current time point, so that the predicted value is close to the actual situation, the convenience of a prediction model is improved, and the processing efficiency is improved.

Fig. 4 is a schematic diagram illustrating a processing procedure of step S204 shown in fig. 2 in an embodiment. As shown in fig. 4, in the embodiment of the present disclosure, when the prediction data of the current time point is estimated from the historical time-series data by using the exponential weighted average method in step S204, step S204 may further include the following steps.

In step S20412, historical time-series data before the previous time point is acquired from the historical time-series data within the target length moving window.

In step S20414, a smoothed value at the previous time point is obtained from the historical time-series data before the previous time point. The smooth value at the previous time point is obtained similarly to the smooth value at the time point, and the smooth value at the previous time point may be calculated by an exponential weighted average method of the historical time series data, which may be specifically referred to as step S206 to step S210 in fig. 2. The obtaining of the smoothed value at the previous time point is different from the obtaining of the smoothed value at the time point in that the historical time-series data before the previous time point are all available as actual data without estimating the data of the "previous time point" at the previous time point.

Step S20416, the smoothed value at the previous time point is added to the data at the previous time point in a weighted manner to obtain the predicted data at the current time point. The weighting can be adjusted according to the specific application scene, so that the obtained prediction data is closer to the actual data.

According to the data prediction method of the current time point provided by the embodiment of the disclosure, the smooth value of the previous time point is obtained by performing weighted addition on the historical time series data before the previous time point, so that the weight occupied by the historical time series data closest to the current time point when the smooth value is calculated can be larger, and the smooth value of the previous time point and the data of the previous time point are subjected to weighted addition to be used as the prediction data of the current time point, thereby further improving the accuracy of the prediction data.

Fig. 5 is a schematic diagram illustrating a processing procedure of step S204 shown in fig. 2 in an embodiment. As shown in fig. 5, in the embodiment of the present disclosure, when the prediction data of the current time point is estimated from the historical time-series data by using interpolation in step S204, step S204 may further include the following steps.

Step S20422, fitting the historical time series data in the target length moving window to obtain a target interpolation function. The discrete historical time sequence data can be fitted to a function which takes time as an independent variable and corresponding data values as dependent variables, and a linear function, a polynomial function and the like can be adopted for fitting to obtain a target interpolation function which covers the most historical time sequence data.

Step S20424, obtaining the prediction data of the current time point according to the target interpolation function. The current time point is out of the time range of the target length moving window, so that the prediction data of the current time point can be obtained by a target interpolation function obtained by fitting by adopting an extrapolation method.

According to the data prediction method of the current time point provided by the embodiment of the disclosure, the prediction data of the current time point is obtained by performing extrapolation on historical time sequence data in a moving window according to the target length, the prediction value of the current time point can be obtained more conveniently, and the processing efficiency is improved.

Fig. 6 is a schematic diagram illustrating a processing procedure of step S206 shown in fig. 2 in an embodiment. As shown in fig. 6, in the embodiment of the present disclosure, the step S206 may further include the following steps.

Step S2062, a target half-life is determined, where the target half-life is a time unit required for the exponential decay weight to be sequentially accumulated to half of the total weight from back to front. For the historical time-series data y of the last n days before the current time point_t-1，y_t-2，…，y_t-nOf y_t-nWeight ω corresponding to term_t-nCan be expressed as:

the target half-life is HN, which can be set according to the business situation, for example, the transaction data of one week, 2 weeks or one month, etc. which is more useful for predicting transactions and more representative of the recent transaction trend of the merchant, the exponentially decaying weights can be accumulated from the back to the front to the HN day in time sequence_HNExpressed as the difference between the total weight (i.e., 1) and the HN's future weight (which may be referred to as the tail weight):

in step S2064, a weighted decreasing rate is obtained according to the target half-life. Let W_HNEqual to 1/2, alpha-1-e is obtained^ln ^(0.5)/HN。

According to the method for acquiring the weighted decreasing rate provided by the embodiment of the disclosure, the weighted decreasing rate is acquired by setting the half-life parameter of the exponential decay weight, so that the weight of the data of the concerned recent days can be set higher, and the smooth value of the acquired current time point can reflect the trend of the actual data more truly.

Fig. 7 is a schematic diagram illustrating a processing procedure of step S210 shown in fig. 2 in an embodiment. As shown in fig. 7, in the embodiment of the present disclosure, the step S210 may further include the following steps.

In step S2102, the historical data at each historical time point in the historical time series data is added according to the exponential decay weight thereof, and a smoothed value at the previous time point is obtained.

Step S2104, the smoothed value of the previous time point and the predicted data of the current time point are added in a weighted manner according to the weighted decreasing rate, and the smoothed value of the current time point is obtained.

In some embodiments, the smoothed value of the current time point is recorded as S_tThe t-th day data estimated by the moving average method is

S can be iteratively calculated from_t：

Where α is the weighted decreasing rate, the smoothed value S at the previous time point_t-1It can also be obtained by iteration (7).

In some embodiments, in the case that the time point of the data value 0 is more or the time window length is shorter, the data amount of the historical time-series data is less, and at this time, the deviation of the smoothed value estimated by the above iterative method from the actual data is larger, and the deviation correction can be performed on the smoothed value by the following steps.

In step S21042, a deviation correction coefficient is obtained from the weighted descent rate and the target length moving window.

Step S21044 is performed to add the smoothed value at the previous time point and the predicted data at the current time point in a weighted manner according to the weighted decreasing rate and the deviation correction coefficient, so as to obtain the smoothed value at the current time point.

In some embodiments, the corrected smoothed value S 'for the current point in time may be'_tExpressed as:

wherein the denominator is S in formula (1)_tSum of coefficients of terms in expanded form of expression, i.e. guaranteed S'_tThe sum of the coefficients of the terms in the expansion is 1, which can avoid the estimated smooth value deviating from the real data greatly when the data amount is small.

According to the smooth value calculation method of the current time point provided by the embodiment of the disclosure, the deviation correction is carried out after the half-life iterative calculation according to the exponential decay weight, so that the obtained smooth value of the current time point can reflect the trend of actual data more truly.

Fig. 8 is a schematic diagram illustrating a processing procedure of step S212 shown in fig. 2 in an embodiment. As shown in fig. 8, in the embodiment of the present disclosure, the step S212 may further include the following steps.

Step S2122, obtaining deviations between the time-series data in the target length moving window and the smoothed value of the current time point, respectively.

And step S2124, after the deviation is squared, weighting and adding according to the exponential decay weight of the corresponding time point, and then solving the square root to obtain the standard deviation of the current time point.

In some embodiments, for example, the calculation S may be used_tThe standard deviation sigma of the t day is calculated by weighting the exponential decay weight_t：

Wherein for the y-th_t-kItem correspondenceWeight ω of (d)_t-k：

According to the method for calculating the standard deviation of the current time point, provided by the embodiment of the disclosure, the deviation between the time series data of each historical time point and the smooth value of the current time point is weighted by adopting the exponential decay weight, so that the obtained standard deviation of the current time point can reflect the dispersion degree of actual data more truly.

FIG. 9 is a flowchart illustrating a method of threshold prediction model evaluation in accordance with an exemplary embodiment. The method shown in fig. 9 may be, for example, a method of obtaining a prediction threshold described above with reference to fig. 2 to 8, and the accuracy of the abnormality prediction is evaluated.

In the development era of internet finance, more and more users carry out payment activities in an internet mode, merchants can more efficiently carry out collection and payment activities with the help of mobile payment, and individuals can more conveniently go out and carry only one mobile phone. In the internet financial scenario, the most common is time series data, payment activities are accumulated into payment data along with one order, and countless payment data form massive transaction behavior data. The abnormal fluctuation condition of transaction behaviors is necessary to be noticed, which is related to the payment security of internet finance, and relates to the interests of a payment platform and a user. In the cross-border order collection project, daily transactions of cross-border order collection merchants form time sequence data, and in this scenario, a threshold value of the next day needs to be output every day for each merchant, so that exception detection is performed on transaction data of each merchant on the next day. In the related technology, the prediction threshold value of t day is directly used for replacing the prediction threshold value of t +1, the obtained threshold value has poor precision and weaker interpretability. The present disclosure provides a rollback algorithm, which allows the prediction at each time to be based on the time point at that time, i.e., a method of predicting the data of t days by using the actual data of t-1 days and then predicting the threshold value of t +1 days.

In testing a threshold prediction model (also referred to as a volatility model), the following two threshold prediction models were first compared:

1) moving average + N sigma scheme: the moving average of the time-series data in the moving window before t days is calculated by using the moving average method shown in fig. 3, then the standard deviation (sigma) of the time-series data in the moving window is calculated by using the moving average, and then the moving average plus N times sigma is used as the upper limit of the prediction threshold. In the test, the setting of the moving window comprises 30 days, 60 days, 90 days, 120 days, 180 days and 360 days, the setting of N comprises 3 times, 4 times, 5 times and 6 times, and the setting of the moving window and the setting of N are arranged and combined to carry out the test.

2) Moving median IQR model: and obtaining the moving median of the time sequence data in the moving window t days ago, then obtaining the quartile on two sides of the median of the time sequence data in the moving window, and then adding the M-times quartile (IQR) to the moving median to be used as the upper limit of the prediction threshold. In the test, the settings of the moving window include 30 days, 60 days, 90 days, 120 days, 180 days, 360 days, and the settings of M include 1.5, 3 times, and so on.

The pair of partial test results of the two models is shown in fig. 10 and 11.

FIG. 10 shows the time series of test results for the amount of a transaction by an overseas merchant using the moving average +6sigma scheme, where the line is a time series of data points with a time window of 90 days, and the points marked on the line are the detected early warning points (outliers detected by the predictive threshold). As shown in fig. 10, the number of early warning points obtained by the moving average +6sigma scheme is large, and there are some redundant early warning points. In the early warning points of continuous time points on a connecting line of the rising trend, early warning is only needed to be carried out at the beginning of the rising trend, and redundant work can be caused to merchant auditors when the number of the continuous early warning points is too large. For the moving average model, when the moving average of the continuous time points is obtained, the data difference between the time series data is only two-end data, so the moving average between the continuous time points is very close, and therefore, continuous early warning points are easy to appear in the continuous data which shows an ascending trend, which indicates that the flexibility of the moving average model is poor.

Fig. 11 shows the time-series item test result of the transaction amount of the oversea merchant using the mobile median +3 times IQR scheme, where the connecting line in the figure is a connecting line of time-series data points with a time window of 90 days, and the point marked on the connecting line is a detected early warning point (abnormal point detected by a prediction threshold). As shown in fig. 11, the early warning points obtained by the mobile median +3 times of the IQR scheme are less than those of the model shown in fig. 10, and no early warning is given at the time point when some data suddenly rises, which indicates that the model is rough and may miss the alarm.

Referring to fig. 9, a model evaluation method 90 provided by the embodiments of the present disclosure may quantify an abnormal prediction condition of an evaluation model. The steps of fig. 9 will be described with reference to a specific application scenario.

In step S902, time-series data of an evaluation time window is acquired.

In some embodiments, for example, the setting of the evaluation time window includes 30 days, 60 days, 90 days, 120 days, 180 days, 360 days, etc., and the data of these windows can be used to obtain the evaluation index respectively, which can be used to select the length of the time window in the threshold prediction.

In step S904, each time point within the evaluation time window is taken as a current time point, and a prediction threshold value of a next time point thereof is obtained. Based on each time point in the evaluation time window, the model to be evaluated is adopted to predict the threshold value of the next time point by using the available historical time data at that time, so as to perform abnormity judgment on the data of the next time point by using the predicted threshold value.

In some embodiments, for example, data for the current point in time (t days) may be obtained for comparison in a variety of ways in predicting the threshold. In the test, the t-day data is obtained by directly using the t-day original value, the moving average method shown in fig. 3 and the exponential weighted average method shown in fig. 4, and the influence on the threshold value and the abnormal point is generally found to be small.

In step S906, predicted abnormality data is obtained from the time-series data of the evaluation time window in accordance with the prediction threshold value of the next time point of each time point within the evaluation time window.

In step S908, based on the time-series data within the evaluation time window, abnormality detection is performed on the data at each time point within the evaluation time window, and reference abnormality data is obtained. The abnormality detection method for reference is based on the "future", that is, after the occurrence of a data point to be judged as abnormal, whether an abnormal point exists in the obtained data is identified in a global view.

In some embodiments, for example, an isolated forest method may be used to obtain reference abnormal data, that is, a data space for constructing time series data within an evaluation time window, the data space is divided into a plurality of subspaces by a random hyperplane, each of the plurality of subspaces includes data of only one time point, and the reference abnormal data is determined from the data of each subspace according to the number of times of dividing the data space when each subspace is obtained, for example, the data of which the number of times of dividing is less than a preset threshold may be regarded as the reference abnormal data.

In some embodiments, a method based on the distance/density between data may also be used to determine whether the data is abnormal, and the data in the region with smaller density and data farther from other data points may be determined as reference abnormal data.

In step S910, a prediction threshold coverage index is obtained from the predicted abnormality data and the reference abnormality data. Since the data of the reference abnormal data is the result of global identification, the coverage of the abnormal data point by the threshold prediction model can be evaluated by taking the data as a reference.

In some embodiments, for example, for an item of merchant transaction amount data, the merchant average coverage rate (the abnormal point of a certain merchant covered the abnormal point of its isolated forest)/the total abnormal point of the isolated forest may be used as the prediction threshold coverage index. The coverage rate of each early warning point of the merchant can also be used as a prediction threshold coverage index, and the coverage rate of each early warning point of the merchant is equal to the average coverage rate of the merchant/average early warning point per month of the merchant by a prediction model. Fig. 12 shows the evaluation results of the moving average +6sigma model using the isolated forest method, in which the detection of outliers does not include holidays. As shown in fig. 12, the merchant average coverage for the moving average +6sigma model is low for time windows of different lengths.

FIG. 13 illustrates a comparison of the moving average +6sigma, moving median +3IQR prediction thresholds with isolated forest method detection outliers in the moving average + N sigma and moving median IQR model test described above, where the detection of outliers includes holidays. The quantile parameter of the isolated forest is set to be 0.02, which means that outliers _ fraction is 0.02, meaning that samples with abnormal scores below 0.02 quantile of all samples are abnormal points.

In combination with the number of early warnings, the evaluation index, and the service scenario, the anomaly detection method for threshold prediction shown in fig. 2 to 9 of the present disclosure is subjected to a comparison test with the moving average +6sigma, and fig. 14 to 15 are test results, where fig. 15 is the same as fig. 14. In the test, the parameters of the threshold prediction method shown in fig. 2 of the present disclosure are set as follows: the target half-life is 7 days, namely the weight of the data from the t day to the t-7 day is reduced to 0.5; the interval prediction parameter is 6, and the interval prediction parameter is not distinguished between holidays and weekdays. As can be seen from fig. 14 to fig. 15, the prediction model of the method disclosed herein can better conform to the fluctuation trend of the actual data, and is more sensitive to outliers, and on the basis of covering the original early warning points of the moving average +6sigma model, outliers of merchants with very stable trends can be found, which are difficult to find by the moving average +6sigma model.

The following describes an application of the method including time information (exponentially decaying weight) shown in fig. 2 to fig. 9 in a service scenario.

In online use, the method has a plurality of parameters including a target half-life period, an interval prediction parameter of a common date, an interval prediction parameter of a special date and the like, and can be quickly adjusted according to online conditions.

In the overseas receipt project, the method is used for merchants of the SEG _ B, and the group of merchants with the characteristics of medium liveness and general data quality account for more than 70% of the historical transaction amount in the general overseas receipt merchants. The investigation and audit result shows that the method has better application effect.

Fig. 16 is a schematic diagram of an anomaly detection result interface using the anomaly detection method shown in fig. 2 to 9 of the present disclosure. As shown in fig. 16, according to the data of company a between 6 months in 2019 and 9 months in 2020, company a is suspected of suspicious transaction as a result of the detection, and a monthly transaction limit is set for company a according to the result of the detection.

Fig. 17 shows abnormality detection data for another merchant using the abnormality detection method shown in fig. 2 to 9 of the present disclosure. As shown in fig. 17, a merchant ALPHA HEALTH CARE (NEW ZEALAND) LIMITED continuously alarms, and after manual review, the merchant is suspected of other illegal behaviors (historical same-name illegal merchants-illegal reimbursement), and finally, the qualitative and processing opinions are that the merchant network checks negative public sentiments of reimbursement, but has direct sales licenses in China, and is judged to be the same subject as the historical same-name and LIMITED merchants, so that the current same history is related to disposal, and the limit is 5 thousand per month.

Fig. 18 is a block diagram illustrating an abnormality detection apparatus according to an exemplary embodiment. The apparatus shown in fig. 18 can be applied to, for example, a server side of the system described above, and can also be applied to a terminal device of the system described above.

Referring to fig. 18, the apparatus 180 provided in the embodiment of the present disclosure may include a historical data obtaining module 1802, a current data predicting module 1804, a weighted rate obtaining module 1806, an exponential weight obtaining module 1808, a current smooth value obtaining module 1810, a current discrete quantity obtaining module 1812, and a threshold predicting module 1814.

Historical data acquisition module 1802 may be used to acquire historical time series data within a moving window of target length prior to a current point in time.

The current data prediction module 1804 may be configured to obtain prediction data for a current time point from historical time series data within a target length moving window.

The weighted rate acquisition module 1806 may be configured to acquire a weighted decreasing rate.

The exponential weight obtaining module 1808 may be configured to determine, according to the weighted decreasing rate, an exponential decay weight of the prediction data at the current time point and the historical data at each historical time point in the historical time series data.

The current smoothed value obtaining module 1810 may be configured to obtain a smoothed value at the current time point according to the predicted data at the current time point and the exponential decay weight thereof, and the historical data at each historical time point in the historical time series data and the exponential decay weight thereof.

The current discrete quantity obtaining module 1812 may be configured to obtain a discrete quantity metric at the current time point according to the predicted data at the current time point, the historical time-series data in the target length moving window, the weighted descent rate, and the smoothed value at the current time point.

The threshold prediction module 1814 may be used to determine a prediction threshold for a next time point based on the smoothed value of the current time point and the discrete degree amount of the current time point, so as to perform anomaly detection on data of the next time point.

Fig. 19 is a block diagram illustrating another abnormality detection apparatus according to an exemplary embodiment. The apparatus shown in fig. 19 can be applied to, for example, a server side of the system described above, and can also be applied to a terminal device of the system described above.

Referring to fig. 19, an apparatus 190 provided in the embodiment of the present disclosure may include a historical data obtaining module 1902, a current data predicting module 1904, a moving average module 19052, or an exponential weighted average module 19054, an interpolation module 19056, a weighted rate obtaining module 1906, an exponential weight obtaining module 1908, a current smooth value obtaining module 1910, a current discrete quantity obtaining module 1912, a threshold predicting module 1914, an evaluation data obtaining module 1916, an evaluation threshold obtaining module 1918, an evaluation abnormal data obtaining module 1920, a reference abnormal data obtaining module 1922, a prediction threshold coverage index obtaining module 1924, where the current data predicting module 1904 may include a prediction method obtaining module 19042 and a prediction data obtaining module 19044, the weighted rate obtaining module 1906 may include a half-life determining module 19062 and a weighted rate calculating module 19064, the current smooth value obtaining module 1910 may include a smooth value calculating module 19102 and a weighted iteration module 19104, the weighted iteration module 19104 may include a correction coefficient obtaining module 191042 and a deviation correction module 191044, the current discrete quantity obtaining module 1912 may include a deviation calculation module 19122 and a standard deviation calculation module 19124, and the threshold prediction module 1914 may include an interval parameter obtaining module 19142 and a prediction upper limit calculation module 19144.

The historical data acquisition module 1902 may be configured to acquire historical time series data within a moving window of a target length prior to a current point in time.

The current data prediction module 1904 may be configured to obtain prediction data of a current time point according to historical time series data in a target length moving window.

The current data prediction module 1904 may also be configured to obtain prediction data of a current time point by using the moving average module 19052, the exponential weighted average module 19054, or the interpolation module 19056 according to historical time series data in a moving window of the target length.

The prediction method obtaining module 19042 may be configured to obtain an input data prediction module to be invoked, where the data prediction module is at least one of the moving average module 19052, the exponentially weighted average module 19054, and the interpolation module 19056.

The prediction data obtaining module 19044 may be configured to call the input data prediction module to be called to obtain prediction data of the current time point according to the historical time series data in the target length moving window.

The moving average module 19052 may be configured to select a target number of historical time series data from the historical time series data within the target length moving window; an average value of the target number of the historical time-series data is obtained as prediction data of the current time point.

The exponentially weighted average module 19054 may be configured to obtain historical time-series data from the historical time-series data within the moving window of the target length before a previous time point; obtaining a smooth value of the previous time point according to historical time sequence data before the previous time point; and weighting and adding the smooth value of the last time point and the data of the last time point to obtain the predicted data of the current time point.

The interpolation module 19056 may be configured to fit historical time series data within a target length moving window to obtain a target interpolation function; and obtaining the prediction data of the current time point according to the target interpolation function.

The weighted rate acquisition module 1906 may be used to acquire a weighted rate of decrease.

Half-life determination module 19062 may be used to determine a target half-life, which is the unit of time required for the exponentially decaying weights to accumulate from back to front in time to half of the total weight.

The weighted rate calculation module 19064 may be used to obtain a weighted rate of decline based on the target half-life.

The exponential weight obtaining module 1908 may be configured to determine an exponential decay weight of the prediction data at the current time point and the historical data at each historical time point in the historical time series data according to the weighted decrease rate.

The current smooth value obtaining module 1910 may be configured to obtain a smooth value at the current time point according to the prediction data at the current time point and the exponential decay weight thereof, and the historical data at each historical time point in the historical time series data and the exponential decay weight thereof.

The smoothed value calculating module 19102 may be configured to add the historical data at each historical time point in the historical time series data according to the exponential decay weight thereof, so as to obtain a smoothed value at the previous time point.

The weighted iteration module 19104 may be configured to perform weighted addition on the smoothed value at the previous time point and the predicted data at the current time point according to the weighted decreasing rate to obtain the smoothed value at the current time point.

The correction factor obtaining module 191042 may be configured to obtain a bias correction factor based on the weighted descent rate and the target length movement window.

The deviation correction module 191044 is configured to perform a weighted addition on the smoothed value at the previous time point and the predicted data at the current time point according to the weighted decreasing rate and the deviation correction coefficient to obtain a smoothed value at the current time point.

The current discrete quantity obtaining module 1912 may be configured to obtain a discrete quantity metric at the current time point according to the predicted data at the current time point, the historical time series data in the target length moving window, the weighted descent rate, and the smoothed value at the current time point. The dispersion measure for the current time point includes a standard deviation for the current time point.

The deviation calculation module 19122 may be configured to obtain deviations of the time-series data within the moving window of the target length from the smoothed value of the current time point, respectively.

The standard deviation calculation module 19124 may be configured to square the deviation, add the squared deviation according to the exponentially decaying weights of the corresponding time points, and obtain the square root, thereby obtaining the standard deviation of the current time point.

The threshold prediction module 1914 may be configured to determine a prediction threshold for a next time point based on the smoothed value of the current time point and the discrete degree amount of the current time point, so as to perform anomaly detection on data of the next time point. The prediction threshold comprises a prediction upper bound.

The interval parameter obtaining module 19142 may be configured to obtain an interval prediction parameter according to the current time point.

The prediction upper limit calculation module 19144 may be configured to add the smoothed value at the current time point and the product of the interval prediction parameter and the standard deviation at the current time point to obtain the prediction upper limit at the next time point.

Evaluation data acquisition module 1916 may be used to acquire time series data for an evaluation time window.

The evaluation threshold obtaining module 1918 is configured to obtain a prediction threshold of a next time point of each time point within the evaluation time window as a current time point.

The estimated anomaly data obtaining module 1920 is operable to obtain predicted anomaly data from the time series data for the evaluation time window based on the prediction threshold for the next time point at each time point within the evaluation time window.

The reference abnormal data obtaining module 1922 may be configured to perform abnormal detection on data at each time point in the evaluation time window according to the time-series data in the evaluation time window, so as to obtain reference abnormal data.

The reference anomaly data acquisition module 1922 may also be used to construct a data space that evaluates the time series data within the time window; dividing the data space into a plurality of subspaces through a random hyperplane, wherein each subspace in the plurality of subspaces only comprises data of one time point; reference abnormal data is determined from data of each subspace according to the number of times of dividing the data space when each subspace is obtained.

The prediction threshold coverage indicator obtaining module 1924 may be configured to obtain a prediction threshold coverage indicator based on the predicted anomaly data and the reference anomaly data.

The specific implementation of each module in the apparatus provided in the embodiment of the present disclosure may refer to the content in the foregoing method, and is not described herein again.

Fig. 20 shows a schematic structural diagram of an electronic device in an embodiment of the present disclosure. It should be noted that the apparatus shown in fig. 20 is only an example of a computer system, and should not bring any limitation to the function and the scope of the application of the embodiments of the present disclosure.

As shown in fig. 20, the apparatus 2000 includes a Central Processing Unit (CPU)2001, which can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)2002 or a program loaded from a storage section 2008 into a Random Access Memory (RAM) 2003. In the RAM 2003, various programs and data necessary for the operation of the device 2000 are also stored. The CPU2001, ROM 2002, and RAM 2003 are connected to each other via a bus 2004. An input/output (I/O) interface 2005 is also connected to bus 2004.

The following components are connected to the I/O interface 2005: an input portion 2006 including a keyboard, a mouse, and the like; an output portion 2007 including a display device such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 2008 including a hard disk and the like; and a communication section 2009 including a network interface card such as a LAN card, a modem, or the like. The communication section 2009 performs communication processing via a network such as the internet. Drive 2010 is also connected to I/O interface 2005 as needed. A removable medium 2011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 2010 as necessary, so that a computer program read out therefrom is mounted in the storage section 2008 as necessary.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 2009, and/or installed from the removable medium 2011. The above-described functions defined in the system of the present disclosure are executed when the computer program is executed by the Central Processing Unit (CPU) 2001.

It should be noted that the computer readable media shown in the present disclosure may be computer readable signal media or computer readable storage media or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer-readable signal medium may include a propagated data signal with computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present disclosure may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a historical data acquisition module, a current data prediction module, a weighted rate acquisition module, an exponential weight acquisition module, and a current smooth value acquisition module. The names of these modules do not constitute a limitation to the module itself in some cases, and for example, the historical data acquisition module may also be described as a "module that acquires historical time-series data from a connected server".

The embodiments of the present disclosure also provide a computer-readable medium, which may be included in the device described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: acquiring historical time sequence data in a target length moving window before a current time point; obtaining prediction data of a current time point according to historical time sequence data in a target length moving window; acquiring a weighted descending rate; determining the exponential decay weight of the prediction data of the current time point and the historical data of each historical time point in the historical time sequence data according to the weighted descending rate; obtaining a smooth value of the current time point according to the prediction data and the exponential decay weight thereof of the current time point, and the historical data and the exponential decay weight thereof of each historical time point in the historical time sequence data; obtaining the discrete distance measurement of the current time point according to the prediction data of the current time point, historical time sequence data in a target length moving window, the weighted descending rate and the smooth value of the current time point; and determining a prediction threshold value of the next time point based on the smooth value of the current time point and the discrete degree quantity of the current time point, so as to be used for carrying out abnormity detection on the data of the next time point.

Exemplary embodiments of the present disclosure are specifically illustrated and described above. It is to be understood that the present disclosure is not limited to the precise arrangements, instrumentalities, or instrumentalities described herein; on the contrary, the disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A time series data abnormality detection method is characterized by comprising the following steps:

acquiring historical time sequence data in a target length moving window before a current time point;

obtaining the prediction data of the current time point according to the historical time sequence data in the target length moving window;

acquiring a weighted descending rate;

determining the prediction data of the current time point and the exponential decay weight of the historical data of each historical time point in the historical time sequence data according to the weighted descending rate;

obtaining a smooth value of the current time point according to the prediction data and the exponential decay weight thereof of the current time point, and the historical data and the exponential decay weight thereof of each historical time point in the historical time sequence data;

obtaining a discrete distance measurement of the current time point according to the prediction data of the current time point, historical time sequence data in the target length moving window, the weighted descending rate and the smooth value of the current time point;

and determining a prediction threshold value of the next time point based on the smooth value of the current time point and the discrete degree quantity of the current time point, so as to be used for carrying out abnormity detection on the data of the next time point.

2. The method according to claim 1, wherein the obtaining the smoothed value at the current time point according to the prediction data at the current time point and the exponential decay weight thereof, and the historical data at each historical time point in the historical time series data and the exponential decay weight thereof comprises:

adding historical data of each historical time point in the historical time sequence data according to the exponential decay weight of the historical data to obtain a smooth value of the previous time point;

and weighting and adding the smooth value of the previous time point and the predicted data of the current time point according to the weighted descending rate to obtain the smooth value of the current time point.

3. The method of claim 2, wherein the weighted-adding the smoothed value at the previous time point and the predicted data at the current time point according to the weighted-down rate to obtain the smoothed value at the current time point comprises:

obtaining a deviation correction coefficient according to the weighted descending rate and the target length moving window;

and according to the weighted descending rate and the deviation correction coefficient, carrying out weighted addition on the smooth value of the previous time point and the predicted data of the current time point to obtain the smooth value of the current time point.

4. The method of claim 1, wherein obtaining the prediction data of the current time point from historical time series data within the target length moving window comprises:

obtaining the prediction data of the current time point by adopting a moving average method, an exponential weighted average method or an interpolation method according to the historical time sequence data in the target length moving window, wherein:

the moving average method includes:

selecting a target number of historical time sequence data from the historical time sequence data in the target length moving window;

obtaining an average value of the target number of historical time-series data as prediction data of the current time point;

the exponential weighted averaging method comprises:

acquiring historical time series data before the previous time point from the historical time series data in the target length moving window;

obtaining a smooth value of the previous time point according to historical time sequence data before the previous time point;

weighting and adding the smooth value of the last time point and the data of the last time point to obtain the predicted data of the current time point;

the interpolation method comprises the following steps:

fitting the historical time sequence data in the target length moving window to obtain a target interpolation function;

and obtaining the prediction data of the current time point according to the target interpolation function.

5. The method of claim 4, wherein obtaining the prediction data of the current time point by using a moving average method, an exponential weighted average method, or an interpolation method according to the historical time series data in the target length moving window comprises:

acquiring an input data prediction method, wherein the data prediction method is at least one of the moving average method, the exponential weighted average method and the interpolation method;

and according to the historical time sequence data in the target length moving window, adopting the input data prediction method to obtain the prediction data of the current time point.

6. The method of claim 1, wherein obtaining the weighted rate of decrease comprises:

determining a target half-life, which is a time unit required by the exponential decay weights to be accumulated to half of the total weight in time from back to front;

obtaining the weighted rate of decline based on the target half-life.

7. The method of claim 1, wherein the amount of dispersion of the current time point comprises a standard deviation of the current time point;

obtaining the discrete degree quantity of the current time point according to the prediction data of the current time point, the historical time sequence data in the target length moving window, the weighted descending rate and the smooth value of the current time point comprises:

respectively obtaining the deviation of the time sequence data in the target length moving window and the smooth value of the current time point;

and after squaring the deviation, weighting and adding according to the exponential decay weight of the corresponding time point, and then solving a square root to obtain the standard deviation of the current time point.

8. The method of claim 7, wherein the prediction threshold comprises an upper prediction limit;

the obtaining a prediction threshold for a next time point based on the smoothed value for the current time point and a dispersion measure comprises:

obtaining interval prediction parameters according to the current time point;

and adding the smooth value of the current time point and the product of the interval prediction parameter and the standard deviation of the current time point to obtain the prediction upper limit of the next time point.

9. The method of claim 1, further comprising:

acquiring time sequence data of an evaluation time window;

taking each time point in the evaluation time window as the current time point, and obtaining a prediction threshold value of the next time point;

obtaining abnormal prediction data from the time series data of the evaluation time window according to the prediction threshold value of the next time point of each time point in the evaluation time window;

according to the time sequence data in the evaluation time window, carrying out abnormity detection on the data of each time point in the evaluation time window to obtain reference abnormal data;

and obtaining a prediction threshold coverage index according to the prediction abnormal data and the reference abnormal data.

10. The method of claim 8, wherein the detecting anomalies in the data at each time point within the evaluation time window comprises:

constructing a data space of time series data within the evaluation time window;

dividing the data space into a plurality of subspaces through a random hyperplane, wherein each subspace of the plurality of subspaces only comprises data of one time point;

and determining the reference abnormal data from the data of each subspace according to the times of dividing the data space when each subspace is obtained.

11. A time-series data abnormality detection apparatus, characterized by comprising:

the historical data acquisition module is used for acquiring historical time series data in a target length moving window before the current time point;

the current data prediction module is used for obtaining prediction data of the current time point according to historical time sequence data in the target length moving window;

a weighted rate obtaining module, configured to obtain a weighted decreasing rate;

the exponential weight acquisition module is used for determining the prediction data of the current time point and the exponential decay weight of the historical data of each historical time point in the historical time sequence data according to the weighted descending rate;

a current smooth value obtaining module, configured to obtain a smooth value of the current time point according to the prediction data of the current time point and the exponential decay weight thereof, and historical data of each historical time point in the historical time series data and the exponential decay weight thereof;

a current discrete magnitude obtaining module, configured to obtain a discrete magnitude of the current time point according to the prediction data of the current time point, the historical time series data in the target length moving window, the weighted descent rate, and the smoothed value of the current time point;

and the threshold prediction module is used for determining a prediction threshold of the next time point based on the smooth value of the current time point and the discrete degree quantity of the current time point so as to detect the abnormality of the data of the next time point.

12. An electronic device, comprising: memory, processor and executable instructions stored in the memory and executable in the processor, characterized in that the processor implements the method according to any of claims 1-10 when executing the executable instructions.

13. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the executable instructions, when executed by a processor, implement the method of any of claims 1-10.