CN116361631A - Method and equipment for detecting time sequence data period, detecting abnormality and scheduling resources - Google Patents

Method and equipment for detecting time sequence data period, detecting abnormality and scheduling resources Download PDF

Info

Publication number
CN116361631A
CN116361631A CN202310350882.3A CN202310350882A CN116361631A CN 116361631 A CN116361631 A CN 116361631A CN 202310350882 A CN202310350882 A CN 202310350882A CN 116361631 A CN116361631 A CN 116361631A
Authority
CN
China
Prior art keywords
period
item
eigenmode function
eigenmode
noise
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310350882.3A
Other languages
Chinese (zh)
Inventor
陈启明
文青松
孙亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba China Co Ltd
Original Assignee
Alibaba China Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba China Co Ltd filed Critical Alibaba China Co Ltd
Priority to CN202310350882.3A priority Critical patent/CN116361631A/en
Publication of CN116361631A publication Critical patent/CN116361631A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/10Pre-processing; Data cleansing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/23Clustering techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/243Classification techniques relating to the number of classes
    • G06F18/2433Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection

Abstract

The application provides a method and equipment for detecting a time sequence data period, detecting abnormality and scheduling resources. According to the method, time series data to be detected are decomposed into first eigenmode functions; whether the first eigenmode function is a noise item and a trend item is checked, the noise item and the trend item in the first eigenmode function are removed, the reserved first eigenmode function is a period item, and the noise item and the trend item in the time sequence data are separated and removed by using a signal decomposition method, so that complex preprocessing is not needed, and the method is simpler, more efficient and more accurate; the periodic items are subjected to similar clustering according to the periods of the periodic items to obtain similar clusters, the periodic items in the same cluster are combined into second eigen-mode functions, so that similar second eigen-mode functions are aggregated, the mode aliasing effect can be relieved, the period of time series data is determined according to the period of the second eigen-mode functions, and the accuracy of period detection can be improved.

Description

Method and equipment for detecting time sequence data period, detecting abnormality and scheduling resources
Technical Field
The present disclosure relates to computer technology, and in particular, to a method and apparatus for time-series data period detection, anomaly detection, and resource scheduling.
Background
With the explosive growth of the internet and many applications, a large amount of time series data is generated by various devices, applications, systems, and the like. Many time series data have periodic characteristics, such as electrocardiographic signals or traffic flow during weekday commute peak hours, etc. Since periodicity is an important feature of time series data, periodicity detection is critical in many time series related tasks, such as in prediction, anomaly detection, classification, clustering, and data compression. Accurate periodic detection not only enables better processing of a single time series, but also facilitates analysis of multiple time series. For example, the similarity may be defined based on a periodic pattern between a plurality of time series, which information may further be used for classification and clustering tasks of the time series.
Accurate periodic detection is very challenging due to the diversity and complexity of real world time series data. At present, a typical period detection method is to remove trend items in time series data based on an HP filtering (Hodrick Prescott Filter) method in a preprocessing stage, then transform the time series data into a time-frequency domain by utilizing discrete wavelet transform, and order the period items on different time scales; the period of the time series data is estimated by using the periodic chart and the autocorrelation function. However, the accuracy of the periodic detection result of the method is seriously dependent on whether the value of the super parameter lambda is proper or not in the HP filtering method in the preprocessing stage, the super parameter lambda depends on human experience, the super parameter lambda is different when used in different scenes, and proper value is often difficult to find, so that the periodic detection method is poor in robustness, poor in universality and low in detection accuracy.
Disclosure of Invention
The application provides a method and equipment for detecting a time sequence data period, detecting abnormality and scheduling resources, which are used for solving the problems of poor robustness, poor universality and low detection accuracy of the conventional period detection method.
In a first aspect, the present application provides a period detection method of time-series data, including: acquiring time sequence data to be detected, and decomposing the time sequence data into a first eigenmode function; checking whether the first eigenmode function is a noise item and a trend item, removing the noise item and the trend item in the first eigenmode function according to a checking result, and taking the reserved first eigenmode function as a period item; according to the period of the period item, carrying out similar clustering on the period item to obtain a class cluster, and combining the period item in the same class cluster into a second eigenmode function; and determining the period of the time series data according to the period of the second eigenmode function.
In a second aspect, the present application provides an anomaly detection method, including: acquiring time sequence data of resource state information of target equipment, wherein the time sequence data comprises the resource state information of the target equipment at a plurality of historical time points; decomposing the time series data into first eigenmode functions; checking whether the first eigenmode function is a noise item and a trend item, removing the noise item and the trend item in the first eigenmode function according to a checking result, and taking the reserved first eigenmode function as a period item; according to the period of the period item, carrying out similar clustering on the period item to obtain a class cluster, and combining the period item in the same class cluster into a second eigenmode function; determining the period of the time sequence data according to the period of the second eigenmode function to obtain the period of the resource state information of the target equipment; and according to the period of the resource state information of the target equipment, carrying out periodic abnormality detection on the target equipment.
In a third aspect, the present application provides a resource scheduling method, including: acquiring time sequence data of resource use information of a target system, wherein the time sequence data comprises the resource use information of the target system at a plurality of historical time points; decomposing the time series data into first eigenmode functions; checking whether the first eigenmode function is a noise item and a trend item, removing the noise item and the trend item in the first eigenmode function according to a checking result, and taking the reserved first eigenmode function as a period item; according to the period of the period item, carrying out similar clustering on the period item to obtain a class cluster, and combining the period item in the same class cluster into a second eigenmode function; determining the period of the time sequence data according to the period of the second eigenmode function to obtain the period of the resource use information of the target system; and predicting the resource use condition of the target system in a future period according to the period of the resource use information of the target system, and carrying out resource scheduling according to the resource use condition of the target system in the future period.
In a fourth aspect, the present application provides a period detection method of time-series data, including: responding to the terminal side equipment to call a period detection interface, and receiving time sequence data to be detected, which is sent by the terminal side equipment; decomposing the time series data into first eigenmode functions; checking whether the first eigenmode function is a noise item and a trend item, removing the noise item and the trend item in the first eigenmode function according to a checking result, and taking the reserved first eigenmode function as a period item; according to the period of the period item, carrying out similar clustering on the period item to obtain a class cluster, and combining the period item in the same class cluster into a second eigenmode function; determining the period of the time series data according to the period of the second eigenmode function; and sending the period of the time series data to the end-side equipment.
In a fifth aspect, the present application provides a cloud server, including: a processor, and a memory communicatively coupled to the processor; the memory stores computer-executable instructions; the processor executes computer-executable instructions stored in the memory to implement the method of any of the above aspects.
In a sixth aspect, the present application provides a computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of the above aspects.
The method and the device for detecting the time sequence data period, detecting abnormality and scheduling resources are characterized in that the time sequence data to be detected is decomposed into a first eigenmode function; whether the first eigenmode function is a noise item and a trend item is checked, the noise item and the trend item in the first eigenmode function are removed according to the checking result, and compared with the preprocessing process of directly removing the trend item from the original time sequence data, the method for separating and removing the noise item and the trend item in the time sequence data by using the signal decomposition method is simpler, more efficient and more accurate; further, according to the period of the period item, performing similar clustering on the period item to obtain a class cluster, and combining the period item in the same class cluster into a second eigenmode function; according to the period of the second eigenmode function, the period of the time series data is determined, the similar second eigenmode function can be aggregated, and the mode aliasing effect of the signal decomposition method can be relieved, so that the accuracy of period detection is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
FIG. 1 is a system architecture diagram of an exemplary time series data period detection suitable for use herein;
FIG. 2 is a flowchart of a method for detecting a period of time series data according to an exemplary embodiment of the present application;
FIG. 3 is an exemplary graph of a first eigenmode function into which time series data provided by an exemplary embodiment of the present application is decomposed;
FIG. 4 is an exemplary diagram of new eigenmode functions after cluster merging according to an exemplary embodiment of the present application;
FIG. 5 is a detailed flow chart of period detection of time series data provided in an exemplary embodiment of the present application;
FIG. 6 is a flowchart of an anomaly detection method according to an exemplary embodiment of the present application;
FIG. 7 is a flowchart of a method for scheduling resources according to an exemplary embodiment of the present application;
FIG. 8 is an interactive flow chart of a cycle detection method according to an exemplary embodiment of the present application;
fig. 9 is a schematic structural diagram of a period detection device for time-series data according to an exemplary embodiment of the present application;
Fig. 10 is a schematic structural diagram of a cloud server according to an embodiment of the present application.
Specific embodiments thereof have been shown by way of example in the drawings and will herein be described in more detail. These drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the accompanying claims.
The terms referred to in this application are explained first:
time series data: is data collected at different points in time for the case where the described phenomenon varies with time. The time series data reflects the state or degree of change over time of something, a phenomenon, or the like. The time sequence data can be understood as a trend term, a period term and a noise term which are overlapped, and the trend term, the period term and the noise term can be obtained after reasonable function transformation and decomposition.
Noise term: is a random noise portion in the time series data.
Trend term: the time series data represent the continuous and long-term change of the sequence, and represent a development trend.
The period term: the part representing the data fluctuation caused by the periodical change in the time series data is a periodical feature.
Empirical mode decomposition (Empirical Mode Decomposition, EMD for short) is a signal decomposition method that decomposes an original signal (time series data) into a plurality of eigenmode functions.
The eigenmode function (Intrinsic Mode Function, IMF for short) is a time-series component into which the signal decomposition method decomposes the original signal.
Modal aliasing effects: it means that one eigenmode function (IMF) contains feature time scales that are very different, or that similar feature time scales are distributed among different eigenmode functions (IMFs). In this embodiment, the similar characteristic time scale distribution is mainly referred to as the modal aliasing effect in different eigenmode functions (IMFs).
Real world time series data has diversity and complexity and accurate periodic detection is very challenging. First, most of time-series data generated in practical applications are non-stationary, and these non-stationary factors, such as trend terms, and other components, can make periodic detection difficult. Second, the actual time series data tends to be nonlinear, and the periodic pattern is complex and dynamic. In one aspect, the periodic pattern may change or deviate from the typical pattern over time. For example, the periodic pattern of daily sales during holidays by a primary online shopping retailer may differ from its typical daily behavior. On the other hand, multiple staggered periodic patterns may occur simultaneously, which makes the problem more complex. For example, traffic congestion time series typically appear as daily and weekly periods, but weekly patterns may change when holidays occur. Third, different time series data typically contains noise of different types and different intensities. This makes developing a versatile robust periodic detection scheme quite challenging.
At present, a typical period detection method is to remove trend items in time series data based on an HP filtering (Hodrick Prescott Filter) method in a preprocessing stage, then transform the time series data into a time-frequency domain by utilizing discrete wavelet transform, and order the period items on different time scales; the period of the time series data is estimated by using the periodic chart and the autocorrelation function. However, the accuracy of the periodic detection result of the method is seriously dependent on whether the super-parameter value is proper or not in the HP filtering method in the preprocessing stage, the super-parameter depends on human experience, the super-parameter is different in use under different scenes, and proper value is often difficult to find, so that the periodic detection method is poor in robustness, poor in universality and low in detection accuracy.
The application provides a period detection method of time series data, which is characterized by decomposing the time series data to be detected into a first eigenmode function; whether the first eigenmode function is a noise item and a trend item is checked, the noise item and the trend item in the first eigenmode function are removed according to a checking result, and the reserved first eigenmode function is used as a period item, compared with a method for directly analyzing and searching original time sequence data and filtering the trend item through an HP filtering mode, the noise item and the trend item in the time sequence data are separated and removed through a signal decomposition method, and the method is simpler, more efficient and more accurate; further, according to the period of the period item, performing similar clustering on the period item to obtain a class cluster, and combining the period item in the same class cluster into a second eigenmode function; according to the period of the second eigenmode function, the period of the time series data is determined, the similar second eigenmode function can be aggregated, and the mode aliasing effect of the signal decomposition method can be relieved, so that the accuracy of period detection is improved.
The period detection method provided by the application can be applied to task scenes such as abnormality detection, resource use condition prediction, compression storage of information, classification and clustering of similar time series data, and other period detection of time series data, and other downstream task scenes based on period detection results, and is not particularly limited herein.
Fig. 1 is a system architecture diagram of an exemplary time-series data period detection applicable to the present application, and as shown in fig. 1, the system architecture may specifically include a cloud server and an end-side device.
The cloud server can be a server cluster arranged at the cloud end, communication links capable of being communicated are arranged between the cloud server and each terminal side device, and communication connection between the cloud server and each terminal side device can be achieved.
The terminal device may specifically be a hardware device with a network communication function, an operation function and an information display function, which includes, but is not limited to, a smart phone, a tablet computer, a desktop computer, an internet of things device, a server, and the like. The terminal side device can acquire the time series data to be detected and send the time series data to be detected to the cloud server.
In practical application, the end side device triggers the cloud server to execute period detection of the time series data by sending a period detection request/instruction to the cloud server or calling an application program interface provided by the cloud server and providing the time series data to be detected to the cloud server. The cloud server obtains time sequence data to be detected provided by the terminal side equipment based on a period detection request/instruction or an application program interface to be called, and decomposes the time sequence data into a first intrinsic mode function; whether the first eigenmode function is a noise item and a trend item is checked, the noise item and the trend item in the first eigenmode function are removed according to the checking result, and the reserved first eigenmode function is used as a period item; according to the period of the period item, carrying out similar clustering on the period item to obtain a class cluster, and combining the period item in the same class cluster into a second eigenmode function; a period of the time series data is determined based on the period of the second eigenmode function. And the cloud server sends the period of the time series data to the end-side equipment.
In addition, in some application scenarios, the cloud server may also perform subsequent processing logic, such as information prediction, anomaly detection, data compression storage, classification, clustering, and the like, of similar time-series data based on the period of the time-series data. The cloud server may perform setting and adjustment according to the needs of the specific task scenario according to the subsequent processing logic executed by the period detection result, which is not specifically limited herein.
The method of the present embodiment may be applied to task scenarios of anomaly detection of target devices/systems, for example. Specifically, the end-side device may acquire resource status information of the target device/system at a plurality of historical time points within a historical period, and arrange the resource status information in chronological order to form time-series data of the resource status information of the target device/system. Such as the utilization of resources such as a Central Processing Unit (CPU) of a cloud computing platform (e.g., a cloud native platform), memory, etc., the power load of a power system, etc. The end side device sends time series data of the resource state information of the target device/system to the cloud server. And the cloud server performs period detection according to the time sequence data of the resource state information of the target equipment/system to obtain the period of the resource state information of the target equipment/system. Based on the period of the resource status information of the target device/system, an abnormality detection function based on the resource status information of the target device/system can be realized. Alternatively, the cloud server may send the period of the resource status information of the target device/system to the end-side device, and the end-side device performs anomaly detection on the target device/system regularly according to the real-time resource status information of the target device/system. Optionally, the cloud server stores the period of resource status information of the target device/system. The terminal side equipment sends an abnormality detection request to the cloud server regularly and carries real-time resource state information of the target equipment/system. The cloud server detects whether the target device/system is abnormal or not according to the real-time resource state information of the target device/system and the period of the resource state information of the target device/system based on the abnormality detection request, and sends an abnormality detection result to the end-side device.
The method of the present embodiment may be applied to task scenarios of information prediction of a target device/system, for example, predicting resource usage of the target device/system in a future period based on a period of resource usage information of the target device/system, and performing resource scheduling. The resource usage information may be a remaining amount/usage amount of resources, a resource demand amount, a resource reserve amount, or the like. Such as the utilization rate of resources such as a CPU and a memory of the cloud computing platform, the power load of a power system, the goods delivery quantity of the electronic commerce platform and the like. Specifically, the end-side device may acquire resource usage information of the target device/system at a plurality of historical time points within a historical period, and arrange the resource usage information in chronological order to form time-series data of the resource usage information of the target device/system. The end-side device sends time-series data of the resource usage information of the target device/system to the cloud server. And the cloud server performs period detection according to the time sequence data of the resource use information of the target equipment/system to obtain the period of the resource use information of the target equipment/system. Based on the period of the resource usage information of the target device/system, the resource usage of the target device/system at a future time period can be predicted. Further, resource scheduling is performed according to the resource usage of the target device/system in the future period. Alternatively, the cloud server may send the resource usage of the target device/system in the future period to the end-side device, and the end-side device performs resource scheduling, such as increasing the resource reserve of the target device/system, according to the resource usage of the target device/system in the future period. Alternatively, the cloud server may schedule resources for the target device/system according to the resource usage of the target device/system in the future period.
The following describes the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems in detail with specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
Fig. 2 is a flowchart of a period detection method of time-series data according to an exemplary embodiment of the present application. The execution body of the embodiment is a cloud server in the system architecture. As shown in fig. 2, the method specifically comprises the following steps:
step S201, time series data to be detected are obtained, and the time series data are decomposed into first eigenmode functions.
In this embodiment, the time series data to be detected includes data of a plurality of time points, and the data of the plurality of time points are arranged into a sequence according to a time sequence, so as to form the time series data.
In the step, the state information of a plurality of different historical time points of the target equipment/system in the historical period is obtained, and the state information of each historical time point is arranged according to the time sequence to form time sequence data.
In practical application, the periodic detection result of the time series data can be applied to various task scenes such as information prediction, anomaly detection, data compression storage, classification, clustering and the like. When the method is applied to different task scenes, the information items specifically included in the state information of each time point included in the time series data of the period to be detected can be different, and the method can be specifically set and adjusted according to the actual needs of the scenes to be used, and is not specifically limited.
For example, the state information included in the time-series data may be the utilization rate of resources such as a CPU and a memory of the cloud computing platform, the power load of the power system, the shipment amount of goods of the electronic commerce platform, and the like.
In this embodiment, after the time-series data to be detected is obtained, the time-series data is decomposed into the first eigenmode function. The step specifically adopts a signal decomposition method capable of separating noise items and trend items in time series data, and specifically adopts any one of the following signal decomposition methods: empirical Mode Decomposition (EMD) method, intrinsic Time-scale decomposition (ITD) method, local mean decomposition (Local Mean Decomposition, LMD) method, etc.
It should be noted that, in this embodiment, "the first eigenmode function" refers to an eigenmode function into which the time-series data to be detected is decomposed, and also refers to a time-series component. Illustratively, the time series data is decomposed using an Empirical Mode Decomposition (EMD) method, resulting in a plurality of eigenmode functions IMFs and residual terms (also referred to as residual terms). In this embodiment, the remainder may be used as an eigenmode function, where the first eigenmode function includes a plurality of time-series components into which the time-series data to be detected is decomposed, and includes the eigenmode function and the remainder. In addition, the remainder may be removed, and the first eigenmode function only includes the eigenmode function IMF obtained by decomposition, and does not include the remainder.
Step S202, whether the first eigenmode function is a noise item and a trend item is checked, the noise item and the trend item in the first eigenmode function are removed according to the checking result, and the reserved first eigenmode function is used as a period item.
After decomposing the time series data to obtain first eigen-mode functions, for each first eigen-mode function, checking whether the first eigen-mode function is a noise item and a trend item, removing the noise item and the trend item in the first eigen-mode function according to the checking result, and taking the reserved first eigen-mode function (a non-noise item and a non-trend item) as a period item.
In this embodiment, by decomposing the time-series data into the first eigenmode functions by using the signal decomposition method, separating the noise term, the trend term and the period term in the time-series data, and then combining the noise test method and the trend term test method, it is tested which of the first eigenmode functions obtained by decomposition are the noise term and which are the trend term, and the period term in the first eigenmode functions can be determined by excluding the noise term and the trend term. Compared with the preprocessing process of directly analyzing and searching the original time sequence data and filtering the trend items through the HP filtering mode in the prior art, the method and the device for removing the noise items and the trend items in the time sequence data by utilizing the signal decomposition method are simpler, more efficient and more accurate in removing the noise items and the trend items, and obtain the period items in the time sequence data.
Alternatively, the checking whether any of the first eigenmode functions is a noise term in this step may be implemented using any method of detecting whether the time series is noise in the prior art, for example, any one of a young-bolg-Box (Ljung-Box) check, a bruse-goldfrey check (breeusch-Godfrey test), a Durbin-Watson check (Durbin-Watson test) check may be used to check whether the first eigenmode function is a noise term, and other noise checking methods may be used currently, which are not specifically limited herein.
Optionally, the step of checking whether any first eigenmode function is a trend term may be implemented by using any unit root checking method used for checking time series stationarity in the prior art, which indicates that the first eigenmode function is a trend term if the checking result is that the first eigenmode function is not stationary, and indicates that the first eigenmode function is not a trend term but a period term if the checking result is that the first eigenmode function is stationary. For example, any one of an Augmented di-fowler (Augmented Dickey-Fuller) test, a PP (philips-Perron) test, a KPSS (kweatkowski-philips-Schmidt-Shim) test may be used, and other unit root test methods, or other time-series stationarity test methods may be used, without being particularly limited thereto.
And step S203, performing similar clustering on the periodic items according to the period of the periodic items to obtain clusters, and combining the periodic items in the same cluster into a second eigenmode function.
After determining the period item in the first eigenmode function, calculating the period of the period item, performing similar clustering on the period item according to the period length of the period item, and gathering the period item with the similar period into the same class cluster to obtain at least one class cluster, wherein one class cluster comprises one or more period items. Further, the periodic entries in the same class of clusters are combined into a new eigenmode function, called the second eigenmode function.
Often, when signal decomposition is performed, insufficient decomposition results in aliasing effects that include characteristic time scales with great differences in one eigenmode function (IMF), and when the decomposition is excessive, aliasing effects with similar characteristic time scales distributed in different eigenmode functions (IMF) occur.
In this embodiment, the time-series data to be detected is fully decomposed in step S202, and the obtained first eigenmode function (IMF) may have similar characteristic time scales distributed in different eigenmode functions (IMFs). In order to avoid that the modal aliasing effect of similar characteristic time scale distribution in different eigenmode functions (IMFs) affects the accuracy of the period detection result, in this embodiment, according to the period length of period items in the eigenmode functions (IMFs), period items with similar periods are gathered into the same cluster, and period items in the same cluster are combined into a new second eigenmode function, so that the modal aliasing effect can be effectively relieved, and the accuracy of the period detection result can be improved.
Step S204, determining a period detection result of the time series data according to the period of the second eigenmode function.
In the step, a Fourier transform method is utilized to calculate the period of the second eigenmode function, and the period detection result of the time series data is determined according to the period of the second eigenmode function. For example, in this step, the period of the second eigenmode function may be used as a period detection result of the time series data, a plurality of periods of the time series data may be detected, and a situation in which a plurality of staggered period modes occur simultaneously may be effectively addressed.
The method for detecting the period of the time series data can be applied to tasks such as information prediction based on the data period, anomaly detection, data compression storage, classification and clustering of similar time series data.
In an alternative embodiment, after determining the period detection result of the time series data, the cloud server may further execute subsequent processing logic to complete the corresponding task, so that the accuracy of task execution may be improved.
For example, the cloud server may predict the state information of the target device/system at a future point in time according to the period of the time series data of the target device/system, so as to improve the accuracy of the state information prediction at the future point in time.
For example, the cloud server may perform anomaly detection on the target device/system according to the period of the time-series data of the target device/system, so as to improve the accuracy of periodic anomaly detection.
For example, the cloud server may compress and store the state information of the target device/system according to the period of the time-series data of the target device/system, so as to improve the quality and effectiveness of the data after the state information is compressed and stored.
In the method of the embodiment, when the time series data is periodically detected, the time series data is decomposed into a first eigenmode function, noise items, trend items and period items in the time series data are separated, then the noise items and trend items in the first eigenmode function are detected by combining a noise detection method and a trend item detection method, and the noise items and the trend items are removed, so that the period items in the time series data can be determined; compared with the preprocessing process of directly analyzing and determining trend item filtering trend item and outlier processing on original time series data in the prior art, the method for processing the time series data by using the signal decomposition method in the embodiment separates and removes noise item and trend item in the time series data, and can remove the noise item and trend item more simply, efficiently and accurately to obtain period item in the time series data. Further, according to the cycle length of the cycle items in the Intrinsic Mode Function (IMF), the cycle items with similar cycles are gathered into the same cluster, the cycle items in the same cluster are combined into a new second intrinsic mode function, so that the mode aliasing effect of signal decomposition can be effectively relieved, the cycle detection result of time series data is determined according to the cycle of the second intrinsic mode function, the precision of the cycle detection result can be improved, and the method is suitable for cycle detection of various task scenes.
In an optional embodiment, in step S202, it is checked whether any of the first eigenmode functions is a noise term, which may be implemented in the following manner:
calculating the arrangement entropy of the eigenmode function, and carrying out noise inspection on the eigenmode function by using a noise inspection method; if the detection result of the noise detection method is noise or the arrangement entropy of the eigenmode function is greater than or equal to the arrangement entropy threshold value, determining that the eigenmode function is a noise item; if the result of the noise test is not noise and the arrangement entropy of the eigenmode function is smaller than the arrangement entropy threshold, determining that the eigenmode function is not a noise item. The eigenmode function is any first eigenmode function.
Wherein the permutation entropy threshold may be set to a fixed value according to a specific task scenario and an empirical value, the permutation entropy of the time series is generally a value between 0 and 1, the permutation entropy threshold may be set to 0.8, 0.85, etc., and specific values of the permutation entropy threshold are not particularly limited herein.
The permutation entropy of the time sequence can measure the complexity of the time sequence, in this embodiment, the permutation entropy of the first eigenmode function is calculated to measure the complexity of the first eigenmode function, if the permutation entropy of the first eigenmode function is greater than or equal to the permutation entropy threshold, it is indicated that the permutation entropy of the first eigenmode function is too large, the complexity of the first eigenmode function is too high, and it is indicated that the first eigenmode function is noise.
In this embodiment, the noise term in the first eigenmode function obtained by decomposition is detected by combining the noise test method and the permutation entropy, so that the noise term existing in the first eigenmode function can be accurately detected, and the missing noise term is prevented from affecting the accuracy of period detection.
When the noise test is performed on the first eigenmode function using the noise test method, noise tests that may be used include, but are not limited to, the poplar-borg-Box test, the bruse-goldfrey test (Breusch-Godfrey test), the Durbin-Watson test (Durbin-Watson test) test. Alternatively, the first eigenmode function may be noise checked using any noise checking method. Optionally, the noise test method may be used to perform noise test on the first eigen mode function respectively, and test results of the multiple noise test methods are synthesized, and when at least one test result is a noise item, it is determined that the test result of the noise test method is a noise item, so that accuracy of period test may be improved.
In an optional embodiment, in step S202, it is checked whether any of the first eigenmode functions is a trend term, which may be implemented in the following manner:
Checking whether the eigenmode function is stationary using at least one sequence stationarity check; if the test result of at least one sequence stability test method is unstable, determining that the eigen-mode function is a trend term; if the test results of the sequence stability test method are all stable, determining that the eigenmode function is not a trend item. The eigenmode function is any first eigenmode function.
In this embodiment, whether the eigenmode function is stable or not is checked by using a plurality of sequence stationarity checking methods, so that a trend term in the first eigenmode function obtained by decomposition can be checked as much as possible, the missing trend term is prevented from affecting the accuracy of period detection, and the accuracy of period checking can be improved.
In examining whether any of the first eigenmode functions is a trend term, sequence stationarity examination methods that may be used include, but are not limited to, the Augmented di-fueller (Augmented di-ckey-Fuller) examination, the PP (philips-Perron) examination, the KPSS (kweatkowski-philips-Schmidt-Shim) examination.
In an optional embodiment, in step S203, similar clustering is performed on the periodic items according to the period of the periodic items to obtain clusters, and the periodic items in the same cluster are combined into the second eigenmode function, which may be implemented specifically as follows:
The period items are arranged according to the sequence from small to large, and the following processing is carried out on the period items in sequence: if the current period item is not clustered into any cluster, the current period item and the period item which meets the preset condition later are clustered into the same cluster; if a class cluster containing a plurality of periodic items exists, the periodic items in the same class cluster are combined to obtain a new eigen-mode function; and performing similar clustering on the new eigenmode functions until a cluster containing a plurality of eigenmode functions does not exist in a clustering result, and taking the new eigenmode functions obtained currently as second eigenmode functions.
Specifically, the period items are arranged in the order of the period from small to large, each period item is processed in turn from the first period item, and the current period item is added to a certain cluster. Specifically, the method includes two cases, in which a current period item is not added to any cluster, a new cluster needs to be created, the current period item and a period item which is after the current period item and meets a preset condition are added to the new cluster, and the next period item is continuously processed. In another case, the current period item is added to a certain cluster, and the next period item is continuously processed until a round of clustering is completed after each period item is traversed. After a round of clustering is completed, if a class cluster containing a plurality of periodic items exists in the class clusters of the round of clustering result (namely, the periodic items needing to be combined exist), the periodic items in the same class cluster are combined to obtain a new eigen-mode function. For a class cluster containing only one period item, the period item in the class cluster is used as a new eigenmode function. And summing the plurality of periodic items in the class cluster for the class cluster containing the plurality of periodic items to obtain a new eigenmode function. After the period item combination is completed, the new eigenmode functions obtained in the current round are clustered in the next round until a cluster containing a plurality of period items does not exist in the clustering result of a certain round, namely, the cluster is stopped when the eigenmode functions do not exist and need to be combined, and the eigenmode function obtained in the last round is used as a second eigenmode function.
Wherein the preset condition is that the period is less than or equal to 2 times of the period of the current period item. Since the period of the period item is arranged from small to large, in addition, the preset condition can be set or adjusted according to the specific application scenario, which is not particularly limited herein.
For example, fig. 3 is a schematic diagram of a first eigenmode function IMF obtained by decomposing time-series data x, and as shown in fig. 3, the time-series data x is decomposed into 5 eigenmode functions IMF1-IMF5 and one residual term, and in this example, 6 components obtained by decomposition are all taken as the first eigenmode function (the residual term is denoted as IMF 6). The IMF1 is checked as a noise item, the IMF6 is a trend item, the IMF2-IMF5 is a period item, and the periods of the IMF2 to the IMF5 are sequentially increased. The period of the IMF5 is smaller than 2 times of the period of the IMF4, the period of the IMF5 is similar to the period of the IMF4, the IMF4 and the IMF5 are clustered into the same class of clusters through similar clustering of period items, and other period items are independently in one class of clusters. And combining the IMF4 and the IMF5 in the same cluster to obtain a new eigenmode function IMF4+5 shown in FIG. 4. The combined imf4+5 in fig. 4 has a much more regular cycle than IMF4 and IMF 5. The data of each time point in the time series data is taken as a sample point, the abscissa in the graphs shown in fig. 3 and 4 is sampling time information of the sample point, and the ordinate is the numerical value of the sample point, and also the state information in the time series data.
In order to avoid that the modal aliasing effect of similar characteristic time scale distribution in different eigenmode functions (IMFs) affects the accuracy of the period detection result, in this embodiment, according to the period length of period items in the eigenmode functions (IMFs) and preset conditions, period items with similar periods are gathered into the same kind of cluster, and period items in the same kind of cluster are combined into a new second eigenmode function, so that the modal aliasing effect can be effectively relieved, and the accuracy of the period detection result can be improved.
In an alternative embodiment, the determining the period of the time-series data in step S204 according to the period of the second eigenmode function may be implemented in the following manner: subtracting trend items in the first eigenmode function from the time sequence data to obtain difference value sequence data; calculating the correlation between the second eigenmode function and the difference sequence data; and determining the period of at least one second eigenmode function as the period of the time series data according to the correlation between the second eigenmode function and the difference series data.
For example, a correlation threshold may be set, and a period of the second eigenmode function having a correlation with the difference sequence data greater than or equal to the correlation threshold is determined as a period of the time sequence data based on the correlation between the second eigenmode function and the difference sequence data and the correlation threshold. Wherein the correlation threshold may be set and fixed by the developer prior to being online, for example the correlation threshold may be set to 0.7. In addition, the correlation threshold controls the number of cycles of the output. The larger the correlation threshold, the fewer the number of second eigenmode functions satisfying the correlation with the difference sequence data greater than or equal to the correlation threshold may be, and the fewer the number of output cycles may be; conversely, the smaller the correlation threshold, the greater the number of cycles output may be. The correlation threshold has definite physical meaning, is easy to determine and adjust, and supports the user to customize the correlation threshold so as to control the number of output periods according to the user requirement.
Alternatively, in calculating the correlation between the second eigenmode function and the difference sequence data, pearson (Pearson) correlation coefficients between the second eigenmode function and the difference sequence data may be calculated.
Alternatively, when calculating the correlation between the second eigenmode function and the difference sequence data, an energy ratio of the second eigenmode function to the difference sequence data may be calculated. In an actual application scenario, the second eigenmode function and the difference sequence data are both discrete time sequences, the discrete time sequences can be understood as discrete time signals, and the energy of the discrete time sequences is obtained by calculating the square sum of values at each discrete time point in the time sequences.
In this embodiment, the time series data is subtracted by the trend term in the first eigenmode function to obtain the difference sequence data, the difference sequence data is obtained by removing the trend term, is a time series dominated by the period term, and outputs the period of the second eigenmode function strongly related to the difference sequence data according to the correlation between the second eigenmode function and the difference sequence data, so that the period with higher accuracy can be output, and the number of the output periods can be flexibly controlled.
In an optional embodiment, in the step S204, when determining the period of the time series data according to the period of the second eigenmode function, it may be checked whether the second eigenmode function is a noise term and a trend term, and the period of the time series data may be determined according to the period of the second eigenmode function that is not the noise term and is not the trend term.
Specifically, after the second eigenmode function is obtained, whether the second eigenmode function is a noise term and a trend term may be checked, and when the checking confirms that the second eigenmode function is a period term, the period of the time-series data is determined according to the period of the second eigenmode function. And when the noise item or the trend item exists in the second eigenmode function through verification, removing the noise item or the trend item in the second eigenmode function, and reserving the second eigenmode function which is not the noise item and is not the trend item, namely obtaining the period item in the second eigenmode function, and determining the period of the time sequence data according to the period of the second eigenmode function which is not the noise item and is not the trend item.
In this embodiment, whether any of the second eigen-mode functions is a noise term is checked, which is the same as the specific implementation manner of checking whether the first eigen-mode function is a noise term, and will not be described here again.
Whether any of the second eigenmode functions is a trend term is the same as the specific implementation manner of whether the first eigenmode function is a trend term, and will not be described in detail here.
Illustratively, fig. 5 is a detailed flowchart of period detection of time series data according to an exemplary embodiment of the present application, and as shown in fig. 5, the detailed flowchart of period detection of time series data is as follows:
step S501, acquiring time series data to be detected.
Step S502, decomposing the time series data into first eigenmode functions.
Steps S501-S502 are similar to the implementation of step S201, and detailed descriptions thereof are omitted herein for reference.
For example, an Empirical Mode Decomposition (EMD) method may be used in this step to decompose the time series data into first eigenmode functions.
Step S503, performing noise test on the first eigenmode function by using a noise test method, and performing noise test according to the arrangement entropy of the first eigenmode function.
Illustratively, in this step, a poplar-borg-Box (Ljung-Box) test method may be used to perform noise test on the first eigenmode function, and the permutation entropy of the first eigenmode function may be calculated, where the first eigenmode function is determined to be a noise term when the test result of the poplar-borg-Box (Ljung-Box) test method is noise or the permutation entropy of the first eigenmode function is greater than or equal to the permutation entropy threshold.
Step S504, checking whether the first eigenmode function is a trend term.
For example, an Augmented di-fueller (Augmented di-fueller) test may be used in this step to verify whether the first eigenmode function is stationary, and the first eigenmode function that is not stationary is determined as the trend term.
Step S505, removing noise items and trend items in the first eigenmode functions, taking the reserved first eigenmode functions as period items, and calculating the periods of the period items.
The noise term and the trend term in the first eigenmode function are checked through the steps S503 and S504, respectively, where the noise term and the trend term are removed, and the remaining first eigenmode function is regarded as a period term.
The period of the period term may be calculated in this step by means of fourier transformation, for example.
Step S506, according to the period of the period item, similar clustering is carried out on the period item to obtain a class cluster, and the period items in the same class cluster are combined into a new eigenmode function.
Specifically, the period items are arranged in the order of the period from small to large, each period item is processed in turn from the first period item, and the current period item is added to a certain cluster. Specifically, the method includes two cases, in which a current period item is not added to any cluster, a new cluster needs to be created, the current period item and a period item which is after the current period item and meets a preset condition are added to the new cluster, and the next period item is continuously processed. In another case, the current period item is added to a certain cluster, and the next period item is continuously processed until a round of clustering is completed after each period item is traversed. After a round of clustering is completed, if a class cluster containing a plurality of periodic items exists in the class clusters of the round of clustering result (namely, the periodic items needing to be combined exist), the periodic items in the same class cluster are combined to obtain a new eigen-mode function.
For a class cluster containing only one period item, the period item in the class cluster is used as a new eigenmode function. And summing the plurality of periodic items in the class cluster for the class cluster containing the plurality of periodic items to obtain a new eigenmode function.
Wherein the preset condition is that the period is less than or equal to 2 times of the period of the current period item. Since the period of the period item is arranged from small to large, in addition, the preset condition can be set or adjusted according to the specific application scenario, which is not particularly limited herein.
Step S507, checking whether the new eigenmode function is a noise item and a trend item, and obtaining a period item in the new eigenmode function.
In this step, whether the new eigenmode function is a noise term may be checked by using the above-mentioned step S503 to check whether the first eigenmode function is a similar implementation manner of the noise term, in this step, whether the new eigenmode function is a trend term may be checked by using the above-mentioned step S504 to check whether the first eigenmode function is a similar implementation manner of the trend term, which will not be described herein. Further, noise terms and trend terms in the new eigenmode functions are removed, and the reserved new eigenmode functions are regarded as periodic terms.
Step S508, calculating the period of the period term in the new eigenmode function.
The period of the period term in the new eigenmode function may be calculated in this step by means of fourier transformation, for example.
Step S509, judging whether the clustering needs to be continued.
For example, if a cluster including a plurality of periodic items (eigenmode functions) exists in the previous clustering result according to the previous round of clustering result, the clustering needs to be continued, and step S506 is continuously performed on the periodic items in the new eigenmode functions obtained in step S507.
If there is no cluster containing a plurality of periodic terms (eigen-mode functions) in the previous round of clustering results, then no further clustering is needed, and steps S510-S512 are performed.
Step S510, taking the new eigenmode function obtained currently as a second eigenmode function.
Step S511, subtracting trend items in the first eigenmode function from the time sequence data to obtain difference value sequence data; and calculating the correlation between the second eigenmode function and the difference sequence data, and determining at least one second eigenmode function as an eigenmode function with strong correlation.
For example, a correlation threshold may be set, and a period of the second eigenmode function having a correlation with the difference sequence data greater than or equal to the correlation threshold is determined as a period of the time sequence data based on the correlation between the second eigenmode function and the difference sequence data and the correlation threshold.
Wherein the correlation threshold may be set and fixed by the developer prior to being online, for example the correlation threshold may be set to 0.7. In addition, the correlation threshold controls the number of cycles of the output. The larger the correlation threshold, the fewer the number of second eigenmode functions satisfying the correlation with the difference sequence data greater than or equal to the correlation threshold may be, and the fewer the number of output cycles may be; conversely, the smaller the correlation threshold, the greater the number of cycles output may be. The correlation threshold has definite physical meaning, is easy to determine and adjust, and supports the user to customize the correlation threshold so as to control the number of output periods according to the user requirement.
Alternatively, in calculating the correlation between the second eigenmode function and the difference sequence data, pearson (Pearson) correlation coefficients between the second eigenmode function and the difference sequence data may be calculated.
Alternatively, when calculating the correlation between the second eigenmode function and the difference sequence data, an energy ratio of the second eigenmode function to the difference sequence data may be calculated. In an actual application scenario, the second eigenmode function and the difference sequence data are both discrete time sequences, the discrete time sequences can be understood as discrete time signals, and the energy of the discrete time sequences is obtained by calculating the square sum of values at each discrete time point in the time sequences.
Step S512, calculating the period of the strongly correlated eigenmode function as the period of the time series data.
The period of the strongly correlated eigenmode functions may be calculated in this step by means of fourier transformation, for example.
The method of the embodiment includes decomposing time series data into a first eigenmode function, separating a noise item, a trend item and a period item in the time series data, and then combining a noise checking method and a trend item checking method to check out the noise item and the trend item in the first eigenmode function, and eliminating the noise item and the trend item to determine the period item in the time series data; compared with the preprocessing process of directly analyzing and determining trend item filtering trend item and outlier processing on original time series data in the prior art, the method for processing the time series data by using the signal decomposition method in the embodiment separates and removes noise item and trend item in the time series data, and can remove the noise item and trend item more simply, efficiently and accurately to obtain period item in the time series data. Further, according to the cycle length of the cycle items in the Intrinsic Mode Function (IMF), the cycle items with similar cycles are gathered into the same cluster, the cycle items in the same cluster are combined into a new second intrinsic mode function, so that the mode aliasing effect of signal decomposition can be effectively relieved, the cycle detection result of time series data is determined according to the cycle of the second intrinsic mode function, the precision of the cycle detection result can be improved, and the method is suitable for cycle detection of various task scenes.
Illustratively, fig. 6 is a flowchart of an anomaly detection method according to an exemplary embodiment of the present application. The anomaly detection method detects the period of the time-series data by using the period detection method provided by any one of the method embodiments, and further performs anomaly detection based on the period of the time-series data. As shown in fig. 6, the method specifically comprises the following steps:
step S601, obtaining time series data of resource status information of a target device, where the time series data includes resource status information of the target device at a plurality of historical time points.
The resource status information of the target device includes, but is not limited to, the remaining amount/usage of the resource, the resource demand, the resource reserve, the running status (failure status, normal status), index information indicating the status of the resource, and the like. Such as the utilization of resources such as the CPU and memory of the cloud computing platform, the power load of the power system, and the like.
Step S602, decomposing the time series data into first eigenmode functions.
Step S603, checking whether the first eigenmode function is a noise item and a trend item, removing the noise item and the trend item in the first eigenmode function according to the checking result, and taking the reserved first eigenmode function as a period item.
Step S604, performing similar clustering on the periodic items according to the period of the periodic items to obtain clusters, and combining the periodic items in the same cluster into a second eigenmode function.
Step S605, determining the period of the time series data according to the period of the second eigenmode function, and obtaining the period of the resource state information of the target device.
In this embodiment, the steps S602 to S605 may be specifically implemented by the period detection method of time series data provided in any of the foregoing embodiments, and details of the foregoing embodiments are specifically referred to herein and are not repeated. In this embodiment, the period of time-series data of the resource status information of the target device is regarded as the period of the resource status information of the target device.
Step S606, according to the period of the resource state information of the target device, the periodic anomaly detection is carried out on the target device.
After obtaining the period of the resource state information of the target device, the cloud server may perform periodic anomaly detection on the target device according to the period of the resource state information of the target device.
In this step, any existing method for detecting the abnormality of the periodic item may be specifically used, and the abnormality of the target device is detected based on the period of the resource status information of the target device, which is not specifically limited herein.
For example, the cloud server may store a period of resource status information for the target device/system. The terminal side equipment sends an abnormality detection request to the cloud server regularly and carries real-time resource state information of the target equipment/system. The cloud server detects whether the target device/system is abnormal or not according to the real-time resource state information of the target device/system and the period of the resource state information of the target device/system based on the abnormality detection request, and sends an abnormality detection result to the end-side device.
In this embodiment, the period detection method based on time-series data realizes an anomaly detection method, and by accurately detecting the period of time-series data of the resource status information of the target device and performing periodic anomaly detection on the target device according to the period of the resource status information of the target device, the accuracy and the effect of periodic anomaly detection can be improved.
Illustratively, fig. 7 is a flowchart of a resource scheduling method according to an exemplary embodiment of the present application. The resource scheduling method utilizes the period detection method provided by any one of the method embodiments to detect the period of the time sequence data, and further realizes the function of resource scheduling based on the period of the time sequence data. As shown in fig. 7, the method specifically comprises the following steps:
Step S701, obtaining time series data of resource usage information of a target system, where the time series data includes resource usage information of the target system at a plurality of history time points.
The resource usage information of the target system includes, but is not limited to, the remaining amount/usage amount of the resource, the resource demand amount, the resource reserve amount, and the like. Such as the utilization of resources such as the CPU and memory of the cloud computing platform, the power load of the power system, and the like.
Step S702, decomposing the time series data into first eigenmode functions.
Step S703, checking whether the first eigenmode function is a noise item and a trend item, and removing the noise item and the trend item in the first eigenmode function according to the checking result, wherein the reserved first eigenmode function is used as a period item.
Step S704, performing similar clustering on the periodic items according to the period of the periodic items to obtain clusters, and combining the periodic items in the same cluster into a second eigenmode function.
Step S705, determining the period of the time series data according to the period of the second eigenmode function, and obtaining the period of the resource use information of the target system.
In this embodiment, the steps S702 to S705 may be specifically implemented by the period detection method of time series data provided in any of the foregoing embodiments, and details of the foregoing embodiments are specifically referred to herein and are not repeated. In this embodiment, the period of time-series data of the resource usage information of the target system is set as the period of the resource usage information of the target system.
Step S706, predicting the resource usage of the target system in the future period according to the period of the resource usage information of the target system, and performing resource scheduling according to the resource usage of the target system in the future period.
After the period of the resource usage information of the target system is obtained, the cloud server can predict the resource usage condition of the target system in the future period according to the period of the resource usage information of the target system, and schedule the resource according to the resource usage condition of the target system in the future period.
For example, when the resource reserve is determined to be insufficient, the resource scheduling is performed to increase the resource reserve of the target system in the future period, so as to satisfy the resource demand of the target system in the future period.
In this embodiment, a method for scheduling resources is implemented by using a period detection method based on time sequence data, which accurately detects a period of resource usage information of a target system, predicts a resource usage condition of the target system in a future period according to the period of the resource usage information of the target system, and performs resource scheduling, so that rationality and effectiveness of resource scheduling can be improved, and resource utilization rate is improved.
Illustratively, fig. 8 is an interactive flowchart of a cycle detection method according to an exemplary embodiment of the present application. As shown in fig. 8, the method specifically comprises the following steps:
step S801, the end-side device invokes a period detection interface provided by the cloud server, and sends time sequence data to be detected to the cloud server.
The period detection interface is an application program interface of the cloud server for providing period detection service of time series data. And triggering the cloud server to execute the method for detecting the period of the time series data by other devices through calling the period detection interface.
Step S802, responding to a period detection interface called by the terminal side equipment, and receiving time series data to be detected sent by the terminal side equipment by the cloud server.
Step 803, the cloud server decomposes the time series data into a first eigenmode function.
Step S804, the cloud server checks whether the first eigenmode function is a noise item and a trend item, and removes the noise item and the trend item in the first eigenmode function according to the checking result, and the reserved first eigenmode function is used as a period item.
And step 805, the cloud server performs similar clustering on the periodic items according to the period of the periodic items to obtain class clusters, and merges the periodic items in the same class cluster into a second eigen-mode function.
Step S806, the cloud server determines the period of the time series data according to the period of the second eigenmode function.
The steps S803 to S806 may be specifically implemented by the period detection method of time series data provided in any of the foregoing embodiments, and detailed descriptions thereof are omitted herein.
Step S807, the cloud server transmits the period of the time-series data to the end-side device.
Step S808, the end-side device receives the period of the time-series data.
In step S809, the terminal device executes subsequent processing logic according to the period of the time-series data.
Subsequent processing logic that the end-side device may execute according to the period of time series data includes, but is not limited to: information prediction, anomaly detection and data compression storage.
For example, the end-side device may predict the state information of the target device at a future point in time based on the period of the time-series data.
For example, the end-side device may perform anomaly detection on the target device according to the period of the time-series data.
For example, the end-side device may compress and store the state information of the target device according to the period of the time-series data. For example, in the case where the state information of the target device is periodically repeated, the state information in one period of the target device may be stored, and the number of storage may be greatly compressed, saving storage space.
The cloud server may detect the period of time series data of fault related information of multiple devices in the same system, and according to the period of time series data of fault related information of different devices, use different devices with similar periods of time series data of fault related information as a device set with related faults, where when one device in the set fails, other devices in the same set are more likely to fail, and may be used in the situations of fault detection, system operation and maintenance, and the like.
Fig. 9 is a schematic structural diagram of a period detection device for time-series data according to an exemplary embodiment of the present application. The period detection device for time series data provided by the embodiment of the application can execute the processing flow provided by the period detection method embodiment for time series data. As shown in fig. 9, the period detection device 90 of the time-series data includes: a time series acquisition module 91, a time series decomposition module 92, a verification module 93, a similarity clustering module 94, and a period determination module 95.
Wherein, the time sequence acquisition module 91 is configured to acquire time sequence data to be detected. The time series decomposition module 92 is configured to decompose the time series data into a first eigenmode function. The checking module 93 is configured to check whether the first eigenmode function is a noise term and a trend term, remove the noise term and the trend term in the first eigenmode function according to the checking result, and keep the first eigenmode function as a period term. The similarity clustering module 94 is configured to perform similarity clustering on the periodic items according to the period of the periodic items to obtain clusters, and combine the periodic items in the same cluster into a second eigenmode function. The period determination module 95 is configured to determine a period of the time-series data according to the period of the second eigenmode function.
In an alternative embodiment, when implementing similar clustering of the periodic items according to the period of the periodic items to obtain clusters, and merging the periodic items in the same cluster into the second eigenmode function, the similar clustering module 94 is further configured to:
the period items are arranged according to the sequence from small to large, and the following processing is carried out on the period items in sequence: if the current period item is not clustered into any cluster, the current period item and the period item which meets the preset condition later are clustered into the same cluster; if a class cluster containing a plurality of periodic items exists, the periodic items in the same class cluster are combined to obtain a new eigen-mode function; and performing similar clustering on the new eigenmode functions until a cluster containing a plurality of eigenmode functions does not exist in a clustering result, and taking the new eigenmode functions obtained currently as second eigenmode functions.
In an alternative embodiment, the preset condition is that the period is less than or equal to 2 times the period of the current period item.
In an alternative embodiment, in implementing the determining of the period of the time series data according to the period of the second eigenmode function, the period determining module 95 is further configured to: subtracting trend items in the first eigenmode function from the time sequence data to obtain difference value sequence data; calculating the correlation between the second eigenmode function and the difference sequence data; and determining the period of at least one second eigenmode function as the period of the time series data according to the correlation between the second eigenmode function and the difference series data.
In an alternative embodiment, in implementing the calculation of the correlation between the second eigenmode function and the difference sequence data, the period determining module 95 is further configured to: calculating a pearson correlation coefficient between the second eigenmode function and the difference sequence data; alternatively, the energy ratio of the second eigenmode function to the difference sequence data is calculated.
In an alternative embodiment, the checking module 93 is further configured to check whether the second eigenmode function is a noise term and a trend term when determining the period of the time series data according to the period of the second eigenmode function; the period determination module 95 is also configured to: the period of the time series data is determined according to the period of the second eigenmode function of the non-noise term and the non-trend term.
In an alternative embodiment, in implementing the checking whether any eigenmode function is a noise term, the checking module 93 is further configured to: calculating the arrangement entropy of the eigenmode function, and carrying out noise inspection on the eigenmode function by using a noise inspection method; if the detection result of the noise detection method is noise or the arrangement entropy of the eigenmode function is greater than or equal to the arrangement entropy threshold value, determining that the eigenmode function is a noise item; if the result of the noise test is not noise and the arrangement entropy of the eigenmode function is smaller than the arrangement entropy threshold, determining that the eigenmode function is not a noise item.
In an alternative embodiment, in implementing the checking whether any eigenmode function is a trend term, the checking module 93 is further configured to: checking whether the eigenmode function is stationary using at least one sequence stationarity check; if the test result of at least one sequence stability test method is unstable, determining that the eigen-mode function is a trend term; if the test results of the sequence stability test method are all stable, determining that the eigenmode function is not a trend item.
In an alternative embodiment, when implementing the acquisition of the time series data to be detected, the time series acquisition module 91 is further configured to: time series data of state information of the target device is acquired, wherein the time series data comprises the state information of the target device at a plurality of historical time points.
The period detection device 90 of the time-series data includes: a post-processing module for: after determining the period of the time series data according to the period of the second eigenmode function, predicting state information of the target device at a future point in time according to the period of the time series data; or, according to the period of the time sequence data, performing anomaly detection on the target equipment; or, according to the period of the time series data, the state information of the target device is compressed and stored.
The device provided in this embodiment of the present application may be specifically configured to execute the solution provided in any of the foregoing method embodiments, and specific functions and technical effects that can be achieved are not described herein.
Fig. 10 is a schematic structural diagram of a cloud server according to an embodiment of the present application. As shown in fig. 10, the cloud server includes: a memory 1001 and a processor 1002. Memory 1001 for storing computer-executable instructions and may be configured to store various other data to support operations on a cloud server. The processor 1002 is communicatively connected to the memory 1001, and is configured to execute the computer-executable instructions stored in the memory 1001, so as to implement the technical solution provided in any one of the method embodiments, and the specific functions and the technical effects that can be implemented are similar, and are not repeated herein.
Optionally, as shown in fig. 10, the cloud server further includes: firewall 1003, load balancer 1004, communication component 1005, power component 1006, and other components. Only some components are schematically shown in fig. 10, which does not mean that the cloud server only includes the components shown in fig. 10.
The embodiment of the application further provides a computer readable storage medium, in which computer executable instructions are stored, and when the computer executable instructions are executed by a processor, the computer executable instructions are used to implement the scheme provided by any one of the method embodiments, and specific functions and technical effects that can be implemented are not described herein.
The embodiment of the application also provides a computer program product, which comprises: the computer program is stored in a readable storage medium, and the computer program can be read from the readable storage medium by at least one processor of the cloud server, so that the at least one processor executes the computer program to enable the cloud server to execute the scheme provided by any one of the method embodiments, and specific functions and technical effects that can be achieved are not repeated herein. The embodiment of the application provides a chip, which comprises: the processing module and the communication interface can execute the technical scheme of the cloud server in the embodiment of the method. Optionally, the chip further includes a storage module (e.g. a memory), where the storage module is configured to store the instructions, and the processing module is configured to execute the instructions stored in the storage module, and execution of the instructions stored in the storage module causes the processing module to execute the technical solution provided in any one of the foregoing method embodiments.
The memory may be an object store (Object Storage Service, OSS). The memory may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
The communication component is configured to facilitate wired or wireless communication between the device in which the communication component is located and other devices. The device where the communication component is located may access a wireless network based on a communication standard, such as a mobile hotspot (WiFi), a mobile communication network of a second generation mobile communication system (2G), a third generation mobile communication system (3G), a fourth generation mobile communication system (4G)/Long Term Evolution (LTE), a fifth generation mobile communication system (5G), or a combination thereof. In one exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.
The power supply component provides power for various components of equipment where the power supply component is located. The power components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the devices in which the power components are located.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, compact disk read-only memory (CD-ROM), optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory. The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should be noted that, the user information (including but not limited to user equipment information, user attribute information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with related laws and regulations and standards, and provide corresponding operation entries for the user to select authorization or rejection.
In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations appearing in a particular order are included, but it should be clearly understood that the operations may be performed out of order or performed in parallel in the order in which they appear herein, merely for distinguishing between the various operations, and the sequence number itself does not represent any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, and are not limited to the "first" and the "second" being different types. The meaning of "a plurality of" is two or more, unless specifically defined otherwise.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards, and provide corresponding operation entries for the user to select authorization or rejection.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It is to be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (14)

1. A period detection method of time-series data, comprising:
acquiring time sequence data to be detected, and decomposing the time sequence data into a first eigenmode function;
checking whether the first eigenmode function is a noise item and a trend item, removing the noise item and the trend item in the first eigenmode function according to a checking result, and taking the reserved first eigenmode function as a period item;
According to the period of the period item, carrying out similar clustering on the period item to obtain a class cluster, and combining the period item in the same class cluster into a second eigenmode function;
and determining the period of the time series data according to the period of the second eigenmode function.
2. The method according to claim 1, wherein the performing similar clustering on the periodic items according to the period of the periodic items to obtain clusters, merging the periodic items in the same cluster into a second eigenmode function, includes:
the period items are arranged according to the sequence from the small period to the large period, and the following processing is sequentially carried out on the period items:
if the current period item is not clustered into any cluster, the current period item and the period item which meets the preset condition later are clustered into the same cluster;
if a class cluster containing a plurality of periodic items exists, the periodic items in the same class cluster are combined to obtain a new eigen-mode function;
and carrying out similar clustering on the new eigenmode functions until a cluster containing a plurality of eigenmode functions does not exist in a clustering result, and taking the new eigenmode functions obtained currently as second eigenmode functions.
3. The method of claim 2, wherein the preset condition is that the period is less than or equal to 2 times the period of the current period term.
4. The method according to claim 1, wherein said determining the period of the time series data from the period of the second eigenmode function comprises:
subtracting trend items in the first eigenmode function from the time sequence data to obtain difference value sequence data;
calculating the correlation between the second eigenmode function and the difference sequence data;
and determining the period of at least one second eigenmode function as the period of the time series data according to the correlation between the second eigenmode function and the difference value series data.
5. The method of claim 4, wherein said calculating a correlation between the second eigenmode function and the difference sequence data comprises:
calculating a pearson correlation coefficient between the second eigenmode function and the difference sequence data;
or alternatively, the process may be performed,
and calculating the energy ratio of the second eigenmode function to the difference sequence data.
6. The method of claim 4, wherein determining the period of the time series data from the period of the second eigenmode function comprises:
Checking whether the second eigenmode function is a noise term and a trend term;
a period of the time series data is determined from a period of the second eigenmode function of a non-noise term and a non-trend term.
7. The method of claim 1 or 6, wherein checking whether any eigenmode function is a noise term comprises:
calculating the arrangement entropy of the eigenmode function, and carrying out noise inspection on the eigenmode function by using a noise inspection method;
if the detection result of the noise detection method is noise or the arrangement entropy of the eigenmode function is greater than or equal to the arrangement entropy threshold value, determining that the eigenmode function is a noise item;
if the detection result of the noise detection method is not noise and the arrangement entropy of the eigenmode function is smaller than the arrangement entropy threshold value, determining that the eigenmode function is not a noise item.
8. The method of claim 1 or 6, wherein checking whether any eigenmode function is a trend term comprises:
checking whether the eigenmode function is stationary using at least one sequence stationarity check;
if the test result of at least one sequence stability test method is unstable, determining that the eigen-mode function is a trend term;
And if the test results of the sequence stability test method are all stable, determining that the eigenmode function is not a trend item.
9. The method according to any one of claims 1-6, wherein the acquiring time series data to be detected comprises:
acquiring time sequence data of state information of target equipment, wherein the time sequence data comprises the state information of the target equipment at a plurality of historical time points;
after determining the period of the time series data according to the period of the second eigenmode function, the method further includes:
predicting state information of the target device at a future point in time according to the period of the time series data;
or alternatively, the process may be performed,
performing anomaly detection on the target equipment according to the period of the time sequence data;
or alternatively, the process may be performed,
and according to the period of the time sequence data, compressing and storing the state information of the target equipment.
10. An abnormality detection method, comprising:
acquiring time sequence data of resource state information of target equipment, wherein the time sequence data comprises the resource state information of the target equipment at a plurality of historical time points;
Decomposing the time series data into first eigenmode functions;
checking whether the first eigenmode function is a noise item and a trend item, removing the noise item and the trend item in the first eigenmode function according to a checking result, and taking the reserved first eigenmode function as a period item;
according to the period of the period item, carrying out similar clustering on the period item to obtain a class cluster, and combining the period item in the same class cluster into a second eigenmode function;
determining the period of the time sequence data according to the period of the second eigenmode function to obtain the period of the resource state information of the target equipment;
and according to the period of the resource state information of the target equipment, carrying out periodic abnormality detection on the target equipment.
11. A method for scheduling resources, comprising:
acquiring time sequence data of resource use information of a target system, wherein the time sequence data comprises the resource use information of the target system at a plurality of historical time points;
decomposing the time series data into first eigenmode functions;
checking whether the first eigenmode function is a noise item and a trend item, removing the noise item and the trend item in the first eigenmode function according to a checking result, and taking the reserved first eigenmode function as a period item;
According to the period of the period item, carrying out similar clustering on the period item to obtain a class cluster, and combining the period item in the same class cluster into a second eigenmode function;
determining the period of the time sequence data according to the period of the second eigenmode function to obtain the period of the resource use information of the target system;
and predicting the resource use condition of the target system in a future period according to the period of the resource use information of the target system, and carrying out resource scheduling according to the resource use condition of the target system in the future period.
12. A period detection method of time-series data, comprising:
responding to the terminal side equipment to call a period detection interface, and receiving time sequence data to be detected, which is sent by the terminal side equipment;
decomposing the time series data into first eigenmode functions;
checking whether the first eigenmode function is a noise item and a trend item, removing the noise item and the trend item in the first eigenmode function according to a checking result, and taking the reserved first eigenmode function as a period item;
according to the period of the period item, carrying out similar clustering on the period item to obtain a class cluster, and combining the period item in the same class cluster into a second eigenmode function;
Determining the period of the time series data according to the period of the second eigenmode function;
and sending the period of the time series data to the end-side equipment.
13. A cloud server, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1-12.
14. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1-12.
CN202310350882.3A 2023-03-29 2023-03-29 Method and equipment for detecting time sequence data period, detecting abnormality and scheduling resources Pending CN116361631A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310350882.3A CN116361631A (en) 2023-03-29 2023-03-29 Method and equipment for detecting time sequence data period, detecting abnormality and scheduling resources

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310350882.3A CN116361631A (en) 2023-03-29 2023-03-29 Method and equipment for detecting time sequence data period, detecting abnormality and scheduling resources

Publications (1)

Publication Number Publication Date
CN116361631A true CN116361631A (en) 2023-06-30

Family

ID=86921342

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310350882.3A Pending CN116361631A (en) 2023-03-29 2023-03-29 Method and equipment for detecting time sequence data period, detecting abnormality and scheduling resources

Country Status (1)

Country Link
CN (1) CN116361631A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117370898A (en) * 2023-12-08 2024-01-09 钛合联(深圳)科技有限公司 Electronic data safety control system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117370898A (en) * 2023-12-08 2024-01-09 钛合联(深圳)科技有限公司 Electronic data safety control system
CN117370898B (en) * 2023-12-08 2024-03-12 钛合联(深圳)科技有限公司 Electronic data safety control system

Similar Documents

Publication Publication Date Title
US10387240B2 (en) System and method for monitoring and measuring application performance using application index
CN110362612B (en) Abnormal data detection method and device executed by electronic equipment and electronic equipment
CN109921950B (en) Network bandwidth demand prediction method and device and electronic equipment
US11165799B2 (en) Anomaly detection and processing for seasonal data
CN110263869B (en) Method and device for predicting duration of Spark task
CN116361631A (en) Method and equipment for detecting time sequence data period, detecting abnormality and scheduling resources
CN114500339B (en) Node bandwidth monitoring method and device, electronic equipment and storage medium
CN112783725A (en) Index acquisition method and device
CN115757035A (en) Data model monitoring method and device, processor and electronic equipment
CN110795324A (en) Data processing method and device
CN109213965A (en) A kind of power system capacity prediction technique, computer readable storage medium and terminal device
CN110413482B (en) Detection method and device
CN109462510B (en) CDN node quality evaluation method and device
CN110764975B (en) Early warning method and device for equipment performance and monitoring equipment
CN115994029A (en) Container resource scheduling method and device
CN116225958A (en) Fault prediction method and device, storage medium and electronic equipment
CN113918636B (en) ETL-based data throughput analysis method
CN115309735A (en) Big data cleaning method and device, computer equipment and storage medium
CN114938339A (en) Data processing method and related device
CN110442508B (en) Test task processing method, device, equipment and medium
CN116028201A (en) Service system capacity prediction method and device
CN112131468A (en) Data processing method and device in recommendation system
CN110825493A (en) Virtual machine tuning method and device
CN113676377B (en) Online user number evaluation method, device, equipment and medium based on big data
CN110543965A (en) Baseline prediction method, baseline prediction device, electronic apparatus, and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination