CN113849700A

CN113849700A - Prediction method and device, equipment and storage medium

Info

Publication number: CN113849700A
Application number: CN202111153035.5A
Authority: CN
Inventors: 陆明; 聂志远; 张心怡
Original assignee: Lenovo Beijing Ltd
Current assignee: Lenovo Beijing Ltd
Priority date: 2021-09-29
Filing date: 2021-09-29
Publication date: 2021-12-28

Abstract

The application discloses a prediction method, a prediction device, equipment and a storage medium; wherein the method comprises the following steps: acquiring first sample data, wherein the first sample data comprises a first parameter value of a storage device at a sampling moment of a first time period; finding out a first target extreme value from the first parameter values of the sampling moments, wherein the time interval between the sampling moment of the first target extreme value and a specific sampling moment is greater than a first threshold value, and the specific sampling moment belongs to the first time period; predicting a second parameter value of the storage device at a sampling instant of a second time period based on the first parameter value between the sampling instant of the first target extremum to the particular sampling instant.

Description

Prediction method and device, equipment and storage medium

Technical Field

The embodiment of the application relates to communication technology, and relates to but is not limited to a prediction method, a prediction device, prediction equipment and a storage medium.

Background

During the use of the storage device, a condition that a certain type of parameter value of the storage device fluctuates sharply in a short period of time may occur, and in this condition, if the parameter value of the storage device continues to develop according to the trend in the future, the operation and the use of the whole system are possibly influenced, so that the prediction and the monitoring of the parameter value of the storage device are necessary.

Disclosure of Invention

In view of this, the prediction method, apparatus, device, and storage medium provided in the embodiments of the present application are implemented as follows:

according to an aspect of an embodiment of the present application, there is provided a prediction method, including: acquiring first sample data, wherein the first sample data comprises a first parameter value of a storage device at a sampling moment of a first time period; finding out a first target extreme value from the first parameter values of the sampling moments, wherein the time interval between the sampling moment of the first target extreme value and a specific sampling moment is greater than a first threshold value, and the specific sampling moment belongs to the first time period; predicting a second parameter value of the storage device at a sampling instant of a second time period based on the first parameter value between the sampling instant of the first target extremum to the particular sampling instant.

According to an aspect of an embodiment of the present application, there is provided a prediction apparatus, including: the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring first sample data, and the first sample data comprises a first parameter value of the storage device at the sampling moment of a first time period; the searching module is used for searching a first target extreme value from the first parameter values of the sampling moments, wherein the time interval between the sampling moment of the first target extreme value and a specific sampling moment is greater than a first threshold value, and the specific sampling moment belongs to the first time period; a prediction module to predict a second parameter value of the storage device at a sampling instant of a second time period based on the first parameter value between the sampling instant of the first target extremum to the particular sampling instant.

According to an aspect of the embodiments of the present application, there is provided an electronic device, including a memory and a processor, the memory storing a computer program executable on the processor, and the processor implementing the method of the embodiments of the present application when executing the program.

According to an aspect of the embodiments of the present application, there is provided a computer-readable storage medium on which a computer program is stored, the computer program, when executed by a processor, implementing the method provided by the embodiments of the present application.

In the embodiment of the present application, the second parameter value of the storage device at the sampling time of the second time period is predicted based on the first parameter value between the sampling time of the first target extremum value and the specific sampling time in the first time period. Therefore, on one hand, the first parameter value between the sampling time of the first target extreme value and the specific sampling time is predicted, but not all the first parameter values of the first time period, so that the more accurate second parameter value can be obtained. This is because: the development trend of the first parameter value between the sampling moment of the first target extreme value and the specific sampling moment is relatively stable, and the development trend of the parameter of the storage device in the future can be represented to a greater extent; on the other hand, the mode of setting the first threshold can avoid the interference of short-term jitter on the direction and the degree of the short-term prediction trend, so that the interval between the sampling time corresponding to the first target extreme value and the specific sampling time is not too short, and the second parameter value of the storage device in the second time period can be accurately and quickly obtained based on a proper amount of parameter values.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.

The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.

Fig. 1 is a schematic flow chart illustrating an implementation of a prediction method according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of data including a target extremum according to an embodiment of the present disclosure;

fig. 3 is a schematic flow chart illustrating an implementation of a prediction method according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a linear relationship obtained by prediction according to an embodiment of the present application;

fig. 5 is a schematic flow chart illustrating an implementation of a prediction method according to an embodiment of the present application;

fig. 6 is a schematic diagram illustrating data showing periodic oscillation according to an embodiment of the present disclosure;

fig. 7 is a schematic diagram of envelope prediction according to an embodiment of the present disclosure;

FIG. 8 is a schematic diagram of a data generation process according to an embodiment of the present disclosure;

fig. 9 is a schematic flow chart illustrating an implementation of a prediction method according to an embodiment of the present application;

FIG. 10 is a schematic diagram of a short-term fluctuation impact analysis report provided by an embodiment of the present application;

fig. 11 is a schematic diagram of actual data breakthrough envelope provided by an embodiment of the present application;

FIG. 12 is a schematic structural diagram of a prediction device according to an embodiment of the present application;

fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, specific technical solutions of the present application will be described in further detail below with reference to the accompanying drawings in the embodiments of the present application. The following examples are intended to illustrate the present application but are not intended to limit the scope of the present application.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

It should be noted that reference to the terms "first \ second \ third" in the embodiments of the present application does not denote a particular ordering with respect to the objects, and it should be understood that "first \ second \ third" may be interchanged under certain circumstances or of a certain order, such that the embodiments of the present application described herein may be performed in an order other than that shown or described herein.

The embodiment of the present application provides a prediction method, which is applied to an electronic device, which may be various types of devices with information processing capability in the implementation process, for example, the electronic device may include a Personal computer (e.g., a desktop computer, a notebook computer, a mini-notebook computer, a tablet computer, etc.), a mobile phone, a Personal Digital Assistant (PDA), a server, a cluster server, and the like. The functions implemented by the method can be implemented by calling program code by a processor in an electronic device, and the program code can be stored in a computer storage medium.

Fig. 1 is a schematic flow chart of an implementation of a prediction method provided in an embodiment of the present application, and as shown in fig. 1, the method may include the following steps 101 to 103:

step 101, obtaining first sample data, wherein the first sample data comprises a first parameter value of a storage device at a sampling time of a first time period.

In the embodiment of the present application, the storage device may be a variety of devices with a storage function, such as a magnetic disk, a flash memory drive, a cloud platform storage device, or a battery.

The type of the parameter value in the sample data is not limited. For example, the parameter value is a disk capacity usage value, a disk capacity usage ratio or a disk capacity remaining ratio, etc.; for another example, the parameter value is a battery capacity usage value, a battery capacity usage ratio or a battery capacity remaining ratio, etc.; as another example, the parameter value is the network speed of the server.

Here, the manner of acquiring the first sample data is not limited, and the first sample data may be extracted from the history data stored in advance, or may be acquired in real time.

In the embodiment of the present application, the sampling interval may be set in advance when the first parameter value of the storage device at the sampling timing of the first time period is acquired. For example, if the sampling interval is set to 60 seconds, the first sampling time is 13:10 minutes, and the second sampling time is 13:11 minutes; if the sampling interval is set to 10 minutes and the first sampling time is 13:10 minutes, then the second sampling time is 13:20 minutes.

In some embodiments, after the first sample data is acquired, the parameter value of the storage device in the second time period is not predicted immediately according to the first sample data, but whether the first sample data fluctuates (for example, whether data is read in or not) is determined first; if the first sample data is the fluctuation data, the prediction is continuously executed, otherwise, the first sample data is not subjected to subsequent processing, so that the prediction on a possibly unused storage device is avoided, and the waste of resources is avoided.

102, finding out a first target extremum from the first parameter values of the sampling moments, wherein a time interval between the sampling moment of the first target extremum and a specific sampling moment is greater than a first threshold, and the specific sampling moment belongs to a first time period.

Here, the position of the specific sampling time is not limited. The specific sampling time may be the last sampling time in the first time period, or may be any other sampling time in the first time period.

Taking the specific sampling time as the last sampling time in the first time period as an example for explanation, as shown in fig. 2, the manner of finding the first target extremum may be: taking a specific sampling time of a first time period (namely, 4 month 1 day to 4 month 21 days in fig. 2) as a starting point, searching forward in sequence, and judging whether a first parameter value corresponding to the current sampling time is a maximum value or a minimum value in first parameter values corresponding to all sampling times before the specific sampling time; if so, continuously determining whether the time interval between the current sampling time and the specific sampling time 203 is greater than a first threshold (if the first threshold is set to be 2 hours), if the time interval is within 2 hours, discarding the extreme point, continuously searching forwards until the next extreme point is found, and the time interval between the sampling time corresponding to the extreme point and the specific sampling time is greater than 2 hours, determining the extreme point as a first target extreme value 204, and recording the corresponding sampling time.

And 103, predicting a second parameter value of the storage device at the sampling time of the second time period based on the first parameter value between the sampling time of the first target extreme value and the specific sampling time.

In some embodiments, the warning information is output when the second parameter value for the second time period is greater than a fourth threshold value. Taking the parameter value as the usage percentage of the disk capacity as an example, in the second time period, if the predicted usage percentage of the disk capacity at a certain time is greater than 70% (fourth threshold), alarm information is output.

The method for outputting the alarm information is not limited, for example, the alarm information is output in a report sending manner; for another example, the monitoring system is integrated according to the strategy to send an alarm through the monitoring system; for another example, the results are saved in a database, and the user logs in a website to receive the notification and browse the analysis report.

In some embodiments, predicting the second parameter value of the storage device at the sampling instant of the second time period may be implemented by performing steps 306 to 307 in the following embodiments.

Fig. 3 is a schematic flow chart of an implementation of the prediction method provided in the embodiment of the present application, and as shown in fig. 3, the method may include the following steps 301 to 308:

step 301, obtaining second sample data, where the second sample data includes a third parameter value of the storage device at the sampling time of the first time period;

and 302, smoothing the third parameter value at each sampling moment to obtain first sample data.

It will be appreciated that, as shown in fig. 2, there will be some small parameter value jitter in the curve (i.e. curve 201) formed by the second sample data, and these small parameter values will interfere with the accuracy of the subsequent search for the first target extremum 204 and the predicted result.

Therefore, in some embodiments, after the second sample data is acquired, the third parameter value in the second sample data may be smoothed, and then the smoothed parameter value and the corresponding sampling time are used as the first sample data (i.e., the smoothed value in the smoothing curve 202). In this way, some small jitter parameter values (noise values) can be filtered out on the basis of not changing the development trend of the original values, so that the subsequent searching of the first target extremum 204 can be effectively performed, and the subsequent prediction error can be reduced.

The smoothing method is not limited to this. For example, taking the average value of the first parameter values corresponding to the N sampling moments as the first parameter value after the smoothing processing; for another example, a median value of the first parameter values corresponding to the M sampling times is taken as the first parameter value after the smoothing processing.

Step 303, finding out a first target extremum from the first parameter values of the sampling moments, wherein a time interval between the sampling moment of the first target extremum and a specific sampling moment is greater than a first threshold, and the specific sampling moment belongs to a first time period;

step 304, determining whether the number of the first parameter values between the sampling time of the first target extreme value and the specific sampling time is larger than a second threshold value; if the number is greater than the second threshold, go to step 305; otherwise, step 308 is performed.

It will be appreciated that the number of first parameter values between the sampling instant of the first target extremum to the particular sampling instant is high, and that the prediction speed will be slow when performing a prediction based on these first parameter values. Therefore, in order to increase the prediction speed and reduce the calculation capability required for prediction, step 305 may be executed when the number of first parameter values is greater than the second threshold, to select a part of the first parameter values from the large number of first parameter values as the prediction data set, and to perform prediction based on the selected first parameter values.

In step 305, a portion of the first parameter values is selected from the first parameter values between the sampling time of the first target extremum and the specific sampling time.

Here, the manner of selecting the partial first parameter values is not limited. For example, only the first target extreme value and the first parameter value corresponding to a specific sampling moment are selected as prediction data; for another example, the user defines a new sampling interval, and resamples the first parameter value between the sampling time of the first target extremum and the specific sampling time to obtain the predicted data.

And step 306, determining the linear relation between the parameter value of the storage device and the sampling time based on the part of the first parameter value.

Here, the method of determining the linear relationship between the parameter value of the storage device and the sampling timing is not limited. For example, a linear regression algorithm or an exponential smoothing algorithm may be used to determine the linear relationship.

As another example, in some embodiments, the linear relationship may be determined by: determining a tangent line of a curve formed by part of first parameter values and corresponding sampling moments by taking a point formed by the first target extreme value and the corresponding sampling moments as a fixed point; wherein the fixed point is a point on the tangent line; the linear relationship represented by the tangent is determined as the linear relationship of the value of the parameter of the storage means to the sampling instant.

For example, as shown in fig. 2, the curve formed by the first parameter value after the first target extreme point and the corresponding sampling time is a curve that increases or decreases approximately monotonically. In this way, the first target extreme point is taken as a fixed point, and rotation is continuously performed to find a tangent line tangent or approximately tangent to the curve, wherein the slope of the tangent line is similar to the slope obtained by the linear regression algorithm. Compared with a linear regression algorithm, the method for determining the linear relation does not need to be subjected to complex data calculation, only needs to search a straight line tangent to the curve and translate the straight line to a specific sampling moment based on the first target extreme point, and is small in calculation amount and high in determination speed.

For another example, in other embodiments, the linear relationship may also be determined by: obtaining a prediction straight line according to a point formed by the first target extreme value and the corresponding sampling time and a point formed by the specific sampling time and the corresponding first parameter value; and taking the linear relation represented by the prediction straight line as the linear relation between the parameter value of the storage device and the sampling time. The linear relation is determined only according to the two data points, the calculation capacity requirement of the prediction algorithm is greatly reduced, and the analysis result is closer to the calculation result of the linear regression algorithm.

And 307, predicting a second parameter value of the storage device at the sampling time of the second time period according to the linear relation and the first parameter value at the specific sampling time.

It will be appreciated that, as shown in fig. 4, after determining the linear relationship between the parameter value of the storage device and the sampling time, the slope of the curve formed by the first parameter value in the first time period and the corresponding time can be obtained, and based on the slope and the specific sampling time 401, the second parameter value (i.e. the value 402 on the prediction curve) of the storage device at the sampling time in the second time period can be predicted. The gray curve is a curve formed by the first parameter value (i.e. the original data line), and the curve blocked by the gray curve is a curve formed by the smooth data values.

And 308, determining the linear relation between the parameter value of the storage device and the sampling time based on the first parameter value, and predicting a second parameter value of the storage device at the sampling time of the second time period according to the linear relation and the first parameter value at the specific sampling time.

In some embodiments, the warning information is output when the second parameter value for the second time period is greater than a fourth threshold value.

In some embodiments, as shown in fig. 5, the second parameter value of the storage device at the sampling time of the second time period may also be predicted by performing the following steps 501 to 509:

step 501, determining a first envelope curve and a second envelope curve of a curve formed by a first parameter value of each sampling time and a corresponding sampling time; wherein the first and second envelope curves are on opposite sides of the curve.

It can be understood that, as shown in fig. 6, if the parameter value of the storage device fluctuates around the alarm threshold (70% in the figure) and exhibits periodic oscillation, when the prediction method in the above embodiment is executed, the predicted value determined according to the linear relationship exhibits a monotone ascending trend, and exceeds the alarm threshold in a short time, but the trend of the original parameter value after the specific sampling time 604 is monotone descending according to the fluctuation trend, and does not exceed the alarm threshold, so that false alarm may be frequently generated. Where 601 is a curve of original data values, 602 is a curve of smoothed data values, and 603 is a curve of predicted values obtained by prediction.

In order to avoid the occurrence of false alarms frequently, in the embodiment of the present application, as shown in fig. 7, the upper and lower envelope curves of the curve composed of the original values are determined, so that the occurrence of false alarms is reduced. Where 701 is a first sample data (i.e., a curve of original values), 702 is a first envelope, 703 is a second envelope, 704 is a smoothed data value, and 705 is a curve of predicted second parameter values (i.e., a curve of predicted values).

Here, the method of determining the first and second envelope is not limited. For example, the envelope of the curve can be obtained by an Empirical Mode Decomposition (EMD) algorithm.

Step 502, finding out a second target extremum from fourth parameter values of each sampling time of the first envelope; and the time interval between the sampling time corresponding to the second target extreme value and the specific sampling time is greater than the first threshold value.

It should be noted that the manner of searching for the second target extremum in the first envelope in step 502 is the same as the manner of searching for the first target extremum in step 102, and is not described herein again.

Step 503, finding out a third target extremum from the fifth parameter values of each sampling time of the second envelope curve; and the time interval between the sampling time corresponding to the third target extreme value and the specific sampling time is greater than the first threshold value.

It should be noted that the manner of searching for the third target extremum in the second envelope in step 503 is the same as the manner of searching for the first target extremum in step 102, and is not described herein again.

And step 504, predicting a sixth parameter value of the storage device at the sampling time of the second time period based on the fourth parameter value between the sampling time corresponding to the second target extreme value and the specific sampling time.

Here, when the prediction is performed based on the fourth parameter value between the sampling time corresponding to the second target extremum in the first envelope and the specific sampling time, the prediction manner is the same as that in steps 306 to 307, and is not described herein again.

And 505, predicting a seventh parameter value of the storage device at the sampling time of the second time period based on a fifth parameter value between the sampling time of the third target extreme value and the specific sampling time.

Here, when the prediction is performed based on the fifth parameter value between the sampling time corresponding to the third target extremum in the second envelope and the specific sampling time, the prediction manner is the same as that in steps 306 to 307, and is not described herein again.

Step 506, determining a second parameter value at the ith time based on the sixth parameter value and the seventh parameter value at the ith time, so as to obtain the second parameter value of the storage device at the sampling time of the second time period; wherein i is any one of the sampling timings of the second time period.

Here, the manner of determining the second parameter value at the ith time based on the sixth parameter value and the seventh parameter value at the ith time is not limited. For example, the sixth parameter value and the seventh parameter value may be averaged to obtain a second parameter value; for another example, the sixth parameter value and the seventh parameter value may be weighted, and the sixth parameter value and the seventh parameter value after weighting may be summed to obtain the second parameter value.

Step 507, determining whether the difference value between the actual parameter value of the storage device at the current moment and the predicted second parameter value at the corresponding moment is greater than a third threshold value; if yes, go to step 508; otherwise, returning to step 501, continuing to predict the eighth parameter value of the storage device at the sampling time of the third time period.

It can be understood that if the difference between the actual parameter value of the storage device at the current time and the predicted second parameter value at the corresponding time is larger, it indicates that the parameter value of the storage device has a larger change (for example, the parameter value suddenly increases, or the parameter value suddenly decreases, or the change trend of the parameter value becomes a parallel line), and the parameter value of the storage device may not appear as the periodic oscillation trend but appear as a monotone rising or monotone falling trend, at this time, step 508 may be executed, the actual parameter value at the current time is determined to be an outlier, and then, according to the actual parameter value before the outlier, the prediction methods such as linear regression in the above embodiments are executed, so as to predict the parameter value of the storage device after the outlier.

If the difference value between the actual parameter value of the storage device at the current moment and the second parameter value at the corresponding moment obtained by prediction is small, which indicates that the parameter value development trend of the storage device is periodic oscillation, the method returns to step 501, and continues to adopt the mode of determining the envelope curve for prediction.

In some embodiments, after generating the upper and lower envelope curves of the curve, a first deviation value may be added to the upper envelope curve to increase its corresponding parameter value; and adding a second deviation value to the lower envelope to reduce the corresponding parameter value, wherein the first deviation value and the second deviation value are opposite values, and the updated upper envelope and the updated lower envelope are used as the abnormal detection envelope. Therefore, the situation that data are too sensitive and the actual parameter value is determined to be an outlier by mistake when the difference value between the actual parameter value and the predicted parameter value is small can be avoided.

Step 508, finding out a fourth target extremum from the actual parameter values at the sampling time before the current time.

Here, the manner of finding the fourth target extremum from the actual parameter value is the same as the manner of finding the first target extremum in step 102, and is not described herein again.

Step 509, predicting an eighth parameter value of the storage device at the sampling time of the third time period based on the actual parameter value between the sampling time of the fourth target extremum and the current time; wherein the number of sampling instants of the third time period is the same as the second time period.

Here, when the parameter value of the storage device in the third time period is predicted based on the actual parameter value between the sampling time corresponding to the fourth target extremum and the current time, the prediction manner is the same as that in steps 306 to 307, and is not described herein again.

In some embodiments, the warning information is output when the eighth parameter value for the third time period is greater than the fourth threshold value.

In the process of predicting the capacity of a disk (such as a Linux mount point or a Windows logical volume) under an operating system in a specific node range, a situation that data of a certain node fluctuates severely in a very short time may occur, and as shown in fig. 8, the data in 801 fluctuates severely in a short time. In this case, if the data of the disk capacity continues to develop according to the trend in the future, the operation and the use of the whole system are likely to be influenced, so that the prediction and the monitoring of the disk capacity are necessary.

Aiming at the situations, the embodiment of the application adopts a prediction method for predicting the extreme situation of short-term data, determines the whole trend of the recent data based on the recent local extreme point and predicts the trend, and can more accurately predict the development trend of the data in a future period of time, thereby realizing the real-time monitoring of the disk capacity of a user. If the point exceeding the alarm threshold value set by the user exists in the prediction result, the system is identified to be abnormal, and an alarm report is immediately sent to the user, so that the user can timely process the risk of the system to be abnormal, and the safe operation and normal use of the system are maintained. In addition, the embodiment of the application can amplify the current data fluctuation trend, so that a user can clearly grasp the influence of the current trend on the system at a glance and prevent the influence from happening in the bud.

In the embodiment of the application, a prediction method for predicting short-term data is adopted, proper local extreme point selection is adopted, data prediction is further performed, prediction and analysis of irregular accidental situations can be achieved, and a prediction result is generated accurately and quickly; processing outliers and noise of the data by using a smoothing algorithm so as to determine appropriate local extreme points and further help to perform short-term trend analysis; resampling is performed on short-term trend data, which significantly reduces the computational power requirements for predictive activities on the basis that the prediction accuracy is not severely affected.

By performing the above method, the following advantages can be obtained: (1) occasionally, the capacity of the disk fluctuates dramatically. The sporadic fluctuation is not regularly traceable, and is not applicable to the traditional algorithm for predicting according to historical data and the monitoring of static threshold values. Occasional fluctuations may also cause disk shortage anomalies. The embodiment of the application solves the problem, and the system can predict and alarm in real time based on accidental fluctuation.

(2) The method can effectively analyze the big data and perform low-computation-power consumption prediction analysis on scenes with low data value density, and the characteristic is prominent. For example, the disk monitoring data of the cloud platform is huge, as shown in formula 1, it is assumed that a certain data center has 10,000 nodes, each node has only 3 logical volumes or mount points to be monitored, the data acquisition frequency of the monitoring system is 60 seconds, and the processing scale of each batch of data may be more than 1 hundred million. The historical data size for each predicted activity can be estimated according to the following formula, typically on the order of minutes or hours for such predictions. The scheme can acquire the data of the volume and can complete analysis and alarm activities within a Service Level Agents (SLAs).

As shown in fig. 9, a flowchart of a prediction method is provided, which is specifically shown in the following steps 901 to 912:

step 901, after the user configures the node range and the disk range to be monitored in the system, the system will start to execute the node subscription message and start the analysis activity.

In step 902, the system obtains cloud platform data or computing and storage resource data (historical capacity data of a disk) from a cloud platform database.

Step 903, after determining the range of the node or disk to be monitored and the monitoring index, the system obtains the corresponding resource monitoring data and status data from the monitoring database.

And 904, analyzing the acquired data according to the characteristics of the data, filtering the data of different nodes through a set rule filter, and identifying an index range which can be used for prediction. Among the typical prediction range cases are: (1) the standard deviation of the capacity, i.e. the degree of variation, is identified. If the capacity is below a certain threshold and the recent standard deviation is low, meaning that there is no change in capacity, then no prediction need be performed. This corresponds to a standard deviation high-pass filter, where standard deviations below a certain value are filtered out. (2) It is also possible to use max-min, if less than a certain threshold, or% max% -min is less than a certain change ratio, no prediction is performed. The goal here is to reduce computational power consumption, most cloud computing resources are not actually used and are identified as unused, and no prediction need be performed on them.

Step 905, data cleaning and missing value filling are performed on the initial data.

Step 906, smoothing the pre-processed data, wherein parameters of the smoothing algorithm can be determined based on a table lookup or a parameter selection algorithm.

Step 907, find local extremum point before t-n time, where n is the maximum value where the latest time fluctuation can be ignored, and is usually set to ignore the latest local extremum point and continue to find the next latest local extremum point if the latest local extremum point occurs within 2 hours from the current time.

Step 908, obtain the data range from the determined local extreme point to the predicted start position.

Step 909, resampling is executed according to the rule, so as to further reduce the requirement of prediction power and improve the efficiency of the monitoring system, and the method can support sampling modes with different granularities to process the original data, such as only using the first point and the last point of the test set; resampling according to user-defined resampling time; or the whole raw data is used directly without sampling.

And step 910, executing prediction to obtain a prediction result within the time t + m, wherein m is the prediction length defined by the user.

And 911, identifying data needing to be alarmed when the prediction result exceeds a threshold value based on a static or dynamic threshold value rule.

And 912, if the condition of needing alarming exists, sending a notice or an alarming message according to the rule, and visually storing the prediction result and the alarming information in a database so as to facilitate the user to check. In some embodiments, the notification or alarm message may be obtained by: (1) sending a report; (2) or sending an alarm through the monitoring system according to the strategy integration monitoring system; (3) or storing the result in a database, and browsing the analysis report by the user logging in the website to receive the notification.

The above implementation process is specifically explained as follows:

(1) data smoothing

In the data smoothing algorithm, different parameters need to be selected according to the length of the history data for data processing. If the smooth parameter selection is too large, the important fluctuation trend of the data can be lost, and the trend of the data cannot be accurately judged; if the smoothing parameters are chosen too small, noise and outliers of the data may not be efficiently processed. The proper smooth parameters can filter out some small jitters on the premise that the original trend of the data waveform is not changed, so that the marking of the extreme point can be effectively carried out, and the error of the prediction process is small.

The smoothing algorithm parameters can be implemented by pre-formed calculations, by table lookup or by rule-based algorithms.

(2) Local extreme point selection method

The correct local extreme points under different scenes are obtained, short-term abnormal fluctuation and medium-short term capacity change trends can be effectively identified, and powerful references are provided for subsequent monitoring and prediction. Different from a common optimization method, the method can effectively identify a global optimal solution or a global extreme point, wherein an appropriate local extreme point needs to be found, and the analysis process is obviously different.

The extreme point selection algorithm idea is as follows:

and a, searching forwards in sequence by taking the last time point of the original data as a start point, and checking whether the data of each time point is an extreme value of the data after the time point.

And b, if the point is identified to be a maximum value or a minimum value at a certain moment, marking the point.

And c, if the distance between the marked local extreme point and the current time is less than a certain window threshold value, for example, the latest 2 hours, ignoring the point, and continuously searching the next nearest local extreme point. Therefore, the situation that the extreme point is too close to the current time and cannot accurately reflect the data trend of the time period is prevented. Jitter in the short term may interfere with the direction and extent of the short term predicted trend.

The data after smoothing and marking the extreme points is shown in fig. 2. Wherein the extreme points are indicated by dots.

(3) Data prediction process

Predictive algorithms typically rely on long-term data precipitation and predict data as stationary data to compute. The method can predict the non-stationary sequence with large fluctuation amplitude. It is understood that the data with fluctuation is called as large fluctuation amplitude and is predicted, and the data with no change in memory capacity (such as a straight line) is called as small fluctuation amplitude.

The prediction process for short-term data is as follows:

step a, determining historical data in the modes of user input, system prefabrication strategy and the like.

And b, starting from the local extreme point obtained by the previous method to the end of the prediction data set, and obtaining the prediction data set.

And c, identifying the number of the data sets, and if the number exceeds a certain threshold value, performing resampling to reduce the data volume. The resampling does not affect the data trend, and has little influence on the precision, so that the computational power requirement is obviously reduced on the basis of not affecting the qualitative judgment of the prediction.

And d, analyzing by using a prediction algorithm based on the data set, such as linear regression, cubic exponential smoothing and the like.

And e, analyzing the short-term or medium-term prediction result through static or dynamic threshold analysis, and sending a notice or an alarm based on different strategies.

Taking the utilization rate of the disk capacity as an example, the data situation within 6 hours in the future is predicted by using the short-term prediction result based on linear regression as shown in fig. 4. The dots are the local extreme point positions, and it can be seen that the other local extreme point on the right side thereof is ignored. Fig. 10 shows the generated short-term fluctuation influence analysis report, and as shown in fig. 10, 1001 is a graph of actual values, 1002 is a graph of smoothed data values, and 1003 is a graph of predicted values obtained by prediction.

In some embodiments, the slope of the prediction algorithm may also be derived based on: the linear regression algorithm shows that the analysis algorithm is intersected with the local extreme point in the calculation process of the local extreme point. If the predicted dataset bunch position is centered, rotate x points to the right, looking for tangency (or near tangency) until, instead of intersecting. This new data location is the starting location of the predicted data. Where x is less than a certain threshold to avoid over-processing. If x still cannot find a tangent (or near tangent) position when it reaches a certain threshold, the original local extreme point continues to be used.

This is because the data after the local extreme point is a curve that rises or falls approximately monotonically. And drawing a straight line tangent with the subsequent curve by taking the local extreme point as the center, thereby obtaining a curve close to the linear regression slope, translating the curve to the end of the prediction curve, namely adjusting the position of the intercept. Since the curve fluctuation after the local extreme point is obtained is small, the simplified algorithm can be used in combination with the service scenario.

In other embodiments, the slope of the prediction algorithm may also be derived based on: since in most cases even the slopes are almost identical based on the curve trend after the local extreme point. Therefore, under the condition of high computational power consumption, a straight line is directly calculated between the local extreme point and the data set cluster position, and the slope of the straight line is used as the slope of the prediction algorithm. The method greatly reduces the calculation force requirement of the prediction algorithm, and the analysis result is closer to the calculation result of the linear regression algorithm. Although the prediction accuracy is lost, it does not therefore produce too large a false alarm as a qualitative analysis and alarm.

In some embodiments, as shown in fig. 6, when the data is close to the threshold and exhibits periodic oscillation, a false alarm of the present algorithm may be caused, and therefore, by combining with the EMD algorithm, the false alarm probability is reduced. The implementation manner of the combination algorithm is as follows from step 1 to step 3:

step 1, generating upper and lower envelope lines of a curve by using an EMD algorithm, and adding a deviation value on the envelope lines to serve as an abnormality detection envelope line.

And step 2, respectively executing prediction on the upper and lower abnormality detection envelopes, and judging that the detection is abnormal if the actual value exceeds the abnormality detection envelope.

a) And actually monitoring the last point in the data, and if the upper and lower anomaly detection envelope thresholds are broken through, executing short-term prediction.

b) As shown in fig. 7, if all data points are within the envelope, the upper and lower envelopes are used for prediction, and whether to trigger the alarm message is determined by whether the data points intersect with a preset threshold.

And 3, triggering an alarm message if the short-term prediction result exceeds an alarm threshold value.

As shown in fig. 11, 111 is a curve of original data values, 112 is an upper envelope, 113 is a lower envelope, 114 is a curve of smoothed data values, and 115 is a curve of predicted data values obtained by prediction. If the actual data at the current time t0 breaks through the upper and lower envelopes at the predicted time t 0' at the time t-1, it means that the data at the current time is outlier data. The previous short-term band effect analysis will now be used to predict the subsequent data, without using the prediction of the upper and lower EMD envelopes.

It should be noted that although the various steps of the methods in this application are depicted in the drawings in a particular order, this does not require or imply that these steps must be performed in this particular order, or that all of the shown steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.

Based on the foregoing embodiments, the present application provides a prediction apparatus, which includes modules included in the prediction apparatus and units included in the modules, and may be implemented by a processor; of course, the implementation can also be realized through a specific logic circuit; in implementation, the processor may be a Central Processing Unit (CPU), a Microprocessor (MPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), or the like.

Fig. 12 is a schematic structural diagram of a prediction apparatus according to an embodiment of the present application, and as shown in fig. 12, the apparatus 120 includes an obtaining module 121, a searching module 122, and a prediction module 123, where:

an obtaining module 121, configured to obtain first sample data, where the first sample data includes a first parameter value of a storage device at a sampling time of a first time period; a searching module 122, configured to search for a first target extremum from the first parameter values at the respective sampling times, where a time interval between a sampling time of the first target extremum and a specific sampling time is greater than a first threshold, and the specific sampling time belongs to the first time period; a prediction module 123 configured to predict a second parameter value of the storage device at a sampling time of a second time period based on the first parameter value between the sampling time of the first target extremum and the specific sampling time.

In some embodiments, the apparatus 120 further comprises a smoothing module, the obtaining module 121, further configured to obtain second sample data, the second sample data including a third parameter value of the storage apparatus at the sampling time of the first time period; and the smoothing module is used for smoothing the third parameter value at each sampling moment to obtain the first sample data.

In some embodiments, the apparatus 120 further comprises a determining module for determining whether the number of the first parameter values between the sampling time of the first target extremum to the specific sampling time is greater than a second threshold value; the selecting module is configured to select a part of the first parameter values from the first parameter values between the sampling time of the first target extremum and the specific sampling time if the number is greater than the second threshold; a prediction module 123 predicts a second parameter value of the storage device at a sampling instant of a second time period based on the portion of the first parameter value.

In some embodiments, the determining module is further configured to determine a linear relationship between the parameter value of the storage device and the sampling time based on the portion of the first parameter value; and the predicting module 123 is configured to predict a second parameter value of the storage device at the sampling time of the second time period according to the linear relationship and the first parameter value at the specific sampling time.

In some embodiments, the determining module is further configured to determine, using a point formed by the first target extremum and the corresponding sampling time as a fixed point, a tangent of a curve formed by the portion of the first parameter values and the corresponding sampling time; wherein the fixed point is a point on the tangent line; and determining the linear relation represented by the tangent line as the linear relation between the parameter value of the storage device and the sampling moment.

In some embodiments, the determining module is further configured to obtain a predicted straight line according to a point formed by the first target extreme value and the corresponding sampling time, and a point formed by the specific sampling time and the corresponding first parameter value; and taking the linear relation represented by the prediction straight line as the linear relation between the parameter value of the storage device and the sampling time.

In some embodiments, the determining module is further configured to determine a first envelope and a second envelope of a curve formed by the first parameter values at the respective sampling instants and the corresponding sampling instants; wherein the first and second envelope lines are on opposite sides of the curve; the searching module 122 is configured to search for a second target extremum from fourth parameter values of each sampling time of the first envelope; wherein a time interval between the sampling time corresponding to the second target extreme value and the specific sampling time is greater than the first threshold; the searching module 122 is further configured to search for a third target extremum from fifth parameter values of each sampling time of the second envelope curve; wherein a time interval between the sampling time corresponding to the third target extremum and the specific sampling time is greater than the first threshold; the predicting module 123 is configured to predict a sixth parameter value of the storage device at the sampling time of the second time period based on the fourth parameter value between the sampling time corresponding to the second target extremum and the specific sampling time; and predicting a seventh parameter value of the storage device at a sampling time of a second time period based on the fifth parameter value between the sampling time of the third target extremum to the specific sampling time; the determining module is further configured to determine the second parameter value at an ith time based on the sixth parameter value and the seventh parameter value at the ith time, so as to obtain the second parameter value of the storage device at a sampling time of a second time period; wherein i is any one of the sampling moments of the second time period.

In some embodiments, the finding module 122 is further configured to find a fourth target extremum from the actual parameter values at the sampling time before the current time if the difference between the actual parameter value at the current time and the corresponding second parameter value is greater than a third threshold; a prediction module 123, configured to predict an eighth parameter value of the storage device at a sampling time of a third time period based on the actual parameter value between the sampling time of the fourth target extremum and the current time; wherein the third time period is the same as the second time period in number of sampling instants.

In some embodiments, the apparatus 120 further includes an output module, configured to output an alarm message when the second parameter value of the second time period and/or the eighth parameter value of the third time period is greater than a fourth threshold.

The above description of the apparatus embodiments, similar to the above description of the method embodiments, has similar beneficial effects as the method embodiments. For technical details not disclosed in the embodiments of the apparatus of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.

It should be noted that, in the embodiment of the present application, the partitioning of the prediction apparatus shown in fig. 12 into modules is schematic, and is only one logical function partitioning, and there may be another partitioning manner in actual implementation. In addition, functional units in the embodiments of the present application may be integrated into one processing unit, may exist alone physically, or may be integrated into one unit by two or more units. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. Or may be implemented in a combination of software and hardware.

It should be noted that, in the embodiment of the present application, if the method described above is implemented in the form of a software functional module and sold or used as a standalone product, it may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing an electronic device to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.

An electronic device is provided in an embodiment of the present application, fig. 13 is a schematic diagram of a hardware entity of the electronic device in the embodiment of the present application, as shown in fig. 13, the electronic device 130 includes a memory 131 and a processor 132, the memory 131 stores a computer program that can be executed on the processor 132, and the processor 132 executes the computer program to implement the steps in the method provided in the embodiment.

It should be noted that the Memory 131 is configured to store instructions and applications executable by the processor 132, and may also buffer data (e.g., image data, audio data, voice communication data, and video communication data) to be processed or already processed by each module in the processor 132 and the electronic device 130, and may be implemented by a FLASH Memory (FLASH) or a Random Access Memory (RAM).

Embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps in the methods provided in the above embodiments.

Embodiments of the present application provide a computer program product containing instructions which, when run on a computer, cause the computer to perform the steps of the method provided by the above-described method embodiments.

Here, it should be noted that: the above description of the storage medium and device embodiments is similar to the description of the method embodiments above, with similar advantageous effects as the method embodiments. For technical details not disclosed in the embodiments of the storage medium, the storage medium and the device of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" or "some embodiments" means that a particular feature, structure or characteristic described in connection with the embodiments is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in some embodiments" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments. The foregoing description of the various embodiments is intended to highlight various differences between the embodiments, and the same or similar parts may be referred to each other, and for brevity, will not be described again herein.

The term "and/or" herein is merely an association relationship describing an associated object, and means that three relationships may exist, for example, object a and/or object B, may mean: the object A exists alone, the object A and the object B exist simultaneously, and the object B exists alone.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice, such as: multiple modules or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or modules may be electrical, mechanical or other.

The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules; can be located in one place or distributed on a plurality of network units; some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, all functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may be separately regarded as one unit, or two or more modules may be integrated into one unit; the integrated module can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read Only Memory (ROM), a magnetic disk, or an optical disk.

Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing an electronic device to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.

The methods disclosed in the several method embodiments provided in the present application may be combined arbitrarily without conflict to obtain new method embodiments.

Features disclosed in several of the product embodiments provided in the present application may be combined in any combination to yield new product embodiments without conflict.

The features disclosed in the several method or apparatus embodiments provided in the present application may be combined arbitrarily, without conflict, to arrive at new method embodiments or apparatus embodiments.

The above description is only for the embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A prediction method, comprising:

acquiring first sample data, wherein the first sample data comprises a first parameter value of a storage device at a sampling moment of a first time period;

finding out a first target extreme value from the first parameter values of the sampling moments, wherein the time interval between the sampling moment of the first target extreme value and a specific sampling moment is greater than a first threshold value, and the specific sampling moment belongs to the first time period;

predicting a second parameter value of the storage device at a sampling instant of a second time period based on the first parameter value between the sampling instant of the first target extremum to the particular sampling instant.

2. The method of claim 1, wherein the obtaining first sample data comprises:

acquiring second sample data, wherein the second sample data comprises a third parameter value of the storage device at the sampling time of the first time period;

and smoothing the third parameter value at each sampling moment to obtain the first sample data.

3. The method of claim 2, wherein predicting a second parameter value for the storage device at a sampling instant of a second time period based on the first parameter value between the sampling instant of the first target extremum to the particular sampling instant comprises:

determining whether the number of the first parameter values between the sampling time of the first target extreme value and the specific sampling time is greater than a second threshold value;

selecting a portion of said first parameter values from said first parameter values between the sampling instants of said first target extremum to said particular sampling instant if said number is greater than said second threshold;

predicting a second parameter value of the storage device at a sampling instant of a second time period based on the portion of the first parameter value.

4. The method of claim 3, wherein predicting a second parameter value of the storage device at a sampling instant of a second time period based on the portion of the first parameter value comprises:

determining a linear relationship of the parameter value of the storage device to the sampling time based on the portion of the first parameter value;

and predicting a second parameter value of the storage device at the sampling time of the second time period according to the linear relation and the first parameter value at the specific sampling time.

5. The method of claim 4, wherein said determining a linear relationship of the parameter values of the storage device to the sampling time based on the portion of the first parameter values comprises:

determining a tangent line of a curve formed by the part of the first parameter values and the corresponding sampling time by taking a point formed by the first target extreme value and the corresponding sampling time as a fixed point; wherein the fixed point is a point on the tangent line;

and determining the linear relation represented by the tangent line as the linear relation between the parameter value of the storage device and the sampling moment.

6. The method of claim 4, wherein said determining a linear relationship of the parameter values of the storage device to the sampling time based on the portion of the first parameter values comprises:

obtaining a prediction straight line according to a point formed by the first target extreme value and the corresponding sampling time and a point formed by the specific sampling time and the corresponding first parameter value;

and taking the linear relation represented by the prediction straight line as the linear relation between the parameter value of the storage device and the sampling time.

7. The method of claim 1, wherein the method further comprises:

determining a first envelope curve and a second envelope curve of a curve formed by the first parameter values of the sampling moments and the corresponding sampling moments; wherein the first and second envelope lines are on opposite sides of the curve;

finding out a second target extreme value from fourth parameter values of each sampling moment of the first envelope curve; wherein a time interval between the sampling time corresponding to the second target extreme value and the specific sampling time is greater than the first threshold;

finding out a third target extreme value from fifth parameter values of each sampling moment of the second envelope curve; wherein a time interval between the sampling time corresponding to the third target extremum and the specific sampling time is greater than the first threshold;

predicting a sixth parameter value of the storage device at the sampling time of a second time period based on the fourth parameter value between the sampling time corresponding to the second target extreme value and the specific sampling time; and

predicting a seventh parameter value of the storage device at a sampling time of a second time period based on the fifth parameter value between the sampling time of the third target extremum and the specific sampling time;

determining the second parameter value at the ith moment based on the sixth parameter value and the seventh parameter value at the ith moment, so as to obtain the second parameter value of the storage device at the sampling moment of a second time period; wherein i is any one of the sampling moments of the second time period.

8. The method of claim 7, wherein the method further comprises:

if the difference value between the actual parameter value at the current moment and the corresponding second parameter value is larger than a third threshold value, finding out a fourth target extreme value from the actual parameter value at the sampling moment before the current moment;

predicting an eighth parameter value of the storage device at a sampling time of a third time period based on the actual parameter value between the sampling time of the fourth target extremum and the current time; wherein the third time period is the same as the second time period in number of sampling instants.

9. The method of any of claims 1 to 8, wherein the method further comprises:

and when the second parameter value of the second time period and/or the eighth parameter value of the third time period are/is larger than a fourth threshold value, outputting alarm information.

10. A prediction apparatus, comprising:

the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring first sample data, and the first sample data comprises a first parameter value of the storage device at the sampling moment of a first time period;

the searching module is used for searching a first target extreme value from the first parameter values of the sampling moments, wherein the time interval between the sampling moment of the first target extreme value and a specific sampling moment is greater than a first threshold value, and the specific sampling moment belongs to the first time period;

a prediction module to predict a second parameter value of the storage device at a sampling instant of a second time period based on the first parameter value between the sampling instant of the first target extremum to the particular sampling instant.