CN114840581A

CN114840581A - Method and device for generating dynamic threshold for equipment early warning based on statistical model

Info

Publication number: CN114840581A
Application number: CN202210563435.1A
Authority: CN
Inventors: 徐玥
Original assignee: Chongqing Cloud Core Intelligent Technology Co ltd
Current assignee: Chongqing Cloud Core Intelligent Technology Co ltd
Priority date: 2022-05-20
Filing date: 2022-05-20
Publication date: 2022-08-02

Abstract

The application provides a method and a device for generating a dynamic threshold value for equipment early warning based on a statistical model. The method comprises the following steps: acquiring historical data of the latest (n +1) days of operation of a plurality of devices, wherein n is a natural number greater than 1; processing the historical data of the last (n +1) days to generate device objects for the plurality of devices; processing the device object based on a statistical model to generate fitting tuning values for result objects of the plurality of devices; and generating a dynamic threshold value for each of the plurality of devices for device pre-warning by using the result objects for the plurality of devices. The method improves the precision of dynamic threshold prediction; and the mathematical models respectively trained by a plurality of devices are encapsulated in the result object, so that network transmission is facilitated, and the storage space occupied by the mathematical models is reduced.

Description

Method and device for generating dynamic threshold for equipment early warning based on statistical model

Technical Field

The invention relates to the technical field of equipment early warning, in particular to a method and a device for generating a dynamic threshold value for equipment early warning based on a statistical model.

Background

At present, an anomaly detection technology is widely applied in various industrial fields, and is particularly widely applied to services of detection, early warning, alarming and the like of mass equipment in an industrial internet scene. This is because, in the production management activities of manufacturing enterprises, the operation state of a device or a system is monitored in real time, and an early warning, an alarm and a response are timely issued for an abnormal state thereof, which is a necessary measure for avoiding production suspension or service interruption.

Most of the current general anomaly detection technologies implement early warning or alarm for mass devices according to a preset static threshold value and in combination with a preset early warning rule. However, the static threshold cannot adapt to normal data fluctuation of industrial equipment due to influence factors such as environmental influence, four-season air temperature, local climate and the like, and false alarms such as false alarm early warning or false alarm can be caused.

Therefore, it is necessary to use dynamic thresholds to warn or alarm the device to effectively reduce or avoid false alarms for the device.

Disclosure of Invention

In order to solve the above problems, the present application provides a method and an apparatus for generating a dynamic threshold for device early warning based on a statistical model, so as to perform device early warning by using the dynamic threshold, so as to effectively avoid the problem of false alarm of the device existing at present.

In a first aspect, the present invention provides a method for generating a dynamic threshold for device early warning based on a statistical model, including:

acquiring historical data of the latest (n +1) days of operation of a plurality of devices, wherein n is a natural number greater than 1;

processing the historical data for the last (n +1) days to generate device objects for the plurality of devices;

processing the device objects based on a statistical model to generate result objects for the plurality of devices;

and generating a dynamic threshold value for each of the plurality of devices for device pre-warning by using the result objects for the plurality of devices.

Further, the processing the historical data of the last (n +1) days to generate device objects for the plurality of devices includes:

preprocessing the historical data of the last (n +1) days to generate a time sequence with a time span of (p +1) days for each device, wherein p is not more than n;

encapsulating the identification corresponding to each device with the time series for each device in a device object for the plurality of devices.

Further, the result object includes a fitted alignment value for each device; the processing the device object based on the statistical model to generate a result object for the plurality of devices includes:

extracting a time sequence for each device from the device object, and dividing the time sequence into training samples and fitting samples;

processing the training samples based on a cubic exponential smoothing statistical model, and determining a cubic exponential smoothing mathematical model for each device;

processing the training sample and the fitting sample according to a cubic exponential smoothing mathematical model aiming at each device, and determining a fitting adjustment value aiming at each device;

and encapsulating the identification corresponding to each device with the cubic exponential smoothing mathematical model and the fitting adjustment value aiming at each device in a result object aiming at the plurality of devices.

Further, the dividing the time series into training samples and fitting samples includes:

and taking the sampling point of the first q days in the time sequence with the time span of (p +1) days as a training sample, and taking the sampling point of the last (p-q +1) days in the time sequence with the time span of (p +1) days as a fitting sample.

Further, the processing the training samples based on the cubic exponential smoothing statistical model to determine a cubic exponential smoothing mathematical model for each device includes:

determining the initial values of the ternary smoothing parameters in the statistical model by the cubic exponential smoothing method;

determining respective initial values of a smooth value sequence, a trend component value sequence and a seasonal component value sequence in the cubic exponential smoothing statistical model;

determining the optimal estimation value, the smooth value sequence, the trend component value sequence and the seasonal component value sequence of the ternary smoothing parameters in the statistical model of the cubic exponential smoothing method based on a least square method according to the training sample, the initial values of the ternary smoothing parameters, the smooth value sequence, the trend component value sequence and the seasonal component value sequence;

and determining the cubic exponential smoothing mathematical model for each device described by the respective most recent data points of the respective optimal estimated values of the ternary smoothing parameters, the respective smoothed value sequences, the respective trend component value sequences, and the respective seasonal component value sequences.

Further, the processing the training samples, the fitting samples, and determining the fitting calibration value for each device according to a cubic exponential smoothing mathematical model for each device includes:

determining an initial value of the fitted alignment value for each device;

predicting a calibration sample of each device according to the training sample, the cubic exponential smoothing mathematical model and the initial value of the fitting calibration value;

and determining a fitting adjustment value for each device based on a least square method according to the adjustment sample and the fitting sample.

Further, the dynamic threshold is used for respectively carrying out early warning on the equipment in a plurality of periods within at least one specified natural day.

In a second aspect, the present invention provides an apparatus for generating a dynamic threshold for equipment early warning based on a statistical model, including:

the device comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring historical data of the latest (n +1) days of operation of a plurality of devices, and n is a natural number greater than 1;

a device object processing unit configured to process the history data of the last (n +1) days and generate device objects for the plurality of devices;

a result object generation unit for processing the device object based on a statistical model to generate a result object for the plurality of devices;

and the dynamic threshold generating unit is used for generating a dynamic threshold for early warning of the equipment for each of the plurality of equipment by using the result objects for the plurality of equipment.

In a third aspect, the invention provides a computer device comprising a memory storing a computer program and a processor executing the computer program to perform the steps of the method described in the first aspect.

In a fourth aspect, the present invention provides an apparatus early warning method, including: generating a dynamic threshold according to any of the methods described in the first aspect; and respectively carrying out early warning on a plurality of devices by utilizing the dynamic threshold.

These and other aspects of the present application will be more readily apparent from the following description of the embodiment(s).

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a schematic flow chart illustrating a method for generating a dynamic threshold for early warning of equipment based on a statistical model according to an embodiment of the present invention;

FIG. 2 is a schematic diagram illustrating an apparatus for generating a dynamic threshold for early warning of a device based on a statistical model according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a system for generating dynamic thresholds for device early warning based on statistical models, according to an embodiment of the present invention;

FIG. 4 is a flow chart of another method for generating dynamic thresholds for device forewarning based on statistical models, in accordance with an embodiment of the present invention;

fig. 5 is a schematic performance comparison diagram of a method for generating a dynamic threshold for device early warning based on a statistical model according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.

To accurately describe the technical contents in the present application and to accurately understand the present application, the terms used in the present specification are given the following explanations or definitions before the description of the specific embodiments.

Time series refers to a set of observed or recorded values of the same variable arranged in chronological order.

Sampling is to express the analog quantity with continuous occurrence time by discrete data points arranged according to the sequence of the occurrence time. For example, the amplitude value of the signal is extracted as a sample at fixed or variable time intervals from the signal continuous on the time axis, so that the signal continuous on the time axis becomes a pulse sequence discrete in time. Where the time interval between samples is called the sampling period (in seconds or s) and its inverse is called the sampling frequency (in number of samples per second or samples/s, or hertz Hz). And continuously sampling, and arranging the obtained numerical values of all sampling points according to the sequence of the sampling time to form a time sequence.

Autoregressive Moving Average (ARMA). The ARMA model is one of the important methods for processing time series.

Exponential smoothing, or Exponential Moving Average, is a Moving Average weighted exponentially down. The weight of each data decreases exponentially with time, and the closer the data is to the current time, the higher the weight is.

Root Mean Square Error (RMSE), the Square Root of the ratio of the sum of squares of the observed and true deviations to the number of observations.

Open source library statmodels are a package of library functions in python for statistical analysis, e.g., testing the model for linear significance without knowledge of the mathematical model, hypothesis testing, regression analysis, time series analysis, etc. The Holt-Winters function in the open source library statmodels can only generate a corresponding mathematical model for the time sequence of one device at a time, and under the application scene of mass devices in the industrial internet, the mass devices correspond to mass models. These massive models not only lead to maintenance difficulties, but also do not facilitate network transmission and interaction between network nodes.

When the deep learning is used for training the mathematical model, a large amount of data needs to be acquired, and a large amount of computing resources, such as NPU (non-trivial partial Unit), GPU (graphics processing Unit) and other computing accelerating devices are used, so that the trained mathematical model can be converged after a large amount of computing time is consumed.

As shown in fig. 1, a method for generating a dynamic threshold for device early warning based on a statistical model according to an embodiment of the present invention includes:

s10: acquiring historical data of the latest (n +1) days of operation of a plurality of devices, wherein n is a natural number greater than 1;

s20: processing the historical data of the last (n +1) days to generate device objects for the plurality of devices;

s30: processing the device objects based on a statistical model to generate result objects for the plurality of devices;

s40: and generating a dynamic threshold value for each of the plurality of devices for device pre-warning by using the result objects for the plurality of devices.

In some embodiments, steps S10-S30 are performed periodically at a first time interval, e.g., one week, one natural month, one quarter. At this time, in the history data of the last (n +1) day acquired in step S10, n is not less than 7, not less than 28, or not less than 120.

In some embodiments, after step S30, step S40 is performed periodically at a second time interval, e.g., the second time interval is one natural day or 2 natural days, etc. Accordingly, the respective dynamic thresholds for the plurality of devices generated in step S40 are used to warn the devices respectively for a plurality of periods within a specified natural day or 2 natural days (e.g., every half hour of 24 hours in a natural day).

In this way, the result object may be used periodically or non-periodically at small time intervals to generate a dynamic threshold for device pre-warning for each of the plurality of devices, and historical data of device operation may be acquired periodically or non-periodically at large time intervals to generate device objects and mathematical model objects for the plurality of devices.

In step S10, the historical data may be obtained from a data warehouse managed by the cloud server through a web service, or may be obtained from an edge-side data warehouse, such as a data warehouse disposed at an edge side of a large number of intelligent sensing devices that are set in an intelligent factory and set by relying on an industrial internet.

In step S10, the multiple devices may be multiple devices located relatively close to each other in geographic location, and may be multiple nodes located at the same level or multiple nodes located at different levels in the industrial internet.

In some embodiments, said processing said historical data of said last (n +1) days to generate device objects for said plurality of devices comprises: preprocessing the historical data of the last (n +1) days to generate a time sequence with a time span of (p +1) days for each device, wherein p is not more than n; encapsulating the identification corresponding to each device with the time series for each device in a device object for the plurality of devices.

Above, naturally, the time span corresponding to the generated time series may be different after the history data of the last (n +1) days of different devices is preprocessed, corresponding to the authenticity and validity of the history data. However, if the preset minimum number of days is satisfied, the time series of the device is considered to be valid, and the device can be packaged into a corresponding device object and used as data for training a mathematical model. During training, the time span corresponding to each time sequence can be determined quickly in advance. Specifically, preprocessing the history data of the last (n +1) days for each device to generate a time series with a time span of (p +1) days for each device may refer to step S32 described later.

In the integrated development environment of python language, a device class having the identifier corresponding to the device and the time sequence for the device is established, and the device class is instantiated to obtain a plurality of device objects for different devices or one device object for all devices. Or, through the deserialization operation of the python-supported pickle module, a python-supported device object is created from a file (such as an EXCEL file) which stores the identifier corresponding to the device and the time sequence of the device.

In some embodiments, the result object includes a fitted alignment value for each device; the processing the device object based on the statistical model to generate a result object for the plurality of devices includes:

Subsequently, through the serialization operation of the python-supported pickle module, the result objects of a plurality of devices in the python program can be saved in the file, and the result objects for the plurality of devices run in the file are saved in the file, so that the integrated development environment separated from the python language is realized, the result objects are permanently stored, and the result objects are conveniently exchanged among a local storage medium, a database and different storage media.

In some embodiments, the dividing the time series into training samples and fitting samples includes: and taking the sampling point of the first q days in the time sequence with the time span of (p +1) days as a training sample, and taking the sampling point of the last (p-q +1) days in the time sequence with the time span of (p +1) days as a fitting sample, wherein q is a natural number less than or equal to p.

In the above, according to the requirements of the model training precision and the fitting calibration value precision, the time sequence can be flexibly divided into training samples and fitting samples having a temporal precedence relationship. Here, the time sequence is divided by default according to the minimum unit of the number of sampling points corresponding to each natural day, that is, the time span corresponding to the training sample and the fitting sample is an integral multiple of the number of natural days. Moreover, the time span corresponding to the training sample is usually larger than the time span corresponding to the fitting sample, for example, the time span corresponding to the training sample is p, and the time span corresponding to the fitting sample is 1.

In some embodiments, the processing the training samples based on a cubic exponential smoothing statistical model to determine a cubic exponential smoothing mathematical model for each device comprises:

determining respective initial values of ternary smoothing parameters in a cubic exponential smoothing method statistical model;

In some embodiments, the initial values of the respective ternary smoothing parameters in the cubic exponential smoothing statistical model may be determined by referring to the description of section 3.2) below.

In some embodiments, the initial values of the smoothed value sequence, the trend component value sequence and the seasonal component value sequence in the cubic exponential smoothing statistical model are determined according to the description of section 3.2) below.

In some embodiments, the optimal estimation value of each of the ternary smoothing parameters, the smoothed value sequence, the trend component value sequence, and the seasonal component value sequence in the cubic exponential smoothing statistical model is determined based on a least square method according to the training sample, the initial value of each of the ternary smoothing parameters, the smoothed value sequence, the trend component value sequence, and the seasonal component value sequence, which are described in 3.2) and 3.3) below.

In some embodiments, a cubic exponential smoothing statistical model, such as the myHolt-Winters function, may be implemented using quadratic development with Holt-Winters functions supported by python language. At this time, the training sample, the initial value of each of the ternary smoothing parameters, the smoothing value sequence, the trend component value sequence, the initial value of each of the seasonal component value sequence, and the number of sampling points k corresponding to the cycle period in the seasonal component value sequence are input parameter values required by the operation of the myHolt-winter function.

In some embodiments, determining the cubic exponential smoothing mathematical model for each device described by the respective most recent data points of the respective optimal estimated values of the ternary smoothing parameters, the respective smoothed value sequences, trend component value sequences, and seasonal component value sequences may refer to the description of section 3.4) below.

Therefore, when the result object is generated, the historical data for training and the intermediate data generated in the calculation process are removed, and the storage space occupied by the mathematical models corresponding to the multiple devices is further reduced. The output result object file is smaller, for example, 90% of the storage space can be saved compared with the result object file generated by directly using ARMA in statunmolds. For example, when a mathematical model for motor bearing temperature is stored as a postfixed file using pkl of python, the size of the file is approximately 3.14kb, which saves at least 90% of storage space compared to 10.5MB for files generated using the ARMA model in statmodels.

In some embodiments, the processing the training samples, the fitting samples, and determining the fitting alignment value for each device according to a cubic exponential smoothing mathematical model for each device comprises:

determining an initial value of the fitted alignment value for each device;

Therefore, compared with a deep learning mode, the method is based on the cubic exponential smoothing statistical model, and can finish the training of the mathematical model aiming at a plurality of devices by using smaller data volume; the method has the advantages that the function of the statistical model based on the cubic exponential smoothing method is realized in a batch mode by utilizing the secondarily developed source code myHolt-Winters function, the time sequences of a plurality of devices are received at the same time to serve as the input of the function, and the result objects obtained by respectively training the time sequences of the devices are packaged to serve as the output of the function; and saving the result objects aiming at the plurality of devices into a file, thereby reducing the storage space occupied by the mathematical models corresponding to the plurality of devices.

Fig. 3 shows a structural block diagram of an equipment early warning system deployed in the method for performing equipment early warning based on a dynamic threshold generated by a statistical model according to the embodiment of the present invention. In the device early warning method according to this embodiment, a dynamic threshold is generated according to any one of the methods described above that generates a dynamic threshold for device early warning based on a statistical model, and early warning is performed on a plurality of devices using the dynamic threshold.

The system comprises an equipment database 1, a mathematical model 2 and an early warning rule 3. The device database 1, for example, running on a database management apparatus, is used to store massive historical data generated by massive devices in the industrial field during operation and dynamic thresholds updated at regular time. The dynamic threshold value updated at regular time is used for device early warning according to the early warning rule 3 described below.

The mathematical model 2 preprocesses the historical data of the equipment to obtain a training sample and a fitting sample. Based on the training samples of each device, a mathematical model for each device is trained using a cubic exponential smoothing statistical model, and a final mathematical model is generated after tuning. And based on the fitting sample, predicting by using the determined cubic exponential smoothing statistical model to obtain a predicted value sequence aiming at each device, storing the predicted value sequences corresponding to a plurality of devices into the device database 1, accessing the device database 1 in a network interconnection mode, and acquiring the predicted value sequences corresponding to each device to perform device early warning.

And the early warning rule 3 acquires a predicted value sequence corresponding to the identifier of the equipment from the equipment database, and selects a predicted value corresponding to the current time from the predicted value sequence as a dynamic threshold value by using a preset early warning rule according to the acquired current time for early warning of the equipment. For example, when the time span corresponding to the predicted value sequence of the equipment is one natural day and the corresponding updating time interval is 30 minutes, the predicted value sequence of the equipment comprises 48 values in total. And calculating the number of minutes from the current time to the time of 00:00:00 in the day, dividing the number of minutes by 30, removing the remainder of the obtained quotient to obtain a value a, and determining the (a +1) th value in the sequence of predicted values as the dynamic threshold corresponding to the current time. If the current time is 2022-4-1100: 15:00, the number of minutes between the current time and the time of 00:00: 00:00 the day is 15, the number of minutes is divided by 30, a remainder is removed from an obtained quotient to obtain a value of 0, a (0+1) th value in the predicted value sequence is a dynamic threshold value corresponding to the current time, and a first numerical value in the predicted value sequence is taken as a dynamic threshold value machine corresponding to the current time for equipment early warning. In the above, when every 30 minutes is taken as a time period, that is, the current time is within the time period of 2022-4-1100: 00:00 to 2022-4-1100: 30:00, the first numerical value in the sequence of common predicted values is used as the dynamic threshold.

If the current time is 2022-4-1100: 35:00, the number of minutes from the current time to the time of the day 00:00:00 is 35, the number of minutes is divided by 30, the remainder is removed from the obtained quotient to obtain a value 1, the (1+1) th value in the sequence of predicted values is the dynamic threshold corresponding to the current time, and the first numerical value in the sequence of predicted values is taken as the threshold of the current time period. And still taking every 30 minutes as a time period, namely, when the current time is within the time period from 2022-4-1100: 31:00 to 2022-4-1101: 00:00, the second numerical value in the common predictive value sequence is used as a dynamic threshold value to carry out equipment early warning. The dynamic threshold corresponding to other time in the day refers to the aforementioned determination, and is not described again. Thus, in a natural day, the dynamic threshold corresponding to each time or time interval is a value in the predicted value sequence.

Thus, the formed early warning rule is as follows: and if the temperature value of the current time of the equipment A is greater than the dynamic threshold value corresponding to the current time, alarming aiming at the equipment A. As shown in fig. 4, the method of the embodiment of the present invention periodically updates the mathematical model with a natural day as the aforementioned first time interval, and periodically generates the dynamic threshold with a natural day as the aforementioned second time interval, which includes the following steps S31 to S35.

Step S31: historical data of a plurality of devices on the last (n +1) days are obtained from a device database every day at a fixed time of 00:00:00 (e.g., automatically triggered by a set rule), and the value of n is configured by a user.

The historical data may include samples of data of different physical meanings (such as state quantities and index quantities) generated by different devices or the same device in operation, and may be a time sequence arranged according to the sequence of sampling time. These data may correspond to the same sampling instants on the time axis, with the same sampling period. The number of samples may be the same between different time series, i.e. have the same time span. For example, the temperature and the rotating speed of a motor during long-term continuous operation are respectively collected as two historical data of the motor, and the two historical data respectively correspond to a temperature time series and a rotating speed time series. For example, the respective temperatures and rotational speeds of a plurality of motors with the same power or different powers on the same production line during long-term continuous operation are respectively collected according to the same sampling frequency, so that a plurality of temperature time series and rotational speed time series are formed and used as historical data of the motors on the production line.

In step S32 described later, the history data acquired for the last (n +1) days of the plurality of devices is preprocessed into the history data for the last (p +1) days or (n +1) days of the plurality of devices.

In step S33 described later, data of the preceding n days or the preceding p days that are temporally further forward (i.e., farther) are used as training samples; in step S34 described later, data of the previous day (i.e., the day before the current day) that is further back in time (i.e., closer) is used as a fitting sample for calculating the fitting adjustment value j.

Step S32: and processing the acquired historical data. For example, the acquired historical data of the latest (n +1) days of the multiple devices is processed to obtain the device objects corresponding to the multiple devices.

Hereinafter, a specific operation of the preprocessing will be described by taking as an example the preprocessing of the history data of the last (n +1) days of a single device.

2.1) dividing the acquired data according to the day, namely the natural day. Corresponding to the historical data of the last (n +1) days of the device acquired in the aforementioned step S31, the data of each (n +1) days is obtained by division, and a time stamp corresponding to each day is printed.

And 2.2) respectively carrying out repeated data processing, abandoning processing or head and tail supplementing processing on the data every day.

Firstly, dividing 24 hours of each natural day into m corresponding time periods and m corresponding time points according to a specified time interval, and determining m corresponding timestamps. If the specified time interval is 30 minutes, 48 time periods and 48 time points are divided, and corresponding 48 time stamps are determined.

In general, there is temporally repetitive data in the acquired data. The reasons for the presence of duplicate data typically include both sampling period and network transmission. For example, under the influence of the factors such as sampling period fluctuation, network transmission delay, network transmission quality, and the like, for a time sequence with a sampling period of 30 seconds, the time of acquiring the first piece of data every day is 00:00:30 during normal sampling and transmission, but under the influence of the network, the time when the data is received by the device database may be 00:00:31 or other times. For example, the sampling period of some data of the device is in the millisecond level (such as vibration of a motor bearing), that is, a plurality of vibration values are collected within one second; for example, the sampling period of other data of the equipment is in the order of seconds (such as the temperature of a motor bearing), namely, only one temperature value is collected in one second. When the data of different classes are stored in a specified time interval, when the sampling period is less than the specified time interval, a plurality of data appear in the specified time interval, and the data are repeated relative to the specified time interval. The network transmission factor is mainly data reporting delay caused by network reasons, and after the data reporting delay collected last time is the same as the data reporting time collected at the current time, the data which are repeated at the current time are generated.

Therefore, according to the divided m time periods, time points or time stamps, temporally repeated data can be removed one by one, that is, duplicate removal. The deduplication method may be to select the most temporally preceding or most temporally following data from among temporally overlapping data, discard other data, select a maximum value, a minimum value, a median value, or the like, or average a number of substitution or aggregation of a plurality of temporally overlapping data.

If the number of the data points in the data of a certain day after the deduplication is performed is less than the preset m data, it indicates that at least one data point for the divided time period is missing in the data of the certain day (if the data is lost, the data will not reach the device database), and the data of the certain day is discarded, that is, the discarding process is performed. If the specified time interval is 30 minutes, after the data of a certain day is subjected to repeated data processing, and the number of data points is less than 48, the data of the day is discarded, that is, the data is not added into the time sequence in the subsequent steps. This is because, if the number of data points on a certain day is too small or insufficient, the data on that day no longer has reference to the training model or the fitting calibration, and therefore the data on that day is discarded, i.e., the discarding process described above.

In addition, when packet loss occurs, data does not reach the device database, and then, in a corresponding time period or time, the data point is empty. Thus, the first piece of data for the day may be aligned to the first time segment as the value at time 00:00:00 of the day, and the subsequent (i +1) th piece of data may be aligned to the (00:00+30 × i:00) th time segment until the last piece of data is the value at time 23:59: 59. At this time, for m, for example, 48 data pieces respectively determine their corresponding timestamps, that is, head and tail are added, or are aligned in time sequence.

In this way, by performing the head and tail compensation processing and the time alignment, it is realized that the data of each day is processed into a time series including m data points which are preset, and in this case, m is an integral multiple of k, so that when resampling is performed in step S35 which will be described later, in accordance with a preset sampling period, for example, 30 minutes, as a sampling period, the time series after resampling includes 48 data points which are temporally shifted from each other for each natural day.

2.3) counting the number of days (p +1) corresponding to the finally obtained data after respectively carrying out repeated data processing, discarding processing or head and tail supplementing processing on the data of each day, wherein p is a natural number less than or equal to n.

2.4) if n is greater than p, this indicates that at least one day of historical data has been discarded. The discarded data is used to supplement the removed empty position with the data of the previous day, and the following data (later in time) is sequentially moved forward in time, so as to finally ensure that the data of the last (p +1) day is a time sequence which is continuous in time. Or, the data of the next day is used for complementing the removed vacant position, the data of the front (more front in time) is sequentially moved backwards in time, and finally the data of the last (p +1) day is ensured to be a time sequence which is continuous in time.

2.5) resampling the daily data by using a larger sampling period to obtain resampled data so as to reduce the data volume. For example, the sampling period is increased from 30 seconds to 30 minutes, i.e., the number of data points is reduced to 1/60 before resampling.

In an industrial internet scenario, a factory or a production line has thousands of devices, and the operating states of the devices are detected, and the number of sampling points generated every day is at least billion. If the model training is performed using raw data, because the amount of data is too large, it may take several hours or even tens of hours to train a single model, which is burdensome and burdensome computational load for plant prediction. Therefore, the data volume can be reduced, the training time of the model can be shortened, and the consumed computing resources can be reduced by utilizing a larger sampling period for resampling.

2.6) splicing the resampled data of the multiple days, namely the (p +1) days into a time sequence with the time span of (p +1) days according to the chronological order.

2.7) packaging the time sequence with the time span of (p +1) days and the identification (such as the unique corresponding identification id) corresponding to the single device into a device object running in a python program.

For the preprocessing of the history data of the recent (n +1) days of the multiple devices, reference may be made to the foregoing 2.1) to 2.7), which is not described again.

After the acquired historical data of the plurality of devices in the last (n +1) days are respectively processed into device objects running in a python program, the device objects corresponding to the plurality of devices are uniformly packaged into device objects applied in step 33, which will be described later, and can be conveniently distributed through a network.

Step S33: the mathematical models corresponding to the plurality of devices are trained using the data processed in step S32, that is, the device objects. The method specifically comprises the following steps:

3.1) traversing the device objects packaged in the step S32, and sequentially acquiring the device objects including each deviceCorresponding equipment objects are extracted, and the time sequence x which corresponds to the equipment and has the time span of (p +1) days is { x } _t And f, wherein 0 ≦ t ≦ m ≦ (p +1) -1, m being the number of corresponding time periods within one natural day or the number of data points after resampling.

3.2) statistical model using cubic exponential smoothing method, time series x ═ x _t And training a mathematical model corresponding to each device.

Specifically, when a cubic exponential smoothing statistical model is used as the mathematical model, the following formulas (1.1) to (1.4) are included:

s _t ＝α(x _t +P _t-k )+(1-α)(s _t-1 +T _t-1 ) (1.1)

T _t ＝β(s _t -s _t-1 )+(1-β)T _t-1 (1.2)

P _t ＝γ(x _t -s _t )+(1-γ)P _t-k (1.3)

x _t+h ＝s _t +hT _t +P _t-k+g (1.4)

wherein, the sequence of smoothing values is expressed as s ═ s _t And the trend component value sequence is recorded as T ═ T _t And recording a sequence of seasonal component values as P ═ P _t And h, wherein the sequence of seasonal component values has an inherent cyclic period within which the number of corresponding sampling points is k. The subscript t in each time series is used for indicating the current time or the t-th data point, t-1 is used for indicating the 1 time or the t-1 th data point before the current time, and t-k is used for indicating the k times or the t-k data points before the current time. t-k + g is used to indicate k-g times before the current time or the t-k + g th data. t + h represents h moments or t + h values after the current moment, and the variable g is the remainder of h on k, namely the remainder after h and k are divided, namely g is less than or equal to h and g is less than or equal to k.

The variables α, β, γ are smoothing parameters, and take values between 0 and 1, respectively, that is, the aforementioned ternary smoothing parameters. The variable k corresponds to the cycle period in which the seasonal component value sequence is intrinsic, and the cycle period is 1 natural day, dataK is 48 when the sampling period of (2) is 30 minutes. The variable h is used to indicate that the h-th data point predicted backward from the current time t is 0 ≦ h ≦ m ≦ p +1, that is, the data of each device operating within (p +1) natural days can be predicted at most by using the mathematical model trained by the historical data of (p +1) natural days. The influence of the initial value of the smooth value sequence s, the initial value of the trend component value sequence T and the initial value of the seasonal component value sequence P on the cubic exponential smoothing statistical model is not obvious, so that s is usually taken for simplicity ₀ ＝x ₀ ,T ₀ ＝x ₁ -x ₀ ，P ₀ ＝0。

Time series x with time span of (p +1) days is set as { x ═ x _t The data of the previous n days or the previous (p +1) days which are more advanced (i.e. farther) in time are used as training samples; the data of the previous day (i.e., the day before) that is further back in time (i.e., closer) is used as a fitting sample for calculating the fitting alignment value j.

And determining the optimal estimated values of the smoothing parameters alpha, beta and gamma by using a training sample and adopting a least square estimation method. Specifically, using the fmin _ l _ bfgs _ b function in the software package script, provided by python, a loss function is constructed as follows (1.5):

wherein x is ^# And for training samples, y is a predicted value sequence predicted by adopting a cubic exponential smoothing statistical model as a mathematical model, q is the number of data points in the training samples or the predicted value sequence, and q is k p. When the cycle period is 1 natural day and the sampling period of data is 30 minutes, q is 48 × p. The above-mentioned loss function is implemented in code, e.g. by python, e.g. by setting the respective initial values of the smoothing parameters alpha, beta, gamma, e.g. in the number of sets [0.3,0.1]And the value intervals, such as the arrays [ (0,1), (0,1)]And the optimized values of alpha, beta and gamma can be obtained by transmitting the functions of fmin _ l _ bfgs _ b supported by a software package of python together, namely the optimal estimated values of alpha, beta and gamma in the least square sense.

3.3) mixing the aboveSubstituting the optimized values of α, β, and γ determined in step 3.2) into equations (1.1) to (1.4) of the cubic exponential smoothing statistical model, so as to calculate a sequence of smoothing values s ═ s { s } corresponding to the specified time span r by using the cubic exponential smoothing statistical model _t T, a sequence of trend component values T ═ T _t P, a sequence of seasonal component values P ═ P _t }, sequence of predicted values y ^* In this case, r is an integer multiple of the number k of data points in the cycle.

3.4) the sequence of smoothed values s ═ s) determined in step 3.3) above _t The last value of the trend component value sequence T ═ T _t The last value of the seasonal component value sequence P ═ P _t The values in the last cycle period, i.e. the latest data point, and the optimized values of α, β, γ determined in the step 3.2), and the default fitting calibration value j ═ 1, corresponding to each device, are packaged as an initial result object for the device.

That is, the time series x with the time span of (p +1) days corresponding to the device extracted in the step 3.1) and the smoothed value series s ═ { s } calculated in the step 3.3) are excluded _t T, a sequence of trend component values T ═ T _t P, a sequence of seasonal component values P ═ P _t The other data after the nearest data point is removed. In this way, the size of the file of the final mathematical model can be reduced.

Repeating the steps 3.2) to 3.4) to obtain initial result objects for each device after encapsulation.

3.5) encapsulating the initial result objects of the plurality of devices into a result object group as a preliminary mathematical model.

Step S34: and calculating a fitting adjustment value j for each device by using the fitting sample, for example, determining an optimal estimation value of the fitting adjustment value corresponding to each device by using a least square estimation method. The method specifically comprises the following steps:

the data of the previous day (i.e., the previous day of the current day) of each device is predicted by using the mathematical model of each device determined in the foregoing step S33, so as to obtain prediction result data. Each data value in the prediction result data is obtained by multiplying a numerical value calculated by a statistical model of a cubic exponential smoothing method by a default fitting adjustment value j.

And determining the optimal estimated value of the fitting calibration value j by using the fitting sample and adopting a least square estimation method. Specifically, using the fmin _ l _ bfgs _ b function in the software package script, provided by python, a loss function is constructed as follows (1.6):

wherein x is ^{^} To fit the sample, y ^{^} In order to obtain predicted result data obtained by predicting data of the previous day of the equipment (i.e., the previous day of the current day) by using the mathematical model of the equipment determined in the foregoing step S33, N is the number of data points in the fitting sample or the predicted result data, N is the number k of data points in the cycle period corresponding to the foregoing sequence of seasonal component values, and when the cycle period is 1 natural day and the sampling period of the data is 30 minutes, N is 48. The loss function is realized by codes, for example, python is adopted, so that the optimal estimated value of the fitting tuning value j can be obtained. And updating the optimal estimation value of the fitting adjustment value j to the corresponding result object, namely updating the fitting adjustment value j corresponding to the equipment in the packaged result object group.

And repeating the steps until the fitting calibration value j of all the devices is updated. Thus, the error of the prediction result can be reduced, and the prediction precision can be improved.

Step S35: and using the trained mathematical model object group to predict data in at least one natural day specified by the equipment in the future. The method specifically comprises the following steps:

traversing a plurality of devices to obtain the corresponding identification of each device, transmitting the identification of the device and the appointed prediction period (such as a natural day) into a mathematical model to obtain the predicted value sequence x of each device in the appointed natural day in the future _h 。

Step S36: the predicted value sequence x acquired in step S35 _h Is multiplied by the fitting calibration value determined in the previous step S34j, obtaining a final prediction result, storing the final prediction result into an equipment database to serve as a dynamic threshold value, and providing the dynamic threshold value for the early warning rule to use.

By adopting the cubic exponential smoothing statistical model (for example, the code is realized as myHolt-Winters), Holt-Winters in statunmodel and ARMA in statunmodel, the test data (including historical data of two devices) stored in the local EXCEL file are respectively processed to obtain a prediction result, and performance comparison experiments such as prediction precision or error, training time length, file size of the mathematical model and the like are carried out, as shown in FIG. 5.

As shown in fig. 5, the smaller the Mean Square Error (MSE) value, the closer the predicted value is to the actual value. It can be seen that the cubic exponential smoothing statistical model of the embodiment of the invention shortens the training time and reduces the file size of the mathematical model under the condition of lower loss and higher precision.

Therefore, the method for generating the dynamic threshold value for the equipment early warning based on the statistical model, provided by the embodiment of the invention, has the advantages that the mathematical models respectively trained by a plurality of equipment are used as result objects to be packaged in one result object, so that the network transmission is facilitated; historical data of the equipment and intermediate data generated by calculation are removed, and only the latest data points (one value or a plurality of values which are the most backward in time) of each time sequence are reserved in the result object, so that the storage space occupied by the mathematical model is favorably reduced; and a fitting calibration value is introduced, so that the dynamic threshold prediction precision is further improved, and the error is reduced.

In some embodiments, as shown in fig. 2, the apparatus for generating a dynamic threshold for a device early warning based on a statistical model according to an embodiment of the present invention includes:

an obtaining unit 100, configured to obtain historical data of a last (n +1) days in which a plurality of devices operate, where n is a natural number greater than 1;

a device object processing unit 200, configured to process the history data of the last (n +1) days to generate device objects for the multiple devices;

a result object generating unit 300, configured to process the device object based on a statistical model, and generate result objects for the plurality of devices;

a dynamic threshold generating unit 400, configured to generate, by using the result objects for the multiple devices, a dynamic threshold for device pre-warning for each of the multiple devices.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the above-described method embodiments when executing the computer program.

In an embodiment, a computer-readable storage medium is provided, in which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

The foregoing description is intended to be illustrative rather than limiting, and it will be appreciated by those skilled in the art that many modifications, variations, or equivalents may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A method for generating a dynamic threshold for device forewarning based on a statistical model, comprising:

processing the historical data of the last (n +1) days to generate device objects for the plurality of devices;

2. The method of claim 1,

the processing the historical data of the last (n +1) days to generate device objects for the plurality of devices, comprising:

3. The method of claim 1,

the result object includes a fitted alignment value for each device;

the processing the device object based on the statistical model to generate a result object for the plurality of devices includes:

4. The method of claim 3,

the dividing the time series into training samples and fitting samples includes:

5. The method of claim 3,

processing the training samples based on a cubic exponential smoothing statistical model to determine a cubic exponential smoothing mathematical model for each device, comprising:

6. The method of claim 5,

the processing the training samples, the fitting samples, and determining the fitting calibration value for each device according to a cubic exponential smoothing mathematical model for each device includes:

determining an initial value of the fitted alignment value for each device;

7. The method according to any one of claims 1 to 6,

the dynamic threshold is used for early warning the equipment in a plurality of periods within at least one designated natural day.

8. An apparatus for generating a dynamic threshold for device forewarning based on a statistical model, comprising:

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.

10. An equipment early warning method is characterized by comprising the following steps:

the method for generating a dynamic threshold value for equipment early warning based on the statistical model according to any one of claims 1 to 7, wherein the dynamic threshold value is generated;

and respectively carrying out early warning on a plurality of devices by utilizing the dynamic threshold.