CN110377447B - Abnormal data detection method and device and server - Google Patents
Abnormal data detection method and device and server Download PDFInfo
- Publication number
- CN110377447B CN110377447B CN201910646013.9A CN201910646013A CN110377447B CN 110377447 B CN110377447 B CN 110377447B CN 201910646013 A CN201910646013 A CN 201910646013A CN 110377447 B CN110377447 B CN 110377447B
- Authority
- CN
- China
- Prior art keywords
- prediction
- data
- monitoring data
- sample
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0766—Error or fault reporting or storing
- G06F11/0787—Storage of error reports, e.g. persistent data storage, storage using memory protection
Abstract
The invention discloses an abnormal data detection method, an abnormal data detection device and a server, wherein the abnormal data detection method comprises the following steps: acquiring monitoring data of a target detection object at the current moment and historical monitoring data of a preset time period before the current moment; according to the historical monitoring data, determining the prediction data of the target detection object at the current moment; determining the prediction deviation of the target detection object at the current moment according to the prediction data and the monitoring data; and carrying out anomaly detection on the monitoring data of the target detection object at the current moment based on a preset anomaly probability function and the prediction deviation. According to the invention, the abnormal degree of the monitoring data is measured by utilizing the difference between the prediction data reflecting the change trend of the data and the actual monitoring data, so that the accuracy and the detection efficiency of abnormal data detection are improved.
Description
Technical Field
The invention relates to the technical field of computers, in particular to an abnormal data detection method, an abnormal data detection device and a server.
Background
Some interactive applications often require monitoring to alert when an anomaly is found. Taking a game as an example, data corresponding to a plurality of game critical paths such as online, login, matching, game-play and the like need to be monitored, so as to give an alarm when abnormal data is found.
In the related art, the change trend of the data is ignored during abnormal data detection, only the difference value between the monitoring data at the current moment and the monitoring data at the previous moment is compared with a fixed threshold, and when the difference value is greater than the fixed threshold, the monitoring data at the current moment is determined to be abnormal data. Obviously, the accuracy of the abnormal data detection method is poor, and the selection of the fixed threshold requires rich operation and maintenance experience, thereby resulting in low data detection efficiency.
Disclosure of Invention
In order to solve the problems in the prior art, embodiments of the present invention provide a method, an apparatus, and a server for detecting abnormal data. The technical scheme is as follows:
in one aspect, a method for detecting abnormal data is provided, where the method includes:
acquiring monitoring data of a target detection object at the current moment and historical monitoring data of a preset time period before the current moment;
according to the historical monitoring data, determining the prediction data of the target detection object at the current moment;
determining the prediction deviation of the target detection object at the current moment according to the prediction data and the monitoring data;
and carrying out anomaly detection on the monitoring data of the target detection object at the current moment based on a preset anomaly probability function and the prediction deviation.
Optionally, the performing, based on the preset abnormal probability function and the prediction deviation, abnormal detection on the monitoring data of the target detection object at the current time includes:
determining an abnormal probability value based on a preset abnormal probability function and the prediction deviation;
judging whether the abnormal probability value is greater than or equal to a preset threshold value or not;
and if the judgment result is yes, determining that the monitoring data of the target detection object at the current moment is abnormal.
In another aspect, an abnormal data detecting apparatus is provided, the apparatus including:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring monitoring data of a target detection object at the current moment and historical monitoring data of a preset time period before the current moment;
the first determining module is used for determining the prediction data of the target detection object at the current moment according to the historical monitoring data;
the second determination module is used for determining the prediction deviation of the target detection object at the current moment according to the prediction data and the monitoring data;
and the anomaly detection module is used for carrying out anomaly detection on the monitoring data of the target detection object at the current moment based on a preset anomaly probability function and the prediction deviation.
Optionally, the first determining module includes:
the third determining module is used for determining the time sequence attribute of the target detection object;
the fourth determining module is used for determining a target prediction strategy matched with the target detection object according to the time sequence attribute;
and the fifth determining module is used for determining the prediction data of the target detection object at the current moment according to the target prediction strategy and the historical monitoring data.
Optionally, when the target prediction strategy is a prediction strategy based on a prediction model, the fifth determining module includes:
a model determination module for determining a prediction model; the predictive model comprises a machine learning model for unsupervised machine learning determination based on sample historical monitoring data of the target detection object;
and the first prediction module is used for predicting the target detection object at the current moment based on the prediction model and the historical monitoring data to obtain the prediction data of the target detection object at the current moment.
Optionally, the apparatus further comprises:
the second prediction module is used for predicting the target detection object at the time point corresponding to each sample historical monitoring data based on the prediction model to obtain the prediction historical data corresponding to each sample historical monitoring data;
a sixth determining module, configured to determine a difference between the prediction historical data and corresponding sample historical monitoring data to obtain a sample prediction difference set;
and the seventh determining module is used for determining the parameter values of the parameters in the preset abnormal probability function according to the sample prediction deviations in the sample prediction deviation set.
Optionally, the seventh determining module includes:
an eighth determining module, configured to determine an average of the sample prediction deviations in the set of sample prediction deviations and a standard deviation of the sample prediction deviations in the set of sample prediction deviations;
and the ninth determining module is used for taking the average value and the standard deviation of the sample prediction deviations in the sample prediction deviation set as the parameter values of the corresponding parameters in the preset abnormal probability function.
Optionally, the seventh determining module includes:
a first average determination module that determines an average of the sample prediction biases in the set of sample prediction biases;
the first function determining module is used for determining a nonlinear fitting function according to regression processing of the sample historical monitoring data;
the first function value determining module is used for determining a function value of the nonlinear fitting function according to the monitoring data at the current moment;
and the tenth determining module is used for taking the average value of the sample prediction deviations in the sample prediction deviation set and the function value of the nonlinear fitting function as the parameter values of the corresponding parameters in the preset abnormal probability function.
Optionally, when the target preset strategy is a prediction strategy based on a mean value, the fifth determining module includes:
the second mean value determining module is used for determining the mean value of the historical monitoring data;
and the third prediction module is used for taking the average value of the historical monitoring data as the prediction data of the target detection object at the current moment.
Optionally, the apparatus further comprises:
the second acquisition module is used for acquiring historical sample monitoring data of the target detection object;
the fourth prediction module is used for determining the average value of the historical monitoring data of the sample to obtain the predicted historical data;
an eleventh determining module, configured to determine a difference between the predicted historical data and each sample historical monitoring data to obtain a sample predicted difference set;
a third average determining module, configured to determine an average of the sample prediction deviations in the sample prediction deviation set;
the second function determining module is used for determining a nonlinear fitting function according to regression processing of the sample historical monitoring data;
the second function value determining module is used for determining the function value of the nonlinear fitting function according to the monitoring data at the current moment;
and the twelfth determining module is used for taking the average value of the sample prediction deviations in the sample prediction deviation set and the function value of the nonlinear fitting function as the parameter values of the corresponding parameters in the preset abnormal probability function.
Optionally, the anomaly detection module includes:
a thirteenth determining module, configured to determine an abnormal probability value based on a preset abnormal probability function and the prediction deviation;
the judging module is used for judging whether the abnormal probability value is greater than or equal to a preset threshold value or not;
and the fourteenth determining module is used for determining that the monitoring data of the target detection object at the current moment is abnormal when the judgment result of the judging module is yes.
In another aspect, a server is provided, which includes a processor and a memory, where at least one instruction, at least one program, a set of codes, or a set of instructions is stored in the memory, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the above abnormal data detection method.
In another aspect, a computer-readable storage medium is provided, in which at least one instruction, at least one program, a set of codes, or a set of instructions is stored, which is loaded and executed by a processor to implement the abnormal data detecting method as described above.
According to the embodiment of the invention, the target detection object is predicted at the current moment through the historical monitoring data of the target detection object to obtain the prediction data, the prediction deviation of the target detection object at the current moment is determined based on the prediction data and the monitoring data, then the abnormality detection is carried out on the monitoring data of the target detection object at the current moment based on the prediction deviation and the preset abnormality probability function, the abnormality degree of the monitoring data is measured by utilizing the difference between the prediction data and the actual monitoring data, and the prediction data fully reflects the change trend of the data, so that the accuracy of the abnormal data detection is greatly improved, the setting of a fixed threshold value is not required to be carried out by depending on rich operation and maintenance experience, and the data detection efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flowchart of an abnormal data detection method according to an embodiment of the present invention;
FIG. 2a is a diagram illustrating an online people curve corresponding to a critical path of a game according to an embodiment of the present invention;
FIG. 2b is a diagram illustrating an authentication success rate curve corresponding to a critical path of a game according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart of a method for determining the prediction data of a target detection object at the current time according to historical monitoring data according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart illustrating another method for determining the prediction data of the target detection object at the current time according to the target prediction strategy and the historical monitoring data according to the embodiment of the present invention;
fig. 5 is a schematic flowchart of another method for determining prediction data of the target detection object at the current time according to the target prediction policy and historical monitoring data according to the embodiment of the present invention;
FIG. 6 is a flow chart illustrating another abnormal data detection method according to an embodiment of the present invention;
FIG. 7a is a schematic diagram illustrating a prediction result of predicting an online people curve at a time point corresponding to each sample historical monitoring data by using a neural network model according to an embodiment of the present invention;
fig. 7b is a schematic diagram of a prediction result obtained by predicting a matching success rate curve at a time point corresponding to each sample historical monitoring data by using an exponential smoothing model according to an embodiment of the present invention;
FIG. 8 is a flow chart illustrating another abnormal data detection method according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of an alarm configuration interface provided by embodiments of the present invention;
fig. 10 is a schematic structural diagram of an abnormal data detection apparatus according to an embodiment of the present invention;
FIG. 11 is a schematic structural diagram of another abnormal data detection apparatus according to an embodiment of the present invention;
FIG. 12 is a schematic structural diagram of another abnormal data detection apparatus according to an embodiment of the present invention;
fig. 13 is a block diagram of a hardware structure of a server according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in other sequences than those illustrated or described herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In the prior art, detection of abnormal data in interactive application is generally based on
Or x (t-1) -x (t) > b, wherein x (t) is the monitoring data at the current moment, x (t-1) is the monitoring data at the previous moment, and a and b are set fixed thresholds. Therefore, the abnormal data detection method does not consider the change trend of the data, and for the data which is in a rapid rising stage and suddenly increases and slows down and the data which is in a continuous rapid falling stage, the abnormal data detection method in the prior art cannot accurately detect the abnormal degree of the data; and the selection of the fixed threshold needs rich operation and maintenance experience, and the data detection efficiency is low.
In view of this, the embodiment of the present invention provides an abnormal data detection method, which predicts the current time through historical monitoring data to obtain predicted data, and measures the abnormal degree of the monitoring data at the current time by using the difference between the predicted data and the monitoring data at the current time, so as to greatly improve the accuracy of abnormal data detection, and set a fixed threshold value without depending on rich operation and maintenance experience, thereby improving data detection efficiency.
Please refer to fig. 1, which is a flowchart illustrating an abnormal data detecting method according to an embodiment of the present invention. It is noted that the present specification provides method steps as described in the examples or flowcharts, but may include more or less steps based on routine or non-inventive efforts. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of sequences, and does not represent a unique order of performance. In actual system or product execution, sequential execution or parallel execution (e.g., parallel processor or multi-threaded environment) may be possible according to the embodiments or methods shown in the figures. Specifically, as shown in fig. 1, the method may include:
s101, acquiring monitoring data of a target detection object at the current moment and historical monitoring data of a preset time period before the current moment.
Wherein the target detection object may be a detection object in a target dimension, the target dimension representing a range of the monitored data. The target detection object represents a detected data object of a certain category, and in this embodiment of the present specification, the target detection object may be divided into different categories according to a data change trend characteristic of the target detection object, where the categories may include, but are not limited to, a time-sequence smooth change category, a time-sequence fluctuating change category, a non-time-sequence fluctuating change category, and the like. The time sequence means that the data at the current moment is associated with the data at the previous moment, and the non-time sequence means that the data at the current moment is not associated with the data at the previous moment.
Taking a game as an example, the target dimension may include but is not limited to a large area, a platform, a province, and the like, that is, the target dimension may be a certain large area, a certain platform, or a certain province, and the like, and the specific division of the large area, the platform, and the province may be set according to the actual situation; the target detection object may be data involved in a game critical path, which may include, but is not limited to, online, login, match, etc., and correspondingly, the target detection object may include, but is not limited to, online population, authentication success rate, match success rate, etc.
In practical application, data of key paths of games, such as online number of people, authentication success rate, matching success rate and the like, are often drawn into corresponding curves, and the change trend of the data is reflected on the corresponding curves. Generally, an online people number curve has two characteristics of superposition and smoothness, as shown in fig. 2a, the current state of the curve changes on the basis of the previous time, and the curve does not have a large-amplitude mutation under normal conditions. The authentication success rate curve shows two characteristics of non-time sequence and volatility, and as shown in fig. 2b, data at the current moment of the curve has no direct relation with data at the previous moment, and the curve is not smooth and is jagged. The matching success rate curve shows two characteristics of time sequence and volatility, and as shown in fig. 7b, data at the current moment of the curve is directly related to data at the previous moment, and the curve is not smooth and is jagged.
In the embodiment of the present specification, the acquisition of the historical monitoring data of the preset time period before the current time may be set according to actual needs. In a specific embodiment, historical monitoring data of a preset number of monitoring granularities before the current time may be obtained according to a preset time interval as the monitoring granularity, for example, the preset time interval may be 5 minutes, and the preset number may be 5, that is, the obtained monitoring data of the current time is xtThe historical monitoring data is { xt-5，xt-4，xt-3，xt-2，xt-1In which xt-1Monitoring data, x, of a time preceding the current timet-2The monitoring data of the first two moments of the current moment, xt-3The monitoring data of the first three moments of the current moment, xt-4The monitoring data of the first four moments of the current moment, xt-5The monitoring data of the first five moments of the current moment.
S103, according to the historical monitoring data, determining the prediction data of the target detection object at the current moment.
In order to improve the accuracy of the predicted data, in this embodiment of the present specification, when determining the predicted data of the target detection object at the current time according to the historical monitoring data, the method may include the steps shown in fig. 3:
s301, determining the time sequence attribute of the target detection object.
Specifically, the time sequence attribute of the target detection object may be determined according to the category of the target detection object, and the time sequence attribute may include a time sequence property and a non-time sequence property. For example, the target detection object of the time-series smooth change category and the time-series fluctuating change category has time-series; the non-temporal volatility variation category is non-temporal.
Taking the data curve corresponding to the game key path as an example, the online number curve and the matching success rate curve have time sequence, and the authentication success rate curve has non-time sequence.
S303, determining a target prediction strategy matched with the target detection object according to the time sequence attribute.
In the embodiments of the present specification, the time sequence corresponds to a prediction strategy based on a prediction model, and the non-time sequence corresponds to a prediction strategy based on a mean value. And determining a target prediction strategy corresponding to the target detection object based on the judgment of the time sequence attribute of the target detection object.
S305, determining the prediction data of the target detection object at the current moment according to the target prediction strategy and the historical monitoring data.
Specifically, when the target prediction policy is a prediction policy based on a prediction model, the method in fig. 4 may be used to determine the prediction data of the target detection object at the current time according to the target prediction policy and the historical monitoring data, as shown in fig. 4, the method may include:
s401, determining a prediction model.
Wherein the predictive model comprises a machine learning model for unsupervised machine learning determination based on sample historical monitoring data of the target test object.
Machine Learning (ML) is a multi-domain cross subject, and relates to multi-domain subjects such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach to make computers have intelligence, and is applied in various fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formal education learning.
Training data input to a machine learning model in an unsupervised machine learning process are not marked and have no determined result, the class of the training data is unknown, and the machine learning model needs to classify the training data according to the similarity between the data to try to minimize the intra-class difference and maximize the inter-class difference.
In the embodiment of the present specification, since there may be relatively large differences in volatility, such as no large fluctuation (i.e., smooth change) and large fluctuation, for the target detection object with time sequence, in order to further improve accuracy of prediction data, different preset machine learning models may be based on when the prediction model is obtained through training, so as to obtain prediction models corresponding to different volatility characteristics. When the prediction model is determined, the volatility of the target detection object can be further analyzed, and the prediction model is selected according to the volatility analysis result. The preset machine learning model may include, but is not limited to, a neural network model, an exponential smoothing model, and the like.
Taking the data curve corresponding to the game key path as an example, the online people number curve and the matching success rate curve have time sequence, but the online people number curve is smooth and has no large fluctuation, and a neural network model can be used as a preset machine learning model to train to obtain a prediction model; the matching success rate curve is not smooth and is zigzag, and an exponential smoothing model can be used as a preset machine learning model to train to obtain a prediction model.
In practical application, sample historical monitoring data of a target detection object with a target dimension can be acquired from a sample data space which is constructed in advance, and the sample data space can be obtained by preprocessing the monitoring data collected in a designated storage space at each monitoring moment. The sample historical monitoring data may be expressed as
Wherein, XnSample historical monitoring data representing a target test object in a target dimension n,
for the historical monitoring data at the T-th time, the monitoring data may be obtained by monitoring according to a preset time interval, for example, the preset time interval may be 5 minutes, and the time corresponding to the T-th time is 5 × T. Monitoring data using sample history
And performing prediction training on the preset machine learning model as input, adjusting the model parameters of the preset machine learning model in the training process until the output prediction result is matched with the corresponding input, and taking the machine learning model corresponding to the current model parameters as the prediction model.
And S403, predicting the target detection object at the current moment based on the prediction model and the historical monitoring data to obtain the prediction data of the target detection object at the current moment.
Specifically, historical monitoring data of a preset time period before the current time is used as input of the prediction model, so that the prediction data of the target detection object at the current time output by the prediction model can be obtained.
When the target preset strategy is a prediction strategy based on a mean value, the method in fig. 5 may be used to determine the prediction data of the target detection object at the current time according to the target prediction strategy and the historical monitoring data, as shown in fig. 5, the method may include:
s501, determining an average value of the historical monitoring data.
For example, the historical monitoring data is { x }t-5，xt-4，xt-3，xt-2，xt-1The average value of the historical monitoring data is (x)t-5+xt-3+xt-2+xt-1)/5。
And S503, taking the average value of the historical monitoring data as the prediction data of the target detection object at the current moment.
And S105, determining the prediction deviation of the target detection object at the current moment according to the prediction data and the monitoring data.
Specifically, the difference between the prediction data and the monitoring data is calculated to obtain the prediction deviation of the target detection object at the current time, which can be expressed as Δ x ═ xpre-xtrueWhere Δ x denotes the prediction deviation, xpreIndicating the prediction data at the current time, xtrueIndicating the monitored data at the current time.
And S107, carrying out anomaly detection on the monitoring data of the target detection object at the current moment based on a preset anomaly probability function and the prediction deviation.
Specifically, the prediction deviation is brought into a preset abnormal probability function, and a corresponding abnormal probability value is calculated through the preset abnormal probability function; judging whether the abnormal probability value is greater than or equal to a preset threshold value, and determining that the monitoring data of the target detection object at the current moment is abnormal when the judgment result is yes; and otherwise, when the judgment result is that the abnormal probability value is smaller than the preset threshold value, determining that the monitoring data of the target detection object at the current moment is normal. The preset threshold value can be set according to the actual form of the abnormal probability function.
In order to improve the efficiency of abnormal data detection and avoid the dependence of the preset threshold on experience, in a specific embodiment, the preset abnormal probability function is set as follows:
where Δ x denotes a prediction deviation, μ denotes a first parameter, and σ denotes a second parameter. Accordingly, the preset threshold is set to 0.99999, that is, when F (Δ x, μ, σ) is greater than or equal to 0.99999, it can be determined that the detection data of the target detection object at the current time is abnormal, whereas when F (Δ x, μ, σ) is less than 0.99999, it can be determined that the detection data of the target detection object at the current time is normal.
In a specific embodiment, the first parameter μ and the second parameter σ in F (Δ x, μ, σ) may be calculated according to historical data of the target detection object in the target dimension, and specifically, the calculation method of the first parameter μ and the second parameter σ is further associated with the target prediction strategy. When the target prediction strategy is a prediction strategy based on a prediction model, as shown in a flowchart of another abnormal data detection method provided in fig. 6, before determining whether the monitored data of the target detection object at the current time is abnormal based on a preset abnormal probability function and the prediction deviation, the method further includes:
s601, predicting the target detection object at the time point corresponding to each sample historical monitoring data based on the prediction model to obtain the prediction historical data corresponding to each sample historical monitoring data.
Specifically, the sample historical detection data can be expressed as
The prediction history data may be expressed as
Wherein the content of the first and second substances,
is shown in
And the prediction historical data of the corresponding time point is obtained by prediction of the prediction model.
For the online population curve corresponding to the game critical path, fig. 7a is a schematic diagram of a prediction result obtained by predicting the curve at a time point corresponding to each sample historical monitoring data by using a neural network model.
For the matching success rate curve corresponding to the aforementioned game critical path, fig. 7b is a schematic diagram of a prediction result obtained by predicting the curve at a time point corresponding to each sample historical monitoring data by using an exponential smoothing model (holt calculator).
S603, determining the difference value between the prediction historical data and the corresponding sample historical monitoring data to obtain a sample prediction difference set.
In particular, the sample prediction error set can be expressed as
Wherein the content of the first and second substances,
and S605, determining parameter values of parameters in the preset abnormal probability function according to the sample prediction deviation in the sample prediction deviation set.
Specifically, if the historical sample monitoring data is not required to be based on corresponding statistical data, and the determination of the number of online people at each time in the online people number curve is independent of the total number of people at that time, the average value of the sample prediction deviations in the sample prediction deviation set and the standard deviation of the sample prediction deviations in the sample prediction deviation set can be calculated, and the average value of the sample prediction deviations in the sample prediction deviation set and the standard deviation of the sample prediction deviations in the sample prediction deviation set are used as the parameter values of the corresponding parameters in the preset abnormal probability function, and the specific calculation formula is expressed as follows:
specifically, if the historical sample monitoring data needs to be based on corresponding statistical data, for example, if the determination of the matching success rate at each time in the matching success rate curve needs to be based on the total matching times at the corresponding time, the parameter values of the parameters in the preset abnormal probability function may be determined according to the following steps:
calculating the average of the sample prediction biases in the set of sample prediction biases, expressed as
And determining a nonlinear fitting function according to regression processing of the historical monitoring data of the sample. Specifically, since the historical sample monitoring data needs to be determined based on the corresponding statistical data, the historical sample monitoring data is expanded before the regression processing is performed, so that each sample data after the expansion includes the historical sample monitoring data and the corresponding statistical data. Taking the matching success rate curve as an example, each sample data after expansion
Wherein, rateTIs the matching success rate, cnt, at the Tth timeTTo correspond to rateTThe total matching times of the data, the sample data after expansion
For expanded sample data
Performing regression processing, wherein the expanded sample data is firstly processed in the regression processing
Performing a discretization process, e.g. as represented by
Then according to Xn' Each data set in the above is calculated to obtain a corresponding data pair of standard deviation sigma and cnt, i.e.
Wherein, the first and the second end of the pipe are connected with each other,
cntkis composed of
May be a discrete value of
Average value of (a).
And the above
The processing method is similar, and is not described herein again. Then according to
Fitting regression to obtain the functional relationship between σ and cnt, e.g. selecting a quadratic function σ ═ a × cnt2+ b cnt + c, where the parameters a, b and c may be determined when the function is fitted.
And determining a function value of the nonlinear fitting function according to the monitoring data at the current moment, and taking an average value of the sample prediction deviations in the sample prediction deviation set and the function value of the nonlinear fitting function as parameter values of corresponding parameters in the preset abnormal probability function. Specifically, the statistical data corresponding to the monitoring data at the current time may be determined, and then the statistical data is substituted into the regression nonlinear function, so as to obtain a function value of the nonlinear function. For example, the cnt corresponding to the monitoring data at the current time is substituted into the above-mentioned quadratic function σ ═ a × cnt2And + b × cnt + c, the function value σ can be obtained. And taking the calculated mu and sigma as parameter values of corresponding parameters in a preset abnormal probability function.
When the target prediction strategy is a mean-based prediction strategy, as shown in a schematic flow chart of another abnormal data detection method provided in fig. 8, before determining whether the monitored data of the target detection object at the current time is abnormal based on a preset abnormal probability function and the prediction deviation, the method further includes:
s801, obtaining sample historical monitoring data of the target detection object.
The sample historical monitoring data may be expressed as
Wherein XnSample historical monitoring data for a target test object representing a target dimension n,
is the monitored data at the Tth moment of the history.
And S803, determining the average value of the historical monitoring data of the sample to obtain the predicted historical data.
Specifically, an average value of the historical sample monitoring data is calculated and used as the predicted historical data of the time point corresponding to each historical sample monitoring data, that is, the average value is
Representing prediction history data.
And S805, determining a difference value between the prediction historical data and the historical monitoring data of each sample to obtain a sample prediction difference set.
In particular, the set of sample prediction deviations may be expressed as
Wherein, the first and the second end of the pipe are connected with each other,
s807, an average of the sample prediction biases in the sample prediction bias set is determined.
Specifically, the average value of the sample prediction bias in the sample prediction bias set is calculated, i.e., expressed as
And S809, determining a nonlinear fitting function according to the regression processing of the historical sample monitoring data.
In the prediction strategy based on the mean value, the monitoring data of the target detection object needs to be determined based on corresponding statistical data, for example, the determination of the authentication success rate at each moment in the authentication success rates needs to be determined based on the total authentication times at the corresponding moment, so that the historical sample monitoring data is expanded before regression processing is performed, and each sample data after expansion comprises the historical sample monitoring data and the corresponding statistical data. Taking the success rate curve of authentication as an example, each sample data after expansion
Wherein, rateTTo authenticate success rate, cntTTo correspond to rateTThe total authentication times of the network, the expanded sample data
For the sample data after expansion
Performing regression processing, wherein the expanded sample data is firstly processed in the regression processing
Performing a discretization process, e.g. as represented by
Then according to Xn' Each data set in the above is calculated to obtain a corresponding data pair of standard deviation sigma and cnt, i.e.
Wherein the content of the first and second substances,
cntkis composed of
May be a discrete value of
Average value of (a).
And the above
The processing is similar, and is not described herein again. Then according to
Fitting regression to obtain the functional relationship between σ and cnt, e.g. selecting a quadratic function σ ═ a × cnt2+ b cnt + c, where the parameters a, b and c may be determined when the function is fitted.
And S811, determining a function value of the nonlinear fitting function according to the monitoring data at the current moment.
Specifically, the statistical data corresponding to the monitoring data at the current time may be determined, and then the statistical data is substituted into the regression nonlinear function, so as to obtain a function value of the nonlinear function. For example, the cnt corresponding to the monitoring data at the current time is substituted into the above-mentioned quadratic function σ ═ a × cnt2And + b × cnt + c, the function value σ can be obtained.
And S813, taking the average value of the sample prediction deviations in the sample prediction deviation set and the function value of the nonlinear fitting function as the parameter value of the corresponding parameter in the preset abnormal probability function.
Specifically, the calculated μ and σ are used as parameter values of corresponding parameters in a preset abnormal probability function.
In practical application, when it is detected that the monitoring data of the target detection object at the current moment is abnormal, alarm information may be sent according to actual settings, the notification mode of the alarm information may include, but is not limited to, WeChat, short message, email, and enterprise-level communication platform (such as RTX), and the notification object of the alarm information may include, but is not limited to, operation and maintenance, operation planning, and the like.
In a particular embodiment, an alarm configuration interface, such as that shown in FIG. 9, may be provided, where monitored dimensions and detection objects options are provided, allowing a user to set target dimensions and target detection objects as desired. The alarm configuration interface may further provide a monitoring period, that is, a preset time interval option for acquiring monitoring data, for example, a period of 5 minutes, where each period corresponds to a time, and the monitoring data of each preset time interval is stored in a specified storage space, so that historical monitoring data is acquired from the specified storage space during subsequent abnormal data detection. In addition, the alarm configuration interface can also allow the setting of the notification mode and the notification object of the alarm information. After the alarm configuration interface in fig. 9 is set and confirmed, the abnormal data detection method provided by the embodiment of the invention can be used for online monitoring.
According to the technical scheme of the embodiment of the invention, the target detection object is predicted at the current moment through the historical monitoring data of the target detection object with the target dimension to obtain the prediction data, the prediction deviation of the target detection object at the current moment is determined based on the prediction data and the monitoring data, then the abnormality detection is carried out on the monitoring data of the target detection object at the current moment based on the prediction deviation and the preset abnormality probability function, the abnormality degree of the monitoring data is measured by utilizing the difference between the prediction data and the actual monitoring data, and the prediction data fully reflects the change trend of the data, so that the accuracy of the abnormality data detection is greatly improved.
In addition, the parameters in the embodiment of the invention are self-trained and determined based on the historical monitoring data of the target detection object, and the setting of a fixed threshold value is not required to be carried out by depending on rich operation and maintenance experience, so that the data detection efficiency is improved.
Corresponding to the abnormal data detection methods provided in the above embodiments, embodiments of the present invention further provide an abnormal data detection apparatus, and since the abnormal data detection apparatus provided in embodiments of the present invention corresponds to the abnormal data detection methods provided in the above embodiments, the implementation of the abnormal data detection method is also applicable to the abnormal data detection apparatus provided in this embodiment, and will not be described in detail in this embodiment.
Please refer to fig. 10, which is a schematic structural diagram illustrating an abnormal data detecting apparatus according to an embodiment of the present invention, wherein the apparatus has a function of implementing the abnormal data detecting method according to the above embodiment of the method, and the function may be implemented by hardware or by hardware executing corresponding software. As shown in fig. 10, the apparatus may include:
the first obtaining module 1010 is configured to obtain monitoring data of a target detection object at a current time and historical monitoring data of a preset time period before the current time;
a first determining module 1020, configured to determine, according to the historical monitoring data, prediction data of the target detection object at a current time;
a second determining module 1030, configured to determine, according to the prediction data and the monitoring data, a prediction deviation of the target detection object at the current time;
the anomaly detection module 1040 is configured to perform anomaly detection on the monitoring data of the target detection object at the current time based on a preset anomaly probability function and the prediction deviation.
In a particular embodiment, the first determining module 1020 may include:
the third determining module is used for determining the time sequence attribute of the target detection object;
the fourth determining module is used for determining a target prediction strategy matched with the target detection object according to the time sequence attribute;
and the fifth determining module is used for determining the prediction data of the target detection object at the current moment according to the target prediction strategy and the historical monitoring data.
Optionally, when the target prediction strategy is a prediction strategy based on a prediction model, the fifth determining module may include:
a model determination module for determining a prediction model; the predictive model comprises a machine learning model for unsupervised machine learning determination based on sample historical monitoring data of the target detection object;
and the first prediction module is used for predicting the target detection object at the current moment based on the prediction model and the historical monitoring data to obtain the prediction data of the target detection object at the current moment.
In a specific embodiment, as shown in the schematic structural diagram of another abnormal data detection apparatus provided in fig. 11, the apparatus may further include:
the second prediction module 1110 is configured to predict, based on the prediction model, a time point corresponding to each sample historical monitoring data of the target detection object, so as to obtain predicted historical data corresponding to each sample historical monitoring data;
a sixth determining module 1120, configured to determine a difference between the prediction historical data and corresponding sample historical monitoring data, to obtain a sample prediction difference set;
a seventh determining module 1130, configured to determine parameter values of parameters in the preset abnormal probability function according to the sample prediction deviations in the sample prediction deviation set.
In a particular embodiment, the seventh determining module 1130 may include:
an eighth determining module, configured to determine an average of the sample prediction deviations in the set of sample prediction deviations and a standard deviation of the sample prediction deviations in the set of sample prediction deviations;
and the ninth determining module is used for taking the average value and the standard deviation of the sample prediction deviation in the sample prediction deviation set as the parameter values of the corresponding parameters in the preset abnormal probability function.
In another specific embodiment, the seventh determining module 1130 may include:
a first mean determination module that determines a mean of the sample prediction deviations in the set of sample prediction deviations;
the first function determining module is used for determining a nonlinear fitting function according to regression processing of the historical monitoring data of the sample;
a first function value determining module, configured to determine a function value of the nonlinear fitting function according to the monitoring data at the current time;
and the tenth determining module is used for taking the average value of the sample prediction deviations in the sample prediction deviation set and the function value of the nonlinear fitting function as the parameter values of the corresponding parameters in the preset abnormal probability function.
Optionally, when the target preset strategy is a prediction strategy based on a mean value, the fifth determining module may include:
the second mean value determining module is used for determining the mean value of the historical monitoring data;
and the third prediction module is used for taking the average value of the historical monitoring data as the prediction data of the target detection object at the current moment.
In another specific embodiment, as shown in a schematic structural diagram of another abnormal data detection apparatus provided in fig. 12, the apparatus may further include:
a second obtaining module 1210, configured to obtain sample historical monitoring data of the target detection object;
a fourth prediction module 1220, configured to determine an average value of the historical sample monitoring data, so as to obtain predicted historical data;
an eleventh determining module 1230, configured to determine a difference between the predicted historical data and each sample historical monitoring data, so as to obtain a sample predicted difference set;
a third average determining module 1240 for determining an average of the sample prediction deviations in the sample prediction deviation set;
a second function determining module 1250, configured to determine a non-linear fitting function according to the regression processing of the sample historical monitoring data;
a second function value determining module 1260, configured to determine a function value of the nonlinear fitting function according to the monitoring data at the current time;
a twelfth determining module 1270, configured to use the average value of the sample prediction deviations in the sample prediction deviation set and the function value of the nonlinear fitting function as the parameter value of the corresponding parameter in the preset abnormal probability function.
Optionally, the anomaly detection module 1040 may include:
a thirteenth determining module, configured to determine an abnormal probability value based on a preset abnormal probability function and the prediction deviation;
the judging module is used for judging whether the abnormal probability value is greater than or equal to a preset threshold value or not;
and a fourteenth determining module, configured to determine that the monitoring data of the target detection object at the current moment is abnormal when a determination result of the determining module is yes.
It should be noted that, when the apparatus provided in the foregoing embodiment implements the functions thereof, only the division of the functional modules is illustrated, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the internal structure of the apparatus may be divided into different functional modules to implement all or part of the functions described above. In addition, the apparatus and method embodiments provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.
The abnormal data detection device of the embodiment of the invention predicts the target detection object at the current moment through the historical monitoring data of the target detection object with the target dimension to obtain the prediction data, determines the prediction deviation of the target detection object at the current moment based on the prediction data and the monitoring data, then performs abnormal detection on the monitoring data of the target detection object at the current moment based on the prediction deviation and the preset abnormal probability function, measures the abnormal degree of the monitoring data by utilizing the difference between the prediction data and the actual monitoring data, and fully reflects the change trend of the data per se through the prediction data, thereby greatly improving the accuracy of abnormal data detection, setting a fixed threshold value without depending on rich operation and maintenance experience, and improving the data detection efficiency.
An embodiment of the present invention provides a server, where the server includes a processor and a memory, where the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the abnormal data detection method provided in the foregoing method embodiment.
The memory may be used to store software programs and modules, and the processor may execute various functional applications and abnormal data detection by operating the software programs and modules stored in the memory. The memory can mainly comprise a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs needed by functions and the like; the storage data area may store data created according to use of the apparatus, and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory may also include a memory controller to provide the processor access to the memory.
The method embodiments provided by the embodiments of the present invention may be executed in a computer terminal, a server, or a similar computing device. Taking an example of the server running on the server, fig. 13 is a block diagram of a hardware structure of the server running an abnormal data detection method according to the embodiment of the present invention, as shown in fig. 13, the server 1300 may have a relatively large difference due to different configurations or performances, and may include one or more Central Processing Units (CPUs) 1310 (the processors 1310 may include but are not limited to Processing devices such as a microprocessor MCU or a programmable logic device FPGA), a memory 1330 for storing data, and one or more storage media 1320 (e.g., one or more mass storage devices) for storing an application program 1323 or data 1322. Memory 1330 and storage medium 1320 may be, among other things, transient storage or persistent storage. The program stored in the storage medium 1320 may include one or more modules, each of which may include a series of instruction operations for the server. Still further, the central processor 1310 may be configured to communicate with the storage medium 1320, to execute a series of instruction operations in the storage medium 1320 on the server 1300. The server 1300 may also include one or more power supplies 1360, one or more wired or wireless network interfaces 1350, one or more input-output interfaces 1340, and/or one or more operating systems 1321 such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
Input/output interface 1340 may be used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the server 1300. In one example, i/o Interface 1340 includes a Network adapter (NIC) that may be coupled to other Network devices via a base station to communicate with the internet. In one example, the input/output interface 1340 can be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
It will be understood by those skilled in the art that the structure shown in fig. 13 is only an illustration and is not intended to limit the structure of the electronic device. For example, server 1300 may also include more or fewer components than shown in FIG. 13, or have a different configuration than shown in FIG. 13.
An embodiment of the present invention further provides a computer-readable storage medium, where the storage medium may be configured in a terminal to store at least one instruction, at least one program, a code set, or a set of instructions related to implementing a video generation method, where the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by the processor to implement the abnormal data detection method provided by the foregoing method embodiment.
Optionally, in this embodiment, the storage medium may include but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, and various media capable of storing program codes.
It should be noted that: the sequence of the above embodiments of the present invention is only for description, and does not represent the advantages or disadvantages of the embodiments. And specific embodiments thereof have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
All the embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and reference may be made to the partial description of the method embodiment for relevant points.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (9)
1. An abnormal data detection method, characterized in that the method comprises:
acquiring monitoring data of a target detection object at the current moment and historical monitoring data of a preset time period before the current moment;
determining the time sequence attribute of the target detection object, and carrying out mobility analysis on the target detection object; determining a target prediction strategy matched with the target detection object according to the time sequence attribute and the fluctuation analysis result; the target prediction strategy comprises a prediction strategy based on a prediction model, wherein the prediction model comprises a neural network model and an exponential smoothing model;
determining the prediction data of the target detection object at the current moment according to the target prediction strategy and the historical monitoring data; determining the prediction deviation of the target detection object at the current moment according to the prediction data and the monitoring data;
based on a preset abnormal probability function and the prediction deviation, carrying out abnormal detection on the monitoring data of the target detection object at the current moment;
the performing anomaly detection on the monitoring data of the target detection object at the current moment based on a preset anomaly probability function and the prediction deviation comprises:
determining an abnormal probability value based on a preset abnormal probability function and the prediction deviation; and judging whether the abnormal probability value is greater than or equal to a preset threshold value.
2. The abnormal data detection method according to claim 1, wherein when the target prediction strategy is a prediction strategy based on a prediction model, the determining the prediction data of the target detection object at the current time according to the target prediction strategy and the historical monitoring data comprises:
determining a prediction model; the predictive model comprises a machine learning model for unsupervised machine learning determination based on sample historical monitoring data of the target detection object;
and predicting the target detection object at the current moment based on the prediction model and the historical monitoring data to obtain the prediction data of the target detection object at the current moment.
3. The abnormal data detecting method according to claim 2, wherein before determining whether the monitored data of the target detection object at the current time is abnormal based on a preset abnormal probability function and the predicted deviation, the method further comprises:
predicting the target detection object at the time point corresponding to each sample historical monitoring data based on the prediction model to obtain the prediction historical data corresponding to each sample historical monitoring data;
determining a difference value between the prediction historical data and corresponding sample historical monitoring data to obtain a sample prediction difference set;
and determining parameter values of parameters in the preset abnormal probability function according to the sample prediction deviation in the sample prediction deviation set.
4. The abnormal data detecting method of claim 3, wherein the determining parameter values of parameters in the preset abnormal probability function according to the sample prediction deviations in the sample prediction deviation set comprises:
determining an average of the sample prediction deviations in the set of sample prediction deviations and a standard deviation of the sample prediction deviations in the set of sample prediction deviations;
and taking the average value and the standard deviation of the sample prediction deviations in the sample prediction deviation set as parameter values of corresponding parameters in the preset abnormal probability function.
5. The abnormal data detecting method of claim 3, wherein the determining parameter values of parameters in the preset abnormal probability function according to the sample prediction deviations in the sample prediction deviation set comprises:
determining an average of the sample prediction biases in the set of sample prediction biases;
determining a nonlinear fitting function according to regression processing of the historical monitoring data of the sample;
determining a function value of the nonlinear fitting function according to the monitoring data at the current moment; and taking the average value of the sample prediction deviations in the sample prediction deviation set and the function value of the nonlinear fitting function as the parameter values of the corresponding parameters in the preset abnormal probability function.
6. The abnormal data detection method of claim 1, wherein when the target prediction strategy is a mean-based prediction strategy, the determining the prediction data of the target detection object at the current moment according to the target prediction strategy and historical monitoring data comprises:
determining an average of the historical monitoring data;
and taking the average value of the historical monitoring data as the prediction data of the target detection object at the current moment.
7. The abnormal data detecting method according to claim 6, wherein before determining whether the monitored data of the target detection object at the current time is abnormal based on a preset abnormal probability function and the predicted deviation, the method further comprises:
obtaining sample historical monitoring data of the target detection object;
determining an average value of the historical monitoring data of the sample to obtain predicted historical data;
determining a difference value between the prediction historical data and the historical monitoring data of each sample to obtain a sample prediction difference set;
determining an average of the sample prediction biases in the set of sample prediction biases;
determining a nonlinear fitting function according to regression processing of the historical monitoring data of the sample;
determining a function value of the nonlinear fitting function according to the monitoring data at the current moment;
and taking the average value of the sample prediction deviations in the sample prediction deviation set and the function value of the nonlinear fitting function as the parameter values of the corresponding parameters in the preset abnormal probability function.
8. An abnormal data detecting apparatus, characterized in that the apparatus comprises:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring monitoring data of a target detection object at the current moment and historical monitoring data of a preset time period before the current moment;
the first determining module is used for determining the time sequence attribute of the target detection object and carrying out mobility analysis on the target detection object; determining a target prediction strategy matched with the target detection object according to the time sequence attribute and the fluctuation analysis result; determining the prediction data of the target detection object at the current moment according to the target prediction strategy and the historical monitoring data; the target prediction strategy comprises a prediction strategy based on a prediction model, wherein the prediction model comprises a neural network model and an exponential smoothing model;
the second determination module is used for determining the prediction deviation of the target detection object at the current moment according to the prediction data and the monitoring data;
the anomaly detection module is used for carrying out anomaly detection on the monitoring data of the target detection object at the current moment based on a preset anomaly probability function and the prediction deviation;
the performing, based on a preset abnormal probability function and the prediction deviation, abnormal detection on the monitoring data of the target detection object at the current moment includes:
determining an abnormal probability value based on a preset abnormal probability function and the prediction deviation;
and judging whether the abnormal probability value is greater than or equal to a preset threshold value.
9. A server comprising a processor and a memory, said memory having stored therein at least one instruction, at least one program, set of codes, or set of instructions, said at least one instruction, said at least one program, set of codes, or set of instructions being loaded and executed by said processor to implement the method of anomaly data detection according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910646013.9A CN110377447B (en) | 2019-07-17 | 2019-07-17 | Abnormal data detection method and device and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910646013.9A CN110377447B (en) | 2019-07-17 | 2019-07-17 | Abnormal data detection method and device and server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110377447A CN110377447A (en) | 2019-10-25 |
| CN110377447B true CN110377447B (en) | 2022-07-22 |
Family
ID=68253680
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910646013.9A Active CN110377447B (en) | 2019-07-17 | 2019-07-17 | Abnormal data detection method and device and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110377447B (en) |
Families Citing this family (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110941648A (en) * | 2019-11-19 | 2020-03-31 | 广州市自来水有限公司 | Abnormal data identification method, system and storage medium based on cluster analysis |
| CN111176931A (en) * | 2019-12-11 | 2020-05-19 | 深圳智链物联科技有限公司 | Operation monitoring method, operation monitoring device, server and storage medium |
| CN111045894B (en) * | 2019-12-13 | 2024-02-13 | 贵州广思信息网络有限公司广州分公司 | Database abnormality detection method, database abnormality detection device, computer device and storage medium |
| CN111124918B (en) * | 2019-12-24 | 2023-09-05 | 中国银行股份有限公司 | Test data prediction method and device and processing equipment |
| CN111125195B (en) * | 2019-12-25 | 2023-09-08 | 亚信科技（中国）有限公司 | Data anomaly detection method and device |
| CN111130940A (en) * | 2019-12-26 | 2020-05-08 | 众安信息技术服务有限公司 | Abnormal data detection method and device and server |
| CN111143169B (en) * | 2019-12-30 | 2024-02-27 | 杭州迪普科技股份有限公司 | Abnormal parameter detection method and device, electronic equipment and storage medium |
| CN111212472B (en) * | 2020-01-02 | 2023-09-05 | 腾讯科技（深圳）有限公司 | Error control method and device and computer readable storage medium |
| CN111173565B (en) * | 2020-01-07 | 2021-02-09 | 北京龙软科技股份有限公司 | Mine monitoring data abnormal fluctuation early warning method and device |
| CN111447193B (en) * | 2020-03-23 | 2022-11-04 | 网宿科技股份有限公司 | Method and device for anomaly detection of real-time data stream |
| CN111414586A (en) * | 2020-03-31 | 2020-07-14 | 北京派得伟业科技发展有限公司 | Environment monitoring method and system |
| CN111614634B (en) * | 2020-04-30 | 2024-01-23 | 腾讯科技（深圳）有限公司 | Flow detection method, device, equipment and storage medium |
| CN111726341B (en) * | 2020-06-02 | 2022-10-14 | 五八有限公司 | Data detection method and device, electronic equipment and storage medium |
| CN111913849B (en) * | 2020-07-29 | 2022-02-11 | 厦门大学 | Unsupervised anomaly detection and robust trend prediction method for operation and maintenance data |
| CN111882338B (en) * | 2020-08-11 | 2023-06-30 | 网易（杭州）网络有限公司 | Abnormality detection method and device for online number of people and electronic equipment |
| CN111984705B (en) * | 2020-08-20 | 2022-05-17 | 广东电网有限责任公司电力科学研究院 | Precision evaluation management method and device for power quality monitoring device |
| CN112444748A (en) * | 2020-10-12 | 2021-03-05 | 武汉蔚来能源有限公司 | Battery abnormality detection method, battery abnormality detection device, electronic apparatus, and storage medium |
| CN112162911B (en) * | 2020-10-14 | 2024-03-29 | 中国民航信息网络股份有限公司 | Abnormal business operation behavior detection method, device, equipment and readable storage medium |
| CN112396244A (en) * | 2020-11-30 | 2021-02-23 | 中国民航信息网络股份有限公司 | Flight booking value processing method and system based on zero booking model |
| CN112308346A (en) * | 2020-11-30 | 2021-02-02 | 中国民航信息网络股份有限公司 | Flight booking value processing method and system based on exponential smoothing model |
| CN112527608A (en) * | 2020-12-10 | 2021-03-19 | 北京自如信息科技有限公司 | Alarm method and device and computer equipment |
| CN112732983B (en) * | 2020-12-31 | 2023-09-12 | 平安科技（深圳）有限公司 | Data detection method and device based on artificial intelligence, server and storage medium |
| CN112995195B (en) * | 2021-03-17 | 2023-01-31 | 北京安天网络安全技术有限公司 | Abnormal behavior prediction method and device |
| CN113126512B (en) * | 2021-04-01 | 2023-07-18 | 青岛海尔空调器有限总公司 | Method and device for reporting data and intelligent household appliance |
| CN113139817A (en) * | 2021-04-28 | 2021-07-20 | 北京沃东天骏信息技术有限公司 | Data classification method, data classification device, medium, and electronic apparatus |
| CN113590392B (en) * | 2021-06-30 | 2024-04-02 | 中国南方电网有限责任公司超高压输电公司昆明局 | Converter station equipment abnormality detection method, device, computer equipment and storage medium |
| CN113419258B (en) * | 2021-07-07 | 2024-03-01 | 东软集团股份有限公司 | Positioning abnormality detection method and related equipment thereof |
| CN113626502A (en) * | 2021-08-13 | 2021-11-09 | 南方电网深圳数字电网研究院有限公司 | Power grid data anomaly detection method and device based on ensemble learning |
| CN114978749B (en) * | 2022-06-14 | 2023-10-10 | 中国电信股份有限公司 | Login authentication method and system, storage medium and electronic equipment |
| CN115865992B (en) * | 2023-03-02 | 2023-08-04 | 中国建材检验认证集团湖南有限公司 | Intelligent water conservancy online monitoring system |
| CN116400126B (en) * | 2023-06-08 | 2023-08-01 | 广东佰林电气设备厂有限公司 | Low-voltage power box with data processing system |
| CN117370917B (en) * | 2023-12-07 | 2024-02-23 | 城光（湖南）节能环保服务股份有限公司 | Urban intelligent street lamp service life prediction method and system |
| CN117474427B (en) * | 2023-12-27 | 2024-03-26 | 大连金马衡器有限公司 | Intelligent pallet cold chain tracing method based on Internet of things technology |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20160047065A (en) * | 2014-10-21 | 2016-05-02 | 삼성에스디에스 주식회사 | System and method for detecting and predicting anomalies based on analysis of text data |
| CN108089962A (en) * | 2017-11-13 | 2018-05-29 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality, device and electronic equipment |
| CN109032829A (en) * | 2018-07-23 | 2018-12-18 | 腾讯科技（深圳）有限公司 | Data exception detection method, device, computer equipment and storage medium |
| CN109615226A (en) * | 2018-12-12 | 2019-04-12 | 焦点科技股份有限公司 | A kind of operation indicator method for monitoring abnormality |
| CN110008079A (en) * | 2018-12-25 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Monitor control index method for detecting abnormality, model training method, device and equipment |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8359209B2 (en) * | 2006-12-19 | 2013-01-22 | Hartford Fire Insurance Company | System and method for predicting and responding to likelihood of volatility |
| CN102867132B (en) * | 2012-10-16 | 2015-10-21 | 南京航空航天大学 | The online fault combination forecasting method of aviation DC converter based on fractional wavelet transform |
| CN104899658A (en) * | 2015-06-12 | 2015-09-09 | 哈尔滨工业大学 | Prediction model selection method based on applicability quantification of time series prediction model |
| US11120127B2 (en) * | 2017-12-27 | 2021-09-14 | Nec Corporation | Reconstruction-based anomaly detection |
| CN108495341B (en) * | 2018-02-08 | 2021-10-22 | 西安电子科技大学 | Resource reservation method and system based on flow direction prediction and mobile communication system |
| CN109714395B (en) * | 2018-12-10 | 2021-10-26 | 平安科技（深圳）有限公司 | Cloud platform resource use prediction method and terminal equipment |
| CN109902874A (en) * | 2019-02-28 | 2019-06-18 | 武汉大学 | A kind of micro-capacitance sensor photovoltaic power generation short term prediction method based on deep learning |
-
2019
- 2019-07-17 CN CN201910646013.9A patent/CN110377447B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20160047065A (en) * | 2014-10-21 | 2016-05-02 | 삼성에스디에스 주식회사 | System and method for detecting and predicting anomalies based on analysis of text data |
| CN108089962A (en) * | 2017-11-13 | 2018-05-29 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality, device and electronic equipment |
| CN109032829A (en) * | 2018-07-23 | 2018-12-18 | 腾讯科技（深圳）有限公司 | Data exception detection method, device, computer equipment and storage medium |
| CN109615226A (en) * | 2018-12-12 | 2019-04-12 | 焦点科技股份有限公司 | A kind of operation indicator method for monitoring abnormality |
| CN110008079A (en) * | 2018-12-25 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Monitor control index method for detecting abnormality, model training method, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110377447A (en) | 2019-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110377447B (en) | Abnormal data detection method and device and server | |
| US11651259B2 (en) | Neural architecture search for convolutional neural networks | |
| EP3620983B1 (en) | Computer-implemented method, computer program product and system for data analysis | |
| Le Son et al. | Remaining useful life estimation based on stochastic deterioration models: A comparative study | |
| US20200371491A1 (en) | Determining Operating State from Complex Sensor Data | |
| Schmidt et al. | Coevolution of fitness predictors | |
| Azzouz et al. | Steady state IBEA assisted by MLP neural networks for expensive multi-objective optimization problems | |
| US20210350234A1 (en) | Techniques to detect fusible operators with machine learning | |
| Rothfuss et al. | Meta-learning priors for safe bayesian optimization | |
| Barcelos et al. | Disco: Double likelihood-free inference stochastic control | |
| JPWO2016084326A1 (en) | Information processing system, information processing method, and program | |
| CN113330462A (en) | Neural network training using soft nearest neighbor loss | |
| CN113835973B (en) | Model training method and related device | |
| Paassen et al. | Gaussian process prediction for time series of structured data. | |
| KR102289396B1 (en) | Application of reinforcement learning for the advancement of forecasting item demand of repair parts of military equipment | |
| Graves et al. | Proximity fuzzy clustering and its application to time series clustering and prediction | |
| Castillo et al. | Determining the significance and relative importance of parameters of a simulated quenching algorithm using statistical tools | |
| Dai et al. | Maximum expected hitting cost of a markov decision process and informativeness of rewards | |
| Li et al. | Continuous probabilistic model building genetic network programming using reinforcement learning | |
| Dimara et al. | Benchmarking of Regression Algorithms for Simulating the Building's Energy | |
| Niknafs et al. | Two-phase interarrival time prediction for runtime resource management | |
| CN110766338A (en) | DPOS (distributed data processing) bifurcation prediction model method based on artificial intelligence and EOS (Ethernet over Ethernet) and IO (input/output) of block chain technology | |
| Corbett et al. | Model predictive quality control of Polymethyl methacrylate | |
| Mebratu et al. | Online Server Fan failure prediction by Vibration Analysis augmented by Hardware and Software Telemetry | |
| Eamrurksiri | Applying Machine Learning to LTE/5G Performance Trend Analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |