CN106815639A - The abnormal point detecting method and device of flow data - Google Patents
The abnormal point detecting method and device of flow data Download PDFInfo
- Publication number
- CN106815639A CN106815639A CN201611229506.5A CN201611229506A CN106815639A CN 106815639 A CN106815639 A CN 106815639A CN 201611229506 A CN201611229506 A CN 201611229506A CN 106815639 A CN106815639 A CN 106815639A
- Authority
- CN
- China
- Prior art keywords
- data
- lstm models
- class
- detected
- historical juncture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Testing Or Calibration Of Command Recording Devices (AREA)
Abstract
The present invention relates to the abnormal point detecting method and device of a kind of flow data, it is used to solve the relatively low technical problem of the degree of accuracy of prior art stream data outlier detection.The method includes：According to the data that every class sensor is detected in the historical juncture, and the actual measurement abnormal point numerical of each historical juncture trains short-term memory recurrent neural network LSTM models long, LSTM models have been trained in formation, wherein, the data for being detected in the historical juncture per sensor described in class are used as an input vector for training the LSTM models；The data that sensor described in every class was detected at current time obtain the abnormal point numerical testing result for having trained LSTM models to export as the input for having trained LSTM models.
Description
Technical field
The present invention relates to data processing field, in particular it relates to the abnormal point detecting method and device of a kind of flow data.
Background technology
Flow data is one group of order, a large amount of, data sequence that rapidly, continuously reaches, and generally, data flow can be regarded
It is a dynamic data set increased without limitation with time duration.For every field, for example, Internet of Things, aviation boat
My god, the meteorological field such as observing and controlling and financial service, the sensor that flow data can be understood as various multiclass detects at each moment
The data for arriving.
Stream data carries out outlier detection, is conducive to predicting the following failure that may occur, and diagnosed.At present,
Stream data carries out the method that the research of abnormality detection is based primarily upon statistics and machine learning.
Wherein, the distribution situation that Statistics-Based Method passes through analysis statisticaling data, finds and does not meet the data distribution
Abnormity point.This method is it should be understood that data distribution situation, while the abnormity point for detecting may be examined by different distributed models
Measure, that is, the mechanism for detecting these abnormity points may not be unique.Method based on machine learning mainly regards outlier detection as
Cluster or classification problem, solve the abnormity point ambiguity problem produced based on statistical method.But prior art is generally using shallow
Layer machine learning model is carried out, and flow data only is directly used in into model learning, does not consider the inner link between data characteristics, abnormal
The degree of accuracy of point detection is relatively low.
The content of the invention
The main object of the present invention is to provide the abnormal point detecting method and device of a kind of flow data, is used to solve existing skill
The relatively low technical problem of the degree of accuracy of art stream data outlier detection.
To achieve these goals, first aspect present invention provides a kind of abnormal point detecting method of flow data, the stream
Data include the data detected in the historical juncture per class sensor, and in the actual measurement abnormal point numerical of each historical juncture
According to methods described includes：
The data that the sensor according to every class is detected in the historical juncture, and the actual measurement abnormal point numerical instruction
LSTM models have been trained in white silk short-term memory recurrent neural network LSTM models long, formation, wherein, sensor is described described in per class
The data that historical juncture detects are used as an input vector for training the LSTM models；
The data that sensor described in every class was detected at current time are obtained as the input for having trained LSTM models
To the abnormal point numerical testing result for having trained LSTM models to export.
Alternatively, the data that basis sensor described in per class is detected in the historical juncture, and the actual measurement
Abnormal point numerical trains short-term memory recurrent neural network LSTM models long, and LSTM models have been trained in formation, including：
Learn often sensor described in class using the flow data sliding window of the LSTM models to be detected in the historical juncture
The data for arriving, obtain the destination probability value P (y=1 | h) of normal point, wherein, the value of normal point label y is 1；
Label and the destination probability value P (y=1 | h) according to normal point, calculate and obtain cross entropy, wherein, the friendship
Pitch loss function of the entropy as model, the similitude for weighing probable value and label；
The parameter of the LSTM models is adjusted according to the loss function.
Alternatively, the flow data sliding window study using LSTM models sensor described in per class is gone through described
The data that the history moment detects, obtain the destination probability value P (y=1 | h) of normal point, including：
Learn each class sensor respectively using the flow data sliding window of the LSTM models to be examined in the historical juncture
The feature of the data for measuring, obtains hiding vector representation；
The logistic regression layer treatment that the hiding vector representation is passed through into the LSTM models, obtains the destination probability value
P (y=1 | h).
Alternatively, the flow data sliding window study using LSTM models sensor described in per class is gone through described
The data that the history moment detects, obtain the destination probability value P (y=1 | h) of normal point, including：
Learn each class sensor respectively using the flow data sliding window of the LSTM models to be examined in the historical juncture
The feature of the data for measuring, obtains first and hides vector representation；
Logistic regression layer treatment of the vector representation by the LSTM models is hidden by described first, described first is obtained general
Rate value P (y=1 | h1)；
Detected in the historical juncture using the flow data sliding window study all the sensors of the LSTM models
The feature of data, obtains second and hides vector representation；
Logistic regression layer treatment of the vector representation by the LSTM models is hidden by described second, the second probable value is obtained
P (y=1 | h2)；
According to the first probable value P (y=1 | h1) and the second probable value P (y=1 | h2) calculated by equation below
Obtain the destination probability value P (y=1 | h)：
P (y=1 | h)=sigmoid (α P (y=1 | h1)+β P (y=1 | h2))；
Wherein, α is more than 0, and alpha+beta=1 more than 0, β, and sigmoid is activation primitive.
Alternatively, the label and the destination probability value P (y=1 | h) according to normal point, calculates and obtains cross entropy,
Including：
The label of normal point and the cross entropy L (θ) of the destination probability value P (y=1 | h) are calculated by equation below：
L (θ)=- (ylogP (y=1 | h)+(1-y) log (1-P (y=1 | h))).
Alternatively, the parameter that the LSTM models are adjusted according to the loss function, including：
Model parameter is adjusted by stochastic gradient descent algorithm according to the loss function L (θ).
Second aspect present invention provides a kind of detection means, including：
Training module, for the data detected in the historical juncture according to every class sensor, and during each history
The actual measurement abnormal point numerical at quarter trains short-term memory recurrent neural network LSTM models long, and LSTM models have been trained in formation, wherein,
The data detected in the historical juncture per sensor described in class are used as an input vector for training the LSTM models；
Detection module, for the data that sensor described in every class is detected at current time to have been trained into LSTM as described
The input of model, obtains the abnormal point numerical testing result for having trained LSTM models to export.
Alternatively, the training module includes：
Data characteristics study module, for being sensed described in the every class of flow data sliding window study using the LSTM models
The data that device is detected in the historical juncture, obtain the destination probability value P (y=1 | h) of normal point, wherein, normal point label y
Value be 1；
Computing module, for the label according to normal point and the destination probability value P (y=1 | h), calculating is intersected
Entropy, wherein, the cross entropy as model loss function, the similitude for weighing probable value and label；
Parameter adjustment module, the parameter for adjusting the LSTM models according to the loss function.
Alternatively, the data characteristics study module specifically for：
Learn each class sensor respectively using the flow data sliding window of the LSTM models to be examined in the historical juncture
The feature of the data for measuring, obtains hiding vector representation；
The logistic regression layer treatment that the hiding vector representation is passed through into the LSTM models, obtains the destination probability value
P (y=1 | h).
Alternatively, the data characteristics study module specifically for：
Learn each class sensor respectively using the flow data sliding window of the LSTM models to be examined in the historical juncture
The feature of the data for measuring, obtains first and hides vector representation；
Logistic regression layer treatment of the vector representation by the LSTM models is hidden by described first, described first is obtained general
Rate value P (y=1 | h1)；
Detected in the historical juncture using the flow data sliding window study all the sensors of the LSTM models
The feature of data, obtains second and hides vector representation；
Logistic regression layer treatment of the vector representation by the LSTM models is hidden by described second, the second probable value is obtained
P (y=1 | h2)；
According to the first probable value P (y=1 | h1) and the second probable value P (y=1 | h2) calculated by equation below
Obtain the destination probability value P (y=1 | h)：
P (y=1 | h)=sigmoid (α P (y=1 | h1)+β P (y=1 | h2))；
Wherein, α is more than 0, and alpha+beta=1 more than 0, β, and sigmoid is activation primitive.
Alternatively, the computing module is used for：
The label of normal point and the cross entropy L (θ) of the destination probability value P (y=1 | h) are calculated by equation below：
L (θ)=- (ylogP (y=1 | h)+(1-y) log (1-P (y=1 | h))).
Alternatively, the parameter adjustment module specifically for：
Model parameter is adjusted by stochastic gradient descent algorithm according to the loss function L (θ).
By above-mentioned technical proposal, the data that every class sensor was detected in the historical juncture are used as training LSTM models
Input vector, based on LSTM models characteristic in itself, its relation between considering different input vectors in training, so as to
When the data detected using various kinds of sensors train LSTM models, it is ensured that LSTM models can learn various kinds of sensors high it
Between contact.Inner link between not considering data characteristics when the abnormity point of flow data is detected compared to existing technology, the present invention
Abnormity point can be more accurately detected using the LSTM models trained.
Other features and advantages of the present invention will be described in detail in subsequent specific embodiment part.
Brief description of the drawings
Accompanying drawing is, for providing a further understanding of the present invention, and to constitute the part of specification, with following tool
Body implementation method is used to explain the present invention together, but is not construed as limiting the invention.In the accompanying drawings：
Fig. 1 is the structural representation of the computing unit of LSTM models；
Fig. 2 is a kind of schematic flow sheet of the abnormal point detecting method of flow data provided in an embodiment of the present invention；
Fig. 3 is the schematic diagram that a kind of flow data provided in an embodiment of the present invention slides study window learning data feature；
Fig. 4 is a kind of schematic diagram of the training process of LSTM models provided in an embodiment of the present invention；
Fig. 5 is the schematic diagram of the training process of another LSTM models provided in an embodiment of the present invention；
Fig. 6 A are a kind of structural representations of detection means provided in an embodiment of the present invention；
Fig. 6 B are the structural representations of another detection means provided in an embodiment of the present invention.
Specific embodiment
Specific embodiment of the invention is described in detail below in conjunction with accompanying drawing.It should be appreciated that this place is retouched
The specific embodiment stated is merely to illustrate and explain the present invention, and is not intended to limit the invention.
In order that those skilled in the art are easier to understand technical scheme provided in an embodiment of the present invention, first below to relating to
And to correlation technique simply introduced.
LSTM (Long Short-term Memory Recurrent Neural Network, recurrent neural net in short-term long
Network) model is a kind of time recurrent neural network.Forget a mechanism f by introducing, solve traditional recurrent neural network
(Recurrent Neural Network, RNN) produce propagation information lose and gradient diffusing phenomenon, more suitable for treatment and
Critical event very long is spaced and postponed in predicted time sequence.
As shown in figure 1, computing unit in LSTM models is as shown in figure 1, as seen from the figure, the calculating of each computing unit
Journey is shown below：
it=σ (W(i)xt+U(i)ht-1+b(i))
ft=σ (W(f)xt+U(f)ht-1+b(f))
ct=ft⊙ct-1+it⊙tanh(W(c)xt+U(c)ht-1+b(c))
ot=σ (W(o)xt+U(o)ht-1+b(o))
ht=ot⊙tanh(ct)
Usually, the abnormity point that flow data is produced is most relevant with its historical data.By analysis of history data, contribute to
Judgement is predicted the abnormal conditions of data.Above formula illustrates any sort sensor and hides vector representation (h in tt) calculating
Method.First, xtIt is the vector representation of such sensor of t, its component is that each sensor is adopted in t such sensor
The numerical value of collection, ht-1It is the hiding vector representation at t-1 moment, W(i), U(i)And b(i)Respectively calculate input gate vector representation
(it) parameter.Based on sigmoid activation primitives, acquisition can be calculated only comprising 0 and 1 vector representation it.Similarly, W(f), U(f)And b(f)Respectively calculate and forget a vector representation (ft) parameter.Then, based on itAnd ftWeighted sum, it is automatic to obtain stream
Data are in t-1 moment important information, while filtering out useless information, calculate the memory elements vector representation for obtaining t
(ct).Equally, based on sigmoid activation primitives, out gate vector o of the flow data in t can be obtainedt.Finally, o is calculatedt
With memory elements ctBetween dot product, obtain flow data t hiding vector representation ht, for calculate the t+1 moment hide to
Amount is represented.
Input vector xtBy after LSTM, calculating and obtaining corresponding hiding vector representation ht, going through before containing t
History flow data change information.Based on htIt is trained or predicts, compared to uses x merelytFor, when being more conducive to capture each
Sensor information is carved, the performance of outlier detection model is improved.
Based on LSTM models, the embodiment of the present invention provides a kind of abnormal point detecting method of flow data, wherein, the fluxion
According to the data detected in the historical juncture including every class sensor, and in the actual measurement abnormal point numerical of each historical juncture
According to.As shown in Fig. 2 the method includes：
The data that S201, the sensor according to per class are detected in the historical juncture, and the actual measurement abnormity point
Data train short-term memory recurrent neural network LSTM models long, and LSTM models have been trained in formation, wherein, sensor described in per class
The data detected in the historical juncture are used as an input vector for training the LSTM models.
Specifically, the data that same sensor is detected in multiple historical junctures are a data sequences, by the data sequence
Used as input vector, each data in data sequence are each element in input vector.
S202, the data that sensor described in every class is detected at current time the defeated of LSTM models is trained into as described
Enter, obtain the abnormal point numerical testing result for having trained LSTM models to export.
With reference to the above-mentioned introduction to LSTM models, the number that the embodiment of the present invention detects every class sensor in the historical juncture
According to the input vector as training LSTM models, in the case of with n class sensors, it is believed that be LSTM model runnings
N chronomere, in each chronomere, the input vector of LSTM models is that a class sensor is detected in the historical juncture
Data.Based on LSTM models characteristic in itself, its relation between considering different input vectors in training, so as to use
During the data training LSTM models that various kinds of sensors is detected, it is ensured that LSTM models can learn between various kinds of sensors high
Contact.Inner link between not considering data characteristics when the abnormity point of flow data is detected compared to existing technology, the present invention is implemented
Example can more accurately detect abnormity point using the LSTM models trained.That is, the embodiment of the present invention will be to fluxion
According to outlier detection regard the classification problem based on time series as, it is contemplated that the relation between various kinds of sensors, improve different
The degree of accuracy of often point detection.
In order that those skilled in the art more understand technical scheme provided in an embodiment of the present invention, below above-mentioned steps enter
Row is described in detail.
Specifically, above-mentioned steps S201 can include：Using the flow data sliding window study of the LSTM models per class
The data that the sensor is detected in the historical juncture, obtain the destination probability value P (y=1 | h) of normal point, wherein, just
Often the value of point label y is 1；Label and the destination probability value P (y=1 | h) according to normal point, calculate and obtain cross entropy, its
In, the cross entropy as model loss function, the similitude for weighing probable value and label；According to the loss function
Adjust the parameter of the LSTM models.
What deserves to be explained is, the size that flow data slides study window illustrates each step-length for learning, and flow data is slided
The size of study window specifically can dynamically be adjusted according to input vector, it is also possible to be preset as fixed value.
Fig. 4 is the schematic diagram that flow data slides study window learning data feature.Wherein, each circle of the Tu Zhong the superiors
Circle represents an element of input vector, namely the data that sensor is detected in a certain historical juncture.Flow data slides study
The size of window is 4, i.e., learn 4 data characteristicses of element every time.K1, k2 and k3 represent each layer neural network parameter, specifically
Computing unit shown in reference picture 1, each layer of neural network parameter includes the ginseng being related in computing unit as shown in Figure 1
Number.By after each layer of neural network parameter treatment, vector representation, h as shown in Figure 4 being hidden accordinglyt-2、ht-1With
ht.As shown in Figure 1, the hiding vector representation of last layer will hide vector representation as next layer.
So, after obtaining hiding vector representation for each input vector successively learning data feature, by hiding vector table
Show and processed by logistic regression layer, that is, obtain the destination probability value P (y=1 | h) of normal point.That is, normal point label y
It is 1 to be worth, and the value of abnormity point label y is 0.Destination probability value P (y=1 | h) refers to the probability that detected data belong to normal point
Value, so that, detected data belong to the probable value as 1-P (y=1 | h) of abnormity point.In addition, loss function is used to weigh general
The similitude of rate value and label, if similar sexual satisfaction predetermined threshold value, can obtain described having trained LSTM moulds with deconditioning
Type；Alternatively, when frequency of training reaches threshold value, it is also possible to deconditioning, obtain described having trained LSTM models.
In a kind of possible implementation of the embodiment of the present invention, the flow data using the LSTM models is slided
The data that window study sensor described in per class is detected in the historical juncture, obtain the destination probability value P (y=1 of normal point
| h), including：Learn each class sensor respectively using the flow data sliding window of the LSTM models to be examined in the historical juncture
The feature of the data for measuring, obtains hiding vector representation；The hiding vector representation is returned by the logic of the LSTM models
Return layer to process, obtain the destination probability value P (y=1 | h).
Illustratively, 3 class sensors are in each historical juncture tn-wTo tnData are detected respectively, wherein, the classification of sensor
The data type that can be detected according to sensor is classified, such as above-mentioned 3 class sensor can be respectively temperature classes sensing
Device, humidity class sensor, pressure class sensor.Also, the quantity of each class sensor can be multiple.With tnMoment illustrates, tn
The flow data at momentWherein, { xt1,xt2It is temperature classes sensor in tnDetection
The data for arriving, { xh1It is humidity class sensor in tnThe data that moment detects, { xp1,xp2,xp3It is pressure class sensor in tn
The data that moment detects.So, the embodiment of the present invention is in this kind of possible implementation, by each class sensor in tn-w
To tnThe data that moment detects are used as the input vector for training LSTM models.
Specifically, as shown in figure 4, using LSTMtStudy temperature classes sensor is in tn-wTo tnThe data sequence that moment detects
The data characteristics of row, using LSTMhStudy temperature classes sensor is in tn-wTo tnThe data characteristics of the data sequence that the moment detects,
Using LSTMpStudy temperature classes sensor is in tn-wTo tnThe data characteristics of the data sequence that the moment detects, in by multiple
After the study of interbed, obtain hide vector representation h respectivelyt, hh, hp.Further, by ht, hh, hpAs the defeated of logistic regression layer
Incoming vector hs, i.e. hs=[ht, hh, hp], by after the activation primitive treatment of logistic regression layer, obtaining destination probability value P (y=1
|h)。
Illustratively, it is assumed that hs=[x1,x2,......xn], it is necessary to learn to y=a1x1+a2x2+ ...+b, i.e. y=
wl.hs+ b, then can do such as down conversion：Y=log (p/ (1-p)) is made, wherein p is the numerical value between [0,1], it is believed that be certain
The possibility that part thing occurs, doing further conversion for y=log (p/ (1-p)) can obtain：
The above-mentioned functional form for being Logic Regression Models, also referred to as sigmoid functions.By above formula understand p [0,1] it
Between it is continuous in whole real number field curve and can lead.When two classification problems is solved, a threshold value can be set, when p is more than
When threshold value, it is divided into classification A, otherwise, is divided into classification B.Then in embodiments of the present invention, classify for normal point and abnormity point
Problem, the data of detection are for the destination probability P (y=1 | h) of normal point：
The data of detection are for the destination probability P (y=0 | h) of abnormity point：
Further, cross entropy Ls of the destination probability P (y=1 | h) and label y between can be obtained by below equation
(θ)：
L (θ)=- y log (P (y=1 | h))+(1-y) log (1-P (y=1 | h))；
What deserves to be explained is, cross entropy is smaller to represent that destination probability is more similar to label, such that it is able to the cross entropy is made
It is loss function, each layer parameter to LSTM models is adjusted.And after parameter adjustment is carried out, the mistake shown in reference picture 4
Journey, continues based on 3 class sensors in tn-wLSTM models are trained to the data that the tn moment detects, until what is trained
Similitude between destination probability and label reaches predetermined threshold value, or, until frequency of training reaches threshold value, deconditioning is obtained
To having trained LSTM models.
In the alternatively possible implementation of the embodiment of the present invention, the flow data using the LSTM models is slided
The data that dynamic window study sensor described in per class is detected in the historical juncture, obtain the destination probability value P (y of normal point
=1 | h), including：Learn each class sensor respectively in the history using the flow data sliding window of the LSTM models
The feature of the data that quarter detects, obtains first and hides vector representation；Vector representation is hidden by described first pass through the LSTM
Model logistic regression layer treatment, obtain the first probable value P (y=1 | h1)；Slided using the flow data of the LSTM models
The feature of the data that dynamic window study all the sensors are detected in the historical juncture, obtains second and hides vector representation；Will
Described second hides vector representation by the layer treatment of the logistic regression of the LSTM models, obtain the second probable value P (y=1 |
h2)；According to the first probable value P (y=1 | h1) and the second probable value P (y=1 | h2) be calculated by equation below
The destination probability value P (y=1 | h)：
P (y=1 | h)=sigmoid (α P (y=1 | h1)+β P (y=1 | h2))
Wherein, α is more than 0, and alpha+beta=1 more than 0, β, and sigmoid is activation primitive.
Example based on Fig. 4 is illustrated.As shown in figure 5, with reference to the above-mentioned process described to Fig. 4, by hs=[ht, hh,
hp] by logistic regression layer activation primitive treatment obtain the first probable value P (y=1 | h1) after, by LSTMaLearn all biographies
Sensor is in tn-wThe feature of the data detected to the tn moment, obtains hiding vector representation ha, by haBy swashing for logistic regression layer
Function treatment living obtain the second probable value P (y=1 | h2).The destination probability value P (y=1 | h) for finally giving is the first probable value P
(y=1 | h1) and the second probable value P (y=1 | h2) weighted sum, wherein, weights α and β can set in advance according to the actual requirements
It is fixed.
After being calculated destination probability value P (y=1 | h) by this kind of possible implementation, it is also possible to calculate intersection
Entropy L (θ)：L (θ)=- (ylogP (y=1 | h)+(1-y) log (1-P (y=1 | h))), and according to L (θ) to the ginseng of LSTM models
Number is adjusted.
In above-mentioned possible implementation, LSTM models training when, the data conduct that each class sensor is detected
The input vector of LSTM models is trained, so as to take into account the contact between adjacent two classes sensor, while in order to capture difference
Contact between class individual sensor, the data that the sensor of all categories is detected are directly inputted as another input vector
LSTM models, contribute to LSTM model learnings to the deeper contact between different classes of sensor, further improve inspection
Survey the degree of accuracy.
Further, the embodiment of the present invention can also pass through SGD (Stochastic according to the cross entropy L (θ)
Gradient Descent, stochastic gradient descent) algorithm is adjusted to model parameter.
Specifically, parameter adjustment is as shown by the following formula：
θ={ Ws (i),Ws (f),Ws (c),Ws (o),Us (i),Us (f),Us (c),Us (o),
bs (i),bs (f),bs (c)bs (o),wl,b(i), s=t, h, p ..., a }
Wherein, l is learning rate, and θ represents the parameter of current each layer of LSTM models,The parameter after adjustment is represented,Represent
Cross entropy L (θ) is to θ derivations.Reference picture 1, the parameter of each computing unit includes W(i), U(i)And b(i), that is, calculate input gate to
Measure the parameter for representing, and W(f), U(f)And b(f), that is, calculate the parameter for forgetting a vector representation, and W(c), U(c)And b(c), that is, calculate the parameter of memory elements vector representation, and W(o), U(o)And b(o), that is, the parameter of out gate vector representation is calculated,
In addition, the relevant parameter that computing is used also including dot product etc..In corresponding to Fig. 4 and Fig. 5, the every LSTM shown in figuretRepresent
Each layer, the computing unit shown in each layer of structure reference picture 1.Being calculated by above-mentioned formula for every layer of each parameter can
In the hope of the parameter after adjustment.Wherein, LSTMh、LSTMpAnd LSTMaSimilarly, so, trained again according to the parameter after adjustment
LSTM models, terminate, the LSTM moulds trained until the satisfactory numerical value of cross entropy or frequency of training reach threshold value
Type.
What deserves to be explained is, can make each layer parameter of LSTM models towards greatest gradient side using stochastic gradient descent algorithm
To convergence, so as to ensure that the cross entropy of each training more comes small so that the cross entropy after training each time becomes closer to meet
It is required that numerical value.
The embodiment of the present invention also provides a kind of detection means 60, a kind of fluxion for implementing the offer of above method embodiment
According to abnormal point detecting method, as shown in Figure 6A, the detection means 60 includes：
Training module 601, for according to the data detected in the historical juncture per class sensor, and each history
The actual measurement abnormal point numerical at moment trains short-term memory recurrent neural network LSTM models long, and LSTM models have been trained in formation, its
In, the data detected in the historical juncture per sensor described in class as train one of the LSTM models be input into
Amount；
Detection module 602, for the data that sensor described in every class is detected at current time to have been trained as described
The input of LSTM models, obtains the abnormal point numerical testing result for having trained LSTM models to export.
Specifically, as shown in Figure 6B, the training module 601 includes：
Data characteristics study module 6011, for described in the every class of flow data sliding window study using the LSTM models
The data that sensor is detected in the historical juncture, obtain the destination probability value P (y=1 | h) of normal point, wherein, normal point
The value of label y is 1；
Computing module 6012, for the label according to normal point and the destination probability value P (y=1 | h), calculating is handed over
Fork entropy, wherein, the cross entropy as model loss function, the similitude for weighing probable value and label；
Parameter adjustment module 6013, the parameter for adjusting the LSTM models according to the loss function.
Alternatively, the data characteristics study module 6011 specifically for：
Learn each class sensor respectively using the flow data sliding window of the LSTM models to be examined in the historical juncture
The feature of the data for measuring, obtains hiding vector representation；
The logistic regression layer treatment that the hiding vector representation is passed through into the LSTM models, obtains the destination probability value
P (y=1 | h).
Alternatively, the data characteristics study module 6011 specifically for：
Learn each class sensor respectively using the flow data sliding window of the LSTM models to be examined in the historical juncture
The feature of the data for measuring, obtains first and hides vector representation；
Logistic regression layer treatment of the vector representation by the LSTM models is hidden by described first, described first is obtained general
Rate value P (y=1 | h1)；
Detected in the historical juncture using the flow data sliding window study all the sensors of the LSTM models
The feature of data, obtains second and hides vector representation；
Logistic regression layer treatment of the vector representation by the LSTM models is hidden by described second, the second probable value is obtained
P (y=1 | h2)；
According to the first probable value P (y=1 | h1) and the second probable value P (y=1 | h2) calculated by equation below
Obtain the destination probability value P (y=1 | h)：
P (y=1 | h)=sigmoid (α P (y=1 | h1)+β P (y=1 | h2))
Wherein, α is more than 0, and alpha+beta=1 more than 0, β, and sigmoid is activation primitive.
Alternatively, the computing module 6012 is used for：
The label of normal point and the cross entropy L (θ) of the destination probability value P (y=1 | h) are calculated by equation below：
L (θ)=- (ylogP (y=1 | h)+(1-y) log (1-P (y=1 | h))).
Alternatively, the parameter adjustment module 6013 specifically for：
Model parameter is adjusted by stochastic gradient descent algorithm according to the loss function L (θ).
Those skilled in the art can be understood that, for convenience and simplicity of description, only with above-mentioned each function mould
The division of block is carried out for example, in practical application, as needed can distribute by different functional modules above-mentioned functions
Complete, will the internal structure of device be divided into different functional modules, to complete all or part of function described above.
The specific work process of foregoing description functional module, may be referred to the corresponding process in preceding method embodiment, no longer go to live in the household of one's in-laws on getting married herein
State.
Using above-mentioned detection device, the data that the detection means detects every class sensor in the historical juncture are used as training
The input vector of LSTM models, based on LSTM models characteristic in itself, between it considers different input vectors in training
Relation, so as to when the data detected using various kinds of sensors train LSTM models, it is ensured that LSTM models can learn height
Contact between various kinds of sensors.Compared to existing technology detect flow data abnormity point when do not consider in data characteristics
Contact, the present invention can more accurately detect abnormity point using the LSTM models trained.
In embodiment provided herein, it should be understood that disclosed apparatus and method, can be by other
Mode is realized.For example, during each functional module in each embodiment of the invention can be integrated in a processing unit, it is also possible to
It is that unit is individually physically present.Above-mentioned integrated unit can both be realized in the form of hardware, it would however also be possible to employ hardware
Plus the form of SFU software functional unit is realized.
The above-mentioned integrated unit realized in the form of SFU software functional unit, can store and be deposited in an embodied on computer readable
In storage media.Above-mentioned SFU software functional unit storage is in a storage medium, including some instructions are used to so that a computer
Equipment (can be personal computer, server, or network equipment etc.) performs the portion of each embodiment methods described of the invention
Step by step.And foregoing storage medium includes：(Random Access Memory, arbitrary access is deposited for USB flash disk, mobile hard disk, RAM
Reservoir), magnetic disc or CD etc. are various can be with the medium of data storage.
The above, specific embodiment only of the invention, but protection scope of the present invention is not limited thereto, and it is any
Those familiar with the art the invention discloses technical scope in, the change or replacement that can be readily occurred in, all should
It is included within the scope of the present invention.Therefore, protection scope of the present invention should be defined by scope of the claims.
Claims (10)
1. a kind of abnormal point detecting method of flow data, it is characterised in that the flow data is included per class sensor in history
The data that quarter detects, and in the actual measurement abnormal point numerical of each historical juncture, methods described includes：
The data that the sensor according to every class is detected in the historical juncture, and the actual measurement abnormal point numerical training length
LSTM models have been trained in short-term memory recurrent neural network LSTM models, formation, wherein, sensor is in the history described in per class
The data that moment detects are used as an input vector for training the LSTM models；
The data that sensor described in every class was detected at current time obtain institute as the input for having trained LSTM models
State the abnormal point numerical testing result for having trained LSTM models to export.
2. method according to claim 1, it is characterised in that basis sensor described in per class is in the historical juncture
The data for detecting, and the actual measurement abnormal point numerical trains short-term memory recurrent neural network LSTM models long, is formed
Training LSTM models, including：
Detected in the historical juncture per sensor described in class using the flow data sliding window study of the LSTM models
Data, obtain the destination probability value P (y=1 | h) of normal point, wherein, the value of normal point label y is 1；
Label and the destination probability value P (y=1 | h) according to normal point, calculate and obtain cross entropy, wherein, the cross entropy
As the loss function of model, the similitude for weighing probable value and label；
The parameter of the LSTM models is adjusted according to the loss function.
3. method according to claim 2, it is characterised in that the flow data sliding window using the LSTM models
The data that study is detected per sensor described in class in the historical juncture, obtain the destination probability value P (y=1 | h) of normal point,
Including：
Learn each class sensor respectively using the flow data sliding window of the LSTM models to be detected in the historical juncture
Data feature, obtain hide vector representation；
The logistic regression layer treatment that the hiding vector representation is passed through into the LSTM models, obtains the destination probability value P (y
=1 | h).
4. method according to claim 2, it is characterised in that the flow data sliding window using the LSTM models
The data that study is detected per sensor described in class in the historical juncture, obtain the destination probability value P (y=1 | h) of normal point,
Including：
Learn each class sensor respectively using the flow data sliding window of the LSTM models to be detected in the historical juncture
Data feature, obtain first hide vector representation；
Logistic regression layer treatment of the vector representation by the LSTM models is hidden by described first, first probable value is obtained
P (y=1 | h1)；
The data detected in the historical juncture using the flow data sliding window study all the sensors of the LSTM models
Feature, obtain second hide vector representation；
Logistic regression layer treatment of the vector representation by the LSTM models is hidden by described second, the second probable value P (y are obtained
=1 | h2)；
According to the first probable value P (y=1 | h1) and the second probable value P (y=1 | h2) be calculated by equation below
The destination probability value P (y=1 | h)：
P (y=1 | h)=sigmoid (α P (y=1 | h1)+β P (y=1 | h2))；
Wherein, α is more than 0, and alpha+beta=1 more than 0, β, and sigmoid is activation primitive.
5. the method according to any one of claim 2 to 4, it is characterised in that the label according to normal point and described
Destination probability value P (y=1 | h), calculate and obtain cross entropy, including：
The label of normal point and the cross entropy L (θ) of the destination probability value P (y=1 | h) are calculated by equation below：
L (θ)=- (ylogP (y=1 | h)+(1-y) log (1-P (y=1 | h))).
6. method according to claim 5, it is characterised in that described that the LSTM models are adjusted according to the loss function
Parameter, including：
Model parameter is adjusted by stochastic gradient descent algorithm according to the loss function L (θ).
7. a kind of detection means, it is characterised in that including：
Training module, for according to the data detected in the historical juncture per class sensor, and each historical juncture
Actual measurement abnormal point numerical trains short-term memory recurrent neural network LSTM models long, and LSTM models have been trained in formation, wherein, per class
The data that the sensor was detected in the historical juncture are used as an input vector for training the LSTM models；
Detection module, for the data that sensor described in every class is detected at current time to have been trained into LSTM models as described
Input, obtain the abnormal point numerical testing result for having trained LSTM models to export.
8. device according to claim 7, it is characterised in that the training module includes：
Data characteristics study module, exists for sensor described in the every class of flow data sliding window study using the LSTM models
The data that the historical juncture detects, obtain the destination probability value P (y=1 | h) of normal point, wherein, the value of normal point label y
It is 1；
Computing module, for the label according to normal point and the destination probability value P (y=1 | h), calculates and obtains cross entropy, its
In, the cross entropy as model loss function, the similitude for weighing probable value and label；
Parameter adjustment module, the parameter for adjusting the LSTM models according to the loss function.
9. device according to claim 8, it is characterised in that the data characteristics study module specifically for：
Learn each class sensor respectively using the flow data sliding window of the LSTM models to be detected in the historical juncture
Data feature, obtain hide vector representation；
The logistic regression layer treatment that the hiding vector representation is passed through into the LSTM models, obtains the destination probability value P (y
=1 | h).
10. device according to claim 8, it is characterised in that the data characteristics study module specifically for：
Learn each class sensor respectively using the flow data sliding window of the LSTM models to be detected in the historical juncture
Data feature, obtain first hide vector representation；
Logistic regression layer treatment of the vector representation by the LSTM models is hidden by described first, first probable value is obtained
P (y=1 | h1)；
The data detected in the historical juncture using the flow data sliding window study all the sensors of the LSTM models
Feature, obtain second hide vector representation；
Logistic regression layer treatment of the vector representation by the LSTM models is hidden by described second, the second probable value P (y are obtained
=1 | h2)；
According to the first probable value P (y=1 | h1) and the second probable value P (y=1 | h2) be calculated by equation below
The destination probability value P (y=1 | h)：
P (y=1 | h)=sigmoid (α P (y=1 | h1)+β P (y=1 | h2))；
Wherein, α is more than 0, and alpha+beta=1 more than 0, β, and sigmoid is activation primitive.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611229506.5A CN106815639A (en) | 2016-12-27 | 2016-12-27 | The abnormal point detecting method and device of flow data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611229506.5A CN106815639A (en) | 2016-12-27 | 2016-12-27 | The abnormal point detecting method and device of flow data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106815639A true CN106815639A (en) | 2017-06-09 |
Family
ID=59110168
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611229506.5A Pending CN106815639A (en) | 2016-12-27 | 2016-12-27 | The abnormal point detecting method and device of flow data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106815639A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109447305A (en) * | 2018-06-23 | 2019-03-08 | 四川大学 | A kind of trend forecasting method based on the long Memory Neural Networks in short-term of quantum weighting |
| CN109871764A (en) * | 2019-01-16 | 2019-06-11 | 深兰科技（上海）有限公司 | A kind of abnormal behaviour recognition methods, device and storage medium |
| CN110287439A (en) * | 2019-06-27 | 2019-09-27 | 电子科技大学 | A kind of network behavior method for detecting abnormality based on LSTM |
| CN110430183A (en) * | 2019-07-31 | 2019-11-08 | 福建师范大学 | The MH-LSTM method for detecting abnormality of dialogue-based characteristic similarity |
| CN111351664A (en) * | 2020-02-13 | 2020-06-30 | 成都运达科技股份有限公司 | Bearing temperature prediction and alarm diagnosis method based on LSTM model |
| CN111611218A (en) * | 2020-04-24 | 2020-09-01 | 武汉大学 | Distributed abnormal log automatic identification method based on deep learning |
| CN112101400A (en) * | 2019-12-19 | 2020-12-18 | 国网江西省电力有限公司电力科学研究院 | Industrial control system abnormality detection method, equipment, server and storage medium |
| CN112585926A (en) * | 2018-08-23 | 2021-03-30 | 摩根士丹利服务集团有限公司 | Distributed system component failure identification |
| CN112862127A (en) * | 2021-04-23 | 2021-05-28 | 北京瑞莱智慧科技有限公司 | Sensor data exception handling method and device, electronic equipment and medium |
| CN115952465A (en) * | 2023-03-10 | 2023-04-11 | 畅捷通信息技术股份有限公司 | Sensor data anomaly detection method and device and computer storage medium |
-
2016
- 2016-12-27 CN CN201611229506.5A patent/CN106815639A/en active Pending
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109447305A (en) * | 2018-06-23 | 2019-03-08 | 四川大学 | A kind of trend forecasting method based on the long Memory Neural Networks in short-term of quantum weighting |
| CN109447305B (en) * | 2018-06-23 | 2021-09-07 | 四川大学 | Trend prediction method based on quantum weighted long-time and short-time memory neural network |
| CN112585926A (en) * | 2018-08-23 | 2021-03-30 | 摩根士丹利服务集团有限公司 | Distributed system component failure identification |
| CN109871764A (en) * | 2019-01-16 | 2019-06-11 | 深兰科技（上海）有限公司 | A kind of abnormal behaviour recognition methods, device and storage medium |
| CN110287439A (en) * | 2019-06-27 | 2019-09-27 | 电子科技大学 | A kind of network behavior method for detecting abnormality based on LSTM |
| CN110430183A (en) * | 2019-07-31 | 2019-11-08 | 福建师范大学 | The MH-LSTM method for detecting abnormality of dialogue-based characteristic similarity |
| CN112101400A (en) * | 2019-12-19 | 2020-12-18 | 国网江西省电力有限公司电力科学研究院 | Industrial control system abnormality detection method, equipment, server and storage medium |
| CN111351664A (en) * | 2020-02-13 | 2020-06-30 | 成都运达科技股份有限公司 | Bearing temperature prediction and alarm diagnosis method based on LSTM model |
| CN111611218A (en) * | 2020-04-24 | 2020-09-01 | 武汉大学 | Distributed abnormal log automatic identification method based on deep learning |
| CN112862127A (en) * | 2021-04-23 | 2021-05-28 | 北京瑞莱智慧科技有限公司 | Sensor data exception handling method and device, electronic equipment and medium |
| CN112862127B (en) * | 2021-04-23 | 2021-07-23 | 北京瑞莱智慧科技有限公司 | Sensor data exception handling method and device, electronic equipment and medium |
| CN115952465A (en) * | 2023-03-10 | 2023-04-11 | 畅捷通信息技术股份有限公司 | Sensor data anomaly detection method and device and computer storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106815639A (en) | The abnormal point detecting method and device of flow data | |
| WO2021042935A1 (en) | Bearing service life prediction method based on hidden markov model and transfer learning | |
| CN108900546A (en) | The method and apparatus of time series Network anomaly detection based on LSTM | |
| US20200231466A1 (en) | Intelligent systems and methods for process and asset health diagnosis, anomoly detection and control in wastewater treatment plants or drinking water plants | |
| CN106326984A (en) | User intention identification method and device and automatic answering system | |
| CN108664690A (en) | Long-life electron device reliability lifetime estimation method under more stress based on depth belief network | |
| US11593648B2 (en) | Methods and systems for detection and isolation of bias in predictive models | |
| CN109523021A (en) | A kind of dynamic network Structure Prediction Methods based on long memory network in short-term | |
| US11379685B2 (en) | Machine learning classification system | |
| CN108647643A (en) | A kind of packed tower liquid flooding state on-line identification method based on deep learning | |
| CN112116002A (en) | Determination method, verification method and device of detection model | |
| Rodríguez‐Borbón et al. | Reliability estimation for accelerated life tests based on a Cox proportional hazard model with error effect | |
| CN110263808B (en) | Image emotion classification method based on LSTM network and attention mechanism | |
| CN111949496B (en) | Data detection method and device | |
| Jin et al. | Toward predictive fault tolerance in a core-router system: Anomaly detection using correlation-based time-series analysis | |
| CN111079348A (en) | Method and device for detecting slowly-varying signal | |
| Yangzhen et al. | A software reliability prediction model: Using improved long short term memory network | |
| Prisacaru et al. | Degradation prediction of electronic packages using machine learning | |
| Ding et al. | A zero-shot soft sensor modeling approach using adversarial learning for robustness against sensor fault | |
| KR20220049573A (en) | Distance-based learning trust model | |
| Lo | Predicting software reliability with support vector machines | |
| CN113156473A (en) | Self-adaptive discrimination method for satellite signal environment of information fusion positioning system | |
| CN107038143A (en) | Belt conveyer scale method for diagnosing faults based on improved multilayer artificial immune network model | |
| CN105468887B (en) | Data analysis system and method | |
| CN115293249A (en) | Power system typical scene probability prediction method based on dynamic time sequence prediction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170609 |
|
| RJ01 | Rejection of invention patent application after publication |