CN106302487A - Agricultural Internet of Things data flow anomaly detects processing method and processing device in real time - Google Patents

Agricultural Internet of Things data flow anomaly detects processing method and processing device in real time Download PDF

Info

Publication number
CN106302487A
CN106302487A CN201610702928.3A CN201610702928A CN106302487A CN 106302487 A CN106302487 A CN 106302487A CN 201610702928 A CN201610702928 A CN 201610702928A CN 106302487 A CN106302487 A CN 106302487A
Authority
CN
China
Prior art keywords
data
moment
sliding window
collection
current time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201610702928.3A
Other languages
Chinese (zh)
Other versions
CN106302487B (en
Inventor
段青玲
肖晓琰
张磊
王剑秦
魏芳芳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Agricultural University
Original Assignee
China Agricultural University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Agricultural University filed Critical China Agricultural University
Priority to CN201610702928.3A priority Critical patent/CN106302487B/en
Publication of CN106302487A publication Critical patent/CN106302487A/en
Application granted granted Critical
Publication of CN106302487B publication Critical patent/CN106302487B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/12—Applying verification of the received information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/12—Applying verification of the received information
    • H04L63/123—Applying verification of the received information received data contents, e.g. message integrity
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Abstract

A kind of agricultural Internet of Things data flow anomaly that the present invention provides detects processing method and processing device in real time, in the method, first determine the size of sliding window according to the time interval of the eigenperiod and collection that gather data, gather measured value and the forecast interval of data prediction current time sensor acquisition further according to the history in sliding window；Then by the actual measured value of current time actual acquisition compared with forecast interval, if not falling within forecast interval, then the data of explanation current time actual acquisition are abnormal data, thus realize the detection to abnormal data.In addition, the method that the present invention provides is after judging that current time data are as abnormal data, utilize the predictive value obtained to substitute abnormal data, such that it is able to realize the process to abnormal data, it is effectively improved the accuracy of the data stream of collection, provides strong data support for automatically controlling of equipment and effective data analysis.

Description

Agricultural Internet of Things data flow anomaly detects processing method and processing device in real time
Technical field
The invention belongs to field of computer technology, process particularly to the Internet of Things data flow anomaly detection in real time of a kind of agricultural Method and device.
Background technology
Internet of Things is widely used in the agricultural productions such as livestock and poultry cultivation, facilities horticulture, aquaculture, becomes acquisition data One of important means.In agricultural Internet of Things, awareness apparatus periodic measurement equipment state and ambient parameter, its time gathered Sequence data has the features such as infinite property, dynamic, temporal correlation, and processes system with the form transmission of data stream to data System.But the environment that agricultural Internet of Things is disposed is relatively more severe, and such as livestock and poultry cultivation place dust is more, aquaculture Internet of Things host In sea water to be deployed in or freshwater aquiculture pond, humid environment, the network equipment is easily damaged；Simultaneously as agricultural production is wide Property, Internet of Things network coverage area is wide, equipment dispersion, needs the most in this environment to transmit number by wireless network According to.
Performance yet with the sensor device in wireless network is not sufficiently stable, and duration can occur data transmission network event Barrier, so that leading to a phenomenon of abnormal data happens occasionally, causes the quality of data to decline, it is impossible to ensure automatically controlling of equipment With effective data analysis.Therefore, the most quickly and efficiently Internet of Things data stream is carried out anomaly data detection be one urgently The problem solved.
In prior art, much more typically the method for the anomaly data detection of Internet of Things data stream is paid attention to according to data stream A certain feature process emphatically.But such method cannot take into account Internet of Things data stream real-time and detection accurately simultaneously The features such as property so that accuracy and the integrity of the data stream received the most effectively are promoted.
Summary of the invention
It is an advantage of the invention to provide a kind of agricultural Internet of Things data flow anomaly and detect processing method and dress in real time Put, use the data accuracy that existing abnormal deviation data examination method obtains, the defect that real-time is inadequate for overcoming.
First aspect, the invention provides a kind of agricultural Internet of Things data flow anomaly and detects processing method in real time, including:
Gather eigenperiod and the time interval of adjacent twice collection of data according in agricultural Internet of Things data stream, obtain Take size q of sliding window；Wherein, size q of described sliding window, the collection comprised in limiting described sliding window The number of historical data；
When the data forward slip collected in Internet of Things data stream is to subsequent time, according in described sliding window Historical data, obtains predictive value and the forecast interval of current time sensor acquisition；
When judging to know that the actual measured value of current time sensor acquisition does not falls within described forecast interval, then ought Front moment actual measured value is categorized as abnormal data, and using described predictive value as the data of current time sensor acquisition；
Update described sliding window；
Wherein, when the data collected in Internet of Things data stream are by tN-1Moment forward slip is to tNDuring the moment, according to described The history comprised in sliding window gathers data, obtains predictive value and the step of forecast interval of current time sensor acquisition Including: according to the t comprised in sliding windowN-qMoment is to tN-1Q the historical data that moment gathers, based on default prediction mould Type, obtains tNThe predictive value of moment sensor acquisition and forecast interval；
Accordingly, the step updating described sliding window includes: by described tNThe data of moment sensor acquisition are added extremely In sliding window, delete the t in sliding windowN-qThe collection data in moment.
Alternatively, described according to agricultural Internet of Things data stream gathers the eigenperiod of data and adjacent twice collection Time interval, the step of size q obtaining sliding window includes:
According to the T and the time interval Δ of adjacent twice collection eigenperiod gathering data in agricultural Internet of Things data stream T, size q based on following formula acquisition sliding window:
q = ∫ t t + T log 2 T / Δ t
Wherein, T represents the eigenperiod gathering data in agricultural Internet of Things in data stream, and Δ t represents internet of things sensors Gather the time interval of data.
Alternatively, described according to the t comprised in sliding windowN-qMoment is to tN-1Q the historical data that moment gathers, based on The forecast model preset, obtains tNThe predictive value of moment sensor acquisition and the step of forecast interval include:
According to the t comprised in sliding windowN-qMoment is to tN-1Q the historical data that moment gathers, builds non-linear support Vector regression model, obtains tNThe predictive value of moment sensor acquisition and forecast interval.
Alternatively, described method also includes:
When judging to know that the actual measured value that current time gathers lacks, described predictive value is sensed as current time The collection data of device.
Alternatively, described method also includes:
When judging to know that the actual measured value of current time sensor falls into described forecast interval, by described actual measurement It is worth the collection data as current time sensor.
Second aspect, the invention provides a kind of agricultural Internet of Things data flow anomaly and detects processing means in real time, including:
Sliding window acquiring size module, for gathering eigenperiod and the phase of data according to agricultural Internet of Things data stream The time interval of adjacent twice collection, obtains size q of sliding window；Wherein, size q of described sliding window, it is used for limiting institute The historical data number comprised in stating sliding window；
Gather data prediction module, for when described sliding window forward slip one time interval, according to described slip Historical data in window, obtains predictive value and the forecast interval of current time sensor acquisition；
Gather data classification and processing module, described pre-for not falling within the actual measured value knowing this moment sensor Time in survey is interval, this moment actual measured value is categorized as abnormal data, and using described predictive value as current time sensor Collection data；
Window more new module, is used for updating described sliding window；
Wherein, when the data collected in Internet of Things data stream are by tN-1Moment forward slip is to tNMoment constantly, described in adopt Collection data prediction module, is further used for: according to the t comprised in sliding windowN-qMoment is to tN-1Q the history number that moment gathers According to, based on default forecast model, obtain tNThe predictive value of moment sensor acquisition and forecast interval；
Accordingly, described window more new module, it is further used for: by described tNThe data of moment sensor acquisition are added extremely In sliding window, delete the t in sliding windowN-qThe collection data in moment.
Alternatively, described sliding window acquiring size module, it is further used for:
Agricultural Internet of Things data stream gathers the eigenperiod of data and the time interval Δ t of adjacent twice collection, base Size q in following formula acquisition sliding window:
q = ∫ t t + T log 2 T / Δ t
Wherein, T represents the eigenperiod gathering data in agricultural Internet of Things in data stream, and Δ t represents internet of things sensors Gather the time interval of data.
Alternatively, described collection data prediction module, it is further used for:
According to the t comprised in sliding windowN-qMoment is to tN-1Q the historical data that moment gathers, builds non-linear support Vector regression model, obtains tNThe predictive value of moment sensor acquisition and forecast interval
Alternatively, the classification of described collection data and processing module, it is further used for:
When judging to know that the actual measured value that current time gathers lacks, described predictive value is sensed as current time The collection data of device.
Alternatively, the classification of described collection data and processing module, it is further used for:
When judging to know that the actual measured value of current time sensor falls into described forecast interval, by described actual measurement It is worth the collection data as current time sensor.
The agriculture Internet of Things data flow anomaly that the embodiment of the present invention provides detects in processing method, in real time first according to collection Cycle and interval obtain sliding window size, further according in sliding window history gather data prediction current time gather Predictive value and forecast interval；Then by the actual measured value of current time actual acquisition compared with forecast interval, if not falling Enter in forecast interval, then the data of explanation current time actual acquisition are abnormal data, thus realize the detection to abnormal data. Additionally, the method that the present invention provides is after judging that current time data are as abnormal data, utilize the predictive value obtained different to substitute Regular data, such that it is able to realize process to abnormal data, is effectively improved the accuracy of the data stream of collection, automatic for equipment Control the data support strong with the offer of effective data analysis.
Accompanying drawing explanation
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, embodiment will be described below The accompanying drawing used required in is briefly described, it should be apparent that, the accompanying drawing in describing below is only some of the present invention Example, for those of ordinary skill in the art, on the premise of not paying creative work, it is also possible to according to these accompanying drawings Obtain other accompanying drawing.
Fig. 1 is that the agriculture Internet of Things data flow anomaly that the present invention provides detects processing method embodiment flow chart in real time；
Fig. 2 is that the agriculture Internet of Things data flow anomaly that the present invention provides detects the signal of processing means example structure in real time Figure.
Detailed description of the invention
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete Describe, it is clear that described embodiment is only a part of embodiment of the present invention rather than whole embodiments wholely.Based on Embodiment in the present invention, the every other reality that ordinary skill people is obtained under not making creative work premise Execute example, broadly fall into the scope of protection of the invention.
First aspect, embodiments provides a kind of agricultural Internet of Things data flow anomaly and detects processing method in real time, As it is shown in figure 1, include:
S1, according to agricultural Internet of Things data stream in gather between the eigenperiod of data and the time of adjacent twice collection Every, obtain size q of sliding window；Wherein, size q of described sliding window, comprise in limiting described sliding window The number of the historical data gathered；
When S2, the data forward slip collected in Internet of Things data stream are to subsequent time, according to described sliding window Interior historical data, obtains predictive value and the forecast interval of current time sensor acquisition；
S3, judge know that the actual measured value of current time sensor acquisition does not falls within described forecast interval time, then Current time actual measured value is categorized as abnormal data, and using described predictive value as the number of current time sensor acquisition According to；
S4, renewal sliding window；
Wherein, when the data collected in Internet of Things data stream are by tN-1Moment forward slip is to tNDuring the moment:
Step S2 may include that according to the t comprised in sliding windowN-qMoment is to tN-1Q the historical data that moment gathers, Based on default forecast model, obtain tNThe predictive value of moment sensor acquisition and forecast interval；
Accordingly, step S4 may include that described tNThe data of moment sensor acquisition are added to sliding window, delete Except the t in sliding windowN-qThe collection data in moment.
The agriculture Internet of Things data flow anomaly that the embodiment of the present invention provides detects in processing method, in real time first according to collection Cycle and interval obtain sliding window size, further according in sliding window history gather data prediction current time gather Predictive value and forecast interval；Then by the actual measured value of current time actual acquisition compared with forecast interval, if not falling Enter in forecast interval, then the data of explanation current time actual acquisition are abnormal data, thus realize the detection to abnormal data. Additionally, the method that the present invention provides is after judging that current time data are as abnormal data, utilize the predictive value obtained different to substitute Regular data, such that it is able to realize process to abnormal data, is effectively improved the accuracy of the data stream of collection, automatic for equipment Control the data support strong with the offer of effective data analysis.
In the specific implementation, it is to be understood that the method in step S1 can be accomplished in several ways, below to step A kind of optional embodiment of rapid S1 illustrates.
First size q of sliding window is determined.Size q of sliding window here, comprises in being used for limiting sliding window History gather data number.Owing to agricultural Internet of Things gathering data stream, there is timing, periodic feature, therefore slide The size of dynamic window q depends on eigenperiod and the time of adjacent twice collection gathering data in agricultural Internet of Things data stream Interval, specifically can pass through formula (1) and try to achieve:
q = ∫ t t + T log 2 T / Δ t - - - ( 1 )
Wherein, T represents the eigenperiod gathering data in agricultural Internet of Things in data stream, and Δ t represents internet of things sensors Gathering the time interval of data, t is the current collection moment.
After obtain the size of sliding window through step S1, a kind of optional embodiment of step S2 can wrap Include: when the data collected in Internet of Things data stream are by tN-1Moment forward slip is to tNDuring the moment, comprise according in sliding window TN-qMoment is to tN-1Q the historical data that moment gathers, builds non-linear support vector regression model, obtains tNMoment senses The predictive value of device collection and forecast interval.
Specifically, utilize support vector regression model that the situation of the internet of things sensors n-th collection of t is carried out Estimate calculating, can pass through formula (2) calculate n-th gather predictive value:
x t ‾ = M ( D t ) + R - - - ( 2 )
Wherein,Represent the predictive value of t；M(Dt) represent non-linear support vector regression model；R represents experience wind Danger, describes with the insensitive function of loss function-e.
For non-linear support vector regression algorithm, need to introduce kernel function, the data of luv space are mapping through φ It is mapped in higher dimensional space, shown in its mapping relations such as formula (3):
Ф: x → φ (x) (3)
In high-dimensional feature space, do approximately linear return, shown in its training set such as formula (4):
Θ={ (φ (x1),y1), (φ (x2),y2) ... (φ (xq),yq)} (4)
Regression problem can be described as: according to given training set Θ, finds RnOn certain real-valued function y=f (x), with This infers the y value corresponding to either mode φ (x).Concrete function is represented by:
F (x)=wT·φ(x)+b (5)
Wherein, w is normal vector；B is intercept.
Use the largest interval model of approximately linear classification, obtain following objective functions:
m a x 1 | | w | |
S.t., yi(wTφ(xi)+b) >=1, i=1 ..., n (6)
This object function is converted into convex quadratic programming problem, and target is optimum the most under certain condition, loss reduction.To convex Quadratic programming problem uses Lagrange duality to transform to the optimization problem of dual variable, obtains by solving its dual problem The optimal solution of primal problem, Lagrangian is:
In the case of requiring constraints to be met, object function is as follows:
Here p is used*Represent the optimal value of this problem, and be of equal value with initial problem.Thus it is converted into this antithesis The solution of problem, namely p*The predictive value gathered for t.
After having tried to achieve predictive value, recycling model residual error, calculate with predictive value as sample average, confidence level is p Forecast interval PI.
Specifically, it is assumed that it is that (p can be p that the actual measured value in t collection falls into the probability of forecast interval PI 95% or 99%), and model residual error meets the Gauss distribution that average is zero, therefore, it was predicted that interval PI bound can be such as formula (9) Shown in:
P I = x t ‾ ± t α / 2 , n - 1 × s 1 + 1 n - - - ( 9 )
Wherein,Represent the predictive value of t, tα/2,n-1The percentile degree of freedom of expression p is n-1 Meet student-t-distribution probability-distribution function, s is the standard deviation of n sample, thus according to formula (9) obtain forecast interval PI。
After obtaining predictive value and forecast interval through step S2, step S3 can be according to forecast interval to current time The actual measured value gathered detects, it is judged that whether it is abnormal data.
Specifically, when judging to know that the actual measured value of t collection does not falls within forecast interval PI, then it is assumed that t The actual measured value that moment gathers is abnormal data, and this actual measured value is now replaced with the t obtained by step S2 The predictive value of sensor acquisition is as the data of current time sensor acquisition.
In the detection to actual measured value, in addition to the situation that above-mentioned abnormal data is detected, it is also possible to go out Existing two kinds of situations:
When judging to know that the actual measured value that t gathers lacks, then it is assumed that current time gathers unsuccessfully, now by t The collection data that the predictive value of moment sensor acquisition gathers as n-th.
Judge know t collection actual measured value fall into described forecast interval time, then it is assumed that current time is adopted The data of collection are not detected by exception, the collection data therefore described actual measured value gathered as t.
Understandable, three kinds of situations described above cover the actual measured value detection that current time gathers substantially The data of actual acquisition each time can be carried out abnormality detection according to said method by contingent several situation, in detection To time abnormal or detect gather unsuccessfully time, utilize predictive value to gather data as current time such that it is able to be effectively improved and adopt The accuracy of the data stream of collection and integrity, provide strong data to prop up for automatically controlling of equipment and effective data analysis Hold.
After the judgement of step S3, step S4 also needs to be updated sliding window.
Specifically, in agricultural Internet of Things, the data stream of sensor acquisition is exactly unlimited according to certain time order composition Set, available ..., xt-1,xt,xt+1... } represent；Sliding window refers to choose the window that fixed data quantity processes as data, Use Dt{xt-q,xt-q+1,…xt-1Represent；After current time actual measured value is carried out anomaly classification, window forward slip, increase xt, delete xt-q；Keeping sliding window size to fix, terminal was the current upper moment gathering the corresponding moment forever, consequently facilitating Collection next time judges.
Second aspect, the embodiment of the present invention also provides for the abnormal detection in real time in real time of a kind of agricultural Internet of Things data stream and processes dress Put, as in figure 2 it is shown, include:
Sliding window acquiring size module, for gathering eigenperiod and the phase of data according to agricultural Internet of Things data stream The time interval of adjacent twice collection, obtains size q of sliding window；Wherein, size q of described sliding window, it is used for limiting institute The historical data number comprised in stating sliding window；
Gather data prediction module, for when described sliding window forward slip one time interval, according to described slip Historical data in window, obtains predictive value and the forecast interval of current time sensor acquisition；
Gather data classification and processing module, described pre-for not falling within the actual measured value knowing this moment sensor Time in survey is interval, this moment actual measured value is categorized as abnormal data, and using described predictive value as current time sensor Collection data；
Window more new module, is used for updating described sliding window；
Wherein, when the data collected in Internet of Things data stream are by tN-1Moment forward slip is to tNMoment constantly, described in adopt Collection data prediction module, is further used for: according to the t comprised in sliding windowN-qMoment is to tN-1Q the history number that moment gathers According to, based on default forecast model, obtain tNThe predictive value of moment sensor acquisition and forecast interval；
Accordingly, described window more new module, it is further used for: by described tNThe data of moment sensor acquisition are added extremely In sliding window, delete the t in sliding windowN-qThe collection data in moment.
In the specific implementation, described sliding window acquiring size module, it is further used for:
Agricultural Internet of Things data stream gathers the eigenperiod of data and the time interval Δ t of adjacent twice collection, base Size q in following formula acquisition sliding window:
q = ∫ t t + T log 2 T / Δ t
Wherein, T represents the eigenperiod gathering data in agricultural Internet of Things in data stream, and Δ t represents internet of things sensors Gather the time interval of data.
In the specific implementation, described collection data prediction module, it is further used for:
According to the t comprised in sliding windowN-qMoment is to tN-1Q the historical data that moment gathers, builds non-linear support Vector regression model, obtains tNThe predictive value of moment sensor acquisition and forecast interval
In the specific implementation, the classification of described collection data and processing module, it is further used for:
When judging to know that the actual measured value that current time gathers lacks, described predictive value is sensed as current time The collection data of device.
In the specific implementation, the classification of described collection data and processing module, it is further used for:
When judging to know that the actual measured value of current time sensor falls into described forecast interval, by described actual measurement It is worth the collection data as current time sensor.
The agriculture Internet of Things data flow anomaly introduced due to the present embodiment detects processing means in real time for can perform basis Agriculture Internet of Things data flow anomaly in inventive embodiments detects the device of processing method in real time, so based on the embodiment of the present invention Described in agriculture Internet of Things data flow anomaly detection method, those skilled in the art will appreciate that the present embodiment Agricultural Internet of Things data flow anomaly detects detailed description of the invention and its various versions of processing means in real time, so at this The agriculture Internet of Things how processing means realizes in the embodiment of the present invention is detected in real time for this agricultural Internet of Things data flow anomaly Data flow anomaly detects processing method in real time and is no longer discussed in detail.As long as those skilled in the art implement the embodiment of the present invention Middle agricultural Internet of Things data flow anomaly detects the device that processing method is used in real time, broadly falls into the model that the application to be protected Enclose.
Algorithm and display are not intrinsic to any certain computer, virtual system or miscellaneous equipment relevant provided herein. Various general-purpose systems can also be used together with based on teaching in this.As described above, construct required by this kind of system Structure be apparent from.Additionally, the present invention is also not for any certain programmed language.It is understood that, it is possible to use various Programming language realizes the content of invention described herein, and the description done language-specific above is to disclose this Bright preferred forms.
In description mentioned herein, illustrate a large amount of detail.It is to be appreciated, however, that the enforcement of the present invention Example can be put into practice in the case of not having these details.In some instances, it is not shown specifically known method, structure And technology, in order to do not obscure the understanding of this description.
Similarly, it will be appreciated that one or more in order to simplify that the disclosure helping understands in each inventive aspect, exist Above in the description of the exemplary embodiment of the present invention, each feature of the present invention is grouped together into single enforcement sometimes In example, figure or descriptions thereof.But, the method for the disclosure should not be construed to reflect an intention that i.e. required guarantor The application claims feature more more than the feature being expressly recited in each claim protected.More precisely, as following Claims reflected as, inventive aspect is all features less than single embodiment disclosed above.Therefore, The claims following detailed description of the invention are thus expressly incorporated in this detailed description of the invention, the most each claim itself All as the independent embodiment of the present invention.
Those skilled in the art are appreciated that and can carry out the module in the equipment in embodiment adaptively Change and they are arranged in one or more equipment different from this embodiment.Can be the module in embodiment or list Unit or assembly are combined into a module or unit or assembly, and can put them in addition multiple submodule or subelement or Sub-component.In addition at least some in such feature and/or process or unit excludes each other, can use any Combine all features disclosed in this specification (including adjoint claim, summary and accompanying drawing) and so disclosed appoint Where method or all processes of equipment or unit are combined.Unless expressly stated otherwise, this specification (includes adjoint power Profit requires, summary and accompanying drawing) disclosed in each feature can be carried out generation by providing identical, equivalent or the alternative features of similar purpose Replace.
Although additionally, it will be appreciated by those of skill in the art that embodiments more in this include institute in other embodiments Including some feature rather than further feature, but the combination of the feature of different embodiment means to be in the scope of the present invention Within and form different embodiments.Such as, in the following claims, embodiment required for protection any it One can mode use in any combination.
The all parts embodiment of the present invention can realize with hardware, or to run on one or more processor Software module realize, or with combinations thereof realize.It will be understood by those of skill in the art that and can use in practice Microprocessor or digital signal processor (DSP) realize in gateway according to embodiments of the present invention, proxy server, system The some or all functions of some or all parts.The present invention is also implemented as performing side as described herein Part or all equipment of method or device program (such as, computer program and computer program).Such The program realizing the present invention can store on a computer-readable medium, or can have the shape of one or more signal Formula.Such signal can be downloaded from internet website and obtain, or provides on carrier signal, or with any other shape Formula provides.
The present invention will be described rather than limits the invention to it should be noted above-described embodiment, and ability Field technique personnel can design alternative embodiment without departing from the scope of the appended claims.In the claims, Any reference marks that should not will be located between bracket is configured to limitations on claims.Word " comprises " and does not excludes the presence of not Arrange element in the claims or step.Word "a" or "an" before being positioned at element does not excludes the presence of multiple such Element.The present invention and can come real by means of including the hardware of some different elements by means of properly programmed computer Existing.If in the unit claim listing equipment for drying, several in these devices can be by same hardware branch Specifically embody.Word first, second and third use do not indicate that any order.These word explanations can be run after fame Claim.

Claims (10)

1. an agriculture Internet of Things data flow anomaly detects processing method in real time, it is characterised in that including:
Gather eigenperiod and the time interval of adjacent twice collection of data according in agricultural Internet of Things data stream, obtain sliding Size q of dynamic window；Wherein, size q of described sliding window, the history of the collection comprised in limiting described sliding window The number of data；
When the data forward slip collected in Internet of Things data stream is to subsequent time, according to the history in described sliding window Data, obtain predictive value and the forecast interval of current time sensor acquisition；
When judging to know that the actual measured value of current time sensor acquisition does not falls within described forecast interval, then by time current Carve actual measured value and be categorized as abnormal data, and using described predictive value as the data of current time sensor acquisition；
Update described sliding window；
Wherein, when the data collected in Internet of Things data stream are by tN-1Moment forward slip is to tNDuring the moment, according to described slip The history comprised in window gathers data, obtains predictive value and the step bag of forecast interval of current time sensor acquisition Include: according to the t comprised in sliding windowN-qMoment is to tN-1Q the historical data that moment gathers, based on default forecast model, Obtain tNThe predictive value of moment sensor acquisition and forecast interval；
Accordingly, the step updating described sliding window includes: by described tNThe data of moment sensor acquisition are added to sliding window In Kou, delete the t in sliding windowN-qThe collection data in moment.
Method the most according to claim 1, it is characterised in that described according to collection data in agricultural Internet of Things data stream Eigenperiod and the time interval of adjacent twice collection, the step of size q obtaining sliding window includes:
According to the T and the time interval △ t of adjacent twice collection, base eigenperiod gathering data in agricultural Internet of Things data stream Size q in following formula acquisition sliding window:
Wherein, T represents the eigenperiod gathering data in agricultural Internet of Things in data stream, and Δ t represents internet of things sensors collection The time interval of data.
Method the most according to claim 1, it is characterised in that described according to the t comprised in sliding windowN-qMoment is to tN-1 Q the historical data that moment gathers, based on default forecast model, obtains tNThe predictive value of moment sensor acquisition and prediction Interval step includes:
According to the t comprised in sliding windowN-qMoment is to tN-1Q the historical data that moment gathers, builds non-linear support vector Regression model, obtains tNThe predictive value of moment sensor acquisition and forecast interval.
Method the most according to claim 1, it is characterised in that described method also includes:
When judging to know that the actual measured value that current time gathers lacks, using described predictive value as current time sensor Gather data.
Method the most according to claim 1, it is characterised in that described method also includes:
When judging to know that the actual measured value of current time sensor falls into described forecast interval, described actual measured value is made Collection data for this moment sensor.
6. an agriculture Internet of Things data flow anomaly detects processing means in real time, it is characterised in that including:
Sliding window acquiring size module, for gathering the eigenperiod and adjacent two of data according to agricultural Internet of Things data stream The time interval of secondary collection, obtains size q of sliding window；Wherein, size q of described sliding window, it is used for limiting described cunning The historical data number comprised in dynamic window；
Gather data prediction module, for when described sliding window forward slip one time interval, according to described sliding window Interior historical data, obtains predictive value and the forecast interval of current time sensor acquisition；
Gather data classification and processing module, for not falling within described Target area in the actual measured value knowing this moment sensor Time interior, this moment actual measured value is categorized as abnormal data, and using described predictive value adopting as current time sensor Collection data；
Window more new module, is used for updating described sliding window；
Wherein, when the data collected in Internet of Things data stream are by tN-1Moment forward slip is to tNMoment constantly, described collection number It is predicted module, be further used for: according to the t comprised in sliding windowN-qMoment is to tN-1Q the historical data that moment gathers, Based on default forecast model, obtain tNThe predictive value of moment sensor acquisition and forecast interval；
Accordingly, described window more new module, it is further used for: by described tNThe data of moment sensor acquisition are added to sliding In window, delete the t in sliding windowN-qThe collection data in moment.
Device the most according to claim 6, it is characterised in that described sliding window acquiring size module, is further used for:
Agricultural Internet of Things data stream in gather the eigenperiod of data and the time interval △ t of adjacent twice collection, based under Size q of formula acquisition sliding window:
Wherein, T represents the eigenperiod gathering data in agricultural Internet of Things in data stream, and Δ t represents internet of things sensors collection The time interval of data.
Device the most according to claim 6, it is characterised in that described collection data prediction module, is further used for:
According to the t comprised in sliding windowN-qMoment is to tN-1Q the historical data that moment gathers, builds non-linear support vector Regression model, obtains tNThe predictive value of moment sensor acquisition and forecast interval.
Device the most according to claim 6, it is characterised in that the classification of described collection data and processing module, uses further In:
When judging to know that the actual measured value that current time gathers lacks, using described predictive value as current time sensor Gather data.
Device the most according to claim 6, it is characterised in that the classification of described collection data and processing module, uses further In:
When judging to know that the actual measured value of current time sensor falls into described forecast interval, by described actual measured value Collection data as current time sensor.
CN201610702928.3A 2016-08-22 2016-08-22 Agriculture internet of things data throat floater real-time detection processing method and processing device Active CN106302487B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610702928.3A CN106302487B (en) 2016-08-22 2016-08-22 Agriculture internet of things data throat floater real-time detection processing method and processing device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610702928.3A CN106302487B (en) 2016-08-22 2016-08-22 Agriculture internet of things data throat floater real-time detection processing method and processing device

Publications (2)

Publication Number Publication Date
CN106302487A true CN106302487A (en) 2017-01-04
CN106302487B CN106302487B (en) 2019-08-09

Family

ID=57662165

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610702928.3A Active CN106302487B (en) 2016-08-22 2016-08-22 Agriculture internet of things data throat floater real-time detection processing method and processing device

Country Status (1)

Country Link
CN (1) CN106302487B (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107346301A (en) * 2016-12-02 2017-11-14 西交利物浦大学 Water quality monitoring noise data real-time detection method based on Double time window checking
CN108304315A (en) * 2017-12-14 2018-07-20 国家计算机网络与信息安全管理中心 A kind of sorting technique and system of software aging abnormal behaviour
CN109617765A (en) * 2019-01-02 2019-04-12 中国联合网络通信集团有限公司 A kind of prediction technique and device of Internet of Things quality of connection
CN109890002A (en) * 2019-01-04 2019-06-14 中国电子科技集团公司电子科学研究院 A kind of method and device of the prediction Risk Degree of Maneuvering Environment of knowledge-based inference
CN110099089A (en) * 2018-01-31 2019-08-06 Sap欧洲公司 The self-tuing on line of multiple data flows in sensor network
CN110311879A (en) * 2018-03-20 2019-10-08 重庆邮电大学 A kind of data flow anomaly recognition methods based on accidental projection angular distribution
CN110839032A (en) * 2019-11-18 2020-02-25 河南牧业经济学院 Internet of things abnormal data identification method and system
CN110971488A (en) * 2019-11-27 2020-04-07 软通动力信息技术有限公司 Data processing method, device, server and storage medium
CN111414586A (en) * 2020-03-31 2020-07-14 北京派得伟业科技发展有限公司 Environment monitoring method and system
CN111599174A (en) * 2020-05-13 2020-08-28 北京百度网讯科技有限公司 Traffic index abnormity detection method and electronic equipment
JP2020170304A (en) * 2019-04-02 2020-10-15 澪標アナリティクス株式会社 Estimation device, estimation method, and program
CN115388931A (en) * 2022-10-27 2022-11-25 河北省科学院应用数学研究所 Credible monitoring method, monitoring terminal and storage medium for sensor abnormal data
CN115859209A (en) * 2023-02-08 2023-03-28 烟台市福山区动物疫病预防控制中心 Animal husbandry poultry breeding abnormity identification method based on feed consumption data

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102014031A (en) * 2010-12-31 2011-04-13 湖南神州祥网科技有限公司 Method and system for network flow anomaly detection
US20120296225A1 (en) * 2011-05-17 2012-11-22 Siemens Medical Solutions Usa, Inc. System for Cardiac Condition Detection Responsive to Blood Pressure Analysis
CN103336906A (en) * 2013-07-15 2013-10-02 哈尔滨工业大学 Sampling GPR method of continuous anomaly detection in collecting data flow of environment sensor
CN103974311A (en) * 2014-05-21 2014-08-06 哈尔滨工业大学 Condition monitoring data stream anomaly detection method based on improved gaussian process regression model

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102014031A (en) * 2010-12-31 2011-04-13 湖南神州祥网科技有限公司 Method and system for network flow anomaly detection
US20120296225A1 (en) * 2011-05-17 2012-11-22 Siemens Medical Solutions Usa, Inc. System for Cardiac Condition Detection Responsive to Blood Pressure Analysis
CN103336906A (en) * 2013-07-15 2013-10-02 哈尔滨工业大学 Sampling GPR method of continuous anomaly detection in collecting data flow of environment sensor
CN103974311A (en) * 2014-05-21 2014-08-06 哈尔滨工业大学 Condition monitoring data stream anomaly detection method based on improved gaussian process regression model

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107346301A (en) * 2016-12-02 2017-11-14 西交利物浦大学 Water quality monitoring noise data real-time detection method based on Double time window checking
CN107346301B (en) * 2016-12-02 2020-09-04 西交利物浦大学 Water quality monitoring noise data real-time detection method based on double-time-window verification
CN108304315A (en) * 2017-12-14 2018-07-20 国家计算机网络与信息安全管理中心 A kind of sorting technique and system of software aging abnormal behaviour
CN110099089A (en) * 2018-01-31 2019-08-06 Sap欧洲公司 The self-tuing on line of multiple data flows in sensor network
CN110311879A (en) * 2018-03-20 2019-10-08 重庆邮电大学 A kind of data flow anomaly recognition methods based on accidental projection angular distribution
CN110311879B (en) * 2018-03-20 2022-02-22 重庆邮电大学 Data flow abnormity identification method based on random projection angle distribution
CN109617765A (en) * 2019-01-02 2019-04-12 中国联合网络通信集团有限公司 A kind of prediction technique and device of Internet of Things quality of connection
CN109890002A (en) * 2019-01-04 2019-06-14 中国电子科技集团公司电子科学研究院 A kind of method and device of the prediction Risk Degree of Maneuvering Environment of knowledge-based inference
JP2020170304A (en) * 2019-04-02 2020-10-15 澪標アナリティクス株式会社 Estimation device, estimation method, and program
CN110839032A (en) * 2019-11-18 2020-02-25 河南牧业经济学院 Internet of things abnormal data identification method and system
CN110971488A (en) * 2019-11-27 2020-04-07 软通动力信息技术有限公司 Data processing method, device, server and storage medium
CN111414586A (en) * 2020-03-31 2020-07-14 北京派得伟业科技发展有限公司 Environment monitoring method and system
CN111599174B (en) * 2020-05-13 2021-11-16 北京百度网讯科技有限公司 Traffic index abnormity detection method and electronic equipment
CN111599174A (en) * 2020-05-13 2020-08-28 北京百度网讯科技有限公司 Traffic index abnormity detection method and electronic equipment
CN115388931A (en) * 2022-10-27 2022-11-25 河北省科学院应用数学研究所 Credible monitoring method, monitoring terminal and storage medium for sensor abnormal data
CN115388931B (en) * 2022-10-27 2023-02-03 河北省科学院应用数学研究所 Credible monitoring method, monitoring terminal and storage medium for sensor abnormal data
CN115859209A (en) * 2023-02-08 2023-03-28 烟台市福山区动物疫病预防控制中心 Animal husbandry poultry breeding abnormity identification method based on feed consumption data

Also Published As

Publication number Publication date
CN106302487B (en) 2019-08-09

Similar Documents

Publication Publication Date Title
CN106302487A (en) Agricultural Internet of Things data flow anomaly detects processing method and processing device in real time
Vega et al. Protocol for automating error removal from yield maps
US9390331B2 (en) System and method for assessing riparian habitats
Cao et al. Using Maxent to model the historic distributions of stonefly species in Illinois streams: the effects of regularization and threshold selections
Rodriguez-Perez et al. Detecting technical anomalies in high-frequency water-quality data using artificial neural networks
AU2011200949B1 (en) System and method for monitoring resources in a water utility network
US10067038B2 (en) Analyzing equipment degradation for maintaining equipment
US20140338428A1 (en) Water leak early detection system and method
CN109784528A (en) Water quality prediction method and device based on time series and support vector regression
KR20180054413A (en) Methd for optimizing crop production efficency and apparatus for the same
US20190164102A1 (en) Operational improvement effect calculation device, operational improvement effect calculation method and recording medium
KR102065435B1 (en) Infrastructure health monitoring system and method
Burgas et al. Principal component analysis for monitoring electrical consumption of academic buildings
CN109643484B (en) Method for learning up-to-date data in consideration of external influence in early warning system and system for the same
CN108875709A (en) One kind flocks together behavioral value method, apparatus, electronic equipment and storage medium
US9654997B2 (en) Method and system for evaluation of sensor observations
CN112258793B (en) Disaster monitoring and early warning method, device, equipment and medium
KR101739546B1 (en) Power data analysis system for avoiding dangerous power situations of consumer
KR20200135078A (en) System for predicting present state and future state of agriculture solar power generation structure based on rnn
Sodhi et al. Low-Cost Water and Energy Efficient Futuristic Irrigation System Using IBM Watson with Bayesian Analysis
CN112703537B (en) Techniques for managing security at an industrial site
Akbari et al. Determining Pleiades satellite data capability for tree diversity modeling
Dai et al. Evaluating the Impact of Systematic Error on the Estimation Performance of a Length‐Based Spawning Potential Ratio
CN107958334A (en) A kind of method for carrying out analytical control to electric power data for power industry
US11204333B2 (en) Method and system for automatic detection of inefficient household thermal insulation

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant