CN104391950A - Method for using hash B + tree structure to detect complex events in manufacturing Internet of Things massive data streams - Google Patents
Method for using hash B + tree structure to detect complex events in manufacturing Internet of Things massive data streams Download PDFInfo
- Publication number
- CN104391950A CN104391950A CN201410708219.7A CN201410708219A CN104391950A CN 104391950 A CN104391950 A CN 104391950A CN 201410708219 A CN201410708219 A CN 201410708219A CN 104391950 A CN104391950 A CN 104391950A
- Authority
- CN
- China
- Prior art keywords
- event
- atomic
- hash
- atomic event
- main chain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method using a hash B + tree structure to detect complex events in manufacturing Internet of Things massive data streams and aims to the solve the problems that current methods for detecting the complex events in the manufacturing Internet of Things massive data streams are long in detecting time, slow in response, low in detecting efficiency, and the like. The method has the advantages that an nondeterministic finite automata (NFA) is used in combination with the hash table B + tree technology to detect the complex events, the detecting capability of the complex events in the manufacturing Internet of Things massive data streams is increased greatly, current complex event mode detecting methods based on the NFA are improved, existing complex event detecting technologies are expanded, and detecting of the complex events in the manufacturing Internet of Things massive data streams can be completed efficiently.
Description
Technical field
The present invention relates to and manufacture thing connection data processing field, more specifically, relate to a kind of Facing to Manufacturing thing connection mass data flow Hash B+ tree construction complex events detecting methods.
Background technology
It is a kind of based on technology such as internet and embedded system, RFID and Sensor Networks for manufacturing thing connection technology, by calculating, communication, organically blending of control technology cooperate with the degree of depth, realize the real-time perception of the medium-and-large-sized complex process of manufacturing industry, Dynamic controlling and information service, thus coordinate the complicated physical process of manufacturing industry, reach process optimization and energy saving of system object technology.In modern manufacturing thing connection, owing to manufacturing production scale expanding day, manufacture process is day by day complicated, production environment is severe, a large amount of RFID, sensor node is deployed to the people that Real-Time Monitoring production scene is gone in production scene, material, equipment, technological process, product, the present situations such as service, above-mentioned RFID, sensor node creates a large amount of about " people in manufacture scene, material, equipment, critical process, product, service " etc. dynamic data, these data form mass data flow with automatic immediate mode, result in mass data processing in manufacture thing connection and be faced with following problem: the feature being difficult to process in time, add that manufacture thing connection scale is very grand, production run exists extensively spatiotemporal, there is dynamic delay in network data transmission, available data disposal route is made to be difficult to support the efficient process manufacturing thing connection mass data flow, develop a kind of efficient mass data intelligent processing method, be necessary with the development adapting to mass data processing in intelligence manufacture thing connection.
Complicated event detection technique, because it can utilize the association between event attribute, the magnanimity manufaturing data stream arrived continuously is constantly filtered by matched rule or algebraic manipulation, find out the sequence of events meeting certain interconnection constraint of needs, thus in recent years in manufacturing industry, obtain increasingly extensive concern.
At present about manufacturing the research carrying out complex events detecting methods in thing connection environment, mainly contain based on automat, based on Petri network, based on Match Tree and the detection method based on digraph method aspect.Based on the detection method of automat, mainly theoretical according to supervisory control, regular expression can be represented by automat.Based on Petri network detection method, mainly with input position node for elementary event, outgoing position node is compound event.By input token (Token), use the migration of token to describe the dynamic process of event detection, calculate transition and guard function, and cause transition and mark position node, after last the position node in sequence is labeled, detect that complicated event occurs.Based on the detection method of Match Tree, the structure mainly through Match Tree realizes complicated event and filters.Based on digraph detection method, the data structure of a main use directed acyclic graph DAG represents complicated event, carrys out presentation of events with the node of digraph, and the limit of digraph represents the composition rule of complicated event, node by marking quoting of dependent event, thus realizes detecting.But due to the restriction of itself design process object-oriented, in current manufacture thing connection environment, complex events detecting methods does not consider the real-time of mass incident stream, high density and magnanimity feature, cause there is detection time when using above-mentioned detection method to detect manufacture thing the United Nations General Assembly scale mass data flow long, reaction velocity is slower, the problems such as throughput is low, therefore cannot be adapted to detect in manufacture thing connection mass data flow.
Along with data flow technique is growing, needs can work out a kind of efficient manufacture thing that is applicable to and join complex events detecting methods in mass data flow.
Summary of the invention
When detecting the compound event manufactured on thing connection magnanimity flow of event, the present invention occurs that detection time is long mainly for above-mentioned current complex events detecting methods, response speed is slow, the problem that detection efficiency is low, propose a kind of Facing to Manufacturing thing connection mass data flow Hash B+ tree construction complex events detecting methods, use automat (NFA) to combine complex events detecting methods in Hash table B and tree construction technology realization manufacture thing connection magnanimity flow of event, the method improve the current complicated event mode detection method based on automat sequence scanning and sequence process, existing complicated event detection technique is expanded, detection thing connection mass data completing complicated event can manufactured more efficiently, substantially increase complicated event detectability in mass data flow.
To achieve these goals, technical scheme of the present invention is:
Complex events detecting methods in a kind of Facing to Manufacturing thing connection mass data flow, comprises the following steps:
A. calculate given pattern matching expression length, and according to given pattern match express generate corresponding NFA, create new Hash show and B+ set and carry out initialization;
B. from atomic event stream, carry out reading atomic event;
C. judge whether the above-mentioned atomic event read is received by NFA; If receive, turn to step D, do not receive, turn to step B;
D. utilize Hash table function to be mapped in corresponding array by this atomic event, adopt Hash table structure or mapping value corresponding to B+ storage of data structure;
E. judge whether this atomic event exists above-mentioned atomic event mapping value in hash data structure or B+ tree construction, if do not exist, the main chain node of this atomic type is then increased at the correspondence position of corresponding construction, the subchain node of this atomic type is inserted again on main chain node, upgrade the minimum time stamp that in main chain node, minimum time stamp occurs for this atomic event simultaneously, and make main chain node Counter numerical value add 1; If exist, then node inserts the subchain node of this atomic type in subchain, main chain node Counter numerical value carries out adding 1 simultaneously;
F. judge the pattern matching expression length whether counter values equals given if not, then to turn to step B; If so, step G is turned to;
G. judge that the minimum time stamp that this atomic event occurs adds whether moving window time TW is greater than now atomic event time of origin stamp, if so, then utilize Hash lookup technology or B+ tree to search technology export dependent event, obtain final complicated event testing result, if not, then step B is turned to mutually.
The present invention utilizes automat (NFA) and the synergy of Hash table B+ tree construction, realize a kind of Facing to Manufacturing thing connection mass data flow complex events detecting methods, overcoming present method, to there is detection time when detecting event on mass data flow long, response speed is slow, the problem of the low grade of detection efficiency, extend existing complicated event detection technique, improve the detectability of compound event on magnanimity flow of event based on automat NFA.
The present invention utilizes Hash table and B+ tree construction Counter numerical value whether to equal given pattern matching expression length and whether minimum time stamp+TW (moving window time) that on major key, atomic event occurs is greater than now atomic event time of origin stamp jointly judges whether complicated event detection completes.Only have meet simultaneously counter values equal minimum time stamp+moving window time TW that on given pattern matching expression length and this sub-key, atomic event occurs be greater than now atomic event time of origin stamp time, Hash table or B+ just can be utilized to set and to search technology export final detection result.
When completing complicated event and detecting, Hash table lookup technique and B+ tree technology of searching is utilized to search huge intermediate result in storage Hash table structure and B+ tree respectively, export corresponding complicated event testing result, accelerate complicated event seek rate, reduce the consumption of time of searching, thus improve complicated event detectability and processing speed on manufacture thing connection mass data flow.
Only have meet simultaneously counter values equal minimum time stamp+moving window time TW that on given pattern matching expression length and this sub-key, atomic event occurs be greater than now atomic event time of origin stamp time, Hash table or B+ just can be utilized to set and to search technology export final detection result.Set up corresponding XFA according to pattern matching expression, and calculate given pattern matching expression length;
Further, Hash table structure or B+ tree construction is selected to store according to Hash table Function Mapping end value, namely described step D utilizes Hash table function to be mapped to after in corresponding array by this atomic event, judge that whether mapping value is unique, if unique, then use Hash table structure to store it, otherwise use B+ tree construction to store it, achieve and utilize Hash table B+ tree construction to carry out quick storage object to the huge intermediate result that magnanimity flow of event produces during detection of complex event.
Further, in described step e, main chain node comprises this atomic event type, the type atom minimum time stamp sum counter, and subchain node comprises this atomic event type and this atom time of origin stamp.
In above-mentioned steps E, need at hash data structure or B+ tree construction, whether this atomic event item is existed to this atomic event and carry out different searching, insert and process operation.
Compared with prior art, the invention has the beneficial effects as follows: a kind of Facing to Manufacturing thing connection mass data flow Hash B+ tree construction complex events detecting methods, have employed the method combined based on automat (NFA) and Hash table B+ tree technology jointly to go to detect complicated event in magnanimity flow of event, overcome now current complex events detecting methods and occur that detection time is long when detecting and manufacturing complicated event on thing connection magnanimity flow of event, response speed is slow, the shortcomings such as detection efficiency is low, the detectability of the event that substantially increases on mass data flow.Present invention improves over the current complicated event mode detection method based on automat, existing complicated event detection technique is expanded, the detection of complicated event can be completed more efficiently in mass data.
Accompanying drawing explanation
Fig. 1 is method anabolic process figure of the present invention.
Fig. 2 is method principle of work schematic diagram of the present invention.
Fig. 3 is that the inventive method compares schematic diagram with SASE method in the time of searching.
Fig. 4 is that the inventive method compares schematic diagram with SASE method in response speed consumption
Fig. 5 is that the inventive method compares schematic diagram with SASE method in handling capacity.
Embodiment
Accompanying drawing, only for exemplary illustration, can not be interpreted as the restriction to this patent;
In order to better the present embodiment is described, some parts of accompanying drawing have omission, zoom in or out, and do not represent the size of actual product;
To those skilled in the art, in accompanying drawing, some known features and explanation thereof may be omitted is understandable.
Embodiment 1
The concrete matching process of the present embodiment to a kind of Facing to Manufacturing thing connection mass data flow Hash B+ tree construction complex events detecting methods is described in detail.In this example, utilize data generator module to go to produce flow of event, generated the number of event type by control data generator module parameter, the probability distribution etc. of flow of event, to realize the demand that the present embodiment desired parameters controls.The present embodiment outfit: Visual C++6.0, test index is: search the time, response speed and handling capacity three aspect, and experiment comparative approach is: SASE method.
The anabolic process figure of the present embodiment as shown in Figure 1, it contains: read atomic event from magnanimity atom pieces stream, non-determined finte-state machine (NFA) matched atoms event, utilizes Hash table B+ storage of data structure relevant atomic event and utilizes Hash table B+ to set the technology of searching and search the large funtion part of dependent event four.
When supposing that pattern matching expression is SEQ (A, B, C), its concrete testing process principle of work as shown in Figure 2, is concluded its specific implementation step and can be divided into following a few step:
(1) length calculating given pattern matching expression SEQ (A, B, C) is 3, and express SEQ (A according to pattern match, B, C) generate corresponding NFA (see Fig. 2), create new Hash and show and initialization operation is carried out to it;
(2) from atomic event stream (see Fig. 2), carry out reading atomic event operation a
f;
(3) atomic event a
fcan be received by NFA; Step (4) is turned to perform;
(4) this atomic event of Hash table Function Mapping a is utilized
fbe mapped in corresponding array, due to a
fmapping value is not unique, uses B+ tree construction to store it;
(5) due to this atomic event a
fdo not exist at B+ tree construction, the main chain node that then utilizing B+ to set insertion method increases this atomic type at this structure correspondence position (comprises this atomic event type, the type atom minimum time stamp sum counter), the subchain node (comprising this atomic event type and this atom time of origin stamp) of this atomic type is inserted again on main chain node, upgrade the minimum time stamp that in main chain node, minimum time stamp occurs for this atomic event simultaneously, and make main chain node Counter numerical value add 1; If exist, then node inserts the subchain node of this atomic type in subchain, main chain node Counter numerical value carries out adding 1 simultaneously;
(6) because main chain node Counter numerical value is 1, be not equal to given pattern matching expression length 3, then turn to step (2) to proceed to detect; Next atomic event operation b is read from atomic event stream
f, then carry out and atomic event a
fcarry out identical testing;
(7) when detection procedure is to c
ftime, because main chain node Counter numerical value 3 equals the length 3 of given pattern matching expression SEQ (A, B, C), and atomic event a on this major key
fminimum time stamp+moving window time the TW occurred is greater than now atomic event c
ftime of origin stabs, so utilize B+ to set the technology of searching search event in storage B+ tree construction, searches and output detections result a
fb
fc
f.Following a
q, b
q, c
qdetect Deng atomic event, detection method and above-mentioned similar.
As detection a
p, b
p, c
pdetect Deng atomic event, detection method is substantially identical with above-mentioned detection, and difference is because its mapping value is unique, needing to use Hash table structure to store, when inserting atomic event operation, needing the insertion function calling Hash table; And when exporting complicated event, need to use the locating function of Hash table and exportable final detection result a
fb
fc
f.
Fig. 3 adopts the inventive method in detection time, to compare schematic diagram with existing SASE method.As seen from Figure 3, under same test condition, compare SASE method, method proposed by the invention can greatly reduce the event detection time.Analyze when its main cause is that the inventive method uses Hash table structure and B+ storage of data structure and searched events stream to detect in testing process and produce intermediate result, save in SASE method when utilizing dynamic instance storehouse store and search and there is a large amount of back tracking operations, thus save it and search the time.
Fig. 4 adopts the inventive method in response speed consumption, to compare schematic diagram with existing SASE method.As seen from Figure 4, under same test condition, compare SASE method, method proposed by the invention can greatly improve event detection response speed.Analyze its main cause and be that the inventive method goes store detection intermediate result and utilize Hash table lookup technique to go to search dependent event in use Hash table structure and B+ tree construction, save SASE method and there is a large amount of back tracking operations when utilizing dynamic instance storehouse store and search, thus improve Whole Response speed.
Fig. 5 adopts the inventive method in handling capacity, to compare schematic diagram with existing SASE method.As seen from Figure 5, under same test condition, compare SASE method, method proposed by the invention can improve event handling amount.Analyze its main cause and be that the inventive method uses Hash table B+ storage of data structure to detect intermediate result and utilizes the Hash table B+ tree technology of searching to go to search dependent event, the quick insertion realizing dependent event stores and fast finding operation, the storage of minimizing event and the elapsed time searched, and then improve entire system processing speed.
The corresponding same or analogous parts of same or analogous label;
Describe in accompanying drawing position relationship for only for exemplary illustration, the restriction to this patent can not be interpreted as;
Obviously, the above embodiment of the present invention is only for example of the present invention is clearly described, and is not the restriction to embodiments of the present invention.For those of ordinary skill in the field, can also make other changes in different forms on the basis of the above description.Here exhaustive without the need to also giving all embodiments.All any amendments done within the spirit and principles in the present invention, equivalent to replace and improvement etc., within the protection domain that all should be included in the claims in the present invention.
Claims (3)
1. a Facing to Manufacturing thing connection mass data flow Hash B+ tree construction complex events detecting methods, is characterized in that, comprise the following steps:
A. calculate given pattern matching expression length, according to given pattern match express generate corresponding NFA, create new Hash show and B+ set and carry out initialization;
B. from atomic event stream, carry out reading atomic event;
C. judge whether the above-mentioned atomic event read is received by NFA; If receive, turn to step D, do not receive, turn to step B;
D. utilize Hash table function to be mapped in corresponding array by this atomic event, adopt Hash table structure or mapping value corresponding to B+ storage of data structure;
E. judge whether this atomic event exists the mapping value of this atomic event in hash data structure or B+ tree construction, if do not exist, the main chain node of this atomic type is then increased at the correspondence position of corresponding construction, the subchain node of this atomic type is inserted again on main chain node, upgrade the minimum time stamp that in main chain node, minimum time stamp occurs for this atomic event simultaneously, and make main chain node Counter numerical value add 1; If exist, then node inserts the subchain node of this atomic type in subchain, main chain node Counter numerical value carries out adding 1 simultaneously;
F. judge the pattern matching expression length whether counter values equals given if not, then to turn to step B; If so, step G is turned to;
G. judge whether the minimum time stamp+moving window time TW that this atomic event occurs is greater than now atomic event time of origin stamp, if so, then utilize Hash lookup technology or B+ tree to search technology export dependent event, obtain final complicated event testing result, if not, then step B is turned to mutually.
2. Facing to Manufacturing thing connection mass data flow Hash B+ tree construction complex events detecting methods according to claim 1, it is characterized in that, described step D utilizes Hash table function to be mapped to after in corresponding array by this atomic event, judge that whether mapping value is unique, if unique, then use Hash table structure to store it, otherwise use B+ tree construction to store it.
3. Facing to Manufacturing thing connection mass data flow Hash B+ tree construction complex events detecting methods according to claim 1, it is characterized in that, in described step e, main chain node comprises this atomic event type, the type atom minimum time stamp sum counter, subchain node comprises this atomic event type and this atom time of origin stamp.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410708219.7A CN104391950A (en) | 2014-11-28 | 2014-11-28 | Method for using hash B + tree structure to detect complex events in manufacturing Internet of Things massive data streams |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410708219.7A CN104391950A (en) | 2014-11-28 | 2014-11-28 | Method for using hash B + tree structure to detect complex events in manufacturing Internet of Things massive data streams |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104391950A true CN104391950A (en) | 2015-03-04 |
Family
ID=52609854
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410708219.7A Pending CN104391950A (en) | 2014-11-28 | 2014-11-28 | Method for using hash B + tree structure to detect complex events in manufacturing Internet of Things massive data streams |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104391950A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106779316A (en) * | 2016-11-25 | 2017-05-31 | 中国电子科技集团公司第三十八研究所 | A kind of radar electric equipment manufacturing things system |
| CN108038264A (en) * | 2017-11-15 | 2018-05-15 | 华南农业大学 | A kind of modeling method for the complicated event detection model shared based on pattern |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070208680A1 (en) * | 2005-10-05 | 2007-09-06 | Siemens Corporate Research Inc | Method and Apparatus For Complex RFID Event Processing |
| CN101883098A (en) * | 2010-06-18 | 2010-11-10 | 大连海事大学 | System and method for distributed complex event detection under RFID (Radio Frequency Identification Devices) equipment network environment |
| CN102508640A (en) * | 2011-10-27 | 2012-06-20 | 西北工业大学 | Distributed radio frequency identification device (RFID) complex event detection method based on task decomposition |
-
2014
- 2014-11-28 CN CN201410708219.7A patent/CN104391950A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070208680A1 (en) * | 2005-10-05 | 2007-09-06 | Siemens Corporate Research Inc | Method and Apparatus For Complex RFID Event Processing |
| CN101883098A (en) * | 2010-06-18 | 2010-11-10 | 大连海事大学 | System and method for distributed complex event detection under RFID (Radio Frequency Identification Devices) equipment network environment |
| CN102508640A (en) * | 2011-10-27 | 2012-06-20 | 西北工业大学 | Distributed radio frequency identification device (RFID) complex event detection method based on task decomposition |
Non-Patent Citations (2)
| Title |
|---|
| 孟培超等: "基于Hash B+树RFID复杂事件检测算法", 《贵州师范大学学报（自然科学版）》 * |
| 王磊: "事件流上复杂事件检测技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106779316A (en) * | 2016-11-25 | 2017-05-31 | 中国电子科技集团公司第三十八研究所 | A kind of radar electric equipment manufacturing things system |
| CN108038264A (en) * | 2017-11-15 | 2018-05-15 | 华南农业大学 | A kind of modeling method for the complicated event detection model shared based on pattern |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102915347B (en) | A kind of distributed traffic clustering method and system | |
| CN103676645B (en) | A kind of method for digging of the correlation rule in time series data stream | |
| CN104346481B (en) | A kind of community detection method based on dynamic synchronization model | |
| CN103678671A (en) | Dynamic community detection method in social network | |
| CN101667197A (en) | Mining method of data stream association rules based on sliding window | |
| CN101419630B (en) | Top-k item digging method and system in data flow | |
| CN105515997B (en) | The higher efficiency range matching process of zero scope expansion is realized based on BF_TCAM | |
| CN102426525A (en) | Panoramic modeling method of multi-application system | |
| CN107832916B (en) | Identification method for critical risk factors and critical risk transmission path of cascade hydropower station based on Bayesian risk network | |
| CN103345496A (en) | Multimedia information searching method and system | |
| CN102169491A (en) | Dynamic detection method for multi-data concentrated and repeated records | |
| CN109582714A (en) | A kind of government affairs item data processing method based on time fading correlation | |
| CN102207935A (en) | Method and system for establishing index | |
| CN104391950A (en) | Method for using hash B + tree structure to detect complex events in manufacturing Internet of Things massive data streams | |
| CN104700311A (en) | Method for discovering neighborhood following community in social network | |
| CN104462095B (en) | A kind of extracting method and device of query statement common portion | |
| CN104834709A (en) | Parallel cosine mode mining method based on load balancing | |
| CN104361058A (en) | Hash structure complex event detection method for mass data flow | |
| CN103927325A (en) | URL (uniform resource locator) classifying method and device | |
| CN102855278B (en) | A kind of emulation mode and system | |
| CN102737134B (en) | Query processing method being suitable for large-scale real-time data stream | |
| CN105354264B (en) | A kind of quick adding method of theme label based on local sensitivity Hash | |
| CN104134112A (en) | Business process model consistency measurement method under semantic constraints | |
| CN108052587B (en) | Big data analysis method based on decision tree | |
| CN104408142A (en) | Detection method for complex events in mass disordered data streams of Internet of Things Manufacturing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150304 |