CN112653749A - Edge computing-based complex event processing system and method for Internet of things - Google Patents

Edge computing-based complex event processing system and method for Internet of things Download PDF

Info

Publication number
CN112653749A
CN112653749A CN202011498669.XA CN202011498669A CN112653749A CN 112653749 A CN112653749 A CN 112653749A CN 202011498669 A CN202011498669 A CN 202011498669A CN 112653749 A CN112653749 A CN 112653749A
Authority
CN
China
Prior art keywords
event
complex
original
internet
things
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011498669.XA
Other languages
Chinese (zh)
Inventor
左敏
尼加提·艾合买提
张青川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Technology and Business University
Original Assignee
Beijing Technology and Business University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Technology and Business University filed Critical Beijing Technology and Business University
Priority to CN202011498669.XA priority Critical patent/CN112653749A/en
Publication of CN112653749A publication Critical patent/CN112653749A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/12Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60Protecting data
    • G06F21/62Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245Protecting personal data, e.g. for financial or medical purposes
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/14Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network

Abstract

The invention relates to a complex event processing system and method based on edge calculation for the Internet of things, which comprises an original event sensing module, an original event processing module and a complex event processing module, wherein the original event sensing module is used for sensing the original event; the original event sensing module acquires an original event from the terminal sensing equipment of the Internet of things and transmits the original event to the original event processing module; the original event processing module preprocesses an original sensing event and then puts the event into an original event library of an edge computing server or transmits the event to an event analyzer according to a preset rule; the complex event processing module takes the simple events from the original event processing module and further performs event pattern detection, event filtering, event abstraction, event hierarchy modeling, event aggregation and conversion and relationship detection between the events through an event parser to reason out more complex and higher-level compound events. The invention can respond to the change of the requirement of the cloud server in real time and process the complex event matched with the edge end in real time, thereby allowing the change of the production plan in the production process and maximizing the production efficiency.

Description

Edge computing-based complex event processing system and method for Internet of things
Technical Field
The invention relates to the field of computer software, in particular to a complex event processing system and method based on edge computing for the Internet of things.
Background
Generally, the internet of things terminal sensing device network lacks sufficient hardware resources to analyze and visualize data collected by the internet of things terminal sensing device. Such internet of things terminal aware device networks are commonly used to acquire data and forward it to a more powerful device where the data can be processed correctly. These devices may be used to monitor, visualize, and perform statistical analysis of instance servers. One challenge with current use of smart internet of things terminal aware device networks is communication with cloud services. Because the embedded system is applied to the internet of things terminal sensing equipment network, there are usually not enough hardware resources to monitor, analyze and visualize data. Furthermore, communication is challenging, as network latency is a limiting factor, especially in real-time applications. To address the bottleneck and privacy security issues of cloud computing, event stream processing is pushed down to devices that are currently capable of computing and communicating at the edge of the network. Based on the current situation, an edge calculation method is provided as a supplement of the terminal sensing equipment network of the Internet of things. By using the edge nodes, some compute-intensive tasks can be closer to the internet of things terminal sensing equipment network, so that data preprocessing is realized.
The edge device is located close to the terminal device, reducing network latency and enabling real-time applications. The edge layer serves as a middle layer and can communicate with the cloud server. One possible scenario is to aggregate and store data generated by the wireless internet of things terminal-aware device network on one edge device. The edge device may be used as a next step to transmit data to the cloud service, and resources of the cloud service, such as storage, computing power, etc., may be used to perform more resource intensive tasks. A potential task may be, for example, filtering large data sets or applying machine learning methods.
A large number of Internet of things terminal sensing devices and intelligent objects deployed in the Internet of things (IoT) provide great potential for real-time monitoring application of the Internet of things, and can detect and react to the real world. The data real-time processing capability of the internet of things is insufficient, and the development of the real-time monitoring application of the internet of things is hindered. The monitoring data volume of the Internet of things is increased rapidly, and the Internet of things monitoring data volume has obvious big data characteristics. The internet of things monitoring system needs to continuously and timely process data when data streams arrive so as to support real-time performance, improve insight capability and decision capability and support complex business logic. Two problems concerning the real-time processing of the data of the internet of things are concerned: 1) how to efficiently translate large amounts of raw perceptual data into meaningful complex events, 2) how to adapt to the complexity and variability of monitoring business logic. A common Complex Event Processing (CEP) system and method for real-time processing of the Internet of things are provided.
Complex Event Processing (CEP) is the primary technique for extracting information. CEP is a data processing technique based on a set of predefined rules that specify how data streams should be processed and which new event streams should be generated as output. The event may be a number of independent events of interest (e.g., above normal temperature) or complex events (e.g., sustained high temperature) that correspond to a particular situation or pattern of traffic.
The invention provides an Internet of things perception event hierarchical model comprising an original event, a simple event and a complex event, and complexity of Internet of things perception event modeling is reduced. The model supports complex time and space semantics, carries out modeling decomposition on the Internet of things events through an event modeling language and defines the complex events. Based on the model, a general complex event processing system architecture based on edge computing is provided, and the architecture deploys the system at the network edge of the original data generation position of the internet of things between the terminal sensing equipment and the cloud application program. And mapping the complex event definition to a CEP rule logic script to detect potential abnormal events in time. The proposed CEP mechanism is generic and applicable to any internet of things and CEP engine.
Disclosure of Invention
The invention solves the problems: the system and the method overcome the defects of the prior art, and provide a complex event processing system and a complex event processing method based on edge calculation for the Internet of things, wherein a real-time processing model meeting the low-delay and high-throughput demand scene is designed for the low-delay and high-throughput demand scene based on the mobile edge calculation technology, and the real-time data stream is preprocessed through the complex event processing technology, so that the purposes of solving the service problem and improving the performance of an edge end are achieved; meanwhile, the simplified complex events are simplified through the event level model, and on the premise of guaranteeing privacy safety and low delay, the information processing capacity of the edge nodes is greatly improved so as to realize high throughput and high concurrence.
The technical scheme of the invention is as follows: an edge computing-based complex event processing system for the internet of things, comprising: the system comprises an original event sensing module, an original event processing module and a complex event processing module;
the original event perception module: the complex event processing system for the Internet of things is deployed at the edge of a network close to the original data generation position of the Internet of things, and is called edge calculation; sensing an original event from the terminal of the Internet of things according to a predefined rule and transmitting the original event to an original event processing module of an edge computing server, wherein the original event is an atomic event which is not processed and can not be decomposed and is generated from a data source of the terminal of the Internet of things at the beginning;
a primitive event processing module: the system comprises a data preprocessing module and an event analyzer; the data preprocessing module of the edge computing server filters the repeated irrelevant data and writes the repeated irrelevant data into an original event library of the edge computing server, if the original event is a point-time event, the event is handed to a complex event processing module of the edge computing server, if the original event is a duration event, the event is firstly stored in the original event library of the edge computing server and is sent to the complex event processing module of the edge computing server when needed; the event analyzer reads an original event from an original event library of the edge computing server in real time, and filters and combines the original event according to a predefined matching rule to obtain a simple event;
a complex event processing module: according to the simple event obtained by the original event processing module of the edge computing server, event mode detection, event filtering, event abstraction, event hierarchical structure modeling, event aggregation and conversion and relationship detection among events are further carried out by an event analyzer in the complex event processing module of the edge computing server so as to obtain the complex event; matching the complex event stream with a rule execution mode in a rule base, if matching is successful, calculating and generating more complex and higher complex events and sending the complex events to a complex event processing engine; the complex events are events describing a series of events to occur according to business logic rules of the internet of things compiled in advance; in the complex event processing module, writing matching rules into a business logic script by using an Event Processing Language (EPL) and storing the business logic script into an edge computing server rule base, matching the complex events by a complex event processing engine according to the rules in the edge computing server rule base, wherein the relationship among the events of the Internet of things comprises events, space, dependence and cause and effect; determining whether to store the edge computing server complex event library or transmit the edge computing server complex event library to a specific subscription cloud server according to the business requirements of the Internet of things; in the application of the intelligent logistics based on the temperature and humidity sensor, based on the matching rule, the detection values of the temperature and humidity sensor are compared, and if the matching is successful, the result is transmitted to the subscribed cloud server and a warning is sent out.
In the complex event processing module, an internet of things complex event matching rule is represented by a directed acyclic graph, which is also called an event detection graph, namely EDG (Op, X); the nodes of the graph are original events or complex events of the Internet of things, the events are placed on leaf nodes, and an event detection graph is generated by connecting the events and the operation characters through edges; the event detection method comprises the steps that | Op | nodes are provided, EDG is an event detection graph, Op is an operator between events of the Internet of things, and X is the number of edges of the event detection graph, namely an operand between the events of the Internet of things; opi ∈ { SEQ, AND, OR } OR Opi is an aggregation operator, AND in the expression, Opi is any value of the IOT event relation operator in the Op retrievable set; there is an | X | side between operations that receive input from each other (Opi → Opj); for the purpose of multi-match optimization, the output operator of the event detection graph EDG is connected to the top layer or the operator; the operation is carried out, namely an AND operator outputs a complex event when all events participating in the Internet of things occur, when all events participating in the Internet of things occur according to a specified time sequence, a sequence operation, namely an SEQ operator outputs a complex event, OR an OR operator outputs a complex event when any event participating in the Internet of things occurs, an aggregation operator receives a frequency parameter specifying the running time, if the operator receiving frequency parameter among the events of the Internet of things is 1, the aggregation operator is run when a new input event is received each time AND is used together with a specified threshold value; then, when the performed aggregation exceeds or falls below the threshold, a complex event is generated.
The invention discloses a complex event processing method based on edge calculation, which comprises the following steps:
original event perception step: acquiring an original event from the terminal sensing equipment of the Internet of things in real time according to a service requirement; generating a tuple for each primitive event; the tuple is packaged according to the format requirement of the IOT service and is transmitted to an original event processing module in an edge computing server close to the IOT sensing equipment according to an IOT transmission protocol required by the IOT service in a Bluetooth or Wi-Fi connection mode and the like;
and (3) original event processing: the method comprises the following steps that a plurality of sockets receive input data streams from different internet of things terminal sensing devices at the same time, deserialize the streams and analyze the corresponding streams into JavaScript object symbol tuples; in addition, the raw event arrival time is appended to each tuple, which is placed in an edge compute server thread safety blocking queue that processes the data in a first-in-first-out order; the edge computing server preprocessing module filters repeated events of irrelevant internet of things and performs a series of preprocessing according to the request data in the edge computing server thread safety blocking queue transmitted from the internet of things original event sensing module according to the business requirements of the internet of things, forms original events and writes the original events into an edge computing server original event library; if the original event is a point-time event, namely an event with high real-time requirement and needing immediate processing, the event analyzer immediately sends the point-time event to a complex event processing module of the edge computing server so as to ensure the real-time processing performance; for the duration event, the event parser acquires the event from the original event library of the edge computing server and processes the related original event to generate the duration event and sends the duration event to the complex event processing module; the event analyzer reads original events from an original event library of the edge computing server in real time, and filters and combines the original events according to a predefined matching rule algorithm to obtain simple events;
complex event processing steps: the simple event enters an event analyzer for preprocessing and then forms a complex event; matching rules in the rule base are analyzed into an event detection graph; carrying out pattern matching on the event stream of the Internet of things and the rules in the rule base; if the matching is successful, determining whether an edge computing server complex event library needs to be stored or transmitted to a specifically subscribed cloud server according to the service requirement of the Internet of things; if the matching is unsuccessful, storing the complex event into an edge computing server complex event library; each event stream is associated with a separate thread security blocking queue, a producer-consumer queue mode is used, namely, producers and consumers do not directly communicate with each other but communicate through the blocking queue, a complex event processing engine serves as a producer, and an edge computing server complex event library or a cloud server can serve as a consumer of a certain event stream; if the data is to be sent to the cloud server, the edge computing device communicates with the cloud server through Wi-Fi or a cellular network.
Compared with the prior art, the invention has the advantages that:
(1) the internet of things terminal sensing equipment network is communicated with the cloud server, high bandwidth and energy consumption are needed, and instantaneity and privacy safety cannot be guaranteed. The invention integrates network, calculation, storage and application core capabilities at the network edge side close to the Internet of things sensing equipment through edge calculation, and has the characteristics of high safety, low delay, low bandwidth, low energy consumption and the like.
(2) The traditional data processing of the internet of things system generally adopts a relational database of a cloud server. Data needs to be stored and indexed prior to processing, which causes significant delays and lacks the ability to collect, process, and analyze large data in real time. The original sensing data is stored and processed on the CEP system which is located at the edge of the Internet of things and is based on edge calculation, so that the communication overhead of network transmission can be reduced, the throughput rate of the cloud server of the Internet of things is improved, and the performance of the whole real-time system of the Internet of things is improved. In addition, the raw data carries various sensitive information. After the edge CEP system processes the data, the service data or the complex event which does not contain sensitive information is sent to the cloud server in the form of the complex event, and privacy protection is facilitated.
Drawings
FIG. 1 is a system architecture of the present invention;
FIG. 2 is a diagram of a hierarchical model of events in the present invention;
FIG. 3 is an event detection diagram in the present invention;
FIG. 4 is a simple event class definition diagram of the Internet of things of the present invention;
FIG. 5 is a state diagram of the NFA of the present invention.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and examples.
As shown in fig. 1, the system of the present invention comprises: the system comprises an original event sensing module, an original event processing module and a complex event processing module; the original event sensing module acquires an original event from the terminal sensing equipment of the Internet of things according to a predefined rule and transmits the original event to the original event processing module; the original event processing module preprocesses an original sensing event and then puts the event into an original event library of an edge computing server or transmits the event to an event analyzer according to a preset rule; the complex event processing module takes the simple events from the original event processing module and further performs event pattern detection, event filtering, event abstraction, event hierarchy modeling, event aggregation and conversion and relationship detection between the events through an event parser to obtain more complex and higher-level complex events.
Each module will be described in detail below.
1. Primitive event awareness module
The whole original event perception module is mainly used for acquiring the perception events of the terminal of the Internet of things in real time, and comprises the steps of data perception, data encapsulation and data transmission of the terminal of the Internet of things, which are introduced one by one.
(1) Data awareness
And the terminal sensing equipment of the Internet of things acquires the required original event in real time.
(2) Data encapsulation
And encapsulating the perceived primitive events according to the requirements of the edge server, and generating a tuple for each primitive event.
(3) Data transmission
And packaging the tuples according to the format requirement of the service of the Internet of things, and transmitting the transmission protocol of the Internet of things required by the service of the Internet of things to an original event processing module in an edge computing server close to the sensing equipment of the Internet of things in a Bluetooth or Wi-Fi connection mode and the like.
2. Primitive event processing module
The module is mainly used for preprocessing data of the original event perception module to generate an original event, transmitting the original event to an event analyzer or storing the original event in an edge server original event library, and finally filtering and combining the original event to obtain a simple event.
2.1 event level model
The event of the internet of things is divided into an original event, a simple event and a complex event according to granularity. The internet of things aware event processing is shown in fig. 2. Fig. 2 shows the entire process from the generation of the internet of things event to the consumption of the internet of things event. The original event is generated by the terminal sensing equipment of the Internet of things. The original events are filtered and combined to generate simple events. Complex events are generated by event pattern matching of simple events. The complex event is sent to the cloud server for event consumption. And the complex event processing is positioned between the terminal sensing equipment of the Internet of things and the cloud server.
Both simple events and complex events are calculated from small granularity events. The difference between the simple events is that the calculation rule of the simple events is universal, the granularity is moderate, and the reusability is good. And the calculation rule of the complex event is customized and determined according to the actual business requirement.
2.2 event preprocessing
The data preprocessing module of the edge computing server filters the repeated irrelevant data and writes the repeated irrelevant data into an original event library of the edge computing server;
the primitive event is an atomic event that originally generated an unprocessed event from the terminal data source of the internet of things and cannot be decomposed. The atomic event e is represented as:
e=(ID,DeviceID,DeviceType,Location,StartTime,EndTime,KeyValue List);
the ID is a unique identifier of the atomic event, which is a serial number. DeviceID is the unique ID of the perceiving device. DeviceType indicates the type of the terminal sensing device of the Internet of things. Location refers to the Location of the terminal sensing device of the internet of things. The KeyValue list is a list composed of key-value pairs, represents a plurality of attributes and values thereof, is extensible, and represents a temperature value of the temperature sensor, and < humidity > represents a humidity value of the humidity sensor of the internet of things. StartTime and EndTime are the start time and end time of an internet of things event. The internet of things point-in-time event satisfies StartTime ═ EndTime. Taking an original event as an example, a Temperature and Humidity sensor ' DS000580 ' located in ' henhouse 5 area ' detects a Temperature of 20 ℃ and a Humidity of 35% at 14.10.2018 at 9:30, which are expressed as e ═ 0001 ', ' DS000580 ', ' area 5 ', 2020-10-89: 30,2020-10-89:31, < Temperature,20>, < humity, 35% >.
If the original event is a point-time event, namely the real-time requirement is high, the event analyzer immediately sends the point-time event to a complex event processing module of the edge computing server so as to ensure the performance of real-time processing;
if the event is the duration event, the event parser acquires the event from the original event library of the edge computing server and processes the related original event to generate the duration event and sends the duration event to the complex event processing module;
if StartTime ≠ EndTime, the IOT event is not a point-time event but a persistent-time event. By analyzing the original event, the duration event may not be the original event, such as a stay event of an RFID tag. The duration event may also be an original event, such as a camera video data event, in which the start time of the video and the end time of the video are recorded in the video event of a duration.
For complex events, Starttime is the minimum start time of all atomic events that make up the complex event, and the end time is the maximum end time of all atomic events that make up the complex event.
For convenience of description, the primitive event may be simply represented as primitive event RAE (d, l, t) to represent the instantaneous temporal and spatial state of a particular device, d refers to the sensing means, l refers to the location-space sensing device, and t refers to when the event occurs; the above raw events are simplified as RAE ═ ("001", "area 5", "2020-10-89: 20"), meaning the temperature and humidity at which the 001 sensor of area No. 5 detected 2020-10-89: 20.
The event analyzer reads an original event from an original event library of the edge computing server in real time, and filters and combines the original event according to a predefined matching rule to obtain a simple event;
2.3 simple event handling
Simple events are introduced into the CEP model. Before the CEP engine, simple event processing is carried out, such as filtering out repeated events, combining and acquiring set events and the like, so that the processing load of the CEP engine can be reduced, and the overall processing performance of the system can be effectively improved.
A simple event is extracted from the original event. The processing process is as follows:
(1) filtering irrelevant events, i.e. raw perceptual events that are not of interest to the application, may be ignored.
(2) The repeated events are filtered. The spatial repetitive events generated by a plurality of terminal sensing devices covering the same area of the Internet of things can be filtered. For example, a single event OE (d, l, t) describing the real-time spatial state of the perceiving device d at time t in the monitoring area l. Most OEs may be useless information for some removable devices, such as RFID tags.
(3) Filtering to obtain important events. For example, for an RFID tag, if it stays in a certain area for a long time, there will be a set of primitive events RAE (d, l, t), but the application usually only concerns the entering area and the leaving area of the tag at two times, i.e. the entering event AE (d, l, t) and the exiting event DE (d, l, t) can filter out important events from the set of primitive events RAE (d, l, t).
In the above expression, AE (d, l, t) indicates that the sensing device d appears in the monitoring area l at time t, and DE (d, l, t) indicates that the sensing device d disappears in the monitoring area l at time t.
SE (d, l, ts, te) describes that device d stays in the monitoring area from ts time to te time. SE (d, l, ts, te) is detected by matching neighboring AE (d, l, t) and DE (d, l, t). SE (d, l, ts, te) is often used, and time reasoning of CEP wastes unnecessary computational cost, so SE (d, l, ts, te) is classified as a simple event.
SE ═ ("001", "area 5", "2020-10-89: 20", "2020-10-89: 30"), describing device # 001 staying in the chicken house monitoring area 5 to 9:30 from 9: 20.
Similarly, the reproduction event RPE (d, l, ts, te) describes that the device d reproduces in the area l, and ts and te are the time when the event of the internet of things disappears and the time when the event of the internet of things reproduces. RPE (d, l, ts, te) is detected by matching adjacent DE (d, l, t) and AE (d, l, t).
According to the definition of the point-time event and the duration event of the internet of things, OE (d, l, t), AE (d, l, t) and DE (d, l, t) are point events, SE (d, l, ts, te) and RPE (d, l, ts, te) are duration events.
The aggregate event is obtained by combining. Many applications focus on the number or set of specific types of internet of things terminal aware device events in a particular area at a particular time, ignoring each individual data event. For example, the number of RFID tags that reach a particular area at a particular time, and the set of RFID tags may be extracted by combination. Event CE Collection (E, l, t)(e.devicetype ═ RFID tag')Where E is the device set, e.size is the number of devices in the set, l refers to the area, t refers to the time of the internet of things event, and the subscript expression defines the attribute constraint, e.g., (e.g., "RFID tag") defines the constraint that the device class is an RFID tag. Taking the above RFID event as an example, events that may occur during the whole device moving process are described. The simple events can represent wider terminal sensing equipment of the Internet of thingsEvents, with the event attributes extended, data attributes are added to the KeyValue list shown in the table.
Simple event definition is common to most internet of things aware devices, and a simple event processing algorithm is as follows:
(1) inputting a raw event RAE (d, l, t);
(2) if d is a device that cannot be moved, then OE ═ RAE (d, l, t), output OE (d, l, t);
(3) if RAE (d, l, t) is an incoming event AE (d, l, t), then AE (d, l, t) is output and an ending event DE (d, l, t) is matched, if matching is successful, then a recurring event RPE (d, l, ts, te) is output; if the matching fails, the device d information is put into a collection event CE (E, l, t);
(4) if RAE (d, l, t) is an exit event DE (d, l, t), output DE (d, l, t) and match the start event AE (d, l, t), if the match is successful, output a simple event SE (d, l, ts, te); if the matching fails, the d device information is eliminated from the event set CE (E, l, t);
in the above algorithm, the reproduced RAE, i.e., the reproduced AE or DE, is filtered and not transmitted to the CEP module, thereby reducing unnecessary load of the CEP module.
The event parser filters and combines the original events from the original event library of the edge computing server in real time according to an event processing algorithm to obtain simple events, including 1) point-time events such as an entry event AE, an exit event DE and a single event OE. 2) A duration event. Upon AE and DE matching, the acquisition of event SE and the recurrence of event RPE are stopped. 3) The collection event CE is obtained from the statistics of AE and DE.
The event parser outputs all simple events as Java objects to the complex event processing module. UMLl class diagram, as shown in fig. 4. The departure event and the single event OccurenameEvent are generalized into point-time events by, for example, an entry event AdditioneEvent; the point-time event StopEvent and the recurring event RepearEvent are generalized into duration events. The point-time events, the aggregate events and the duration events are generalized to internet-of-things events.
3. Complex event processing module
The complex event processing module is mainly used for reasoning complex events according to simple events.
Complex events are a data processing technique based on a set of predefined rules that specify how a data stream should be processed and which new event streams should be generated as output.
Simply speaking, one or more event streams composed of simple events are matched through a certain rule, and then data desired by a user is output, and complex events meeting the rule are output.
Inputting: one or more event streams consisting of simple events;
and (3) treatment: identifying internal relations among the simple events, wherein a plurality of simple events which accord with a certain rule form a complex event; and (3) outputting: complex events that satisfy rules
3.1 event detection graph model
An event detection graph model, the original event may relate to a suspicious mobile phone call or credit card transaction, or an abnormal sensor temperature measurement, an application defines CEP matching rules composed of operators, and each networking complex event matching rule is represented by a directed acyclic graph, also called the graph is an event detection graph as shown in fig. 3. The nodes of the graph are original events or complex events of the Internet of things, sensing events of the Internet of things are placed on leaf nodes, and edges are used for connecting the events and the operation characters to generate an event detection graph.
The CEP matching rule is a main method for identifying whether the simple event accords with the business rule, the original event is generated by the sensor equipment, and the original event is filtered and combined to generate the simple event; the complex event is generated by matching an event pattern of the simple event, the complex event is sent to the application program to use the event, and the complex event processing is positioned between the terminal sensing equipment of the Internet of things and the application program.
3.2 Complex event definition
Let the atomic event be e:
e=(ID,DeviceID,DeviceType,Location,StartTime,EndTime,KeyValue List) ①
e is the event set, as follows:
E={e|e is an event defined as ①.} ②
the event set consists of many atomic events, so the complex event is represented as:
C=f(E1,E2...,En),(En∈E,n>0) ③
many atomic events constitute simple events, a plurality of simple events are combined to generate an event set, and a plurality of event sets can generate a complex event when a certain condition is met.
In (c), f is an event constructor. The function contains various event operators. Complex events are typically defined by applying an event constructor to constituent events, which may be simple events or complex events.
Let E1Temperature and humidity of area No. 01,001 temperature and humidity sensor 2018, 10, 14, 9, 30, the same principle is E30The humiture is 10 o' clock at 14.10.2018, so SE is (E1, E2) and the complex event C is avg (E)1,E2,..E30) The average value of the temperature and humidity sensors No. 1 and 001 in the area within 30 minutes.
The event operators are as follows:
a) e1 ^ E2 defines that E1 and E2 occur without time limitation. (E1 ^ E2) T indicates that both E1 and E2 occurred within T time.
b) E1V E2 indicates either E1 or E2 occurs.
c)
Figure BDA0002842963010000101
E indicates that E1 did not occur.
d) Aggregation, defining statistics or attribute values of instances of a set of event instances, including count (E), sum (E, key), max (E, key), min (E, key), avg (E, key), and so on. It calculates the number of instances, the sum, the maximum, the minimum, and the average of the key attribute for each instance, respectively.
e) ET defines the valid time range for complex event E, where T represents the time window duration in units of s (seconds), m (minutes), h (hours), etc.
3.3 Complex event cases
And defining complex events for the real-time monitoring application program of the Internet of things by using an event modeling language.
Example (c): internet of things monitoring system for fruit transportation and storage
The optimal storage conditions for fruits are assumed to be 5-10 ℃ and 85-90% humidity. Fruit supermarket fruits are supplied by trucks. Two temperature and humidity sensor nodes DS000560 and DS000570 are respectively deployed on a fruit supermarket and a supplier truck to detect temperature and humidity. Two RFID readers a and B are placed at the entrance of the supermarket and at the door of the supplier truck, respectively. Each box of fruit on a supplier truck has a unique RFID tag.
The requirements for smoothly transporting fruits from a truck to a supermarket are that 1) the temperature of the fruit supermarket is 5-10 ℃, and the humidity is 85% -90%. 2) The temperature in the vehicle is 5-10 ℃ and the humidity is 85% -90%. 3) Less than 10 minutes is required for the fruit to be transported from the truck to the supermarket. If the conditions are met, the fruits can be brought into the supermarket, and if not, alarm processing is carried out.
The CEP system is deployed at an edge compute server at the edge of the network near the location of raw data generation. The CEP system processes a large number of original events to generate complex events, and then determines whether to store an edge computing server complex event library or transmit the complex events to a specific subscription cloud server according to the business requirements of the Internet of things. Because the data volume of the complex event is far smaller than the data volume of the original event, the data volume transmitted to the cloud can be greatly reduced. The data volume of the event of the Internet of things transmitted through the network is reduced, and the real-time processing performance of the whole Internet of things system can be effectively improved.
Edge computing based CEP systems are able to process raw events in a timely manner as raw data is generated and transmitted to an edge computing server. The real-time performance of processing the original event at the edge is better than that of transmitting the original data to the cloud for processing.
The CEP system on the edge server can run on the gateway of the Internet of things or on a separate computer in the same local area network with the gateway of the Internet of things.
3.4 Complex event processing Engine
And a CEP rule engine is based on to realize a complex event processing module on the edge computing server. The original event stream performs pattern matching with matching rules in the rule base. And if the matching is successful, sending the inferred and generated complex event to the cloud server and the subscribed cloud application program.
The definition of the matching rule is the basis for finding complex events. Different application systems define different rule bases and obtain different high-level complex events.
The rule base is predefined according to application requirements, and a large number of rules are stored in the rule base. Rules are used to describe the determination of complex events. For example, "if an RFID tag enters an area but more than two faces are recognized in the video, the system should alarm" if an abnormal event occurs, or "if the temperature C2-C1>10 ℃ and C2 occurs 10 seconds after C1, the system should alarm" if the temperature rises too fast. It can be seen that the rule consists of two parts. The first part is a condition, representing a pattern of events, describing the relationship between the original events. The second part is the calculation result, which is the specific processing of the complex event occurrence.
The Flink CEP is a complex library of event handlers implemented on top of the Flink that allows for the detection of patterns of events in the event stream with the opportunity to master important items in the data.
To improve efficiency, the pattern matching rules will be written on top of the base patterns provided by the Flink CEP.
The Flink CEP is a Complex Event Processing (CEP) library implemented on top of the Flink that allows patterns of events to be detected in the event stream, giving an opportunity to master important items in the data.
In fact, the Flink CEP first requires a user to create and define individual rules, and then the rules of the front and back logical relations are strung together through a linked list to form a logical expression of pattern matching.
Then, the user is required to use the non-deterministic finite automata compiler to split the rules and create non-deterministic finite automata objects, wherein the non-deterministic finite automata objects comprise all states matched with the rules and expressions for conversion among the states.
A state diagram of a non-deterministic finite automata is given in fig. 5.
Flink CEP-three state transition edges:
take, which indicates that the event matching is successful, updates the current state to a new state and advances to a 'next' state;
proctored, when an event comes, the current state does not change, and the event directly 'advances' to the next target state in a state transition diagram;
IGNORE, when the event comes, if the matching is not successful, the current event is ignored, and no change occurs in the current state.
3.5 CEP use of Flink
(1) Create Maven project, add Flink CEP dependent to pom.
(2) The input data stream is obtained using the Pattern API, DataStream < Event > Event1 ═ input.keyby (new KeySelector < Event, Integer > () i.e. taken from the input stream to Event 1; Event1.getname (). equals the temperature from the Event1 Event in real time.
(3) Defining a set of Pattern matching rules, Pattern, where (new iterative condition < Event > () { filter (Event1, temperature ═ 42) }), which will match to this Pattern when the temperature te temperature of Event1 Event is 42;
(4) binding the set of rules with the data stream generates a pattern, where pattern < Event > pattern ═ cep. pattern (Event1, pattern), that is, Event stream Event1 is matched with pattern matching rule pattern.
(5) The output data stream is extracted from the patternStream by a select method, select (new Pattern SelectFunction < Event > () { select (Map < String, List < Event > > Pattern) }, i.e. the matched data is extracted from the data stream according to a predefined pattern.
(6) And acquiring an output event stream extracted by the select method, return create alert (pattern), namely returning an alarm according to the matching rule pattern, and warning that the temperature exceeds 42 ℃ for further processing.
The invention can horizontally expand and contract the system scale according to the actual operation load of the event processing node, so that the whole system can bear the high Regularsover. The node manager can dynamically adjust the single processing common point model in the system, when a certain event processing node fails, a normal node can be randomly selected from the corresponding edge node group to be copied so as to ensure the high integrity of the whole event processing node group, thereby ensuring the continuity of upper-layer services, greatly improving the overall reliability of the complex event processing system, and meeting the data analysis requirements of large data flow, high concurrency and high real-time.
The invention has not been described in detail and is within the skill of the art.
In this specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Claims (3)

1. An edge computing-based complex event processing system for the internet of things, comprising: the system comprises an original event sensing module, an original event processing module and a complex event processing module;
the original event perception module: the complex event processing system for the Internet of things is deployed at the edge of a network close to the original data generation position of the Internet of things, and is called edge calculation; sensing an original event from the terminal of the Internet of things according to a predefined rule and transmitting the original event to an original event processing module of an edge computing server, wherein the original event is an atomic event which is not processed and can not be decomposed and is generated from a data source of the terminal of the Internet of things at the beginning;
a primitive event processing module: the system comprises a data preprocessing module and an event analyzer; the data preprocessing module of the edge computing server filters the repeated irrelevant data and writes the repeated irrelevant data into an original event library of the edge computing server, if the original event is a point-time event, the event is handed to a complex event processing module of the edge computing server, if the original event is a duration event, the event is firstly stored in the original event library of the edge computing server and is sent to the complex event processing module of the edge computing server when needed; the event analyzer reads an original event from an original event library of the edge computing server in real time, and filters and combines the original event according to a predefined matching rule to obtain a simple event;
a complex event processing module: according to the simple event obtained by the original event processing module of the edge computing server, event mode detection, event filtering, event abstraction, event hierarchical structure modeling, event aggregation and conversion and relationship detection among events are further carried out by an event analyzer in the complex event processing module of the edge computing server so as to obtain the complex event; matching the complex event stream with a rule execution mode in a rule base, if matching is successful, calculating and generating more complex and higher complex events and sending the complex events to a complex event processing engine; the complex events are events describing a series of events to occur according to business logic rules of the internet of things compiled in advance; in the complex event processing module, writing matching rules into a business logic script by using an Event Processing Language (EPL) and storing the business logic script into an edge computing server rule base, matching the complex events by a complex event processing engine according to the rules in the edge computing server rule base, wherein the relationship among the events of the Internet of things comprises events, space, dependence and cause and effect; determining whether to store the edge computing server complex event library or transmit the edge computing server complex event library to a specific subscription cloud server according to the business requirements of the Internet of things; in the application of the intelligent logistics based on the temperature and humidity sensor, based on the matching rule, the detection values of the temperature and humidity sensor are compared, and if the matching is successful, the result is transmitted to the subscribed cloud server and a warning is sent out.
2. The edge-computing-based complex event processing system of claim 1, wherein: in the complex event processing module, an internet of things complex event matching rule is represented by a directed acyclic graph, which is also called an event detection graph, namely EDG (Op, X); the nodes of the graph are original events or complex events of the Internet of things, the events are placed on leaf nodes, and an event detection graph is generated by connecting the events and the operation characters through edges; the event detection method comprises the steps that | Op | nodes are provided, EDG is an event detection graph, Op is an operator between events of the Internet of things, and X is the number of edges of the event detection graph, namely an operand between the events of the Internet of things; opi ∈ { SEQ, AND, OR } OR Opi is an aggregation operator, AND in the expression, Opi is any value of the IOT event relation operator in the Op retrievable set; there is an | X | side between operations that receive input from each other (Opi → Opj); for the purpose of multi-match optimization, the output operator of the event detection graph EDG is connected to the top layer or the operator; the operation is carried out, namely an AND operator outputs a complex event when all events participating in the Internet of things occur, when all events participating in the Internet of things occur according to a specified time sequence, a sequence operation, namely an SEQ operator outputs a complex event, OR an OR operator outputs a complex event when any event participating in the Internet of things occurs, an aggregation operator receives a frequency parameter specifying the running time, if the operator receiving frequency parameter among the events of the Internet of things is 1, the aggregation operator is run when a new input event is received each time AND is used together with a specified threshold value; then, when the performed aggregation exceeds or falls below the threshold, a complex event is generated.
3. An edge computing-based complex event processing method for the Internet of things is characterized by comprising the following steps:
original event perception step: acquiring an original event from the terminal sensing equipment of the Internet of things in real time according to a service requirement; generating a tuple for each primitive event; the tuple is packaged according to the format requirement of the IOT service and is transmitted to an original event processing module in an edge computing server close to the IOT sensing equipment according to an IOT transmission protocol required by the IOT service in a Bluetooth or Wi-Fi connection mode and the like;
and (3) original event processing: the method comprises the following steps that a plurality of sockets receive input data streams from different internet of things terminal sensing devices at the same time, deserialize the streams and analyze the corresponding streams into JavaScript object symbol tuples; in addition, the raw event arrival time is appended to each tuple, which is placed in an edge compute server thread safety blocking queue that processes the data in a first-in-first-out order; the edge computing server preprocessing module filters repeated events of irrelevant internet of things and performs a series of preprocessing according to the request data in the edge computing server thread safety blocking queue transmitted from the internet of things original event sensing module according to the business requirements of the internet of things, forms original events and writes the original events into an edge computing server original event library; if the original event is a point-time event, namely an event with high real-time requirement and needing immediate processing, the event analyzer immediately sends the point-time event to a complex event processing module of the edge computing server so as to ensure the real-time processing performance; for the duration event, the event parser acquires the event from the original event library of the edge computing server and processes the related original event to generate the duration event and sends the duration event to the complex event processing module; the event analyzer reads an original event from an original event library of the edge computing server in real time, and filters and combines the original event according to a predefined matching rule to obtain a simple event;
complex event processing steps: the simple event enters an event analyzer for preprocessing and then forms a complex event; matching rules in the rule base are analyzed into an event detection graph; carrying out pattern matching on the event stream of the Internet of things and the rules in the rule base; if the matching is successful, determining whether an edge computing server complex event library needs to be stored or transmitted to a specifically subscribed cloud server according to the service requirement of the Internet of things; if the matching is unsuccessful, storing the complex event into an edge computing server complex event library; each event stream is associated with a separate thread security blocking queue, a producer-consumer queue mode is used, namely, producers and consumers do not directly communicate with each other but communicate through the blocking queue, a complex event processing engine serves as a producer, and an edge computing server complex event library or a cloud server serves as a consumer of a certain event stream; if the data is to be sent to the cloud server, the edge computing device communicates with the cloud server through Wi-Fi or a cellular network.
CN202011498669.XA 2020-12-18 2020-12-18 Edge computing-based complex event processing system and method for Internet of things Pending CN112653749A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011498669.XA CN112653749A (en) 2020-12-18 2020-12-18 Edge computing-based complex event processing system and method for Internet of things

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011498669.XA CN112653749A (en) 2020-12-18 2020-12-18 Edge computing-based complex event processing system and method for Internet of things

Publications (1)

Publication Number Publication Date
CN112653749A true CN112653749A (en) 2021-04-13

Family

ID=75354792

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011498669.XA Pending CN112653749A (en) 2020-12-18 2020-12-18 Edge computing-based complex event processing system and method for Internet of things

Country Status (1)

Country Link
CN (1) CN112653749A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113448555A (en) * 2021-06-30 2021-09-28 深信服科技股份有限公司 Association analysis method, device, equipment and storage medium
CN113596043A (en) * 2021-08-03 2021-11-02 中国电信股份有限公司 Attack detection method, attack detection device, storage medium and electronic device
CN113641712A (en) * 2021-08-20 2021-11-12 平安银行股份有限公司 Hit processing method, device and equipment for complex event and storage medium
WO2022222623A1 (en) * 2021-04-20 2022-10-27 International Business Machines Corporation Composite event estimation through temporal logic

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105138385A (en) * 2015-08-25 2015-12-09 北京邮电大学 Complex event processing method for Internet of Things
CN106294824A (en) * 2016-08-17 2017-01-04 广东工业大学 Manufacture Internet of Things towards the complex events detecting methods of uncertain data stream and system
US20170060622A1 (en) * 2015-02-27 2017-03-02 Savi Technology, Inc. System and method for a data processing architecture
CN109885588A (en) * 2019-01-23 2019-06-14 齐鲁工业大学 A kind of complex events detecting methods and system
CN111708645A (en) * 2020-06-12 2020-09-25 北京思特奇信息技术股份有限公司 Event processing method and system based on stream processing

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170060622A1 (en) * 2015-02-27 2017-03-02 Savi Technology, Inc. System and method for a data processing architecture
CN105138385A (en) * 2015-08-25 2015-12-09 北京邮电大学 Complex event processing method for Internet of Things
CN106294824A (en) * 2016-08-17 2017-01-04 广东工业大学 Manufacture Internet of Things towards the complex events detecting methods of uncertain data stream and system
CN109885588A (en) * 2019-01-23 2019-06-14 齐鲁工业大学 A kind of complex events detecting methods and system
CN111708645A (en) * 2020-06-12 2020-09-25 北京思特奇信息技术股份有限公司 Event processing method and system based on stream processing

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
许冬冬 等: "基于事件共享机制的物联网复杂事件处理方法", 《计算机应用》 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022222623A1 (en) * 2021-04-20 2022-10-27 International Business Machines Corporation Composite event estimation through temporal logic
GB2620538A (en) * 2021-04-20 2024-01-10 Ibm Composite event estimation through temporal logic
CN113448555A (en) * 2021-06-30 2021-09-28 深信服科技股份有限公司 Association analysis method, device, equipment and storage medium
CN113448555B (en) * 2021-06-30 2024-04-09 深信服科技股份有限公司 Correlation analysis method, device, equipment and storage medium
CN113596043A (en) * 2021-08-03 2021-11-02 中国电信股份有限公司 Attack detection method, attack detection device, storage medium and electronic device
CN113641712A (en) * 2021-08-20 2021-11-12 平安银行股份有限公司 Hit processing method, device and equipment for complex event and storage medium
CN113641712B (en) * 2021-08-20 2023-08-15 平安银行股份有限公司 Hit processing method, device, equipment and storage medium for complex event

Similar Documents

Publication Publication Date Title
CN112653749A (en) Edge computing-based complex event processing system and method for Internet of things
WO2020119481A1 (en) Network traffic classification method and system based on deep learning, and electronic device
Khare et al. Big data in IoT
US9165333B2 (en) Systems and methods for supply chain event visualization
Sun et al. Efficient rule engine for smart building systems
US20180075038A1 (en) Anomaly detection by multi-level tolerance relations
Lan et al. A universal complex event processing mechanism based on edge computing for internet of things real-time monitoring
CN107203199A (en) A kind of industry control network safe early warning method and system
Ramachandran et al. Large-scale situation awareness with camera networks and multimodal sensing
WO2015094269A1 (en) Hybrid flows containing a continuous flow
CN111225280B (en) Lightweight video analysis system based on embedded platform
CN110034966B (en) Data flow classification method and system based on machine learning
CN115858796A (en) Fault knowledge graph construction method and device
CN112559237B (en) Operation and maintenance system troubleshooting method and device, server and storage medium
CN114338746A (en) Analysis early warning method and system for data collection of Internet of things equipment
Zaman et al. A recommender system architecture for predictive telecom network management
Burlachenko et al. Devising a method for the active coordination of video cameras in optical navigation based on the multi-agent approach
CN113505048A (en) Unified monitoring platform based on application system portrait and implementation method
Chakravarthy et al. Adapting stream processing framework for video analysis
CN114430331A (en) Network security situation sensing method and system based on knowledge graph
CN113434573A (en) Multi-dimensional image retrieval system, method and equipment
Curry et al. Multimodal event processing: A neural-symbolic paradigm for the internet of multimedia things
US20230412769A1 (en) Scalable Visual Computing System
CN112861894A (en) Data stream classification method, device and system
CN116910144A (en) Computing power network resource center, computing power service system and data processing method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210413

RJ01 Rejection of invention patent application after publication