CN116049234A - Event processing method, system, electronic device and computer readable medium - Google Patents

Event processing method, system, electronic device and computer readable medium Download PDF

Info

Publication number
CN116049234A
CN116049234A CN202211730095.3A CN202211730095A CN116049234A CN 116049234 A CN116049234 A CN 116049234A CN 202211730095 A CN202211730095 A CN 202211730095A CN 116049234 A CN116049234 A CN 116049234A
Authority
CN
China
Prior art keywords
rule
event
processed
computing node
key
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211730095.3A
Other languages
Chinese (zh)
Inventor
梁润庭
马越
张健
刘海东
何宇彤
罗旋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Volcano Engine Technology Co Ltd
Original Assignee
Beijing Volcano Engine Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Volcano Engine Technology Co Ltd filed Critical Beijing Volcano Engine Technology Co Ltd
Priority to CN202211730095.3A priority Critical patent/CN116049234A/en
Publication of CN116049234A publication Critical patent/CN116049234A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24564Applying rules; Deductive queries
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24568Data stream processing; Continuous queries
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/30Creation or generation of source code
    • G06F8/31Programming languages or programming paradigms
    • G06F8/315Object-oriented languages
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/40Transformation of program code
    • G06F8/41Compilation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/448Execution paradigms, e.g. implementations of programming paradigms
    • G06F9/4498Finite state machines
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application discloses an event processing method, an event processing system, electronic equipment and a computer readable medium, wherein when the event processing system at least comprises a first computing node and a second computing node, the working principle of the event processing system is as follows: after acquiring an event to be processed, firstly carrying out grouping key extraction processing on the event to be processed by the first computing node by utilizing a rule matching granularity key extractor corresponding to at least one candidate rule to obtain a grouping key of the event to be processed; and then the first computing node sends the event to be processed and the grouping key to the second computing node so that the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule, thereby realizing the purpose of simultaneously running a plurality of CEP rules in the system and effectively improving the event processing performance of the system.

Description

Event processing method, system, electronic device and computer readable medium
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to an event processing method, an event processing system, an electronic device, and a computer readable medium.
Background
Composite event processing (Complex Event Processing, CEP) is an analysis technique based on event streams in a dynamic environment. Wherein, the events are usually meaningful state changes, through analyzing the relationship among the events, utilizing the technologies of filtering, association, aggregation and the like, the query rules are formulated according to the time sequence relationship among the events and the aggregation relationship, the event sequence meeting the requirements is continuously queried from the event stream, and finally the complex composite events are obtained through analysis.
In addition, CEP can generally involve three aspects of event sequence matching, event combination matching, and event window aggregation. Wherein the event sequence matching refers to finding a certain event sequence from the unordered event stream (e.g., a sequence of a first event followed by a second event a preset number of times, followed by a third event). The event combination matching refers to finding a certain event combination (e.g., a combination of a first event or a second event) from the event stream. The event window aggregation refers to whether or not there is a first event in N neighboring events continuously extracted from the event stream. N is a positive integer, N represents the window size.
Furthermore, in some application areas, the analysis process may be performed for the event stream, typically by means of pre-configured CEP rules. Among other things, CEP rules can generally include three core elements, rule matching granularity, rule matching logic, and rule matching result processing logic. The rule matching granularity is typically used to aggregate events with the same attributes together for analysis (e.g., events occurring at the same location, prices of the same stock, etc.), so that the rule matching granularity can indicate which attributes (e.g., location or stock identity, etc.) are grouped for the event stream. The rule matching logic is logic that matches sequences of events (e.g., detects the presence or absence of a sequence of "event 2→event 4→event 1" from a sequence of events of "event 1→event 2→event 4→event 9→event 1"). The rule matching result processing logic is logic that refers to processing of results determined based on the rule matching logic (e.g., after matching results in a sequence of "event 2→event 4→event 1", the time interval between event 2→event 1 is calculated, etc.).
However, some execution systems (e.g., the Flink system) have drawbacks with respect to the execution of CEP rules, resulting in poor event handling performance of the system.
Disclosure of Invention
In order to solve the technical problems, the application provides an event processing method, an event processing system, electronic equipment and a computer readable medium, which can improve event processing performance.
In order to achieve the above purpose, the technical scheme provided by the application is as follows:
the application provides an event processing method, which is applied to an event processing system, wherein the event processing system comprises a first computing node and a second computing node;
the method comprises the following steps:
after acquiring an event to be processed, the first computing node performs grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to at least one candidate rule to obtain a grouping key of the event to be processed;
the first computing node sends the event to be processed and the grouping key to the second computing node;
and the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule.
In one possible implementation, the first computing node includes at least one first computing unit; the first computing unit comprises a rule matching granularity key extractor corresponding to the at least one candidate rule; the second computing node comprises at least one second computing unit; the second computing unit comprises a rule matching logic executor corresponding to the at least one candidate rule and a rule matching result processing logic executor corresponding to the at least one candidate rule;
The first computing node performs grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to at least one candidate rule to obtain a grouping key of the event to be processed, and the grouping key comprises the following steps:
a first target unit in the at least one first computing unit performs grouping key extraction processing on the event to be processed by utilizing a rule matching granularity key extractor corresponding to at least one candidate rule to obtain a grouping key of the event to be processed;
the first computing node sends the event to be processed and the grouping key to the second computing node, including:
the first target unit sends the event to be processed and the grouping key to a second target unit in the at least one second computing unit;
the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule, and the processing comprises the following steps:
and the second target unit processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule.
In one possible embodiment, the second target unit has processed at least one event to be referenced over a historical period of time; the grouping key of the event to be referred is the same as the grouping key of the event to be processed; and the end time of the historical time period is not later than the acquisition time of the event to be processed.
In one possible embodiment, the method further comprises:
the first computing node utilizes a rule event filter corresponding to the at least one candidate rule to filter the event to be processed to obtain a filtering result of the event to be processed;
the first computing node performs grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to at least one candidate rule to obtain a grouping key of the event to be processed, and the grouping key comprises the following steps:
if the screening processing result indicates that the event to be processed meets the event screening condition corresponding to the target rule in the at least one candidate rule, the first computing node performs grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to the target rule to obtain a grouping key of the event to be processed;
The second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule, and the processing comprises the following steps:
and the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the target rule and the rule matching result processing logic executor corresponding to the target rule.
In one possible implementation, the first computing node includes at least one first computing unit; the first computing unit comprises a rule event filter corresponding to the at least one candidate rule and a rule matching granularity key extractor corresponding to the at least one candidate rule; the second computing node comprises at least one second computing unit; the second computing unit comprises a rule matching logic executor corresponding to the at least one candidate rule and a rule matching result processing logic executor corresponding to the at least one candidate rule;
the first computing node performs screening processing on the event to be processed by using a rule event filter corresponding to the at least one candidate rule to obtain a screening processing result, which comprises the following steps:
A third target unit in the at least one first computing unit screens the event to be processed by using a rule event screening device corresponding to the at least one candidate rule to obtain a screening result;
the first computing node performs grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to the target rule to obtain a grouping key of the event to be processed, and the method comprises the following steps:
the third target unit performs grouping key extraction processing on the event to be processed by utilizing a rule matching granularity key extractor corresponding to the target rule to obtain a grouping key of the event to be processed;
the first computing node sends the event to be processed and the grouping key to the second computing node, including:
the third target unit sends the event to be processed and the grouping key to a second target unit in the at least one second computing unit;
the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the target rule and the rule matching result processing logic executor corresponding to the target rule, and the processing comprises the following steps:
And the second target unit processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the target rule and the rule matching result processing logic executor corresponding to the target rule.
In one possible embodiment, the second target unit has processed at least one event to be referenced over a historical period of time; the grouping key of the event to be referred is the same as the grouping key of the event to be processed, and the screening processing result of the event to be referred is the same as the screening processing result of the event to be processed; and the end time of the historical time period is not later than the acquisition time of the event to be processed.
In one possible implementation manner, the number of the target rules is M, and M is a positive integer.
In one possible implementation, the event processing system further includes a rule updating unit;
the method further comprises the steps of:
after the rule event filter corresponding to the rule to be used is obtained, the rule updating unit sends the rule event filter corresponding to the rule to be used to each first computing unit; each first computing unit updates a rule event filter corresponding to the at least one candidate rule by using the rule event filter corresponding to the rule to be used;
After the rule matching granularity key extractor corresponding to the rule to be used is obtained, the rule updating unit sends the rule matching granularity key extractor corresponding to the rule to be used to each first computing unit; each first computing unit updates a rule matching granularity key extractor corresponding to the at least one candidate rule by using the rule matching granularity key extractor corresponding to the rule to be used;
after the rule matching logic executor corresponding to the rule to be used is obtained, the rule updating unit sends the rule matching logic executor corresponding to the rule to be used to each second computing unit; each second computing unit updates a rule matching logic executor corresponding to the at least one candidate rule by using the rule matching logic executor corresponding to the rule to be used;
after the rule matching result processing logic executor corresponding to the rule to be used is obtained, the rule updating unit sends the rule matching result processing logic executor corresponding to the rule to be used to each second computing unit; and updating the rule matching result processing logic executor corresponding to the at least one candidate rule by each second computing unit by using the rule matching result processing logic executor corresponding to the rule to be used.
The application also provides an event processing system, which comprises a first computing node and a second computing node;
the first computing node is used for carrying out grouping key extraction processing on the event to be processed by utilizing a rule matching granularity key extractor corresponding to at least one candidate rule after the event to be processed is acquired, so as to obtain a grouping key of the event to be processed;
the first computing node is further configured to send the event to be processed and the grouping key to the second computing node;
the second computing node is configured to process the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule, and the rule matching result processing logic executor corresponding to the at least one candidate rule.
The application also provides an electronic device, comprising: a processor and a memory;
the memory is used for storing instructions or computer programs;
the processor is configured to execute the instructions or the computer program in the memory, so that the electronic device executes the event processing method provided by the application.
The present application also provides a computer readable medium having instructions or a computer program stored therein, which when run on a device, cause the device to perform the event processing method provided herein.
The present application provides a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for performing the event processing methods provided herein.
Compared with the prior art, the application has at least the following advantages:
for the event processing system (for example, a link system) provided by the application, when the system at least comprises a first computing node and a second computing node, the system can complete a CEP processing process for an event stream by means of the two nodes, and the working principle of the system is as follows: after acquiring an event to be processed (for example, a newly generated event in an event stream), firstly performing grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to at least one candidate rule by using the first computing node to obtain a grouping key of the event to be processed; and then the first computing node sends the event to be processed and the grouping key to the second computing node so that the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule, thereby realizing the purpose of simultaneously running a plurality of CEP rules in the system and effectively improving the event processing performance of the system.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of an NFA provided in an embodiment of the present application;
FIG. 2 is a schematic diagram illustrating an operation principle of an event processing system according to an embodiment of the present application;
FIG. 3 is a flowchart of a method for event processing according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram illustrating the operation of another event processing system according to an embodiment of the present application;
FIG. 5 is a schematic diagram of an event processing system according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to better understand the technical solutions of the present application, some technical terms related to the present application are first described below.
The non-deterministic finite state machine (Nondeterministic Finite Automata, NFA) refers to a finite state automaton that can have multiple possible next states for each state and input symbol pair. In addition, the NFA may be configured to implement rule matching logic configured for one CEP rule, and the implementation procedure may specifically be: for rule matching logic configured for a CEP rule, the rule matching logic may be parsed and compiled to obtain a series of state vertices and edges for state transitions between the vertices, such that the vertices and edges may be used to construct NFAs (e.g., NFAs shown in fig. 1) corresponding to the CEP rule. As an example, when the CEP matching rule is a rule of "complete event 1, then complete event 2", the NFA generated based on the CEP matching rule is the NFA shown in fig. 1, so that the specific meaning of the NFA is as follows: when event 1 is acquired, the state machine jumps to the state where event 1 is completed. At this point the state machine has two ways of jumping (non-deterministic): after acquiring other events (not event 2), the state machine keeps the current state unchanged; but after event 2 is acquired, the state machine jumps to the state where the match was successful, so that the match can be completed.
Flink refers to a framework and distributed processing engine, and can be used to perform stateful calculations on unbounded and bounded data streams. In addition, the flank is designed to run in all common clustered environments, performing computations at memory execution speed and on an arbitrary scale. Furthermore, flink may be used to implement CEP rules.
The Flink CEP refers to a complex event processing library in the Flink, so that the Flink can implement analysis processing for a series of behavior information (events) based on the complex event processing library, for example, by analyzing the relationship between events: the time sequence relation, the time interval, the progressive relation and the aggregation relation continuously inquire out the event sequence meeting the requirements from the event stream, and finally, more complex compound events are obtained.
Based on the relevant content of the technical nouns, the relevant content of the technical scheme of the application is described below.
Indeed, for a Flink system, in some application scenarios, the system may implement CEP rules in such a way that "1 CEP rule corresponds to 1 Flink task" for ease of understanding, as described below in connection with fig. 2.
As an example, as shown in fig. 2, a specific process in which CEP rules are executed by the link system includes steps 1-6 below.
Step 1: the Flink code is developed by the relevant personnel to define rule matching granularity, rule matching logic, and rule matching result processing logic for one CEP rule.
Step 2: compiling the Flink code to obtain an event grouping key extractor, an NFA component and a rule matching result processing function corresponding to the CEP rule, and submitting the event grouping key extractor, the NFA component and the rule matching result processing function as a Flink task. The event grouping key extractor is used for performing grouping key extraction processing on an event according to the rule matching granularity of the CEP rule; the NFA component is configured to implement rule matching logic of the CEP rule; the rule matching result processing function is for processing a matching result determined based on the rule matching logic.
Step 3: when the Flink system is executing the Flink task, the system continuously reads the event from the event stream, invokes the event grouping key extractor to extract the grouping key corresponding to the event from each event, and constructs a binary group corresponding to the event according to the grouping key corresponding to the event and the event.
Step 4: hash grouping is carried out on the event by pressing a grouping key corresponding to the event, and the event with the same grouping key is sent to the same CEP rule calculation unit for rule matching calculation. Wherein the CEP rule calculation unit comprises the above NFA component and a rule matching result processing function.
Step 5: after the CEP rule calculation unit receives the event, it is processed by the above NFA component according to the above rule matching logic and saves the matched intermediate results.
Step 6: and when the NFA component determines that the matching is finished, the rule matching result processing function is called to process the stored matching result so as to obtain an event processing result for the event stream.
Based on the above related content of fig. 2, for the flank system, since the system can independently develop and deploy a set of rule matching codes for each service scenario according to the rule requirement of each service scenario, the system development efficiency is low; because the system executes the CEP rule in a mode that 1 CEP rule corresponds to 1 Flink task, the number of the Flink tasks becomes very huge along with the increase of the CEP rule, so that huge task operation and maintenance cost is brought to the system, and the event processing performance of the system is easily affected.
Based on the foregoing related content of the flank system, in order to solve the technical problems shown in the background section, an embodiment of the present application provides an event processing method applied to an event processing system (for example, a flank system), where the event processing system includes at least a first computing node and a second computing node, the method includes: after acquiring an event to be processed (for example, a newly generated event in an event stream), firstly performing grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to at least one candidate rule by using the first computing node to obtain a grouping key of the event to be processed; and then the first computing node sends the event to be processed and the grouping key to the second computing node so that the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule, thereby realizing the purpose of simultaneously running a plurality of CEP rules in the system and effectively improving the event processing performance of the system.
In order to make the present application solution better understood by those skilled in the art, the following description will clearly and completely describe the technical solution in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
In order to better understand the technical solutions provided in the present application, the event processing method provided in the present application is described below with reference to some drawings. As shown in fig. 3, when the event processing system includes a first computing node and a second computing node, the event processing method applied to the event processing system provided in the embodiment of the present application includes the following S1 to S3.
S1: after acquiring the event to be processed, the first computing node performs grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to at least one candidate rule to obtain a grouping key of the event to be processed.
The event processing system is used for carrying out event processing on the event stream according to some CEP rules; moreover, embodiments of the present application are not limited to this event processing system, which may be a Flink system, for example.
In addition, the event processing system may include at least a first computing node and a second computing node. Wherein the first computing node is configured to determine at least a grouping key for an event; moreover, the embodiment of the present application is not limited to this first computing node, and for example, it may be a node having an event preprocessing function in the Flink system shown in fig. 4. The second computing node is configured to perform some processing (e.g., processing based on rule matching logic, processing based on rule matching result processing logic, etc.) on the event according to the grouping key determined by the first computing node; moreover, the embodiment of the present application is not limited to this second computing node, and for example, it may be a node with CEP rule matching function in the Flink system shown in fig. 4. It should be noted that, in order to better understand the event processing system, the following description is made in connection with the relevant content of the event processing method (e.g., S1-S3 or steps 11-14, etc.) provided in the present application.
The event to be processed refers to an event acquired from the event stream; moreover, embodiments of the present application are not limited to the event to be processed, for example, in some application scenarios (for example, when performing real-time processing on an event in an event stream), the event to be processed may refer to a newly generated event in the event stream. It should be noted that, the embodiment of the present application is not limited to the above-mentioned method for acquiring the event to be processed.
The ith candidate rule is a CEP rule (e.g., rule a or rule B shown in fig. 4) that is pre-configured for the link system by a pointer. Wherein I is a positive integer, I is less than or equal to I, I is a positive integer, and I represents the number of the at least one candidate rule.
The rule matching granularity key extractor corresponding to the ith candidate rule refers to a rule matching granularity key extractor obtained by compiling (or parsing) the ith candidate rule such that the "rule matching granularity key extractor corresponding to the ith candidate rule" is used to extract a grouping key of an event (i.e., "matching granularity key" shown in fig. 4) according to the rule matching granularity configured for the ith candidate rule, so that the grouping key can represent a certain attribute of the event, so that some processing (e.g., hash grouping processing, etc.) can be performed based on the grouping key later. Wherein I is a positive integer, I is less than or equal to I, and I is a positive integer.
The above "grouping key of a to-be-processed event" refers to extracting a processing result from the grouping key of the to-be-processed event so that the "grouping key of the to-be-processed event" can represent a certain attribute of the event so that some processing (e.g., hash grouping processing, etc.) can be performed on the to-be-processed event subsequently based on the "grouping key of the to-be-processed event". It should be noted that, the embodiment of the present application is not limited to the determination process of the "grouping key of events to be processed" above.
Indeed, to better enhance the event handling effect, the present embodiment further provides a possible implementation manner of the above first computing node, where the first computing node includes at least one first computing unit, and each first computing unit includes the rule matching granularity key extractor corresponding to the above "at least one candidate rule" (for example, the extractor is stored in advance in each first computing unit), so that each first computing unit may be used to determine the grouping key of an event.
Based on this, the present embodiment also provides a possible implementation manner of the above S1, in this implementation manner, when the above first computing node includes at least one first computing unit, S1 may specifically be: after the event to be processed is acquired, a first target unit in the at least one first computing unit performs grouping key extraction processing on the event to be processed by utilizing a rule matching granularity key extractor corresponding to at least one candidate rule, so as to obtain a grouping key of the event to be processed. The first target unit is a first computing unit for performing grouping key extraction processing on the event to be processed; moreover, the embodiment of the present application is not limited to the determining process of the first target unit, and may be implemented, for example, by any method that can select a computing unit for performing data processing from a distributed system, which exists in the present or future.
Based on the above description of S1, when the first computing node includes a plurality of first computing units (for example, the computing units-1 and-2 shown in fig. 4), after the first computing node acquires the event to be processed, a first target unit (for example, the computing unit-1 or the computing unit-2 shown in fig. 4) may be determined from the first computing units, and the first target unit performs a packet key extraction process on the event to be processed by using a rule matching granularity key extractor corresponding to at least one candidate rule stored therein, so as to obtain a packet key of the event to be processed, so that the packet key can represent a certain attribute of the event, so that some processes (for example, hash packet processing and the like) can be performed on the event to be processed based on the "packet key of the event to be processed" later.
S2: the first computing node sends the event to be processed and the grouping key of the event to be processed to the second computing node.
In this embodiment, for a processing system including a first computing node and a second computing node, after the first computing node determines a grouping key of an event to be processed, the first computing node may send a binary group of < the event to be processed, the grouping key of the event to be processed > to the second computing node, so that the second computing node can implement some processing procedures for the event to be processed (for example, a processing procedure based on rule matching logic and a processing procedure based on rule matching result processing logic) based on the grouping key.
Indeed, to better enhance the event handling effect, the present embodiment further provides a possible implementation manner of the above second computing node, where the second computing node includes at least one second computing unit (for example, the computing unit-3 and the computing unit-4 shown in fig. 4), and each of the second computing units includes "the rule matching logic executor corresponding to the at least one candidate rule" and "the rule matching result processing logic executor corresponding to the at least one candidate rule" hereinafter (for example, both the two executors are stored in each of the second computing units in advance), so that each of the second computing units may be used to perform the corresponding processing procedure (for example, the processing procedure based on the rule matching logic and the processing procedure based on the rule matching result processing logic) for one event.
Based on this, the present embodiment also provides a possible implementation manner of the above S2, in this implementation manner, when the above first computing node includes at least one first computing unit, and the above second computing node includes at least one second computing unit, S2 may specifically be: after a first target unit of the first computing units determines a grouping key of an event to be processed, the first target unit sends the event to be processed and the grouping key of the event to be processed to a second target unit of the at least one second computing unit, so that some processing procedures (such as processing procedures based on rule matching logic, processing procedures based on rule matching result processing logic, and the like) for the event to be processed are realized by the second target unit based on the grouping key. The second target unit is a second computing unit for executing a subsequent processing procedure for the event to be processed; moreover, the embodiment of the present application is not limited to the determination process of the second target unit, and may be implemented, for example, by any method that can select a computing unit for performing data processing from a distributed system, which exists in the present or future.
In fact, in some application scenarios, if the above at least one second computing unit has access to a certain storage space, the result obtained by performing rule matching processing on the event (that is, the intermediate matching result shown in fig. 4) may be stored in the storage space, so that the second computing units may obtain the result from the storage space, so in order to improve the event processing flexibility, different events in the event sequence conforming to a certain CEP rule may be obtained by matching different second computing units. Based on this, the embodiment of the present application provides a possible implementation manner of the above second target unit, in this implementation manner, the second target unit may be any one of the second computing units, so that the second target unit may complete a subsequent execution process of a CEP rule by using the content stored in the storage space.
In addition, in some application scenarios (for example, a link system), if, for at least one second computing unit, these second computing units can only access the storage space corresponding to the second computing unit, in order to better meet the self-requirements of CEP, all events of the same grouping key can be sent to the same second computing unit for subsequent processing. Based on this, the present application embodiment also provides another possible implementation manner of the above second target unit, in which the second target unit processes at least one event to be referenced in a historical period. The event to be referred refers to the event which is processed by the second computing node and belongs to the same hash packet with the event to be processed. It can be seen that the grouping key of the event to be referenced is the same as the grouping key of the event to be processed (i.e., the events to be referenced belong to the same hash grouping as the event to be processed).
In addition, the embodiment of the present application does not limit the above historical period, for example, the end time of the historical period is no later than the acquisition time of the event to be processed, so that the historical period may refer to a period before the event processing procedure for the event to be processed is triggered.
Also, the embodiment of the present application is not limited to the implementation of the above step "the first target unit sends the pending event and the grouping key of the pending event to the second target unit in the at least one second computing unit", and may specifically be, for example: after the first target unit determines the packet key of the event to be processed, the first target unit may firstly perform hash packet processing on the event to be processed according to the packet key to obtain a packet to be used (i.e., a hash packet to which the event to be processed belongs), so that the packet to be used includes at least one event to be referred to and the event to be processed, thereby enabling the packet to be used to represent some events with the same packet key; then, determining a second target unit which has processed the events to be referred from the second computing units according to the packets to be used by the first target unit; finally, the first target unit sends the event to be processed and the grouping key of the event to be processed to the second target unit so that the second target unit can realize some processing procedures for the event to be processed based on the grouping key.
S3: the second computing node processes the event to be processed according to the grouping key of the event to be processed, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule.
The rule matching logic executor corresponding to the ith candidate rule refers to a rule matching logic executor (for example, NFA) obtained by compiling (or parsing) the ith candidate rule, so that the "rule matching logic executor corresponding to the ith candidate rule" is used for performing event matching processing for an event according to rule matching logic configured for the ith candidate rule. Wherein I is a positive integer, I is less than or equal to I, and I is a positive integer.
The rule matching result processing logic executor corresponding to the i-th candidate rule refers to a rule matching result processing logic executor (e.g., rule matching result processing function) obtained by compiling (or parsing) the i-th candidate rule, so that the "rule matching result processing logic executor corresponding to the i-th candidate rule" is used for processing some event matching results (e.g., "intermediate matching results" shown in fig. 4) according to the rule matching result processing logic configured for the i-th candidate rule. Wherein I is a positive integer, I is less than or equal to I, and I is a positive integer.
In fact, in order to better improve the event processing effect, the embodiment of the present application further provides a possible implementation manner of the above S3, where, when the above second computing node includes at least one second computing unit, S3 may specifically be: after the second target unit in the second computing units obtains the event to be processed and the grouping key of the event to be processed, the second target unit processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule. It should be noted that, the embodiments of the present application are not limited to the processing procedure of the two types of actuators for the event to be processed, and may be implemented by using any existing or future processing procedure of the two types of actuators corresponding to one CEP rule for the event.
Based on the above description of S1 to S3, for the event processing method applied to the event processing system (for example, the link system) provided in the embodiment of the present application, when the event processing system includes at least a first computing node and a second computing node, the method includes: after acquiring an event to be processed (for example, a newly generated event in an event stream), firstly performing grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to at least one candidate rule by using the first computing node to obtain a grouping key of the event to be processed; and then the first computing node sends the event to be processed and the grouping key to the second computing node so that the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule, thereby realizing the purpose of simultaneously running a plurality of CEP rules in the system and effectively improving the event processing performance of the system.
Indeed, to better enhance event handling performance, the present embodiments also provide a possible implementation of the above event handling method, in which the method may specifically include the following steps 11-14.
Step 11: after the event to be processed is acquired, the first computing node utilizes a rule event filter corresponding to at least one candidate rule to filter the event to be processed, and a filtering result of the event to be processed is obtained.
The rule event filter corresponding to the ith candidate rule refers to a rule event filter obtained by compiling (or parsing) the ith candidate rule, so that the "rule event filter corresponding to the ith candidate rule" is used for filtering events of interest (i.e., events meeting the event filtering conditions) of the ith candidate rule from an event stream according to the event filtering conditions configured for the ith candidate rule, so that the events of interest (e.g., grouping key extraction processing, processing based on rule matching logic, processing based on rule matching result processing logic, etc.) can be processed for the events of interest by the ith candidate rule. Wherein I is a positive integer, I is less than or equal to I, and I is a positive integer.
The above "filtering processing result of the event to be processed" is used for describing which candidate rules the event to be processed meets the event filtering condition corresponding to, so that the "filtering processing result of the event to be processed" can represent the concerned event of which candidate rules the event to be processed belongs to, so that the candidate rules concerned about the event to be processed can be used for processing the event to be processed subsequently.
Indeed, to better enhance the event processing effect, the embodiment of the present application further provides a possible implementation manner of the above first computing node, where the first computing node includes at least one first computing unit, and each first computing unit includes the above "rule event filter corresponding to at least one candidate rule" and the above "rule matching granularity key extractor corresponding to at least one candidate rule" (for example, the extractor and the filter are stored in each first computing unit in advance), so that each first computing unit may be used not only to determine a grouping key of an event, but also to perform a filtering process for an event, so that the above event processing system can be prevented from executing some processing procedures that are not practical as much as possible.
Based on this, the present embodiment also provides a possible implementation manner of the above step 11, in this implementation manner, when the above first computing node includes at least one first computing unit, step 11 may specifically be: after the event to be processed is acquired, a third target unit in the at least one first computing unit performs screening processing on the event to be processed by using a rule event screening device corresponding to the at least one candidate rule, so as to obtain a screening processing result. The third target unit is a first computing unit for performing event screening processing and grouping key extraction processing on the event to be processed; moreover, the embodiment of the present application is not limited to the determination process of the third target unit, and may be implemented, for example, by any method that can select a computing unit for performing data processing from a distributed system, which exists in the present or future.
Based on the above description of step 11, when the event processing system including the first computing node includes a plurality of first computing units (for example, the computing units-1 and-2 shown in fig. 4), after the first computing node acquires the event to be processed, a third target unit may be determined from the first computing units, and the third target unit performs screening processing on the event to be processed by using a rule event filter corresponding to at least one candidate rule stored in the third target unit, so as to obtain a screening processing result of the event to be processed, so that the screening processing result can indicate which candidate rule of interest of the event to be processed belongs to, so that only the candidate rule of interest of the event to be processed needs to be processed for the event to be processed, and thus, it is unnecessary to execute which candidate rule of not interest of the event to be processed for the event to be processed, and computing resources can be effectively saved.
Step 12: if the screening processing result indicates that the event to be processed meets the event screening condition corresponding to the target rule in the at least one candidate rule, the first computing node performs grouping key extraction processing on the event to be processed by using the rule matching granularity key extractor corresponding to the target rule, and a grouping key of the event to be processed is obtained.
The target rule refers to a candidate rule focusing on the event to be processed, so that CEP matching processing is needed to be performed on the event to be processed by using the target rule subsequently; moreover, embodiments of the present application do not limit the number of target rules, which may be M, for example. M is a positive integer. That is, the target rule may be one or a plurality of.
In addition, in step 12 above, the first computing node performs the grouping key extraction process on the event to be processed by using the rule matching granularity key extractor corresponding to the target rule, so as to obtain the related content of the grouping key of the event to be processed, which is similar to the related content of S1 above, and for convenience of understanding, the following description is made with reference to an example.
As an example, when the above first computing node includes at least one first computing unit, step 12 may specifically be: after a third target unit in the first computing units determines a screening processing result of an event to be processed, if the screening processing result indicates that the event to be processed meets an event screening condition corresponding to a target rule in at least one candidate rule, the third target unit performs grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to the target rule to obtain a grouping key of the event to be processed.
Step 13: the first computing node sends the event to be processed and the grouping key of the event to be processed to the second computing node.
It should be noted that, the relevant content of step 13 is similar to that of step S2 above, and for ease of understanding, the following description is made with reference to examples.
As an example, when the above first computing node comprises at least one first computing unit and the above second computing node comprises at least one second computing unit, step 13 may specifically be: after a third target unit in the first computing units determines a grouping key of the event to be processed, the third target unit sends the event to be processed and the grouping key to a second target unit in the at least one second computing unit. Wherein, please refer to the above S2 for the relevant content of the second target unit.
Indeed, in order to better improve the event handling effect, the present embodiment also provides a possible implementation manner of the above second target unit, where the second target unit has processed at least one event to be referenced in a historical time period. The grouping key of the event to be referred is the same as the grouping key of the event to be processed (i.e., the events to be referred and the event to be processed belong to the same hash grouping), and the filtering result of the event to be referred is the same as the filtering result of the event to be processed (i.e., the event to be referred and the event to be processed all meet the event filtering condition corresponding to the same CEP rule).
Step 14: the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the target rule and the rule matching result processing logic executor corresponding to the target rule.
It should be noted that, the relevant content of step 14 is similar to that of step S3 above, and for ease of understanding, the following description is made with reference to examples.
As an example, when the above second computing node includes at least one second computing unit, step 14 may specifically be: after the second target unit in the second computing units obtains the event to be processed and the grouping key of the event to be processed, the second target unit processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the target rule and the rule matching result processing logic executor corresponding to the target rule.
Based on the above descriptions of steps 11 to 14, it can be seen that, in the event processing system (for example, a link system) provided in the embodiment of the present application, when the event processing system includes at least a first computing node, the first computing node may not only be used to perform a grouping key determination process for an event, but also be used to screen CEP rules focusing on the event from all CEP rules, so that only those CEP rules focusing on the event are used to process the event subsequently, so that occurrence of an invalid processing procedure can be reduced, and thus event processing performance of the system can be effectively improved.
Indeed, to better improve the event handling performance, the present embodiments also provide a possible implementation of the above event handling system, in which the event handling system comprises not only the above first computing node and second computing node, but also a rule updating unit. Wherein the rule updating unit is configured to perform an update process for a CEP rule stored in the event processing system (e.g., store a new CEP rule in the event processing system, or update a CEP rule stored in the event processing system with a new CEP rule, etc.) in response to a rule update operation triggered by a user. The rule update operation is used to input a new CEP rule to the event processing system.
In addition, the embodiment of the present application does not limit the working principle of the rule updating unit, and for example, it may specifically be: after the rule updating unit receives the rule to be used submitted by the user, the rule updating unit can compile (or analyze) the rule to be used to obtain at least one component corresponding to the rule to be used, and the at least one component is broadcasted and stored into each computing unit in the corresponding computing node by the rule updating unit. Wherein, the rule to be used refers to a new CEP rule configured by the user.
The above "at least one component" may include at least one of the rule event filter corresponding to the above rule to be used, the rule matching granularity key extractor corresponding to the rule to be used, the rule matching logic executor corresponding to the rule to be used, and the rule matching result processing logic executor corresponding to the rule to be used.
The above "rule event filter corresponding to a rule to be used" refers to a rule event filter obtained by compiling (or parsing) the rule to be used, so that the "rule event filter corresponding to a rule to be used" is used for filtering events of interest of the rule to be used (i.e., events satisfying the event filtering conditions) from an event stream according to event filtering conditions configured for the rule to be used, so that the events of interest thereof can be processed (for example, packet key extraction processing, processing based on rule matching logic, processing based on rule matching result processing logic, and the like) by means of the rule to be used later.
The above "rule matching granularity key extractor corresponding to a rule to be used" refers to a rule matching granularity key extractor obtained by compiling (or parsing) the rule to be used so that the "rule matching granularity key extractor corresponding to a rule to be used" is used to extract a grouping key of an event according to a rule matching granularity configured for the rule to be used so that the grouping key can represent a certain attribute of the event so that some processing (e.g., hash grouping processing, etc.) can be performed based on the grouping key later.
The "rule matching logic executor corresponding to a rule to be used" refers to a rule matching logic executor (for example, NFA) obtained by compiling the rule to be used, so that the "rule matching logic executor corresponding to the rule to be used" is used for performing event matching processing for an event according to rule matching logic configured for the rule to be used.
The above "rule matching result processing logic executor corresponding to a rule to be used" refers to a rule matching result processing logic executor (e.g., rule matching result processing function) obtained by compiling (or parsing) the rule to be used, so that the "rule matching result processing logic executor corresponding to a rule to be used" is used for processing some event matching results (e.g., "intermediate matching results" shown in fig. 4) according to rule matching result processing logic configured for the rule to be used.
In order to better understand the above rule updating unit, the following description is made in connection with an example.
As an example, in one possible implementation, when the above event processing system comprises a rule updating unit, a first computing node and a second computing node, the first computing node comprising at least one first computing unit, and the second computing node comprising at least one second computing unit, the event processing method not only comprises some of the steps described above (e.g. S1-S3 or steps 11-step 14), but also comprises the following steps 21-step 24. It should be noted that the execution time of the steps 21 to 24 is not limited in the embodiment of the present application.
Step 21: after the rule event filter corresponding to the rule to be used is obtained, the rule updating unit sends the rule event filter corresponding to the rule to be used to each first computing unit, so that each first computing unit updates the rule event filter corresponding to at least one candidate rule by using the rule event filter corresponding to the rule to be used.
It should be noted that, in the embodiment of the present application, the implementation of "the first computing unit updates the rule event filter corresponding to the at least one candidate rule with the rule event filter corresponding to the rule to be used" in the above step 21 is not limited, for example, in some application scenarios (for example, when the historical version of the rule to be used is not stored in the first computing unit), the rule event filter corresponding to the rule to be used may be directly stored in the first computing unit, so that the rule event filter corresponding to the at least one candidate rule stored in the first computing unit includes the rule event filter corresponding to the rule to be used. As another example, in other application scenarios (e.g., when a historical version of the rule to be used has been stored in the first computing unit), the rule event filter corresponding to the history rule stored in the first computing unit may be replaced with the rule event filter corresponding to the rule to be used, such that the rule event filter corresponding to the at least one candidate rule stored in the first computing unit includes the rule event filter corresponding to the rule to be used, but does not include the rule event filter corresponding to the history rule. Wherein the history rule refers to a history version of the rule to be used.
Step 22: after the rule matching granularity key extractor corresponding to the rule to be used is obtained, the rule updating unit sends the rule matching granularity key extractor corresponding to the rule to be used to each first computing unit, so that each first computing unit updates the rule matching granularity key extractor corresponding to at least one candidate rule by using the rule matching granularity key extractor corresponding to the rule to be used.
It should be noted that the embodiment of step 22 is similar to the embodiment of step 21 above.
Step 23: after the rule matching logic executor corresponding to the rule to be used is obtained, the rule updating unit sends the rule matching logic executor corresponding to the rule to be used to each second computing unit, so that each second computing unit updates the rule matching logic executor corresponding to at least one candidate rule by using the rule matching logic executor corresponding to the rule to be used.
It should be noted that the embodiment of step 23 is similar to the embodiment of step 21 above.
Step 24: after the rule matching result processing logic executor corresponding to the rule to be used is obtained, the rule updating unit sends the rule matching result processing logic executor corresponding to the rule to be used to each second computing unit, so that each second computing unit updates the rule matching result processing logic executor corresponding to at least one candidate rule by using the rule matching result processing logic executor corresponding to the rule to be used.
It should be noted that the embodiment of step 24 is similar to the embodiment of step 21 above.
Based on the above description of steps 21 to 24, for an event processing system (for example, a link system) including the above rule updating unit, the rule updating unit may broadcast and store each component corresponding to a new CEP rule into all computing units of corresponding computing nodes in the system, so as to implement dynamic update processing for the CEP rule without restarting a task, which can better improve event processing performance of the system.
Based on the event processing method provided in the embodiment of the present application, the embodiment of the present application further provides an event processing system, which is explained and illustrated in the following with reference to fig. 5. Fig. 5 is a schematic structural diagram of an event processing system according to an embodiment of the present application. It should be noted that, for technical details of the event processing system provided in the embodiments of the present application, please refer to the related content of the event processing method.
As shown in fig. 5, an event processing system 500 provided in an embodiment of the present application includes a first computing node 501 and a second computing node 502;
The first computing node 501 is configured to perform, after obtaining an event to be processed, a grouping key extraction process on the event to be processed by using a rule matching granularity key extractor corresponding to at least one candidate rule, so as to obtain a grouping key of the event to be processed;
the first computing node 501 is further configured to send the event to be processed and the grouping key to the second computing node;
the second computing node 502 is configured to process the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule, and the rule matching result processing logic executor corresponding to the at least one candidate rule.
In a possible implementation, the first computing node 501 includes at least one first computing unit; the first computing unit comprises a rule matching granularity key extractor corresponding to the at least one candidate rule; the second computing node 502 comprises at least one second computing unit; the second computing unit comprises a rule matching logic executor corresponding to the at least one candidate rule and a rule matching result processing logic executor corresponding to the at least one candidate rule;
A first target unit in the at least one first computing unit is used for carrying out grouping key extraction processing on the event to be processed by utilizing a rule matching granularity key extractor corresponding to at least one candidate rule to obtain a grouping key of the event to be processed;
the first target unit is further configured to send the event to be processed and the grouping key to a second target unit in the at least one second computing unit;
the second target unit is configured to process the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule, and the rule matching result processing logic executor corresponding to the at least one candidate rule.
In one possible embodiment, the second target unit has processed at least one event to be referenced within a historical time period; the grouping key of the event to be referred is the same as the grouping key of the event to be processed; and the end time of the historical time period is not later than the acquisition time of the event to be processed.
In a possible implementation manner, the first computing node 501 is further configured to perform screening processing on the event to be processed by using a rule event screener corresponding to the at least one candidate rule, so as to obtain a screening processing result of the event to be processed;
The first computing node 501 is specifically configured to: if the screening processing result indicates that the event to be processed meets the event screening condition corresponding to the target rule in the at least one candidate rule, performing grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to the target rule to obtain a grouping key of the event to be processed;
the second computing node 502 is specifically configured to: and processing the event to be processed according to the grouping key, the rule matching logic executor corresponding to the target rule and the rule matching result processing logic executor corresponding to the target rule.
In a possible implementation, the first computing node 501 includes at least one first computing unit; the first computing unit comprises a rule event filter corresponding to the at least one candidate rule and a rule matching granularity key extractor corresponding to the at least one candidate rule; the second computing node 502 comprises at least one second computing unit; the second computing unit comprises a rule matching logic executor corresponding to the at least one candidate rule and a rule matching result processing logic executor corresponding to the at least one candidate rule;
A third target unit in the at least one first computing unit is used for screening the event to be processed by using a rule event screening device corresponding to the at least one candidate rule to obtain a screening processing result;
the third target unit is further configured to perform grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to the target rule, so as to obtain a grouping key of the event to be processed;
the third target unit is further configured to send the event to be processed and the grouping key to a second target unit in the at least one second computing unit;
the second target unit is configured to process the event to be processed according to the grouping key, the rule matching logic executor corresponding to the target rule, and the rule matching result processing logic executor corresponding to the target rule.
In one possible embodiment, the second target unit has processed at least one event to be referenced within a historical time period; the grouping key of the event to be referred is the same as the grouping key of the event to be processed, and the screening processing result of the event to be referred is the same as the screening processing result of the event to be processed; and the end time of the historical time period is not later than the acquisition time of the event to be processed.
In one possible implementation manner, the number of the target rules is M, and M is a positive integer.
In a possible implementation manner, the event processing system further comprises a rule updating unit;
the rule updating unit is used for sending the rule event filter corresponding to the rule to be used to each first computing unit after the rule event filter corresponding to the rule to be used is obtained;
each first computing unit is further configured to update a rule event filter corresponding to the at least one candidate rule with a rule event filter corresponding to the rule to be used;
the rule updating unit is further configured to send the rule matching granularity key extractor corresponding to the rule to be used to each first computing unit after obtaining the rule matching granularity key extractor corresponding to the rule to be used;
each first computing unit is further configured to update a rule matching granularity key extractor corresponding to the at least one candidate rule with a rule matching granularity key extractor corresponding to the rule to be used;
the rule updating unit is further configured to send the rule matching logic executor corresponding to the rule to be used to each second computing unit after the rule matching logic executor corresponding to the rule to be used is obtained;
Each second computing unit is further configured to update a rule matching logic executor corresponding to the at least one candidate rule with a rule matching logic executor corresponding to the rule to be used;
the rule updating unit is further configured to send the rule matching result processing logic executor corresponding to the rule to be used to each second computing unit after obtaining the rule matching result processing logic executor corresponding to the rule to be used;
and each second computing unit is further configured to update the rule matching result processing logic executor corresponding to the at least one candidate rule by using the rule matching result processing logic executor corresponding to the rule to be used.
Based on the above description of the event processing system 500, it can be known that, for the event processing system 500 provided in the embodiment of the present application, when the system includes at least a first computing node and a second computing node, the system can complete a CEP processing procedure for an event stream by using the two nodes, and the working principle of the system is as follows: after acquiring an event to be processed (for example, a newly generated event in an event stream), firstly performing grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to at least one candidate rule by using the first computing node to obtain a grouping key of the event to be processed; and then the first computing node sends the event to be processed and the grouping key to the second computing node so that the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule, thereby realizing the purpose of simultaneously running a plurality of CEP rules in the system and effectively improving the event processing performance of the system.
In addition, the embodiment of the application also provides electronic equipment, which comprises a processor and a memory: the memory is used for storing instructions or computer programs; the processor is configured to execute the instructions or the computer program in the memory, so that the electronic device executes any implementation of the event processing method provided in the embodiments of the present application.
Referring to fig. 6, a schematic structural diagram of an electronic device 600 suitable for use in implementing embodiments of the present disclosure is shown. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 6 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.
As shown in fig. 6, the electronic device 600 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
In general, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, magnetic tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 shows an electronic device 600 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 609, or from storage means 608, or from ROM 602. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 601.
The electronic device provided by the embodiment of the present disclosure belongs to the same inventive concept as the method provided by the above embodiment, and technical details not described in detail in the present embodiment can be seen in the above embodiment, and the present embodiment has the same beneficial effects as the above embodiment.
The embodiments of the present application also provide a computer readable medium having stored therein instructions or a computer program, which when run on a device, cause the device to perform any implementation of the event processing method provided by the embodiments of the present application.
It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (Hyper Text Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the method described above.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. Where the names of the units/modules do not constitute a limitation of the units themselves in some cases.
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
It should be noted that, in the present description, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different manner from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the system or device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant points refer to the description of the method section.
It should be understood that in this application, "at least one" means one or more, and "a plurality" means two or more. "and/or" for describing the association relationship of the association object, the representation may have three relationships, for example, "a and/or B" may represent: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
It is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. An event processing method, wherein the method is applied to an event processing system, and the event processing system comprises a first computing node and a second computing node;
the method comprises the following steps:
after acquiring an event to be processed, the first computing node performs grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to at least one candidate rule to obtain a grouping key of the event to be processed;
the first computing node sends the event to be processed and the grouping key to the second computing node;
and the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule.
2. The method of claim 1, wherein the first computing node comprises at least one first computing unit; the first computing unit comprises a rule matching granularity key extractor corresponding to the at least one candidate rule; the second computing node comprises at least one second computing unit; the second computing unit comprises a rule matching logic executor corresponding to the at least one candidate rule and a rule matching result processing logic executor corresponding to the at least one candidate rule;
the first computing node performs grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to at least one candidate rule to obtain a grouping key of the event to be processed, and the grouping key comprises the following steps:
a first target unit in the at least one first computing unit performs grouping key extraction processing on the event to be processed by utilizing a rule matching granularity key extractor corresponding to at least one candidate rule to obtain a grouping key of the event to be processed;
the first computing node sends the event to be processed and the grouping key to the second computing node, including:
the first target unit sends the event to be processed and the grouping key to a second target unit in the at least one second computing unit;
The second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule, and the processing comprises the following steps:
and the second target unit processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule.
3. The method of claim 2, wherein the second target unit has processed at least one event to be referenced within a historical time period; the grouping key of the event to be referred is the same as the grouping key of the event to be processed; and the end time of the historical time period is not later than the acquisition time of the event to be processed.
4. The method according to claim 1, wherein the method further comprises:
the first computing node utilizes a rule event filter corresponding to the at least one candidate rule to filter the event to be processed to obtain a filtering result of the event to be processed;
The first computing node performs grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to at least one candidate rule to obtain a grouping key of the event to be processed, and the grouping key comprises the following steps:
if the screening processing result indicates that the event to be processed meets the event screening condition corresponding to the target rule in the at least one candidate rule, the first computing node performs grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to the target rule to obtain a grouping key of the event to be processed;
the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule and the rule matching result processing logic executor corresponding to the at least one candidate rule, and the processing comprises the following steps:
and the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the target rule and the rule matching result processing logic executor corresponding to the target rule.
5. The method of claim 4, wherein the first computing node comprises at least one first computing unit; the first computing unit comprises a rule event filter corresponding to the at least one candidate rule and a rule matching granularity key extractor corresponding to the at least one candidate rule; the second computing node comprises at least one second computing unit; the second computing unit comprises a rule matching logic executor corresponding to the at least one candidate rule and a rule matching result processing logic executor corresponding to the at least one candidate rule;
The first computing node performs screening processing on the event to be processed by using a rule event filter corresponding to the at least one candidate rule to obtain a screening processing result, which comprises the following steps:
a third target unit in the at least one first computing unit screens the event to be processed by using a rule event screening device corresponding to the at least one candidate rule to obtain a screening result;
the first computing node performs grouping key extraction processing on the event to be processed by using a rule matching granularity key extractor corresponding to the target rule to obtain a grouping key of the event to be processed, and the method comprises the following steps:
the third target unit performs grouping key extraction processing on the event to be processed by utilizing a rule matching granularity key extractor corresponding to the target rule to obtain a grouping key of the event to be processed;
the first computing node sends the event to be processed and the grouping key to the second computing node, including:
the third target unit sends the event to be processed and the grouping key to a second target unit in the at least one second computing unit;
the second computing node processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the target rule and the rule matching result processing logic executor corresponding to the target rule, and the processing comprises the following steps:
And the second target unit processes the event to be processed according to the grouping key, the rule matching logic executor corresponding to the target rule and the rule matching result processing logic executor corresponding to the target rule.
6. The method of claim 5, wherein the second target unit has processed at least one event to be referenced over a historical period of time; the grouping key of the event to be referred is the same as the grouping key of the event to be processed, and the screening processing result of the event to be referred is the same as the screening processing result of the event to be processed; the end time of the history time period is not later than the acquisition time of the event to be processed;
and/or the number of the groups of groups,
the number of the target rules is M, and M is a positive integer.
7. The method of claim 5, wherein the event processing system further comprises a rule updating unit;
the method further comprises the steps of:
after the rule event filter corresponding to the rule to be used is obtained, the rule updating unit sends the rule event filter corresponding to the rule to be used to each first computing unit; each first computing unit updates a rule event filter corresponding to the at least one candidate rule by using the rule event filter corresponding to the rule to be used;
After the rule matching granularity key extractor corresponding to the rule to be used is obtained, the rule updating unit sends the rule matching granularity key extractor corresponding to the rule to be used to each first computing unit; each first computing unit updates a rule matching granularity key extractor corresponding to the at least one candidate rule by using the rule matching granularity key extractor corresponding to the rule to be used;
after the rule matching logic executor corresponding to the rule to be used is obtained, the rule updating unit sends the rule matching logic executor corresponding to the rule to be used to each second computing unit; each second computing unit updates a rule matching logic executor corresponding to the at least one candidate rule by using the rule matching logic executor corresponding to the rule to be used;
after the rule matching result processing logic executor corresponding to the rule to be used is obtained, the rule updating unit sends the rule matching result processing logic executor corresponding to the rule to be used to each second computing unit; and updating the rule matching result processing logic executor corresponding to the at least one candidate rule by each second computing unit by using the rule matching result processing logic executor corresponding to the rule to be used.
8. An event processing system, the event processing system comprising a first computing node and a second computing node;
the first computing node is used for carrying out grouping key extraction processing on the event to be processed by utilizing a rule matching granularity key extractor corresponding to at least one candidate rule after the event to be processed is acquired, so as to obtain a grouping key of the event to be processed;
the first computing node is further configured to send the event to be processed and the grouping key to the second computing node;
the second computing node is configured to process the event to be processed according to the grouping key, the rule matching logic executor corresponding to the at least one candidate rule, and the rule matching result processing logic executor corresponding to the at least one candidate rule.
9. An electronic device, the device comprising: a processor and a memory;
the memory is used for storing instructions or computer programs;
the processor for executing the instructions or computer program in the memory to cause the electronic device to perform the method of any of claims 1-7.
10. A computer readable medium, characterized in that it has stored therein instructions or a computer program which, when run on a device, causes the device to perform the method of any of claims 1-7.
CN202211730095.3A 2022-12-30 2022-12-30 Event processing method, system, electronic device and computer readable medium Pending CN116049234A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211730095.3A CN116049234A (en) 2022-12-30 2022-12-30 Event processing method, system, electronic device and computer readable medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211730095.3A CN116049234A (en) 2022-12-30 2022-12-30 Event processing method, system, electronic device and computer readable medium

Publications (1)

Publication Number Publication Date
CN116049234A true CN116049234A (en) 2023-05-02

Family

ID=86119400

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211730095.3A Pending CN116049234A (en) 2022-12-30 2022-12-30 Event processing method, system, electronic device and computer readable medium

Country Status (1)

Country Link
CN (1) CN116049234A (en)

Similar Documents

Publication Publication Date Title
CN111581291A (en) Data processing method and device, electronic equipment and readable medium
CN111324342B (en) Method, device, medium and electronic equipment for generating interface layer code
CN114422267B (en) Flow detection method, device, equipment and medium
CN110390493B (en) Task management method and device, storage medium and electronic equipment
US20210200806A1 (en) Method and apparatus for parallel processing of information
CN111628938B (en) Branch merging method, device, electronic equipment and computer storage medium
CN111858381A (en) Application program fault tolerance capability test method, electronic device and medium
CN111857720A (en) Method and device for generating user interface state information, electronic equipment and medium
CN116629984A (en) Product information recommendation method, device, equipment and medium based on embedded model
CN116860286A (en) Page dynamic update method, device, electronic equipment and computer readable medium
CN116049234A (en) Event processing method, system, electronic device and computer readable medium
CN112948341B (en) Method and apparatus for identifying anomalous network device logs
CN112507676B (en) Method and device for generating energy report, electronic equipment and computer readable medium
CN115145964A (en) Time sequence data integration method, device, equipment and medium
CN111309323B (en) Parameter initialization method and device and electronic equipment
CN111209042B (en) Method, device, medium and electronic equipment for establishing function stack
CN110941683B (en) Method, device, medium and electronic equipment for acquiring object attribute information in space
CN112115154A (en) Data processing and data query method, device, equipment and computer readable medium
CN112799797A (en) Task management method and device
CN110908867B (en) Task completion judging method and device, storage medium and electronic equipment
CN116303529A (en) Object acquisition method, device, electronic equipment and computer readable medium
CN113392361B (en) Service processing method, device, electronic equipment and computer readable medium
CN116166856A (en) Processing method, device, equipment and storage medium of table data
US10931761B2 (en) Interconnecting nodes of entity combinations
CN117520399A (en) Data storage method, apparatus, electronic device, and computer readable medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination