WO2019214685A1 - 一种消息的处理方法、装置和系统 - Google Patents
一种消息的处理方法、装置和系统 Download PDFInfo
- Publication number
- WO2019214685A1 WO2019214685A1 PCT/CN2019/086218 CN2019086218W WO2019214685A1 WO 2019214685 A1 WO2019214685 A1 WO 2019214685A1 CN 2019086218 W CN2019086218 W CN 2019086218W WO 2019214685 A1 WO2019214685 A1 WO 2019214685A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- processing
- message
- node
- processing node
- rule
- Prior art date
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 15
- 238000007781 pre-processing Methods 0.000 claims abstract description 122
- 238000000034 method Methods 0.000 claims abstract description 61
- 230000008569 process Effects 0.000 claims abstract description 35
- 230000004044 response Effects 0.000 claims abstract description 20
- 238000012098 association analyses Methods 0.000 claims description 127
- 238000004590 computer program Methods 0.000 claims description 8
- 238000012797 qualification Methods 0.000 description 24
- 238000004458 analytical method Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000007726 management method Methods 0.000 description 7
- 230000008859 change Effects 0.000 description 6
- 238000012097 association analysis method Methods 0.000 description 5
- 238000010219 correlation analysis Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
- H04L41/0636—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis based on a decision tree analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/02—Standardisation; Integration
- H04L41/0246—Exchanging or transporting network management information using the Internet; Embedding network management web servers in network elements; Web-services-based protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/30—Routing of multiclass traffic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/64—Routing or path finding of packets in data switching networks using an overlay routing layer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/30—Definitions, standards or architectural aspects of layered protocol stacks
- H04L69/32—Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
- H04L69/322—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions
- H04L69/329—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions in the application layer [OSI layer 7]
Definitions
- the present application relates to data processing, for example, to a method, apparatus and system for processing a message.
- streaming processing has been widely used in e-commerce, news aggregation, social networking sites, data mining and machine learning.
- Streaming processing has low latency, high performance, distributed, scalable and fault-tolerant features, which can be easily done. Extensions to threads, processes, and servers.
- Figure 1 shows the system framework of streaming processing.
- messages are processed from at least one data source and processed by multiple nodes.
- S1 and S2 in the figure are data sources, and B1 to B5 are used to process messages.
- the advantage of streaming is the ability to extend the processing of messages.
- Each stage of message processing can be extended, including in-process extensions and inter-process extensions. In this way, under the condition of sufficient hardware, the infinite expansion of message processing capability can be achieved.
- An embodiment of the present application provides a method for processing a message, including: a pre-processing node, in response to receiving a first message of an event sent by a data source, searching for a processing object in a rule instance that has been created by the system, where the processing object includes the a rule instance of the first message; the pre-processing node establishes a processing path according to the found rule instance, where the processing path includes addressing information of the processing node where the found rule instance is located; and the pre-processing node according to the processing a path, the first message is sent to the processing node, to process the first message by the searched rule instance at the processing node.
- the embodiment of the present application further provides a method for processing a message, including: a processing node receiving a message of an event sent by a pre-processing node or a previous processing node; and the processing node responding to determining that the message is the one event
- the first message processes the first message by using a corresponding rule instance on the processing node.
- the embodiment of the present application further provides a message preprocessing node, including: a message receiving module, configured to receive a message sent by a data source; and an instance searching module, configured to: after the message receiving module receives the first message of an event, Finding a processing object in a rule instance that has been created by the system, the processing object includes a rule instance of the first message, and the path establishing module is configured to establish a processing path according to the found rule instance, where the processing path includes the searching The address instance to which the rule instance is located is located; the message sending module is configured to send the first message to the processing node according to the processing path.
- a message preprocessing node including: a message receiving module, configured to receive a message sent by a data source; and an instance searching module, configured to: after the message receiving module receives the first message of an event, Finding a processing object in a rule instance that has been created by the system, the processing object includes a rule instance of the first message, and the path establishing module is configured to establish a processing path
- the embodiment of the present application further provides a message processing system, including a data source, a preprocessing node, and a processing node, where: the data source is configured to send all messages of an event to the preprocessing node;
- the pre-processing node is the pre-processing node described in the foregoing embodiment;
- the processing node is configured to receive a message of the one event sent by the pre-processing node, such as determining that the message is the first message of the one event, The first message is processed using the found rule instance on the processing node.
- the embodiment of the present application further provides a message processing apparatus, including a memory, a processor, and a computer program stored on the memory and operable on the processor, where the processor implements the computer program A method as in the embodiments of the present application.
- the embodiment of the present application further provides a computer readable storage medium having stored thereon a computer program, which is executed by a processor to implement a method as in the embodiment of the present application.
- FIG. 1 is a schematic diagram of streaming processing
- FIG. 3 is a block diagram of a preprocessing node according to Embodiment 1 of the present application.
- FIG. 4 is a flowchart of a message processing method performed by a processing node according to Embodiment 1 of the present application;
- FIG. 5 is a schematic diagram of a staged processing of a message according to Embodiment 2 of the present application.
- FIG. 6 is a flowchart of performing association analysis on a message according to Embodiment 2 of the present application.
- FIG. 7 is a schematic diagram of association analysis of an alarm message according to Embodiment 3 of the present application.
- FIG. 8 is a schematic diagram of association analysis of an alarm message according to Embodiment 4 of the present application.
- FIG. 9 is a schematic diagram of association analysis of an alarm message according to Embodiment 5 of the present application.
- FIG. 10 is a schematic diagram of association analysis of an alarm message according to Embodiment 6 of the present application.
- FIG. 11 is a schematic diagram of association analysis of news data according to Embodiment 7 of the present application.
- This embodiment provides a method for processing a message, which can dynamically calculate a processing path of a message in a streaming processing mode, and implement message distribution and processing according to the processing path.
- the processing system of the message in this embodiment can refer to FIG. 1, and adopts a streaming processing framework such as a jstorm stream processing framework, including a data source and a node for processing a message.
- the nodes are classified into two types, one is called a preprocessing node, and the other is called a preprocessing node.
- a class is called a processing node.
- the pre-processing node is configured to receive a message of the event from the data source, establish a processing path of the message, and distribute the message to the processing node, and after receiving the message, the processing node processes the message by using the rule instance established according to the processing rule.
- the processing method of the message in this embodiment includes step 110, step 120, and step 130.
- step 110 the pre-processing node, in response to receiving the first message of an event sent by the data source, looks up a processing object in the rule instance that the system has created, the processing object including a rule instance of the first message.
- an event is used only to refer to an event and does not refer to a particular event.
- the first message of the event is the first message of the event that the data source collected in chronological order.
- the rule instance includes a rule instance created according to an association analysis rule, the one event is an alarm event, the first message of an event is a generation message of an alarm event, and the processing of the message is an association analysis of the message.
- the event of the present application is not limited thereto, and may be an action performed by the device, a thing that occurs, and the like.
- the message of the event may be any data related to the event, and the message of the same event carries the key value of the event, such as a keyword, a name, an identifier, etc., and all the messages of the same event can be identified by the key value.
- the alarm of the network management is the network element.
- the description information is the time and description of the alarm.
- the key information can be defined as the time when the news is generated.
- the description information is the person and description of the news.
- the definition of specific key information should be defined by the application according to the actual application scenario.
- the pre-processing node and the processing node are both logical nodes, and the rule instance is created according to the processing rule, and the business processing logic that can be executed on the processing node to execute the processing rule.
- the pre-processing node and the processing node may be implemented with resource elements having processing capabilities, such as processes or threads, while the rule instance is a program running on the processing node.
- a rule instance needs to define the processing object of the rule instance.
- a rule instance that performs association analysis on a message for an alarm event can define a processing object by a qualified condition, such as by specifying a network element identifier.
- the alarm code the processing object is defined as an alarm event generated by the specified network element with the specified alarm code.
- the processing object information of the rule instance that has been created by the system is saved as described above, and other information such as the identifier, parameter, and type of the rule instance may be saved.
- the information may also be stored in other nodes, and the pre-processing node reads the information from other nodes when needed.
- the created rule instance is created according to at least one processing rule, and each processing rule corresponds to a service scenario and has different service processing logic.
- a processing rule at least one rule instance may be created, and the processing objects and parameters of different rule instances may be different.
- at least one rule instance can be run on a processing node.
- the processing node can determine which rule instance to run for processing according to the identifier of the rule instance written in the message.
- the processing nodes that are created according to different processing rules are different in processing nodes.
- the created rule instances may be divided into at least one group according to the processing rule, and the rule instances created according to the same processing rule are grouped into one group.
- a plurality of rule instances created according to the same processing rule only one rule instance processes an event, so that one event has only one branchless association analysis path, and only one processing result. But this is not a must.
- step 120 the pre-processing node establishes a processing path according to the found rule instance, where the processing path includes addressing information of the processing node where the found rule instance is located.
- the addressing information of the processing node where the found rule instance is located is represented by information that can uniquely identify the rule instance.
- the addressing information of the processing node where the found rule instance is located includes the identifier of the found rule instance; or the addressing information of the processing node where the found rule instance is located includes the found rule.
- the ID of the instance is created according to the at least one processing rule.
- the types of the rule instances created according to the same processing rule are the same.
- the types of the rule instances created according to different processing rules are different. Because the processing node where each rule instance is located is configured in the system, the information that uniquely identifies the rule instance can be used as the addressing information to ensure that the result of each addressing is the same.
- the specific addressing method can be various, for example, the rule.
- the identifier and type of the instance are hashed, and the corresponding relationship between the hash value and the processing node is found according to the obtained hash value, thereby determining the processing node where the rule instance is located.
- These addressing methods can be implemented using the mechanisms provided by the streaming framework.
- step 130 the pre-processing node sends the first message to the processing node according to the processing path, to process the first message by the found rule instance.
- processing the first message includes performing association analysis on the first message, but the application is not limited thereto, and may be various other processes.
- the pre-processing node sends the first message to the processing node, and writes the parameter and identifier of the found rule instance into the first message, and Sending the first message to the processing node, the parameters and identifiers of the found rule instance may be used when processing the first message.
- the pre-processing node may send the first message to the multiple processing nodes in a broadcast manner, and if necessary, simultaneously send parameters of the rule instance.
- the serial distribution mode is adopted, that is, the pre-processing node sends the first message to the first processing node in the processing path, and each processing node in the processing path is sent to the next processing node.
- the pre-processing node sends addressing information of the first message and all processing nodes after the first processing node in the processing path to the first processing node; each of the first message is received Processing nodes process the first message using the found rule instance on each processing node, and in response to determining that each processing node is not the last one of the plurality of processing nodes, the first The addressing information of the message and all processing nodes subsequent to the next processing node in the processing path is sent to the next processing node.
- the addressing information sent by the foregoing pre-processing node and the processing node is required for the next processing node to continue to transmit, but should not be understood as only transmitting the addressing information.
- the pre-processing node may also process all processing in the processing path.
- the addressing information of the node is sent to the first processing node, and after the first processing node finds its own addressing information, the addressing information of the subsequent processing node can be determined to implement the continuous transmission of the message.
- the preprocessing node saves the processing path, and in response to receiving a subsequent message of the one event sent by the data source, Sending the subsequent message to the processing node according to the saved processing path.
- the specific sending mode is similar to the sending mode of the first message, but the processing node may not process these subsequent events when performing the association analysis. Of course, if it is other business processing logic, it can also process subsequent events.
- all the messages of an event are collected and distributed to the same pre-processing node by the same data source by using the mechanism supported by the stream processing framework.
- the present application implements the processing of the message according to the created rule instance by establishing a processing path of the message and distributing it according to the processing path.
- the embodiment further provides a pre-processing node for the message. As shown in FIG. 3, the message receiving module 10, the instance searching module 20, the path establishing module 30, and the message sending module 40 are provided.
- the message receiving module 10 is configured to receive a message sent by the data source.
- the instance lookup module 20 is configured to: after the message receiving module receives the first message of an event, look up a processing object in a rule instance that has been created by the system, where the processing object includes a rule instance of the first message.
- the path establishing module 30 is configured to establish a processing path according to the found rule instance, where the processing path includes addressing information of the processing node where the found rule instance is located.
- the message sending module 40 is configured to send the first message to the processing node for processing according to the processing path.
- the path establishing module is configured to save the processing path after establishing the processing path
- the message sending module is further configured to receive, by the message receiving module, the data source. After the subsequent message of an event is described, the subsequent message is sent to the processing node according to the saved processing path.
- the created rule instance is different according to the processing node where the rule instance created according to the different processing rule is created according to the at least one processing rule.
- the created rule instances are divided into at least one group, and the rule instances created according to the same processing rule are grouped into one group; the instance lookup module searches for the processing object in the created rule instance, and the processing is performed.
- the object includes a rule instance of the first message, including: the pre-processing node looks up a rule instance in each group, and searches in each group for at most one processing object including a rule instance of the one event.
- the addressing information of the processing node where the found rule instance is located includes the identifier of the found rule instance, or includes the The identification and type of the rule instance, wherein the created rule instance is created according to at least one processing rule, and the rule instance created according to the same processing rule has the same type, and the type of the rule instance created according to different processing rules is different.
- the sending, by the message sending module, the first message to the processing node includes: writing the parameter and identifier of the found rule instance into the first message, and sending the first message to The processing node.
- the processing path established by the path establishing module includes addressing information of multiple processing nodes; the message sending module is configured to: the first message and the first processing node in the processing path The addressing information of all the processing nodes is sent to the first processing node, and the first message carries the parameters and identifiers of the found rule instance.
- the rule instance includes a rule instance created according to an association analysis rule.
- This embodiment further provides a message processing method performed by the processing node. As shown in FIG. 4, the method includes step 210 and step 220.
- step 210 the processing node receives a message of an event sent by the pre-processing node or the last processing node.
- step 220 the processing node processes the first message with a corresponding rule instance on the processing node in response to determining that the message is the first message of the one event.
- the corresponding rule instance on the processing node in this step is the rule instance found by the pre-processing node in the above, that is, the processing object includes a rule instance of an event.
- an event is an alarm event, and the rule instance only needs to perform association analysis on the first message of one event, and only distributes other messages of one event.
- the application is not limited to this.
- the message is sent in a serial distribution manner, and the processing node receives a message of the one event, and receives a subsequent processing node in the processing path of the one event (refers to a processing node after the processing node) Addressing information; the processing node saves after receiving the addressing information, and sends the first message to the subsequent processing node according to the addressing information; the processing node receives the event And following the message, sending the subsequent message to the subsequent processing node according to the saved addressing information.
- the method further includes: generating the event in response to processing the first message.
- the new message can be sent to the data source responsible for collecting the message of the one event according to its business processing logic.
- the message of an event is still processed sequentially by the original data source, the preprocessing node, and the processing node on the processing path.
- the embodiment further provides a message processing system, including a data source, a preprocessing node, and a processing node, where: the data source is configured to send all messages of an event to the same preprocessing node; a pre-processing node as described above; the processing node, configured to receive a message of the one event sent by the pre-processing node, in response to determining that the message is a first message of the one event, utilizing The found rule instance on the processing node processes the first message.
- the data source is configured to send all messages of an event to the same preprocessing node
- a pre-processing node as described above
- the processing node configured to receive a message of the one event sent by the pre-processing node, in response to determining that the message is a first message of the one event, utilizing The found rule instance on the processing node processes the first message.
- the processing node receives the addressing information of the subsequent processing node in the processing path of the one event in response to receiving the message of the one event; the processing node determines that the message is the one event After the first message, further configured to save the addressing information of the subsequent processing node, send the first message to the subsequent processing node according to the addressing information; and, in response to receiving the subsequent message of the one event, according to saving The addressing information is sent to the subsequent processing node.
- the method further includes: generating the event in response to processing the first message.
- a new message the new message is sent to a data source responsible for collecting the message of the one event.
- the rule instance includes a rule instance created according to an association analysis rule.
- the embodiment further provides a message processing apparatus comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor executes the computer program Any processing as in the method of the present embodiment can be achieved.
- the message processing of this embodiment adopts streaming processing, and the processing path of the dynamically generated message is used for message distribution. And the messages of the same event are distributed according to the same processing path, and the sequential processing of all messages of the same event can be guaranteed.
- This embodiment relates to a method for association analysis of messages.
- the association analysis path of the message is dynamically calculated, and the association analysis is performed on the message distribution according to the association analysis path, and a method for ensuring that all messages of the same event are processed in order is provided.
- the network management is used as an application scenario.
- the scenario involves a large number of alarm events. Each alarm event generates multiple messages.
- the messages can be classified into: generating a message, generating a message when the event is generated; and subsequent messages are generated after the event is generated. Messages such as event change messages, clear messages, etc.
- association analysis is performed on the first message of the event, that is, the relationship between the multiple events is analyzed, so that the user can quickly obtain useful information.
- association analysis methods are defined as association rules. Therefore, different scenarios can correspond to different association rules.
- a rule instance defines the following.
- qualification It is used to define the processing object of the rule. It can also be defined as the effective scope of the definition rule.
- the qualification condition defines which events the rule takes effect and can be specified by the user. For example, the user can specify at least one network element and at least one alarm code. At least one alarm level or the like is a qualification condition of the rule instance.
- Business processing logic refers to the processing logic when performing association analysis on messages that meet the qualification conditions.
- Parameter The input parameter of the rule instance to perform the association analysis, which can be specified by the user, for example, the time when the alarm is delayed.
- the user can create at least one rule instance according to an association rule, and the qualifications and parameters of each rule instance are specified by the user.
- a message that meets the qualification condition of a certain rule instance that is, the message is a processing object of the rule instance
- the associated analysis node where the rule instance is located, and the relationship between the events is obtained. And finally show it to the user.
- the association analysis method of the message in this embodiment is divided into two stages.
- the preprocessing stage it is determined whether the first message of the event meets the qualification condition of at least one rule instance that has been created in the system.
- the event-based first message matches the determination result of the qualification condition of at least one rule instance that has been created in the system, creates an association analysis path, and distributes the first message of the event and the parameter of the rule instance to the corresponding association analysis node according to the association analysis path.
- Two-stage processing the event-based first message does not meet the judgment result of the qualification condition of at least one rule instance that has been created in the system, and the event will not be processed by the rule instance.
- the rule instance on the association analysis node performs association analysis on the first message of the received event according to the business processing logic and parameters.
- the association analysis method of the message in this embodiment is as shown in FIG. 6. It is assumed that the user has created a plurality of rule instances for performing association analysis in the system, and the method includes steps 310 to 390.
- step 310 the data source collects a message of the event and sends it to the pre-processing node.
- This embodiment is based on a streaming processing framework. Events are collected in multiple data sources, and all messages of the same event are collected and distributed in the same data source.
- step 320 the pre-processing node determines the type of the message, and based on the judgment result of the message generation type being the event generation message, the process proceeds to step 340, and the process proceeds to step 330 based on the determination result that the message type is not the event generation message.
- step 330 the association analysis path of the event is obtained from the cache according to the key value of the message, and the process proceeds to step 360.
- step 340 the created rule instances are obtained, and the rule instances are divided into multiple groups according to the corresponding association rules, wherein the rule instances created according to the same association rule are divided into one group; the messages are sequentially and the qualification conditions of each group of rule instances are defined. Comparing, in each set of rule instances, finding at most one processing object includes a rule instance of the message, sequentially arranging the identifiers and types of all the found rule instances, establishing an association analysis path, and setting parameters of the rule instance, Basic information such as the identity of the rule instance is written to the message.
- the processing object includes a rule instance of the message, meaning that the message conforms to the qualification condition of the rule instance.
- the identifiers and types of the multiple rule instances may be arranged according to a preset sorting manner, for example, the identifier and type of the rule instance A, the identifier and type of the rule instance B, and the rule instance.
- the identification and type of a rule instance is the addressing information of the associated analysis node where the rule instance is located.
- the specific addressing process can be implemented by the mechanism provided by the streaming framework.
- the parameters of the rule instance can also be sent as separate information, depending on the format of the information sent.
- step 350 the pre-processing node associates the association analysis path with the key value of the message and stores it in the cache for use by subsequent messages.
- step 360 the message is sent to the next associated analysis node in the associated analysis path, and the addressing information of all associated analysis nodes after the next associated analysis node in the associated analysis path is sent with the message.
- the next association analysis node is the first association analysis node in the association analysis path.
- the next association analysis node is the first association analysis node after the association analysis node in the association analysis path.
- step 370 the association analysis node that receives the message determines the type of the message, and based on the determination result of the message generation type being the event generation message, proceeds to step 380, and proceeds to the step based on the determination result of the message generation type not being the event generation message. 390.
- step 380 the association analysis node that receives the message obtains the parameters of the rule instance carried in the message, and performs association analysis on the message by using the rule instance running on the message.
- step 390 the associated sending node that receives the message determines whether there is addressing information of the next analyzing node in the associated analysis path, and the processing ends based on the determination result of the addressing information without the next node; The result of the addressing information of a node is returned to step 360.
- the system has created multiple rule instances and association rules based on the same association rule.
- the reported alarm messages will match one of the rule instances.
- the process flow includes steps 1 through 12.
- step one the data source S1 receives the alert message A1.
- the data source can be an alarm collection module on the network element.
- step 2 the data source S1 distributes the alarm message A1 to the pre-processing node P2 according to the network element information of the alarm message A1.
- step 3 after receiving the alarm message A1, the pre-processing node P2, through the pre-processing of the association analysis, finds that the alarm message A1 meets the qualification condition of the rule instance M1, and generates an association analysis path [M1].
- the association analysis path [M1] represents the addressing information of the association analysis node M1, and can be specifically represented by the identifier and type of the rule instance, M represents the type, and 1 represents the ID, and other subsequent embodiments are the same.
- step 4 the pre-processing node P2 sends the alarm message A1 to the association analysis node M1 according to the association analysis path, and then continues the subsequent processing flow.
- association analysis node and the rule instance on it use the same label.
- the association analysis node where the rule instance M1 is located is represented as the association analysis node M1.
- the processing is distributed to the association analysis node M1, that is, the association analysis of the alarm message A1 is performed by the rule instance M1 on the association analysis node M1.
- Other embodiments are the same.
- step five the data source S2 receives the alert message A2.
- step 6 the data source S2 distributes the alarm message A2 to the pre-processing node P3 according to the network element information of the alarm message A2.
- step 7 after receiving the alarm message A2, the pre-processing node P3 finds that the alarm message A2 meets the qualification condition of the rule instance M1 by the pre-processing of the association analysis, thereby generating the association analysis path [M1] of the alarm message A2.
- step 8 the pre-processing node P3 sends the alarm message A2 to the association analysis node M1 according to the association analysis path, and then continues the subsequent processing flow.
- step 9 data source S1 receives alert message A3.
- step 10 the data source S1 distributes the alarm message A3 to the pre-processing node P1 according to the network element information of the alarm message A3.
- the pre-processing node P1 After receiving the alarm message A3, the pre-processing node P1 finds that the alarm message A3 meets the qualification condition of the rule instance M1 by the pre-processing of the association analysis, thereby generating the association analysis path [M1] of the alarm message A3.
- step 12 the pre-processing node P1 sends the alarm message A3 to the association analysis node M1 according to the association analysis path, and then continues the subsequent processing flow.
- a plurality of sets of rule instances are created according to various association rules.
- M1 to M3 are a group
- R1 to R3 are a group
- C1 to C3 are a group
- the reported alarm messages are all met.
- the qualification of at least one rule instance, the process flow includes steps 1 to 12.
- step one the data source S1 receives the alert message A1.
- step 2 the data source S1 distributes the alarm message A1 to the pre-processing node P2 according to the network element information of the alarm message A1.
- step 3 after receiving the alarm message A1, the pre-processing node P2, through the pre-processing of the association analysis, finds that the alarm message A1 meets the qualification conditions of the rule instances M1, R2, and C2, thereby generating an association analysis path of the alarm message A1 [M1 , R2, C2].
- step 4 the pre-processing node P2 sends the alarm message A1 to the association analysis node M1 according to the association analysis path, and M1 sends the message to the association analysis node R2 for processing, and R2 sends the message to the association analysis node C2 for processing, and then continues to follow. Process flow.
- step five the data source S2 receives the alert message A2.
- step 6 the data source S2 distributes the alarm message A2 to the pre-processing node P3 according to the network element information of the alarm message A2.
- step 7 after receiving the alarm message A2, the pre-processing node P3, through the pre-processing of the association analysis, finds that the alarm message A2 meets the qualification conditions of the rule instances M2, R2, and C1, thereby generating an association analysis path of the alarm message A2 [M2 , R2, C1].
- step 8 the pre-processing node P3 sends the alarm message A2 to the association analysis node M2 according to the association analysis path, and M2 sends the message to the association analysis node R2 for processing, and R2 sends the message to the association analysis node C1 for processing, and then Continue the subsequent processing.
- step 9 data source S1 receives alert message A3.
- step 10 the data source S1 distributes the alarm message A3 to the pre-processing node P1 according to the network element information of the alarm message A3.
- the pre-processing node P1 After receiving the alarm message A3, the pre-processing node P1 finds that the alarm message A3 meets the qualification conditions of the rule instances M1 and C2 through the pre-processing of the association analysis, thereby generating the association analysis path [M1, C2].
- step 12 the pre-processing node P1 sends the alarm message A3 to the association analysis node M1 according to the association analysis path, and the M1 sends the message to the association analysis node C2 for processing, and then continues the subsequent processing flow.
- step one the data source S1 receives the alert message A1.
- step 2 the data source S1 sends the alarm message A1 to the pre-processing node P2 according to the network element information generating the alarm message A1.
- step 3 the pre-processing node P2 finds that the alarm message A1 meets the qualification condition of the rule instance M1 by the pre-processing of the association analysis, and generates an association analysis path [M1] of the alarm message A1.
- step 4 the pre-processing node P2 sends the alarm message A1 to the association analysis node M1 according to the association analysis path, and then continues the subsequent processing flow.
- step 5 the data source S1 receives the change message C1 of the alarm to which the alarm message A1 belongs.
- step 6 the data source S1 sends C1 to the pre-processing node P2 for processing according to the network element information generated by C1.
- step 7 the pre-processing node P2 obtains the association analysis path [M1] of the alarm message A1 by querying.
- step 8 the pre-processing node P2 sends C1 to the associated processing node M1 for processing according to the association analysis path, and then continues the subsequent processing flow.
- step IX the data source S1 receives the clear message D1 of the alarm to which the alarm message A1 belongs.
- step 10 the data source S1 sends D1 to the pre-processing node P2 for processing according to the network element information generated by D1.
- step 11 the pre-processing node P2 obtains the association analysis path [M1] of the alarm message A1 by querying.
- step 12 the pre-processing node P2 sends the D1 to the association analysis node M1 according to the association analysis path, and then continues the subsequent processing flow.
- a new message may be generated during the process of correlating the alarm message. This example illustrates the related process for this case.
- step one the data source S1 receives the alert message A1.
- step 2 the data source S1 sends the alarm message A1 to the pre-processing node P2 according to the network element information generating the alarm message A1.
- step 3 the pre-processing node P2 finds that the alarm message A1 meets the qualification condition of the rule instance M1 by the pre-processing of the association analysis, and generates an association analysis path [M1] of the alarm message A1.
- step 4 the pre-processing node P2 sends an alarm message A1 to the association analysis node M1 according to the association analysis path. After the alarm message A1 is processed by the association analysis node M1, a new alarm message A1 to which the alarm message A1 belongs is generated. .
- step five the association analysis node M1 sends a change message C1 to the data source S1 for processing.
- step 6 the data source S1 distributes the change message C1 according to the network element information generating the alarm message A1, and sends the change message C1 to the pre-processing node P2 for processing.
- step 7 the pre-processing node P2 finds the association analysis path [M1] of the alarm message A1.
- step 8 the pre-processing node P2 sends the change message C1 to the association analysis node M1 according to the found association analysis path, and then continues the subsequent processing flow.
- each association analysis node can send messages to the data through message middleware.
- the message of this embodiment is news data, and the embodiment relates to association analysis of news data.
- multiple data sources in the system can obtain news data, each news data belongs to a different event, and is identified by a different key value.
- the processing flow includes steps 1 to 12.
- step one the data source S1 acquires the news data N1.
- step two the data source S1 distributes the news data N1 to the pre-processing node P2 according to the key value of the news data N1.
- step 3 the pre-processing node P2 finds that the news data N1 meets the qualification conditions of the rule instances M1 and E3 by the pre-processing of the association analysis, and generates the association analysis path [M1, E3] of the news data N1.
- step 4 the pre-processing node P2 sends the news data N1 to the association analysis node M1 according to the association analysis path, and the association analysis node M1 sends the message to the association analysis node E3 for processing, and then continues the subsequent processing flow.
- step five the data source S3 obtains the news data N2.
- step 6 the data source S3 is distributed to the pre-processing node P1 for processing according to the key value of the news data N2.
- step 7 the pre-processing node P1 finds that the news data N2 meets the qualification conditions of the rule instances M2 and E3 by the pre-processing of the association analysis, and generates the association analysis path [M2, E3] of the news data N1.
- step 8 the pre-processing node P1 sends the news data N2 to the association analysis node M2 according to the association analysis path, and the association analysis node M2 sends the news data N2 to the association analysis node E3 for processing, and then continues the subsequent processing flow.
- step 9 the data source S2 acquires the news data N3.
- step 10 the data source S2 distributes the news data N3 to the pre-processing node P1 for processing according to the key value of the news data N3.
- step 11 the pre-processing node P1 finds that the news data N3 meets the qualification conditions of the rule instances M1 and E1 by the pre-processing of the association analysis, and generates the association analysis path [M1, E1] of the news data N3.
- step 12 the pre-processing node P1 sends the news data N3 to the association analysis node M1 according to the association analysis path, and the association analysis node M1 sends the news data N3 to the association analysis node E1 for processing, and then continues the subsequent processing flow.
- the news data of the same event may also have a subsequent additional report, which is equivalent to changing the message, and can be processed in the same manner as the above embodiment.
- Such software may be distributed on a computer readable medium, which may include computer storage media (or non-transitory media) and communication media (or transitory media).
- computer storage medium includes volatile and nonvolatile, implemented in any method or technology for storing information, such as computer readable instructions, data structures, program modules or other data. Sex, removable and non-removable media.
- the computer storage medium includes, but is not limited to, a random access memory (RAM), a read only memory (ROM), and an electrically erasable programmable read-only memory (EEPROM).
- Flash memory or other memory technology
- CD-ROM Compact Disc Read-Only Memory
- DVD Digital Video Disc
- magnetic cassette magnetic tape
- disk storage or other magnetic storage device Or any other medium that can be used to store the desired information and that can be accessed by the computer.
- communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and can include any information delivery media. .
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer And Data Communications (AREA)
- Debugging And Monitoring (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
一种消息的处理方法、装置和系统,预处理节点响应于接收数据源发送的一个事件的首消息,在系统已创建的规则实例中查找处理对象,所述处理对象包括所述首消息的规则实例;所述预处理节点根据查找到的规则实例建立处理路径,所述处理路径包括所述查找到的规则实例所在处理节点的寻址信息;所述预处理节点根据所述处理路径,将所述首消息发送给所述处理节点,以由所述查找到的规则实例对所述首消息进行处理。
Description
本申请要求在2018年05月09日提交中国专利局、申请号为201810438739.9的中国专利申请的优先权,该申请的全部内容通过引用结合在本申请中。
本申请涉及数据处理,例如涉及一种消息的处理方法、装置和系统。
目前流式处理已经被大量用于电子商务、新闻聚合、社交网站、数据挖掘和机器学习等领域,流式处理具有低延迟、高性能、分布式、可扩展和容错的特点,可以方便的做到线程、进程、服务器级的扩展。目前已经有了很多成熟的流式处理框架,例如:storm、spark等。
图1所示是流式处理的系统框架,在流式处理中,消息从至少一个数据源出发,经过多个节点处理,图中的S1和S2为数据源,B1~B5为对消息进行处理的节点。流式处理的优势就在于消息处理的扩展能力,消息处理中的每个阶段都可以得到扩展,包括进程内的扩展和进程间的扩展。这样在硬件充足的境况下,可以达到消息处理能力的无限扩展。
在电信领域,随着第5代移动通信技术(5-Generation,5G)网络时代的来临,移动网络设备接入数量将得到极大提高,预计到了2020年,将有500亿移动设备接入网络。这样,随之而来的是网络管理设备数量的极大提升,网络管理软件需要处理大量的消息。引入流式处理可以极大的提升传统网管的消息处理能力,但传统网管有一些自己的业务要求,根据关联分析规则对消息的关联分析就是其中之一。以告警消息为例,网管系统中告警消息的关联分析用于找出不同事件产生的告警消息之间的关联关系,以确定告警产生的主要原因,使用户能够快速定位并解决问题。又如,网管系统中还存在根据归并规则对告警消息进行归并处理的需求,等等。在大数据分析领域,同样也有根据各种规则对消息进行处理的需求,例如对不同新闻事件的消息进行关联分析,对不同购买事件的消息进行归并处理等等。
在流式处理中如何根据相应规则实现对消息的这些处理,是需要关注的重点。此外,某些特定消息需要按序处理,在流式处理时如何实现,也是需要关注的重点。
发明内容
以下是对本文详细描述的主题的概述。本概述并非是为了限制权利要求的保护范围。
本申请实施例提供了一种消息的处理方法,包括:预处理节点响应于接收数据源发送的一个事件的首消息,在系统已创建的规则实例中查找处理对象,所述处理对象包括所述首消息的规则实例;所述预处理节点根据查找到的规则实例建立处理路径,所述处理路径包括所述查找到的规则实例所在处理节点的寻址信息;所述预处理节点根据所述处理路径,将所述首消息发送给所述处理节点,以在所述处理节点由所述查找到的规则实例对所述首消息进行处理。
本申请实施例还提供了一种消息的处理方法,包括:处理节点接收预处理节点或上一处理节点发送的一个事件的消息;所述处理节点响应于确定所述消息为所述一个事件的首消息,利用所述处理节点上的相应规则实例对所述首消息进行处理。
本申请实施例还提供了一种消息的预处理节点,包括:消息接收模块,设置为接收数据源发送的消息;实例查找模块,设置为在所述消息接收模块接收到一个事件的首消息后,在系统已创建的规则实例中查找处理对象,所述处理对象包括所述首消息的规则实例;路径建立模块,设置为根据查找到的规则实例建立处理路径,所述处理路径包括所述查找到的规则实例所在处理节点的寻址信息;消息发送模块,设置为根据所述处理路径,将所述首消息发送给所述处理节点。
本申请实施例还提供了一种消息的处理系统,包括数据源、预处理节点和处理节点,其中:所述数据源,设置为将一个事件的所有消息发送给所述预处理节点;所述预处理节点为前述实施例所述的预处理节点;所述处理节点,设置为接收所述预处理节点发送的所述一个事件的消息,如确定所述消息为所述一个事件的首消息,利用所述处理节点上的所述查找到的规则实例对所述首消息进行处理。
本申请实施例还提供了一种消息的处理装置,包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序,所述处理器执行所述计算机程序时实现如本申请实施例的方法。
本申请实施例还提供了一种计算机可读存储介质,其上存储有计算机程序,该计算机程序被处理器执行时实现如本申请实施例的方法。
在阅读并理解了附图和详细描述后,可以明白其他方面。
图1是流式处理的示意图
图2是本申请实施例一消息处理方法的流程图;
图3是本申请实施例一预处理节点的模块图;
图4是本申请实施例一处理节点执行的消息处理方法的流程图;
图5是本申请实施例二对消息进行分阶段处理的示意图;
图6是本申请实施例二对消息进行关联分析的流程图;
图7是本申请实施例三对告警消息进行关联分析的示意图;
图8是本申请实施例四对告警消息进行关联分析的示意图;
图9是本申请实施例五对告警消息进行关联分析的示意图;
图10是本申请实施例六对告警消息进行关联分析的示意图;
图11是本申请实施例七对新闻数据进行关联分析的示意图。
下文中将结合附图对本申请的实施例进行详细说明。需要说明的是,在不冲突的情况下,本申请中的实施例及实施例中的特征可以相互任意组合。
实施例一
本实施例提供了一种消息的处理方法,可以在流式处理方式下,动态计算消息的处理路径,根据处理路径实现对消息的分发和处理。
本实施例消息的处理系统可参见图1,采用流式处理框架如jstorm流处理框架,包括数据源和对消息进行处理的节点,这些节点分为两类,一类称为预处理节点,一类称为处理节点。其中,预处理节点设置为从数据源接收事件的消息,建立消息的处理路径及将消息分发给处理节点,而处理节点在接收到消息后,利用根据处理规则建立的规则实例对消息进行处理。
本实施例消息的处理方法如图2所示,包括步骤110,步骤120和步骤130。
在步骤110中,预处理节点响应于接收数据源发送的一个事件的首消息,在系统已创建的规则实例中查找处理对象,所述处理对象包括所述首消息的规则实例。
这里的“一个事件”只是用于指代某一事件,并非指代特殊的事件。事件的首消息是数据源按时间顺序采集到的该事件的第一个消息。在一个示例中,所述规则实例包括根据关联分析规则创建的规则实例,所述一个事件为告警事件,一个事件的首消息是告警事件的产生消息,而对消息的处理是对消息进行关联分析。但本申请的事件不局限于此,如可以是设备执行的一个动作,也可以是发生的一件事情,等等。事件的消息可以是与事件相关的任何数据,同一事件的消息携带有该事件的键值,如可以是关键字,名称、标识等,通过键值可以识别出同一事件的所有消息。例如网管的告警,关键信息为网元,描述信息为告警产生的时间、描述等;对于新闻事件,可以定义关键信息为新闻产生 的时间,描述信息为新闻涉及的人物,描述等。具体的关键信息的定义应该由应用根据实际的应用场景来定义。
本实施例中,预处理节点和处理节点均为逻辑节点,而规则实例根据处理规则创建,可以在处理节点上运行以执行该处理规则的业务处理逻辑。在一个示例中,预处理节点和处理节点可以用进程或线程等具有处理能力的资源单元来实现,而规则实例是在处理节点上运行的程序。
一个规则实例除了需要业务处理逻辑外,还需要定义该规则实例的处理对象,例如,在一个针对告警事件的消息进行关联分析的规则实例,可以通过限定条件定义处理对象,如通过指定网元标识和告警码,将处理对象定义为在指定网元产生的具有指定告警码的告警事件。本实施例在预处理节点中,保存系统已创建的规则实例的处理对象信息如上述限定条件,还可以保存规则实例的标识、参数、类型等其他信息。在另一实施例中,这些信息也可以保存在其他节点,预处理节点在需要使用时再从其他节点读取到这些信息。
本实施例中,所述已创建的规则实例根据至少一种处理规则创建,每一种处理规则对应于一种业务场景,具有不同的业务处理逻辑。根据一种处理规则可以创建至少一个规则实例,不同的规则实例的处理对象、参数可以不同。本实施例中,一个处理节点上可以运行至少一个规则实例,有多个规则实例时,处理节点可以根据消息中写入的规则实例的标识来确定运行哪一个规则实例进行处理。本实施例中,根据不同处理规则创建的规则实例所在的处理节点不同。
本实施例中,可以根据处理规则将已创建的规则实例分为至少一组,根据同一处理规则创建的规则实例分为一组。在系统已创建的规则实例中查找处理对象,所述处理对象包括所述首消息的规则实例时,采用分组查找的方式,且每一组至多查找到一个处理对象包括所述一个事件的规则实例,也即在该组规则实例中查找到一个规则实例后即可结束对该组规则实例的查找。这样,在根据同一处理规则创建的多个规则实例中,只由一个规则实例对一个事件进行处理,使得一个事件仅一条无分支的关联分析路径,只有一种处理结果。但这并非是必须的。
在步骤120中,所述预处理节点根据查找到的规则实例建立处理路径,所述处理路径包括所述查找到的规则实例所在处理节点的寻址信息。
本实施例中,查找到的规则实例所在处理节点的寻址信息通过可唯一标识该规则实例的信息来表示。例如,所述查找到的规则实例所在处理节点的寻址信息包括所述查找到的规则实例的标识;或者,所述查找到的规则实例所在处理节点的寻址信息包括所述查找到的规则实例的标识和类型,其中,已创建的规则实例根据至少一种处理规则创建,根据同一处理规则创建的规则实例的类型相同,根据不同处理规则创建的规则实例的类型不同。因为系统中配置有各 规则实例所在的处理节点,因而可唯一标识规则实例的信息可作为寻址信息,保证每次寻址的结果相同,具体的寻址方法可以有多种,例如,对规则实例的标识和类型进行哈希运算,根据得到的哈希值查找哈希值与处理节点的对应关系,从而确定规则实例所在的处理节点。这些寻址方法可以利用流式处理框架提供的机制来实现。
在步骤130中,所述预处理节点根据所述处理路径,将所述首消息发送给所述处理节点,以由所述查找到的规则实例对所述首消息进行处理。
在一个示例中,对所述首消息进行处理包括对所述首消息进行关联分析,但本申请不局限于此,还可以是各种其他的处理。
本步骤中,如果规则实例定义有参数,所述预处理节点将所述首消息发送给所述处理节点时,将所述查找到的规则实例的参数和标识写入所述首消息中,并将首消息发送给所述处理节点,所述查找到的规则实例的参数和标识可以在对所述首消息进行处理时使用。
本步骤中,在处理节点有多个时,预处理节点可以采用广播的方式,将所述首消息分别发送给所述多个处理节点,必要时同时发送规则实例的参数。本实施例中,则采用了串行分发方式,也即预处理节点将首消息发送给处理路径中的第一个处理节点,而处理路径中的每一个处理节点再发送到下一个处理节点。
例如,所述预处理节点将所述首消息和所述处理路径中第一个处理节点之后的所有处理节点的寻址信息发送给所述第一个处理节点;接收到所述首消息的每个处理节点利用每个处理节点上的所述查找到的规则实例对所述首消息进行处理,响应于确定每个处理节点不是所述多个处理节点中的最后一个处理节点,将所述首消息和所述处理路径中下一个处理节点之后的所有处理节点的寻址信息发送给所述下一个处理节点。
上述预处理节点和处理节点发送的寻址信息是下一处理节点继续发送所需要的,但不应理解为只能发送这些寻址信息,例如,预处理节点也可以将处理路径中的所有处理节点的寻址信息发送给上述第一个处理节点,而上述第一个处理节点从中查找到自己的寻址信息之后,就可以确定后续处理节点的寻址信息而实现消息的继续发送。
本实施例中,所述预处理节点根据查找到的规则实例建立处理路径之后,所述预处理节点保存所述处理路径,响应于接收所述数据源发送的所述一个事件的后续消息,则根据保存的所述处理路径将所述后续消息发送给所述处理节点。具体的发送方式与首消息的发送方式类似,但处理节点在进行关联分析时,可以不对这些后续事件进行处理。当然,如果是其他的业务处理逻辑,也可以对后续事件进行处理。本实施例中,利用流处理框架支持的机制,一个事件的 所有消息在同一个数据源采集并分发给相同的预处理节点。
本申请通过建立消息的处理路径并按处理路径分发,可以在流式处理中,根据创建的规则实例实现对消息的处理。
本实施例还提供了一种消息的预处理节点,如图3所示,包括消息接收模块10,实例查找模块20,路径建立模块30以及消息发送模块40。
消息接收模块10,设置为接收数据源发送的消息。
实例查找模块20,设置为在所述消息接收模块接收到一个事件的首消息后,在系统已创建的规则实例中查找处理对象,所述处理对象包括所述首消息的规则实例。
路径建立模块30,设置为根据查找到的规则实例建立处理路径,所述处理路径包括所述查找到的规则实例所在处理节点的寻址信息。
消息发送模块40,设置为根据所述处理路径,将所述首消息发送给所述处理节点进行处理。
本实施例中,所述路径建立模块,设置为建立所述处理路径之后,保存所述处理路径;所述消息发送模块,还设置为在所述消息接收模块接收到所述数据源发送的所述一个事件的后续消息后,根据保存的所述处理路径将所述后续消息发送给所述处理节点。
本实施例中,所述已创建的规则实例是根据至少一种处理规则创建的根据不同处理规则创建的规则实例所在的处理节点不同。
本实施例中,所述已创建的规则实例分为至少一组,根据同一处理规则创建的规则实例分为一组;所述实例查找模块在已创建的规则实例中查找处理对象,所述处理对象包括所述首消息的规则实例,包括:所述预处理节点在每一组中查找规则实例,在每一组中查找至多一个处理对象包括所述一个事件的规则实例。
本实施例中,在所述路径建立模块建立的所述处理路径中,所述查找到的规则实例所在处理节点的寻址信息包括所述查找到的规则实例的标识,或者包括所述查找到的规则实例的标识和类型,其中,已创建的规则实例根据至少一种处理规则创建,根据同一处理规则创建的规则实例的类型相同,根据不同处理规则创建的规则实例的类型不同。
本实施例中,所述消息发送模块将所述首消息发送给所述处理节点,包括:将所述查找到的规则实例的参数和标识写入所述首消息中,并将首消息发送给所述处理节点。
本实施例中,所述路径建立模块建立的所述处理路径包括多个处理节点的寻址信息;所述消息发送模块设置为:将所述首消息和所述处理路径中第一个处理节点之后的所有处理节点的寻址信息发送给所述第一个处理节点,所述首 消息携带所述查找到的规则实例的参数和标识。
本实施例中,所述规则实例包括根据关联分析规则创建的规则实例。
本实施例还提供了处理节点执行的消息处理方法,如图4所示,包括步骤210和步骤220。
在步骤210中,处理节点接收预处理节点或上一处理节点发送的一个事件的消息。
在步骤220中,所述处理节点响应于确定所述消息为所述一个事件的首消息,利用所述处理节点上的相应规则实例对所述首消息进行处理。
本步骤中所述处理节点上的“相应规则实例”即上文中预处理节点查找到的规则实例,也即处理对象包括一个事件的规则实例。本实施例中,一个事件为告警事件,规则实例只需要对一个事件的首消息做关联分析,对一个事件的其他消息只做分发。但本申请不局限于此。
本实施例中,采用串行分发方式发送消息,所述处理节点响应于接收所述一个事件的消息,接收所述一个事件的处理路径中后续处理节点(指在所述处理节点之后的处理节点)的寻址信息;所述处理节点接收到所述寻址信息之后进行保存,根据所述寻址信息将所述首消息发送给所述后续处理节点;所述处理节点接收到所述一个事件的后续消息时,根据保存的所述寻址信息,将所述后续消息发送给所述后续处理节点。
本实施例中,所述处理节点利用所述处理节点上的相应规则实例对所述首消息进行处理之后,所述方法还包括:响应于对所述首消息进行处理时生成所述一个事件的新消息,可以根据其业务处理逻辑,将所述新消息发送给负责采集所述一个事件的消息的数据源。这样,一个事件的消息仍然是由原来的数据源、预处理节点和处理路径上的处理节点进行按序处理。
本实施例还提供了一种消息处理系统,包括数据源、预处理节点和处理节点,其中:所述数据源,设置为将一个事件的所有消息发送给同一预处理节点;所述预处理节点为如上所述的任一种预处理节点;所述处理节点,设置为接收所述预处理节点发送的所述一个事件的消息,响应于确定所述消息为所述一个事件的首消息,利用所述处理节点上的所述查找到的规则实例对所述首消息进行处理。
本实施例中,所述处理节点响应于接收所述一个事件的消息,接收所述一个事件的处理路径中后续处理节点的寻址信息;所述处理节点确定所述消息是所述一个事件的首消息之后,还设置为保存所述后续处理节点的寻址信息,根据所述寻址信息将所述首消息发送给后续处理节点;及,响应于接收所述一个 事件的后续消息,根据保存的所述寻址信息,将所述后续消息发送给所述后续处理节点。
本实施例中,所述处理节点利用所述处理节点上的相应规则实例对所述首消息进行处理之后,所述方法还包括:响应于对所述首消息进行处理时生成所述一个事件的新消息,将所述新消息发送给负责采集所述一个事件的消息的数据源。
本实施例中,所述规则实例包括根据关联分析规则创建的规则实例。
本实施例还提供了一种消息处理装置,包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序,其特征在于,所述处理器执行所述计算机程序时可以实现如本实施例方法的任何处理。
本实施例的消息处理采用了流式处理,动态生成消息的处理路径用于消息的分发。且同一事件的消息根据相同的处理路径分发,可以保障同一事件的所有消息的按序处理。
实施例二
本实施例涉及一种消息的关联分析方法。在流式处理方式下,动态计算消息的关联分析路径,根据关联分析路径对消息分发做关联分析,并提供了一种保障同一事件的其所有消息按序处理的方法。
本实施例以网管为应用场景,该场景涉及大量的告警事件,每个告警事件会产生多个消息,这些消息可以分为:产生消息,事件产生时生成的消息;后续消息,事件产生后生成的消息,如事件的改变消息、清除消息等。
同一事件的不同消息具有相同的键值(key),需要被按序处理。
本实施例中,对事件的首消息进行关联分析,也即分析多个事件之间的关系,以便用户快速得到有用的信息。在实际应用中,存在多种应用场景,每种场景下都可以有自己的关联分析方法,这些关联分析方法被定义为关联规则,因而不同的场景可以对应不同的关联规则。
本实施例中,一个规则实例定义了以下内容。
限定条件:用于定义规则的处理对象,也可以说是定义规则的生效范围,限定条件定义了规则对哪些事件生效,可以由用户指定,例如,用户可以指定至少一个网元,至少一个告警码,至少一种告警级别等作为规则实例的限定条件。
业务处理逻辑:指对符合限定条件的消息进行关联分析时的处理逻辑。
参数:即规则实例执行关联分析的输入参数,可以由用户指定,例如可以 是告警延迟的时间。
用户可以根据一种关联规则创建至少一个规则实例,每一规则实例的限定条件和参数由用户指定。通过本实施例方法,符合某一规则实例的限定条件的消息(即该消息是该规则实例的处理对象)会被分发到该规则实例所在的关联分析节点一起分析,从而获得事件之间的关系,最终展示给用户。
参见图5,本实施例消息的关联分析方法分为两个阶段。
预处理阶段,判断事件的首消息是否符合系统中已创建的至少一个规则实例的限定条件。基于事件的首消息符合系统中已创建的至少一个规则实例的限定条件的判断结果,创建关联分析路径,根据关联分析路径将事件的首消息和规则实例的参数分发到相应的关联分析节点进行第二阶段的处理;基于事件的首消息不符合系统中已创建的至少一个规则实例的限定条件的判断结果,事件将不被规则实例处理。
关联分析处理,关联分析节点上的规则实例根据业务处理逻辑和参数对接收到的事件的首消息进行关联分析。
本实施例消息的关联分析方法如图6所示,假定用户已经在系统中创建了多个用于进行关联分析的规则实例,所述方法包括步骤310至步骤390。
在步骤310中,数据源采集事件的消息并发送给预处理节点。
本实施例以流式处理框架为基础,事件在多个数据源采集,同一个事件的所有消息在同一个数据源采集和分发。
在步骤320中,预处理节点判断消息的类型,基于消息的类型为事件的产生消息的判断结果,进入步骤340,基于消息的类型不为事件的产生消息的判断结果,进入步骤330。
在步骤330中,根据消息的键值从缓存获取事件的关联分析路径,进入步骤360。
在步骤340中,获取已创建的规则实例,这些规则实例按对应的关联规则划分为多组,其中根据同一关联规则创建的规则实例划分为一组;将消息依次与每组规则实例的限定条件进行比较,在每一组规则实例中查找到至多一个处理对象包括所述消息的规则实例,将所有查找到的规则实例的标识和类型顺序排列,建立关联分析路径,并将规则实例的参数、规则实例的标识等基本信息写入消息中。
本步骤中,处理对象包括所述消息的规则实例,意味着所述消息符合该规则实例的限定条件。本步骤如果一共查找到多个规则实例,该多个规则实例的标识和类型可以按照预先设定的排序方式排列,例如,规则实例A的标识和类型、规则实例B的标识和类型、规则实例C的标识和类型。一个规则实例的标 识和类型是作为该规则实例所在关联分析节点的寻址信息,具体的寻址过程可以由流式处理框架提供的机制来实现。
本步骤中,规则实例的参数也可以作为单独的信息发送,这取决于所发送信息的格式。
在步骤350中,预处理节点将关联分析路径与消息的键值关联,存入缓存,供后续消息使用。
在步骤360中,将消息发送到关联分析路径中的下一个关联分析节点,同时将关联分析路径中下一个关联分析节点之后的所有关联分析节点的寻址信息与消息一起发送。
本步骤由预处理节点执行时,所述下一个关联分析节点即关联分析路径中的第一个关联分析节点。本步骤由关联发析路径上的一个关联分析节点执行时,所述下一个关联分析节点即关联分析路径中该关联分析节点之后的第一个关联分析节点。
在步骤370中,接收到消息的关联分析节点判断消息的类型,基于消息的类型为事件的产生消息的判断结果,进入步骤380,基于消息的类型不为事件的产生消息的判断结果,进入步骤390。
在步骤380中,接收到消息的关联分析节点获取消息中携带的规则实例的参数,利用其上运行的规则实例对消息进行关联分析。
在步骤390中,接收到消息的关联发析节点判断关联分析路径中是否还有下一个分析节点的寻址信息,基于没有下一节点的寻址信息的判断结果,处理结束;基于还有下一节点的寻址信息判断结果,返回步骤360。
实施例三
当多条告警消息上报时,如果都符合同一条关联规则的限定条件,则会被分发到同一个关联分析节点处理,下面以多条告警消息符合同一个关联规则的限定为例,说明该情况下的关联分析处理。
请参见图7,系统中已根据同一关联规则创建了多个规则实例,类型的关联规则,上报的告警消息都将符合其中一条规则实例,则处理流程包括步骤一至步骤十二。
在步骤一中,数据源S1接收到告警消息A1。
数据源可以是网元上的告警采集模块。
在步骤二中,数据源S1根据告警消息A1的网元信息将告警消息A1分发到预处理节点P2处理。
在步骤三中,预处理节点P2收到告警消息A1后,通过关联分析的预处理, 发现告警消息A1符合规则实例M1的限定条件,生成关联分析路径[M1]。
关联分析路径[M1]表示关联分析节点M1的寻址信息,具体可以用规则实例的标识和类型来表示,M表示类型,1表示ID,后续的其他实施例同此。
在步骤四中,预处理节点P2根据关联分析路径将告警消息A1发送给关联分析节点M1处理,然后继续后续的处理流程。
上述关联分析节点和其上的规则实例采用相同的标记,如规则实例M1所在的关联分析节点表示为关联分析节点M1。分发到关联分析节点M1处理,即由关联分析节点M1上的规则实例M1对告警消息A1进行关联分析。其他实施例同此。
在步骤五中,数据源S2接收到告警消息A2。
在步骤六中,数据源S2根据告警消息A2的网元信息将告警消息A2分发到预处理节点P3上处理。
在步骤七中,预处理节点P3收到告警消息A2后,通过关联分析的预处理,发现告警消息A2符合规则实例M1的限定条件,从而生成告警消息A2的关联分析路径[M1]。
在步骤八中,预处理节点P3根据关联分析路径将告警消息A2发送给关联分析节点M1处理,然后继续后续的处理流程。
在步骤九中,数据源S1接收到告警消息A3。
在步骤十中,数据源S1根据告警消息A3的网元信息将告警消息A3分发到预处理节点P1上处理。
在步骤十一中,预处理节点P1收到告警消息A3后,通过关联分析的预处理,发现告警消息A3符合规则实例M1的限定条件,从而生成告警消息A3的关联分析路径[M1]。
在步骤十二中,预处理节点P1根据关联分析路径将告警消息A3发送给关联分析节点M1处理,然后继续后续的处理流程。
实施例四
多条告警符合多个关联规则时,告警的消息根据自己的关联分析路径分别处理,本实施例将对该场景下消息的关联分析方法进行说明。
请参见图8,系统中根据多种关联规则创建了多组规则实例,图中的M1~M3为一组,R1~R3为一组,C1~C3为一组,上报的告警消息都符合其中至少一个规则实例的限定条件,处理流程包括步骤一至步骤十二。
在步骤一中,数据源S1接收到告警消息A1。
在步骤二中,数据源S1根据告警消息A1的网元信息将告警消息A1分发 到预处理节点P2上处理。
在步骤三中,预处理节点P2收到告警消息A1后,通过关联分析的预处理,发现告警消息A1符合规则实例M1、R2、C2的限定条件,从而生成告警消息A1的关联分析路径[M1,R2,C2]。
在步骤四中,预处理节点P2根据关联分析路径将告警消息A1发送给关联分析节点M1处理,M1再将消息发送给关联分析节点R2处理,R2再发送给关联分析节点C2处理,然后继续后续的处理流程。
在步骤五中,数据源S2接收到告警消息A2。
在步骤六中,数据源S2根据告警消息A2的网元信息将告警消息A2分发到预处理节点P3上处理。
在步骤七中,预处理节点P3收到告警消息A2后,通过关联分析的预处理,发现告警消息A2符合规则实例M2、R2、C1的限定条件,从而生成告警消息A2的关联分析路径[M2,R2,C1]。
在步骤八中,预处理节点P3根据关联分析路径将告警消息A2发送给关联分析节点M2处理,M2再将消息发送给关联分析节点R2处理,R2再将消息发送给关联分析节点C1处理,然后继续后续的处理流程。
在步骤九中,数据源S1接收到告警消息A3。
在步骤十中,数据源S1根据告警消息A3的网元信息将告警消息A3分发到预处理节点P1上处理。
在步骤十一中,预处理节点P1收到告警消息A3后,通过关联分析的预处理,发现告警消息A3符合规则实例M1、C2的限定条件,从而生成告警消息A3的关联分析路径[M1,C2]。
在步骤十二中,预处理节点P1根据关联分析路径将告警消息A3发送给关联分析节点M1处理,M1再将消息发送给关联分析节点C2处理,然后继续后续的处理流程。
实施例五
一条告警的上报消息生成关联分析路径后,该告警后续的所有消息都将根据该关联分析路径处理,本示例以同一条告警的不同消息为例对消息的关联分析进行说明。
请参见图9,系统已创建多个规则实例,上报的告警符合其中的一个规则实例,处理流程包括步骤一至步骤十二。
在步骤一中,数据源S1接收到告警消息A1。
在步骤二中,数据源S1将根据产生告警消息A1的网元信息将告警消息A1 发送给预处理节点P2处理。
在步骤三中,预处理节点P2通过关联分析的预处理,发现告警消息A1符合规则实例M1的限定条件,生成告警消息A1的关联分析路径[M1]。
在步骤四中,预处理节点P2根据关联分析路径将告警消息A1发送给关联分析节点M1处理,然后继续后续的处理流程。
在步骤五中,数据源S1接收到告警消息A1所属告警的改变消息C1。
在步骤六中,数据源S1根据C1产生的网元信息将C1发送给预处理节点P2处理。
在步骤七中,预处理节点P2通过查询,得到告警消息A1的关联分析路径[M1]。
在步骤八中,预处理节点P2根据关联分析路径将C1发送给关联处理节点M1处理,然后继续后续的处理流程。
在步骤九中,数据源S1接收到告警消息A1所属告警的清除消息D1。
在步骤十中,数据源S1根据D1产生的网元信息将D1发送给预处理节点P2处理。
在步骤十一中,预处理节点P2通过查询,得到告警消息A1的关联分析路径[M1]。
在步骤十二中,预处理节点P2根据关联分析路径将D1发送给关联分析节点M1处理,然后继续后续的处理流程。
实施例六
对告警消息进行关联分析的处理过程中可能会产生新消息,本示例针对这种情况说明相关的处理过程。
参见图10。假定系统中存已创建多个用于关联分析的规则实例,而上报的告警符合其中一个规则实例,相关处理流程包括步骤一至步骤八。
在步骤一中,数据源S1接收到告警消息A1。
在步骤二中,数据源S1将根据产生告警消息A1的网元信息将告警消息A1发送给预处理节点P2处理。
在步骤三中,预处理节点P2通过关联分析的预处理,发现告警消息A1符合规则实例M1的限定条件,生成告警消息A1的关联分析路径[M1]。
在步骤四中,预处理节点P2根据关联分析路径将告警消息A1发送给关联分析节点M1处理,告警消息A1经过关联分析节点M1的处理后,产生一个新的告警消息A1所属告警的改变消息C1。
在步骤五中,关联分析节点M1将改变消息C1发送给数据源S1处理。
在步骤六中,数据源S1将改变消息C1按产生告警消息A1的网元信息分发,将改变消息C1发送给预处理节点P2处理。
在步骤七中,预处理节点P2查找到告警消息A1的关联分析路径[M1]。
在步骤八中,预处理节点P2根据查找到的关联分析路径将改变消息C1发送给关联分析节点M1处理,然后继续后续的处理流程。
在流式处理框架中,每个关联分析节点都可以通过消息中间件将消息发送到数据。
实施例七
本实施例的消息为新闻数据,本实施例涉及新闻数据的关联分析。
请参见图11,本示例中,系统中的多个数据源均可获取新闻数据,每个新闻数据属于不同的事件,用不同的键值标识,处理流程包括步骤一至步骤十二。
在步骤一中,数据源S1获取新闻数据N1。
在步骤二中,数据源S1根据新闻数据N1的键值,将新闻数据N1分发到预处理节点P2处理。
在步骤三中,预处理节点P2通过关联分析的预处理,发现新闻数据N1符合规则实例M1、E3的限定条件,生成新闻数据N1的关联分析路径[M1,E3]。
在步骤四中,预处理节点P2根据关联分析路径将新闻数据N1发送给关联分析节点M1处理,关联分析节点M1再将消息发送给关联分析节点E3处理,然后继续后续的处理流程。
在步骤五中,数据源S3获取新闻数据N2。
在步骤六中,数据源S3根据新闻数据N2的键值分发到预处理节点P1处理。
在步骤七中,预处理节点P1通过关联分析的预处理,发现新闻数据N2符合规则实例M2、E3的限定条件,生成新闻数据N1的关联分析路径[M2,E3]。
在步骤八中,预处理节点P1根据关联分析路径,将新闻数据N2发送给关联分析节点M2处理,关联分析节点M2再将新闻数据N2发送给关联分析节点E3处理,然后继续后续的处理流程。
在步骤九中,数据源S2获取新闻数据N3。
在步骤十中,数据源S2根据新闻数据N3的键值将新闻数据N3分发到预处理节点P1处理。
在步骤十一中,预处理节点P1通过关联分析的预处理,发现新闻数据N3符合规则实例M1、E1的限定条件,生成新闻数据N3的关联分析路径[M1,E1]。
在步骤十二中,预处理节点P1根据关联分析路径将新闻数据N3发送给关联分析节点M1处理,关联分析节点M1再将新闻数据N3发送给关联分析节点 E1处理,然后继续后续的处理流程。
同一事件的新闻数据也可能存在后续的追加报道,相当于改变消息,此时可以按照上述实施例的相同方式进行处理。
本领域普通技术人员可以理解,上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施方式中,在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分;例如,一个物理组件可以具有多个功能,或者一个功能或步骤可以由若干物理组件合作执行。某些物理组件或所有物理组件可以被实施为由处理器,如中央处理器、数字信号处理器或微处理器执行的软件,或者被实施为硬件,或者被实施为集成电路,如专用集成电路。这样的软件可以分布在计算机可读介质上,计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的,术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于随机存取存储器(Random Access Memory,RAM)、只读内存(Read Only Memory,ROM)、带电可擦可编程只读存储器(Electrically Erasable Programmable Read-Only Memory,EEPROM)、闪存或其他存储器技术、只读光盘(Compact Disc Read-Only Memory,CD-ROM)、数字多功能盘(Digital Video Disc,DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外,本领域普通技术人员公知的是,通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据,并且可包括任何信息递送介质。
Claims (20)
- 一种消息的处理方法,包括:预处理节点响应于接收数据源发送的一个事件的首消息,在系统已创建的规则实例中查找处理对象,所述处理对象包括所述首消息的规则实例;所述预处理节点根据查找到的规则实例建立处理路径,所述处理路径包括所述查找到的规则实例所在处理节点的寻址信息;所述预处理节点根据所述处理路径,将所述首消息发送给所述处理节点,以由所述查找到的规则实例对所述首消息进行处理。
- 如权利要求1所述的处理方法,所述预处理节点根据查找到的规则实例建立处理路径之后,还包括:所述预处理节点保存所述处理路径,响应于接收所述数据源发送的所述一个事件的后续消息,根据保存的所述处理路径将所述后续消息发送给所述处理节点。
- 如权利要求1所述的处理方法,其中:所述已创建的规则实例根据至少一种处理规则创建,根据不同处理规则创建的规则实例所在的处理节点不同。
- 如权利要求3所述的处理方法,其中:所述已创建的规则实例分为至少一组,根据同一处理规则创建的规则实例分为一组;所述预处理节点在已创建的规则实例中查找处理对象,所述处理对象包括所述首消息的规则实例,包括:所述预处理节点在每一组中查找规则实例,在每一组中查找至多一个所述处理对象,所述处理对象包括所述一个事件的规则实例。
- 如权利要求3所述的处理方法,其中:所述查找到的规则实例所在处理节点的寻址信息包括所述查找到的规则实例的标识;或者所述查找到的规则实例所在处理节点的寻址信息包括所述查找到的规则实例的标识和类型,其中,根据同一处理规则创建的规则实例的类型相同,根据不同处理规则创建的规则实例的类型不同。
- 如权利要求1所述的处理方法,其中:所述预处理节点将所述首消息发送给所述处理节点,包括:所述预处理节点将所述查找到的规则实例的参数和标识写入所述首消息中,并将所述首消息发送给所述处理节点。
- 如权利要求1-6中任一项所述的处理方法,其中:所述处理路径包括多个处理节点的寻址信息,所述预处理节点将所述首消息发送给所述多个处理节点,以由所述查找到的规则实例对所述首消息进行处理,包括:所述预处理节点将所述首消息和所述处理路径中第一个处理节点之后的所有处理节点的寻址信息发送给所述第一个处理节点;接收到所述首消息的每个所述处理节点利用每个所述处理节点上的所述查找到的规则实例对所述首消息进行处理,响应于确定每个所述处理节点不是所述多个处理节点中的最后一个处理节点,将所述首消息和所述处理路径中下一个处理节点之后的所有处理节点的寻址信息发送给所述下一个处理节点。
- 如权利要求1-6中任一项所述的处理方法,其中:所述规则实例包括根据关联分析规则创建的规则实例。
- 一种消息的处理方法,包括:处理节点接收预处理节点或上一处理节点发送的一个事件的消息;所述处理节点响应于确定所述消息为所述一个事件的首消息,利用所述处理节点上的相应规则实例对所述首消息进行处理。
- 如权利要求9所述的处理方法,还包括:所述处理节点响应于接收所述一个事件的消息,接收所述一个事件的处理路径中后续处理节点的寻址信息;所述处理节点接收所述一个事件的处理路径中后续处理节点的寻址信息之后,保存所述寻址信息,根据所述寻址信息将所述首消息发送给所述后续处理节点;及,响应于接收所述一个事件的后续消息,根据保存的所述寻址信息,将所述后续消息发送给所述后续处理节点。
- 如权利要求9所述的处理方法,所述处理节点利用所述处理节点上的相应规则实例对所述首消息进行处理之后,还包括:响应于对所述首消息进行处理时生成了所述一个事件的新消息,将所述新消息发送给负责采集所述一个事件的消息的数据源。
- 一种消息的预处理节点,包括:消息接收模块,设置为接收数据源发送的消息;实例查找模块,设置为在所述消息接收模块接收到一个事件的首消息后,在系统已创建的规则实例中查找处理对象,所述处理对象包括所述首消息的规则实例;路径建立模块,设置为根据查找到的规则实例建立处理路径,所述处理路径包括所述查找到的规则实例所在处理节点的寻址信息;消息发送模块,设置为根据所述处理路径,将所述首消息发送给所述处理节点进行处理。
- 如权利要求12所述的预处理节点,其中所述路径建立模块,设置为建立所述处理路径之后,保存所述处理路径;所述消息发送模块,还设置为在所述消息接收模块接收到所述数据源发送的所述一个事件的后续消息后,根据保存的所述处理路径将所述后续消息发送给所述处理节点。
- 如权利要求12所述的预处理节点,其中:在所述路径建立模块建立的所述处理路径中,所述查找到的规则实例所在处理节点的寻址信息包括所述查找到的规则实例的标识,或者包括所述查找到的规则实例的标识和类型,其中,所述已创建的规则实例根据至少一种处理规则创建,根据同一处理规则创建的规则实例的类型相同,根据不同处理规则创建的规则实例的类型不同。
- 如权利要求12所述的预处理节点,其中:所述路径建立模块建立的所述处理路径包括多个处理节点的寻址信息;所述消息发送模块设置为:将所述首消息和所述处理路径中第一个处理节点之后的所有处理节点的寻址信息发送给所述第一个处理节点,所述首消息携带所述查找到的规则实例的参数和标识。
- 一种消息处理系统,包括数据源、预处理节点和处理节点,其中:所述数据源,设置为将一个事件的所有消息发送给所述预处理节点;所述预处理节点为权利要求12-15任一所述的预处理节点;所述处理节点,设置为接收所述预处理节点发送的所述一个事件的消息,响应于确定所述消息为所述一个事件的首消息,利用所述处理节点上的所述预处理节点查找到的规则实例对所述首消息进行处理。
- 如权利要求16所述的处理系统,其中:所述处理节点设置为:接收所述一个事件的消息时,接收所述一个事件的处理路径中后续处理节点的寻址信息;所述处理节点设置为:确定所述消息是所述一个事件的首消息之后,还保存所述后续处理节点的寻址信息,根据所述寻址信息将所述首消息发送给后续处理节点;及,响应于接收所述一个事件的后续消息,根据保存的所述寻址信息,将所述后续消息发送给所述后续处理节点。
- 如权利要求16或17所述的处理系统,其中:所述规则实例包括根据关联分析规则创建的规则实例。
- 一种消息处理装置,包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序,所述处理器执行所述计算机程序时实现如权利要求1-11所述的方法。
- 一种计算机可读存储介质,所述存储介质存储有计算机程序,所述计算机程序被处理器执行时实现如权利要求1-11所述的方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP19799823.0A EP3793171B1 (en) | 2018-05-09 | 2019-05-09 | Message processing method, apparatus, and system |
JP2020562588A JP7133037B2 (ja) | 2018-05-09 | 2019-05-09 | メッセージ処理方法、装置およびシステム |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810438739.9 | 2018-05-09 | ||
CN201810438739.9A CN108737522B (zh) | 2018-05-09 | 2018-05-09 | 一种消息的处理方法、装置和系统 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019214685A1 true WO2019214685A1 (zh) | 2019-11-14 |
Family
ID=63937315
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2019/086218 WO2019214685A1 (zh) | 2018-05-09 | 2019-05-09 | 一种消息的处理方法、装置和系统 |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP3793171B1 (zh) |
JP (1) | JP7133037B2 (zh) |
CN (1) | CN108737522B (zh) |
WO (1) | WO2019214685A1 (zh) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108737522B (zh) * | 2018-05-09 | 2021-07-20 | 中兴通讯股份有限公司 | 一种消息的处理方法、装置和系统 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101252441A (zh) * | 2008-02-20 | 2008-08-27 | 深圳市永达电子有限公司 | 基于可设定信息安全目标的获得性安全保障方法及系统 |
US20120102503A1 (en) * | 2010-10-20 | 2012-04-26 | Microsoft Corporation | Green computing via event stream management |
CN105723335A (zh) * | 2013-09-16 | 2016-06-29 | 麦塔奈迪克斯有限公司 | 数据流探索 |
CN107968840A (zh) * | 2017-12-15 | 2018-04-27 | 华北电力大学(保定) | 一种大规模电力设备监测报警数据实时处理方法及系统 |
CN108737522A (zh) * | 2018-05-09 | 2018-11-02 | 中兴通讯股份有限公司 | 一种消息的处理方法、装置和系统 |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7315903B1 (en) * | 2001-07-20 | 2008-01-01 | Palladia Systems, Inc. | Self-configuring server and server network |
US7831034B2 (en) * | 2006-07-20 | 2010-11-09 | Microsoft Corporation | Management of telephone call routing using a directory services schema |
JP2013038647A (ja) * | 2011-08-09 | 2013-02-21 | Nec Corp | メッセージ転送システム、制御装置、メッセージ転送ノード、メッセージ転送方法およびプログラム |
FR2989190A1 (fr) * | 2012-04-05 | 2013-10-11 | Comptacom | Entite de traitement d'un flux de donnees, procede et programme d'ordinateur correspondant |
JP5941859B2 (ja) * | 2013-03-05 | 2016-06-29 | 株式会社エヌ・ティ・ティ・データ | 検証装置、検証方法、及びプログラム |
CN104618142B (zh) * | 2014-12-30 | 2017-07-07 | 北京奇安信科技有限公司 | 消息订阅方法、处理节点设备和总线设备 |
US10305858B2 (en) * | 2015-12-18 | 2019-05-28 | Nicira, Inc. | Datapath processing of service rules with qualifiers defined in terms of dynamic groups |
CN106936620B (zh) * | 2015-12-31 | 2020-09-01 | 北京神州泰岳软件股份有限公司 | 一种告警事件的处理方法和处理装置 |
US20180089324A1 (en) * | 2016-09-26 | 2018-03-29 | Splunk Inc. | Dynamic resource allocation for real-time search |
CN107944293B (zh) * | 2017-11-20 | 2019-09-24 | 上海携程商务有限公司 | 虚拟资产保护方法、系统、设备及存储介质 |
-
2018
- 2018-05-09 CN CN201810438739.9A patent/CN108737522B/zh active Active
-
2019
- 2019-05-09 EP EP19799823.0A patent/EP3793171B1/en active Active
- 2019-05-09 WO PCT/CN2019/086218 patent/WO2019214685A1/zh unknown
- 2019-05-09 JP JP2020562588A patent/JP7133037B2/ja active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101252441A (zh) * | 2008-02-20 | 2008-08-27 | 深圳市永达电子有限公司 | 基于可设定信息安全目标的获得性安全保障方法及系统 |
US20120102503A1 (en) * | 2010-10-20 | 2012-04-26 | Microsoft Corporation | Green computing via event stream management |
CN105723335A (zh) * | 2013-09-16 | 2016-06-29 | 麦塔奈迪克斯有限公司 | 数据流探索 |
CN107968840A (zh) * | 2017-12-15 | 2018-04-27 | 华北电力大学(保定) | 一种大规模电力设备监测报警数据实时处理方法及系统 |
CN108737522A (zh) * | 2018-05-09 | 2018-11-02 | 中兴通讯股份有限公司 | 一种消息的处理方法、装置和系统 |
Non-Patent Citations (1)
Title |
---|
See also references of EP3793171A4 |
Also Published As
Publication number | Publication date |
---|---|
JP7133037B2 (ja) | 2022-09-07 |
CN108737522A (zh) | 2018-11-02 |
EP3793171A4 (en) | 2022-02-23 |
EP3793171B1 (en) | 2024-04-03 |
JP2021524085A (ja) | 2021-09-09 |
CN108737522B (zh) | 2021-07-20 |
EP3793171A1 (en) | 2021-03-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11586673B2 (en) | Data writing and reading method and apparatus, and cloud storage system | |
US20160164963A1 (en) | Method, system, and device for managing server hardware resources in a cloud scheduling environment | |
CN108848034B (zh) | 一种网络设备及表项学习方法 | |
KR20190020105A (ko) | 스트리밍 데이터의 분배 처리 방법 및 디바이스 | |
CN111581397A (zh) | 一种基于知识图谱的网络攻击溯源方法、装置及设备 | |
CN109033109B (zh) | 数据处理方法及系统 | |
US11863439B2 (en) | Method, apparatus and storage medium for application identification | |
WO2017028394A1 (zh) | 一种基于实例的分布式数据恢复方法和装置 | |
CN110955704A (zh) | 一种数据管理方法、装置、设备及存储介质 | |
CN110928851A (zh) | 处理日志信息的方法、装置、设备及存储介质 | |
CN110851473A (zh) | 一种数据处理方法、装置和系统 | |
CN103716384A (zh) | 跨数据中心实现云存储数据同步的方法和装置 | |
US8712100B2 (en) | Profiling activity through video surveillance | |
CN110336813B (zh) | 一种访问控制方法、装置、设备及存储介质 | |
CN108399175B (zh) | 一种数据存储、查询方法及其装置 | |
WO2019214685A1 (zh) | 一种消息的处理方法、装置和系统 | |
US10853892B2 (en) | Social networking relationships processing method, system, and storage medium | |
CN104038566A (zh) | 一种虚拟交换设备地址学习的方法、装置及系统 | |
US11940890B2 (en) | Timing index anomaly detection method, device and apparatus | |
CN112579552A (zh) | 日志存储及调用方法、装置及系统 | |
CN113596746B (zh) | 集群消息的处理方法、装置、电子设备和介质 | |
CN113052700B (zh) | 一种确定微服务调用链的方法及装置 | |
CN103795577A (zh) | 日志服务器处理日志的方法和装置 | |
US20140244650A1 (en) | Distributed event processing | |
CN113132233B (zh) | 数据处理方法、软件定义网络控制器和数据处理系统 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19799823 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2020562588 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2019799823 Country of ref document: EP Effective date: 20201209 |