CN111708645A - Event processing method and system based on stream processing - Google Patents
Event processing method and system based on stream processing Download PDFInfo
- Publication number
- CN111708645A CN111708645A CN202010537106.0A CN202010537106A CN111708645A CN 111708645 A CN111708645 A CN 111708645A CN 202010537106 A CN202010537106 A CN 202010537106A CN 111708645 A CN111708645 A CN 111708645A
- Authority
- CN
- China
- Prior art keywords
- event
- matching
- subscription
- processing
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 14
- 238000004806 packaging method and process Methods 0.000 claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 19
- 238000007781 pre-processing Methods 0.000 claims abstract description 19
- 238000004140 cleaning Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/547—Messaging middleware
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses an event processing method and system based on stream processing, and relates to the field of real-time data processing. The method comprises the following steps: the preprocessing unit preprocesses the event, sends the event to the message middleware in a message mode and forwards the event to the stream processing platform; the stream processing platform matches the event with a preset event matching rule, and sends the event to the subscription relation processing unit after the matching is successful; the subscription relation processing unit looks up subscription information of the event and sends the subscription information and the event to the packaging unit; and the packaging unit packages and distributes the event according to the subscription information. The invention can save more storage resources, only needs to carry out standardized processing on the source data when a new event message is accessed, and realizes the subsequent processing through configuration, thereby greatly saving the development workload, reducing the data processing time delay from a minute level to a second level, and being suitable for processing the data with high real-time requirement.
Description
Technical Field
The invention relates to the field of real-time data processing, in particular to an event processing method and system based on stream processing.
Background
With the development of network technology, in the era of mobile internet, massive data is generated every day, and for data with higher real-time requirements, if the data volume is smaller, the data is cleaned and filtered and then stored in a relational database, and meanwhile, the database is inquired or a service is called to verify whether a combination condition is satisfied. And if the data volume is large, storing the data in a non-relational database after data cleaning and filtering, and simultaneously inquiring the non-relational database to verify whether the combination condition is established.
However, as the requirement for the time delay of message processing is higher and higher, by storing data first and then processing, not only the time delay of data processing is longer, but also the cost of data storage is higher, storing data in a database, newly adding a type of data requires new creation or extension table model, and developing corresponding data storage and data processing codes, and the development workload is larger, which is not beneficial to subsequent extension.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide an event processing method and system based on stream processing, aiming at the defects of the prior art.
The technical scheme for solving the technical problems is as follows:
an event processing method based on stream processing comprises the following steps:
the method comprises the steps that a preprocessing unit preprocesses an event, sends the preprocessed event to a message middleware in a message mode, and forwards the message to a stream processing platform through the message middleware;
the stream processing platform matches the event with a preset event matching rule, and sends the event to a subscription relationship processing unit after the matching is successful;
the subscription relation processing unit refers to subscription information of the event and sends the subscription information and the event to a packaging unit;
and the packaging unit packages and distributes the event according to the subscription information.
Another technical solution of the present invention for solving the above technical problems is as follows:
an event processing system based on stream processing, comprising: preprocessing unit, message middleware, stream processing platform, subscription relation processing unit and packaging unit, wherein:
the preprocessing unit is used for preprocessing an event, sending the preprocessed event to the message middleware in a message mode, and forwarding the message to the stream processing platform through the message middleware;
the stream processing platform is used for matching the event with a preset event matching rule, and sending the event to the subscription relationship processing unit after the matching is successful;
the subscription relation processing unit is used for looking up subscription information of the event and sending the subscription information and the event to the packaging unit;
the packaging unit is used for packaging and distributing the event according to the subscription information.
The invention has the beneficial effects that: the invention firstly forwards the preprocessed event to the stream processing platform through the message middleware in a message mode, then the stream processing platform carries out matching and subscription distribution according to the preset event matching rule, the processing process of the event does not need to store source data and process data, so that more storage resources can be saved.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a flow chart of an event processing method based on stream processing according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating an event processing method according to another embodiment of the present invention;
fig. 3 is a structural framework diagram provided by an embodiment of the stream processing-based event processing system of the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth to illustrate, but are not to be construed to limit the scope of the invention.
As shown in fig. 1, a schematic flow chart is provided for an embodiment of an event processing method based on stream processing, where the method is suitable for processing a real-time event, and includes:
and S1, the preprocessing unit preprocesses the event, sends the preprocessed event to the message middleware in a message mode, and forwards the message to the stream processing platform through the message middleware.
It should be understood that, because events may have data loss and the like during the collection process, or data in different formats may occur, preprocessing is required to complete the data and unify the formats for subsequent processing.
For example, data cleansing and data normalization may be performed on the data.
Preferably, kafka message middleware can be used, and when the stream processing platform receives the kafka message, the kafka brooker address, the subject and consumer group code of the message, the number of threads for receiving the message, and the like need to be set.
And S2, the stream processing platform matches the event with a preset event matching rule, and sends the event to the subscription relation processing unit after the matching is successful.
It should be understood that the specific event matching rule can be set according to actual requirements, and when the execution result of the rule expression is true, the event rule matching is successful; if the execution result of the expression is false, the event rule matching is not successful, and the piece of data is discarded. For example, taking "watch Tencent video" event as an example, the regular expression of "watch Tencent video" rule may be "WEBID ═ 5_ 30097'". If the user is watching the flight video, the value of WEBID of the source data is '5_30097', so that the execution result of the expression is '5_30097' to '5_30097', which is true. If other web sites are visited, the source data WEBID is other value, so the expression result is false.
And S3, the subscription relation processing unit consults the subscription information of the event and sends the subscription information and the event to the packaging unit.
It should be understood that the subscription information includes fields and output formats of subscription, subscription time period, subscription frequency, output topic, etc., for example, still taking "watch flight video" event as an example, the subscription system real-time marketing system (number 100) has the subscription fields of "mobile phone number", "website name", "watching time", the output format of 1(1 represents JSON, 2 represents comma-separated character string), the subscription event of 0900-2000, the subscription frequency of 1DD/0HH/0-1 (representing 1 day and 1 time at most), and the output topic of Tdata 000.
And S4, the packaging unit packages and distributes the event according to the subscription information.
For example, it may be determined whether the current time is a subscription time, whether a preset subscription frequency is reached from the last output, and the like, for example, if the subscription frequency is at most 1 time in 1 day before the last packaged event is 2 hours, it is obviously not in accordance with the requirement of the subscription frequency, the data is discarded. And writing the current number, the event and the current time into a cache if the subscription requirement is met, and continuing to the next step.
In the embodiment, a preprocessed event is forwarded to a stream processing platform through message middleware in a message mode, then the stream processing platform performs matching and subscription distribution according to a preset event matching rule, source data and process data do not need to be stored in the event processing process, more storage resources can be saved, when a new event message is accessed, only the source data needs to be processed in a standardized mode, subsequent processing is realized through configuration, the development workload can be greatly saved, the data processing delay is reduced from a minute level to a second level, and the method is suitable for processing data with high real-time requirements.
Optionally, in some possible embodiments, the preprocessing unit preprocesses the event, specifically including:
the preprocessing unit cleans the events and standardizes the cleaned events.
It should be understood that the event cleaning is to check the important field information of the source data, and to perform the completion processing for the completable source data with the missing important field information, otherwise, to discard the important field information.
For example, for a missing key field, such as a missing mobile phone number, the corresponding mobile phone number may be found by the IMSI. If the mobile phone number and the IMSI are both empty, the piece of data is discarded.
It should be appreciated that because source data formats are typically very different, e.g., may be xml format, json format, and also separated by characters, normalization is required. The standardized format can be json format, and besides existing field information of the source data, the source data type coding and processing time stamp field can be added.
By cleaning and standardizing the source data, the subsequent processing of the data can be facilitated, and the processing efficiency is improved.
Optionally, in some possible embodiments, the event matching rule includes: the event matching method comprises a simple event matching rule and a complex event matching rule, wherein the simple event matching rule comprises the following steps: at least one of event source coding, event identification, event name, failure identification and regular expression; the complex event matching rules include: at least one simple event, and a time of establishment of each simple event.
It should be understood that the event rule configuration data may be stored in the mysql database, and the rule data contains event source code, event identification, event name, event generation and failure identification, rule expression, and the like. For example, taking "watch Tencent video" as an example, the event source code may be 1000001, the event identifier may be 4129715169713152, the event name may be "watch Tencent video", the event invalidation identifier may be 1000(1000 stands for valid, 1100 stands for invalid), and the regular expression may be "WEBID ═ 5_ 30097'".
It should be appreciated that the simple event rule loaded requires that two conditions be satisfied: 1. the event is valid 2. the event is subscribed or the attributed complex event is subscribed.
The event subscription means that a description is provided for which systems subscribe to an event, which fields subscribe to and output format (json or character string segmentation), time period and frequency of output (the same number and the same event are once a day, once an hour or once a month), and output subject.
When an event is processed, it may be sent over a communication link to a system that has subscribed to the event.
It should be noted that the complex event may be defined according to actual requirements, the complex event rule defines which simple events constitute a complex event, and how long the simple events are established, the complex event is established. For example, if a user is running a super-set of events and the user watches an Tencent video event simultaneously within half an hour, then a complex event may be defined, each complex event consisting of at least one simple event.
It should be understood that the simple events composing the complex event do not necessarily need to have any relationship, and from the practical point of view, if the success rate of marketing recommendation is higher if the complex event is satisfied, the simple events can be combined into the complex event. For example, a user watching a flight video uses a large amount of traffic, the user may have a long idle time when arriving at an airport, and two simple events of watching the flight video and the user arriving at the airport can be combined into a complex event, so that the probability of successfully recommending traffic package packages to the client is higher, and the availability of data is improved.
Optionally, in some possible embodiments, the streaming processing platform matches the event with a preset event matching rule, and sends the event to the subscription relationship processing unit after the matching is successful, where the method specifically includes:
the stream processing platform judges whether the event is a simple event or a complex event, if the event is the simple event, the event is matched with the simple event matching rule, and after the matching is successful, the event is sent to the subscription relation processing unit;
if the event is a complex event, splitting the event into simple events through a complex event matching rule, matching the split simple events with the simple event matching rule, and sending the simple events to a complex event matching unit after matching is successful;
the complex event matching unit inquires the matching duration of the complex event to which the simple event belongs and the list of the associated simple events, stores the associated simple events in a cache, sets the validity period as the matching duration, traverses the list of the simple events, and inquires whether the split simple events exist in the cache.
Specifically, traversal may be performed by an event ID, and if the traversed event ID is the same as the received data event ID, then existence is deemed.
For example, if the complex event B is composed of four simple events a, B, c, and d, this step will determine whether the a, B, c, and d events of the user are stored in the cache, for example, when the a event comes, it will be checked whether the B, c, and d events exist in the redis, and when the same d event comes, it will be checked whether the a, B, and c events exist in the redis.
If all the simple events exist, all the simple events are assembled and then serve as matching data to be sent to the subscription relationship processing unit, for example, all the field information of the simple events can be combined into a new json message to be sent to a subscription relationship matching link.
It should be noted that the associated simple events may be stored in a distributed cache.
It should be understood that for a simple event, it is only necessary to determine whether the execution result of the regular expression is true, for example, as in the "view flight video" rule of the above example, the regular expression is "WEBID ═ 5_30097 '", and if the user is viewing the flight video, the WEBID value of the source data is "5 _ 30097", so that the execution result of the expression is "5 _30097' ═ 5_30097', which is true, the matching is successful. If the user is watching the sourdy video, the WEBID value of the source data is '5_30086', so that the execution result of the expression is '5_30086' ═ 5_30097', which is false, the matching fails.
If the match fails, the piece of data may be discarded.
Optionally, in some possible embodiments, the encapsulating unit encapsulates and distributes the event according to the subscription information, and specifically includes:
and the packaging unit judges whether the current time is within the subscription time period of the event according to the subscription information, if so, judges whether the subscription frequency requirement is met according to the subscription information, if so, assembles the event according to the subscription format according to the subscription information, and writes the assembled data into the corresponding message middleware theme.
For example, assuming that the subscription period is 9 am to 11 am, if the current event is 10 pm, the current time is not within the subscription period of the event, and the data is discarded. If the current event is 10 am, the current time is in the subscription time period of the event, the subscription frequency requirement is continuously judged, if the current event is satisfied, the assembled data is written into a cache, the data assembly is carried out according to the subscription format, the json format and the character separation format are supported, and then the assembled data is written into the corresponding message middleware theme.
It is understood that some or all of the alternative embodiments described above may be included in some embodiments.
As shown in fig. 2, an event processing flow chart including all the real-time modes is provided, the events are cleaned and standardized, the processed data are matched according to a simple event rule and a complex event rule, subscription processing is performed, and finally the events are output and packaged.
As shown in fig. 3, a structural framework diagram is provided for an embodiment of the event processing system based on stream processing, which is applicable to processing of real-time events, and includes: preprocessing unit 1, message middleware 2, stream processing platform 3, subscription relationship processing unit 4 and packaging unit 5, wherein:
the preprocessing unit 1 is configured to preprocess an event, send the preprocessed event to the message middleware 2 in a message manner, and forward the message to the stream processing platform 3 through the message middleware 2;
the stream processing platform 3 is used for matching the event with a preset event matching rule, and sending the event to the subscription relationship processing unit 4 after the matching is successful;
the subscription relation processing unit 4 is configured to look up subscription information of an event, and send the subscription information and the event to the packaging unit 5;
the packaging unit 5 is used for packaging and distributing the event according to the subscription information.
In the embodiment, a preprocessed event is firstly forwarded to the stream processing platform 3 through the message middleware 2 in a message mode, then the stream processing platform 3 performs matching and subscription distribution according to a preset event matching rule, source data and process data do not need to be stored in the event processing process, more storage resources can be saved, when a new event message is accessed, only the source data needs to be processed in a standardized mode, and subsequent processing is realized through configuration, so that the development workload can be greatly saved, the data processing delay is reduced from a minute level to a second level, and the method is suitable for processing data with high real-time requirements.
Optionally, in some possible embodiments, the pre-processing unit 1 is specifically configured to perform cleaning on the event, and perform standardized processing on the cleaned event.
Optionally, in some possible embodiments, the event matching rule includes: the event matching method comprises a simple event matching rule and a complex event matching rule, wherein the simple event matching rule comprises the following steps: at least one of event source coding, event identification, event name, failure identification and regular expression; the complex event matching rules include: at least one simple event, and a time of establishment of each simple event.
Optionally, in some possible embodiments, the stream processing platform 3 is specifically configured to determine that the event is a simple event or a complex event, match the event with a simple event matching rule if the event is a simple event, and send the event to the subscription relationship processing unit 4 after matching is successful;
if the event is a complex event, splitting the event into simple events through a complex event matching rule, matching the split simple events with the simple event matching rule, and sending the simple events to a complex event matching unit after matching is successful;
the complex event matching unit is used for inquiring the matching duration of the complex event to which the simple event belongs and the list of the associated simple events, storing the associated simple events in the cache, setting the validity period as the matching duration, traversing the list of the simple events, inquiring whether the split simple events exist in the cache, and if all the split simple events exist, splicing all the simple events and sending the spliced simple events as matching data to the subscription relation processing unit 4.
Optionally, in some possible embodiments, the packaging unit 5 is configured to determine, according to the subscription information, whether the current time is within a subscription time period of the event, if so, determine, according to the subscription information, whether a subscription frequency requirement is met, if so, assemble the event according to a subscription format according to the subscription information, and write the assembled data into the corresponding message middleware 2 theme.
It is understood that some or all of the alternative embodiments described above may be included in some embodiments.
It should be noted that the above embodiments are product embodiments corresponding to the previous method embodiments, and for the description of each optional implementation in the product embodiments, reference may be made to corresponding descriptions in the above method embodiments, and details are not described here again.
The reader should understand that in the description of this specification, reference to the description of the terms "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
While the invention has been described with reference to specific embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. An event processing method based on stream processing is characterized by comprising the following steps:
the method comprises the steps that a preprocessing unit preprocesses an event, sends the preprocessed event to a message middleware in a message mode, and forwards the message to a stream processing platform through the message middleware;
the stream processing platform matches the event with a preset event matching rule, and sends the event to a subscription relationship processing unit after the matching is successful;
the subscription relation processing unit refers to subscription information of the event and sends the subscription information and the event to a packaging unit;
and the packaging unit packages and distributes the event according to the subscription information.
2. The event processing method based on stream processing according to claim 1, wherein the preprocessing unit preprocesses the event, specifically comprising:
the preprocessing unit cleans the events and standardizes the cleaned events.
3. The stream processing-based event processing method according to claim 1 or 2, wherein the event matching rule includes: a simple event matching rule and a complex event matching rule, wherein the simple event matching rule comprises: at least one of event source coding, event identification, event name, failure identification and regular expression; the complex event matching rule comprises: at least one simple event, and a time of establishment of each simple event.
4. The event processing method based on stream processing according to claim 3, wherein the stream processing platform matches the event with a preset event matching rule, and sends the event to a subscription relationship processing unit after matching is successful, specifically comprising:
the stream processing platform judges whether the event is a simple event or a complex event, if the event is the simple event, the event is matched with the simple event matching rule, and when the matching is successful, the event is sent to a subscription relationship processing unit;
if the event is a complex event, splitting the event into simple events through the complex event matching rule, matching the split simple events with the simple event matching rule, and sending the simple events to a complex event matching unit after matching is successful;
the complex event matching unit inquires the matching duration of the complex event to which the simple event belongs and a list of associated simple events, stores the associated simple events in a cache, sets the validity period as the matching duration, traverses the list of simple events, inquires whether the split simple events exist in the cache, and sends all the simple events after being spliced as matching data to the subscription relation processing unit if all the simple events exist.
5. The event processing method based on stream processing according to claim 1, wherein the encapsulating unit encapsulates and distributes the event according to the subscription information, specifically comprising:
and the packaging unit judges whether the current time is within the subscription time period of the event according to the subscription information, if so, judges whether the subscription frequency requirement is met according to the subscription information, if so, assembles the event according to a subscription format according to the subscription information, and writes the assembled data into a corresponding message middleware theme.
6. An event processing system based on stream processing, comprising: preprocessing unit, message middleware, stream processing platform, subscription relation processing unit and packaging unit, wherein:
the preprocessing unit is used for preprocessing an event, sending the preprocessed event to the message middleware in a message mode, and forwarding the message to the stream processing platform through the message middleware;
the stream processing platform is used for matching the event with a preset event matching rule, and sending the event to the subscription relationship processing unit after the matching is successful;
the subscription relation processing unit is used for looking up subscription information of the event and sending the subscription information and the event to the packaging unit;
the packaging unit is used for packaging and distributing the event according to the subscription information.
7. The stream processing-based event processing system according to claim 6, wherein the preprocessing unit is specifically configured to clean the events and standardize the cleaned events.
8. The stream processing-based event processing system according to claim 6 or 7, wherein the event matching rule includes: a simple event matching rule and a complex event matching rule, wherein the simple event matching rule comprises: at least one of event source coding, event identification, event name, failure identification and regular expression; the complex event matching rule comprises: at least one simple event, and a time of establishment of each simple event.
9. The event processing system according to claim 8, wherein the stream processing platform is specifically configured to determine that the event is a simple event or a complex event, match the event with the simple event matching rule if the event is a simple event, and send the event to the subscription relationship processing unit after matching is successful;
if the event is a complex event, splitting the event into simple events through the complex event matching rule, matching the split simple events with the simple event matching rule, and sending the simple events to a complex event matching unit after matching is successful;
the complex event matching unit is used for inquiring the matching duration of the complex event to which the simple event belongs and a list of associated simple events, storing the associated simple events in a cache, setting the validity period as the matching duration, traversing the list of simple events, inquiring whether the split simple events exist in the cache, and if all the split simple events exist, splicing all the simple events and sending the spliced simple events as matching data to the subscription relationship processing unit.
10. The event processing system according to claim 6, wherein the packaging unit is configured to determine whether a current time is within a subscription period of the event according to the subscription information, if so, determine whether a subscription frequency requirement is met according to the subscription information, if so, assemble the event according to a subscription format according to the subscription information, and write the assembled data into a corresponding message middleware theme.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010537106.0A CN111708645A (en) | 2020-06-12 | 2020-06-12 | Event processing method and system based on stream processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010537106.0A CN111708645A (en) | 2020-06-12 | 2020-06-12 | Event processing method and system based on stream processing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111708645A true CN111708645A (en) | 2020-09-25 |
Family
ID=72540817
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010537106.0A Pending CN111708645A (en) | 2020-06-12 | 2020-06-12 | Event processing method and system based on stream processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111708645A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112291313A (en) * | 2020-10-15 | 2021-01-29 | 长春创世麒麟科技有限公司 | Event-driven data transfer system and method |
| CN112653749A (en) * | 2020-12-18 | 2021-04-13 | 北京工商大学 | Edge computing-based complex event processing system and method for Internet of things |
| CN113781101A (en) * | 2021-08-09 | 2021-12-10 | 深圳市惟客数据科技有限公司 | Event center management method and device based on real-time calculation and readable medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103455611A (en) * | 2013-09-06 | 2013-12-18 | 北京邮电大学 | Data processing method for RFID (Radio Frequency Identification) middleware |
| CN103458033A (en) * | 2013-09-04 | 2013-12-18 | 北京邮电大学 | System for providing services of event-driven service-oriented internet of things and working method thereof |
| CN105488223A (en) * | 2015-12-28 | 2016-04-13 | 中国民航信息网络股份有限公司 | Analyzing and subscribing method and system of business data events |
| CN109271261A (en) * | 2018-08-29 | 2019-01-25 | 中国建设银行股份有限公司 | Method, equipment and storage medium is uniformly processed in a kind of event |
| US20190236194A1 (en) * | 2018-01-31 | 2019-08-01 | Splunk Inc. | Dynamic query processor for streaming and batch queries |
| CN110222091A (en) * | 2019-06-10 | 2019-09-10 | 帷幄匠心科技（杭州）有限公司 | A kind of mass data real-time statistic analysis method |
| CN110543464A (en) * | 2018-12-12 | 2019-12-06 | 广东鼎义互联科技股份有限公司 | Big data platform applied to smart park and operation method |
| CN110647385A (en) * | 2019-08-23 | 2020-01-03 | 南京万米信息技术有限公司 | Distributed transaction execution method, device and system |
| CN111245907A (en) * | 2019-12-31 | 2020-06-05 | 畅捷通信息技术股份有限公司 | Event processing method and system based on rule matching |
-
2020
- 2020-06-12 CN CN202010537106.0A patent/CN111708645A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103458033A (en) * | 2013-09-04 | 2013-12-18 | 北京邮电大学 | System for providing services of event-driven service-oriented internet of things and working method thereof |
| CN103455611A (en) * | 2013-09-06 | 2013-12-18 | 北京邮电大学 | Data processing method for RFID (Radio Frequency Identification) middleware |
| CN105488223A (en) * | 2015-12-28 | 2016-04-13 | 中国民航信息网络股份有限公司 | Analyzing and subscribing method and system of business data events |
| US20190236194A1 (en) * | 2018-01-31 | 2019-08-01 | Splunk Inc. | Dynamic query processor for streaming and batch queries |
| CN109271261A (en) * | 2018-08-29 | 2019-01-25 | 中国建设银行股份有限公司 | Method, equipment and storage medium is uniformly processed in a kind of event |
| CN110543464A (en) * | 2018-12-12 | 2019-12-06 | 广东鼎义互联科技股份有限公司 | Big data platform applied to smart park and operation method |
| CN110222091A (en) * | 2019-06-10 | 2019-09-10 | 帷幄匠心科技（杭州）有限公司 | A kind of mass data real-time statistic analysis method |
| CN110647385A (en) * | 2019-08-23 | 2020-01-03 | 南京万米信息技术有限公司 | Distributed transaction execution method, device and system |
| CN111245907A (en) * | 2019-12-31 | 2020-06-05 | 畅捷通信息技术股份有限公司 | Event processing method and system based on rule matching |
Non-Patent Citations (1)
| Title |
|---|
| 朱奕健 等: "基于通信运营商数据的大数据实时流处理系统", 《中国新通信》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112291313A (en) * | 2020-10-15 | 2021-01-29 | 长春创世麒麟科技有限公司 | Event-driven data transfer system and method |
| CN112653749A (en) * | 2020-12-18 | 2021-04-13 | 北京工商大学 | Edge computing-based complex event processing system and method for Internet of things |
| CN113781101A (en) * | 2021-08-09 | 2021-12-10 | 深圳市惟客数据科技有限公司 | Event center management method and device based on real-time calculation and readable medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111708645A (en) | Event processing method and system based on stream processing | |
| CN107889069B (en) | Short message gateway selection method, device, server and readable storage medium | |
| CN109002484B (en) | Method and system for sequentially consuming data | |
| US20140359649A1 (en) | Methods and apparatus for audience data collection and analysis in a content delivery network | |
| CN109120461B (en) | A kind of service feature end-to-end monitoring method, system and device | |
| CN113037823A (en) | Message transmission system and method | |
| CN108076098A (en) | A kind of method for processing business and system | |
| CN103401934A (en) | Method and system for acquiring log data | |
| CN107332719A (en) | A kind of method that daily record is analyzed in real time in CDN system | |
| CN106210132B (en) | A kind of web camera configuration method, apparatus and system | |
| CN106375360B (en) | Graph data updating method, device and system | |
| CN109947729B (en) | Real-time data analysis method and device | |
| CN105512910A (en) | Target user screening method and apparatus | |
| CN105847231A (en) | Service publishing method, device and system | |
| CN111339357A (en) | Recommendation method and device based on live user behaviors | |
| CN114817190A (en) | Log synchronization method, device, system, equipment and storage medium | |
| CN110956485B (en) | Message subscription method and component based on RocktMQ and message subscription system | |
| CN106156258B (en) | Method, device and system for counting data in distributed storage system | |
| CN114238295A (en) | Data sorting method and device based on grouping | |
| CN113240139A (en) | Alarm cause and effect evaluation method, fault root cause positioning method and electronic equipment | |
| CN111813503A (en) | Micro-service application open system based on container cloud | |
| CN112468840A (en) | Tamper-proof system and method for third-party EPG server in IPTV system | |
| CN111431664A (en) | Dispatching data packet download method and device based on JSON data protocol | |
| CN111582996B (en) | Service information display method and device | |
| CN105847886A (en) | Internet television big data intelligent service platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200925 |