CN115145964A - Time sequence data integration method, device, equipment and medium - Google Patents
Time sequence data integration method, device, equipment and medium Download PDFInfo
- Publication number
- CN115145964A CN115145964A CN202210912546.9A CN202210912546A CN115145964A CN 115145964 A CN115145964 A CN 115145964A CN 202210912546 A CN202210912546 A CN 202210912546A CN 115145964 A CN115145964 A CN 115145964A
- Authority
- CN
- China
- Prior art keywords
- cep
- rule
- state machine
- nfa
- deterministic state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000010354 integration Effects 0.000 title claims abstract description 35
- 238000012544 monitoring process Methods 0.000 claims abstract description 28
- 238000012216 screening Methods 0.000 claims abstract description 14
- 238000004590 computer program Methods 0.000 claims description 15
- 238000001914 filtration Methods 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 5
- 230000002123 temporal effect Effects 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 14
- 230000004044 response Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000004080 punching Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24568—Data stream processing; Continuous queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a time sequence data integration method, which comprises the following steps: monitoring a CEP rule base, and intercepting an access request of a current non-deterministic state machine NFA object when monitoring that a current CEP rule is updated; generating a new non-deterministic state machine NFA object and replacing the current non-deterministic state machine NFA with the new non-deterministic state machine NFA object; analyzing the updated CEP rule base to obtain a signal set of the CEP rule base; screening out data corresponding to the signal set from the monitored one or more data streams; screening out rule description events from data corresponding to the signal set; searching context information associated with the description information from a time sequence library; the description information is integrated with the context information. The invention can dynamically load a new matching rule under the condition of not restarting a real-time Flink (real-time computing) program, thereby having the real-time response processing capability of low delay, high throughput and high efficiency when processing data.
Description
Technical Field
The invention belongs to the technical field of data processing, and particularly relates to a time sequence data integration method, a time sequence data integration device, time sequence data integration equipment and a time sequence data integration medium.
Background
Complex Event Processing (CEP) is Event Processing that combines data from multiple sources to infer events or patterns that indicate more Complex situations. The goal of complex event processing is to identify meaningful events (e.g., opportunities, threats) and quickly respond to them. The data collected from the terminal equipment of the internet of things generally have certain time sequence, and meanwhile disorder and delay conditions are easy to occur.
Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, the present invention provides a method, an apparatus, a device and a medium for integrating time series data to solve the above-mentioned technical problems.
The invention provides a time sequence data integration method, which comprises the following steps:
monitoring a CEP rule base, and when monitoring that a current CEP rule is updated, intercepting an access request of a current non-deterministic state machine (NFA) object, wherein the CEP rule base comprises a plurality of CEP rules;
generating a new non-deterministic state machine NFA object according to the updated CEP rule, and replacing the current non-deterministic state machine NFA with the new non-deterministic state machine NFA object;
analyzing the updated CEP rule base to obtain a signal set of the CEP rule base, wherein each CEP rule corresponds to a signal;
screening out data corresponding to the signal set from the monitored one or more data streams;
screening out rule description events from the data corresponding to the signal set by using rule matching conditions;
according to the description information of the rule description event, searching context information associated with the description information from a time sequence library;
and integrating the description information and the context information, and writing the description information and the context information into a downstream application.
In an embodiment of the invention, the CEP rule includes an event determination basic rule and a data item to be integrated in an information element of an event.
In an embodiment of the present invention, the monitoring CEP rule base includes:
actively notified by the program;
or monitoring a log of the CEP rule base, and capturing change data from the log through the Dbus tool, thereby realizing monitoring of the CEP rules.
In an embodiment of the present invention, before intercepting an access request of a current non-deterministic state machine NFA object, the method further includes:
the state of the current data stream is saved and a new save point savepoint is generated.
In an embodiment of the invention, before replacing the current non-deterministic state machine NFA with the new non-deterministic state machine NFA object, the method further comprises:
emptying intermediate state storage data related to the current non-deterministic state machine NFA object and initializing a new non-deterministic state machine NFA.
In an embodiment of the present invention, the description information includes a body and a time when a rule describes an event, the context information includes information associated with the body, and the downstream application includes at least one of Kafka message middleware, a database, and a storage medium.
The invention provides a time sequence data integration device, which comprises:
the monitoring module is used for monitoring a CEP rule base and intercepting an access request of a current non-deterministic state machine NFA object when monitoring that a current CEP rule is updated, wherein the CEP rule base comprises a plurality of CEP rules;
an updating module, configured to generate a new non-deterministic state machine NFA object according to the updated CEP rule, and replace the current non-deterministic state machine NFA with the new non-deterministic state machine NFA object;
the analysis module is used for analyzing the updated CEP rule base to obtain a signal set of the CEP rule base, wherein each CEP rule corresponds to a signal;
a first filtering module for filtering out data corresponding to the signal set from the monitored one or more data streams;
the second screening module screens out rule description events from the data corresponding to the signal set by using rule matching conditions;
the searching module is used for searching context information related to the description information from a time sequence library according to the description information of the rule description event;
and the integration module is used for integrating the description information and the context information and writing the description information and the context information into downstream application.
The invention provides an electronic device, comprising:
one or more processors;
a storage device for storing one or more programs which, when executed by the one or more processors, cause the electronic device to implement the steps of the above-described time series data integration method.
The present invention provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor of a computer, causes the computer to perform the steps of the above-mentioned time series data integration method.
The invention has the beneficial effects that: the invention discloses a time sequence data integration method, which comprises the following steps: monitoring a CEP rule base, and intercepting an access request of a current non-deterministic state machine NFA object when monitoring that a current CEP rule is updated, wherein the CEP rule base comprises a plurality of CEP rules; generating a new non-deterministic state machine NFA object according to the updated CEP rule, and replacing the current non-deterministic state machine NFA with the new non-deterministic state machine NFA object; analyzing the updated CEP rule base to obtain a signal set of the CEP rule base, wherein each CEP rule corresponds to a signal; screening out data corresponding to the signal set from the monitored one or more data streams; screening out rule description events from the data corresponding to the signal set by using rule matching conditions; according to the description information of the rule description event, searching context information associated with the description information from a time sequence library; and integrating the description information and the context information, and writing the description information and the context information into downstream application. The invention clears and updates the NFA of the non-deterministic state machine when the CEP rule is dynamically updated, initializes the NFA of the new non-deterministic state machine, and can dynamically load the new matching rule under the condition of not restarting a real-time Flink (real-time computing) program, thereby having the real-time response processing capability of low delay, high throughput and high efficiency when processing data; the CEP rules include various types, and various event scenes can be comprehensively identified; in addition, the rule base and the time sequence data element information to be fused are stored in a background database, when the rule changes, only the configuration needs to be updated, and the logic elements identified by the event are stored in the database, so that the analysis and the management are convenient.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort. In the drawings:
FIG. 1 is a schematic diagram of an implementation environment of a time series data integration method according to an exemplary embodiment of the present application;
FIG. 2 is a flow chart illustrating a method of time series data integration in accordance with an exemplary embodiment of the present application;
FIG. 3 is a block diagram of a time series data integration apparatus shown in an exemplary embodiment of the present application;
FIG. 4 illustrates a schematic structural diagram of a computer system suitable for use in implementing the electronic device of an embodiment of the present application.
Detailed Description
Other advantages and effects of the present invention will become apparent to those skilled in the art from the disclosure herein, wherein the embodiments of the present invention are described in detail with reference to the accompanying drawings and preferred embodiments. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, and are not intended to limit the scope of the present invention.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
In the following description, numerous details are set forth to provide a more thorough explanation of embodiments of the present invention, however, it will be apparent to one skilled in the art that embodiments of the present invention may be practiced without these specific details, and in other embodiments, well-known structures and devices are shown in block diagram form, rather than in detail, to avoid obscuring embodiments of the present invention.
FIG. 1 is a diagram illustrating an exemplary environment for implementing a method for integrating time series data according to the present application. Referring to fig. 1, the implementation environment includes a terminal device 101 and a server 102, and the terminal device 101 and the server 102 communicate with each other through a wired or wireless network. The server can monitor a CEP rule base, and when monitoring that a current CEP rule is updated, the server can intercept an access request of a current non-deterministic state machine NFA object, wherein the CEP rule base comprises a plurality of CEP rules; generating a new non-deterministic state machine NFA object according to the updated CEP rule, and replacing the current non-deterministic state machine NFA with the new non-deterministic state machine NFA object; analyzing the updated CEP rule base to obtain a signal set of the CEP rule base, wherein each CEP rule corresponds to a signal; screening out data corresponding to the signal set from the monitored one or more data streams; screening out rule description events from the data corresponding to the signal set by using rule matching conditions; according to the description information of the rule description event, searching context information associated with the description information from a time sequence library; and integrating the description information and the context information, and writing the description information and the context information into a downstream application.
It should be understood that the number of terminal devices 101 and servers 102 in fig. 1 is merely illustrative. There may be any number of terminal devices 101 and servers 102, as desired.
The terminal device 101 corresponds to a client, and may be any electronic device having a user input interface, including but not limited to a smart phone, a tablet, a notebook computer, a vehicle-mounted computer, and the like, where the user input interface includes but not limited to a touch screen, a keyboard, a physical key, an audio pickup device, and the like.
The server 102 corresponds to a server, may be a server providing various services, may be an independent physical server, may also be a server cluster or a distributed system formed by a plurality of physical servers, and may also be a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a CDN (Content Delivery Network), a big data and an artificial intelligence platform, which is not limited herein.
The terminal device 101 may communicate with the server 102 through a wireless network such as 3G (third generation mobile information technology), 4G (fourth generation mobile information technology), 5G (fifth generation mobile information technology), and the like, which is not limited herein.
Embodiments of the present application respectively provide a time series data integration method, a time series data integration apparatus, an electronic device, and a computer readable storage medium, and the embodiments will be described in detail below.
Referring to fig. 2, fig. 2 is a flowchart illustrating a time series data integration method according to an exemplary embodiment of the present application. The method may be applied to the implementation environment shown in fig. 1 and specifically executed by the terminal device 101 in the implementation environment. It should be understood that the method may be applied to other exemplary implementation environments and is specifically executed by devices in other implementation environments, and the embodiment does not limit the implementation environment to which the method is applied.
Referring to fig. 2, fig. 2 is a flowchart illustrating an exemplary time series data integration method according to the present application, which at least includes steps S210 to S270, and the following is detailed:
step S210, a CEP rule base is monitored, when the fact that the current CEP rule is updated is monitored, an access request of a current non-determinacy state machine NFA object is intercepted, and the CEP rule base comprises a plurality of CEP rules;
monitoring the CEP rule library is to monitor the CEP rule, and may be understood as monitoring a data result of the CEP rule, and determining whether the CEP rule is updated, that is, determining whether the data result of the CEP rule is updated.
Note that a plurality of CEP rules including data items to be integrated in the information elements of the event and the event determination basic rule are stored in the CEP rule library.
In one embodiment, the listening of the CEP rule base is done proactively by the program.
In another embodiment, the monitoring of the CEP rule base is to monitor a log of the CEP rule base, and the change data is captured from the log by the Dbus tool, so as to realize the monitoring of the CEP rule. Please enter a header DBus to focus on data collection and real-time data stream calculation, collect source data in a non-intrusive way through simple and flexible configuration, and converge data generated in a business process by adopting a highly available stream calculation framework.
It should be noted that the CEP rules include three types, namely strict neighbor, loose neighbor, and non-deterministic loose neighbor, and can be identified according to the three types of rules. Wherein,
strict nearest neighbor: indicating that all events occur in strict order without any mismatch in between, as determined by the next state immediately following. As for pattern "a strict neighbor b", the sequence of events [ a, c, b1, b2] does not match.
Loose neighbor: events indicating that a mismatch is allowed to occur in the middle are determined by the next subsequent data. For example, for b1, b2 satisfying state b, for pattern "a loose neighbor b", the sequence of events [ a, c, b1, b2] is matched to { a, b 1).
Non-deterministic loose nearest neighbor: indicating further relaxation of the condition, events that have been previously matched can also be reused, as determined by the next subsequent data. For example for b1, b2 satisfying state b, for pattern "a non-deterministic loose neighbor b", the sequence of events [ a, c, b1, b2] is matched to { a, b 1), { a, b 2).
The invention can comprehensively identify various event scenes by supporting three CEP rule modes of strict neighbor, loose neighbor and uncertain loose. And different types of event recognition scenes can be met through configuration change, application programs do not need to be re-developed and deployed, and the method is simple and efficient.
When monitoring that the current CEP rule is updated, intercepting an access request of a current Non-deterministic state machine NFA (NFA) object. The current NFA object corresponds to the current CEP rule. The NFA is a non-deterministic state machine that defines a pattern for matching complex events. The monitoring of the current CEP rule is in response to an update request of a user, and the update request is a request for updating the current CEP rule to a new CEP rule. The request carries the updated CEP rules.
It should be understood that since the working mechanism of Flink is to translate CEP rules into NFA objects, the subsequent pattern matching process is an operation on the NFA objects. When it is monitored that the current CEP rule is updated, an access request of the current NFA object needs to be intercepted, and then the NFA object is correspondingly changed according to the updated CEP rule.
It should be noted that when the current CEP rule is not monitored to be updated, the monitoring is continued.
It should be further noted that, before intercepting the access request of the current non-deterministic state machine NFA object, the state of the current data stream should be saved to generate a new saving point savepoint, and save state information such as watermark of the upstream data stream.
Step S220, generating a new non-deterministic state machine NFA object according to the updated CEP rule, and replacing the current non-deterministic state machine NFA with the new non-deterministic state machine NFA object;
it should be noted that different rule engines generate NFA objects according to their execution differences. The Groovy rules engine has JIT (Just In Time) characteristics and needs to generate new execution bytecodes online. The Drools rule engine can generate a new NFA object by using the rope punching generation rule description file.
Based on rule engines such as Groovy and Drools, the CEO rules are stored in a database (rule base), and the flag CEP rules can be dynamically changed by configuration updating, so that the CEP rule base can be dynamically updated.
In an embodiment, before replacing the current non-deterministic state machine NFA with the new non-deterministic state machine NFA object, the method further comprises:
emptying intermediate state storage data related to the current non-deterministic state machine NFA object and initializing a new non-deterministic state machine NFA.
Specifically, after the new non-deterministic state machine NFA object is replaced, an application program interface of the Flink state storage object is called, the intermediate state storage data in the state storage object is emptied, and the NFA object is initialized through the application program interface, so that the data of the intermediate state storage object is restored to an initial value.
The invention clears and updates the NFA of the non-deterministic state machine when the CEP rule is dynamically updated, initializes the NFA of the new non-deterministic state machine, and can dynamically load the new matching rule under the condition of not restarting a real-time Flink (real-time computing) program, thereby having the real-time response processing capability of low delay, high throughput and high efficiency when processing data.
Step S230, analyzing the updated CEP rule base to obtain a signal set of the CEP rule base, wherein each CEP rule corresponds to a signal;
it should be noted that the CEP rule and the data elements to be subsequently integrated are stored in the CEP rule base as a plurality of signals, and all event-related signals form a signal set.
And storing the rule base and the time sequence data element information to be fused in a background database, and only updating the configuration when the rule changes. And the logic elements of the event identification are stored by the database, so that the analysis and management are convenient.
Step S240, screening out data corresponding to the signal set from the monitored one or more data streams;
in addition to the data corresponding to the signal set related to the event, the data stream or data streams also include other data, so that the data needs to be screened to obtain target data, and thus, the data amount processed by the Flink stream can be reduced by screening the data corresponding to the signal set from the monitored data stream or data streams.
Step S250, utilizing rule matching conditions to screen out rule description events from the data corresponding to the signal set;
specifically, rule matching conditions are applied to find rule description events from the data stream. And the Flink loss processing engine calls the rule engine to perform event matching and identifies the occurrence of the event.
Step S260, according to the description information of the rule description event, searching context information related to the description information from a time sequence library;
the description information comprises a body and time when a rule description event occurs, the context information comprises information associated with the body, and the downstream application comprises at least one of Kafka message middleware, a database and a storage medium. The subject may be an agent, a participant, a terminal device where an event occurs, and the like.
And step S270, integrating the description information and the context information, and writing the description information and the context information into a downstream application.
In an embodiment, the downstream application comprises at least one of Kafka message middleware, a database, and a storage medium.
Fig. 3 is a block diagram of a time-series data integration apparatus according to an exemplary embodiment of the present application. The device can be applied to the implementation environment shown in fig. 1 and is specifically configured in the terminal equipment. The apparatus may also be applied to other exemplary implementation environments and specifically configured in other devices, and the embodiment does not limit the implementation environment to which the apparatus is applied.
As shown in fig. 3, the present application provides a time series data integration apparatus, which includes:
a monitoring module 310, configured to monitor a CEP rule base, and intercept an access request of a current non-deterministic state machine NFA object when it is monitored that a current CEP rule is updated, where the CEP rule base includes multiple CEP rules;
an update module 320, configured to generate a new non-deterministic state machine NFA object according to the updated CEP rule, and replace the current non-deterministic state machine NFA with the new non-deterministic state machine NFA object;
the analysis module 330 is configured to analyze the updated CEP rule base to obtain a signal set of the CEP rule base, where each CEP rule corresponds to one signal;
a first filtering module 340 configured to filter out data corresponding to the signal set from the monitored one or more data streams;
a second filtering module 350, configured to filter rule description events from the data corresponding to the signal set by using rule matching conditions;
the searching module 360 is configured to search, according to the description information of the rule description event, context information associated with the description information from a time sequence library;
and an integrating module 370, configured to integrate the description information and the context information, and write the description information and the context information into a downstream application.
It should be noted that the time series data integration apparatus provided in the foregoing embodiment and the time series data integration method provided in the foregoing embodiment belong to the same concept, and specific ways of performing operations by each module and unit have been described in detail in the method embodiments, and are not described herein again. In practical applications, the time-series data integration apparatus provided in the above embodiment may distribute the functions through different functional modules according to needs, that is, divide the internal structure of the apparatus into different functional modules to complete all or part of the functions described above, which is not limited herein.
An embodiment of the present application further provides an electronic device, including: one or more processors; a storage device, configured to store one or more programs, which when executed by the one or more processors, enable the electronic device to implement the time-series data integration method provided in the above-described embodiments.
FIG. 4 illustrates a schematic structural diagram of a computer system suitable for use in implementing the electronic device of an embodiment of the present application. It should be noted that the computer system 400 of the electronic device shown in fig. 4 is only an example, and should not bring any limitation to the functions and the application scope of the embodiments of the present application.
As shown in fig. 4, the computer system 400 includes a Central Processing Unit (CPU) 401, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, according to a program stored in a Read-Only Memory (ROM) 402 or a program loaded from a storage section 408 into a Random Access Memory (RAM) 403. In the RAM 403, various programs and data necessary for system operation are also stored. The CPU 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An Input/Output (I/O) interface 405 is also connected to the bus 404.
The following components are connected to the I/O interface 405: an input portion 406 including a keyboard, a mouse, and the like; an output section 407 including a Display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 408 including a hard disk and the like; and a communication section 407 including a Network interface card such as a LAN (Local Area Network) card, a modem, or the like. The communication section 407 performs communication processing via a network such as the internet. A driver 410 is also connected to the I/O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 410 as necessary, so that a computer program read out therefrom is mounted into the storage section 408 as necessary.
In particular, according to embodiments of the application, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising a computer program for performing the method illustrated in flowchart 2. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 409, and/or installed from the removable medium 411. The computer program executes various functions defined in the system of the present application when executed by a Central Processing Unit (CPU) 401.
It should be noted that the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM), a flash Memory, an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer-readable signal medium may include a propagated data signal with a computer program embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present application may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.
Another aspect of the present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor of a computer, causes the computer to execute the time-series data integration method as described above. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist separately without being incorporated in the electronic device.
Another aspect of the application also provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the time-series data integration method provided in the above embodiments.
The foregoing embodiments are merely illustrative of the principles of the present invention and its efficacy, and are not to be construed as limiting the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.
Claims (9)
1. A method for time series data integration, the method comprising:
monitoring a CEP rule base, and intercepting an access request of a current non-deterministic state machine NFA object when monitoring that a current CEP rule is updated, wherein the CEP rule base comprises a plurality of CEP rules;
generating a new non-deterministic state machine NFA object according to the updated CEP rule, and replacing the current non-deterministic state machine NFA with the new non-deterministic state machine NFA object;
analyzing the updated CEP rule base to obtain a signal set of the CEP rule base, wherein each CEP rule corresponds to a signal;
screening out data corresponding to the signal set from the monitored one or more data streams;
screening out rule description events from the data corresponding to the signal set by using rule matching conditions;
according to the description information of the rule description event, searching context information associated with the description information from a time sequence library;
and integrating the description information and the context information, and writing the description information and the context information into a downstream application.
2. The method of claim 1, wherein the CEP rules include event decision rules and data items to be integrated in the information elements of events.
3. The method of claim 1, wherein the listening to the CEP rule base comprises:
actively notified by the program;
or monitoring a log of the CEP rule base, and capturing change data from the log through the Dbus tool, thereby realizing monitoring of the CEP rules.
4. The method of temporal data integration according to claim 1, characterized in that before intercepting an access request of a current non-deterministic state machine NFA object, the method further comprises:
the state of the current data stream is saved and a new save point savepoint is generated.
5. The method of time-series data integration according to claim 1, wherein before replacing a current non-deterministic state machine NFA with said new non-deterministic state machine NFA object, said method further comprises:
emptying intermediate state storage data related to the current non-deterministic state machine NFA object and initializing a new non-deterministic state machine NFA.
6. The time-series data integration method of claim 1, wherein the description information comprises a body and a time of a rule description event, the context information comprises information associated with the body, and the downstream application comprises at least one of Kafka message middleware, a database, and a storage medium.
7. An apparatus for integrating time series data, the apparatus comprising:
the monitoring module is used for monitoring a CEP rule base and intercepting an access request of a current non-deterministic state machine NFA object when monitoring that a current CEP rule is updated, wherein the CEP rule base comprises a plurality of CEP rules;
an updating module, configured to generate a new non-deterministic state machine NFA object according to the updated CEP rule, and replace the current non-deterministic state machine NFA with the new non-deterministic state machine NFA object;
the analysis module is used for analyzing the updated CEP rule base to obtain a signal set of the CEP rule base, wherein each CEP rule corresponds to a signal;
a first filtering module for filtering out data corresponding to the signal set from the monitored one or more data streams;
the second screening module screens out rule description events from the data corresponding to the signal set by using rule matching conditions;
the searching module is used for searching context information related to the description information from a time sequence library according to the description information of the rule description event;
and the integration module is used for integrating the description information and the context information and writing the description information and the context information into downstream application.
8. An electronic device, characterized in that the electronic device comprises:
one or more processors;
storage means for storing one or more programs which, when executed by the one or more processors, cause the electronic device to carry out the steps of the time-series data integration method according to any one of claims 1 to 6.
9. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor of a computer, causes the computer to carry out the steps of the time series data integration method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210912546.9A CN115145964A (en) | 2022-07-30 | 2022-07-30 | Time sequence data integration method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210912546.9A CN115145964A (en) | 2022-07-30 | 2022-07-30 | Time sequence data integration method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115145964A true CN115145964A (en) | 2022-10-04 |
Family
ID=83414416
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210912546.9A Pending CN115145964A (en) | 2022-07-30 | 2022-07-30 | Time sequence data integration method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115145964A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117893638A (en) * | 2024-03-18 | 2024-04-16 | 上海朋熙半导体有限公司 | Timing diagram generation method, device and equipment of fusion state machine and storage medium |
-
2022
- 2022-07-30 CN CN202210912546.9A patent/CN115145964A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117893638A (en) * | 2024-03-18 | 2024-04-16 | 上海朋熙半导体有限公司 | Timing diagram generation method, device and equipment of fusion state machine and storage medium |
| CN117893638B (en) * | 2024-03-18 | 2024-06-11 | 上海朋熙半导体有限公司 | Timing diagram generation method, device and equipment of fusion state machine and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110347716B (en) | Log data processing method, device, terminal equipment and storage medium | |
| US11502930B2 (en) | Method and system for generating alerts using parameter based network monitoring for alert conditions | |
| CN112527848B (en) | Report data query method, device and system based on multiple data sources and storage medium | |
| CN111049878A (en) | Method, device, electronic device and storage medium for visualizing micro-service call chain | |
| CN110069495A (en) | Date storage method, device and terminal device | |
| US20130013549A1 (en) | Hardware-assisted approach for local triangle counting in graphs | |
| CN116436772B (en) | Real-time alarm method and device, electronic equipment and storage medium | |
| CN109360106B (en) | Sketch construction method, system, medium and computer system | |
| CN112115113A (en) | Data storage system, method, device, equipment and storage medium | |
| CN111985197A (en) | Template generation method based on medical information | |
| CN110245684B (en) | Data processing method, electronic device, and medium | |
| CN115145964A (en) | Time sequence data integration method, device, equipment and medium | |
| CN113256355B (en) | Method, device, medium, equipment and system for determining integral rights and interests in real time | |
| CN110825589A (en) | Anomaly detection method and device for micro-service system and electronic equipment | |
| CN114356712A (en) | Data processing method, device, equipment, readable storage medium and program product | |
| CN112052398A (en) | Media information recommendation method and device, electronic equipment and storage medium | |
| CN115563160A (en) | Data processing method, data processing device, computer equipment and computer readable storage medium | |
| CN113111367B (en) | Security information management method, device and system | |
| US11720414B2 (en) | Parallel execution controller for partitioned segments of a data model | |
| CN116860541A (en) | Service data acquisition method, device, computer equipment and storage medium | |
| CN114564286A (en) | Rule engine warning method and rule engine warning system | |
| CN113590447A (en) | Buried point processing method and device | |
| CN113569256A (en) | Vulnerability scanning method and device, vulnerability scanning system, electronic equipment and computer readable medium | |
| CN113162958A (en) | Method, device and equipment for intelligently pushing message and storage medium | |
| Shah et al. | Multi-GPU approach for development of parallel and scalable pub-sub system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |