CN117591560A - Event discovery and tracking method and system based on hydropower station real-time signals - Google Patents
Event discovery and tracking method and system based on hydropower station real-time signals Download PDFInfo
- Publication number
- CN117591560A CN117591560A CN202311605270.0A CN202311605270A CN117591560A CN 117591560 A CN117591560 A CN 117591560A CN 202311605270 A CN202311605270 A CN 202311605270A CN 117591560 A CN117591560 A CN 117591560A
- Authority
- CN
- China
- Prior art keywords
- event
- real
- signal
- time
- time data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 77
- 238000001514 detection method Methods 0.000 claims abstract description 22
- 238000013507 mapping Methods 0.000 claims abstract description 17
- 230000004927 fusion Effects 0.000 claims abstract description 7
- 238000012544 monitoring process Methods 0.000 claims description 24
- 238000005065 mining Methods 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000009825 accumulation Methods 0.000 claims description 7
- 238000003032 molecular docking Methods 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 7
- 238000012216 screening Methods 0.000 claims description 6
- 238000012423 maintenance Methods 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims description 4
- 238000007499 fusion processing Methods 0.000 claims description 4
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000739 chaotic effect Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000012258 culturing Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24568—Data stream processing; Continuous queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Economics (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Water Supply & Treatment (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Public Health (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
The invention discloses an event discovery and tracking method and system based on hydropower station real-time signals, comprising the following steps: based on historical signal data and historical operation instruction data of the hydropower station, a typical association relation event library is established, unique identification signals or signal combinations of the events are obtained, and a mapping relation between the events and the signals is obtained; establishing a device-event-signal relationship map according to the mapping relationship between the event and the signal and based on the relationship between the device and the event; acquiring multi-source real-time data, and carrying out data fusion on the acquired multi-source real-time data to obtain a fused real-time data stream; according to the fused real-time data stream and based on the relation graph of the equipment-event-signal, adopting an event finding and tracking algorithm to carry out three processes of event finding, event dynamic tracking and event overtime detection on the fused real-time data stream, and finding an event signal of the real-time data stream; the method improves the operation efficiency, safety and reliability of the hydropower station.
Description
Technical Field
The invention relates to the technical field of hydropower stations, in particular to an event discovery and tracking method and system based on real-time signals of hydropower stations.
Background
Centralized and unified monitoring of signals of the large-ferry subordinate 8 stations faces challenges of event signal description complexity, extremely reliance on manual judgment, empirical analysis and the like, and multiple events can overlap at the same time, so that a large number of signals are gushed in a short time. However, there are still a number of problems in relying on only experienced technicians to perform signal analysis and take corresponding measures, firstly, tasks have various device types, the amount of signals is large, complex data processing is required, alarms are frequently generated, monitoring work is difficult, and there is a high risk that even experienced technicians are required to consume a great deal of time and manpower resources to analyze the signals. Second, operational safety is challenging and signal monitoring requires manual 24-hour uninterrupted monitoring. This high intensity and high pressure work requires a high level of concentration on the part of the technician, keeping a clear head, in order to discover anomalies in time and take corrective countermeasures. In addition, signal data is complex, emergency and non-emergency events are interleaved, and even experienced technicians require significant time to sort and analyze the signals in view of the "zero fault tolerance" security requirements of signal monitoring. Finally, the experience of technicians is largely from the accumulation of processed events, so culturing experienced technicians typically requires a considerable amount of time. Thus, the use of the above method would result in a low efficiency hydropower station operation and a too high technician dependence would aggravate the technician's effort.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides an event discovery and tracking method and system based on a hydropower station real-time signal, a device-event-signal relation map is constructed through historical signal data and historical operation instruction data of the hydropower station, and the system is utilized to analyze collected data streams so as to realize automatic analysis and monitoring of the signals, thereby reducing the working intensity of technicians and improving the operation efficiency of the hydropower station.
In order to achieve the aim of the invention, the invention adopts the following technical scheme:
an event discovery and tracking method based on hydropower station real-time signals comprises the following steps:
s1, based on historical signal data and historical operation instruction data of a hydropower station, a typical association relation event library is established, unique identification signals or signal combinations of the events are obtained, and a mapping relation between the events and the signals is obtained;
s2, establishing a relationship map of equipment-event-signal based on the relationship between the equipment and the event according to the mapping relationship between the event and the signal obtained in the step S1;
s3, acquiring multi-source real-time data, and carrying out data fusion on the acquired multi-source real-time data to obtain a fused real-time data stream;
s4, according to the fused real-time data stream obtained in the step S3, based on the relation graph of the equipment-event-signal established in the step S2, adopting an event finding and tracking algorithm to carry out three processes of event finding, event dynamic tracking and event overtime detection on the fused real-time data stream, and finding an event signal of the real-time data stream.
Further, the step S1 specifically includes:
s11, acquiring events of each scene involved in the daily scheduling process of the historical hydropower station, analyzing the association relation between the events and the characteristic signals, and based on the logic description of the association relation between the characteristic signals and the events, eventing the characteristic signals and establishing a typical association relation event library;
s12, acquiring an operation instruction and an instruction event corresponding to the operation instruction, which are involved in the daily scheduling process of the historical hydropower station, acquiring signal data corresponding to the instruction event by adopting a frequent pattern mining method based on signal data of a historical monitoring system, and inputting the signal data into a typical association relation event library established in the step S11 to obtain a typical association relation event library;
s13, collecting each event in the typical association relation event library obtained in the step S12, and obtaining a signal of each event;
s14, coding the signal of each event obtained in the step S13 by adopting a continuous digital coding method to obtain an event coded signal;
s15, according to the event coded signals obtained in the step S14, carrying out full-permutation and combination on the coded signals of each event to obtain various combinations of each event, and traversing the combinations to obtain the number of combinations after the event is traversed;
s16, judging whether the number of combinations obtained in the step S15 after the event traversal is the same as the number of combinations of other events, if so, executing the step S17, otherwise, stopping searching for the combinations exceeding the current length to obtain the signal combination length;
s17, continuing to increase the number of combinations of the events until the number of combinations of the events is equal to the number of all signals contained in the events, so as to obtain the signal combination length;
s18, obtaining a unique identification signal or signal combination of the event according to the signal combination length obtained in the steps S16 and S17.
Further, in step S12, a frequent pattern mining method is adopted to obtain signal data corresponding to the instruction event, and the process of inputting the signal data into the typical association relation event library established in step S11 is as follows:
acquiring signal data of a history monitoring system of an operation instruction in different time periods, sequencing the signal data of the history monitoring system according to time sequence to obtain signal data of the same operation instruction in different time periods, finding out signal data with higher occurrence frequency in different time periods by adopting a frequent pattern mining method, collecting the signal data to form signal subsets, judging whether the signal subsets form complete events, and if so, adding the signal data in the signal subsets into a typical association relation event library to be used as confirmed events.
Further, the frequent pattern mining method adopted in step S12 is implemented based on Apriori algorithm, and the algorithm steps are as follows:
firstly, calculating the support degree of each item, namely a single signal, in a data set, reserving the item with the support degree higher than a preset threshold value, namely the minimum support degree, generating a candidate 1-item set, and constructing a frequent 1-item set;
then, generating a candidate 2-item set by using the constructed frequent 1-item set, and screening by using the support degree of a single signal in the data set again to obtain the frequent 2-item set;
the generation of a larger candidate set continues and screening continues until a new candidate set cannot be generated.
Further, step S2 specifically includes:
s21, taking each device as a node, taking each event as a node, and identifying the nodes by using different identifiers;
s22, defining the relationship between the main equipment nodes and the relationship between the sub equipment nodes by using undirected edges, and defining the relationship between the sub equipment nodes and the main equipment nodes and the relationship between the event nodes and the equipment nodes by using directed edges;
s23, adding all the equipment nodes and the event nodes into the graph according to the undirected edges and the directed edges defined in the step S22, adding the directed edges between the event nodes and the equipment nodes, the directed edges between the sub-equipment nodes and the main equipment nodes, the undirected edges between the main equipment nodes and the undirected edges between the sub-equipment nodes, and establishing a relation map of equipment-event-signals.
Further, the step S3 specifically includes:
s31, acquiring signal data flow and operation instruction data generated in the daily scheduling process of the hydropower station to obtain multi-source real-time data;
s32, carrying out data fusion on the multi-source real-time data obtained in the step S31, associating signal data streams and operation instruction data in the multi-source real-time data, and unifying data formats of the signal data streams and the operation instruction data to obtain fused real-time data streams.
Further, the event searching process in step S4 is as follows:
analyzing the fused real-time data stream obtained in the step S3, and if operation instruction data exists in the real-time data stream, identifying a confirmed event corresponding to the real-time data stream through the operation instruction data;
if the real-time data stream is in a single form, matching the real-time data stream with the event in the equipment-event-signal relation map established in the step S2, and determining a unique identification signal or signal combination of the matched confirmed event;
if the real-time data stream corresponds to a plurality of events, enabling a storage space for each event in the cache, backing up the real-time data stream to each cache, and continuously accumulating the real-time data stream at the same time;
judging whether a unique identification signal or signal combination exists in the current accumulated real-time data stream according to the unique identification signal or signal combination of the matched confirmed event, if so, identifying the confirmed event corresponding to the current accumulated real-time data stream, if not, marking the current accumulated real-time data stream as an event to be confirmed, and continuously accumulating the real-time data stream;
and when the accumulated real-time data streams reach the set quantity threshold, performing similarity calculation on the event to be confirmed and the matched multiple confirmed events to obtain an event with highest similarity and exceeding the threshold, and marking the event as the event of the real-time data streams.
Further, the process of event dynamic tracking in step S4 is as follows:
when the real-time data stream corresponds to a plurality of events, creating a plurality of event caches, monitoring and maintaining the event caches, backing up the accumulation of the real-time data stream to the matched event caches, if the identified event is the unique event, reserving the identified event caches, and deleting other matched event caches.
Further, the event timeout detection in step S4 includes fixed time-limit timeout detection and dynamic time-limit timeout detection, and the specific process is as follows:
the fixed time limit timeout detection sets a fixed time for the determined event according to the determined event identified in the real-time data stream, and when the accumulated time of the real-time data stream exceeds the set fixed time, the current real-time data stream is submitted;
the dynamic time limit timeout detection determines timeout time according to the actual condition of the real-time data stream, and after the real-time data stream is identified as a determined event and an end signal of the event is identified, the current real-time data stream is not submitted according to fixed time, but accumulation of the real-time data stream is ended in advance.
An event discovery and tracking system based on real-time signals of a hydropower station, comprising:
the event library establishing module is used for establishing a typical association relation event library based on historical signal data and historical operation instruction data of the hydropower station, and obtaining a mapping relation between the event and the signal by acquiring a unique identification signal or a signal combination of the event;
the device event relation map module is used for establishing a relation map of the device-event-signal based on the relation between the device and the event according to the mapping relation between the event and the signal in the typical association relation event library established in the event library establishing module;
the data real-time docking and preprocessing module is used for acquiring multi-source real-time data, and carrying out fusion processing on the multi-source real-time data according to a time sequence to obtain a fused real-time data stream;
and the event discovery and tracking module is used for realizing signal event discovery, multi-buffer construction, maintenance and event tracking of the device-event-signal relationship map established based on the device event relationship map module according to the fused real-time data flow obtained by the data real-time docking and preprocessing module.
The invention has the following beneficial effects:
according to the event discovery and tracking method and system based on the hydropower station real-time signal, the event can be discovered and tracked in time by processing the hydropower station real-time data stream, and the running efficiency, safety and reliability of the hydropower station are improved.
Drawings
FIG. 1 is a schematic flow chart of an event discovery and tracking method based on hydropower station real-time signals;
fig. 2 is a schematic structural diagram of an event discovery and tracking system based on real-time signals of a hydropower station.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and all the inventions which make use of the inventive concept are protected by the spirit and scope of the present invention as defined and defined in the appended claims to those skilled in the art.
As shown in fig. 1, an event discovery and tracking method based on a real-time signal of a hydropower station includes the following steps S1-S4:
s1, based on historical signal data and historical operation instruction data of a hydropower station, a typical association relation event library is established, unique identification signals or signal combinations of the events are obtained, and a mapping relation between the events and the signals is obtained.
In this embodiment, various scenes involved in the daily scheduling process of the historical hydropower station, such as start-stop, switching operation, accidents, etc., are studied first, an event of each scene is obtained, and an association relationship between the event and a characteristic signal is established, specifically, a mapping relationship between the event and the signal is established, and the characteristic signal associated with each event is defined, where the characteristic signal includes a reason signal, a trigger signal, an end signal, a key signal, etc. of the event. And secondly, acquiring operation instructions related in the daily scheduling process of the historical hydropower station and events corresponding to the operation instructions. The method specifically comprises the steps of using signal data of a history monitoring system, applying a frequent pattern mining technology to obtain signal data corresponding to an operation instruction, and supplementing the signal data into a typical association relation event library to further enrich event information so as to provide more accurate event discovery and tracking capability.
Specifically, step S1 specifically includes S11-S18:
s11, acquiring events of all scenes involved in the daily scheduling process of the historical hydropower station, analyzing the association relation between the events and the characteristic signals, and based on the logic description of the association relation between the characteristic signals and the events, eventing the characteristic signals and establishing a typical association relation event library.
S12, acquiring an operation instruction and an instruction event corresponding to the operation instruction, which are related in the daily scheduling process of the historical hydropower station, acquiring signal data corresponding to the instruction event by adopting a frequent pattern mining method based on signal data of a historical monitoring system, and inputting the signal data into a typical association relation event library established in the step S11 to obtain a typical association relation event library.
In this embodiment, signal data acquired by the history Kafka platform, that is, signal data of the history monitoring system, are ordered to ensure that the signal data are ordered according to time sequence, time periods where the operation instructions are issued are determined according to time information given by the operation instructions, signal data of the same operation instructions in different time periods are obtained, the ordered signal data are analyzed by using a frequent pattern mining method, signal data frequently appearing in different time periods are found, the signal data are collected to form signal subsets, the signal subsets are verified and confirmed by technicians, and the technicians evaluate the correlation of the signal subsets and the integrity of events according to knowledge in the field. And judging whether the signal subset forms a complete event, if so, adding the signal data in the signal subset into a typical association relation event library, and taking the signal data as a confirmed event. In addition, the frequent pattern mining method adopted in the embodiment is realized based on an Apriori algorithm, and can find the association relation between data in a large-scale data set, so that the accuracy of event matching is enhanced.
Specifically, in step S12, a frequent pattern mining method is adopted to obtain signal data corresponding to the instruction event, and the process of inputting the signal data into the typical association relationship event library established in step S11 is as follows:
acquiring signal data of a history monitoring system of an operation instruction in different time periods, sequencing the signal data of the history monitoring system according to time sequence to obtain signal data of the same operation instruction in different time periods, finding out signal data with higher occurrence frequency in different time periods by adopting a frequent pattern mining method, collecting the signal data to form signal subsets, judging whether the signal subsets form complete events, and if so, adding the signal data in the signal subsets into a typical association relation event library to be used as confirmed events.
Specifically, the frequent pattern mining method adopted in step S12 is implemented based on Apriori algorithm, and the algorithm steps are as follows:
firstly, calculating the support degree of each item, namely a single signal, in a data set, reserving the item with the support degree higher than a preset threshold, namely the minimum support degree, generating a candidate 1-item set, and constructing a frequent 1-item set.
And then, generating a candidate 2-item set by using the constructed frequent 1-item set, and screening by using the support degree of the single signal in the data set again to obtain the frequent 2-item set.
The generation of a larger candidate set continues and screening continues until a new candidate set cannot be generated.
S13, collecting each event in the typical association relation event library obtained in the step S12, and obtaining a signal of each event.
S14, coding the signal of each event acquired in the step S13 by adopting a continuous digital coding method to acquire an event coded signal.
S15, according to the event coded signals obtained in the step S14, carrying out full permutation and combination on the coded signals of each event to obtain various combinations of each event, and traversing the combinations to obtain the number of combinations after the event is traversed.
S16, judging whether the number of combinations obtained in the step S15 after the event traversal is the same as the number of combinations of other events, if so, executing the step S17, otherwise, stopping searching for the combinations exceeding the current length to obtain the signal combination length.
And S17, continuously increasing the number of combinations of the events until the number of combinations of the events is equal to the number of all signals contained in the events, and obtaining the signal combination length.
S18, obtaining a unique identification signal or signal combination of the event according to the signal combination length obtained in the steps S16 and S17.
Steps S13-S18 are processes for acquiring a unique identification signal or combination of signals for each event. The specific process is as follows: firstly, collecting signals of each event in a typical association relation event library in the embodiment, and endowing each signal with a unique digital code so as to ensure that the signals have unique identifiers and avoid chaotic conflict; secondly, carrying out standardization and normalization processing on the collected signals, mapping the signals to a unified form, and carrying out continuous processing on the signals so as to represent the signals into a compact continuous coding form; finally, for each event, carrying out full permutation and combination on signals of the event, carrying out various possible combinations on all signals of each event to explore the combination situation of different signals, checking whether other events have the same signal combination, if the current signal combination number can uniquely represent the event, stopping searching for the combination exceeding the current signal combination length, namely marking the combination as the signal combination belonging to the event, if the current signal combination number cannot uniquely represent the event, continuing to increase the signal combination number length until the signal combination size is equal to the number of all signals contained in the event, and if the signal combination length of the event is 1, then the signal combination is the unique identification signal of the current event, and at the moment, all other combinations with the signal combination length exceeding 1 do not need to be traversed; if the signal combination length of the event exceeds 1 and the combination does not exist for other events, the signal combination represents the signal combination of the current event.
S2, according to the mapping relation between the event and the signal obtained in the step S1, and based on the relation between the equipment and the event, a relation map of the equipment-event-signal is established.
In this embodiment, the mapping relationship between the event and the signal has been obtained in the above step, so that the relationship map of the device-event-signal can be obtained as long as the association relationship between the event and the device is clarified. The specific process is as follows: first, the node definition treats each device and event as a node and is identified using different identifiers, such as: "Device1", "Device2", "Event1", "Event2". Second, the definition of an edge, by which a different relationship is represented. The undirected edge points from one master device to another master device or from one sub-device to another sub-device for representing the relationship of the devices on the same level; the directed edge slave sub-device points to the master device node and is used for representing the hierarchical relationship between the master device and the sub-device and indicating that the sub-device belongs to the master device; in addition, there are directed edges of the event node to the device node for indicating an association between the event and the device, indicating that the device is associated with the event. Finally, the component graph, that is, at the stage of the component graph, adds all devices and event nodes into the graph, adds directed edges between the event and the device to represent the relationship between them, and simultaneously adds the directed edges between the sub-device and the main device to build the hierarchical structure. To represent a unified hierarchy of devices, undirected edges between the master devices and undirected edges between the child devices are also added. Furthermore, the signals in the constructed device-event-signal relationship graph represent attributes of the event nodes.
Specifically, step S2 specifically includes S21-S23:
s21, each device is used as a node, each event is used as a node, and different identifiers are used for identifying the nodes.
S22, defining the relation between the main equipment nodes and the sub equipment nodes by using undirected edges, and defining the relation between the sub equipment nodes and the main equipment nodes and the relation between the event nodes and the equipment nodes by using directed edges.
S23, adding all the equipment nodes and the event nodes into the graph according to the undirected edges and the directed edges defined in the step S22, adding the directed edges between the event nodes and the equipment nodes, the directed edges between the sub-equipment nodes and the main equipment nodes, the undirected edges between the main equipment nodes and the undirected edges between the sub-equipment nodes, and establishing a relation map of equipment-event-signals.
S3, acquiring multi-source real-time data, and carrying out data fusion on the acquired multi-source real-time data to obtain a fused real-time data stream.
In the embodiment, the multi-source real-time data is obtained by acquiring signal data flow and operation instruction data generated in the daily scheduling process of the hydropower station. Because the signal data flow generated during daily scheduling is large and involves a variety of sensors and device types, the resulting signal data flow includes signals generated by different devices of different systems. The term "system" as used in this embodiment is intended to refer to a different component, element, part, section or assembly at a different level. And meanwhile, fusing and processing the obtained multi-source real-time data, and obtaining the real-time data stream with a unified format by correlating and unifying the operation instruction data and the signal data stream in the multi-source real-time data. Analyzing and cleaning a signal data stream from a Kafka platform, extracting useful information, analyzing an operation instruction subordinate to the signal data stream, acquiring time, an object, instruction content and the like of the operation instruction data, associating the signal data stream with the instruction data stream, unifying formats of the signal data stream and the operation instruction data of the Kafka platform, and completing unified formatting and association operation on multi-source real-time data of different sources and types to obtain a fused real-time data stream.
Specifically, step S3 specifically includes S31-S32:
s31, acquiring signal data flow and operation instruction data generated in the daily scheduling process of the hydropower station, and obtaining multi-source real-time data.
S32, carrying out data fusion on the multi-source real-time data obtained in the step S31, associating signal data streams and operation instruction data in the multi-source real-time data, and unifying data formats of the signal data streams and the operation instruction data to obtain fused real-time data streams.
S4, according to the fused real-time data stream obtained in the step S3, based on the relation graph of the equipment-event-signal established in the step S2, adopting an event finding and tracking algorithm to carry out three processes of event finding, event dynamic tracking and event overtime detection on the fused real-time data stream, and finding an event signal of the real-time data stream.
In this embodiment, the event discovery and tracking algorithm includes: the acquired real-time data streams are monitored, collected and analyzed in real time, and are converted into events and separated in the form of events, wherein the process comprises three stages of tracking the beginning, the duration and the end of the events. Therefore, the event discovery and tracking algorithm is adopted to event the fused real-time data stream, and the three processes comprise event searching, event dynamic tracking and event overtime monitoring. In addition, to further refine the event discovery algorithm, the algorithm in this embodiment is calculated based on the similarity of Jaccard similarity coefficients, which can accurately identify and classify signal streams to determine the degree of coincidence with a determined event. The Jaccard similarity coefficient is a calculation method for comparing the similarity between two sets, and is mainly used for measuring the proportion of common elements in total elements of the two sets, so as to judge the similarity degree between the sets. The value range of the Jaccard similarity coefficient is between 0 and 1, wherein 0 indicates that the two sets have no common elements, namely are completely dissimilar; 1 indicates that the elements of the two sets are identical, i.e. completely similar. Thus, in practical application, the closer the similarity coefficient is to 1, the more similar the two sets are represented. Therefore, by adopting the method, the event to which the current real-time data stream belongs can be determined. Most importantly, the result of the event discovery and tracking algorithm can also be used as the early data of scenes such as fault diagnosis, anomaly detection and the like of the hydropower station, and support is provided for the scenes.
The specific process of event searching is as follows: when the real-time data stream carries operation instruction data, directly identifying an event corresponding to the current real-time data stream through the operation instructions; when only a single real-time data stream arrives, the real-time data stream is matched with the event in the established device-event-signal relationship map, and the unique identification signal or the combined signal of the matched determined event is determined. If the real-time data signal stream corresponds to a plurality of events, a memory space is enabled in the buffer memory for each event, and is backed up into each buffer memory, and the real-time data stream is continuously accumulated. Determining whether the identification signal exists in the current real-time data stream according to the matched unique identification signal or the combined signal of the determined event, and if so, determining the event corresponding to the current real-time data stream; if the matched determined event has no unique identification signal or combined signal, marking the current real-time data stream as an event to be confirmed, continuing to accumulate until the accumulated real-time data stream reaches a set threshold value, and performing similarity calculation on the event to be confirmed and the matched multiple known events to determine the event with the highest similarity and exceeding the threshold value, and marking the event as the event to which the current real-time data stream belongs. In this embodiment, similarity calculation is performed on an event to be confirmed and a plurality of matched known events to determine an event with the highest similarity and exceeding a threshold value, a similarity matching algorithm is adopted, and the calculation process is as follows: and extracting signals from the matched events as feature sets, carrying out similarity calculation on the matched signal sets in the buffer area and each feature set, and finally comparing the calculated similarity with a preset threshold value.
Specifically, the event searching process in step S4 is as follows:
and (3) analyzing the fused real-time data stream obtained in the step (S3), and if operation instruction data exist in the real-time data stream, identifying a confirmed event corresponding to the real-time data stream through the operation instruction data.
If the real-time data stream is in a single form, matching the real-time data stream with the event in the device-event-signal relationship map established in step S2, and determining a unique identification signal or signal combination of the matched identified event.
If the real-time data stream corresponds to a plurality of events, a storage space is started for each event in the cache, the real-time data stream is backed up to each cache, and meanwhile, the real-time data stream is continuously accumulated.
Judging whether a unique identification signal or signal combination exists in the current accumulated real-time data stream according to the unique identification signal or signal combination of the matched confirmed event, if so, identifying the confirmed event corresponding to the current accumulated real-time data stream, and if not, marking the current accumulated real-time data stream as an event to be confirmed and continuously accumulating the real-time data stream.
And when the accumulated real-time data streams reach the set quantity threshold, performing similarity calculation on the event to be confirmed and the matched multiple confirmed events to obtain an event with highest similarity and exceeding the threshold, and marking the event as the event of the real-time data streams.
In this embodiment, the monitoring and maintenance are performed on the caches of the events in a dynamic manner, and the cache state is monitored and the cache content is updated periodically, so as to track the development process of the events while retaining signal data related to the events, and only necessary information is retained and the use of the caches is optimized through unique event confirmation and cache management policies. The entire dynamic tracking process covers the entire life cycle of signal events from the beginning of recognition, gradually accumulating to the end of the final. The purpose of which is to monitor the signal flow.
Specifically, the process of event dynamic tracking in step S4 is:
when the real-time data stream corresponds to a plurality of events, creating a plurality of event caches, monitoring and maintaining the event caches, backing up the accumulation of the real-time data stream to the matched event caches, if the identified event is the unique event, reserving the identified event caches, and deleting other matched event caches.
In this embodiment, in the event searching process, when the real-time data stream is identified as the determined event, but the real-time data stream is delayed and has not arrived at the end of the signal, because the real-time limitation cannot accumulate the signal without limit, a timeout detection is introduced, and once the determined event is confirmed, by means of the time limit information preset by the event itself, when the event is ended and the event signal is submitted. Timeout detection includes two modes, namely a fixed time limit and a dynamic time limit.
Specifically, the event timeout detection in step S4 includes fixed time-limit timeout detection and dynamic time-limit timeout detection, and the specific procedure is as follows:
the fixed time limit timeout detection sets a fixed time for the determined event based on the determined event identified in the real-time data stream, and when the accumulated time of the real-time data stream exceeds the set fixed time, the current real-time data stream is submitted.
The dynamic time limit timeout detection determines timeout time according to the actual condition of the real-time data stream, and after the real-time data stream is identified as a determined event and an end signal of the event is identified, the current real-time data stream is not submitted according to fixed time, and accumulation of the real-time data stream is ended in advance.
As shown in fig. 2, an event discovery and tracking system based on real-time signals of a hydropower station includes:
the event library establishing module establishes a typical association relation event library based on historical signal data and historical operation instruction data of the hydropower station, and obtains a mapping relation between the event and the signal by acquiring a unique identification signal or a signal combination of the event.
In this embodiment, the event library creating module is responsible for collecting and storing event signal data of the historical hydropower station and is used for creating event signals of the hydropower station, and the module includes an event library probability table, wherein all events and corresponding time limit information are covered. Meanwhile, the module also comprises an event signal list, wherein the related relations and attribute information such as key signals, reason signals, ending signals, time sequence of the signals and the like of the event are recorded.
And the equipment event relation graph module is used for establishing a relation graph of equipment-event-signal based on the relation between equipment and the event according to the mapping relation between the event and the signal in the typical incidence relation event library established in the event library establishing module.
In this embodiment, the device event relationship map module creates a hydropower station device event relationship map according to the correspondence between the event and the device, and determines the event associated with each device and the signal associated with the event by analyzing the map.
And the data real-time docking and preprocessing module is used for acquiring multi-source real-time data, and carrying out fusion processing on the multi-source real-time data according to a time sequence to obtain a fused real-time data stream.
In this embodiment, the data real-time docking and preprocessing module is responsible for acquiring signal data streams and operation instruction data generated by related devices from a real-time monitoring system of the hydropower station, and performing fusion processing of multiple source signal streams according to a time sequence.
And the event discovery and tracking module is used for realizing signal event discovery, multi-buffer construction, maintenance and event tracking of the device-event-signal relationship map established based on the device event relationship map module according to the fused real-time data flow obtained by the data real-time docking and preprocessing module.
In this embodiment, the event discovery and tracking module is based on the fused real-time data stream, so as to realize signal event discovery, multi-buffer construction, maintenance and event tracking of the device-event-signal relationship map established by the device event relationship map module, thereby realizing intelligent monitoring of a large number of real-time data streams, quickly discovering and tracking event states, assisting technicians of the hydropower station in operating the working states of the hydropower station, improving working efficiency, preventing accidents and timely making emergency response.
The principles and embodiments of the present invention have been described in detail with reference to specific examples, which are provided to facilitate understanding of the method and core ideas of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Those of ordinary skill in the art will recognize that the embodiments described herein are for the purpose of aiding the reader in understanding the principles of the present invention and should be understood that the scope of the invention is not limited to such specific statements and embodiments. Those of ordinary skill in the art can make various other specific modifications and combinations from the teachings of the present disclosure without departing from the spirit thereof, and such modifications and combinations remain within the scope of the present disclosure.
Claims (10)
1. The event discovery and tracking method based on the hydropower station real-time signal is characterized by comprising the following steps of:
s1, based on historical signal data and historical operation instruction data of a hydropower station, a typical association relation event library is established, unique identification signals or signal combinations of the events are obtained, and a mapping relation between the events and the signals is obtained;
s2, establishing a relationship map of equipment-event-signal based on the relationship between the equipment and the event according to the mapping relationship between the event and the signal obtained in the step S1;
s3, acquiring multi-source real-time data, and carrying out data fusion on the acquired multi-source real-time data to obtain a fused real-time data stream;
s4, according to the fused real-time data stream obtained in the step S3, based on the relation graph of the equipment-event-signal established in the step S2, adopting an event finding and tracking algorithm to carry out three processes of event finding, event dynamic tracking and event overtime detection on the fused real-time data stream, and finding an event signal of the real-time data stream.
2. The method for event discovery and tracking based on real-time signals of a hydropower station according to claim 1, wherein the step S1 specifically comprises:
s11, acquiring events of each scene involved in the daily scheduling process of the historical hydropower station, analyzing the association relation between the events and the characteristic signals, and based on the logic description of the association relation between the characteristic signals and the events, eventing the characteristic signals and establishing a typical association relation event library;
s12, acquiring an operation instruction and an instruction event corresponding to the operation instruction, which are involved in the daily scheduling process of the historical hydropower station, acquiring signal data corresponding to the instruction event by adopting a frequent pattern mining method based on signal data of a historical monitoring system, and inputting the signal data into a typical association relation event library established in the step S11 to obtain a typical association relation event library;
s13, collecting each event in the typical association relation event library obtained in the step S12, and obtaining a signal of each event;
s14, coding the signal of each event obtained in the step S13 by adopting a continuous digital coding method to obtain an event coded signal;
s15, according to the event coded signals obtained in the step S14, carrying out full-permutation and combination on the coded signals of each event to obtain various combinations of each event, and traversing the combinations to obtain the number of combinations after the event is traversed;
s16, judging whether the number of combinations obtained in the step S15 after the event traversal is the same as the number of combinations of other events, if so, executing the step S17, otherwise, stopping searching for the combinations exceeding the current length to obtain the signal combination length;
s17, continuing to increase the number of combinations of the events until the number of combinations of the events is equal to the number of all signals contained in the events, so as to obtain the signal combination length;
s18, obtaining a unique identification signal or signal combination of the event according to the signal combination length obtained in the steps S16 and S17.
3. The method for discovering and tracking events based on real-time signals of hydropower stations according to claim 2, wherein the process of acquiring signal data corresponding to command events by adopting a frequent pattern mining method in step S12 and inputting the signal data into a typical association relation event library established in step S11 is as follows:
acquiring signal data of a history monitoring system of an operation instruction in different time periods, sequencing the signal data of the history monitoring system according to time sequence to obtain signal data of the same operation instruction in different time periods, finding out signal data with higher occurrence frequency in different time periods by adopting a frequent pattern mining method, collecting the signal data to form signal subsets, judging whether the signal subsets form complete events, and if so, adding the signal data in the signal subsets into a typical association relation event library to be used as confirmed events.
4. The method for event discovery and tracking based on hydropower station real-time signals according to claim 2, wherein the frequent pattern mining method adopted in step S12 is implemented based on Apriori algorithm, and the algorithm steps are as follows:
firstly, calculating the support degree of each item, namely a single signal, in a data set, reserving the item with the support degree higher than a preset threshold value, namely the minimum support degree, generating a candidate 1-item set, and constructing a frequent 1-item set;
then, generating a candidate 2-item set by using the constructed frequent 1-item set, and screening by using the support degree of a single signal in the data set again to obtain the frequent 2-item set;
the generation of a larger candidate set continues and screening continues until a new candidate set cannot be generated.
5. The method for event discovery and tracking based on real-time signals of a hydropower station according to claim 1, wherein step S2 specifically comprises:
s21, taking each device as a node, taking each event as a node, and identifying the nodes by using different identifiers;
s22, defining the relationship between the main equipment nodes and the relationship between the sub equipment nodes by using undirected edges, and defining the relationship between the sub equipment nodes and the main equipment nodes and the relationship between the event nodes and the equipment nodes by using directed edges;
s23, adding all the equipment nodes and the event nodes into the graph according to the undirected edges and the directed edges defined in the step S22, adding the directed edges between the event nodes and the equipment nodes, the directed edges between the sub-equipment nodes and the main equipment nodes, the undirected edges between the main equipment nodes and the undirected edges between the sub-equipment nodes, and establishing a relation map of equipment-event-signals.
6. The method for event discovery and tracking based on real-time signals of a hydropower station according to claim 1, wherein the step S3 specifically comprises:
s31, acquiring signal data flow and operation instruction data generated in the daily scheduling process of the hydropower station to obtain multi-source real-time data;
s32, carrying out data fusion on the multi-source real-time data obtained in the step S31, associating signal data streams and operation instruction data in the multi-source real-time data, and unifying data formats of the signal data streams and the operation instruction data to obtain fused real-time data streams.
7. The method for event discovery and tracking based on real-time signals of hydropower stations according to claim 1, wherein the event searching in step S4 comprises the following steps:
analyzing the fused real-time data stream obtained in the step S3, and if operation instruction data exists in the real-time data stream, identifying a confirmed event corresponding to the real-time data stream through the operation instruction data;
if the real-time data stream is in a single form, matching the real-time data stream with the event in the equipment-event-signal relation map established in the step S2, and determining a unique identification signal or signal combination of the matched confirmed event;
if the real-time data stream corresponds to a plurality of events, enabling a storage space for each event in the cache, backing up the real-time data stream to each cache, and continuously accumulating the real-time data stream at the same time;
judging whether a unique identification signal or signal combination exists in the current accumulated real-time data stream according to the unique identification signal or signal combination of the matched confirmed event, if so, identifying the confirmed event corresponding to the current accumulated real-time data stream, if not, marking the current accumulated real-time data stream as an event to be confirmed, and continuously accumulating the real-time data stream;
and when the accumulated real-time data streams reach the set quantity threshold, performing similarity calculation on the event to be confirmed and the matched multiple confirmed events to obtain an event with highest similarity and exceeding the threshold, and marking the event as the event of the real-time data streams.
8. The method for event discovery and tracking based on real-time signals of hydropower stations according to claim 1, wherein the process of event dynamic tracking in step S4 is as follows:
when the real-time data stream corresponds to a plurality of events, creating a plurality of event caches, monitoring and maintaining the event caches, backing up the accumulation of the real-time data stream to the matched event caches, if the identified event is the unique event, reserving the identified event caches, and deleting other matched event caches.
9. The method for event discovery and tracking based on real-time signals of a hydropower station according to claim 1, wherein the event timeout detection in step S4 comprises a fixed time-limit timeout detection and a dynamic time-limit timeout detection, and the specific procedures are as follows:
the fixed time limit timeout detection sets a fixed time for the determined event according to the determined event identified in the real-time data stream, and when the accumulated time of the real-time data stream exceeds the set fixed time, the current real-time data stream is submitted;
the dynamic time limit timeout detection determines timeout time according to the actual condition of the real-time data stream, and after the real-time data stream is identified as a determined event and an end signal of the event is identified, the current real-time data stream is not submitted according to fixed time, but accumulation of the real-time data stream is ended in advance.
10. An event discovery and tracking system based on real-time signals of a hydropower station, comprising:
the event library establishing module is used for establishing a typical association relation event library based on historical signal data and historical operation instruction data of the hydropower station, and obtaining a mapping relation between the event and the signal by acquiring a unique identification signal or a signal combination of the event;
the device event relation map module is used for establishing a relation map of the device-event-signal based on the relation between the device and the event according to the mapping relation between the event and the signal in the typical association relation event library established in the event library establishing module;
the data real-time docking and preprocessing module is used for acquiring multi-source real-time data, and carrying out fusion processing on the multi-source real-time data according to a time sequence to obtain a fused real-time data stream;
and the event discovery and tracking module is used for realizing signal event discovery, multi-buffer construction, maintenance and event tracking of the device-event-signal relationship map established based on the device event relationship map module according to the fused real-time data flow obtained by the data real-time docking and preprocessing module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311605270.0A CN117591560A (en) | 2023-11-24 | 2023-11-24 | Event discovery and tracking method and system based on hydropower station real-time signals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311605270.0A CN117591560A (en) | 2023-11-24 | 2023-11-24 | Event discovery and tracking method and system based on hydropower station real-time signals |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117591560A true CN117591560A (en) | 2024-02-23 |
Family
ID=89918034
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311605270.0A Pending CN117591560A (en) | 2023-11-24 | 2023-11-24 | Event discovery and tracking method and system based on hydropower station real-time signals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117591560A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102282552A (en) * | 2008-12-23 | 2011-12-14 | 黄家正 | System, method and computer program for pattern based intelligent control, monitoring and automation |
| CN110046260A (en) * | 2019-04-16 | 2019-07-23 | 广州大学 | A kind of darknet topic discovery method and system of knowledge based map |
| CN111768077A (en) * | 2020-05-28 | 2020-10-13 | 国网浙江省电力有限公司绍兴供电公司 | Intelligent power grid trip event identification method based on knowledge graph |
| US20210064624A1 (en) * | 2019-06-25 | 2021-03-04 | Google Llc | Using live data streams and/or search queries to determine information about developing events |
-
2023
- 2023-11-24 CN CN202311605270.0A patent/CN117591560A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102282552A (en) * | 2008-12-23 | 2011-12-14 | 黄家正 | System, method and computer program for pattern based intelligent control, monitoring and automation |
| CN110046260A (en) * | 2019-04-16 | 2019-07-23 | 广州大学 | A kind of darknet topic discovery method and system of knowledge based map |
| US20210064624A1 (en) * | 2019-06-25 | 2021-03-04 | Google Llc | Using live data streams and/or search queries to determine information about developing events |
| CN111768077A (en) * | 2020-05-28 | 2020-10-13 | 国网浙江省电力有限公司绍兴供电公司 | Intelligent power grid trip event identification method based on knowledge graph |
Non-Patent Citations (1)
| Title |
|---|
| 高志;樊锐轶;米超;王大海;胡庆博;冯超;: "设备监控信息大数据与设备模型的互校验及实用化事件分析", 电力大数据, no. 03, 21 March 2020 (2020-03-21) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107886238B (en) | Business process management system and method based on mass data analysis | |
| CN111145378B (en) | Automatic video inspection system and method for road traffic incident | |
| CN108398934B (en) | equipment fault monitoring system for rail transit | |
| CN111612018A (en) | Underground pipe gallery monitoring method and system based on multi-source heterogeneous data fusion | |
| CN115330268B (en) | Comprehensive emergency command method and system for coping with mine disasters | |
| CN115527340A (en) | Intelligent construction site safety monitoring system and method based on unmanned aerial vehicle and surveillance camera | |
| CN103049365B (en) | Information and application resource running state monitoring and evaluation method | |
| CN210666820U (en) | Pedestrian abnormal behavior detection system based on DSP edge calculation | |
| CN112862233A (en) | Fault relevance analysis system and method based on Internet of vehicles data | |
| CN117078227A (en) | Environment monitoring operation and maintenance platform based on identification analysis | |
| CN116986246A (en) | Intelligent inspection system and method for coal conveying belt | |
| CN111932060A (en) | Method and system for performing mobile inspection on cable line based on front-end and back-end fusion data | |
| CN115952919A (en) | Intelligent risk prediction method based on process mining | |
| CN111680027A (en) | Method and system for realizing intelligent cloud management based on knowledge drive | |
| CN114023076A (en) | Specific vehicle tracking method based on multi-source heterogeneous data | |
| CN117591560A (en) | Event discovery and tracking method and system based on hydropower station real-time signals | |
| CN112907911A (en) | Intelligent anomaly identification and alarm algorithm based on equipment process data | |
| CN112816933A (en) | Maintenance diagnosis method and system for electric energy meter terminal | |
| CN116845863A (en) | Terminal power grid abnormal equipment access alarm method and device based on space-time track | |
| CN105933677A (en) | Industrial field video data processing system | |
| CN113837672B (en) | Equipment maintenance management method based on industrial Internet | |
| CN112153341B (en) | Task supervision method, device and system, electronic equipment and storage medium | |
| CN114168409A (en) | Service system running state monitoring and early warning method and system | |
| CN112711508A (en) | Intelligent operation and maintenance service system facing large-scale client system | |
| CN111200315B (en) | Transformer substation monitoring background fault diagnosis system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |