CN104092578B - Data stream monitoring method based on pipe method - Google Patents
Data stream monitoring method based on pipe method Download PDFInfo
- Publication number
- CN104092578B CN104092578B CN201310741890.7A CN201310741890A CN104092578B CN 104092578 B CN104092578 B CN 104092578B CN 201310741890 A CN201310741890 A CN 201310741890A CN 104092578 B CN104092578 B CN 104092578B
- Authority
- CN
- China
- Prior art keywords
- data
- pipeline
- data flow
- monitoring
- flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 67
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000008859 change Effects 0.000 claims abstract description 37
- 230000006978 adaptation Effects 0.000 claims description 5
- 230000005540 biological transmission Effects 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims description 4
- 238000002347 injection Methods 0.000 claims description 3
- 239000007924 injection Substances 0.000 claims description 3
- 230000003068 static effect Effects 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 abstract description 5
- 238000004883 computer application Methods 0.000 abstract description 3
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 239000000243 solution Substances 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses the data stream monitoring method based on pipe method, the data for making types of applications system by application system interface are integrated, the delta data of the multiple application systems of real-time reception, construct data flow, build pipeline model, the linkage set up between data stream association, data flow observation item and data flow, and by supervisor engine, to a plurality of monitoring analysis of the data flow progress based on curved section in a pipeline.The present invention can set up incidence relation between data flow, continuation Monitoring Data associates situation of change in computer application system, and the data monitoring with curved surface, waveform, form etc. for expression way directly perceived is exported as a result, it is possible to realize the function that linkage data flow is monitored based on the period.
Description
Technical field
The present invention relates to information monitoring technology, the data stream monitoring method of pipe method is based particularly on.
Background technology
With information technology during enterprise production and management by further strengthened research, in various information system produce
The data such as increasing production, project, fund and goods and materials, continuation long-term to these data changes the data flow to be formed
Monitoring and analysis effectively enterprise can be helped to make business decision, so, gradually paid attention to by enterprise.Due to these data
Different systems are resulted from, the monitoring method to data is all distributed in respective system, is only capable of by several time points
Retrieval and inquisition, statistical means etc. can be realized.Also Some Enterprises employ the data monitoring mode based on data center, its is each
The monitored object that business department is directed to concentrates on the data flow in each field, for example, production division focuses on production, product data
The change of stream, Finance Department is absorbed in the change and trend analysis of funds data stream.
Current data stream monitoring method, can be monitored for specific data stream, be implied however, ignoring business datum
Information relevance, the retardance that has of the linkage of Change of Data Stream and Change of Data Stream, it is comprehensive set up data flow it
Between incidence relation, so that comprehensively monitoring and analysis can not be carried out to the linkage of a plurality of data flow.
The content of the invention
It is an object of the invention to provide a kind of data stream monitoring method based on pipe method, it can set up between data flow
Incidence relation, continuation Monitoring Data associates situation of change in computer application system, and exports with curved surface, waveform, form
Deng the data monitoring for expression way directly perceived as a result, it is possible to realize the function based on period monitoring linkage data flow.
The object of the present invention is achieved like this：A kind of data stream monitoring method based on pipe method, 1. builds data
Structure：By data flow constructor build a storage service data flow data structure, define data flow static attribute-
Name identification, code identification, description and monitored item set, inherit dynamic attribute-data access, monitored item and pass in the structure shown here
Increase, build data flow model, after data flow model is built, be saved in pipe database；2. pipeline model is built：Pass through pipe
Road constructor constructs a pipeline, defines pipeline identification, the data structure for 1. walking construction is added in pipeline, every pipeline
The a plurality of data flow for having a business association relation can be added；Pipeline constructor：For constructing a pipeline, a pipeline is exactly about
The data structure of several data flows of beam, pipeline includes data flow chained list, a data stream association relation table, and data stream association is closed
It is that table includes the relation pair of data flow relation pair and data stream monitoring；Phase between data flow relation pair statement two data streams
Mutually influence relation；3. input channel is set up：Application system calls the interface in the pipeline constructor of this method, when business datum becomes
Business datum, is delivered in the pipeline constructor of this method, sets up this method by the transmission of trigger data during change by interface
Data exchange channels between pipeline constructor and application system, the data of transmission include pipeline identification, stream identification, monitoring
Item mark, monitored item type, the data of monitored item；4. supervisor engine is started：The data structure and pipeline built by constructor,
Loading and instantiation, a pipeline is loaded from pipe database in monitoring flow, receives the industry that application system is transmitted through interface
Business data, start pipeline and simultaneously business datum are loaded into the monitored item of each data flow, the business datum of real-time reception is preserved
In monitoring data storehouse, and in real-time monitoring data flow monitored item change, record change linkage, time point；5. start
Link adaptation module：According to the actual change order of monitoring data, the time interval of change in monitoring data storehouse, repair automatically
The priority of the monitored item of correction data stream and association order；6. curved surface monitored results are exported：By relevance line in time shaft
The monitored item of data flow during upper interception change, so as to form a curved surface constituted based on the period, by monitoring item data；⑦
Close supervisor engine：After supervisor engine is loaded and instantiates pipeline, monitored in real time after injection business datum；Work as pipeline
After being manually stopped, supervisor engine stops pipeline example, and the pipeline in monitoring data storehouse is placed in into suspended state, no longer received
The real time data of the pipeline；When manual-lock pipeline or after receive data flow done state, supervisor engine stops pipeline example
Change, unload pipeline model, close the conduit acceptor passage, pipeline data is removed from monitoring data storehouse.
The reason for technical solution of the present invention is generated：Business datum forms business data flow with the change of time shaft, different
The relevance existed between business is lain in the change of business data flow, but because business is carried by different application systems, because
This, application system interface can form data integration passage, and pipe method can constrain the business data flow with implication relation, pass through
The task of monitored item execution real-time monitoring data flow can effectively be disclosed in different business in a period of time between associated data stream
Linkage between data flow.
Advantages of the present invention：First, pipeline effect of restraint can bind the business data flow being present in different application systems, lead to
The incidence relation between data flow between incidence relation, data stream monitoring is crossed to the real-time monitoring of data flow, just can effectively be disclosed
The linkage of business datum change；2nd, the means of monitoring curved surface can effectively eliminate the business datum in time point monitor mode
Change the not explicitly of association, and be illustrated in the relation that influences each other of data variation in a period of time.
The present invention integrates the data of types of applications system by application system interface, the multiple application systems of real-time reception
The delta data of system, construct data flow, build pipeline model, set up data stream association, data flow observation item and data flow it
Between linkage, and by supervisor engine, in a pipeline a plurality of data flow carry out the monitoring analysis based on curved section.
The present invention can set up incidence relation between data flow, and the association of continuation Monitoring Data becomes in computer application system
Change situation, and export with curved surface, waveform, form etc. for expression way directly perceived data monitoring as a result, it is possible to realize based on the time
The function of section monitoring linkage data flow.
Brief description of the drawings
Below in conjunction with accompanying drawing, the invention will be further described.
Fig. 1 is workflow diagram of the invention；
Fig. 2 is inventive pipeline model construction schematic diagram；
Fig. 3 is the monitoring principle schematic of the invention based on curved surface.
Embodiment
A kind of data stream monitoring method based on pipe method, as shown in figure 1,1. building data structure：Pass through data flow
Constructor builds the data structure of a storage service data flow, defines static attribute-name identification, the coding mark of data flow
Know, describe and monitored item set, succession dynamic attribute-data access, monitored item are incremental in the structure shown here, build data flow mould
Type, after data flow model is built, is saved in pipe database；2. pipeline model is built：One is constructed by pipeline constructor
Individual pipeline, defines pipeline identification, and the data structure for 1. walking construction is added in pipeline, and every pipeline, which can be added, a plurality of to be had
The data flow of business association relation；Pipeline constructor：For constructing a pipeline, a pipeline is exactly to constrain several data flows
Data structure, pipeline includes data flow chained list, a data stream association relation table, and data stream association relation table includes data flow
The relation pair of relation pair and data stream monitoring；The relation that influences each other between data flow relation pair statement two data streams；③
Set up input channel：Application system calls the interface in the pipeline constructor of this method, the trigger data when business datum changes
Transmission, business datum is delivered in the pipeline constructor of this method by interface, set up the pipeline constructor of this method with
Data exchange channels between application system, the data of transmission include pipeline identification, stream identification, monitored item mark, monitoring
Item type, the data of monitored item；4. supervisor engine is started：The data structure and pipeline built by constructor, in monitoring flow
Middle loading and instantiation, a pipeline is loaded from pipe database, receives the business datum that application system is transmitted through interface, is started
Business datum is simultaneously loaded into the monitored item of each data flow by pipeline, and the business datum of real-time reception is stored in into monitoring data storehouse
In, and in real-time monitoring data flow monitored item change, record change linkage, time point；5. linkage adaptive mode is started
Block：According to the actual change order of monitoring data, the time interval of change in monitoring data storehouse, carry out the automatic prison for correcting data flow
Control the priority and association order of item；6. curved surface monitored results are exported：When intercepting change on a timeline by relevance line
Data flow monitored item, so as to form a curved surface constituted based on the period, by monitoring item data；7. monitoring is closed to draw
Hold up：After supervisor engine is loaded and instantiates pipeline, monitored in real time after injection business datum；When pipeline is manually stopped
Afterwards, supervisor engine stops pipeline example, and the pipeline in monitoring data storehouse is placed in into suspended state, the reality of the pipeline is no longer received
When data；When manual-lock pipeline or after receive data flow done state, supervisor engine stops pipeline example, unloads pipeline
Model, closes the conduit acceptor passage, pipeline data is removed from monitoring data storehouse.
The present invention builds the business datum integrated channel of types of applications system by application system interface, passes through data flow structure
Make device and pipeline constructor sets up the data entity based on data stream monitoring, the change of business datum is loaded into number by supervisor engine
According to the linkage in stream and pipeline, monitoring each business data flow change in real time, linkage adaptation module is then according in data flow
The change frequency of monitored item, the influence order to monitored item in other data flows, dynamically adjust incidence relation in data flow
The priority of priority and monitored item, finally, by the change of the data stream nodes with interaction relation formed in a period of time
Situation is shown by output module.The process description of the present invention is as follows：1. data flow constructor：For constructing a storage
The data structure of business data flow, by stream identification, confirms a business data flow, including stream identification, data flow
Title, monitoring necklace watch, each business data flow include multiple monitored item.2. monitored item：Data in one business data flow
Item is used for change and the time point of real-time recorded data, is also the business datum of user's concern and monitoring, is connect by application system
Mouth receives data, starts in supervisor engine after pipeline, in the monitored item for reloading data flow.3. pipeline constructor：For
A pipeline is constructed, a pipeline is exactly the data structure for constraining several data flows, and pipeline includes data flow chained list, a number
According to stream incidence relation table, data stream association relation table includes the relation pair of data flow relation pair and data stream monitoring；Data flow
Relation pair describes the relation that influences each other between two data streams, and this relation that influences each other passes through monitored item relation pair quilt
Embody；A pipeline model is constructed, is exactly unique mark of the generation for each pipeline, adds in the duct
Constraint between the data flow and interpolation data stream that define（Define data flow relation）, pipeline model, which builds schematic diagram, such as schemes
Shown in 2.4. supervisor engine：For loading a pipeline, receiving the business datum transmitted through application system interface, start pipeline simultaneously
In the monitored item that business datum is loaded into each data flow, with the change of monitored item in real-time monitoring data flow, record change
Linkage and time point.5. linkage adaptation module：Because the incidence relation of data flow in pipeline is set in a model,
After supervisor engine starts pipeline, loading data, adaptation module can change according to the variable condition and linkage of actual monitored
Priority and association order in situation amendment incidence relation.6. curved surface monitors output module：For exporting institute in a pipeline
There is the state of data flow linkage change, a monitored item of the state by each data flow in different time points is associated to constituting；By
In the time delay of data variation, the linkage closure line that the monitored item relation pair connected by data variation is constituted, in pipeline
Time shaft on constitute a curved section, the curved section expresses the connection between each business data flow within a period of time
Dynamic property, its monitoring principle schematic based on curved surface is as shown in Figure 3.
Claims (1)
1. a kind of data stream monitoring method based on pipe method, it is characterised in that：1. data structure is built：Pass through data flow structure
Make the data structure that device builds a storage service data flow, define static attribute-name identification of data flow, code identification,
Description and monitored item set, succession dynamic attribute-data access, monitored item are incremental in the structure shown here, build data flow model,
After data flow model is built, it is saved in pipe database；2. pipeline model is built：One pipe is constructed by pipeline constructor
Road, defines pipeline identification, and the data structure for 1. walking construction is added in pipeline, and every pipeline, which can be added, a plurality of has business
The data flow of incidence relation；Pipeline constructor：For constructing a pipeline, a pipeline is exactly the number for constraining several data flows
According to structure, pipeline includes data flow chained list, a data stream association relation table, and data stream association relation table includes data flow relation
Pair and data stream monitoring relation pair；The relation that influences each other between data flow relation pair statement two data streams；3. set up
Input channel：Application system calls the interface in the pipeline constructor of this method, the biography of trigger data when business datum changes
Pass, business datum is delivered in the pipeline constructor of this method by interface, set up the pipeline constructor and application of this method
Data exchange channels between system, the data of transmission include pipeline identification, stream identification, monitored item mark, monitored item class
Type, the data of monitored item；4. supervisor engine is started：The data structure and pipeline built by constructor, adds in monitoring flow
Carry and instantiate, a pipeline is loaded from pipe database, receive the business datum that application system is transmitted through interface, start pipeline
And be loaded into business datum in the monitored item of each data flow, the business datum of real-time reception is stored in monitoring data storehouse,
And in real-time monitoring data flow monitored item change, record change linkage, time point；5. linkage adaptation module is started：
According to the actual change order of monitoring data, the time interval of change in monitoring data storehouse, carry out the automatic monitoring for correcting data flow
The priority of item and association order；6. curved surface monitored results are exported：When intercepting change on a timeline by relevance line
The monitored item of data flow, so as to form a curved surface constituted based on the period, by monitoring item data；7. supervisor engine is closed：
After supervisor engine is loaded and instantiates pipeline, monitored in real time after injection business datum；After pipeline is manually stopped,
Supervisor engine stops pipeline example, and the pipeline in monitoring data storehouse is placed in into suspended state, the real-time of the pipeline is no longer received
Data；When manual-lock pipeline or after receive data flow done state, supervisor engine stops pipeline example, unloads pipeline mould
Type, closes the conduit acceptor passage, pipeline data is removed from monitoring data storehouse.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310741890.7A CN104092578B (en) | 2013-12-29 | 2013-12-29 | Data stream monitoring method based on pipe method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310741890.7A CN104092578B (en) | 2013-12-29 | 2013-12-29 | Data stream monitoring method based on pipe method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104092578A CN104092578A (en) | 2014-10-08 |
| CN104092578B true CN104092578B (en) | 2017-09-19 |
Family
ID=51640260
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310741890.7A Active CN104092578B (en) | 2013-12-29 | 2013-12-29 | Data stream monitoring method based on pipe method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104092578B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9524775B1 (en) | 2015-10-20 | 2016-12-20 | International Business Machines Corporation | Correlation detector |
| CN111599174B (en) * | 2020-05-13 | 2021-11-16 | 北京百度网讯科技有限公司 | Traffic index abnormity detection method and electronic equipment |
| CN112181963A (en) * | 2020-09-28 | 2021-01-05 | 新奥数能科技有限公司 | Data processing method, device, server and storage medium for digital enterprise |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102262563A (en) * | 2011-08-09 | 2011-11-30 | 南京南瑞继保电气有限公司 | Method for data interaction and modeling of dispatching subsystem |
| CN102819249A (en) * | 2012-08-03 | 2012-12-12 | 山东康威通信技术股份有限公司 | Linkage control system and control method for power tunnel |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030135575A1 (en) * | 2002-01-14 | 2003-07-17 | Richard Marejka | Self-monitoring and trending service system with cascaded pipeline linking numerous client systems |
-
2013
- 2013-12-29 CN CN201310741890.7A patent/CN104092578B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102262563A (en) * | 2011-08-09 | 2011-11-30 | 南京南瑞继保电气有限公司 | Method for data interaction and modeling of dispatching subsystem |
| CN102819249A (en) * | 2012-08-03 | 2012-12-12 | 山东康威通信技术股份有限公司 | Linkage control system and control method for power tunnel |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104092578A (en) | 2014-10-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Design of liveness-enforcing supervisors for flexible manufacturing systems using Petri nets | |
| AU2011271002B2 (en) | Processing related datasets | |
| CN104092578B (en) | Data stream monitoring method based on pipe method | |
| CN103218263B (en) | The dynamic defining method of MapReduce parameter and device | |
| CN101589366A (en) | Parallelization and instrumentation in a producer graph oriented programming framework | |
| Baumgrass et al. | BPMN extension for business process monitoring | |
| KR101032819B1 (en) | Manufacturing prediction server | |
| US20160283610A1 (en) | Hybrid flows containing a continous flow | |
| US7653452B2 (en) | Methods and computer systems for reducing runtimes in material requirements planning | |
| CN107357825A (en) | A kind of project document management system based on database | |
| CN108052314A (en) | The method and device that a kind of plug-in unit based on Gradle is packaged | |
| CN112379884B (en) | Method and system for realizing flow engine based on Spark and parallel memory calculation | |
| CN108614690A (en) | Software development methodology, system based on component-oriented and high in the clouds and storage medium | |
| US8781609B2 (en) | Signal processing network | |
| CN104731800B (en) | Data analysis set-up | |
| Brodsky et al. | Analysis and optimization in smart manufacturing based on a reusable knowledge base for process performance models | |
| CN108829802A (en) | It is associated with log back method and device | |
| US8918765B2 (en) | Auto-documenting based on real-time analysis of code execution | |
| CN104700187A (en) | Bill generation method | |
| CN102929853A (en) | DCS (Data Communication System) project data generating system and method based on Excel sheet association | |
| Liu et al. | Project time-cost trade-off optimization by maximal flow theory | |
| US10459920B2 (en) | Support actual and virtual SQL dataflow by streaming infrastructure | |
| Sukhorukov et al. | Robotization of business processes of enterprises of housing and communal services | |
| CN103324776A (en) | Linear hybrid system invariance generating method based on abstract interpretation | |
| Bettoni et al. | Towards sustainability assessment: Reference data model for integrated product, process, supply chain design |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |