CN109067565A - A kind of fusion calculation method and apparatus for isomery timing operation/maintenance data - Google Patents
A kind of fusion calculation method and apparatus for isomery timing operation/maintenance data Download PDFInfo
- Publication number
- CN109067565A CN109067565A CN201810722221.8A CN201810722221A CN109067565A CN 109067565 A CN109067565 A CN 109067565A CN 201810722221 A CN201810722221 A CN 201810722221A CN 109067565 A CN109067565 A CN 109067565A
- Authority
- CN
- China
- Prior art keywords
- data
- fusion
- isomery
- maintenance data
- maintenance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/142—Network analysis or design using statistical or mathematical methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Algebra (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Pure & Applied Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a kind of fusion calculation methods for isomery timing operation/maintenance data, comprising the following steps: passes through agreement or the multi-source heterogeneous operation/maintenance data of API type collection；The multi-source heterogeneous operation/maintenance data is fused to diagram data based on connection topology；Using the diagram data, monitoring and early warning report data is generated by stream calculation.For O&M scenarios the Caspian Sea amount, isomery, the data for having temporal characteristics, it is excavated by construction and connects topological attribute, novel fusion is carried out, to research and develop multiple dimensioned multi-level stream calculation system, general data is opened up to the connecting-type of such special tectonic and are handled.Thus, it is possible to establish IT O&M environment Internet of Things, physical machine, virtual machine isomeric data Fusion Model frame so that IT O&M has more global data basis；For fused data, deployment distribution and streaming computing system are researched and developed, more real-time, accurately early warning and diagnosis is realized, promotes IT O&M actual effect.
Description
Technical field
The present invention relates to big data processing technology fields, are used for isomery timing operation/maintenance data more specifically to one kind
Fusion calculation method and apparatus.
Background technique
The network O&M management system (IT O&M) of a new generation starts from the management of network-oriented equipment to network-oriented industry
The management transition of business.This network management thought services network, business is as network management object, passes through real-time monitoring and network service phase
The equipment of pass, application are provided by simulating the service quality of client's real-time measurement network service by the business of collection network business
Material realizes comprehensive, multi-angle of view monitoring network service operation situation purpose, to realize the fault management of network service, property
It can management and configuration management.
The literal meaning of data fusion is more intuitive, even if industry is under no theoretical and frame guidance, many data are searched
Collection, the conventional treatment handled have also agreed the theory of many data fusions without prior consultation.Data fusion also there are many close similar title, than
As data merge (merge), data collaborative (synergy), aggregation of data (integration), data combination
(combination) etc..Generally defined according to one that L.Wald in 1998 is proposed: data fusion is a formal frame
Frame defines different amalgamation mode and tool under this frame, by these modes and tool by the data from different sources into
Row joint, its object is to obtain the information of better quality, and the improvement of quality depends on application.The advantages of this definition, exists
In: 1) it is a framework that it, which highlight data fusion, and be not limited to tool and method itself.2) fusion knot is emphasized in this definition
The quality evaluation of fruit is with the application service that is serviced.The development of this theory frame, with the IT O&M itself mentioned before from poly-
Coke in NSM to BSM evolution, framework with service play the same tune on different musical instruments on associated thinking of development it is wonderful.
In machine learning scope, data fusion, which refers to from multiple information sources, integrates information, to be formed to related entities
Specifically, data accumulation comprehensively, unified.This process of data fusion may operate in several levels: data level (feature extraction with
Before), characteristic layer (before attribute specification) and decision-making level (after data independent attribute explanation).Pixel-based fusion it is main
Method has: HIS transformation, PCA transformation, high-pass filtering method, weigthed sums approach, Wavelet Transform Fusion method etc..Feature level fusing method
Mainly have: Dempster-shafer rationalistic method, Bayes' assessment, entropy method, the cum rights method of average, voting method etc..Decision level data
Fusion method mainly has: Bayes' assessment, expert system, neural network, fuzzy set etc..
The data fusion difficulty that IT O&M faces essentially consists in the presence of heterogeneous data source, is mainly reflected in: 1) physics is whole
Hold fragmentation, mobile, instrumentation trend；2) equipment and each level virtualization of network, especially container virtualization, SDN
Network virtualization.
IT system generally has a multiple nodes, multiple links, and the data of acquisition have multi-source, each across period, structuring
Different feature.On the one hand as the internet of things era is arrived, the temperature and humidity fire-fighting electric power such as data center, network center, equipment machine room
Deng sensor deployment it is increasingly intensive.And all kinds of physical equipments are in system, network and application layer operation/maintenance data monitoring data format
It is become more diversified with structured features.In addition with cloud and virtualization development, especially Docker class container virtualization technology
So that the old device virtualization of low side is also gradually universal, the extension of their high resiliency and instant feature transport virtualization system
Dimension data has more instantaneity and burst.To realize global O&M monitoring and decision, to all kinds of half structures, it is non-structural, when decline
Storage, synthesis, fusion and the retrieval etc. of property data seem particularly necessary.
Summary of the invention
The purpose of the present invention is to provide a kind of fusion calculation method and apparatus for isomery timing operation/maintenance data.
The technical solution used to solve the technical problems of the present invention is that: a kind of melting for isomery timing operation/maintenance data is provided
Close calculation method, comprising the following steps:
Pass through agreement or the multi-source heterogeneous operation/maintenance data of API type collection；
The multi-source heterogeneous operation/maintenance data is fused to diagram data based on connection topology；
Using the diagram data, monitoring and early warning report data is generated by stream calculation.
It, will be described based on connection topology in the fusion calculation method provided by the present invention for isomery timing operation/maintenance data
The step that multi-source heterogeneous operation/maintenance data is fused to diagram data includes:
By data reasoning, when the multi-source heterogeneous operation/maintenance data is expanded by triple sequential log format as five-tuple
Sequence journal format；
Fusion calculation is carried out to the multi-source heterogeneous operation/maintenance data of five-tuple sequential log format, generates the figure number
According to.
In the fusion calculation method provided by the present invention for isomery timing operation/maintenance data, to five-tuple sequential log
The multi-source heterogeneous operation/maintenance data of format carries out fusion calculation, after the step for generating the diagram data further include:
Fusion Indexs measure is carried out to the diagram data.
In the fusion calculation method provided by the present invention for isomery timing operation/maintenance data, using the diagram data, lead to
Cross stream calculation generate monitoring and early warning report data the step include:
According to the difference of fusion frequency, classified by stream calculation to the diagram data；
According to the sorted diagram data, the monitoring and early warning report data is generated.
In the fusion calculation method provided by the present invention for isomery timing operation/maintenance data, according to fusion frequency not
Together, in the step classified to the diagram data by stream calculation, the diagram data is divided into large scale graph key section
Point, mesoscale graph key node and small scalogram key node.
Correspondingly, the present invention also provides a kind of fusion calculation devices for isomery timing operation/maintenance data, comprising:
Acquisition module, for passing through agreement or the multi-source heterogeneous operation/maintenance data of API type collection；
Fusion Module, for the multi-source heterogeneous operation/maintenance data to be fused to diagram data based on connection topology；
Computing module generates monitoring and early warning report data by stream calculation for utilizing the diagram data.
In the fusion calculation device provided by the present invention for isomery timing operation/maintenance data, the Fusion Module includes:
Unit is expanded, for passing through data reasoning, by the multi-source heterogeneous operation/maintenance data by triple sequential log format
Expanding is five-tuple sequential log format；
Fusion calculation unit carries out based on fusion by the multi-source heterogeneous operation/maintenance data to five-tuple sequential log format
It calculates, generates the diagram data.
In the fusion calculation device provided by the present invention for isomery timing operation/maintenance data, the Fusion Module is also wrapped
It includes:
Detection unit, for carrying out fusion Indexs measure to the diagram data.
In the fusion calculation device provided by the present invention for isomery timing operation/maintenance data, the computing module includes:
Taxon classifies to the diagram data by stream calculation for the difference according to fusion frequency；
Generation unit, for generating the monitoring and early warning report data according to the sorted diagram data.
In the fusion calculation device provided by the present invention for isomery timing operation/maintenance data, the taxon will be described
Diagram data is divided into large scale graph key node, mesoscale graph key node and small scalogram key node.
Fusion calculation method and apparatus for isomery timing operation/maintenance data of the invention, which have the advantages that, to be directed to
O&M scenarios the Caspian Sea amount, isomery, the data for having temporal characteristics excavate it by construction and connect topological attribute, carry out novel
Fusion, to research and develop multiple dimensioned multi-level stream calculation system, opens up general data to the connecting-type of such special tectonic and handles.By
This, can establish IT O&M environment Internet of Things, physical machine, virtual machine isomeric data Fusion Model frame so that IT O&M has more
Global data basis；For fused data, deployment distribution and streaming computing system are researched and developed, is realized more real-time, accurately pre-
Alert and diagnosis, promotes IT O&M actual effect.
Detailed description of the invention
Fig. 1 is the flow chart of the fusion calculation method first embodiment for isomery timing operation/maintenance data of the invention；
Fig. 2 show the flow chart of step S2 shown in FIG. 1；
Fig. 3 show the flow chart of step S3 shown in FIG. 1；
Fig. 4 is the schematic diagram of the fusion calculation device for isomery timing operation/maintenance data of the invention；
Fig. 5 is the structural schematic diagram of a kind of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
In order to make those skilled in the art more fully understand the present invention program, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
The embodiment of a part of the invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
The model that the present invention protects all should belong in member's every other embodiment obtained without making creative work
It encloses.
Description and claims of this specification and term " first ", " second ", " third " and " in above-mentioned attached drawing
Four " etc. are not use to describe a particular order for distinguishing different objects.In addition, term " includes " and " having " and it
Any deformation, it is intended that cover and non-exclusive include.Such as it contains the process, method of a series of steps or units, be
System, product or equipment are not limited to listed step or unit, but optionally further comprising the step of not listing or list
Member, or optionally further comprising other step or units intrinsic for these process, methods, product or equipment.
Below in conjunction with drawings and examples the present invention will be further explained explanation.
Fig. 1 is the flow chart of the fusion calculation method first embodiment for isomery timing operation/maintenance data of the invention；Such as
Shown in Fig. 1, fusion calculation method provided by the present invention for isomery timing operation/maintenance data the following steps are included:
S1, pass through agreement or the multi-source heterogeneous operation/maintenance data of API type collection；
S2, the multi-source heterogeneous operation/maintenance data is fused to by diagram data based on connection topology；
S3, using the diagram data, pass through stream calculation and generate monitoring and early warning report data.
In step sl, it is acquired, by agreement or API type collection operation/maintenance data, including physical machine data based on association
The acquisition of virtual machine virtual network data and the acquisition of Internet of Things sensing data of view, wherein partial virtual Environmental Support API access obtains
Data, the environment such as general networks, equipment then make full use of the agreements such as TCP/IP, UDP, SNMP, PING, HTTP, ICMP, SSH to adopt
Collect data.
Further, IT operation/maintenance data type can usually be defined as the log number with timestamp from the point of view of abstract angle
According to, but its data is still multifarious.Some daily record datas, such as web log, structuring is relatively preferable, and every log includes
Field is generally sanctified by usage, concentrate on client (user) IP address, access port, the response time, request time, the address url,
Request method, status code, page-size, source, browser language, user browser other information, browser version, browser
Type etc..But it is collected by the system log of syslog, syslog-ng, rsyslog collection, by scribe and fluentd
Business diary etc. is customized the log reported based on http (s) post agreement, or is the vSphere based on VMware
The virtual machine management systems such as Client, based on ping/traceroute/SNMP etc. to the sampling instrument of data flow or package,
The tired log of SDN software definition lamination, their schema format and disunity.Therefore, in this application, collected data
It is mainly stored by NoSQL, to solve the problems, such as acquisition, data are polynary, isomery, Schema extension and calamity changeable, and obtained
Standby effect.But partial structured or intermediate data is also stored by relational database cluster.
In step s 2, the isomery time series data Fusion Model based on connection topology expands triple log in O&M environment
It fills for five-tuple format, so as to set up under each timeslice in entire O&M environment clear between each node and company abundant
Relationship is connect, the topological structure and figure shape data of their static dynamic networks of fusion and membership are constructed.Therefore, such as Fig. 2 institute
Show, step S2 includes following sub-step:
Step S21: by data reasoning, it is by the expansion of triple sequential log format by the multi-source heterogeneous operation/maintenance data
Five-tuple sequential log format；
Step S22: fusion calculation is carried out to the multi-source heterogeneous operation/maintenance data of five-tuple sequential log format, generates institute
State diagram data.
Specifically, every daily record data that (such as some data center) acquires in entire O&M environment, generally all meets
Following triplet format:
<timestamp, entity or service ID, various dimensions multi-format log content>
Therefore, in the step s 21, the relevant object of log is placed into the connection structure belonging to them and is considered.At this
This log of a timestamp, according to topological structure belonging to the entity or service ID (assuming that being numbered is node A), service
Group and log content can associate with another entity (assuming that being numbered is node B), so that every log will
It is expanded into following five-tuple:
< timestamp, node A (entity or service ID), node B (entity or service ID), connection attribute, various dimensions multi-format
Log content >
Wherein, the set that connection attribute is formed in response to IT O&M scenarios is different, two sections including but not limited in static topological
Level above and below in the inner node connection of physical connection, dynamic network (such as SDN), instantaneous web, tcp connection, network layer between point
Set membership, controller and the administrative relationships of controlled unit etc. in virtual environment.
Once log is extended for five-tuple, these data have several big features: 1) because node to and connection attribute deposit
To be implied by connection topology, or defined according to science, they form figure (Graph) data；2) because of log content
Various dimensions multi-format, the structuring of data content meets isomery feature；3) log naturally has the not stopping pregnancy of fine-time stamp
It is raw, it is the time series data of suitable high frequency.But the diagram data that daily record data is formed has its own spy compared to conventional static diagram data
Color: node is dynamic change with connection side；The attribute type and event content for connecting side are related to text, description etc.；Time window
The graph structure that the size (such as 1 second or 1 minute section) of mouth obtains is different.
Therefore, in step S22, the operation/maintenance data of the five-tuple sequential log format obtained to extension is needed to melt
It closes, here in addition to using conventional Various types of data fusion, pretreatment, ETL technology, it is also necessary to the fusion of isomery diagram data is used,
This includes multi-connection polymerization between time series data indexing, node, figure connection beta pruning etc..
Further, after fusion obtains diagram data, it is also necessary to it is evaluated and tested, therefore, step S2 further include:
Step S23, fusion Indexs measure is carried out to the diagram data.
It in step S23, is evaluated and tested based on more wheel multi-expert sampling and comprehensive, to the accurate of IT operation/maintenance data fusion frame
Degree, integrity degree, private index are evaluated and tested.
In step s3, the diagram data obtained for fusion, research and development deployment distribution and streaming computing system, are realized more real
When, accurately early warning and diagnosis, promoted IT O&M actual effect.Because operation/maintenance data has timing fine granularity characteristic, pass through me
Data fusion, according to the difference of fusion frequency, the diagram data scale of acquisition is different.And because what graph structure interior nodes had
It is under the jurisdiction of in hierarchy, so they are also equipped with the feature of layering fragment.Therefore, as shown in figure 3, step S3 includes following son
Step:
Step S31, according to the difference of fusion frequency, classified by stream calculation to the diagram data；
Step S32, according to the sorted diagram data, the monitoring and early warning report data is generated.
In this application, the diagram data obtained to fusion, introduces Spark for it in the form of RDD data structure
Streaming stream processing system, connecting data according to it by Map operation, filter operation, collectingN operation etc. will
It is divided into large scale graph key node, mesoscale graph key node and small scalogram key node.Further according to sorted figure number
According to the machine learning module such as research and development invasion judgement, exception monitoring, intelligent patrol detection, near real-time early warning.
In the first embodiment of the fusion calculation method for isomery timing operation/maintenance data of the invention, for O&M field
Scape the Caspian Sea amount, isomery, the data for having temporal characteristics excavate it by construction and connect topological attribute, carry out novel fusion, from
And multiple dimensioned multi-level stream calculation system is researched and developed, general data are opened up to the connecting-type of such special tectonic and are handled.Thus, it is possible to
Establish IT O&M environment Internet of Things, physical machine, virtual machine isomeric data Fusion Model frame so that IT O&M has more global number
According to basis；For fused data, deployment distribution and streaming computing system are researched and developed, realizes more real-time, accurately early warning and diagnosis,
Promote IT O&M actual effect.
Fig. 4 is the schematic diagram of the fusion calculation device for isomery timing operation/maintenance data of the invention, as shown in figure 4,
In the present embodiment, the fusion calculation device 400 for isomery timing operation/maintenance data includes:
Acquisition module 410, for passing through agreement or the multi-source heterogeneous operation/maintenance data of API type collection；
Fusion Module 420, for the multi-source heterogeneous operation/maintenance data to be fused to diagram data based on connection topology；
Computing module 430 generates monitoring and early warning report data by stream calculation for utilizing the diagram data.
Specifically, in an embodiment of the present invention, acquisition module passes through agreement or API type collection operation/maintenance data, packet
Include the acquisition of physical machine data, the acquisition of virtual machine virtual network data and the acquisition of Internet of Things sensing data based on agreement, wherein part
Virtual environment support API access obtain data, the environment such as general networks, equipment then make full use of TCP/IP, UDP, SNMP,
The agreements such as PING, HTTP, ICMP, SSH acquire data.
Specifically, in an embodiment of the present invention, isomery time series data Fusion Model of the Fusion Module based on connection topology
Triple log in O&M environment is extended for five-tuple format, so as to set up entire O&M environment under each timeslice
In clear between each node and connection relationship abundant, construct the topological structure of their static dynamic networks of fusion and membership
With figure shape data.Therefore, the Fusion Module includes:
Unit is expanded, for passing through data reasoning, by the multi-source heterogeneous operation/maintenance data by triple sequential log format
Expanding is five-tuple sequential log format；
Fusion calculation unit carries out based on fusion by the multi-source heterogeneous operation/maintenance data to five-tuple sequential log format
It calculates, generates the diagram data.
Further, after fusion obtains diagram data, it is also necessary to be evaluated and tested to it, therefore, the Fusion Module is also
Include:
Detection unit, for carrying out fusion Indexs measure to the diagram data.
Specifically, in an embodiment of the present invention, because operation/maintenance data has timing fine granularity characteristic, pass through us
Data fusion, according to the difference of fusion frequency, the diagram data scale of acquisition is different.And the because person in servitude that graph structure interior nodes have
Belong in hierarchy, so they are also equipped with the feature of layering fragment.Therefore, the computing module includes:
Taxon classifies to the diagram data by stream calculation for the difference according to fusion frequency；
Generation unit, for generating the monitoring and early warning report data according to the sorted diagram data.
In this application, the diagram data obtained to fusion, introduces Spark for it in the form of RDD data structure
Streaming stream processing system, connecting data according to it by Map operation, filter operation, collectingN operation etc. will
It is divided into large scale graph key node, mesoscale graph key node and small scalogram key node.Further according to sorted figure number
According to the machine learning module such as research and development invasion judgement, exception monitoring, intelligent patrol detection, near real-time early warning.
About the device in above-described embodiment, wherein modules execute the concrete mode of operation in related this method
Embodiment in be described in detail, no detailed explanation will be given here.
Referring to Fig. 5, Fig. 5 is the structural schematic diagram of a kind of electronic equipment provided in an embodiment of the present invention, the electronic equipment
500, may include:
Processor 510, memory 520, communication interface 530 and bus 540；
The processor 510, the memory 520 and the communication interface 530 are connected and are completed by the bus 540
Mutual communication；
The memory 520 stores executable program code；
The processor 510 run by reading the executable program code stored in the memory 520 with it is described
The corresponding program of executable program code, for executing a kind of fusion calculation method for isomery timing operation/maintenance data；Its
In, a kind of fusion calculation method for isomery timing operation/maintenance data, comprising the following steps:
Pass through agreement or the multi-source heterogeneous operation/maintenance data of API type collection；
The multi-source heterogeneous operation/maintenance data is fused to diagram data based on connection topology；
Using the diagram data, monitoring and early warning report data is generated by stream calculation.
As can be seen that using technical solution provided in an embodiment of the present invention, it can establish IT O&M environment Internet of Things, physics
The Fusion Model frame of machine, virtual machine isomeric data, so that IT O&M has more global data basis；For fused data, grind
Deployment distribution and streaming computing system are sent out, more real-time, accurately early warning and diagnosis is realized, promotes IT O&M actual effect.
The embodiment of the invention also provides a kind of storage mediums, wherein the storage medium is described for storing application program
Application program for executing a kind of fusion calculation for isomery timing operation/maintenance data described in the embodiment of the present invention at runtime
Method.
The embodiment of the invention also provides a kind of application programs, wherein the application program for executing this hair at runtime
A kind of fusion calculation method for isomery timing operation/maintenance data described in bright embodiment.
It should be noted that for the various method embodiments described above, for simple description, therefore, it is stated as a series of
Combination of actions, but those skilled in the art should understand that, the present invention is not limited by the sequence of acts described because
According to the present invention, some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art should also know
It knows, the embodiments described in the specification are all preferred embodiments, and related actions and modules is not necessarily of the invention
It is necessary.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, there is no the portion being described in detail in some embodiment
Point, reference can be made to the related descriptions of other embodiments.
In several embodiments provided herein, it should be understood that disclosed device, it can be by another way
It realizes.For example, the apparatus embodiments described above are merely exemplary, such as the division of the unit, it is only a kind of
Logical function partition, there may be another division manner in actual implementation, such as multiple units or components can combine or can
To be integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutual
Coupling, direct-coupling or communication connection can be through some interfaces, the indirect coupling or communication connection of device or unit,
It can be electrical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially
The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words
It embodies, which is stored in a storage medium, including some instructions are used so that a computer
Equipment (can for personal computer, server or network equipment etc.) execute each embodiment the method for the present invention whole or
Part steps.And storage medium above-mentioned includes: that USB flash disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
Reservoir (RAM, Random Access Memory), mobile hard disk, magnetic or disk etc. be various to can store program code
Medium.
The above, above embodiments are only to illustrate technical solution of the present invention, rather than its limitations；Although referring to aforementioned
Invention is explained in detail for embodiment, wherein those skilled in the art should understand that: it still can be right
Technical solution documented by foregoing embodiments is modified or equivalent replacement of some of the technical features；And this
It modifies or replaces, the range for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should be included within scope of the presently claimed invention.
Claims (10)
1. a kind of fusion calculation method for isomery timing operation/maintenance data, which comprises the following steps:
Pass through agreement or the multi-source heterogeneous operation/maintenance data of API type collection；
The multi-source heterogeneous operation/maintenance data is fused to diagram data based on connection topology；
Using the diagram data, monitoring and early warning report data is generated by stream calculation.
2. the fusion calculation method according to claim 1 for isomery timing operation/maintenance data, which is characterized in that based on company
Topology, which is connect, by the step that the multi-source heterogeneous operation/maintenance data is fused to diagram data includes:
By data reasoning, the multi-source heterogeneous operation/maintenance data is expanded by triple sequential log format as five-tuple timing day
Will format；
Fusion calculation is carried out to the multi-source heterogeneous operation/maintenance data of five-tuple sequential log format, generates the diagram data.
3. the fusion calculation method according to claim 2 for isomery timing operation/maintenance data, which is characterized in that five
The multi-source heterogeneous operation/maintenance data of tuple sequential log format carries out fusion calculation, generate the diagram data the step it
Afterwards further include:
Fusion Indexs measure is carried out to the diagram data.
4. the fusion calculation method according to claim 1 for isomery timing operation/maintenance data, which is characterized in that utilize institute
Diagram data is stated, includes: by the step that stream calculation generates monitoring and early warning report data
According to the difference of fusion frequency, classified by stream calculation to the diagram data；
According to the sorted diagram data, the monitoring and early warning report data is generated.
5. the fusion calculation method according to claim 4 for isomery timing operation/maintenance data, which is characterized in that in basis
In the step classified by stream calculation to the diagram data, the diagram data is divided into greatly for the difference of fusion frequency
Scalogram key node, mesoscale graph key node and small scalogram key node.
6. a kind of fusion calculation device for isomery timing operation/maintenance data characterized by comprising
Acquisition module, for passing through agreement or the multi-source heterogeneous operation/maintenance data of API type collection；
Fusion Module, for the multi-source heterogeneous operation/maintenance data to be fused to diagram data based on connection topology；
Computing module generates monitoring and early warning report data by stream calculation for utilizing the diagram data.
7. the fusion calculation device according to claim 6 for isomery timing operation/maintenance data, which is characterized in that described to melt
Molding block includes:
Unit is expanded, for by data reasoning, the multi-source heterogeneous operation/maintenance data to be expanded by triple sequential log format
For five-tuple sequential log format；
Fusion calculation unit carries out fusion calculation for the multi-source heterogeneous operation/maintenance data to five-tuple sequential log format,
Generate the diagram data.
8. the fusion calculation device according to claim 7 for isomery timing operation/maintenance data, which is characterized in that described to melt
Mold block further include:
Detection unit, for carrying out fusion Indexs measure to the diagram data.
9. the fusion calculation device according to claim 6 for isomery timing operation/maintenance data, which is characterized in that the meter
Calculating module includes:
Taxon classifies to the diagram data by stream calculation for the difference according to fusion frequency；
Generation unit, for generating the monitoring and early warning report data according to the sorted diagram data.
10. the fusion calculation device according to claim 9 for isomery timing operation/maintenance data, which is characterized in that described
The diagram data is divided into large scale graph key node, mesoscale graph key node and small scalogram key node by taxon.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810722221.8A CN109067565A (en) | 2018-07-03 | 2018-07-03 | A kind of fusion calculation method and apparatus for isomery timing operation/maintenance data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810722221.8A CN109067565A (en) | 2018-07-03 | 2018-07-03 | A kind of fusion calculation method and apparatus for isomery timing operation/maintenance data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109067565A true CN109067565A (en) | 2018-12-21 |
Family
ID=64818626
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810722221.8A Pending CN109067565A (en) | 2018-07-03 | 2018-07-03 | A kind of fusion calculation method and apparatus for isomery timing operation/maintenance data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109067565A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109617750A (en) * | 2019-01-31 | 2019-04-12 | 国网电子商务有限公司 | A kind of service method for early warning and gateway |
| CN114595773A (en) * | 2022-03-10 | 2022-06-07 | 广东泰云泽科技有限公司 | Multi-source heterogeneous twin data fusion method and system based on factory production process |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20120056330A (en) * | 2010-11-25 | 2012-06-04 | 국방과학연구소 | Apparatus and method for threat data integration |
| CN104283318A (en) * | 2014-10-24 | 2015-01-14 | 国家电网公司 | Power equipment integrated monitoring and early warning system based on big data and analysis method thereof |
| CN106708815A (en) * | 2015-07-15 | 2017-05-24 | 中兴通讯股份有限公司 | Data processing method, device and system |
| CN107404400A (en) * | 2017-07-20 | 2017-11-28 | 中国电子科技集团公司第二十九研究所 | A kind of network situation awareness implementation method and device |
| CN107832354A (en) * | 2017-10-23 | 2018-03-23 | 珠海许继芝电网自动化有限公司 | A kind of power distribution network intelligence O＆M managing and control system of distributed caching |
-
2018
- 2018-07-03 CN CN201810722221.8A patent/CN109067565A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20120056330A (en) * | 2010-11-25 | 2012-06-04 | 국방과학연구소 | Apparatus and method for threat data integration |
| CN104283318A (en) * | 2014-10-24 | 2015-01-14 | 国家电网公司 | Power equipment integrated monitoring and early warning system based on big data and analysis method thereof |
| CN106708815A (en) * | 2015-07-15 | 2017-05-24 | 中兴通讯股份有限公司 | Data processing method, device and system |
| CN107404400A (en) * | 2017-07-20 | 2017-11-28 | 中国电子科技集团公司第二十九研究所 | A kind of network situation awareness implementation method and device |
| CN107832354A (en) * | 2017-10-23 | 2018-03-23 | 珠海许继芝电网自动化有限公司 | A kind of power distribution network intelligence O＆M managing and control system of distributed caching |
Non-Patent Citations (2)
| Title |
|---|
| 李杰，刘林阴，陈慧丽等: "《嵌入式Linux系统 开发原理与实战》", 31 December 2017 * |
| 赵和平，何熊文，刘崇华等: "《国之重器出版工程 空间数据系统》", 1 May 2018 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109617750A (en) * | 2019-01-31 | 2019-04-12 | 国网电子商务有限公司 | A kind of service method for early warning and gateway |
| CN114595773A (en) * | 2022-03-10 | 2022-06-07 | 广东泰云泽科技有限公司 | Multi-source heterogeneous twin data fusion method and system based on factory production process |
| CN114595773B (en) * | 2022-03-10 | 2022-11-15 | 天津科技大学 | Multi-source heterogeneous twin data fusion method and system based on factory production process |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Rathore et al. | Efficient graph-oriented smart transportation using internet of things generated big data | |
| Ganz et al. | A practical evaluation of information processing and abstraction techniques for the internet of things | |
| Yang et al. | A time efficient approach for detecting errors in big sensor data on cloud | |
| CN107683597A (en) | Network behavior data collection and analysis for abnormality detection | |
| CN107690776A (en) | For the method and apparatus that feature is grouped into the case for having selectable case border in abnormality detection | |
| CN108259371A (en) | A kind of network flow data analysis method and device based on stream process | |
| CN103905440B (en) | Network security situation awareness analysis method based on log and SNMP information fusion | |
| US11392733B2 (en) | Multi-dimensional event model generation | |
| Herrero et al. | RT-MOVICAB-IDS: Addressing real-time intrusion detection | |
| CN101095307A (en) | Network management appliance | |
| Latif et al. | Blockchain and iot based formal model of smart waste management system using tla+ | |
| CN113347170B (en) | Intelligent analysis platform design method based on big data framework | |
| CN104486116A (en) | Multidimensional query method and multidimensional query system of flow data | |
| CN111917747A (en) | Campus network security situation awareness system and method | |
| CN106254137A (en) | The alarm root-cause analysis system and method for supervisory systems | |
| CN103916478B (en) | The method and apparatus that streaming based on distributed system builds data side | |
| CN109067565A (en) | A kind of fusion calculation method and apparatus for isomery timing operation/maintenance data | |
| CN107846460A (en) | A kind of recurrence system and method for Military Information System information flow | |
| CN110460662A (en) | The processing method and system of internet of things data | |
| CN114430331A (en) | Network security situation sensing method and system based on knowledge graph | |
| Fournier-Viger et al. | Discovering alarm correlation rules for network fault management | |
| Wang et al. | Anomaly detection in wireless sensor networks based on KNN | |
| Mateen et al. | Software QualityAssurance in Internet of Things | |
| CN111756691A (en) | Acquisition strategy conflict detection method and device, electronic equipment and storage medium | |
| Chang et al. | Smartparcels: Cross-layer iot planning for smart communities |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181221 |
|
| RJ01 | Rejection of invention patent application after publication |