CN106572190A - Autonomous collection method for operational data of information communication - Google Patents
Autonomous collection method for operational data of information communication Download PDFInfo
- Publication number
- CN106572190A CN106572190A CN201611003412.6A CN201611003412A CN106572190A CN 106572190 A CN106572190 A CN 106572190A CN 201611003412 A CN201611003412 A CN 201611003412A CN 106572190 A CN106572190 A CN 106572190A
- Authority
- CN
- China
- Prior art keywords
- data
- real
- time
- performance
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000004891 communication Methods 0.000 title abstract description 26
- 238000011160 research Methods 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims abstract description 10
- 230000006870 function Effects 0.000 claims description 6
- 230000009467 reduction Effects 0.000 claims description 4
- 238000012517 data analytics Methods 0.000 claims description 3
- 230000002085 persistent effect Effects 0.000 claims description 3
- 230000003014 reinforcing effect Effects 0.000 claims description 3
- 239000012634 fragment Substances 0.000 claims description 2
- 238000004458 analytical method Methods 0.000 abstract description 35
- 238000005516 engineering process Methods 0.000 abstract description 18
- 238000007726 management method Methods 0.000 abstract description 12
- 230000002708 enhancing effect Effects 0.000 abstract 1
- 238000012544 monitoring process Methods 0.000 description 25
- 230000008901 benefit Effects 0.000 description 7
- 235000019580 granularity Nutrition 0.000 description 6
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000013499 data model Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 238000005206 flow analysis Methods 0.000 description 3
- 239000003795 chemical substances by application Substances 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 230000032683 aging Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013481 data capture Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 239000000975 dye Substances 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000010304 firing Methods 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000006317 isomerization reaction Methods 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000003012 network analysis Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000010223 real-time analysis Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/465—Distributed object oriented systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Abstract
The invention discloses an autonomous collection method for the operational data of information communication. The method comprises the steps of conducting the research on a kernel customized for capturing packets and maximally reducing the loss of the packet capturing performance, wherein the real-time gripping capability reaches a line speed; adopting a multi-threaded service data parsing engine, maximally exerting the parallel processing capability of a CPU unit; realizing the centralized management through a unified platform in the distributed deployment mode; designing a real-time distributed processing mode for service data; calculating the data of key performance indexes for display purpose by a pretreatment program, inputting the data into a key value type memory database, and exporting the data into a file for persistently storing the data after a period of time; designing a multi-level cache mode and enhancing the real-time performance. According to the technical scheme of the invention, in combination with the current operation state of the power system, the key points of the operation performance of an information communication system and the information communication acquisition technology are studied. Meanwhile, the corresponding standardized finishing operation is conducted, so that the data support is provided for an intelligent analysis system. Meanwhile, the method improves the performance indexes of equipment applications.
Description
Technical field
The present invention relates to information processing technology neighborhood, particularly relates to a kind of information communication service data self-collection method.
Background technology
Conclusion data and proceduring data in being present in various information communication equipment or system, are the business that carries out point
Analysis, safety analysis, the important evidence of performance evaluation, these conclusions or procedural data, generally there is following several presence sides
Formula：All kinds of daily records of information communication device or system；Information communication device or the SNMP information of system output；The network equipment is
The Flow information of system output；The information original flow analyzed by packet capturing and is obtained.This four category information each has certain lacking
Point, it is impossible to provide effective data supporting for analysis system.
The content of the invention
In view of this, it is an object of the invention to propose a kind of information communication service data self-collection method, Neng Gouwei
Intelligent analysis system provides data supporting, and help improves the performance indications of equipment application.
A kind of information communication service data self-collection method provided based on the above-mentioned purpose present invention, including：
The kernel that research aims at packet capturing and customizes, has reached the real-time Grasping skill of linear speed, and the reduction for maximizing degree is grabbed
Bag performance is lost；
Using multi-threaded business data analytics engine, the parallel processing capability of CPU groups is maximally utilized；
Using distributed deployment mode, managed concentratedly by the unified platform；
The real-time distributed processing mode of business datum is designed with, the business datum produced by analytics engine is designed, in real time
Spue in memory database cluster, when the demand for having big data quantity, by increasing hardware server to build cluster, by process
Data scale is different, builds different scales cluster；
Data spue to before data-base cluster, and in real time data are done with classification pretreatment, and the data of same business can be whole
Manage under a fragment, and temporally longitudinal cutting, to improve treatment effeciency during computing；After data are stored in, temporally granularity
Real-time running data library inquiry function, by the statistics for calculating relevant database is put into, for front end displaying；It is simultaneously pre-
Processing routine calculate some for show KPI Key Performance Indicator numerical value, in being put into key assignments type memory database, for a period of time after
Unification exports to file persistent storage；
Multi-level buffer mode, reinforcing real-time performance are designed with, it is whole except the business datum of memory database is cached in real time
Each level of individual system also has the caching of oneself.
From the above it can be seen that the information communication service data self-collection method that the present invention is provided, with reference to electric power
The current operation present situation of system, investigates the key point of information communication system runnability, and Research Information Letter acquisition technique is gone forward side by side
Row respective standardization is arranged, and can provide data supporting for intelligent analysis system, and help improves the performance indications of equipment application.
Description of the drawings
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
The accompanying drawing to be used needed for having technology description is briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with
Other accompanying drawings are obtained according to these accompanying drawings.
Fig. 1 is the information communication service data self-collection method schematic diagram of the embodiment of the present invention.
Specific embodiment
To make the object, technical solutions and advantages of the present invention become more apparent, below in conjunction with specific embodiment, and reference
Accompanying drawing, the present invention is described in more detail.
The embodiment of the present invention is directed to conclusion or procedural data, and the definition of existing four category information and feature are sketched
It is as follows：
(1) all kinds of daily records of information communication device or system
System journal is the information of hardware in record system, software and system problem, while can be with monitoring system
Raw event.
In power industry various information system, all kinds of daily records are all there are, daily, or even all the time, all kinds of letters
Breathization system can all export and store substantial amounts of daily record data.
User can check the reason for mistake occurs, or the trace that attacker stays when finding under attack by it
Mark.System journal includes system journal, application log and security log.
System journal is a kind of very crucial component, because system journal can allow user to be fully understood by the ring of oneself
Border.This system log message is very crucial for the basic reason or reduction system firing area that determine failure
, because system journal can allow user to understand failure or all events attacked before occurring.For virtualized environment formulation
A set of good system journal strategy also it is critical that, because system journal need and many different external modules carry out
Association.Good system journal can prevent user from the angle analysis problem of mistake, it is to avoid waste valuable fault-remove time.Separately
A kind of outer reason is that, by means of system journal, keeper is likely to find before some from incognizant problem, several
In the middle of the environment of all just deployment systems daily record.
(2) information communication device or the SNMP information of system output
SNMP is the abbreviation of English " Simple Network Management Protocol ", and Chinese means " simple
NMP ".SNMP is environmental management agreement the most frequently used at present.
In power informatization framework, SNMP is widely adopted, and serves power information in the form of " webmaster " system
In the routine work of change department.
SNMP is designed to unrelated with agreement, thus it can in IP, IPX, AppleTalk, OSI and other use
Used on host-host protocol.SNMP is series of protocols group and specification, and they provide in a kind of equipment from network and collect
The method of network management information.SNMP also Reports a Problem to network management workstation for equipment and is provided a method that with mistake.
At present, almost all of network equipment manufacturer all realizes the support to SNMP.Leading the SNMP of trend is
The common communication protocol of the one equipment collection management information from network.The manager of equipment collects these information and records
In management information bank (MIB).The characteristic of these information reporting facilitys, data throughout, channel overloa4 and mistake etc..MIB has public affairs
Common form, so the snmp management instrument from multiple manufacturers can collect mib information, presents on management console and is
System keeper.
SNMP provides a kind of unification, cross-platform equipment control method.
(3) disparate networks equipment or the Flow information of system output
Flow technologies, are the important technology branch fields of network analysis in recent years, to a certain extent, have become one kind
Tendency technology.According to different vendor and technical standard, Flow is divided into NetFlow, the Sflow of foreign countries, and domestic China again
For the Netstream of company.
In the network architecture of power industry, the network equipment of Flow information can be exported, mainly Cisco of the U.S. is with
The route and switching equipment of Huawei of state.
NetFlow is cisco company-developed technology, and it is both a kind of switching technology, is again a kind of flow analysis technology,
It is also simultaneously one of billing technology of industry main flow.It can answer the following problem about IP flows：Who at what time,
Where, using which kind of agreement, access who, specific flow are how many problems such as.The occupation rate of market advantage of NetFlow is
One of flow analysis technology of current main flow.
To guarantee the uniformity and compatibility of network management, the Netstream technologies of Huawei Company, with NetFlow in skill
Art essence, it is all more similar on the form of Flow, but both still have details difference.
Based in the analysis method of Flow, network flow (Network Flow) generally defined as gives source node and purpose
One-way data bag/the frame sequence transmitted between node.Generally, the network equipment (3 layer switch, router etc.) itself provides base
In the analytic function in IP packet header, it is responsible for the analysis and arrangement of network flow data, according to the good data of certain condition and definition
To stream collector (Flow Collector) output data, then have the software of correlation again carries out the flow data for collecting to form
Arrange, analysis and client represent.This method have it is cheap, dispose and configure the characteristics of facilitating, be suitable for it is long-term,
Data acquisition and issuance under large traffic environment.
Because the standard of Flow technologies can not be completely unified, the technical staff of IETF is working out IPFIX (IP
Flow Information Export) specification so that the form of traffic statistics tends to standardization in network.IPFIX is based on
The NetFlow V9 designs of Cisco, are a kind of for data output, the form based on template, with very strong extensibility.
(4) information original flow analyzed by packet capturing and obtained
Packet capturing (packet capture), is most basic, is also the most frequently used network traffic information acquisition methods, extensive
Network traffic analysis and Network Safety Analysis field are applied to, nearly 90% gateway class safety means, are all to adopt packet capturing technology
And realize.
In the information-based department of power industry, in many network failures or security incident, technical staff can adopt and grab
The method of bag, to failure or accident depth analysis are carried out.
The most frequently used packet snapping method is the method analyzed by bypass flow, carries out real-time or offline data analysis.This
The mode of kind is known as network monitoring or Sniffer again.
Sniffer is divided to for two kinds of software and hardware, the Sniffer of software have Sniffer Pro, Network Monitor,
PacketBone etc., its advantage is to be easily installed deployment, it is easy to which study is used, while being also easy to exchange；Have the disadvantage to capture
All of transmission on network, in some cases also just cannot real awareness network failure and ruuning situation.The Sniffer of hardware
Commonly referred to protocol analyzer, typically all gyp, and price also costly, but can possess the link for supporting all kinds of extensions
Capture ability and high performance data capture in real time the function of analysis.
As above, collecting method conventional at this stage is simply described.Brief comparison will be done to these methods below：
1) merits and demerits of log approach：
Advantage：Obtain convenient, clear data
Shortcoming：
Daily record mostly is conclusive information, lacks procedural information, therefore, belong to Incomplete information.
Resource consumption cannot be avoided.After in order to avoid opening daily record, information system pressure itself is excessive, many information systems
And it is not turned on necessary journal function.
Isomerization outstanding problem.Because different business systems are developed by different developers, and industry is not believed daily record
The clear and definite and full and accurate specification of breath, this has resulted in the daily record of the operation system of different developers exploitation, in output format and interior
Rong Shang, there is very big difference.Very big technology barrier is caused to further analysis.
2) merits and demerits of SNMP methods：
Advantage：In being this several method, standard is most perfect, most popular method.In principle, almost all of network
Equipment, using component, can export SNMP information by configuring or installing Agent.
Shortcoming：Due to being with network element to manage object, so the output information of SNMP mostly is the shape of the network equipment or system
State amount, is combined not closely with business, and shortage " can flow " information type that feature is described.
And, lack the related information content of performance KQI, it is impossible to realize the judgement to network and application performance.Additionally,
On acquisition method, the method for poll also has many defects, just repeats no more here.
3) merits and demerits of Flow methods：
Advantage：Suitable for the technical standard of flow analysis, further, it is possible to the real time data for supporting big flow is exported.
Shortcoming：Lack the analysis ability to application layer message.
By taking the NetFlow of Cisco System Co. as an example, the most frequently used standard is V5 standards, but this standard is only focused in TCP layer
Information, without the ability that the upper layer information to OSI is analyzed, and V9 standards, although there is the analysis ability of application layer, but
Its version popularization is again very undesirable, and many network equipments do not support V9 standards.Therefore, using Flow technologies, can not be fine
Application layer message is analyzed.
4) merits and demerits of packet capturing analysis：
It is defined by real-time packet capturing analysis method.
Advantage：Method versatility is good, and flexibility ratio is high, and the depth and content of analysis can be with self-defined, it might even be possible to which analysis is negative
Carry the content of section.
Shortcoming：When in the face of big flow network environment, there is a big difference in performance for common packet capturing analysis method, it is impossible to
Meet the real-time analysis demand of user.
In the embodiment of the present invention, with reference to the current operation present situation of power system, information communication system runnability is investigated
Key point, Research Information Letter acquisition technique, and respective standard arrangement is carried out, data supporting is provided for intelligent analysis system,
Help improves the performance indications of equipment application.Research contents mainly includes the following aspects：Research Information Letter service data
Collection standard.The unified resource of Research Information Letter and operation fused data model.Research Information Letter service data from
Primary standard acquisition technique.
(1) the collection standard of Research Information Letter service data
By the analysis to current several data acquisition modes pluses and minuses, this project is mainly using bypass and the data of SNMP
Acquisition mode communicates intelligent management analysis work, so can both collect complete performance analysis data source, and
Data collecting system can be avoided to interfere to current network.The mode of Flow and agent, all to application system performance itself
Have a certain impact, all different degrees of occupancy system own resources, the mode for passing into others' hands is exactly in simple terms by using class
Like technologies such as tcpdump, the packet transmitted in network is completely intercepted and captured, the Internet letter of packet is extracted by analysis
Cease (source, destination address, source, destination interface) and application layer message (web protocol, database protocol etc.) to analyze network transmission matter
Amount and application system running quality, even with the parsing for specific protocol type, can analyze the information of transaction message,
As client is charged by intelligent electric meter, customer ID etc. can be resolved to.The mode of bypass can intercept and capture major part to intellectual analysis
The useful data source of system, but it there is also leak, because bypass analysis are taken from the mirror image flow of the network equipments such as switch,
So the problem of some equipment hardware performance itself, because it will not be transmitted in a network, so cannot be resolved to, now
Need to be analyzed with reference to SNMP information assists.
Data form after bypass analysis are disposed may be output as the SQL of standard, can so unify all data and adopt
The form of collection, realizes the standardization of data source, for the data source that later stage intelligent analysis system provides precise and high efficiency.
(2) the unified resource of Research Information Letter and operation fused data model.
It is long by the automatic normalization operation standard storehouse of the construction information communication resource, including network KPI (KPI Key Performance Indicator)
Phase runs baseline and sets up fortune using KQI (Key Quality Indicator) longtime running baselines and hardware resource longtime running baseline
Row fused data model, notices that such index is not respective individualism, and they are the relations for affecting one another, and network is unexpected
The Data Concurrent of big flow likely results in application system overload, slow so as to cause application system to access, by setting up fusion
Data model, can understand the index for confirming and seeing their this performance impact, for example be exactly to find network access quantity
With the equilibrium valve of server performance pressure, application server performance was so both can ensure that, large-scale investment can have been avoided again
Waste, in order to realize that this target needs to obtain data below output result：
Internet status monitoring：Network traffics size, packet rate, little packet rate, network transmission packet loss, bandwidth profit
With rate, real-time session quantity, application/communication disruption quantity etc..
Database performance is monitored：For distinct type data-base sentence delay of communication analysis, using time-delay analysis,
Byte count sizes etc.
Web performance monitorings：Web page loading time-delay analysis, delay of communication analysis, the return code accessed based on url
Statistics, loading type statistics (get or post), visited site statistics etc..
Performance of middle piece is monitored：For weblogic performance evaluations (the key technical indexes refers to web performance evaluations)
Transaction message is parsed：Such as xml agreements parsing.
Other status monitorings：For main frame (CPU, internal memory, network interface card, disk), operating system, disk and file system, etc.
Status monitoring.
(3) the self-developed standard acquisition technique of Research Information Letter service data.
It is the information communication service data self-collection method schematic diagram of the embodiment of the present invention with reference to Fig. 1.
Described information communication service data self-collection method, comprises the following steps：
The kernel that research aims at packet capturing and customizes, has reached the real-time Grasping skill of linear speed, and the reduction for maximizing degree is grabbed
Bag performance is lost.
Using multi-threaded business data analytics engine, the parallel processing capability of CPU groups is maximally utilized.
To avoid performance bottleneck from being designed with distributed deployment mode, managed concentratedly by the unified platform.
For ensure data analysis it is ageing, Project design adopt the real-time distributed processing mode of business datum, design by
The business datum that analytics engine is produced, spues in real time in memory database (mongodb) cluster, when the need for having big data quantity
Ask, can be different by processing data scale by increasing hardware server to build cluster, different scales cluster can be built.
Data spue to before data-base cluster, and in real time data can be done with classification pretreatment, the data meeting of same business
Arrange under a shard, and temporally longitudinal cutting, to improve treatment effeciency during computing.After data are stored in, temporally grain
Degree real-time running data library inquiry function, by the statistics for calculating relevant database is put into, for front end displaying.Simultaneously
Preprocessor calculate some for show KPI numerical value, in being put into key assignments type memory database, unified derivation after a period of time
To file persistent storage.
Multi-level buffer mode, reinforcing real-time performance are designed with, it is whole except the business datum of memory database is cached in real time
Each level of individual system also has the caching of oneself.
In the present embodiment, also including service data monitoring collection end to end, the monitoring of so-called service data end to end and
Collection is embodied in from user and initiates access request, to request through some network nodes, using component and server, storage, then
To the complete loops that user side is returned to using respond request.
Aforesaid this monitoring and collection are dynamic, are embodied in two aspects：
1. by lasting regular monitoring, Dynamic Discovery quality of service problem.Busy can be for example defined on to supervise per 5 minutes
Quality of service of control, once there occurs quality of service problem, control management platform can be perceived rapidly, so as to by troubleshooting
Time restriction reduces the impact of traffic failure in a very short time；
2. dynamically monitored by business.When finding that a certain quality of service of information communication system is not good, such as video council
View shake is severe, now can dynamically start the quality of service monitoring among network between two forward node and perceive, and reaches
To the purpose of rapid fault location.
This network end to end and service dynamic are perceived and can be divided into three kinds of granularities：
Granularity 1：The monitoring of network level.The network quality of the main monitoring TCP/IP Internets of network level monitoring and transport layer.
The monitoring of network level can be found that the basic forwarding problems such as bandwidth deficiency, packet loss, shake, time delay, is suitable for the bandwidth monitor of short-term
Monitor with long-term transfer quality；
Granularity 2：Service level is monitored.The flow of service level monitoring simulation actual services, main monitoring TCP/IP application layers
Network quality.The monitoring of this rank can be found that business personnel's application layer failure that network level can't find.It is this to monitor and true
Real service level monitors performance and disposal ability to monitoring system and all there is certain requirement；
Granularity 3：Actual services level is monitored.The network quality of direct monitoring actual services, and real business system in existing network
System merges completely, and this monitoring is best able to react the network quality problem of actual services, and the requirement highest to business needs identification
Actual services and it is marked or dyes.
Information communication system service data is gathered and monitored needs comprehensive these three different monitoring granularities, there is provided with network
Based on level monitoring, supplemented by service level monitoring, important business truly monitor the business monitoring service end to end of this fusion.
Those of ordinary skill in the art should be understood：The discussion of any of the above embodiment is exemplary only, not
It is intended to imply that the scope of the present disclosure (including claim) is limited to these examples；Under the thinking of the present invention, above example
Or can also be combined between the technical characteristic in different embodiments, step can be realized with random order, and be existed such as
Many other changes of upper described different aspect of the invention, for simple and clear their no offers in details.
In addition, to simplify explanation and discussing, and in order to obscure the invention, can in the accompanying drawing for being provided
It is connected with the known power ground of integrated circuit (IC) chip and other parts with illustrating or can not illustrate.Furthermore, it is possible to
Device is shown in block diagram form, to avoid obscuring the invention, and this have also contemplated that following facts, i.e., with regard to this
The details of the embodiment of a little block diagram arrangements be depend highly on the platform that will implement the present invention (that is, these details should
It is completely in the range of the understanding of those skilled in the art).Elaborating detail (for example, circuit) to describe the present invention's
In the case of exemplary embodiment, it will be apparent to those skilled in the art that can be without these details
In the case of or implement the present invention in the case that these details are changed.Therefore, these descriptions are considered as explanation
It is property rather than restricted.
Although invention has been described to have been incorporated with specific embodiment of the invention, according to retouching above
State, many replacements of these embodiments, modification and modification will be apparent for those of ordinary skills.Example
Such as, other memory architectures (for example, dynamic ram (DRAM)) can use discussed embodiment.
All such within the broad range that embodiments of the invention are intended to fall into claims replace,
Modification and modification.Therefore, all any omission, modification, equivalent, improvement within the spirit and principles in the present invention, made
Deng should be included within the scope of the present invention.
Claims (1)
1. a kind of information communicates service data self-collection method, it is characterised in that include：
The kernel that research aims at packet capturing and customizes, has reached the real-time Grasping skill of linear speed, maximizes the reduction packet capturing of degree
Can loss；
Using multi-threaded business data analytics engine, the parallel processing capability of CPU groups is maximally utilized；
Using distributed deployment mode, managed concentratedly by the unified platform；
The real-time distributed processing mode of business datum is designed with, the business datum produced by analytics engine is designed, is spued in real time
To in memory database cluster, when the demand for having big data quantity, by increasing hardware server to build cluster, by processing data
Scale is different, builds different scales cluster；
Data spue to before data-base cluster, and in real time data are done with classification pretreatment, and the data of same business can be arranged
Under one fragment, and temporally longitudinal cutting, to improve treatment effeciency during computing；After data are stored in, temporally granularity is real-time
Runtime database query function, by the statistics for calculating relevant database is put into, for front end displaying；Pre-process simultaneously
Program calculate some for show KPI Key Performance Indicator numerical value, in being put into key assignments type memory database, a period of time after unify
Export to file persistent storage；
It is designed with multi-level buffer mode, reinforcing real-time performance, except the business datum of memory database is cached in real time, whole system
Each level of system also has the caching of oneself.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611003412.6A CN106572190A (en) | 2016-11-15 | 2016-11-15 | Autonomous collection method for operational data of information communication |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611003412.6A CN106572190A (en) | 2016-11-15 | 2016-11-15 | Autonomous collection method for operational data of information communication |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106572190A true CN106572190A (en) | 2017-04-19 |
Family
ID=58542162
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611003412.6A Pending CN106572190A (en) | 2016-11-15 | 2016-11-15 | Autonomous collection method for operational data of information communication |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106572190A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108712306A (en) * | 2018-05-10 | 2018-10-26 | 国网宁夏电力有限公司信息通信公司 | A kind of information system automation inspection platform and method for inspecting |
| CN109491984A (en) * | 2018-10-09 | 2019-03-19 | 湖北省农村信用社联合社网络信息中心 | Hash packet data library fragment poll method for sorting |
| CN110046202A (en) * | 2019-03-07 | 2019-07-23 | 中国人民解放军海军工程大学 | The integrated power system real time data releasing method of key value database based on memory |
| CN111064575A (en) * | 2019-11-12 | 2020-04-24 | 卡斯柯信号(郑州)有限公司 | Method for analyzing network packet capturing applied to signal system of domestic password encryption |
| CN111178790A (en) * | 2020-02-28 | 2020-05-19 | 华兰生物工程重庆有限公司 | Electric power detection system |
| CN113014429A (en) * | 2021-02-24 | 2021-06-22 | 紫光云技术有限公司 | Link relation monitoring system |
-
2016
- 2016-11-15 CN CN201611003412.6A patent/CN106572190A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108712306A (en) * | 2018-05-10 | 2018-10-26 | 国网宁夏电力有限公司信息通信公司 | A kind of information system automation inspection platform and method for inspecting |
| CN109491984A (en) * | 2018-10-09 | 2019-03-19 | 湖北省农村信用社联合社网络信息中心 | Hash packet data library fragment poll method for sorting |
| CN110046202A (en) * | 2019-03-07 | 2019-07-23 | 中国人民解放军海军工程大学 | The integrated power system real time data releasing method of key value database based on memory |
| CN110046202B (en) * | 2019-03-07 | 2023-05-26 | 中国人民解放军海军工程大学 | Real-time data management method for integrated power system based on memory key value database |
| CN111064575A (en) * | 2019-11-12 | 2020-04-24 | 卡斯柯信号(郑州)有限公司 | Method for analyzing network packet capturing applied to signal system of domestic password encryption |
| CN111178790A (en) * | 2020-02-28 | 2020-05-19 | 华兰生物工程重庆有限公司 | Electric power detection system |
| CN113014429A (en) * | 2021-02-24 | 2021-06-22 | 紫光云技术有限公司 | Link relation monitoring system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106572190A (en) | Autonomous collection method for operational data of information communication | |
| US6321264B1 (en) | Network-performance statistics using end-node computer systems | |
| Lee et al. | Network monitoring: Present and future | |
| Yu | Network telemetry: towards a top-down approach | |
| EP2742646B1 (en) | A method, apparatus and communication network for root cause analysis | |
| US10296551B2 (en) | Analytics for a distributed network | |
| EP1367771B1 (en) | Passive network monitoring system | |
| CN107645398A (en) | A kind of method and apparatus of diagnostic network performance and failure | |
| CN110855493B (en) | Application topological graph drawing device for mixed environment | |
| CN101933290A (en) | Method for configuring acls on network device based on flow information | |
| Trammell et al. | mPlane: an intelligent measurement plane for the internet | |
| CN111726410B (en) | Programmable real-time computing and network load sensing method for decentralized computing network | |
| CN104618128A (en) | Multi-thread based node network detecting and analyzing method and system | |
| CN109547257A (en) | Method for controlling network flow, device, equipment, system and storage medium | |
| Duffield et al. | Trajectory engine: A backend for trajectory sampling | |
| Pekár et al. | Issues in the passive approach of network traffic monitoring | |
| CN111800311B (en) | Real-time sensing method for decentralized computing state | |
| Cisco | NetFlow Services Solutions Guide | |
| Dressler et al. | History-high speed network monitoring and analysis | |
| Botta et al. | A customer service assurance platform for mobile broadband networks | |
| Ranjitha et al. | A case for cross-domain observability to debug performance issues in microservices | |
| KR20080087197A (en) | Method and system for traffic analysis | |
| Ehrlich et al. | Passive flow monitoring of hybrid network connections regarding quality of service parameters for the industrial automation | |
| Subramoni et al. | Visualize and Analyze your Network Activities using OSU INAM | |
| CN115021974B (en) | Local area network safety probe equipment set |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170419 |
|
| WD01 | Invention patent application deemed withdrawn after publication |