CN102664769A - Acquisition and analysis system for network packets and realization method thereof - Google Patents
Acquisition and analysis system for network packets and realization method thereof Download PDFInfo
- Publication number
- CN102664769A CN102664769A CN2012101164866A CN201210116486A CN102664769A CN 102664769 A CN102664769 A CN 102664769A CN 2012101164866 A CN2012101164866 A CN 2012101164866A CN 201210116486 A CN201210116486 A CN 201210116486A CN 102664769 A CN102664769 A CN 102664769A
- Authority
- CN
- China
- Prior art keywords
- data
- acquisition
- network
- probe device
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses an acquisition and analysis system for network packets. The system comprises an acquisition control center device and at least one data acquisition probe device, wherein the acquisition control center device is connected with the data acquisition probe device through a network. The acquisition control center device is used for sending network packet data to the data acquisition probe device, acquiring filter scheme information, presetting acquisition contents, and sending acquisition control commands such as start, pause, resuming, and stop. The data acquisition probe device is used for receiving the acquisition control commands sent by the acquisition control center device, acquiring the network packets data and sending working state information of the data acquisition probe device to the acquisition control center device. Acquisition behavior and process of each data acquisition probe device are under centralized control of the acquisition control center device. The data acquisition probe device relies on a probe background service process and a data acquisition process so that operating of the data acquisition probe device one by one is avoided and operation control is simpler.
Description
Technical field
The present invention relates to the computer software engineering field, relate in particular to a kind of network package collection and resolution system and its implementation.
Background technology
The TCP/IP network package generally is made up of Ethernet stem, IP stem, TCP/UDP stem, application data and five parts of Ethernet afterbody; Wherein netting very much stem, IP stem, TCP/UDP stem and four parts of Ethernet afterbody has clear and definite and the set form definition; Common network data acquisition and analytical tool; Like network package protocal analysis softwares such as EtherPeek, Sniffr; Through analysis to these four parts, can obtain protocol type, the source/information such as purpose IP address of package, can be network traffics analysis and statistics, network performance analysis, network security and management etc. and provide support.The application data partial content is the key data source of reflection application system performance by user definition and tissue, is the main contents that network package is resolved.An application system often comprises many network package, and the formal definition of the application data part of these packages is not quite similar, and the form of the application data part in the different application systems network package differs widely especially.The network package analytical tool that is generally the application system service only is in order to satisfy the application system demand; The result data form is solidificated in the analytical tool; Be difficult to adapt to the demand that data format changes; Can't realize the network package parsing on the general meaning, also lack the versatility platform that integrates the network package data acquisition, resolves.
Summary of the invention
To the problems referred to above; The object of the present invention is to provide concentrated-distributed network package collection of a kind of good versatility and centralized network package to resolve system and its implementation, solved the problem that lacks the collection of versatility network package and resolve platform in one.
For achieving the above object, a kind of network package collection according to the invention and resolution system comprise an acquisition controlling center fixture and at least one data collecting probe device, wherein,
The acquisition controlling center fixture through network connection data acquisition probe device, sends network package data acquisition filtering scheme information, preset collection content, reaches the acquisition controlling instruction to this data collecting probe device;
The data collecting probe device receives the control information that the acquisition controlling center fixture sends, and the collection network packet data also sends its work state information to the acquisition controlling center fixture.
Preferably, said data collecting probe device is made up of probe network communication module, data acquisition module and data memory module, wherein,
The probe network communication module is realized the information exchange between data collecting probe device and the acquisition controlling center fixture;
Data acquisition module, the network package data qualified according to the filtering scheme information gathering, and the network package data that collect are carried out the memory queue buffer memory;
Data memory module, with the network package storage of memory queue buffer memory in the node disk file.
Said acquisition controlling center fixture is made up of central site network communication module, acquisition control module, data download module, data resolution module and result formats editor module, wherein,
The central site network communication module realizes the information exchange between acquisition controlling center fixture and the data collecting probe device;
Acquisition control module, the gatherer process of monitoring data collection probe apparatus;
Data download module downloads to above-mentioned node disk file on the node disk on the acquisition controlling center fixture;
The result formats editor module, the form that preset network package data parsing is appeared, and with form preservation as a result;
Data resolution module is resolved the network package data according to preset analytical form.
For achieving the above object, a kind of network package collection according to the invention and parsing implementation method may further comprise the steps:
1), starts probe backstage service processes at the acquisition node deploy data collecting probe device of computer network;
2) at the Control Node deploy acquisition controlling center fixture of computer network;
3) start the acquisition controlling center fixture, realization is connected with each data collecting probe device;
4) dispose the data acquisition filtering scheme information of each data collecting probe device, and send the data collecting probe device to;
5) log-on data acquisition probe device on the acquisition controlling center fixture begins collection, storage networking packet data;
6) on the acquisition controlling center fixture, stop the data collecting probe device, finish the collection of network package data;
The data file of 7) at the acquisition controlling center fixture each data collecting probe device being gathered, preserving downloads to the acquisition controlling center fixture;
8) on the acquisition controlling center fixture, open the network package data file that to resolve, it is resolved to the form as a result of user's appointment.
Beneficial effect of the present invention is:
1, the collection behavior of each data collecting probe device and process are by acquisition controlling center fixture centralized control; Simultaneously; The data collecting probe device is realized by probe backstage service processes and data acquisition process; Probe backstage service processes is gathered process according to the direct control data of the order of acquisition controlling center fixture, and this mode has been avoided the troublesome operation of operating data acquisition probe device (as starting, stopping) one by one, makes operation control more succinct;
2, data acquisition thread and storage thread are separated the collection and the storing process of network package data; Solved the contradiction between high-speed data acquisition and the low speed magnetic disc i/o; Guaranteed the high speed property of data acquisition, reduced because of the data loss rate of gathering, storage speed does not match and causes;
3, the user can be at any time according to the application system demand, and the formal definition of change analysis result makes the network package collection satisfy different application systems with the parsing platform and resolves requirement, has good versatility;
4, the formal definition content is preserved with the XML form as a result, has very strong versatility and editability.
Description of drawings
Fig. 1 is the structural representation of said network package collection of the embodiment of the invention and resolution system;
Fig. 2 is the deployment sketch map of said network package collection of the embodiment of the invention and resolution system;
Fig. 3 is an acquisition controlling center fixture runnable interface sketch map;
Fig. 4 is system information flow process figure.
Embodiment
Below in conjunction with Figure of description the present invention is done further description.
As shown in Figure 1, said a kind of network package collection of the embodiment of the invention and resolution system comprise an acquisition controlling center fixture and at least one data collecting probe device, wherein,
The acquisition controlling center fixture; Through network connection data acquisition probe device, send network package data acquisition filtering scheme information, preset collection content, reach and send the acquisition controlling instruction that starts, suspends, recovers, stops gatherer process to this data collecting probe device;
The data collecting probe device receives the control information that the acquisition controlling center fixture sends, and the collection network packet data also sends its work state information to the acquisition controlling center fixture.
Following mask body describes acquisition controlling center fixture and data collecting probe device.
At first, said data collecting probe device is made up of probe network communication module, data acquisition module and data memory module, wherein,
The probe network communication module, the ICP/IP protocol of employing computer network is realized the information exchange between data collecting probe device and the acquisition controlling center fixture;
Data acquisition module is responsible on the computer node network layer, gathering qualified network package data according to filtering scheme information, and the network package data that collect is carried out the memory queue buffer memory, to realize the collection of network package data in high speed, reduces packet loss;
Data memory module is responsible for network package storage with the memory queue buffer memory in the node disk file, and the phenomenon that causes computer node decreased performance and loss of data to avoid expending memory source has in a large number guaranteed the complete preservation to institute's image data.
Once more, said acquisition controlling center fixture is made up of central site network communication module, acquisition control module, data download module, data resolution module and result formats editor module, wherein,
The central site network communication module, the ICP/IP protocol of employing computer network is realized the information exchange between acquisition controlling center fixture and the data collecting probe device;
Acquisition control module, the gatherer process of monitoring data collection probe apparatus, such as: the position, work at present state, the gatherer process that show probe apparatus;
Data download module downloads to above-mentioned node disk file on the node disk on the acquisition controlling center fixture;
The result formats editor module, the form that preset network package data parsing is appeared, and form as a result preserved with the XML form;
Data resolution module is resolved the network package data according to preset analytical form, and display result.
A kind of network package collection and parsing implementation method may further comprise the steps:
1), starts probe backstage service processes at the acquisition node deploy data collecting probe device of computer network;
2) at the Control Node deploy acquisition controlling center fixture of computer network;
3) start the acquisition controlling center fixture, realization is connected with each data collecting probe device;
4) dispose the data acquisition filtering scheme information of each data collecting probe device, and send the data collecting probe device to;
5) log-on data acquisition probe device on the acquisition controlling center fixture begins collection, storage networking packet data;
6) on the acquisition controlling center fixture, stop the data collecting probe device, finish the collection of network package data;
The data file of 7) at the acquisition controlling center fixture each data collecting probe device being gathered, preserving downloads to the acquisition controlling center fixture;
8) on the acquisition controlling center fixture, open the network package data file that to resolve, it is resolved to the form as a result of user's appointment.
Specifically, network communication module, data acquisition module and data memory module are realized by two independent processes: network communication module realizes that by said probe backstage service processes data acquisition module and data memory module include in the data acquisition process.Probe backstage service processes receives the control command that the acquisition controlling center fixture sends through the network ICP/IP protocol, and controls the data acquisition of said data acquisition process in view of the above through pipe technology.Data acquisition module and data memory module are realized relatively independent data acquisition thread, storage thread respectively.In memory array, and the network package data that will be saved to the node disk file are passed to the storage thread to the data acquisition thread with the network package metadata cache of being gathered.
The acquisition controlling center fixture shows the data collecting probe device of disposing in the computer network with the tabulation mode; Each row is represented a data acquisition probe device, and displaying contents has: probe title, operating state, host's host name, host's host IP address, communication port numbers, image data amount and filtering scheme information.
Be illustrated in figure 2 as the deployment sketch map of network package collection and resolution system, guarantee that the probe backstage service processes (backstage service .exe) in the data collecting probe device is in running status.Shown in Figure 3 is acquisition controlling center fixture runnable interface sketch map, provides the information such as title, position and operating state of the acquisition probe device of having disposed in the computer network in the interface.Be illustrated in figure 4 as system information flow process figure.Can see contact and the brief overview between the course of work between above-mentioned each module in the drawings, make the work purpose of system and process come into plain view.
Specifically introduce its use according to this flow process below:
On the acquisition controlling center fixture; Dispose the acquisition filter scheme information of each data collecting probe device; Comprise and intend the employed agreement of collection network package, its source IP address scope etc., completion is set after, through computer network communication filtering scheme information is passed to each data collecting probe device.
On the acquisition controlling center fixture, start gatherer process, after probe backstage service processes was received this order, log-on data was gathered process (probe .exe), begins to gather its host's meshed network packet data, and image data is deposited in host's node disk file.In the gatherer process, can on the acquisition controlling center fixture, suspend gatherer process as required, at this moment, probe backstage service processes notification data collection process stops the network package data acquisition; Can assign the recovery acquisition through the acquisition controlling center fixture after suspending gatherer process, receive this order after, the data acquisition process begins to continue the collection network packet data from current time.Suspend or recover gatherer process, do not influence the process that collection network package writes disk file.When the metadata cache formation had data, the data acquisition process deposited data in disk file automatically, and when metadata cache formation free of data, the data acquisition process stops to write the disk file process automatically.
After acquisition tasks is accomplished; Can be on the acquisition controlling center fixture; Stop gatherer process, after probe backstage service processes is received this order, notification data collection process is finished gatherer process; The data acquisition process stops network package collection action immediately, and continues the data in the metadata cache formation are write disk file.After all data all were written into disk file in the metadata cache formation, the data acquisition process stopped automatically.
The acquisition controlling center fixture downloads to this locality with the network package data file in each node.The user is according to the application system demand, newly-built or edited result data format, and save as formatted file as a result.
The acquisition controlling center fixture is according to form document definition as a result, the network package data file of appointment resolved to structurized, concrete parameter, for use in correlation analysis or other application of application system.Wherein, network package analysis result formal definition comprises like the lower part: major key, field name, Field ID, field type, field length, coded system, Major key, repeat indication and transform expression formula.
More than; Be merely preferred embodiment of the present invention, but protection scope of the present invention is not limited thereto, any technical staff who is familiar with the present technique field is in the technical scope that the present invention discloses; The variation that can expect easily or replacement all should be encompassed within protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection range that claim was defined.
Claims (9)
1. network package collection and resolution system is characterized in that, comprise an acquisition controlling center fixture and at least one data collecting probe device, wherein,
The acquisition controlling center fixture through network connection data acquisition probe device, sends network package data acquisition filtering scheme information, preset collection content, reaches the acquisition controlling instruction to this data collecting probe device;
The data collecting probe device receives the control information that the acquisition controlling center fixture sends, and the collection network packet data also sends its work state information to the acquisition controlling center fixture.
2. network package collection according to claim 1 and resolution system is characterized in that, said data collecting probe device is made up of probe network communication module, data acquisition module and data memory module, wherein,
The probe network communication module is realized the information exchange between data collecting probe device and the acquisition controlling center fixture;
Data acquisition module, the network package data qualified according to the filtering scheme information gathering, and the network package data that collect are carried out the memory queue buffer memory;
Data memory module, with the network package storage of memory queue buffer memory in the node disk file.
Said acquisition controlling center fixture is made up of central site network communication module, acquisition control module, data download module, data resolution module and result formats editor module, wherein,
The central site network communication module realizes the information exchange between acquisition controlling center fixture and the data collecting probe device;
Acquisition control module, the gatherer process of monitoring data collection probe apparatus;
Data download module downloads to above-mentioned node disk file on the node disk on the acquisition controlling center fixture;
The result formats editor module, the form that preset network package data parsing is appeared, and with form preservation as a result;
Data resolution module is resolved the network package data according to preset analytical form.
A network package collection with resolve implementation method, it is characterized in that, may further comprise the steps:
1), starts probe backstage service processes at the acquisition node deploy data collecting probe device of computer network;
2) at the Control Node deploy acquisition controlling center fixture of computer network;
3) start the acquisition controlling center fixture, realization is connected with each data collecting probe device;
4) dispose the data acquisition filtering scheme information of each data collecting probe device, and send the data collecting probe device to;
5) log-on data acquisition probe device on the acquisition controlling center fixture begins collection, storage networking packet data;
6) on the acquisition controlling center fixture, stop the data collecting probe device, finish the collection of network package data;
The data file of 7) at the acquisition controlling center fixture each data collecting probe device being gathered, preserving downloads to the acquisition controlling center fixture;
8) on the acquisition controlling center fixture, open the network package data file that to resolve, it is resolved to the form as a result of user's appointment.
4. network package collection according to claim 3 and parsing implementation method; It is characterized in that; Network communication module, data acquisition module and data memory module are realized by two independent processes: network communication module realizes that by said probe backstage service processes data acquisition module and data memory module include in the data acquisition process.
5. network package collection according to claim 4 and parsing implementation method; It is characterized in that; Probe backstage service processes receives the control command that the acquisition controlling center fixture sends through the network ICP/IP protocol, and controls the data acquisition of said data acquisition process in view of the above through pipe technology.
6. network package collection according to claim 5 and parsing implementation method is characterized in that data acquisition module and data memory module are realized relatively independent data acquisition thread, storage thread respectively.
7. network package collection according to claim 6 and parsing implementation method; It is characterized in that; In memory array, and the network package data that will be saved to the node disk file are passed to the storage thread to the data acquisition thread with the network package metadata cache of being gathered.
8. network package collection according to claim 3 and parsing implementation method; It is characterized in that; The acquisition controlling center fixture shows the data collecting probe device of disposing in the computer network with the tabulation mode; Each row is represented a data acquisition probe device, and displaying contents has: probe title, operating state, host's host name, host's host IP address, communication port numbers, image data amount and filtering scheme information.
9. the network package collection and parsing implementation trifling according to claim 3; It is characterized in that network package analysis result formal definition comprises like the lower part: major key, field name, Field ID, field type, field length, coded system, Major key, repeat indication and transform expression formula.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012101164866A CN102664769A (en) | 2012-04-19 | 2012-04-19 | Acquisition and analysis system for network packets and realization method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012101164866A CN102664769A (en) | 2012-04-19 | 2012-04-19 | Acquisition and analysis system for network packets and realization method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102664769A true CN102664769A (en) | 2012-09-12 |
Family
ID=46774188
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012101164866A Pending CN102664769A (en) | 2012-04-19 | 2012-04-19 | Acquisition and analysis system for network packets and realization method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102664769A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105718295A (en) * | 2016-01-27 | 2016-06-29 | 四川长虹电器股份有限公司 | Data collecting and analyzing method and system |
| CN106254172A (en) * | 2016-07-14 | 2016-12-21 | 东软集团股份有限公司 | Heterogeneous applications collecting method and device |
| CN107167839A (en) * | 2017-06-05 | 2017-09-15 | 王伟巍 | A kind of IMF OBS multi-channel data acquisition application systems |
| CN112947115A (en) * | 2021-02-05 | 2021-06-11 | 西安羚控电子科技有限公司 | Generalized data monitoring method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101286895A (en) * | 2008-05-22 | 2008-10-15 | 上海交通大学 | Dynamic configurable data monitoring system and method for distributed network |
| CN101514627A (en) * | 2008-03-05 | 2009-08-26 | 中国科学院自动化研究所 | Remote monitoring system for oil well pressure by capillary steel pipes |
| CN201345048Y (en) * | 2008-12-26 | 2009-11-11 | 中国铁路通信信号上海工程有限公司 | Data acquisition and analysis system |
| CN101619989A (en) * | 2008-07-04 | 2010-01-06 | 中国铁路通信信号上海工程有限公司 | System and method for acquiring and analyzing remote data of bridge |
-
2012
- 2012-04-19 CN CN2012101164866A patent/CN102664769A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101514627A (en) * | 2008-03-05 | 2009-08-26 | 中国科学院自动化研究所 | Remote monitoring system for oil well pressure by capillary steel pipes |
| CN101286895A (en) * | 2008-05-22 | 2008-10-15 | 上海交通大学 | Dynamic configurable data monitoring system and method for distributed network |
| CN101619989A (en) * | 2008-07-04 | 2010-01-06 | 中国铁路通信信号上海工程有限公司 | System and method for acquiring and analyzing remote data of bridge |
| CN201345048Y (en) * | 2008-12-26 | 2009-11-11 | 中国铁路通信信号上海工程有限公司 | Data acquisition and analysis system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105718295A (en) * | 2016-01-27 | 2016-06-29 | 四川长虹电器股份有限公司 | Data collecting and analyzing method and system |
| CN106254172A (en) * | 2016-07-14 | 2016-12-21 | 东软集团股份有限公司 | Heterogeneous applications collecting method and device |
| CN107167839A (en) * | 2017-06-05 | 2017-09-15 | 王伟巍 | A kind of IMF OBS multi-channel data acquisition application systems |
| CN112947115A (en) * | 2021-02-05 | 2021-06-11 | 西安羚控电子科技有限公司 | Generalized data monitoring method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110943870B (en) | Intelligent station panoramic data monitoring and analyzing system and method | |
| EP1921527B1 (en) | Adjustable data collection rate for embedded historians | |
| CN101848214B (en) | Random positioning playback method and system based on RDP (remote desktop protocol) audit data | |
| CN102664769A (en) | Acquisition and analysis system for network packets and realization method thereof | |
| CN103635881B (en) | The management method and terminal of application program | |
| CN111106955B (en) | Intelligent station communication gateway machine and communication method | |
| WO2007070803A3 (en) | System and method for web-based control of remotely located devices using ready on command architecture | |
| CN111918230B (en) | Data acquisition method, data transmission method, gateway, equipment and storage medium | |
| CN103617098A (en) | Intelligent backup method and system based on data changes | |
| CN101018150A (en) | A collection method and system of the telecom device performance data | |
| CN107124344A (en) | Train is changed and data storage control method with CAN ethernet communications | |
| CN111966465B (en) | Method, system, equipment and medium for modifying host configuration parameters in real time | |
| US7840725B2 (en) | Capture of data in a computer network | |
| CN106534272A (en) | System and method for processing parameters of coal-fired unit | |
| CN104320301A (en) | Intranet special line flow monitoring method and system | |
| CN106789191A (en) | A kind of automatic method for restarting of distributed deployment service processes and device | |
| CN106292518A (en) | Remote PLC monitoring and debugging system and method | |
| CN108052385A (en) | A kind of Container Management method, system, equipment and computer storage media | |
| CN109412939B (en) | Communication gateway for recording industrial network communication period process data and working method | |
| CN102289368A (en) | Method and system for obtaining serial printing information | |
| CN113423025B (en) | Data management terminal with artificial intelligence | |
| CN115086104A (en) | Method for supporting data disconnection retransmission and serial server | |
| CN104317747B (en) | A kind of data buffer storage of grid receiver and dispensing device and method | |
| CN111127250A (en) | Electric power data monitoring event analysis system and method | |
| CN101866296A (en) | Windows file system based installation package silencing method and tool |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20120912 |