CN102546392B - System and method used for sampling network messages and based on transmission control protocol (tcp) connection - Google Patents
System and method used for sampling network messages and based on transmission control protocol (tcp) connection Download PDFInfo
- Publication number
- CN102546392B CN102546392B CN201110383695.2A CN201110383695A CN102546392B CN 102546392 B CN102546392 B CN 102546392B CN 201110383695 A CN201110383695 A CN 201110383695A CN 102546392 B CN102546392 B CN 102546392B
- Authority
- CN
- China
- Prior art keywords
- tcp
- message
- sampling
- judge module
- management table
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a system used for sampling network messages and based on transmission control protocol (tcp) connection, which comprises a tcp message judging module, a tcp identification judging module, a sampling proportion judging module, a tcp searching module connected with the tcp identification judging module and a tcp connection management table respectively connected with the tcp searching module and the sampling proportion judging module. The invention further provides a method used for sampling network messages and based on the tcp connection. The tcp connection management table is built in the message sampling system and used for managing processes of building, transmitting and closing the tcp connection. The system and the method used for sampling network messages and based on the tcp connection can sample messages continuously and improve effectiveness of sampled data.
Description
Technical field
The invention belongs to network data processing field, specifically relate to a kind of network message sampling system and method connecting based on tcp.
Background technology
Current network data processing system need to carry out reception & disposal to the message on network, in the situation that network traffics are very large, for reducing system load, the general method that adopts packet sampling, that is to say, system does not receive all messages, but extract a certain amount of message from all messages, analyzes.
Packet sampling is often based on message, and by the method for accounting message counting, according to certain sampling ratio, stochastical sampling message, analyzes such as extract a message from every 10 messages.But in actual applications, a lot of network data analysis work is for single message, but connect for tcp.
The patent No. is: the invention that ZL200910073048.4, name are called " a kind of self adaptive network traffic sampling method for abnormality detection " has disclosed a kind of self adaptive network traffic sampling method, utilize the method for accounting message counting to sample to the data packet messages of transmission, but like this message is sampled and tended to lose some message in tcp connection, the data that tcp is connected are imperfect, cause existing when analytical sampling data the problem that sampled data validity is not high.
Summary of the invention
The object of the invention is to, propose a kind of network message sampling system and method connecting based on tcp, can carry out continuous acquisition to message, improve the validity of sampled data.
For achieving the above object, the invention provides a kind of network message sampling system connecting based on tcp, described packet sampling system comprises: tcp message judge module, tcp sign judge module and sampling ratio judge module; It is characterized in that, the tcp being connected with described tcp sign judge module searches module, and described tcp searches module and is connected with tcp admin table respectively with described sampling ratio judge module; Described tcp message judge module, described tcp sign judge module are connected successively with described sampling ratio judge module.
In optimal technical scheme provided by the invention, the four-tuple structure that each list item in described tcp connection management table connects for sign tcp.
In the second optimal technical scheme provided by the invention, whether described tcp message judge module is that tcp message judges to the message of input, determine it is in sampling ratio stochastical sampling, continue to process next incoming message and still this message transmissions is arrived to tcp sign judge module.
In the 3rd optimal technical scheme provided by the invention, the tcp sign of described tcp sign judge module judgement message, determines described message transmissions is still searched to module by described tcp message transmissions to described tcp to described sampling ratio judge module.
In the 4th optimal technical scheme provided by the invention, described sampling ratio judge module judges whether the data volume of current sampling has reached the sampling ratio of systemic presupposition, judgement is to abandon the message of receiving, continue to process next incoming message, or the message received of sampling, and the four-tuple structure of the message of receiving described in extracting being transferred in described tcp connection management table.
In the 5th optimal technical scheme provided by the invention, described four-tuple structure comprises source ip, object ip, source port and destination interface.
In the 6th optimal technical scheme provided by the invention, described tcp searches module after receiving the tcp message that described tcp sign judge module sends, extract the four-tuple structure of this tcp message, and search whether there be the list item identical with the tcp message of described extraction in described tcp connection management table.
In the 7th optimal technical scheme provided by the invention, provide a kind of network message method of sampling connecting based on tcp, its improvements are, the described network message method of sampling comprises the steps:
(1) in packet sampling system, set up a tcp connection management table;
(2) type of judgement incoming message, if this type of message is TCP message, checks the tcp flag of this message; Otherwise in sampling ratio stochastical sampling, then continue to process next incoming message;
(3) the tcp flag of judgement message, if this tcp message belongs to, newly-built tcp connects, and carry out step 4; If this tcp message belongs to the data message that built tcp connects, carry out step 5; If this tcp message belongs to the message of closing of tcp connection, carry out step 6;
(4) judge whether the data volume of current sampling has reached the sampling ratio of systemic presupposition, if reached sampling ratio, abandons the message of receiving, continue to process next incoming message; Otherwise the message that sampling is received, and the four-tuple structure of the message of receiving described in extracting, be inserted in described tcp connection management table;
(5) extract the four-tuple structure of tcp message, in described tcp connection management table, search whether there be the list item identical with the tcp message of described extraction, if found, this tcp message of sampling; Otherwise abandon this tcp message, continue to process next incoming message;
(6) extract the four-tuple structure of tcp message, in described tcp connection management table, search whether there be the list item identical with the tcp message of described extraction; If found, this tcp message of sampling, and from described tcp connection management table, delete this tcp connection; Otherwise abandon this tcp message, continue to process next incoming message.
Compared with the prior art, a kind of network message sampling system and method connecting based on tcp provided by the invention, can in packet sampling system, set up a tcp connection management table, management tcp connects the process of setting up, transmitting, close, and realizes the packet sampling connecting based on tcp by tcp connection management table; When a new tcp connection is found, if need sampling, just it to be managed in tcp connection table, messages all in whole transmitting procedure are all sampled, until connection closed.If a newfound tcp connects, do not need sampling, just in tcp connection management table, do not set up the option that this tcp connects, follow-up message all abandons; And, avoided existing packet sampling system when message is sampled, to tend to lose some message in tcp connection, the data that tcp is connected are imperfect, cause existing when analytical sampling data the problem that sampled data validity is not high.
Accompanying drawing explanation
Fig. 1 is the structural representation of network message sampling system.
Fig. 2 is the flow chart of the network message method of sampling.
Embodiment
As shown in Figure 1, the network message sampling system connecting based on tcp, comprise: tcp message judge module, tcp sign judge module and sampling ratio judge module, the tcp being connected with described tcp sign judge module searches module, and searches with described tcp the tcp connection management table that module is connected with described sampling ratio judge module respectively; Described tcp message judge module, described tcp sign judge module are connected successively with described sampling ratio judge module.
Each list item in described tcp connection management table is the four-tuple structure that sign tcp connects; Described four-tuple structure comprises source ip, object ip, source port and destination interface.Whether described tcp message judge module is that tcp message judges to the message of input, judgment result is that "No", in sampling ratio stochastical sampling, then continues to process next incoming message; Judgment result is that "Yes", this message transmissions is identified to judge module to tcp.Described tcp sign judge module can judge the sign of tcp message, if tcp sign shows that described tcp message belongs to newly-built tcp and connects, described tcp message transmissions is arrived to described sampling ratio judge module; If tcp sign shows described tcp message and belongs to the message of closing that data message that built tcp connects or tcp connect, described tcp message transmissions searched to module to described tcp.Described sampling ratio judge module judges whether the data volume of current sampling has reached the sampling ratio of systemic presupposition, judgment result is that "Yes", abandons the message of receiving, continues to process next incoming message; Judgment result is that "No", the message received of sampling, and the four-tuple structure of the message of receiving described in extracting is also transferred in described tcp connection management table.Described tcp searches module after receiving the tcp message that described tcp sign judge module sends, and extracts the four-tuple structure of this tcp message, and search whether there be the list item identical with the tcp message of described extraction in described tcp connection management table.
As shown in Figure 2, for a kind of network message method of sampling connecting based on tcp, comprise the steps:
(1) in packet sampling system, set up a tcp connection management table;
(2) to the message of input, whether be that tcp message judges, judgment result is that "No", in sampling ratio stochastical sampling, then continue to process next incoming message; Judgment result is that "Yes", check the tcp flag of message;
(3) the tcp flag of message is judged, this tcp message of judgment result displays belongs to that newly-built tcp connects, and carry out step 4; This tcp message of judgment result displays belongs to the data message that built tcp connects, and carry out step 5; This tcp message of judgment result displays belongs to the message of closing of tcp connection, carry out step 6;
(4) judge whether the data volume of current sampling has reached the sampling ratio of systemic presupposition, judgment result is that "Yes", abandons the message of receiving, continue to process next incoming message; Judgment result is that "No", the message that sampling is received, and the four-tuple structure of the message of receiving described in extracting, be inserted in described tcp connection management table;
(5) extract the four-tuple structure of tcp message, in described tcp connection management table, search whether there be the list item identical with the tcp message of described extraction, lookup result is "Yes", this tcp message of sampling; Lookup result is "No", abandons this tcp message, continues to process next incoming message;
(6) extract the four-tuple structure of tcp message, in described tcp connection management table, search whether there be the list item identical with the tcp message of described extraction, lookup result is "Yes", this tcp message of sampling, and from described tcp connection management table, delete this tcp connection; Lookup result is "No", abandons this tcp message, continues to process next incoming message.
Need statement, content of the present invention and embodiment are intended to prove the practical application of technical scheme provided by the present invention, should not be construed as limiting the scope of the present invention.Those skilled in the art inspired by the spirit and principles of the present invention, can do various modifications, be equal to and replace or improve.But in the protection range that these changes or modification are all awaited the reply in application.
Claims (2)
1. the network message sampling system connecting based on tcp, described packet sampling system comprises: tcp message judge module, tcp sign judge module and sampling ratio judge module; It is characterized in that, the tcp being connected with described tcp sign judge module searches module, and described tcp searches module and is connected with tcp connection management table respectively with described sampling ratio judge module; Described tcp message judge module, described tcp sign judge module are connected successively with described sampling ratio judge module;
The four-tuple structure that each list item in described tcp connection management table connects for sign tcp;
Whether described tcp message judge module is that tcp message judges to the message of input, and definite is in sampling ratio stochastical sampling, continues to process next incoming message and still this message transmissions is identified to judge module to tcp;
The tcp sign of described tcp sign judge module judgement message, determines described tcp message transmissions is still searched to module by described tcp message transmissions to described tcp to described sampling ratio judge module;
Described sampling ratio judge module judges whether the data volume of current sampling has reached the sampling ratio of systemic presupposition, judgement is to abandon the message of receiving, continue to process next incoming message, or the message received of sampling, and the four-tuple structure of the message of receiving described in extracting being transferred in described tcp connection management table;
Described tcp searches module after receiving the tcp message that described tcp sign judge module sends, and extracts the four-tuple structure of this tcp message, and search whether there be the list item identical with the tcp message of described extraction in described tcp connection management table;
The network message method of sampling connecting based on tcp of the described network message sampling system connecting based on tcp, is characterized in that, the described network message method of sampling comprises the steps:
(1) in packet sampling system, set up a tcp connection management table;
(2) type of judgement incoming message, if this type of message is TCP message, checks the tcp flag of this message; Otherwise in sampling ratio stochastical sampling, then continue to process next incoming message;
(3) the tcp flag of judgement message, if this tcp message belongs to, newly-built tcp connects, and carries out step (4); If this tcp message belongs to the data message that built tcp connects, carry out step (5); If this tcp message belongs to the message of closing of tcp connection, carry out step (6);
(4) judge whether the data volume of current sampling has reached the sampling ratio of systemic presupposition, if reached sampling ratio, abandons the message of receiving, continue to process next incoming message; Otherwise the message that sampling is received, and the four-tuple structure of the message of receiving described in extracting, be inserted in described tcp connection management table;
(5) extract the four-tuple structure of tcp message, in described tcp connection management table, search whether there be the list item identical with the tcp message of described extraction, if found, this tcp message of sampling; Otherwise abandon this tcp message, continue to process next incoming message;
(6) extract the four-tuple structure of tcp message, in described tcp connection management table, search whether there be the list item identical with the tcp message of described extraction; If found, this tcp message of sampling, and from described tcp connection management table, delete this tcp connection; Otherwise abandon this tcp message, continue to process next incoming message.
2. packet sampling system according to claim 1, is characterized in that, described four-tuple structure comprises source ip, object ip, source port and destination interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110383695.2A CN102546392B (en) | 2011-11-28 | 2011-11-28 | System and method used for sampling network messages and based on transmission control protocol (tcp) connection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110383695.2A CN102546392B (en) | 2011-11-28 | 2011-11-28 | System and method used for sampling network messages and based on transmission control protocol (tcp) connection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102546392A CN102546392A (en) | 2012-07-04 |
| CN102546392B true CN102546392B (en) | 2014-08-27 |
Family
ID=46352381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110383695.2A Active CN102546392B (en) | 2011-11-28 | 2011-11-28 | System and method used for sampling network messages and based on transmission control protocol (tcp) connection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102546392B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1897541A (en) * | 2005-07-15 | 2007-01-17 | 华为技术有限公司 | Method for realizing network sampling |
| CN102130789A (en) * | 2011-04-15 | 2011-07-20 | 北京网御星云信息技术有限公司 | Method, device and system for measuring and sampling streams based on application groups |
| CN102143085A (en) * | 2011-04-27 | 2011-08-03 | 北京网御星云信息技术有限公司 | Multi-dimensional network situation awareness method, equipment and system |
-
2011
- 2011-11-28 CN CN201110383695.2A patent/CN102546392B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1897541A (en) * | 2005-07-15 | 2007-01-17 | 华为技术有限公司 | Method for realizing network sampling |
| CN102130789A (en) * | 2011-04-15 | 2011-07-20 | 北京网御星云信息技术有限公司 | Method, device and system for measuring and sampling streams based on application groups |
| CN102143085A (en) * | 2011-04-27 | 2011-08-03 | 北京网御星云信息技术有限公司 | Multi-dimensional network situation awareness method, equipment and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102546392A (en) | 2012-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101741862B (en) | System and method for detecting IRC bot network based on data packet sequence characteristics | |
| CN101388763B (en) | SQL injection attack detection system supporting multiple database types | |
| EP3651438B1 (en) | Data transmission based on application- and protocol-adaptive compression strategies | |
| WO2016107210A1 (en) | Redundant industrial ethernet system with multistage packet filtering and service classification control | |
| CN103259728A (en) | OFS in-band communication method and OFS | |
| CN101119246B (en) | Data packet sampling statistic method and apparatus | |
| CN104702584A (en) | Modbus communication access control method based on rule self-learning | |
| CN106506347B (en) | A kind of multi-protocol data communication gate equipment for air traffic control system | |
| US20120173712A1 (en) | Method and device for identifying p2p application connections | |
| CN104915348A (en) | Method and system for publishing air quality in real time | |
| CN102497297A (en) | System and method for realizing deep packet inspection technology based on multi-core and multi-thread | |
| CN110868404A (en) | Industrial control equipment automatic identification method based on TCP/IP fingerprint | |
| CN102546392B (en) | System and method used for sampling network messages and based on transmission control protocol (tcp) connection | |
| CN107733716A (en) | Distributed file system log analysis method, system, equipment and storage medium | |
| CN106657145A (en) | Method for automatically finding database based on communication protocol and SQL grammar | |
| CN101026502B (en) | Broad band network comprehensive performance management flatform | |
| CN103001966A (en) | Processing and identifying method and device for private network IP | |
| CN104125106A (en) | Network purity detection device and method based on classified decision tree | |
| CN109274551A (en) | A kind of accurate efficient industry control resource location method | |
| CN105828306A (en) | Junk short message detecting method and device | |
| CN109121109A (en) | A kind of overhead transmission line connector temperature detection method and system based on microgrid ad hoc network | |
| CN103702301A (en) | Real-time sensing control system for inter-internet short message service | |
| CN201725421U (en) | Complex data test and control terminal for oil field based on technology of Internet of things | |
| CN107196931A (en) | A kind of deep message detection method based on network isolating device | |
| CN102006289A (en) | Spoofed source address filtering method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220725 Address after: 100193 No. 36 Building, No. 8 Hospital, Wangxi Road, Haidian District, Beijing Patentee after: Dawning Information Industry (Beijing) Co.,Ltd. Patentee after: DAWNING INFORMATION INDUSTRY Co.,Ltd. Address before: 100084 Beijing Haidian District City Mill Street No. 64 Patentee before: Dawning Information Industry (Beijing) Co.,Ltd. |
|
| TR01 | Transfer of patent right |