CN106209698B - Protocol assembly method based on Libnids - Google Patents
Protocol assembly method based on Libnids Download PDFInfo
- Publication number
- CN106209698B CN106209698B CN201610532683.4A CN201610532683A CN106209698B CN 106209698 B CN106209698 B CN 106209698B CN 201610532683 A CN201610532683 A CN 201610532683A CN 106209698 B CN106209698 B CN 106209698B
- Authority
- CN
- China
- Prior art keywords
- memory
- data packet
- node
- flow structure
- packet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000013467 fragmentation Methods 0.000 claims abstract description 18
- 238000006062 fragmentation reaction Methods 0.000 claims abstract description 18
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000009545 invasion Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/9057—Arrangements for supporting packet reassembly or resequencing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/18—Multiprotocol handlers, e.g. single devices capable of handling multiple protocols
Abstract
The protocol assembly method based on Libnids that the invention discloses a kind of, the corresponding four-tuple of first packet is obtained according to first packet that the data packet received is IP fragmentation, later, flow structure is searched according to the corresponding four-tuple of described first packet, and the sequence number queue being added for saving IP fragmentation sequence number is rebuild to the flow structure found；When the data packet received is non-first packet of IP fragmentation, corresponding HOST is found according to the destination IP of the data packet, corresponding flow structure is searched in several corresponding flow structures of the HOST found, which is write direct in the corresponding memory of the flow structure.Scattered random memory is changed to big memory by the present invention, it has been effectively saved the time of distribution and the release of memory, has improved the efficiency of memory management, has effectively saved the time consumed by file operation, it is operated without the file copy etc. in thread, saves the very big time.
Description
Technical field
The invention belongs to technical field of network security, and in particular to a kind of protocol assembly method based on Libnids.
Background technique
With increasing rapidly for Internet application, network security problem has become focus of attention, by network data
Effective management of content can reduce the security risk of network to a certain extent.The basis that network content management is realized is network number
According to the protocol assembly of packet, including packet capture, protocol analysis, application layer data reduction three parts.Protocol assembly technology is exactly
All messages in network are captured, message is recombinated and restored according to the agreement of standard regulation, is carried out to underlying protocol
On the basis of parsing, mainly the agreement of application layer is analyzed.
Currently, for network packet protocol assembly, there are many technologies and Related product to occur, but due to Netowrk tape
Wide rapid growth, the network information become increasingly abundant, therefore all to the protocol assembly of network packet in terms of performance and function
It puts forward new requirements.
Libnids is the professional programming interface for network invasion monitoring exploitation, it provides the function of TCP data stream recombination
The function that energy and IP fragmentation are recombinated, so can be competent at various agreement Libnids of the analysis based on Transmission Control Protocol.
Libnids also can be regarded as the ICP/IP protocol stack that network packet can be reverted to TCP layer.Currently, common
Protocol assembly technology is all based on Libnids mostly, primary disadvantage is that using one process serial process mode, memory management
Low efficiency causes network throughput small.
Summary of the invention
In view of this, the main purpose of the present invention is to provide a kind of protocol assembly method based on Libnids.
In order to achieve the above objectives, the technical scheme of the present invention is realized as follows:
The embodiment of the present invention provides a kind of protocol assembly method based on Libnids, this method are as follows: according to the number received
The corresponding four-tuple of first packet is obtained according to first packet that packet is IP fragmentation to be corresponded to according to first packet later
Four-tuple search flow structure, and the sequence being added for saving IP fragmentation sequence number is rebuild to the flow structure that finds
Number queue；When the data packet received is non-first packet of IP fragmentation, correspondence is found according to the destination IP of the data packet
HOST, search corresponding flow structure in several corresponding flow structures of the HOST found, which write direct
In the corresponding memory of the flow structure.
In above scheme, this method further include: when the data packet received is not for IP fragmentation, according to the data packet pair
The four-tuple answered searches corresponding flow structure and later writes direct the data packet in the corresponding memory of the flow structure.
In above scheme, before this method further include: one new c file of creation is used for managing internal memory and .h file
For defining the structure of memory node；Configuration file is read first when starting, obtains the distribution of each protocol node
The maximum number of connections that space size and each agreement allow, the memory headroom of needs is calculated according to the two information；Distribution
Memory headroom creates node according to different agreements, and the pointer of node is directed toward corresponding memory address, and initializes in node
Hold；The global linked list head of creation strings together all nodes, is managed respectively the linked list head of different agreement；Create free
Linked list head distributes to thread use, the node of thread release is rejoined again in free chain for saving idle node；?
Mutex is created in linked list head, and mutually exclusive operation is carried out to the thread of application node in multithreading.
In above scheme, it is described the data packet is write direct in the corresponding memory of the flow structure after, this method is also
Data convert is carried out including reading the data packet saved in memory.
It is described to write direct the data packet in the corresponding memory of the flow structure in above scheme, specifically:
The pointer variable of memory node is added in the half-connection structural body struct half_stream of Libnids, is receiving connection
Data determine that the interface that node application is then called when needing space obtains a memory node；In connection closed or surpass
When, pass to protocol processes queue for memory node as parameter, protocol processes thread also can be directly from memory
Data are obtained, until protocol processes thread completes all parsing work and then all memory informations are write disk
In file, the realizing interface of memory node is finally called；Libnids writes half that memory judges connection directly inside Libnids
In the processing queue whether connection data need, and corresponding node is inserted into corresponding thread when connecting terminates.
It is described to write direct the data packet in the corresponding memory of the flow structure in above scheme, specifically: it is receiving
To IP fragmentation first packet when create memory, the initial address of the memory is stored in the half-connection of flow structure, simultaneously also
The sequence number initial value of in store first data packet, the current offset of memory and an address offset queue；When one
When the data packet for needing to be written reaches, the offset address in memory is determined according to sequence number, is write direct；If the sequence number with
The current offset saved in half-connection is equal, then illustrates that the data packet is continuously, if unequal, to illustrate the data packet
It is arrived first than data packet before, needs to save its sequence number into address offset queue, when each current offset is modified all
Need to be compared with the value in this queue, if find it is equal if prove after data packet arrived, need handle
Current offset continues to be modified to the place for not recording data packet.
Compared with prior art, beneficial effects of the present invention:
(1) scattered random memory is changed to big memory, be effectively saved memory distribution and release when
Between, improve the efficiency of memory management.
It (2) should be disposable write operation, when effectively saving consumed by file operation by frequent operating writing-file
Between.
(3) it is operated without the file copy etc. in thread, saves the very big time.
Detailed description of the invention
Fig. 1 is flow chart of the invention；
Fig. 2 is that Libnids writes memory detailed design figure.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
The embodiment of the present invention provides a kind of protocol assembly method based on Libnids, as shown in Figure 1, this method are as follows: according to
The data packet received is that first packet of IP fragmentation obtains described first and wrap corresponding four-tuple, later, according to described the
The corresponding four-tuple of one packet searches flow structure, and rebuilds and be added for saving IP fragmentation sequence to the flow structure found
The sequence number queue of row number；When the data packet received is non-first packet of IP fragmentation, according to the destination IP of the data packet
Corresponding HOST is found, corresponding flow structure is searched in several corresponding flow structures of the HOST found, by the data
Packet writes direct in the corresponding memory of the flow structure.
This method further include: when the data packet received is not for IP fragmentation, according to the corresponding four-tuple of the data packet
Corresponding flow structure is searched, later, which is write direct in the corresponding memory of the flow structure.
Before this method further include: one new c file of creation is for managing internal memory and .h file for defining memory
The structure of node；Configuration file is read first when starting, obtains the space size of each protocol node distribution, and every
A kind of maximum number of connections that agreement allows, the memory headroom of needs is calculated according to the two information；Storage allocation space, according to
Different agreements creates node, and the pointer of node is directed toward corresponding memory address, and initializes node content；Create global chain
Gauge outfit strings together all nodes, is managed respectively the linked list head of different agreement；Creation free linked list head is for saving
Idle node, distributes to thread use, and the node of thread release is rejoined again in free chain；It is created in linked list head mutual
Reprimand amount carries out mutually exclusive operation to the thread of application node in multithreading.
It is described the data packet is write direct in the corresponding memory of the flow structure after, this method further include read memory
The data packet of middle preservation carries out data convert.
It is described to write direct the data packet in the corresponding memory of the flow structure, as shown in Fig. 2, specifically:
The pointer variable of memory node is added in the half-connection structural body struct half_stream of Libnids, is receiving connection
Data determine that the interface that node application is then called when needing space obtains a memory node；In connection closed or surpass
When, pass to protocol processes queue for memory node as parameter, protocol processes thread also can be directly from memory
Data are obtained, until protocol processes thread completes all parsing work and then all memory informations are write disk
In file, the realizing interface of memory node is finally called；Libnids writes half that memory judges connection directly inside Libnids
In the processing queue whether connection data need, and corresponding node is inserted into corresponding thread when connecting terminates.
It is described to write direct the data packet in the corresponding memory of the flow structure, specifically: receiving IP fragmentation
Memory is created when first packet, the initial address of the memory is stored in the half-connection of flow structure, while also in store first
The sequence number initial value of a data packet, the current offset of memory and an address offset queue；It needs to be written when one
When data packet reaches, the offset address in memory is determined according to sequence number, is write direct；If protected in the sequence number and half-connection
The current offset deposited is equal, then illustrates that the data packet is continuously, if unequal, to illustrate the data packet than number before
It is arrived first according to packet, needs to save its sequence number into address offset queue, each current offset requires when modifying and this
Value in queue is compared, if find it is equal if prove after data packet arrived, need current offset
Continue to be modified to the place for not recording data packet.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention.
Claims (4)
1. a kind of protocol assembly method based on Libnids, which is characterized in that this method are as follows: the data packet received is IP points
When first packet of piece, the corresponding four-tuple of first packet is obtained, later, according to the corresponding four-tuple of described first packet
Flow structure is searched, and the sequence number queue being added for saving IP fragmentation sequence number is rebuild to the flow structure found；
When the data packet received is non-first packet of IP fragmentation, corresponding HOST is found according to the destination IP of the data packet,
Corresponding flow structure is searched in several corresponding flow structures of the HOST found, which is write direct into the flow structure
Corresponding memory in；It is described to write direct the data packet in the corresponding memory of the flow structure, specifically: in Libnids
Half-connection structural body struct half_stream in be added memory node pointer variable, receiving connection data, really
Then call the interface of node application to obtain a memory node when needing space surely；Connection closed or time-out when
It waits, passes to protocol processes queue for memory node as parameter, protocol processes thread also can directly obtain number from memory
According to, until protocol processes thread completes all parsing work and then writes all memory informations in disk file,
Finally call the realizing interface of memory node；Libnids writes the half-connection data that memory directly judges connection inside Libnids
In the processing queue for whether needing, and corresponding node being inserted into corresponding thread when connecting terminates；
Alternatively, described write direct the data packet in the corresponding memory of the flow structure, specifically: receiving IP fragmentation
Memory is created when first packet, the initial address of the memory is stored in the half-connection of flow structure, while also in store first
The sequence number initial value of a data packet, the current offset of memory and an address offset queue；It needs to be written when one
When data packet reaches, the offset address in memory is determined according to sequence number, is write direct；If protected in the sequence number and half-connection
The current offset deposited is equal, then illustrates that the data packet is continuously, if unequal, to illustrate the data packet than number before
It is arrived first according to packet, needs to save its sequence number into address offset queue, each current offset requires when modifying and this
Value in queue is compared, if find it is equal if prove after data packet arrived, need current offset
Continue to be modified to the place for not recording data packet.
2. the protocol assembly method according to claim 1 based on Libnids, which is characterized in that this method further include: when
When the data packet received is not IP fragmentation, corresponding flow structure is searched according to the corresponding four-tuple of the data packet, later, by this
Data packet is write direct in the corresponding memory of the flow structure.
3. the protocol assembly method according to claim 1 or 2 based on Libnids, which is characterized in that before this method also
It include: the structure for creating a new c file and being used to define memory node for managing internal memory and .h file；In starting
When read configuration file first, obtain each protocol node distribution space size and each agreement allow maximum
Connection number calculates the memory headroom of needs according to the two information；Storage allocation space creates section according to different agreements
The pointer of point, node is directed toward corresponding memory address, and initializes node content；Global linked list head is created by all nodes
It strings together, the linked list head of different agreement is managed respectively；Creation free linked list head is distributed to for saving idle node
Thread uses, and the node of thread release is rejoined again in free chain；Mutex is created in linked list head, it is right in multithreading
The thread of application node carries out mutually exclusive operation.
4. the protocol assembly method according to claim 3 based on Libnids, which is characterized in that described by the data packet
After writing direct in the corresponding memory of the flow structure, this method further includes reading the data packet saved in memory to carry out data
Reduction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610532683.4A CN106209698B (en) | 2016-07-07 | 2016-07-07 | Protocol assembly method based on Libnids |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610532683.4A CN106209698B (en) | 2016-07-07 | 2016-07-07 | Protocol assembly method based on Libnids |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106209698A CN106209698A (en) | 2016-12-07 |
| CN106209698B true CN106209698B (en) | 2019-07-23 |
Family
ID=57472557
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610532683.4A Active CN106209698B (en) | 2016-07-07 | 2016-07-07 | Protocol assembly method based on Libnids |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106209698B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109547389B (en) * | 2017-08-08 | 2021-12-07 | 中国移动通信集团宁夏有限公司 | Code stream file recombination method and device |
| CN109189505B (en) * | 2018-07-06 | 2021-11-09 | 武汉斗鱼网络科技有限公司 | Method and system for reducing storage space occupied by object serialization |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6937606B2 (en) * | 2001-04-20 | 2005-08-30 | International Business Machines Corporation | Data structures for efficient processing of IP fragmentation and reassembly |
| CN101674234A (en) * | 2009-08-21 | 2010-03-17 | 曙光信息产业(北京)有限公司 | Fragments-reassembling method of IP messages and device thereof |
-
2016
- 2016-07-07 CN CN201610532683.4A patent/CN106209698B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6937606B2 (en) * | 2001-04-20 | 2005-08-30 | International Business Machines Corporation | Data structures for efficient processing of IP fragmentation and reassembly |
| CN101674234A (en) * | 2009-08-21 | 2010-03-17 | 曙光信息产业(北京)有限公司 | Fragments-reassembling method of IP messages and device thereof |
Non-Patent Citations (2)
| Title |
|---|
| 基于Libnids库的Internet网络协议还原系统研究;石金龙等;《电子技术》;20140331(第3期);第13-19页 |
| 高效IP分片重组的设计与实现;王永等;《微处理器》;20081031(第5期);第172-177页 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106209698A (en) | 2016-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Miano et al. | Creating complex network services with ebpf: Experience and lessons learned | |
| CN111371779B (en) | Firewall based on DPDK virtualization management system and implementation method thereof | |
| CN105224692B (en) | Support the system and method for the SDN multilevel flow table parallel searchs of multi-core processor | |
| US8347050B2 (en) | Append-based shared persistent storage | |
| CN112558948A (en) | Method and device for identifying message under mass flow | |
| CN110032449A (en) | A kind of method and device for the performance optimizing GPU server | |
| CN111966284B (en) | OpenFlow large-scale flow table elastic energy-saving and efficient searching system and method | |
| US20110276578A1 (en) | Obtaining file system view in block-level data storage systems | |
| CN102487374B (en) | Access control list realization method and apparatus thereof | |
| US20050097300A1 (en) | Processing system and method including a dedicated collective offload engine providing collective processing in a distributed computing environment | |
| CN108287905B (en) | Method for extracting and storing network flow characteristics | |
| CN106357463B (en) | The access link tracking implementation method and its system of non-invasive | |
| CN103368851A (en) | Openflow flow table storage and optimization method based on resource reuse | |
| CN105653474B (en) | A kind of allocating cache controller towards coarseness dynamic reconfigurable processor | |
| CN106209698B (en) | Protocol assembly method based on Libnids | |
| CN112100146B (en) | Efficient erasure correction distributed storage writing method, system, medium and terminal | |
| US10073878B1 (en) | Distributed deduplication storage system with messaging | |
| WO2021104393A1 (en) | Method for achieving multi-rule flow classification, device, and storage medium | |
| US20120271866A1 (en) | Conservative garbage collecting and tagged integers for memory management | |
| CN109547288B (en) | Programmable flow measuring method for protocol independent forwarding network | |
| CN108596824A (en) | A kind of method and system optimizing rich metadata management based on GPU | |
| Robin et al. | P4KP: QoS-Aware Top-K best path using programmable switch | |
| WO2017197982A1 (en) | Packet processing method, device and system, and computer storage medium | |
| Xie et al. | Index–Trie: Efficient archival and retrieval of network traffic | |
| CN114884893A (en) | Forwarding and control definable cooperative traffic scheduling method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Protocol reduction method based on Libnids Effective date of registration: 20200403 Granted publication date: 20190723 Pledgee: Shaanxi cultural industry finance Company limited by guarantee Pledgor: XI'AN JIAOTONG UNIVERSITY JUMP NETWORK TECHNOLOGY Co.,Ltd. Registration number: Y2020610000035 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Date of cancellation: 20220520 Granted publication date: 20190723 Pledgee: Shaanxi cultural industry finance Company limited by guarantee Pledgor: XI'AN JIAOTONG UNIVERSITY JUMP NETWORK TECHNOLOGY Co.,Ltd. Registration number: Y2020610000035 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Protocol restoration method based on Libnids Effective date of registration: 20220523 Granted publication date: 20190723 Pledgee: Shaanxi cultural industry finance Company limited by guarantee Pledgor: XI'AN JIAOTONG UNIVERSITY JUMP NETWORK TECHNOLOGY Co.,Ltd. Registration number: Y2022610000268 |
|
| PC01 | Cancellation of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Date of cancellation: 20230608 Granted publication date: 20190723 Pledgee: Shaanxi cultural industry finance Company limited by guarantee Pledgor: XI'AN JIAOTONG UNIVERSITY JUMP NETWORK TECHNOLOGY Co.,Ltd. Registration number: Y2022610000268 |