CN101841470B - High-speed capturing method of bottom-layer data packet based on Linux - Google Patents
High-speed capturing method of bottom-layer data packet based on Linux Download PDFInfo
- Publication number
- CN101841470B CN101841470B CN201010134367A CN201010134367A CN101841470B CN 101841470 B CN101841470 B CN 101841470B CN 201010134367 A CN201010134367 A CN 201010134367A CN 201010134367 A CN201010134367 A CN 201010134367A CN 101841470 B CN101841470 B CN 101841470B
- Authority
- CN
- China
- Prior art keywords
- data
- function
- ring
- packet
- capture device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a high-speed capturing method of a bottom-layer data packet based on Linux. By setting a virtual capturing equipment module (VUKM module) to modify a network card driver, the high-speed capturing method leads the data packet reaching to a network card to be capable of bypassing a kernel protocol to be directly passed to a subsequence module for processing so as to realize memory sharing of a user space and a kernel space; and the kernel space transmits the data packet to an upper-layer analysis processing interface module at a high speed, and provides a mechanism for leading an upper-layer application program and the network card to access the VUKM module in a conflict-free manner so as to make further processing to the captured data packet. The high-speed capturing method can acquire the original data packet by the network card at a high speed under the gigabit network environment, and can overcome the defect of the traditional data packet capturing technology, thus improving the acquisition efficiency.
Description
Technical field
The present invention relates under a kind of Linux environment to belong to information security field based on the data packet high-speed catching method of PCI-Express.
Background technology
Along with the arrival in gigabit networking epoch, traditional message capturing mechanism has become the performance bottleneck of whole system.One of which, network message is received by linux kernel with the mechanism that single message drives, in check each bar message and all indistinguishably carry out operations such as buffering application and header check; Its two, mh only reads out a message from kernel through system call at every turn; Its three, message need be sent to the user's space at mh place through memory copying repeatedly.Common network interface card and Libpcap interface just are enough to guarantee to operate in the network message handling procedure of Linux user's space, as: tcpdump, ethereal and snort etc. catch network message linear speed.
This bottleneck problem mainly shows as packet loss and sharply rises and " interruption is flooded "; That is to say; System can't promptly catch network packet; And most cpu clock all is used to handle the interruption that network interface card produces, and has no time to attend to other tasks of processing, and the performance of system is sharply descended.If modification NIC driver; Make the packet that arrives network interface card walk around kernel protocol stack; And directly hand to the subsequent module for processing of structure and realize the direct visit of the user program of user's space to these packets, can improve the efficiency of catching packet greatly.
The traditional data bag is caught by drives interrupts; Because the overhead that interrupts is expensive, when the speed of packet arrival was too fast, system can be absorbed in continuous Interrupt Process and can't jump out; Therefore produce and interrupt livelock, thereby cause the rapid decline of message throughput.Under the identical situation of bandwidth, if the very little packet of transmission, the generation meeting that interrupt this moment is very frequent, and the disposal ability of system can drop to very low.Since adopt drives interrupts to have problems fully, the system that has then adopts complete method of driving to receive message.But when the poll frequency of default during much larger than frequency that packet arrives, it is meaningless that this poll just becomes.Therefore; Adopt drives interrupts or poll to drive the throughput that all can not well improve message separately, a kind of implementation method preferably adopts the driving of half poll to transmit control exactly, when offered load is low; Message time of advent is at random, this appropriate to the occasion employing interrupt processing mechanism; When offered load is higher, should adopt polling mechanism.
In the traditional data capture technique, network data transmission needs repeatedly memory copying, consumes a large amount of cpu cycle and memory source.The basic thought of zero-copy is: the process that packet is transmitted from the network equipment to the user program space, reduce data copy number of times and system call, realize that zero of CPU participates in, thoroughly eliminate CPU burden in this respect.The main implementation technique of zero-copy is DMA data transmission technology and memory-mapped technology.Zero duplication technology at first utilizes the DMA technology that network packet is directly delivered in the pre-assigned address space of system kernel, avoids the participation of CPU; Simultaneously, the region of memory of storage packet in the system kernel is mapped to the application program of user's space, application program directly conducts interviews to this piece internal memory, thereby has reduced the memory copying of system kernel to user's space, has reduced the expense of system call simultaneously.Concrete data in the kernel calling party space do not receive the restriction of operating system, but from security consideration, the data in the operating system restricting user access kernel spacing.Therefore operating system receives packets need and copies user's space to from kernel spacing, if data volume is very big, can reduce the performance of system.
The memory-mapped technology will be constructed a user's space and the shared buffer circle of kernel spacing.What deposit in this buffering area is the address of packet in internal memory that linux kernel is caught.Kernel is whenever caught a packet, just this packet storage addresses is placed in the shared buffer of annular.Memory-mapped is set up " physical address " of the memory on the physical equipment with a sector address of " virtual address space " of kernel or user space processes and is associated.Through associated virtual address, the process of consumer process or kernel spacing realizes the direct access to memory on the physical equipment.If so if physical space of the virtual address associated together of the virtual address of kernel spacing and user's space, they just could share this block space so, and have saved memory copying.
Because the arrival in network gigabit epoch makes the traditional data bag catch and has run into the bottleneck on the performance.It is the prerequisite that intruding detection system, network firewall, network protocol analysis, high-performance router, high performance communication system and other network monitoring system are given full play to its performance that but high-speed message is caught.
Summary of the invention
Goal of the invention:
The present invention seeks under the gigabit networking environment, the bottleneck problem on the performance that runs into to present packet capture technology proposes under a kind of Linux environment the data packet high-speed catching method based on the RT8110 PCI-Express.
Technical scheme:
The present invention adopts following technical scheme for realizing above-mentioned purpose:
A kind of high-speed capturing method of the bottom data bag based on Linux comprises preparation process and job step:
(1), preparation process:
(I), the Interrupt Process function of revising in the NIC driver is submitted packet response code partly to; Make the NIC driver of revising to satisfy the network packet that arrives network interface card and can walk around the virtual capture device module that kernel protocol stack is directly handed to subsequent builds, keep supplying the user space processes of layer protocol analyzing subsystem and obtain;
(II), make up virtual capture device module, the virtual capture device module of registration realizes memory-mapped in kernel, makes the application program shared drive of the user's space of kernel and upper-layer protocol analyzing subsystem;
Said virtual capture device module comprises receiving encircles, sends ring, idle ring and data buffer zone; Wherein:
The data buffer zone is physically and all continuous in logic list structure, is used to deposit the packet that arrives network interface card, to the user's space of upper-layer protocol analyzing subsystem data is provided with " zero-copy " mode;
The user's space that the reception ring is used for the upper-layer protocol analyzing subsystem obtains the data buffering plot structure of having filled;
Idle ring is used for the data buffering plot structure that the user's space with the upper-layer protocol analyzing subsystem disposes and returns to the data buffer zone;
Send ring and be used for the data that the user's space with the upper-layer protocol analyzing subsystem need send and insert the data buffering plot structure, send from network interface card then;
(III), the protocal analysis interface is set; Be used to provide the interface of upper-layer protocol analyzing subsystem and virtual capture device module, make the upper-layer protocol analyzing subsystem resolve, discern the bottom data bag that obtains from virtual capture device module and subsequent treatment such as classification;
(2), job step:
A, the virtual capture device module of initialization; And to this module of kernel registration; Calculate and receive ring, the idle size of encircling, sending ring and buffering area data buffer zone, the size that receives ring, transmission ring, idle ring and data buffer zone is the integral multiple of page-size among the linux;
B, at kernel spacing, be to receive ring, send ring, idle ring and storage allocation space, data buffer zone, call all pages that kernel function will distribute and pin, make these pages in the process that system moves not by the internal memory that swaps out;
C, these memory headrooms of virtual capture device module all are mapped in the user's space go;
D, in virtual capture device module, a data buffering area structure is hung in the reception ring;
E, arrive network interface card when the packet of external network, network interface card is handed to the data buffer zone in the virtual capture device module then to interrupt or the NAPI mode is handled the packet of arrival, sends signal to the protocal analysis interface simultaneously, has informed that packet arrives;
After F, virtual capture device module are obtained packet, this packet is hung the data buffering plot structure in the reception ring;
G, protocal analysis interface are sk_ data buffering plot structure with the packet encapsulation of data buffer zone, and the upper-layer protocol analyzing subsystem are handed in the sk_ data buffer zone carry out analyzing and processing;
After H, upper-layer protocol analyzing subsystem disposed, the data buffer zone that will store this packet through the protocal analysis interface again returned to the free time ring of virtual capture device module; The data that to send simultaneously send to sends ring;
I, transmission ring are inserted the data buffering plot structure with the said data that need to send of H step, send from network interface card then.
In the high-speed capturing method of the bottom data bag based on Linux of the present invention when kernel is virtual capture device module assignment memory headroom; The data buffer zone is organized into the form of chained list; Deposit the kernel spacing first address of this sheet internal memory in the data structure of data buffer zone, NIC driver can directly directly be visited this sheet internal memory through the kernel spacing first address of depositing in the physical address and perhaps begun just can visit this sheet buffering area from a node of chained list.
When said (III) step is provided with the protocal analysis interface, the registration function that 5 chained lists are deposited different levels being set in the high-speed capturing method of the bottom data bag based on Linux of the present invention, is respectively application level function analytic function, IP layer function analytic function, TCP functional analysis function, data link layer functions analytic function, additional function function; Behind the chained list of system initialization logical layer correspondence, when packet arrived, the upper-layer protocol analyzing subsystem can call the registration function that has existed in 5 chained lists one by one through the protocal analysis interface, accomplishes protocal analysis and processing with this; If data are application layer datas, calling application layer function analytic function then; If data, are then called IP layer function analytic function from network layer; If data have the TCP header part, then call TCP functional analysis function; If data are data link layers, then call the data link layer functions analytic function; If data comprise above-mentioned with exterior portions, then call the additional function function.
Network interface card in the high-speed capturing method of the bottom data bag based on Linux of the present invention is the RT8110 PCI-Express.
Beneficial effect:
Adopt the present invention, can under the gigabit networking environment, obtain raw data packets at a high speed, can overcome the drawback of traditional data bag capture technique, improve and obtain efficient through network interface card.
Description of drawings
Fig. 1 is the position of the present invention in whole system.
Fig. 2 is the logic connection layout of the present invention in whole system.
Fig. 3 is the DFD of the inventive method.
Fig. 4 is an amended NIC driver workflow diagram of the present invention.
Fig. 5 is a VUKM module data flow graph of the present invention.
Fig. 6 is a VUKM module initialization flow chart of the present invention.
Fig. 7 is a protocal analysis interface data flow graph of the present invention.
Fig. 8 is a registration function organizational form sketch map of the present invention.
Fig. 9 is a protocal analysis interface initialization flow chart of the present invention.
Figure 10 is a final testing result of the present invention.
Embodiment
Below in conjunction with accompanying drawing technical scheme of the present invention is elaborated:
From structured flowchart illustrated in figures 1 and 2, can find out the present invention, i.e. the position of bottom data bag capture systems in whole DFI sample acquiring platform.After external network data bag among Fig. 1 arrives network interface card; Get into linux kernel through NIC driver; Amended then NIC driver will be handed to virtual capture device module (hereinafter to be referred as the VUKM module) by the network packet that network interface card is caught; To realize the data sharing of kernel spacing and user's space; The protocal analysis interface that is in the upper strata at last extracts and hands to the protocal analysis subsystem of user's space with network packet from the VUKM module, and the protocal analysis subsystem is resolved, discerned and classify network data flow then, and stamps corresponding tag value and pass to the feature selection module that follow-up Filter module and Wrapper combine; Remove redundant and the incoherent characteristic of classification, obtain optimal feature subset.As shown in Figure 2, agreement identification is not emphasis of the present invention with the feature selecting step, be that prior art is not done at this and given unnecessary details, and catching of bottom data bag is key content of the present invention, shows the workflow that is described in detail bottom data bag capture systems respectively:
1. amended NIC driver
Shown in Figure 3 is bottom data bag capture platform DFD; The packet of external network arrives network interface card; Network interface card is not given kernel protocol stack then and is handled, but hand to the buffering area that the VUKM module provides to interrupt or the NAPI mode is handled the packet of arrival; Send the SIGUSR1 signal to the protocal analysis interface simultaneously, informed that packet arrives.
The RT8110 NIC driver is the NIC driver that Linux carries, and supports NAPI technology (Linux is risen by kernel version 2 .4.23 and begins to support the NAPI technology).When packet arrived network interface card, its NIC driver workflow was following:
The flow process of NAPI mode: directly call in the Interrupt Process function _ nettif_rx_schedule, and add this equipment in the POLL processing queue, and develop soft interruption.Soft Interrupt Process function net_rx_action this moment () carries out calling dev-＞poll () function (dev-＞poll is embodied as rtl8110_poll in this driving) after quota etc. calculates to each equipment, carries out polling operation.This function is actual call be Interrupt Process function rtl8110_rx_interrupt (); And in Interrupt Process function rtl8110_rx_interrupt () call macro rtl8110_rx_sk, this is grand will to call netif_receive_skb () function and accomplish last submit operation.
The flow process of NON_NAPI mode: after Interrupt Process function rtl8110_interrupt () makes some necessary processing; Call rtl8110_rx_interrupt; And this Interrupt Process function will wrap from network interface card DMA to internal memory BD and encircle; Call rtl8110_rx_skb again, and rtl8110_rx_skb adds reception formation (packet in this formation is by soft Interrupt Process function net_rx_action ()) processing to calling netif_rx.Soft Interrupt Process function net_rx_action () calls dev-＞poll (under the NON_NAPI pattern; Dev-＞poll is process_backlog () function), process_backlog () function then calls netif_receive_skb accomplish after last submit operation.
The RT8110 network interface card has adopted the NAPI technology; In the RT8110 NIC driver, no matter submit packet in which way to; Its operation all realizes in rtl8110_rx_interrupt () function, and Interrupt Process function rtl8110_rx_interrupt () major function is through netifx () function or netif_receive_skb () function the Frame in the DMA buffer memory to be submitted to the upper strata with the skb structure.Therefore we only need to revise the response code that Interrupt Process function rtl8110_rx_interrupt () submits the packet part to.In order to revise the RT8110 NIC driver,
Need in this driver, add following code:
if(vukm?dev?state){
Vukm clean data buffer zone of rxfreebd ();
Vukm data buffer zone=vukm malloc data buffer zone f ();
If (vukm data buffer zone)
Memcpy (the vukm data buffer zone-＞packet, skb-＞data, pkt size);
Vukm insert data buffer zone to rxbd (vukm data buffer zone, pkt size);
}Else{
Printk (KERN DEBUG " nsi vukm:No Empty data buffer zone fer! N ");
}
}
Amended NIC driver can be walked around kernel protocol stack with network packet and directly hand to the VUKM module; But the epimere code need at first be done some simultaneous operations; Then packet is handed to the function vukm_insert_ data buffer zone _ to_rxbd () in the VUKM module, its major function is that a data buffering area structure is hung in the reception ring.Fig. 4 is amended NIC driver workflow diagram.
2. make up the VUMK module
Fig. 5 is a VUKM module data flow graph; After the VUKM module is obtained network packet from amended NIC driver; This packet is hung into the reception ring; And then packet sent to the protocal analysis interface, the protocal analysis interface returns to the data buffer zone f of this packet of storage on the free time ring of VUKM module after handling corresponding network packet.The key that " zero-copy " realized is exactly a ring structure, and packet then leaves in the data buffer zone, and this buffering area distributes in kernel, one of the size location 2K of this structure.In order to reduce memory fragmentation as much as possible, so this module adopts the good internal memory of application in advance.At kernel spacing; For receiving ring, transmission ring, idle ring and data buffer zone (receiving the integral multiple that the size of encircling, sending ring, idle ring and data buffer zone is page-size among the linux) storage allocation space, the vukm module all is mapped to these internal memories in the user's space and goes with vmalloc ().
Fig. 6 is a VUKM module initialization flow chart, and in the process that linux system starts, the VUKM module is initialised, and to this module device of kernel registration, and begin to calculate and distribute the size that receives ring, idle ring, sends ring and buffering area data buffer zone.And calling all pages that set_bit () function that kernel provides and atomic_inc () function will distribute pins; Make these pages in the process of system operation not by the internal memory that swaps out; The buffer data buffering area is organized into physically and all continuous in logic list structure, to the upper strata data is provided with " zero-copy " mode then.Vukm module initialization function is vukm_init ().
Usually the realization of " zero-copy " mode be in user's space storage allocation space as buffering area, network interface card then is to realize through the user's physical address mapping table that in the VUKM module, is incorporated as the user buffering area definition if will visit this buffering area.And the VUKM module of packet capture platform of the present invention changes at the kernel spacing storage allocation and (pins this sheet internal memory; Prevent to swap out); And be organized into the form of chained list; Kaddr item in the buffer data buffer data structure is deposited the kernel spacing first address of this sheet internal memory, and NIC driver can be directly directly visited this sheet internal memory or network interface card through the kernel spacing first address of depositing among the kaddr and begun just can visit this sheet buffering area from a node of chained list.
In the linux system, the application program that is in user's attitude is in the kernel state data because the setting of authority can't be visited.But memory-mapped mechanism is visited the problem of kernel state data for the application program lack of competence that has solved user's attitude well; It makes user's attitude and kernel state share a slice region of memory; The packet that arrives network interface card directly is passed to the data buffer zone that is positioned at this sheet shared drive zone, and user's attitude application program just can be taken data away from the data buffer zone.
3. the protocal analysis interface is provided with
Fig. 7 is a protocal analysis interface data flow graph; Realize memory-mapped through registration VUKM module in the kernel; Thereby realized the application program shared drive of kernel state and user's attitude, the protocal analysis interface has then been realized the mechanism of packet from network interface card to the protocal analysis subsystem.This interface is a sk_ data buffering plot structure with the packet encapsulation of data buffer zone; And the registration function that the sk_ data buffer zone is handed to the protocal analysis subsystem on upper strata and provided carried out analyzing and processing; After finishing, the VUKM module is returned in the data buffer zone of storage packet.In the bottom data capture platform; The protocal analysis interface has then been realized the transmission mechanism of packet from network interface card to the protocal analysis subsystem; The protocal analysis subsystem needs a unified development interface; Therefore the mode with registration function is that the upper strata provides this interface, and we are provided with the registration function that 5 chained lists are deposited different levels.
Fig. 8 is a registration function organizational form sketch map of the present invention; The function that in native system, is used to register is funcmod_reg (); This function confirms which chained list regfunc () function should add in, after packet arrives according to the value of the shape ginseng wfunlevel that imports into; The registration function that has existed in five chained lists be can call one by one, protocal analysis and processing accomplished with this.
Fig. 9 is the initialization flowchart of protocal analysis interface; System initialization at first; The corresponding chained list of 5 logical layers of system call mod_call_init () function, and initialization then, protocal analysis subsystem only need when initialization, to call each call back function of mod_call_reg () function registration.If user's input " S " order then begins the initializtion protocol analyzing subsystem, wait for receiving packet; If all registered function information are then printed in user's input " P " order; If user's input " Q " order, then EP (end of program).
The present invention has following beneficial effect: adopt the present invention; Can be under the gigabit networking environment; At a high speed catch raw data packets through network interface card, can overcome bottleneck and the drawback that takes place to interrupt to flood on the performance that traditional data bag capture technique brings, improve the efficient of system.
4. function and performance test
We test the performance that this method is caught packet in real network environment.Figure 10 is the test result of this method, and 0xc3c6d00b is a memory address, and 001d is the packet in this sheet internal memory, and is visible, and network flow is in the storage continuously of internal memory Chinese style.0xc3c6d00b, 0xc3c6d01b, 0xc3c6d02b represent the continuous memory location of a network flow in this sheet internal memory, and corresponding is the primitive network packet of in this sheet internal memory, storing thereafter.
Table 1 is this packet capture system system acquisition The performance test results under various message lengths.
Define following performance index:
Critical give out a contract for a project (ns) at interval: send packet at interval by certain hour, when the transmission time interval of two adjacent data bags reached this value, system can produce the packet loss phenomenon.
Maximum rate (p/s): system under the prerequisite that packet loss does not take place, the packet maximum number order that per second can be caught.
Maximum stream flow (bps): system under the prerequisite that packet loss does not take place, the data message total length that per second can be caught.
Table 1 system testing conclusive table
| Long data packet (B) | Critical give out a contract for a project (ns) at interval | Maximum rate (p/s) | Maximum stream flow (bps) |
| 64 | About 1053 | About 938900 | About 490200000 |
| 128 | About 1068 | About 942000 | About 967400000 |
| 512 | About 4300 | About 239000 | About 968500000 |
| 1500 | About 11900 | About 82800 | About 964000000 |
It is thus clear that system tends towards stability when message length is 128B, near the network gigabit wire speed.
Step of the present invention is described below on the whole, specific as follows:
Steps A, modification NIC driver make the NIC driver of revising can satisfy the network packet that arrives network interface card and can walk around the virtual capture device module that kernel protocol stack is directly handed to subsequent builds, supply user space processes to obtain; The RT8110 network interface card has adopted NAPI (new application programming interfaces) technology; In the RT8110 NIC driver, no matter submit packet in which way to; Its operation all realizes in the Interrupt Process function, and the major function of Interrupt Process function is that the Frame in the DMA buffer memory is submitted to the upper strata with the skb structure.Therefore only need to revise the response code that the Interrupt Process function is submitted the packet part to, just can achieve the goal:
The first step if the virtual capture device module that makes up is in available opening, then will receive data buffering plot structure complete restitution on encircling in the metadata cache chained list, be virtual capture device module assignment memory headroom.
In second step, if the success of the Memory Allocation of virtual device modules, the packet that then will arrive network interface card is directly sent to the memory headroom of having distributed, and a data buffering area structure is hung on the reception ring.
The 3rd one, if the Memory Allocation of virtual device modules is unsuccessful, then printing does not have unnecessary Memory Allocation.
Step B, the virtual capture device module of structure: its role is to make the network packet that arrives network interface card can walk around kernel protocol stack and realize of the direct visit of the user program of user's space, give upper-layer protocol analyzing and processing interface by the kernel spacing high-speed transfer to these packets.The main realization is to realize memory-mapped through the virtual capture device module of registration in kernel, thereby reaches the application program shared drive of kernel and user's space, improves data packet transmission efficient.
Said virtual capture device module comprises the data buffer zone that receives ring, transmission ring, idle ring and store data bag;
Receive ring function: amended NIC driver will reach the packet of network interface card and put into the data buffer zone; Then the data buffer zone is hung on the reception ring; Because virtual capture device module makes that user's space and kernel spacing can shared drives, so user's space can directly obtain the data buffering plot structure of having filled from reception encircles.
Idle ring function: the data buffering plot structure that user's space will dispose is hung into idle ring; And idle ring returns to buffering area with hanging the data buffering plot structure of; If have network packet to reach network interface card this moment; Then amended NIC driver can be obtained the data buffering plot structure of having given back once more, and inserts packet.
Send ring function: if user's space has data to send from network interface card, the data that then will send are inserted the data buffering plot structure, hang then and to send on the ring, send from network interface card through sending ring.
The first step, network interface card is directly passed to shared data buffer zone with data, and network interface card and upper-layer protocol analyzing subsystem can be encircled and the shared data buffer zone of idle ring visit through receiving by ensuring escapement from confliction;
In second step, the application program that is positioned at user's space can be visited receive ring, transmission ring, idle ring and data buffer zone;
In the process that linux system starts, virtual capture device module is initialised, and to this module device of kernel registration, and begin to calculate and distribute the size that receives ring, transmission ring, idle ring and buffering area data buffer zone.The buffer data buffering area is organized into physically and all continuous in logic list structure, to the upper strata data is provided with " zero-copy " mode then.Network interface card then is to realize through the user's physical address mapping table that in virtual capture device module, is incorporated as the user buffering area definition if will visit this buffering area.Kaddr item in the buffer data buffer data structure is deposited the kernel spacing first address of this sheet internal memory, and NIC driver can be directly directly visited this sheet internal memory or network interface card through the kernel spacing first address of depositing among the kaddr and begun just can visit this sheet buffering area from a node of chained list.
The application program of user's space is because the setting of authority can't access process kernel state data.But memory-mapped mechanism has solved the problem of the application program lack of competence visit kernel state data of user's attitude well; It makes user's attitude and kernel state share a slice region of memory; The packet that arrives network interface card directly is passed to the data buffer zone that is arranged in this sheet shared drive zone, and the application program of user's attitude just can be taken data away from the data buffer zone.
Step C, the protocal analysis interface is set: the protocal analysis interface provides the interface of upper-layer protocol analyzing subsystem (DPI system) with the bottom data capture systems, and it is regional to visit same block cache when being positioned at the application program read data of user side during with network interface card DMA transmission data.
The first step, the protocal analysis subsystem needs a unified development interface, and therefore the mode with registration function is that the upper strata provides this interface.System initialization, the corresponding chained list of system initialization logical layer then.
Second step, wait for user's input, if user's input " S " (beginning) order then begins the initializtion protocol analyzing subsystem, wait for receiving packet; If all registered function information are then printed in the order of user's input " P " (stopping), return after finishing; If the order of user's input " Q " (withdrawing from), then EP (end of program).
The 3rd step; The protocal analysis interface is a sk_ data buffer zone f structure with the data encapsulation in the data buffer area; And the registration function that this structure is handed to the upper-layer protocol analyzing subsystem and provided carried out analyzing and processing; After finishing, the data buffer zone of storing packet is returned to virtual capture device module.
Claims (4)
1. high-speed capturing method based on the bottom data bag of Linux is characterized in that: comprise preparation process and job step:
(1), preparation process:
(I), the Interrupt Process function of revising in the NIC driver is submitted packet response code partly to; Make the NIC driver of revising to satisfy the network packet that arrives network interface card and walk around the virtual capture device module that kernel protocol stack is directly handed to subsequent builds, keep supplying the user space processes of layer protocol analyzing subsystem and obtain;
(II), make up virtual capture device module, the virtual capture device module of registration realizes memory-mapped in kernel, makes the application program shared drive of the user's space of kernel and upper-layer protocol analyzing subsystem;
Said virtual capture device module comprises receiving encircles, sends ring, idle ring and data buffer zone; Wherein:
The data buffer zone is physically and all continuous in logic list structure, is used to deposit the packet that arrives network interface card, to the user's space of upper-layer protocol analyzing subsystem data is provided with " zero-copy " mode;
Receiving ring is used for making the user's space of upper-layer protocol analyzing subsystem to obtain the data buffering plot structure of having filled from receiving ring;
Idle ring is used for the data buffering plot structure that the user's space with the upper-layer protocol analyzing subsystem disposes and returns to the data buffer zone;
Send ring and be used for the data that the user's space with the upper-layer protocol analyzing subsystem need send and insert the data buffering plot structure, send from network interface card then;
(III), the protocal analysis interface is set; Be used to provide the interface of upper-layer protocol analyzing subsystem and virtual capture device module, make the upper-layer protocol analyzing subsystem resolve, discern and the subsequent treatment of classifying the bottom data bag that obtains from virtual capture device module;
(2), job step:
A, the virtual capture device module of initialization, and, calculate the size that receives ring, free time ring, sends ring and data buffer zone to this module of kernel registration, receive the integral multiple that the size of encircling, sending ring, idle ring and data buffer zone is page-size among the linux;
B, at kernel spacing, be to receive ring, send ring, idle ring and storage allocation space, data buffer zone, call all pages that kernel function will distribute and pin, make these pages in the process that system moves not by the internal memory that swaps out;
C, these memory headrooms of virtual capture device module all are mapped in the user's space go;
D, in virtual capture device module, a data buffering area structure is hung in the reception ring;
E, arrive network interface card when the packet of external network; Network interface card is with the packet that interrupts or newly application programming interfaces (NAPI) mode is handled arrival; Hand to the data buffer zone in the virtual capture device module then, send signal to the protocal analysis interface simultaneously, informed that packet arrives;
After F, virtual capture device module are obtained packet, this packet is hung the data buffering plot structure in the reception ring;
G, protocal analysis interface are sk_ data buffering plot structure with the packet encapsulation of data buffer zone, and sk data buffering plot structure are handed to the upper-layer protocol analyzing subsystem carry out analyzing and processing;
After H, upper-layer protocol analyzing subsystem disposed, the data buffer zone that will store this packet through the protocal analysis interface again returned to the free time ring of virtual capture device module; The data that to send simultaneously send to sends ring;
I, transmission ring are inserted the data buffering plot structure with the said data that need to send of H step, send from network interface card then.
2. the high-speed capturing method of the bottom data bag based on Linux according to claim 1; It is characterized in that: among the step B of said job step; When kernel is virtual capture device module assignment memory headroom; The data buffer zone of virtual capture device module is organized into the form of chained list; Deposit said kernel in the data structure of data buffer zone and be the kernel spacing first address of the memory headroom that virtual capture device module distributed; It is the memory headroom that virtual capture device module is distributed that NIC driver is directly directly visited kernel through the kernel spacing first address of depositing in the physical address, and perhaps the node from chained list begins the data buffer zone with regard to accesses virtual capture device module.
3. the high-speed capturing method of the bottom data bag based on Linux according to claim 1; It is characterized in that: when said (III) step is provided with the protocal analysis interface, the registration function that 5 chained lists are deposited different levels being set, is respectively application level function analytic function, IP layer function analytic function, TCP functional analysis function, data link layer functions analytic function, additional function function; Behind the chained list of system initialization logical layer correspondence, when packet arrived, the upper-layer protocol analyzing subsystem can call the registration function that has existed in 5 chained lists one by one through the protocal analysis interface, accomplishes protocal analysis and processing with this; If data are application layer datas, calling application layer function analytic function then; If data, are then called IP layer function analytic function from network layer; If data have the TCP header part, then call TCP functional analysis function; If data are data link layers, then call the data link layer functions analytic function; If data comprise above-mentioned with exterior portions, then call the additional function function.
4. the high-speed capturing method of the bottom data bag based on Linux according to claim 1, it is characterized in that: said network interface card is the RT8110 PCI-Express.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010134367A CN101841470B (en) | 2010-03-29 | 2010-03-29 | High-speed capturing method of bottom-layer data packet based on Linux |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010134367A CN101841470B (en) | 2010-03-29 | 2010-03-29 | High-speed capturing method of bottom-layer data packet based on Linux |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101841470A CN101841470A (en) | 2010-09-22 |
| CN101841470B true CN101841470B (en) | 2012-10-10 |
Family
ID=42744604
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201010134367A Expired - Fee Related CN101841470B (en) | 2010-03-29 | 2010-03-29 | High-speed capturing method of bottom-layer data packet based on Linux |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101841470B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108881026A (en) * | 2018-06-01 | 2018-11-23 | 武汉绿色网络信息服务有限责任公司 | A kind of BGP message forwarding method and device for realizing router based on linux system |
| CN109309605A (en) * | 2018-11-26 | 2019-02-05 | 北京邮电大学 | Band network telemetry system and method |
Families Citing this family (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102497434B (en) * | 2011-12-16 | 2014-11-05 | 中国科学院计算技术研究所 | Establishing method of kernel state virtual network equipment and packet transmitting and receiving methods thereof |
| CN103490939A (en) * | 2012-06-11 | 2014-01-01 | 中兴通讯股份有限公司 | Data packet processing method and data packet processing device |
| US9069658B2 (en) * | 2012-12-10 | 2015-06-30 | Google Inc. | Using a virtual to physical map for direct user space communication with a data storage device |
| CN103441941A (en) * | 2013-08-13 | 2013-12-11 | 广东睿江科技有限公司 | High performance data message capture method and device based on Linux |
| CN103617142B (en) * | 2013-09-09 | 2017-03-15 | 南京邮电大学 | A kind of express network collecting method based on pf_ring |
| CN104636175A (en) * | 2013-11-06 | 2015-05-20 | 沈阳高精数控技术有限公司 | ARM platform networking method for embedded real-time system |
| CN104092581A (en) * | 2014-06-30 | 2014-10-08 | 国家电网公司 | Quick message processing method in message recording and analyzing system of intelligent substation |
| CN105635045B (en) * | 2014-10-28 | 2019-12-13 | 北京启明星辰信息安全技术有限公司 | Tcpdump packet capture implementation method and device based on drive zero copy mode system |
| CN104901844B (en) * | 2015-05-13 | 2019-01-22 | 国家计算机网络与信息安全管理中心 | High Precision Time Stamps acquisition methods, device and network interface card based on PCIE |
| CN106293888A (en) * | 2015-06-04 | 2017-01-04 | 北京国双科技有限公司 | The packet capturing of data, deposit method and the packet capturing of data, deposit device |
| CN106330776A (en) * | 2015-06-30 | 2017-01-11 | 中兴通讯股份有限公司 | Message processing method and device |
| CN106775462A (en) * | 2016-11-30 | 2017-05-31 | 郑州云海信息技术有限公司 | A kind of method and apparatus that memory copying is reduced during read-write |
| CN106850547A (en) * | 2016-12-15 | 2017-06-13 | 华北计算技术研究所（中国电子科技集团公司第十五研究所） | A kind of data restoration method and system based on http protocol |
| CN106775490A (en) * | 2016-12-29 | 2017-05-31 | 郑州云海信息技术有限公司 | A kind of kernel log storage method and device based on nonvolatile memory |
| CN107181738B (en) * | 2017-04-25 | 2020-09-11 | 中国科学院信息工程研究所 | Software intrusion detection system and method |
| CN107204939A (en) * | 2017-05-27 | 2017-09-26 | 南京南瑞继保电气有限公司 | A kind of message processing method based on two-level cache |
| CN107730352A (en) * | 2017-09-30 | 2018-02-23 | 携程计算机技术（上海）有限公司 | Order based reminding method, system, storage medium, electronic installation and server |
| CN110011933B (en) * | 2018-01-05 | 2021-05-18 | 华为技术有限公司 | Method, apparatus and computer readable storage medium for transmitting data packet |
| US11106491B2 (en) * | 2018-04-06 | 2021-08-31 | Beijing Didi Infinity Technology And Development Co., Ltd. | Method and system for kernel routine callbacks |
| CN110535813B (en) * | 2018-05-25 | 2022-04-22 | 网宿科技股份有限公司 | Method and device for processing coexistence of kernel mode protocol stack and user mode protocol stack |
| CN111327645B (en) * | 2018-11-28 | 2023-11-21 | 鸿合科技股份有限公司 | Network sharing method and device and electronic equipment |
| CN109413106A (en) * | 2018-12-12 | 2019-03-01 | 中国航空工业集团公司西安航空计算技术研究所 | A kind of ICP/IP protocol stack implementation method |
| CN109688066B (en) * | 2018-12-29 | 2020-11-13 | 合肥埃科光电科技有限公司 | Gateway filtering driving method based on GigE Vision |
| CN110086571A (en) * | 2019-04-10 | 2019-08-02 | 广州华多网络科技有限公司 | A kind of data transmission and received method, apparatus and data processing system |
| CN110083363B (en) * | 2019-04-22 | 2022-04-01 | 珠海网博信息科技股份有限公司 | Method for intercepting wireless data packet in Linux kernel dynamic injection mode |
| CN110445580B (en) * | 2019-08-09 | 2022-04-19 | 浙江大华技术股份有限公司 | Data transmission method and device, storage medium, and electronic device |
| CN110730157A (en) * | 2019-08-31 | 2020-01-24 | 苏州浪潮智能科技有限公司 | Storage system intrusion detection method, system, terminal and storage medium |
| CN111181736B (en) * | 2019-12-31 | 2022-04-05 | 奇安信科技集团股份有限公司 | Data transmission method, device, system and medium |
| CN111371759B (en) * | 2020-02-25 | 2022-06-21 | 深信服科技股份有限公司 | Network data packet reading method, device, equipment and readable storage medium |
| CN111917835A (en) * | 2020-07-13 | 2020-11-10 | 北京天空卫士网络安全技术有限公司 | System, method and device for monitoring network data |
| CN114513382B (en) * | 2020-11-16 | 2023-07-25 | 沈阳中科数控技术股份有限公司 | Real-time Ethernet field bus network data packet processing method based on network card |
| CN114513383B (en) * | 2020-11-16 | 2023-07-21 | 沈阳中科数控技术股份有限公司 | Real-time Ethernet field bus data packet processing method based on original socket |
| CN113114532A (en) * | 2021-04-07 | 2021-07-13 | 国网上海市电力公司 | Substation automation network data monitoring method based on portable equipment |
| CN115051778B (en) * | 2022-06-07 | 2023-06-27 | 西安微电子技术研究所 | Linux modularized PRP protocol stack system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1435977A (en) * | 2002-02-01 | 2003-08-13 | 联想(北京)有限公司 | Method for detecting and responding of fire wall invasion |
| CN101135980A (en) * | 2006-08-29 | 2008-03-05 | 飞塔信息科技（北京）有限公司 | Device and method for realizing zero copy based on Linux operating system |
| CN101227341A (en) * | 2007-12-18 | 2008-07-23 | 浪潮电子信息产业股份有限公司 | Method for fast catching Ethernet card on Linux system |
-
2010
- 2010-03-29 CN CN201010134367A patent/CN101841470B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1435977A (en) * | 2002-02-01 | 2003-08-13 | 联想(北京)有限公司 | Method for detecting and responding of fire wall invasion |
| CN101135980A (en) * | 2006-08-29 | 2008-03-05 | 飞塔信息科技（北京）有限公司 | Device and method for realizing zero copy based on Linux operating system |
| CN101227341A (en) * | 2007-12-18 | 2008-07-23 | 浪潮电子信息产业股份有限公司 | Method for fast catching Ethernet card on Linux system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108881026A (en) * | 2018-06-01 | 2018-11-23 | 武汉绿色网络信息服务有限责任公司 | A kind of BGP message forwarding method and device for realizing router based on linux system |
| CN108881026B (en) * | 2018-06-01 | 2020-02-21 | 武汉绿色网络信息服务有限责任公司 | Linux system based BGP message forwarding method and device for realizing router |
| CN109309605A (en) * | 2018-11-26 | 2019-02-05 | 北京邮电大学 | Band network telemetry system and method |
| CN109309605B (en) * | 2018-11-26 | 2020-08-25 | 北京邮电大学 | In-band network telemetry system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101841470A (en) | 2010-09-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101841470B (en) | High-speed capturing method of bottom-layer data packet based on Linux | |
| US20220245072A1 (en) | System and method for facilitating dynamic command management in a network interface controller (nic) | |
| CN101917350B (en) | Network card drive-based zero copy Ethernet message capturing and transmitting implementation method under Linux | |
| US8051212B2 (en) | Network interface adapter with shared data send resources | |
| US7836212B2 (en) | Reflecting bandwidth and priority in network attached storage I/O | |
| CN102123090B (en) | IP (Internet protocol) fragment processing method based on two-level table storage and transport layer information inquiry | |
| US20080022016A1 (en) | Network memory pools for packet destinations and virtual machines | |
| CN101702121B (en) | Device for controlling network flow of program in Windows system | |
| CN110086571A (en) | A kind of data transmission and received method, apparatus and data processing system | |
| CN101873337A (en) | Zero-copy data capture technology based on rt8169 gigabit net card and Linux operating system | |
| CN101135980A (en) | Device and method for realizing zero copy based on Linux operating system | |
| CN109783250A (en) | A kind of message forwarding method and the network equipment | |
| CN100535886C (en) | Data-transmission system between nodes, and device and method | |
| CN110109852A (en) | System and method for realizing TCP _ IP protocol by hardware | |
| CN101304373A (en) | Method and system for implementing high-efficiency transmission chunk data in LAN | |
| CN103617142B (en) | A kind of express network collecting method based on pf_ring | |
| CN104506379A (en) | Method and system for capturing network data | |
| CN107220200A (en) | Time triggered Ethernet data management system and method based on dynamic priority | |
| CN109565455A (en) | Packet descriptor storage in packeting memory with cache | |
| US11010165B2 (en) | Buffer allocation with memory-based configuration | |
| CN101957808A (en) | Communication method among various CPUs (Central Processing Units), system and CPU | |
| CN113127139B (en) | Memory allocation method and device based on DPDK of data plane development kit | |
| CN110519180A (en) | Network card virtualization queue scheduling method and system | |
| CN102916902A (en) | Method and device for storing data | |
| WO2014092551A1 (en) | System and method for optimal memory management between cpu and fpga unit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20121010 Termination date: 20150329 |
|
| EXPY | Termination of patent right or utility model |