CN106713462A - Network data packet processing method and device - Google Patents
Network data packet processing method and device Download PDFInfo
- Publication number
- CN106713462A CN106713462A CN201611220118.0A CN201611220118A CN106713462A CN 106713462 A CN106713462 A CN 106713462A CN 201611220118 A CN201611220118 A CN 201611220118A CN 106713462 A CN106713462 A CN 106713462A
- Authority
- CN
- China
- Prior art keywords
- data
- network
- user space
- buffering area
- network equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/30—Definitions, standards or architectural aspects of layered protocol stacks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a network data packet processing method and device. The method comprises the steps of establishing a mapping relationship between a user mode memory space and a buffer area of a network device in a drive of a kernel mode; obtaining data from the buffer area of the network device; directly converting the obtained from the kernel mode to the user mode; and processing the data through utilization of a CPU. The device comprises a mapping relationship establishment unit for establishing the mapping relationship between the user mode memory space and the buffer area of the network device in the drive of the kernel mode; a data obtaining unit for obtaining the data from the buffer area of the network device; a data conversion unit for directly converting the obtained from the kernel mode to the user mode; and a data processing unit for processing the data through utilization of the CPU. According to the method and the device, the system resources and time occupied by each data packet can be reduced, and the network data packet processing performance is improved.
Description
Technical field
The invention belongs to technical field of data processing, more particularly to a kind of processing method of network data packets and device.
Background technology
With the development of the network communications technology, the network bandwidth is improved rapidly, and 1Gbit, 10Gbit network are gradually popularized,
40Gbit networks large-scale application in telecom operators, large enterprise's level backbone network.The exponential growth of network traffics, it is right
The performance of the network packets such as fire wall, IDS/IPS, application gateway analyzing and processing class networking products proposes new challenge.Pass
The processing method of network data packets of system is faced with that packet loss is high, time delay is big and system load is high etc. under high speed network environment and asks
Topic, it is impossible to meet the requirement of the network packet real-time processing under high speed network environment.
The content of the invention
To solve the above problems, the invention provides a kind of processing method of network data packets and device, each can be reduced
System resource and time that processing data packets take, improve network packet process performance.
A kind of processing method of network data packets that the present invention is provided, including：
It is that User space memory headroom sets up mapping relations with the buffering area of the network equipment in the driving of kernel state；
Data are obtained from the buffering area of the network equipment；
The data of acquisition are directly changed into User space from kernel state；
The data are processed using CPU.
Preferably, it is described to obtain number from the buffering area of the network equipment in above-mentioned processing method of network data packets
According to for：
Using the poll thread data are obtained from the buffering area of the network equipment.
Preferably, in above-mentioned processing method of network data packets, set from the network using the poll thread described
Before obtaining data in standby buffering area, also include：
Each described poll thread is tied on a corresponding CPU.
Preferably, it is described in the driving of kernel state in above-mentioned processing method of network data packets, it is empty User space internal memory
Between set up mapping relations with the buffering area of the network equipment before, also include：
Device drives are split as the driving of the kernel state and the User space memory headroom.
A kind of network packet processing unit that the present invention is provided, including：
Mapping relations set up unit, in the driving of kernel state, being User space memory headroom slow with the network equipment
Rush area and set up mapping relations；
Data capture unit, for obtaining data from the buffering area of the network equipment；
Date Conversion Unit, for the data of acquisition to be directly changed into User space from kernel state；
Data processing unit, for being processed the data using CPU.
Preferably, in above-mentioned network packet processing unit, the data capture unit specifically for：
Using the poll thread data are obtained from the buffering area of the network equipment.
Preferably, in above-mentioned network packet processing unit, also include：
Binding unit, for each described poll thread to be tied into a corresponding CPU.
Preferably, in above-mentioned network packet processing unit, also include：
Split cells, for the driving that device drives are split as the kernel state and the User space memory headroom.
The above-mentioned processing method of network data packets and device provided by foregoing description, the present invention, due to including：
It is that User space memory headroom sets up mapping relations with the buffering area of the network equipment in the driving of kernel state；From the network equipment
Buffering area in obtain data；The data of acquisition are directly changed into User space from kernel state；The data are carried out using CPU
Treatment.The method and device avoid kernel mode to the copy procedure of User space, can simplify protocol layer processing procedure, shielding
Socket layers of protocol processes, therefore, it is possible to reduce system resource and the time of each processing data packets occupancy, improve network number
According to bag process performance.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
The accompanying drawing to be used needed for having technology description is briefly described, it should be apparent that, drawings in the following description are only this
Inventive embodiment, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
The accompanying drawing of offer obtains other accompanying drawings.
The schematic diagram of the first processing method of network data packets that Fig. 1 is provided for the embodiment of the present application；
The schematic diagram of the first network packet processing unit that Fig. 2 is provided for the embodiment of the present application.
Specific embodiment
Core concept of the invention is to provide a kind of processing method of network data packets and device, can reduce each data
System resource and time that bag treatment takes, improve network packet process performance.
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.It is based on
Embodiment in the present invention, it is every other that those of ordinary skill in the art are obtained under the premise of creative work is not made
Embodiment, belongs to the scope of protection of the invention.
The first processing method of network data packets that the embodiment of the present application is provided is as shown in figure 1, Fig. 1 is the embodiment of the present application
The schematic diagram of the first processing method of network data packets for providing.The method comprises the following steps：
S1：It is that User space memory headroom sets up mapping relations with the buffering area of the network equipment in the driving of kernel state；
It should be noted that network protocol stack is in network data, multiple data duplication is no longer carried out, but will
Data in hardware are copied directly to User space, it is to avoid the copy procedure of kernel mode to User space, do so can simplify agreement
Layer processing procedure, the protocol processes of socket layers of shielding.
S2：Data are obtained from the buffering area of the network equipment；
In this step, can be obtained using modes such as interruption or polls, be not intended to limit herein.
S3：The data of acquisition are directly changed into User space from kernel state；
It should be noted that in the implementation of kernel protocol stack of the prior art, physics of the data from arrival network interface card
Equipment starts, it is necessary to experience Firmware-Device Driver-Ethernet-IP-TCP/UDP-Sockets-System
Many processes of Call Interface, can all be related to data directly mobile in internal memory or index movement in those steps,
The conversion from kernel state to User space is also related to, unnecessary expense is brought, and utilizes the step, the data in hardware are straight
Connect and copy User space to, it becomes possible to solve this problem.
S4：The data are processed using CPU.
It should be noted that the above method by way of rewriting linux kernel network protocol stack to network packet
Processing procedure is optimized, and realizes improving the purpose of network packet process performance, when treatment thread is realized, it is to avoid in treatment
During called using system, the expense called into system of reduction.
Above-mentioned the first processing method of network data packets provided by foregoing description, the embodiment of the present application, due to
Including：It is that User space memory headroom sets up mapping relations with the buffering area of the network equipment in the driving of kernel state；From the net
Data are obtained in the buffering area of network equipment；The data of acquisition are directly changed into User space from kernel state；Using CPU to the number
According to being processed.This method avoid the copy procedure of kernel mode to User space, protocol layer processing procedure can be simplified, shielded
Socket layers of protocol processes, therefore, it is possible to reduce system resource and the time of each processing data packets occupancy, improve network number
According to bag process performance.
Second processing method of network data packets that the embodiment of the present application is provided, is at above-mentioned the first network packet
On the basis of reason method, also including following technical characteristic：
It is described from the buffering area of the network equipment obtain data be：
Using the poll thread data are obtained from the buffering area of the network equipment.
It is substantial amounts of to notify system by the way of interrupt it should be noted that driving implementation of the prior art
The variation of data, when there is system call statement in program, program goes to system when calling, first by the similar finger such as traps
Order, saving scene goes to carry out system to call, and is performed in kernel state, then restoring scene, and each process can have two stacks, one
Individual kernel state stack and a User space stack.Kernel state stack will be turned to by User space stack when interrupting and performing.Needed when system is called
Enter the switching of broker's storehouse.And kernel code is distrusted, it is necessary to carry out extra inspection user.The return course that system is called
There are many extra works, such as check the need for scheduling etc..But with continuing to develop for computer technology, interrupt resources are more next
More the bottleneck as systematic function, replaces, cpu performance more and more higher, and poll thread turns into better way, therefore,
During the embodiment of the present application is to the optimization of driving, interrupt mode is replaced using polling mode, there are three aspect advantages using the method：Subtract
Few occupancy to system break resource；Reduce the influence of system call；Due to accessing external equipment by the way of poll, can be with
Reduce the abnormal influence to system of external equipment.In addition, to be avoided during writing the poll function of network data bag
Use to system call interfaces, can take direct internal memory operation, such as memory address operation, the table handling of internal memory chain, internal memory
Data manipulation, for example, change user's application, using new interface, such as the init for redefining, start, stop, reset,
The interfaces such as read, write perform the access to the network equipment, can so play the effect for improving process performance.
The third processing method of network data packets that the embodiment of the present application is provided, is at above-mentioned second network packet
On the basis of reason method, also including following technical characteristic：
Before the utilization poll thread obtains data from the buffering area of the network equipment, also include：
Each described poll thread is tied on a corresponding CPU.
In this case, according to the core number of current CPU in system, set and be not more than network data
Polling threads, and bound with CPU, it is ensured that each thread is run in a core all the time, and thread can be avoided to exist
The consumption switched between core cpu.
The 4th kind of processing method of network data packets that the embodiment of the present application is provided, be it is above-mentioned the first to the third network
On the basis of data package processing method, also including following technical characteristic：
It is described in the driving of kernel state, be User space memory headroom with the buffering area of the network equipment set up mapping relations it
Before, also include：
Device drives are split as the driving of the kernel state and the User space memory headroom.
Specifically, being exactly to rewrite network device driver, original Device Driver are split into the drive in kernel spaces
The driving two parts in dynamic and User spaces, basis is provided for data are converted into User space from kernel state.
The first network packet processing unit that the embodiment of the present application is provided is as shown in Fig. 2 Fig. 2 is the embodiment of the present application
The schematic diagram of the first the network packet processing unit for providing, the device includes：
Mapping relations set up unit 201, in the driving of kernel state, being User space memory headroom and the network equipment
Buffering area sets up mapping relations, it is necessary to illustrate, network protocol stack no longer carries out multiple data in network data
Replicate, but the data in hardware are copied directly to User space, it is to avoid the copy procedure of kernel mode to User space, do so can
To simplify protocol layer processing procedure, the protocol processes of socket layers of shielding；
Data capture unit 202, for obtaining data from the buffering area of the network equipment, can using interrupt or
The modes such as poll are obtained, and are not intended to limit herein；
Date Conversion Unit 203, for the data of acquisition to be directly changed into User space from kernel state, can be by hardware
Data be copied directly to User space, reduce the expense in terms of unnecessary internal memory；
Data processing unit 204, for being processed the data using CPU, it is to avoid using being in processing procedure
System is called, the expense that reduction is called into system.
Above-mentioned the first the network packet processing unit provided by foregoing description, the embodiment of the present application, it is to avoid
Copy procedure of the kernel mode to User space, can simplify protocol layer processing procedure, the protocol processes of socket layers of shielding, therefore
System resource and the time of each processing data packets occupancy can be reduced, network packet process performance is improved.
Second network packet processing unit that the embodiment of the present application is provided, is at above-mentioned the first network packet
On the basis of reason device, also including following technical characteristic：
The data capture unit specifically for：
Using the poll thread data are obtained from the buffering area of the network equipment.
It is substantial amounts of to notify system by the way of interrupt it should be noted that driving implementation of the prior art
The variation of data, when there is system call statement in program, program goes to system when calling, first by the similar finger such as traps
Order, saving scene goes to carry out system to call, and is performed in kernel state, then restoring scene, and each process can have two stacks, one
Individual kernel state stack and a User space stack.Kernel state stack will be turned to by User space stack when interrupting and performing.Needed when system is called
Enter the switching of broker's storehouse.And kernel code is distrusted, it is necessary to carry out extra inspection user.The return course that system is called
There are many extra works, such as check the need for scheduling etc..But with continuing to develop for computer technology, interrupt resources are more next
More the bottleneck as systematic function, replaces, cpu performance more and more higher, and poll thread turns into better way.Therefore, exist
During the embodiment of the present application is to the optimization of driving, interrupt mode is replaced using polling mode, there are three aspect advantages using the method：Subtract
Few occupancy to system break resource；Reduce the influence of system call；Due to accessing external equipment by the way of poll, can be with
Reduce the abnormal influence to system of external equipment.In addition, to be avoided during writing the poll function of network data bag
Use to system call interfaces, can take direct internal memory operation, such as memory address operation, the table handling of internal memory chain, internal memory
Data manipulation, for example, change user's application, using new interface, such as the init for redefining, start, stop, reset,
The interfaces such as read, write perform the access to the network equipment, can so play the effect for improving process performance.
The third network packet processing unit that the embodiment of the present application is provided, is at above-mentioned second network packet
On the basis of reason device, also including following technical characteristic：
Binding unit, for each described poll thread to be tied into a corresponding CPU.
In this case, according to the core number of current CPU in system, set and be not more than network data
Polling threads, and bound with CPU, it is ensured that each thread is run in a core all the time, and thread can be avoided to exist
The consumption switched between core cpu.
The 4th kind of network packet processing unit that the embodiment of the present application is provided, be it is above-mentioned the first to the third network
In data packet processing on the basis of any one, also including following technical characteristic：
Split cells, for the driving that device drives are split as the kernel state and the User space memory headroom.
Specifically, being exactly to rewrite network device driver, original Device Driver are split into the drive in kernel spaces
The driving two parts in dynamic and User spaces, basis is provided for data are converted into User space from kernel state.
In sum, the embodiment of the present application is provided the above method and device, can reduce the treatment of each network packet
Time, advantage is as follows：The number of times that data are copied in internal memory is reduced, prevents from repeating to copy；The occupancy to system resource is reduced,
Including system hard break, traps and cpu resource etc.；Reduce the abnormal influence to the stability of a system of external equipment；Improve network
Processing data packets performance.
The foregoing description of the disclosed embodiments, enables professional and technical personnel in the field to realize or uses the present invention.
Various modifications to these embodiments will be apparent for those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, the present invention
The embodiments shown herein is not intended to be limited to, and is to fit to and principles disclosed herein and features of novelty phase one
The scope most wide for causing.
Claims (8)
1. a kind of processing method of network data packets, it is characterised in that including：
It is that User space memory headroom sets up mapping relations with the buffering area of the network equipment in the driving of kernel state；
Data are obtained from the buffering area of the network equipment；
The data of acquisition are directly changed into User space from kernel state；
The data are processed using CPU.
2. processing method of network data packets according to claim 1, it is characterised in that described from the slow of the network equipment
Rushing acquisition data in area is：
Using the poll thread data are obtained from the buffering area of the network equipment.
3. processing method of network data packets according to claim 2, it is characterised in that utilize the poll thread described
Before obtaining data from the buffering area of the network equipment, also include：
Each described poll thread is tied on a corresponding CPU.
4. the processing method of network data packets according to claim any one of 1-3, it is characterised in that described in kernel state
It is before User space memory headroom sets up mapping relations with the buffering area of the network equipment, also to include in driving：
Device drives are split as the driving of the kernel state and the User space memory headroom.
5. a kind of network packet processing unit, it is characterised in that including：
Mapping relations set up unit, the buffering area in the driving of kernel state, being User space memory headroom and the network equipment
Set up mapping relations；
Data capture unit, for obtaining data from the buffering area of the network equipment；
Date Conversion Unit, for the data of acquisition to be directly changed into User space from kernel state；
Data processing unit, for being processed the data using CPU.
6. network packet processing unit according to claim 5, it is characterised in that the data capture unit is specifically used
In：
Using the poll thread data are obtained from the buffering area of the network equipment.
7. network packet processing unit according to claim 6, it is characterised in that also include：
Binding unit, for each described poll thread to be tied into a corresponding CPU.
8. the network packet processing unit according to claim any one of 5-7, it is characterised in that also include：
Split cells, for the driving that device drives are split as the kernel state and the User space memory headroom.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611220118.0A CN106713462A (en) | 2016-12-26 | 2016-12-26 | Network data packet processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611220118.0A CN106713462A (en) | 2016-12-26 | 2016-12-26 | Network data packet processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106713462A true CN106713462A (en) | 2017-05-24 |
Family
ID=58903435
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611220118.0A Pending CN106713462A (en) | 2016-12-26 | 2016-12-26 | Network data packet processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106713462A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107749893A (en) * | 2017-11-09 | 2018-03-02 | 北京精强远科技有限公司 | The quick method for receiving and storing data is realized in a kind of shared-file system |
| CN108712308A (en) * | 2018-06-06 | 2018-10-26 | 郑州云海信息技术有限公司 | The method and apparatus that the network equipment is detected in virtual network |
| CN112698934A (en) * | 2019-10-22 | 2021-04-23 | 华为技术有限公司 | Resource scheduling method and device, PMD scheduling device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101267361A (en) * | 2008-05-09 | 2008-09-17 | 武汉飞思科技有限公司 | A high-speed network data packet capturing method based on zero duplication technology |
| CN101478549A (en) * | 2009-01-20 | 2009-07-08 | 电子科技大学 | Operation method for memory sharing media server and functional module construction |
| CN102326149A (en) * | 2011-07-28 | 2012-01-18 | 华为技术有限公司 | Realization method and apparatus for internal memory migration |
| EP2552081A2 (en) * | 2006-07-10 | 2013-01-30 | Solarflare Communications Inc | Interrupt management |
| CN103731364A (en) * | 2014-01-16 | 2014-04-16 | 赛特斯信息科技股份有限公司 | X86 platform based method for achieving trillion traffic rapid packaging |
-
2016
- 2016-12-26 CN CN201611220118.0A patent/CN106713462A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2552081A2 (en) * | 2006-07-10 | 2013-01-30 | Solarflare Communications Inc | Interrupt management |
| CN101267361A (en) * | 2008-05-09 | 2008-09-17 | 武汉飞思科技有限公司 | A high-speed network data packet capturing method based on zero duplication technology |
| CN101478549A (en) * | 2009-01-20 | 2009-07-08 | 电子科技大学 | Operation method for memory sharing media server and functional module construction |
| CN102326149A (en) * | 2011-07-28 | 2012-01-18 | 华为技术有限公司 | Realization method and apparatus for internal memory migration |
| CN103731364A (en) * | 2014-01-16 | 2014-04-16 | 赛特斯信息科技股份有限公司 | X86 platform based method for achieving trillion traffic rapid packaging |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107749893A (en) * | 2017-11-09 | 2018-03-02 | 北京精强远科技有限公司 | The quick method for receiving and storing data is realized in a kind of shared-file system |
| CN108712308A (en) * | 2018-06-06 | 2018-10-26 | 郑州云海信息技术有限公司 | The method and apparatus that the network equipment is detected in virtual network |
| CN108712308B (en) * | 2018-06-06 | 2021-11-26 | 郑州云海信息技术有限公司 | Method and device for detecting network equipment in virtual network |
| CN112698934A (en) * | 2019-10-22 | 2021-04-23 | 华为技术有限公司 | Resource scheduling method and device, PMD scheduling device, electronic equipment and storage medium |
| CN112698934B (en) * | 2019-10-22 | 2023-12-15 | 华为技术有限公司 | Resource scheduling method and device, PMD scheduling device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105577567B (en) | Network packet method for parallel processing based on Intel DPDK | |
| US10439987B2 (en) | Systems and methods for securing network traffic flow in a multi-service containerized application | |
| CN110535813B (en) | Method and device for processing coexistence of kernel mode protocol stack and user mode protocol stack | |
| CN103200128B (en) | The methods, devices and systems that a kind of network packet processes | |
| CN106776395B (en) | A kind of method for scheduling task and device of shared cluster | |
| CN103064742A (en) | Automatic deployment system and method of hadoop cluster | |
| CN108494817A (en) | Data transmission method, relevant apparatus and system | |
| CN110677305A (en) | Automatic scaling method and system in cloud computing environment | |
| CN109981403A (en) | Virtual machine network data traffic monitoring method and device | |
| CN103312720A (en) | Data transmission method, equipment and system | |
| CN102662740A (en) | Asymmetric multi-core system and realization method thereof | |
| US20230080588A1 (en) | Mqtt protocol simulation method and simulation device | |
| CN106713462A (en) | Network data packet processing method and device | |
| CN108512782A (en) | Accesses control list is grouped method of adjustment, the network equipment and system | |
| CN109739668A (en) | A kind of method of data transmission between virtual machines, system and equipment | |
| US20140068165A1 (en) | Splitting a real-time thread between the user and kernel space | |
| CN103618762A (en) | System and method for enterprise service bus state pretreatment based on AOP | |
| CN107402782A (en) | A kind of method and device for being used to load plug-in unit in live software | |
| US20230028837A1 (en) | Scaling for split-networking datapath | |
| CN105025063A (en) | Online data transmission method for large-scale terminal equipment | |
| CN106131162B (en) | A method of network service agent is realized based on IOCP mechanism | |
| US11784946B2 (en) | Method for improving data flow and access for a neural network processor | |
| CN112511633B (en) | Method, system, equipment and medium for block transmission of massive small files | |
| CN112492055A (en) | Method, device and equipment for redirecting transmission protocol and readable storage medium | |
| CN102902593A (en) | Protocol distribution processing system based on cache mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170524 |