CN108270676B - Network data processing method and device based on Intel DPDK - Google Patents

Network data processing method and device based on Intel DPDK Download PDF

Info

Publication number
CN108270676B
CN108270676B CN201611267423.5A CN201611267423A CN108270676B CN 108270676 B CN108270676 B CN 108270676B CN 201611267423 A CN201611267423 A CN 201611267423A CN 108270676 B CN108270676 B CN 108270676B
Authority
CN
China
Prior art keywords
dpdk
address
message
addresses
protocol stack
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201611267423.5A
Other languages
Chinese (zh)
Other versions
CN108270676A (en
Inventor
陈晓科
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Potevio Information Technology Co Ltd
Original Assignee
Potevio Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Potevio Information Technology Co Ltd filed Critical Potevio Information Technology Co Ltd
Priority to CN201611267423.5A priority Critical patent/CN108270676B/en
Publication of CN108270676A publication Critical patent/CN108270676A/en
Application granted granted Critical
Publication of CN108270676B publication Critical patent/CN108270676B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00Routing or path finding of packets in data switching networks
    • H04L45/74Address processing for routing
    • H04L45/745Address table lookup; Address filtering
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00Routing or path finding of packets in data switching networks
    • H04L45/54Organization of routing tables
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00Network arrangements, protocols or services for addressing or naming
    • H04L61/50Address allocation
    • H04L61/5007Internet protocol [IP] addresses

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention provides a network data processing method and a device based on Intel DPDK, wherein the method comprises the following steps: constructing an Intel DPDK operating environment of a data surface development kit; adding one or more IP addresses by using a DPDK, and establishing a routing information table corresponding to the one or more IP addresses; and forwarding the message received by the IP address to a corresponding working thread of the DPDK for processing, and sending the message sent by the IP address to a corresponding destination address in the routing information table. The invention adopts the method of adding IP nodes, realizes the p2p access of all IP multi-nodes, expands the data packet interactive channel of the cross-server, and can conveniently establish a DPDK-based multi-server low-cost high-efficiency interconnected distributed large system.

Description

Network data processing method and device based on Intel DPDK
Technical Field
The invention relates to the technical field of data processing, in particular to a network data processing method and device based on Intel DPDK.
Background
The DPDK (Data Plane Development Kit) is a software Development Kit of a high-speed network Data packet proposed by Intel corporation, and has been sourced, a system adopting an Intel CPU and a network card is mainly supported in the early stage, other CPUs such as IBM Power, Cavium and the like and a part of non-Intel network cards are supported, the DPDK directly reads and writes the network card through a bypass L inux system network protocol stack, and the wire speed transceiving under the packet flow of a 10Gbit network can be realized by combining the binding of different cores in a multi-core CPU.
As shown in fig. 1, in the conventional DPDK model, a data plane and a control plane are separated, the control plane is reserved for L inux system itself and bound to a separate CPU core, the data plane is handed to an application program, and an environment without thread scheduling, system call and network card interrupt is created for the data plane, so that efficient analysis and parallel processing of data are realized.
However, this DPDK application method is only suitable for DPDK of a single server, and when a large service area system is built using a plurality of servers, it is difficult to expand the system scale. For example, there is no suitable interaction mode for the message interaction between multiple servers, the DPDK between servers or the DPDK message interaction from a protocol stack to another server. And a plurality of modules between the operating system protocol stack and the DPDK in the same server share one channel, and the other modules can be influenced by the fact that one module does not release the buffer. In addition, the mode of 'message passing pointer' easily causes the management confusion of the pointer and the memory risk is increased, thereby causing the stability of the system to be reduced.
Disclosure of Invention
In view of the above, the present invention is proposed to provide a method and an apparatus for network data processing based on Intel DPDK, which overcome or at least partially solve the above problems.
In a first aspect, the present invention provides a network data processing method based on Intel DPDK, where the method includes:
constructing an Intel DPDK operating environment of a data surface development kit;
adding one or more IP addresses by using a DPDK, and establishing a routing information table corresponding to the one or more IP addresses;
and forwarding the message received by the IP address to a corresponding working thread of the DPDK for processing, and sending the message sent by the IP address to a corresponding destination address in the routing information table.
Optionally, the adding one or more IP addresses by using the DPDK, and establishing a routing information table corresponding to the one or more IP addresses includes:
adding one or more IP addresses on a data path between a physical network port and an IP protocol stack of an operating system;
sending an address resolution protocol (arp) request broadcast message to the operating system IP protocol stack and the physical network port, and sending an arp response message to the operating system IP protocol stack received by the IP address or an arp request sent by the physical network port;
and establishing a routing information table corresponding to the one or more IP addresses according to the arp request broadcast message and the arp response message.
When the IP node is increased by the method, the broadcast arp and the arp message of which the destination address is not the IP/MAC added to the DPDK at the position need to be transmitted through and can not be cut off in the DPDK, otherwise, other IP nodes are influenced.
Optionally, the adding one or more IP addresses by using the DPDK, and establishing a routing information table corresponding to the one or more IP addresses includes:
when the port parameter which is transmitted when rte _ kni _ init () is called is larger than the number of physical network ports, adding one or more IP addresses on a data path between the DPDK and an IP protocol stack of an operating system; (different from the former method of adding ip nodes, the virtual network port is added here, and the actual physical network port is not bound);
sending an arp request broadcast message to the operating system IP protocol stack, and sending an arp response message to the arp request received by the IP address and sent by the operating system IP protocol stack;
and establishing a routing information table corresponding to the one or more IP addresses according to the arp request broadcast message and the arp response message.
Optionally, the forwarding the packet received by the IP address to a corresponding working thread of the DPDK for processing, and sending the packet sent by the IP address to a corresponding destination address in the routing information table includes:
each DPDK module interacts with the IP protocol stack of the operating system through one IP address;
when a DPDK module using the IP address needs to send a data packet from a physical network port, the IP address is connected to the physical network port through a brct1 bridge mode, so that the data packet is sent from the physical network port.
Optionally, before adding one or more IP addresses by using the DPDK and establishing a routing information table corresponding to the one or more IP addresses, the method further includes:
and creating a plurality of working threads and performing parallel processing on the network data packets.
Optionally, the creating a plurality of work threads and performing parallel processing on the network data packet includes:
allocating an RX memory ring for each first working thread in charge of processing messages, and allocating an RX memory ring and a TX memory ring for each second working thread in charge of receiving and sending messages;
when a first working thread sends a message, storing the message through a memory cache mbuf, and sending a data mbuf pointer to a TX memory ring of a second working thread so as to send the message through the second working thread;
and when the second working thread receives the message, forwarding the message to the RX memory ring of the first working thread.
In a second aspect, the present invention provides a network data processing apparatus based on Intel DPDK, where the apparatus includes:
the operating environment construction unit is used for constructing an Intel DPDK operating environment of a data surface development suite;
the device comprises an IP address adding unit, a routing information table establishing unit and a control unit, wherein the IP address adding unit is used for adding one or more IP addresses by using a DPDK and establishing the routing information table corresponding to the one or more IP addresses;
and the message processing unit is used for forwarding the message received by the IP address to a corresponding working thread of the DPDK for processing, and sending the message sent by the IP address to a corresponding destination address in the routing information table.
Optionally, the IP address adding unit is specifically configured to:
adding one or more IP addresses on a data path between a physical network port and an IP protocol stack of an operating system;
sending an address resolution protocol (arp) request broadcast message to the operating system IP protocol stack and the physical network port, and sending an arp response message to the operating system IP protocol stack received by the IP address or an arp request sent by the physical network port;
and establishing a routing information table corresponding to the one or more IP addresses according to the arp request broadcast message and the arp response message.
Optionally, the IP address adding unit is specifically configured to:
when the port parameter which is transmitted when rte _ kni _ init () is called is larger than the number of physical network ports, adding one or more IP addresses on a data path between the DPDK and an IP protocol stack of an operating system;
sending an arp request broadcast message to the operating system IP protocol stack, and sending an arp response message to the arp request received by the IP address and sent by the operating system IP protocol stack;
and establishing a routing information table corresponding to the one or more IP addresses according to the arp request broadcast message and the arp response message.
Optionally, the packet processing unit is specifically configured to:
each DPDK module interacts with the IP protocol stack of the operating system through one IP address;
when a DPDK module using the IP address needs to send a data packet from a physical network port, the IP address is connected to the physical network port through a brct1 bridge mode, so that the data packet is sent from the physical network port.
According to the technical scheme, the Intel DPDK-based network data processing method and the device provided by the invention have the advantages that an Intel DPDK operating environment is constructed; adding one or more IP addresses by using a DPDK, and establishing a routing information table corresponding to the one or more IP addresses; and forwarding the message received by the IP address to a corresponding working thread of the DPDK for processing, and sending the message sent by the IP address to a corresponding destination address in the routing information table. Therefore, the method for adding the IP nodes is adopted, the p2p access of all IP multi-nodes is realized, the data packet interaction channel of the cross-server is expanded, and a DPDK-based multi-server low-cost high-efficiency interconnected distributed large system can be conveniently established; virtual ip nodes can be added by using the DPDK, so that each module independently has the ip nodes and independently has mbuf receiving buffer, and other modules cannot be influenced by the buffer management problem of one module.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a DPDK model in the prior art;
fig. 2 is a schematic flowchart of a network data processing method based on Intel DPDK according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating a network data processing method based on Intel DPDK according to another embodiment of the present invention;
fig. 4 is a schematic structural diagram of an all-IP DPDK model 1 according to another embodiment of the present invention;
fig. 5 is a flowchart illustrating a network data processing method based on Intel DPDK according to another embodiment of the present invention;
fig. 6 is a schematic structural diagram of an all-IP DPDK model 2 according to another embodiment of the present invention;
fig. 7 is a schematic structural diagram of a network data processing apparatus based on an Intel DPDK according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a network data processing apparatus based on an Intel DPDK according to another embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 2 is a schematic flowchart of a network data processing method based on Intel DPDK in an embodiment of the present invention, and as shown in fig. 2, the method includes:
s1: and constructing an Intel DPDK operating environment of a data surface development suite.
Specifically, the constructing of the Intel DPDK operating environment specifically includes:
setting a kernel starting parameter isoLCpus as 1-n (n is the serial number of the last CPU core from 0), so that the reserved CPU cores are specially used for processing DPDK data plane transactions; setting the CPU affinity of the interrupt to close the CPU core interrupt of the DPDK to be operated; setting a huge page memory; loading an Intel network card UIO kernel driving module and an kni kernel module; binding the related physical network port to an Intel UIO driver; calling rte _ kni _ init () starts the network working mechanism of DPDK.
S2: one or more IP addresses are added by utilizing the DPDK, and a routing information table corresponding to the one or more IP addresses is established.
Specifically, the method comprises two schemes: firstly, one or more ip/mac addresses are added in a data path between a physical network port and a protocol stack, so that a plurality of ip nodes aiming at DPDK are added in a full-switched network, and all nodes can conveniently carry out direct point-to-point message interaction; secondly, when rte _ kni _ init () is called, more numbers than the actual physical network ports are transmitted, one or more IP addresses are added between the DPDK and the operating system IP protocol stack, so as to generate a virtual network port for each DPDK module, which connects the operating system protocol stack and the DPDK.
S3: and forwarding the message received by the IP address to a corresponding working thread of the DPDK for processing, and sending the message sent by the IP address to a corresponding destination address in the routing information table.
Specifically, the DPDK module may perform packet interaction through the added one or more IP addresses.
Therefore, the Intel DPDK operating environment is constructed in the embodiment; adding one or more IP addresses by using a DPDK, and establishing a routing information table corresponding to the one or more IP addresses; and forwarding the message received by the IP address to a corresponding working thread of the DPDK for processing, and sending the message sent by the IP address to a corresponding destination address in the routing information table. Therefore, the method for adding the IP nodes is adopted in the embodiment, the p2p access of all IP multi-nodes is realized, the data packet interaction channel of the cross-server is expanded, and a DPDK-based multi-server low-cost high-efficiency interconnected distributed large system can be conveniently established; virtual ip nodes can be added by using the DPDK, so that each module independently has the ip nodes and independently has mbuf receiving buffer, and other modules cannot be influenced by the buffer management problem of one module.
In an optional embodiment of the present invention, before the step S2, the method further includes the following steps:
and creating a plurality of working threads and performing parallel processing on the network data packets.
In this embodiment, a plurality of work threads are created to implement parallel processing of network data packets. When the added ip node receives the message, the message can be forwarded to the corresponding DPDK working thread for processing, and when the DPDK module needs to send the message from the ip, the routing information table is inquired to obtain the source mac and the destination mac, so that the message is sent to the corresponding destination mac. Namely, a plurality of ip nodes aiming at DPDK are further added in the full-switched network, so that all the nodes can conveniently carry out direct point-to-point message interaction.
Fig. 3 is a schematic flowchart of a network data processing method based on Intel DPDK according to another embodiment of the present invention, as shown in fig. 3, the method includes:
s301: and constructing an Intel DPDK operating environment.
S302: and creating a plurality of working threads and performing parallel processing on the network data packets.
Specifically, the method specifically comprises the following steps: allocating an RX memory ring for each first working thread in charge of processing messages, and allocating an RX memory ring and a TX memory ring for each second working thread in charge of receiving and sending messages; when a first working thread sends a message, storing the message through a memory cache mbuf, and sending a data mbuf pointer to a TX memory ring of a second working thread so as to send the message through the second working thread; and when the second working thread receives the message, forwarding the message to the RX memory ring of the first working thread. The method of separating the message sending and receiving and processing has the maximum flexibility while obtaining high efficiency.
S303: one or more IP addresses are added on the data path between the physical portal and the operating system IP protocol stack.
As shown in fig. 4, after adding one or more IP addresses to a data path between a physical network port and an operating system IP protocol stack, the full IP DPDK model 1 corresponding to this embodiment is obtained, as shown in fig. 4, and the network port corresponding to the newly added IP address is presented facing to both directions of the operating system protocol stack and the physical network port.
S304: and sending an address resolution protocol arp request broadcast message to the operating system IP protocol stack and the physical network port, and sending an arp response message to the operating system IP protocol stack received by the IP address or the arp request sent by the physical network port.
Specifically, a worker thread responsible for accepting/sending a message periodically sends an arp broadcast to both the operating system IP protocol stack and the physical port (generally, a free arp, which is sent only for an IP (named ADDED _ IP) address ADDED to a DPDK on the port, and each ADDED _ IP is bound to a different mac).
It should be understood that, in this embodiment, the network port corresponding to the added IP address is oriented to the operating system IP protocol stack and the physical network port, so that the arp broadcast message is sent to the operating system IP protocol stack and the physical network port, and the received arp request may be sent by the operating system IP protocol stack or the physical network port.
It should be noted that, for an arp packet received from any direction, the broadcast arp and the arp packet whose destination address is not the ip/MAC added to the DPDK at this point, transparent transmission is required and cannot be cut off in the DPDK, otherwise other ip nodes will be affected.
S305: and establishing a routing information table corresponding to the one or more IP addresses according to the arp request broadcast message and the arp response message.
Specifically, a DPDK routing information table is established according to the arp message, and the state of the ip node in the DPDK routing information table is periodically queried to update the state of the ip node.
S306: and forwarding the message received by the IP address to a corresponding working thread of the DPDK for processing, and sending the message sent by the IP address to a corresponding destination address in the routing information table.
Specifically, a message (from an operating system IP protocol stack or a physical network port) sent to an IP address is distributed to different DPDK modules (each DPDK module corresponds to a working thread) for processing, a DPDK routing information table is queried for each packet sent from an ADDED _ IP to obtain a source mac and a destination mac, and the mac layer is filled in and then sent out.
The method in the embodiment is equivalent to adding a plurality of ip nodes aiming at the DPDK in the full switching network, so that all the nodes can conveniently perform direct point-to-point message interaction, and the method for adding the DPDK ip nodes is adopted to realize that the p2p access of the full ip multi-node can conveniently establish a multi-server large system with low cost.
Fig. 5 is a schematic flowchart of a network data processing method based on Intel DPDK in another embodiment of the present invention, as shown in fig. 5, the method includes:
s501: and constructing an Intel DPDK operating environment.
S502: and creating a plurality of working threads and performing parallel processing on the network data packets.
S503: when the incoming port parameter is greater than the number of physical ports at the time rte _ kni _ init () is called, one or more IP addresses are added on the data path between the DPDK and the operating system IP protocol stack.
Specifically, rte _ kni _ init () is called to transfer more numbers than the actual physical network ports, for example, DPDK actual physical network ports are 3, if 2 virtual network ports are added, 5 is used as a port parameter to transfer rte _ kni _ init (), while rte _ kni _ init () is called to transfer more numbers than the actual physical network ports, rte _ kni _ alloc () is called for each newly added virtual network port to generate a virtual network port { 1 st parameter is transferred to mbuf pool pointer and 3 rd parameter is transferred to NU LL empty pointer } during calling, and one or more ip/mac addresses (this ip is called "DPDK local ip") are added between the physical network port and the protocol stack.
It should be noted that the first parameter passed by rte _ kni _ alloc () (no matter whether a physical port or a virtual port under DPDK generates a network node through the function, and the return value of the function is a handle for performing a transceiving function) may be an independently applied mbuf pool (which cannot appear in rte _ kni _ alloc () of other ports and is exclusive to the virtual port), or may be an mbuf pool shared with other ports.
As shown in fig. 6, one or more IP addresses are added to a data path between a DPDK and an IP protocol stack of an operating system, that is, a virtual network port for communicating the IP protocol stack of the operating system and the DPDK is generated for each DPDK module, so that the full IP DPDK model 1 corresponding to this embodiment is obtained, as shown in fig. 6, the virtual network port corresponding to the newly added IP address is only presented to the IP protocol stack of the operating system, and thus, both arp and ordinary IP transceiving are only presented to the protocol stack.
Understandably, different from the method for adding the ip node in the above embodiment, the virtual network port is added in this embodiment, and the actual physical network port is not bound.
S504: sending an arp broadcast message to the operating system IP protocol stack, and sending an arp response message to an arp request sent by the operating system IP protocol stack and received by the IP address.
Specifically, the independent ip is generated by generating an arp address broadcast through interaction with an arp of a protocol stack.
Understandably, the network port corresponding to the added IP address in this embodiment is only oriented to the IP protocol stack of the operating system, so the arp broadcast message is only sent to the IP protocol stack of the operating system, and the received arp request is only sent by the IP protocol stack of the operating system.
S505: and establishing a routing information table corresponding to the one or more IP addresses according to the arp broadcast message and the arp response message.
S506: and forwarding the message received by the IP address to a corresponding working thread of the DPDK for processing, and sending the message sent by the IP address to a corresponding destination address in the routing information table.
Specifically, the method comprises the following steps: each DPDK module interacts with the IP protocol stack of the operating system through one IP address; when a DPDK module using the IP address needs to send a data packet from a physical network port, the IP address is connected to the physical network port through a brct1 bridge mode, so that the data packet is sent from the physical network port. For example, brctl addifbr0 vtehv 0 is the addition of virtual port vtehv 0 to bridge 0.
In this embodiment, the network ports corresponding to the added IP addresses are all protocol stack-oriented, that is, only packet interaction is performed between the network ports and the IP protocol stack of the operating system, and when a module using the network port needs to send a data packet from a physical network port (for example, send an IP packet to another server), the network port may be connected to an actual physical network port packet output in a brctl bridge manner, or any physical network port packet output that is taken over by a DPDK may be used (because the modules under the DPDK are generally threads under the same process and share resources with each other).
In this embodiment, a method of adding virtual ip nodes by using DPDKs is adopted, so that each DPDK module independently has ip nodes and independently has mbuf receiving buffers, and other modules are not affected by the buffer management problem of one module. Further, the virtual network port generated by using the DPDK may be connected to an actual physical network port in a brctl bridge (virtual bridge) manner, so as to implement an ip full-connectivity system architecture.
Therefore, the embodiment adopts different methods for different risk levels (for example, adding a single virtual ip node for OM and debugging report does not affect the protocol layer message because the protocol layer message is important, and adding an ip node for the service message and the protocol layer message according to the scheme 1), so that different channels can be realized, the stability of the system can be improved, and the problem of positioning and troubleshooting can be facilitated.
Fig. 7 is a schematic structural diagram of a network data processing apparatus based on Intel DPDK in an embodiment of the present invention, as shown in fig. 7, the apparatus includes: an operating environment constructing unit 701, an IP address adding unit 702, and a message processing unit 703. Wherein:
the operating environment construction unit 701 is used for constructing an Intel DPDK operating environment of a data plane development suite; the IP address adding unit 702 is configured to add one or more IP addresses by using a DPDK, and establish a routing information table corresponding to the one or more IP addresses; the message processing unit 703 is configured to forward the message received by the IP address to a corresponding working thread of the DPDK for processing, and send the message sent by the IP address to a corresponding destination address in the routing information table.
In this embodiment, an Intel DPDK operating environment is constructed by the operating environment constructing unit 701; the IP address adding unit 702 adds one or more IP addresses by using DPDK, and establishes a routing information table corresponding to the one or more IP addresses; the message processing unit 703 forwards the message received by the IP address to a corresponding working thread of the DPDK for processing, and sends the message sent by the IP address to a corresponding destination address in the routing information table. Therefore, the method for adding the IP nodes is adopted in the embodiment, the p2p access of all IP multi-nodes is realized, the data packet interaction channel of the cross-server is expanded, and a DPDK-based multi-server low-cost high-efficiency interconnected distributed large system can be conveniently established; virtual ip nodes can be added by using the DPDK, so that each module independently has the ip nodes and independently has mbuf receiving buffer, and other modules cannot be influenced by the buffer management problem of one module.
In an optional embodiment of the present invention, the IP address adding unit 702 is specifically configured to:
adding one or more IP addresses on a data path between a physical network port and an IP protocol stack of an operating system;
sending an address resolution protocol (arp) request broadcast message to the operating system IP protocol stack and the physical network port, and sending an arp response message to the operating system IP protocol stack received by the IP address or an arp request sent by the physical network port;
and establishing a routing information table corresponding to the one or more IP addresses according to the arp request broadcast message and the arp response message.
In an optional embodiment of the present invention, the IP address adding unit 702 is specifically configured to:
when the port parameter which is transmitted when rte _ kni _ init () is called is larger than the number of physical network ports, adding one or more IP addresses on a data path between the DPDK and an IP protocol stack of an operating system;
sending an arp request broadcast message to the operating system IP protocol stack, and sending an arp response message to the arp request received by the IP address and sent by the operating system IP protocol stack;
and establishing a routing information table corresponding to the one or more IP addresses according to the arp request broadcast message and the arp response message.
Further, the message processing unit 703 is specifically configured to:
each DPDK module interacts with the IP protocol stack of the operating system through one IP address;
when a DPDK module using the IP address needs to send a data packet from a physical network port, the IP address is connected to the physical network port through a brct1 bridge mode, so that the data packet is sent from the physical network port.
In an alternative embodiment of the present invention, as shown in fig. 8, the apparatus further comprises: a worker thread creation unit 704 configured to:
and creating a plurality of working threads and performing parallel processing on the network data packets.
In an optional embodiment of the present invention, the worker thread creating unit 704 is specifically configured to:
allocating an RX memory ring for each first working thread in charge of processing messages, and allocating an RX memory ring and a TX memory ring for each second working thread in charge of receiving and sending messages;
when a first working thread sends a message, storing the message through a memory cache mbuf, and sending a data mbuf pointer to a TX memory ring of a second working thread so as to send the message through the second working thread;
and when the second working thread receives the message, forwarding the message to the RX memory ring of the first working thread.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
In the description of the present invention, it should be noted that the terms "upper", "lower", and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, which are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation, and operate, and thus, should not be construed as limiting the present invention. Unless expressly stated or limited otherwise, the terms "mounted," "connected," and "connected" are intended to be inclusive and mean, for example, that they may be fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (6)

1. A network data processing method based on Intel DPDK is characterized by comprising the following steps:
constructing an Intel DPDK operating environment of a data surface development kit;
adding one or more IP addresses by using a DPDK, and establishing a routing information table corresponding to the one or more IP addresses;
forwarding the message received by the IP address to a corresponding working thread of a DPDK for processing, and sending the message sent by the IP address to a corresponding destination address in the routing information table;
the adding one or more IP addresses by using the DPDK, and establishing a routing information table corresponding to the one or more IP addresses includes:
adding one or more IP addresses on a data path between a physical network port and an IP protocol stack of an operating system;
sending an address resolution protocol (arp) request broadcast message to the operating system IP protocol stack and the physical network port, and sending an arp response message to the operating system IP protocol stack received by the IP address or an arp request sent by the physical network port;
establishing a routing information table corresponding to the one or more IP addresses according to the arp request broadcast message and the arp response message;
or comprises the following steps:
when the port parameter which is transmitted when rte _ kni _ init () is called is larger than the number of physical network ports, adding one or more IP addresses on a data path between the DPDK and an IP protocol stack of an operating system;
sending an arp request broadcast message to the operating system IP protocol stack, and sending an arp response message to the arp request received by the IP address and sent by the operating system IP protocol stack;
and establishing a routing information table corresponding to the one or more IP addresses according to the arp request broadcast message and the arp response message.
2. The method according to claim 1, wherein when the DPDK is used to add one or more IP addresses and establish a routing information table corresponding to the one or more IP addresses, the method includes: when the port parameter which is transmitted when rte _ kni _ init () is called is larger than the number of physical network ports, adding one or more IP addresses on a data path between the DPDK and an IP protocol stack of an operating system; sending an arp request broadcast message to the operating system IP protocol stack, and sending an arp response message to the arp request received by the IP address and sent by the operating system IP protocol stack; when a routing information table corresponding to the one or more IP addresses is established according to the arp request broadcast message and the arp response message, the forwarding the message received by the IP address to a corresponding working thread of the DPDK for processing, and sending the message sent by the IP address to a corresponding destination address in the routing information table includes:
each DPDK module interacts with the IP protocol stack of the operating system through one IP address;
when a DPDK module using the IP address needs to send a data packet from a physical network port, the IP address is connected to the physical network port through a brct1 bridge mode, so that the data packet is sent from the physical network port.
3. The method according to any of claims 1-2, wherein before adding one or more IP addresses using DPDK and establishing a routing information table corresponding to the one or more IP addresses, the method further comprises:
and creating a plurality of working threads and performing parallel processing on the network data packets.
4. The method of claim 3, wherein creating the plurality of worker threads for parallel processing of the network packet comprises:
allocating an RX memory ring for each first working thread in charge of processing messages, and allocating an RX memory ring and a TX memory ring for each second working thread in charge of receiving and sending messages;
when a first working thread sends a message, storing the message through a memory cache mbuf, and sending a data mbuf pointer to a TX memory ring of a second working thread so as to send the message through the second working thread;
and when the second working thread receives the message, forwarding the message to the RX memory ring of the first working thread.
5. An Intel DPDK-based network data processing apparatus, comprising:
the operation environment construction unit is used for constructing an Intel DPDK operation environment of a data surface development suite;
the device comprises an IP address adding unit, a routing information table establishing unit and a control unit, wherein the IP address adding unit is used for adding one or more IP addresses by using a DPDK and establishing the routing information table corresponding to the one or more IP addresses;
the message processing unit is used for forwarding the message received by the IP address to a corresponding working thread of the DPDK for processing, and sending the message sent by the IP address to a corresponding destination address in the routing information table;
the IP address adding unit is specifically configured to:
adding one or more IP addresses on a data path between a physical network port and an IP protocol stack of an operating system;
sending an address resolution protocol (arp) request broadcast message to the operating system IP protocol stack and the physical network port, and sending an arp response message to the operating system IP protocol stack received by the IP address or an arp request sent by the physical network port;
establishing a routing information table corresponding to the one or more IP addresses according to the arp request broadcast message and the arp response message;
or specifically for:
when the port parameter which is transmitted when rte _ kni _ init () is called is larger than the number of physical network ports, adding one or more IP addresses on a data path between the DPDK and an IP protocol stack of an operating system;
sending an arp request broadcast message to the operating system IP protocol stack, and sending an arp response message to the arp request received by the IP address and sent by the operating system IP protocol stack;
and establishing a routing information table corresponding to the one or more IP addresses according to the arp request broadcast message and the arp response message.
6. The apparatus according to claim 5, wherein when the IP address adding unit is specifically configured to: when the port parameter which is transmitted when rte _ kni _ init () is called is larger than the number of physical network ports, adding one or more IP addresses on a data path between the DPDK and an IP protocol stack of an operating system; sending an arp request broadcast message to the operating system IP protocol stack, and sending an arp response message to the arp request received by the IP address and sent by the operating system IP protocol stack; when a routing information table corresponding to the one or more IP addresses is established according to the arp request broadcast message and the arp response message, the packet processing unit is specifically configured to:
each DPDK module interacts with the IP protocol stack of the operating system through one IP address;
when a DPDK module using the IP address needs to send a data packet from a physical network port, the IP address is connected to the physical network port through a brct1 bridge mode, so that the data packet is sent from the physical network port.
CN201611267423.5A 2016-12-31 2016-12-31 Network data processing method and device based on Intel DPDK Expired - Fee Related CN108270676B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611267423.5A CN108270676B (en) 2016-12-31 2016-12-31 Network data processing method and device based on Intel DPDK

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611267423.5A CN108270676B (en) 2016-12-31 2016-12-31 Network data processing method and device based on Intel DPDK

Publications (2)

Publication Number Publication Date
CN108270676A CN108270676A (en) 2018-07-10
CN108270676B true CN108270676B (en) 2020-07-28

Family

ID=62770342

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611267423.5A Expired - Fee Related CN108270676B (en) 2016-12-31 2016-12-31 Network data processing method and device based on Intel DPDK

Country Status (1)

Country Link
CN (1) CN108270676B (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110798366B (en) * 2018-08-01 2023-02-24 阿里巴巴集团控股有限公司 Task logic processing method, device and equipment
CN109669792A (en) * 2018-12-24 2019-04-23 网宿科技股份有限公司 Analyze the method and service server of data message
CN111859062B (en) * 2019-04-30 2023-09-22 大唐移动通信设备有限公司 Network data processing method and device based on DPDK
CN112506674B (en) * 2019-09-16 2024-02-02 北京华耀科技有限公司 System and method for communication between user mode TCP/IP protocol stack and local application in Linux system
CN111049762A (en) * 2019-12-23 2020-04-21 上海金仕达软件科技有限公司 Data acquisition method and device, storage medium and switch
CN114422489B (en) * 2020-10-13 2024-01-26 中国电信股份有限公司 Information transmission method and system based on service grid
CN113691619B (en) * 2021-08-24 2023-01-10 北京天融信网络安全技术有限公司 Message processing method and device, electronic equipment and storage medium
CN113672410B (en) * 2021-08-25 2023-08-25 北京天融信网络安全技术有限公司 Data processing method and electronic device
CN114301619B (en) * 2021-11-16 2024-01-30 北京威努特技术有限公司 Industrial control firewall without hardware management port and implementation method of simulation management port of industrial control firewall
CN115242895B (en) * 2022-07-19 2023-04-18 杭州迪普科技股份有限公司 DPDK-based local access method and device
CN115412502B (en) * 2022-11-02 2023-03-24 之江实验室 Network port expansion and message rapid equalization processing method
CN116170404B (en) * 2023-02-17 2023-09-29 通明智云(北京)科技有限公司 DPDK-based data forwarding method and device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105099957A (en) * 2015-08-20 2015-11-25 电子科技大学 Data packet forwarding method based on software table look-up
CN105357151A (en) * 2015-11-19 2016-02-24 成都科来软件有限公司 DPDK-based packet capture and mirror image flow forwarding method
CN105577567A (en) * 2016-01-29 2016-05-11 国家电网公司 Network data packet parallel processing method based on Intel DPDK
CN105939349A (en) * 2016-05-25 2016-09-14 电子科技大学 Method for realizing follow-up safe access of user data
CN106209506A (en) * 2016-06-30 2016-12-07 瑞斯康达科技发展股份有限公司 A kind of virtualization deep-packet detection flow analysis method and system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160275026A1 (en) * 2015-03-20 2016-09-22 Intel Corporation Weakly ordered doorbell

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105099957A (en) * 2015-08-20 2015-11-25 电子科技大学 Data packet forwarding method based on software table look-up
CN105357151A (en) * 2015-11-19 2016-02-24 成都科来软件有限公司 DPDK-based packet capture and mirror image flow forwarding method
CN105577567A (en) * 2016-01-29 2016-05-11 国家电网公司 Network data packet parallel processing method based on Intel DPDK
CN105939349A (en) * 2016-05-25 2016-09-14 电子科技大学 Method for realizing follow-up safe access of user data
CN106209506A (en) * 2016-06-30 2016-12-07 瑞斯康达科技发展股份有限公司 A kind of virtualization deep-packet detection flow analysis method and system

Also Published As

Publication number Publication date
CN108270676A (en) 2018-07-10

Similar Documents

Publication Publication Date Title
CN108270676B (en) Network data processing method and device based on Intel DPDK
CN102334112B (en) Method and system for virtual machine networking
US10698717B2 (en) Accelerator virtualization method and apparatus, and centralized resource manager
EP2880828B1 (en) System and method for virtual ethernet interface binding
EP3355553A1 (en) Reliable load-balancer using segment routing and real-time application monitoring
CN109408257B (en) Data transmission method and device for Network On Chip (NOC) and electronic equipment
CN107278359A (en) The method of Message processing, main frame and system in cloud computing system
CN102571587B (en) Method and equipment for forwarding messages
KR20130128461A (en) Communication control system, switch node, and communication control method
JP2020520612A (en) Packet transmission method, edge device, and machine-readable storage medium
EP3588869B1 (en) Technologies for hairpinning network traffic
CN102857414A (en) Forwarding table writing method and device and message forwarding method and device
CN102307141B (en) Message forwarding method and device
CN111211999A (en) OVS-based real-time virtual network implementation method
CN113472624A (en) Method for realizing virtual network data packet forwarding based on vDPA and application
JP7046983B2 (en) Packet transmission method and equipment
CN111371694A (en) Shunting method, device and system, processing equipment and storage medium
CN106850440B (en) Router, routing method and chip for multi-address shared data routing packet
US20240022501A1 (en) Data Packet Sending Method and Device
WO2014101502A1 (en) Memory access processing method based on memory chip interconnection, memory chip, and system
CN107483628B (en) DPDK-based one-way proxy method and system
CN106790162B (en) Virtual network optimization method and system
KR101924002B1 (en) Chip multi processor and router for chip multi processor
CN101262425A (en) Multi-cast forward method based on network address translation
CN111147611B (en) Electronic device providing fast packet forwarding with reference to additional network address translation tables

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20200728

Termination date: 20211231