CN118055068A - Message processing method, device, equipment and medium based on DPDK - Google Patents
Message processing method, device, equipment and medium based on DPDK Download PDFInfo
- Publication number
- CN118055068A CN118055068A CN202211435253.2A CN202211435253A CN118055068A CN 118055068 A CN118055068 A CN 118055068A CN 202211435253 A CN202211435253 A CN 202211435253A CN 118055068 A CN118055068 A CN 118055068A
- Authority
- CN
- China
- Prior art keywords
- message
- processed
- core
- cpu
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 21
- 238000012545 processing Methods 0.000 claims abstract description 167
- 238000000034 method Methods 0.000 claims abstract description 56
- 230000008569 process Effects 0.000 claims description 33
- 230000008859 change Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 17
- 230000010355 oscillation Effects 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 230000003287 optical effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention discloses a message processing method, device, equipment and medium based on DPDK. Wherein the method comprises the following steps: receiving a message to be processed through a first CPU core, and sending the message to be processed to a second CPU core; judging whether a message to be processed has a corresponding message forwarding table or not through a CPU second core; if the message does not exist, the first service processing is carried out on the message to be processed through the CPU second check, and forwarding processing judgment is carried out on the message to be processed according to the first service processing result; if yes, the message to be processed is forwarded back to the first core of the CPU through the second core of the CPU, and second business processing is carried out through the third core of the CPU according to the judging result of whether the message to be processed needs to be processed or not; the real-time performance of the first service and the message forwarding is higher than that of the second service. According to the technical scheme, the problems of message forwarding flow oscillation and large bandwidth flow forwarding fluctuation can be effectively solved, so that the message forwarding efficiency and the message forwarding stability are improved.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a medium for processing a packet based on DPDK.
Background
DPDK (DATA PLANE Development Kit) is an architecture provided in data plane applications for fast processing of data packets. Two forwarding models existing for DPDK are divided into run to completion model (fig. 1A) and pipeline model (fig. 1B). In either of the above models, the implementation mechanism of the DPDK needs to process core isolation and process core binding, and in the packet traffic scenario of increasing the number of service modules or large bandwidth, the CPU with a small number of processing cores will generate serious forwarding performance degradation and forwarding traffic bandwidth jitter and oscillation, which brings huge loss to the service.
In the related art, a DPDK data packet distribution mode is adopted to perform dynamic load balancing of traffic. However, this solution has high requirements on CPU performance, requires at least 5 cores of CPU, and has high cost. In addition, when the operations such as configuration change and the like caused by the increase of DPDK service modules easily cause the problems of forwarding flow oscillation and large bandwidth flow forwarding fluctuation of a CPU, the efficiency and stability of message forwarding are poor, and the practical application requirements cannot be met. Therefore, how to effectively improve the efficiency and stability of message forwarding is one of the problems to be solved in the current message processing technology.
Disclosure of Invention
The invention provides a message processing method, device, equipment and medium based on DPDK, which can effectively solve the problems of forwarding flow oscillation and large bandwidth flow forwarding fluctuation generated by a CPU, thereby improving the message forwarding efficiency and the message forwarding stability of the CPU.
According to an aspect of the present invention, there is provided a DPDK-based message processing method, including:
Receiving a message to be processed through a first CPU core, and sending the message to be processed to a second CPU core;
Judging whether the message to be processed has a corresponding message forwarding table or not through the CPU second core;
if not, the CPU performs a second check on the message to be processed to perform a first service processing, and performs forwarding processing judgment on the message to be processed according to a first service processing result;
If yes, the message to be processed is transmitted back to the first CPU core through the second CPU core, and second service processing is performed through the third CPU core according to the judging result of whether the message to be processed needs to be subjected to the second service processing or not;
the real-time performance of the first service and the message forwarding is higher than that of the second service.
According to another aspect of the present invention, there is provided a DPDK-based message processing apparatus, including:
The message to be processed sending module is used for receiving a message to be processed through the first core of the CPU and sending the message to be processed to the second core of the CPU;
the message forwarding table judging module is used for judging whether the message to be processed has a corresponding message forwarding table or not through the CPU second core;
The first service processing module is used for carrying out first service processing on the message to be processed through the CPU second check if the message to be processed does not exist, and carrying out forwarding processing judgment on the message to be processed according to a first service processing result;
The second service processing module is used for forwarding the message to be processed back to the first CPU core through the second CPU core if the message to be processed exists, and performing second service processing through the third CPU core according to a judging result of whether the message to be processed needs to be subjected to the second service processing or not;
the real-time performance of the first service and the message forwarding is higher than that of the second service.
According to another aspect of the present invention, there is provided a DPDK-based message processing electronic device, including:
At least one processor; wherein the processor includes at least four CPU cores;
And a memory communicatively coupled to the at least one processor; wherein,
The memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the DPDK based message processing method according to any one of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement a DPDK based message processing method according to any one of the embodiments of the present invention when executed.
According to the technical scheme, the first core of the CPU receives a message to be processed and sends the message to be processed to the second core of the CPU; judging whether a message to be processed has a corresponding message forwarding table or not through a CPU second core; if the message does not exist, the first service processing is carried out on the message to be processed through the CPU second check, and forwarding processing judgment is carried out on the message to be processed according to the first service processing result; if yes, the message to be processed is forwarded back to the first core of the CPU through the second core of the CPU, and second business processing is carried out through the third core of the CPU according to the judging result of whether the message to be processed needs to be processed or not; the real-time performance of the first service and the message forwarding is higher than that of the second service. According to the technical scheme, the problems of forwarding flow oscillation and large bandwidth flow forwarding fluctuation generated by the CPU can be effectively solved, so that the message processing performance of the CPU and the stability of message forwarding are improved, and the actual application requirements can be better met.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1A is a schematic diagram of a run to completion model in DPDK according to the prior art;
Fig. 1B is a schematic diagram of a pipeline model in DPDK according to the prior art;
Fig. 2 is a flowchart of a DPDK-based message processing method according to a first embodiment of the present invention;
fig. 3 is a schematic diagram of an optimized DPDK forwarding model according to one embodiment of the present invention;
Fig. 4 is a flowchart of a DPDK-based message processing method according to a second embodiment of the present invention;
FIG. 5 is a flowchart of a business process refreshing process according to a second embodiment of the present invention;
fig. 6 is a flowchart of a white list matching process according to a second embodiment of the present invention;
Fig. 7 is a schematic structural diagram of a DPDK-based message processing apparatus according to a third embodiment of the present invention;
Fig. 8 is a schematic structural diagram of an electronic device implementing a DPDK-based message processing method according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," "target," and the like in the description and claims of the present invention and in the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 2 is a flowchart of a DPDK-based message processing method according to an embodiment of the present invention, where the method may be performed by a DPDK-based message processing apparatus, and the DPDK-based message processing apparatus may be implemented in hardware and/or software, and the DPDK-based message processing apparatus may be configured in an electronic device with data processing capability. As shown in fig. 2, the method includes:
S110, receiving the message to be processed through the first core of the CPU, and sending the message to be processed to the second core of the CPU.
Wherein the CPU first core may be configured to receive a message from a physical port. The message to be processed may refer to a message waiting to be processed. The second core of CPU can be used for receiving the message sent by the first core of CPU, and carrying out the first service processing on the received message. The first service may refer to a service that is strongly related to the real-time forwarding of the message.
In this embodiment, a polling mode may be adopted to receive a message to be processed from a physical port through the CPU first core, so as to complete the transceiving of a plurality of data packets at one time. It should be noted that, the network card receiving and transmitting packet descriptor is generally 16 bytes or 32 bytes, the CPU buffer basic unit is 64 bytes, and 2 or 4 descriptors can be stored. Adjacent caches can be accessed each time by the CPU anticipation mechanism, and if 4 cache locations are processed at a time, 16 descriptors can be updated. After the first core of the CPU receives the message to be processed and performs simple processing, the message to be processed can be packaged into an inter-core ring queue (lock-free queue), and the ring queue is sent to the second core of the CPU.
S120, judging whether the message to be processed has a corresponding message forwarding table or not through a CPU second core.
The message forwarding table may be used to characterize whether the CPU second core forwards the message to be processed for the first time. If the CPU second core forwards the message to be processed for the first time, a message forwarding table corresponding to the message to be processed does not exist; if the CPU second core is not forwarding the message to be processed for the first time, a message forwarding table corresponding to the message to be processed exists.
In this embodiment, after the second core of the CPU receives the ring queue sent by the first core of the CPU, the to-be-processed packet is first read from the ring queue, and then it is determined whether the to-be-processed packet has a corresponding packet forwarding table. Optionally, the judging, by the CPU second core, whether the message to be processed has a corresponding message forwarding table includes: extracting attribute information of the message to be processed through the CPU second check, distributing corresponding message caches for the message to be processed, and storing the attribute information in the message caches; wherein, the attribute information at least comprises message quintuple information and interface information; judging whether the message to be processed has a corresponding message forwarding table or not according to the attribute information.
The attribute information may be used to describe source information and destination information of the message to be processed. Specifically, the attribute information at least includes message quintuple information and interface information. The message quintuple information may include a source IP, a destination IP, a source port, a destination port, and a protocol.
In this embodiment, first, attribute information extraction is performed on a message to be processed through a second core of the CPU, so as to obtain five-tuple information (source IP, destination IP, source port, destination port, protocol) and interface information of the message, and meanwhile, a corresponding message buffer is allocated to the message to be processed (a message to be processed is allocated to a message buffer), and the attribute information is stored in the message buffer, and then, whether the message to be processed has a corresponding message forwarding table can be determined according to the attribute information.
By the arrangement, whether the message to be processed has a corresponding message forwarding table or not can be rapidly and accurately judged according to the attribute information.
And S130, if the message does not exist, performing first service processing on the message to be processed through a CPU second check, and performing forwarding processing judgment on the message to be processed according to a first service processing result.
In this embodiment, if there is no message forwarding table corresponding to the message to be processed, the CPU performs the first service processing on the message to be processed through the second check of the CPU, and performs forwarding processing determination on the message to be processed according to the result of the first service processing. Wherein the first service processing result comprises success or failure. Specifically, if the message to be processed does not have a corresponding message forwarding table, the read message to be processed and the set message buffer memory are sequentially sent to each service module, and according to the functional attribute of each service module, the first service processing is performed on the message to be processed through each service module. If the processing result of each service module is successful, the first service processing result is indicated to be successful; if the processing result of the service module is failure, the first service processing result is failure.
In this embodiment, after the first service processing is performed on the to-be-processed packet, forwarding processing judgment may be performed on the to-be-processed packet according to the result of the first service processing, so as to determine whether forwarding processing is required on the to-be-processed packet. Optionally, forwarding the message to be processed according to the first service processing result includes: if the first service processing result is successful, a message forwarding table is created for the message to be processed according to the first service processing result and attribute information in the message cache, and the message to be processed is forwarded back to the first core of the CPU through the second core of the CPU; otherwise, deleting the message to be processed and the corresponding message buffer.
In this embodiment, if the first service processing result is successful, a processing flag may be set for a packet buffer corresponding to the to-be-processed packet according to the first service processing result, and a packet forwarding table is established at the same time, and the packet forwarding table is set to record attribute information in the packet buffer, and then the to-be-processed packet is packaged into an inter-core ring queue, and the ring queue is forwarded back to the CPU first core through the CPU second core; if the first service processing result is failure, the message to be processed and the corresponding message buffer are deleted directly, so that the CPU resource occupation is reduced, and the CPU resource waste is avoided.
By the arrangement, when the first service processing result is successful, a message forwarding table can be created for the message to be processed, and the message to be processed is forwarded back to the CPU first core; and deleting the message to be processed and the corresponding message buffer when the first service processing result is failure, thereby reducing the CPU resource occupation and avoiding the CPU resource waste.
S140, if yes, the message to be processed is transmitted back to the first core of the CPU through the second core of the CPU, and the second business processing is carried out through the third core of the CPU according to the judging result of whether the message to be processed needs to be processed or not; the real-time performance of the first service and the message forwarding is higher than that of the second service.
The third core of the CPU may be configured to perform a second service processing on the message to be processed. The second service may be understood as a service with low real-time requirement for forwarding the message. It should be noted that, the real-time performance of the first service and the message forwarding is higher than that of the second service.
In this embodiment, if there is a message forwarding table corresponding to the message to be processed, the message to be processed may be forwarded back to the first core of the CPU through the second core of the CPU, and it is determined whether the message to be processed needs to be processed by the second service. If the message to be processed is determined to need to be subjected to the second service processing, the CPU can be used for third checking the message to be processed to carry out the second service processing. Specifically, the third core of the CPU can acquire the message cache in the second core of the CPU in a polling mode, and if the message cache exists, the message cache is sequentially sent to the service module to process the service module again; and if the message cache does not exist, continuing to poll and read the message cache in the second core of the CPU.
In this embodiment, optionally, performing, by the third core of the CPU, second service processing according to a determination result that the message to be processed needs to be subjected to second service processing includes: performing second service processing on the message to be processed according to the message buffer of the message to be processed through a third CPU core; if the second service processing result is successful, updating a message forwarding table corresponding to the message to be processed according to the second service processing result, and deleting the message cache; otherwise, deleting the message forwarding table and the message buffer corresponding to the message to be processed, and ending the forwarding processing of the message to be processed.
In this embodiment, the second service processing may be performed on the to-be-processed message according to the message buffer of the to-be-processed message by the third core of the CPU, and further processing the to-be-processed message according to the result of the second service processing. Wherein the second service processing result may include success or failure. Specifically, if the second service processing result is successful, the message forwarding table corresponding to the message to be processed can be updated according to the second service processing result, and the message cache is deleted; if the second service processing result is failure, deleting a message forwarding table and a message buffer corresponding to the message to be processed, and ending the forwarding processing of the message to be processed.
By the arrangement, the message forwarding table corresponding to the message to be processed can be updated or deleted according to the second service processing result, and the forwarding processing of the message to be processed is finished while the message forwarding table is deleted, so that the accuracy of the forwarding processing of the message to be processed is ensured.
According to the technical scheme, the first core of the CPU receives a message to be processed and sends the message to be processed to the second core of the CPU; judging whether a message to be processed has a corresponding message forwarding table or not through a CPU second core; if the message does not exist, the first service processing is carried out on the message to be processed through the CPU second check, and forwarding processing judgment is carried out on the message to be processed according to the first service processing result; if yes, the message to be processed is forwarded back to the first core of the CPU through the second core of the CPU, and second business processing is carried out through the third core of the CPU according to the judging result of whether the message to be processed needs to be processed or not; the real-time performance of the first service and the message forwarding is higher than that of the second service. According to the technical scheme, the problems of forwarding flow oscillation and large bandwidth flow forwarding fluctuation generated by the CPU can be effectively solved, so that the message forwarding efficiency and the message forwarding stability of the CPU are improved, and the practical application requirements can be better met.
In this embodiment, optionally, the first core and the second core of the CPU belong to a DPDK binding core, and the third core of the CPU belongs to a DPDK unbinding core, where the DPDK binding core is used to process a first DPDK service, and the DPDK unbinding core is used to process a second DPDK service and other non-DPDK services; the first DPDK service comprises a real-time DPDK message forwarding service, and the second DPDK service comprises a service based on message caching.
Fig. 3 is a schematic diagram of an optimized DPDK forwarding model according to a first embodiment of the present invention. As shown in fig. 3, the DPDK forwarding model is obtained by optimizing the existing pipeline forwarding model by taking a 4-core CPU as an example. The physical core 3 is a first core of the CPU, belongs to a DPDK binding core, and may be used to receive and send a message to be processed. The physical core 2 is a second core of the CPU, belongs to a DPDK binding core, and can be used for performing a first service processing on a message to be processed, i.e. processing a service which is strongly related to forwarding real-time of the message to be processed. The physical core 1 is a third core of the CPU, belongs to a DPDK unbinding core, and can be used for performing second service processing on a message to be processed, i.e. processing a service which is not high in real-time performance with the message to be processed. The remaining physical 0 cores may accept user configuration and pass parameters to the data thread, etc., as the master core.
By the arrangement, the second service processing and the message forwarding can be decoupled, so that the message forwarding efficiency is improved.
Example two
Fig. 4 is a flowchart of a DPDK-based message processing method according to a second embodiment of the present invention, where the present embodiment is optimized based on the above embodiments. The concrete optimization is as follows: after judging that the message to be processed has the corresponding message forwarding table through the CPU second core, the method further comprises the following steps: judging whether the message to be processed needs to be subjected to second business processing or not; if the business process is changed, the configuration of the first business is changed and/or the message forwarding table is refreshed, determining that the message to be processed needs to be processed by the second business; the second service is determined according to the service flow change, the configuration change of the first service and/or the message forwarding table refreshing operation.
As shown in fig. 4, the method of this embodiment specifically includes the following steps:
s210, receiving the message to be processed through the first core of the CPU, and sending the message to be processed to the second core of the CPU.
S220, judging whether the message to be processed has a corresponding message forwarding table or not through the CPU second core.
And S230, if the message does not exist, performing first service processing on the message to be processed through a CPU second check, and performing forwarding processing judgment on the message to be processed according to a first service processing result.
The specific implementation of S210-S230 may be referred to in the detailed description of S110-S130, and will not be described herein.
S240, if so, the message to be processed is forwarded back to the first core of the CPU through the second core of the CPU, and whether the message to be processed needs to be processed by the second service is judged.
In this embodiment, if there is a corresponding message forwarding table in the message to be processed, the second core of the CPU is required to further determine whether the message to be processed needs to be processed by the second service processing, while the message to be processed is forwarded back to the first core of the CPU. It should be noted that, if the second service processing needs to be performed, that is, the service module processing needs to be performed again, the message to be processed may be continuously sent at this time, and the message to be processed in the message buffer waits for the third core of the CPU to perform the second service processing.
S250, if the business process change, the configuration change of the first business and/or the message forwarding table refreshing operation occur, determining that the message to be processed needs to be processed by the second business; the second service is determined according to the service flow change, the configuration change of the first service and/or the message forwarding table refreshing operation.
In this embodiment, if any one or more of a service flow change, a configuration change of the first service, and a message forwarding table refreshing operation occurs, it may be determined that the message to be processed needs to be processed by the second service. The second service is determined according to the service flow change, the configuration change of the first service and/or the message forwarding table refreshing operation. Wherein, the configuration change may include addition, deletion, or update of the configuration, etc.
Fig. 5 is a flowchart of a business process refreshing process according to a second embodiment of the present invention. Wherein physical core 2 represents the second core of the CPU, physical core 1 represents the third core of the CPU, and fig. 5 is an example of a 4-core CPU. As shown in fig. 5, first, N IP packets (packets to be processed) sent by the CPU first core are read concurrently by the CPU second core. And then extracting attribute information of the IP message through the CPU second check, distributing a corresponding message buffer for the IP message, and storing the attribute information in the message buffer so as to judge whether the IP message has a corresponding message forwarding table according to the attribute information. If the IP message does not have a corresponding message forwarding table, the read IP message and the set message buffer memory are sequentially sent to each service module (module 1 to module N) for first service processing. After each service module is processed, judging whether the message needs to be discarded according to the processing result of the service module. If the service module processing result is successful, the message does not need to be discarded; if the processing result of the service module is failure, the message needs to be discarded. Only if the processing results of the N service modules are successful, the success of the first service processing result can be indicated; and as long as one of the N service modules fails, indicating that the first service processing result is failed. If the first service processing result is successful, a message forwarding table is created for the message to be processed according to the first service processing result and attribute information in the message cache, and the message to be processed is forwarded back to the first core of the CPU through the second core of the CPU; otherwise, deleting the message to be processed and the corresponding message buffer.
If the IP message has a corresponding message forwarding table, the attribute information in the message forwarding table is synchronized into the message cache, and whether the message forwarding table refreshing operation exists is further judged. If the refreshing operation of the message forwarding table does not exist, the IP message is directly forwarded back to the first core of the CPU through the second core of the CPU, and the IP message and the corresponding message cache are deleted; if the message forwarding table refreshing operation exists, the IP message is forwarded back to the first core of the CPU through the second core of the CPU, and meanwhile, the processing identification of the storage module is transmitted to the message cache, and the processing of the third core of the CPU is waited. The third core of the CPU can acquire the message cache in the second core of the CPU in a polling mode. If the message cache exists, the message cache is sequentially sent to each service module (module 1 to module N) to carry out service module processing again; and if the message cache does not exist, continuing to poll and read the message cache in the second core of the CPU. If the second service processing result is successful, updating a message forwarding table corresponding to the message to be processed according to the second service processing result, and deleting the message cache; otherwise, deleting the message forwarding table and the message buffer corresponding to the message to be processed, and ending the forwarding processing of the message to be processed.
S260, performing second business processing on the message to be processed through a third check of the CPU; the real-time performance of the first service and the message forwarding is higher than that of the second service.
According to the technical scheme, after the CPU second core judges that the message to be processed has the corresponding message forwarding table, whether the message to be processed needs to be subjected to second service processing or not is judged; if the business process is changed, the configuration of the first business is changed and/or the message forwarding table is refreshed, determining that the message to be processed needs to be processed by the second business; the second service is determined according to the service flow change, the configuration change of the first service and/or the message forwarding table refreshing operation. According to the technical scheme, the problems of forwarding flow oscillation and large bandwidth flow forwarding fluctuation generated by the CPU can be effectively solved, the message forwarding efficiency and the message forwarding stability of the CPU can be further improved, and therefore the practical application requirements can be better met.
In this embodiment, optionally, the first service includes a white list matching process or message validity verification. Fig. 6 is a flowchart of a white list matching process according to a second embodiment of the present invention. Where physical core 2 represents the CPU second core, physical core 1 represents the CPU third core, and is assumed to be a 4-core CPU in fig. 6. The white list matching process in fig. 6 may refer to the process of the business process refresh process in fig. 5, except that the process of determining whether the message forwarding table refresh operation exists by the second core of the CPU in fig. 5 is changed to determine whether the white list flag exists, and the process of performing the business process refresh process by the third core of the CPU in fig. 5 is changed to perform the white list matching process.
Example III
Fig. 7 is a schematic structural diagram of a DPDK-based message processing apparatus according to a third embodiment of the present invention, where the apparatus may execute the DPDK-based message processing method according to any embodiment of the present invention, and the apparatus has functional modules and beneficial effects corresponding to the execution method. As shown in fig. 7, the apparatus includes:
a message to be processed sending module 310, configured to receive a message to be processed through a first core of a CPU, and send the message to be processed to a second core of the CPU;
a message forwarding table judging module 320, configured to judge, by using the CPU second core, whether the message to be processed has a corresponding message forwarding table;
The first service processing module 330 is configured to perform, if not, first service processing on the to-be-processed packet by the CPU second check, and perform forwarding processing determination on the to-be-processed packet according to a first service processing result;
The second service processing module 340 is configured to, if so, forward the message to be processed back to the first CPU core through the second CPU core, and perform second service processing according to a result of determining whether the message to be processed needs to be subjected to second service processing through the third CPU core;
the real-time performance of the first service and the message forwarding is higher than that of the second service.
Optionally, the message forwarding table determining module 320 is specifically configured to:
Extracting attribute information of the message to be processed through the CPU second check, distributing corresponding message caches for the message to be processed, and storing the attribute information in the message caches; wherein, the attribute information at least comprises message quintuple information and interface information;
Judging whether the message to be processed has a corresponding message forwarding table or not according to the attribute information.
Optionally, the first service processing module 330 is specifically configured to:
If the first service processing result is successful, a message forwarding table is created for the message to be processed according to the first service processing result and attribute information in the message cache, and the message to be processed is forwarded back to the first CPU core through the second CPU core;
otherwise, deleting the message to be processed and the corresponding message buffer.
Optionally, the second service processing module 340 is specifically configured to:
Performing second service processing on the message to be processed according to the message cache of the message to be processed through the third CPU core;
If the second service processing result is successful, updating a message forwarding table corresponding to the message to be processed according to the second service processing result, and deleting the message cache;
Otherwise, deleting the message forwarding table and the message buffer corresponding to the message to be processed, and ending the forwarding processing of the message to be processed.
Optionally, the apparatus further includes:
The second service processing judging module is used for judging whether the message to be processed needs to be subjected to second service processing or not after judging that the message to be processed has a corresponding message forwarding table through the CPU second core;
the second service processing determining module is used for determining that the message to be processed needs to be processed by a second service if the service flow is changed, the configuration of the first service is changed and/or the message forwarding table is refreshed; and the second service is determined according to the service flow change, the configuration change of the first service and/or the message forwarding table refreshing operation.
Optionally, the first service includes white list matching processing or message validity checking.
Optionally, the first core of the CPU and the second core of the CPU belong to a DPDK binding core, and the third core of the CPU belongs to a DPDK unbinding core, where the DPDK binding core is configured to process a first DPDK service, and the DPDK unbinding core is configured to process a second DPDK service and other non-DPDK services;
the first DPDK service comprises a real-time DPDK message forwarding service, and the second DPDK service comprises a service based on message caching.
The message processing device based on DPDK provided by the embodiment of the invention can execute the message processing method based on DPDK provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example IV
Fig. 8 shows a schematic diagram of the structure of an electronic device 10 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 8, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, wherein the processor includes at least four CPU cores; the memory stores computer programs executable by at least one processor, and the processor 11 can perform various suitable actions and processes according to the computer programs stored in a Read Only Memory (ROM) 12 or the computer programs loaded from a storage unit 18 into a Random Access Memory (RAM) 13. In the RAM13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the DPDK-based message processing method.
In some embodiments, the DPDK-based message processing method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the memory unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the DPDK based message processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the DPDK based message processing method in any other suitable way (e.g. by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A message processing method based on DPDK is characterized by comprising the following steps:
Receiving a message to be processed through a first CPU core, and sending the message to be processed to a second CPU core;
Judging whether the message to be processed has a corresponding message forwarding table or not through the CPU second core;
if not, the CPU performs a second check on the message to be processed to perform a first service processing, and performs forwarding processing judgment on the message to be processed according to a first service processing result;
If yes, the message to be processed is transmitted back to the first CPU core through the second CPU core, and second service processing is performed through the third CPU core according to the judging result of whether the message to be processed needs to be subjected to the second service processing or not;
the real-time performance of the first service and the message forwarding is higher than that of the second service.
2. The method according to claim 1, wherein determining, by the CPU second core, whether the message to be processed has a corresponding message forwarding table, includes:
Extracting attribute information of the message to be processed through the CPU second check, distributing corresponding message caches for the message to be processed, and storing the attribute information in the message caches; wherein, the attribute information at least comprises message quintuple information and interface information;
Judging whether the message to be processed has a corresponding message forwarding table or not according to the attribute information.
3. The method of claim 2, wherein forwarding the message to be processed according to the first service processing result includes:
If the first service processing result is successful, a message forwarding table is created for the message to be processed according to the first service processing result and attribute information in the message cache, and the message to be processed is forwarded back to the first CPU core through the second CPU core;
otherwise, deleting the message to be processed and the corresponding message buffer.
4. The method according to claim 2, wherein the performing, by the third core of the CPU, the second service processing according to the determination result that the message to be processed needs to perform the second service processing includes:
Performing second service processing on the message to be processed according to the message cache of the message to be processed through the third CPU core;
If the second service processing result is successful, updating a message forwarding table corresponding to the message to be processed according to the second service processing result, and deleting the message cache;
Otherwise, deleting the message forwarding table and the message buffer corresponding to the message to be processed, and ending the forwarding processing of the message to be processed.
5. The method according to claim 2, wherein after determining, by the CPU second core, that the message to be processed has a corresponding message forwarding table, the method further comprises:
Judging whether the message to be processed needs to be subjected to second business processing or not;
if the business process is changed, the configuration of the first business is changed and/or the message forwarding table is refreshed, determining that the message to be processed needs to be subjected to second business processing; and the second service is determined according to the service flow change, the configuration change of the first service and/or the message forwarding table refreshing operation.
6. The method according to any of claims 2-5, wherein the first service comprises a white list matching process or a message validity check.
7. The method of claim 1, wherein the first core and the second core belong to a DPDK bundling core, and the third core belongs to a DPDK unbinding core, wherein the DPDK bundling core is configured to process a first DPDK traffic, and wherein the DPDK unbinding core is configured to process a second DPDK traffic and other non-DPDK traffic;
the first DPDK service comprises a real-time DPDK message forwarding service, and the second DPDK service comprises a service based on message caching.
8. A DPDK-based message processing apparatus, said apparatus comprising:
The message to be processed sending module is used for receiving a message to be processed through the first core of the CPU and sending the message to be processed to the second core of the CPU;
the message forwarding table judging module is used for judging whether the message to be processed has a corresponding message forwarding table or not through the CPU second core;
The first service processing module is used for carrying out first service processing on the message to be processed through the CPU second check if the message to be processed does not exist, and carrying out forwarding processing judgment on the message to be processed according to a first service processing result;
The second service processing module is used for forwarding the message to be processed back to the first CPU core through the second CPU core if the message to be processed exists, and performing second service processing through the third CPU core according to a judging result of whether the message to be processed needs to be subjected to the second service processing or not;
the real-time performance of the first service and the message forwarding is higher than that of the second service.
9. A DPDK-based message processing electronic device, the electronic device comprising:
At least one processor; wherein the processor includes at least four CPU cores;
And a memory communicatively coupled to the at least one processor; wherein,
The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the DPDK based message processing method according to any one of claims 1-7.
10. A computer readable storage medium storing computer instructions for causing a processor to implement the DPDK based message processing method according to any one of claims 1-7 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211435253.2A CN118055068A (en) | 2022-11-16 | 2022-11-16 | Message processing method, device, equipment and medium based on DPDK |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211435253.2A CN118055068A (en) | 2022-11-16 | 2022-11-16 | Message processing method, device, equipment and medium based on DPDK |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118055068A true CN118055068A (en) | 2024-05-17 |
Family
ID=91052614
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211435253.2A Pending CN118055068A (en) | 2022-11-16 | 2022-11-16 | Message processing method, device, equipment and medium based on DPDK |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118055068A (en) |
-
2022
- 2022-11-16 CN CN202211435253.2A patent/CN118055068A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114500633B (en) | Data forwarding method, related device, program product and data transmission system | |
| CN113778644B (en) | Task processing method, device, equipment and storage medium | |
| CN115514718B (en) | Data interaction method, control layer and equipment based on data transmission system | |
| CN116347518A (en) | User plane data packet processing system, method, computer equipment and storage medium | |
| CN115525666A (en) | Real-time data updating method and device, electronic equipment and storage medium | |
| CN116578502A (en) | Access request processing device, processing method, equipment and storage medium | |
| CN112883041B (en) | Data updating method and device, electronic equipment and storage medium | |
| CN116383207A (en) | Data tag management method and device, electronic equipment and storage medium | |
| CN118055068A (en) | Message processing method, device, equipment and medium based on DPDK | |
| CN114615273B (en) | Data transmission method, device and equipment based on load balancing system | |
| CN114553894B (en) | Data synchronization method, device, system and storage medium | |
| US20180349215A1 (en) | Transmission of large messages in computer systems | |
| CN114793234B (en) | Message processing method, device, equipment and storage medium | |
| CN116306407B (en) | Verification method, device, equipment and storage medium of Network On Chip (NOC) | |
| CN115174447B (en) | Network communication method, device, system, equipment and storage medium | |
| EP3993366A2 (en) | Network load balancer, request message distribution method, program product and system | |
| CN114924806B (en) | Dynamic synchronization method, device, equipment and medium for configuration information | |
| CN114338543B (en) | Network access speed limiting method, device, equipment and storage medium | |
| CN117997886A (en) | Message processing method and device, electronic equipment and storage medium | |
| CN117472913A (en) | Data processing method, device, equipment and storage medium | |
| CN118018533A (en) | Streaming media data transmission method, device, equipment and medium | |
| CN117395211A (en) | Data forwarding method, device, switch and medium | |
| CN115145859A (en) | Data transmission method and device, electronic equipment and storage medium | |
| CN115587119A (en) | Database query method and device, electronic equipment and storage medium | |
| CN114827159A (en) | Network request path optimization method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |