CN115567590A - Data packet scheduling method, device, equipment and readable storage medium - Google Patents
Data packet scheduling method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN115567590A CN115567590A CN202211439802.3A CN202211439802A CN115567590A CN 115567590 A CN115567590 A CN 115567590A CN 202211439802 A CN202211439802 A CN 202211439802A CN 115567590 A CN115567590 A CN 115567590A
- Authority
- CN
- China
- Prior art keywords
- data packet
- data
- scheduled
- destination address
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/74—Address processing for routing
- H04L45/745—Address table lookup; Address filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The application discloses a data packet scheduling method, a device, equipment and a readable storage medium, wherein the method comprises the following steps: acquiring an external data packet; determining a data packet to be scheduled, which accords with a preset data type, in the external data packet; matching the data packet to be scheduled with data in a preset cache table; and if the first destination address corresponding to the data packet to be scheduled in the preset cache table is obtained through matching, carrying out redirection processing on the data packet to be scheduled, and forwarding the redirected data packet to be scheduled to the first destination address. According to the method and the device, the external data packet is obtained, the data packet to be scheduled which accords with the preset data type is selected, the data packet to be scheduled is matched with the data in the preset cache table, if the first destination address corresponding to the data packet to be scheduled is obtained through matching, the data packet to be scheduled is redirected and forwarded to the first destination address, the data packet to be scheduled is prevented from being forwarded again through a user state, and therefore the scheduling performance of the data packet is improved.
Description
Technical Field
The present application relates to the field of data processing, and in particular, to a method, an apparatus, a device, and a readable storage medium for scheduling a data packet.
Background
In the application of the network technology simulation verification platform, in order to simulate a more real and complex scene, the network technology simulation verification platforms in various places are mutually linked to form a combined simulation verification platform. In order to coordinate and monitor network traffic, the combined simulation and verification platform often selects a main platform to coordinate and schedule other platforms. In order to enable the main platform to supervise and control the user and the sub-platforms, all sub-platforms and the traffic of the user need to be forwarded through the main platform.
In a conventional data scheduling solution, scheduling policies and algorithms are operated in a user mode, which results in that when a host platform forwards data packets, all data packets need to pass through a complex network protocol stack in a kernel, and data copying between the user mode and the kernel mode needs to be performed multiple times, so that a scheduling service in the user mode makes a decision on a forwarding address of the data packet.
Due to the complexity of a network protocol stack in a kernel and frequent data transmission between a kernel mode and a user mode, a lot of unnecessary machine resources are occupied, and finally, the performance of the traditional data scheduling solution is sharply reduced in a large data flow scene.
Disclosure of Invention
In view of this, the present application provides a method, an apparatus, a device and a readable storage medium for scheduling a data packet, which aim to improve the scheduling performance of the data packet.
In order to achieve the above object, the present application provides a data packet scheduling method, including the following steps:
acquiring an external data packet;
determining a data packet to be scheduled which accords with a preset data type in the external data packets;
matching the data packet to be scheduled with data in a preset cache table;
and if the first destination address corresponding to the data packet to be scheduled in the preset cache table is obtained through matching, redirecting the data packet to be scheduled, and forwarding the redirected data packet to be scheduled to the first destination address.
Illustratively, after the matching the data packet to be scheduled with the data in the preset cache table, the method includes:
if the first destination address is not matched, initializing the data packet to be scheduled, and outputting the initialized data packet to be scheduled to a user state;
after the user state works out the scheduling strategy of the initialized data packet to be scheduled, a second destination address of the scheduling strategy is obtained;
and updating the second destination address to the preset cache table to ensure that the initialized data packet to be scheduled is matched with the preset cache table to obtain the second destination address.
Illustratively, if the first destination address is not obtained by matching, the initializing the to-be-scheduled data packet includes:
if the first destination address is not matched, calculating a first hash value and a first key value group of the data packet to be scheduled;
and updating the data of the first hash value and the first key value group to the preset cache table so as to complete the initialization processing of the data packet to be scheduled.
Illustratively, the updating the second destination address to the preset cache table includes:
analyzing the scheduling strategy to obtain an analyzed second destination address and a second hash value;
matching a second key value group corresponding to the second hash value in the preset cache table;
determining a data packet to be updated based on the second key value group;
and updating the second destination address to the data packet to be updated, and updating the second destination address to the preset cache table.
Illustratively, the updating the second destination address to the data packet to be updated and the updating the second destination address to the preset cache table include:
determining whether the second destination address in the preset cache table is consistent with the data packet to be updated or not based on the second hash value and the second key value group;
if the data packets are inconsistent, updating the second destination address to the data packet to be updated, updating the second destination address to the preset cache table, and marking the matching times of the updating process as one time;
if the second destination address is consistent with the data packet to be updated, determining the matching times of the second destination address and the data packet to be updated;
if the matching times are more than or equal to two times, marking the data packet to be updated to be valid;
and if the matching times are less than two times, increasing the matching times for one time.
Illustratively, the matching the data packet to be scheduled with the data in the preset cache table includes:
calculating to obtain key value data of the data packet to be scheduled: the key value data comprises a protocol, a source port, a destination port and a source address;
combining at least two data in the key value data to obtain a retrieval key value;
and matching the retrieval key value with data in a preset cache table.
For example, if the first destination address corresponding to the data packet to be scheduled in the preset cache table is obtained through matching, performing redirection processing on the data packet to be scheduled includes:
and if the first destination address corresponding to the data packet to be scheduled in the preset cache table is obtained through matching, transmitting the data packet to be scheduled to a construction function, so that the construction function can use the first destination address to update the data packet to be scheduled, and use the current node data to update the source port and the source address of the data packet to be scheduled.
Illustratively, to achieve the above object, the present application further provides a packet scheduling apparatus, including:
the acquisition module is used for acquiring an external data packet;
the determining module is used for determining a data packet to be scheduled in the external data packet based on a preset data type;
the matching module is used for matching the data packet to be scheduled with data in a preset cache table;
and the judging module is used for redirecting the data packet to be scheduled and forwarding the redirected data packet to be scheduled to the first destination address if the first destination address corresponding to the data packet to be scheduled in the preset cache table is obtained through matching.
Illustratively, to achieve the above object, the present application further provides a data packet scheduling apparatus, where the apparatus includes: a memory, a processor and a packet scheduler stored on the memory and operable on the processor, the packet scheduler being configured to implement the steps of the packet scheduling method as described above.
Illustratively, to achieve the above object, the present application further provides a computer readable storage medium, on which a data packet scheduling program is stored, and the data packet scheduling program, when executed by a processor, implements the steps of the data packet scheduling method as described above.
Compared with the prior art that multiple data copies are performed between a kernel state and a user state, so that decisions such as scheduling strategies and forwarding addresses made at the user state are obtained, the kernel state forwards data packets according to decision data, and the scheduling performance of the data packets is low, in the application, external data packets are obtained, the data packets which accord with the preset data types are determined to be the data packets to be scheduled from the external data packets according to the preset data types, the data packets to be scheduled are matched with data in a preset cache table, if a first destination address which corresponds to the data packets to be scheduled in the preset cache table is obtained through matching, the data packets to be scheduled are redirected, and the redirected data packets to be scheduled are forwarded to the first destination address, namely the first destination address of the data packets to be scheduled is directly determined in the kernel state, the data packets to be scheduled are redirected, the data packets to be scheduled are directly forwarded to the first destination address in the kernel state, the scheduling data from the user state is further avoided, the multiple data copies between the user state and the kernel state are avoided, and the scheduling performance of the data packets to be scheduled is improved due to this application.
Drawings
Fig. 1 is a schematic flow chart of a first embodiment of a packet scheduling method according to the present application;
fig. 2 is a schematic flowchart of a second embodiment of a data packet scheduling method according to the present application;
FIG. 3 is a flowchart illustrating a data update procedure according to a second embodiment of the data packet scheduling method of the present application;
FIG. 4 is a diagram illustrating a third embodiment of a method for scheduling packets according to the present application;
fig. 5 is a schematic structural diagram of a hardware operating environment according to an embodiment of the present application.
The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application.
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a first embodiment of a data packet scheduling method according to the present application.
While the present embodiments provide an embodiment of a packet scheduling method, it should be noted that, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that shown or described here. For convenience of description, the following omits to perform various steps of the body description packet scheduling method, which includes:
step S110: acquiring an external data packet;
in the application of a network technology simulation verification platform, the main platform forwards the traffic corresponding to the branch platform and the user mode through the main platform, so that the main platform assists in scheduling of the branch platform and monitors the traffic, but the scheduling strategy and algorithm are in the user mode, so that when the main platform forwards the data packet needs to pass through a network protocol stack in the kernel mode and reach the user mode, and the data packet is copied for multiple times between the user mode and the kernel mode, so that the data such as the scheduling strategy of the user mode and the destination address of the scheduling can be obtained.
The external data packet is a data packet which needs to be scheduled through the kernel mode.
The obtaining process is to obtain an external Data packet from an XDP (eXpress Data Path) hook.
Step S120: determining a data packet to be scheduled which accords with a preset data type in the external data packets;
the external data packet is forwarded through the kernel mode, and is divided into two forwarding modes according to the actual forwarding condition, wherein one mode is that the traditional data packet is copied for multiple times through the kernel mode and the user mode, and is finally forwarded to the destination address through the kernel mode, and the other mode is that in the embodiment, the destination address of the data packet is determined through the kernel mode, and the data packet is directly forwarded in the kernel mode.
Illustratively, in a manner of directly forwarding a data packet in a kernel mode, a data type of the data packet needs to be determined first, and the data packet directly forwarded in the kernel mode is selected, where the data packet is a data packet to be scheduled, and other data packets all use a conventional forwarding mode.
The number of external data packets is huge, and the data types of the external data packets cover various types, for example: video, audio, etc. For an actual application scenario, when different types of data are forwarded, the data amount is emphasized, for example, in a scenario of forwarding a video data packet and an audio data packet, the amount of the video data packet exceeds 70% or 80% of the total external data packet (a weight ratio formulated according to an actual situation), the forwarding data of the current scenario is emphasized on forwarding the video data packet, so that the external data packet can be filtered according to the situation, the video data packet is correspondingly processed by using a mode of directly forwarding the video data packet to a destination address in a kernel mode, and the audio video is still forwarded in the kernel mode and in a mode of multiple copies of a user mode.
Meanwhile, all data types included in the external data packet can be directly forwarded by using a kernel mode, and similarly, a data packet to be scheduled with a preset data type can be selected, wherein the preset data type can be any one type or multiple types in the data packet, and the preset data type is determined according to an actual use scene and use purposes.
When determining the data packet to be scheduled, the rx filter can be used to analyze the data packet in the rx filter, and the data packet related to the user mode is found by filtering according to the analyzed quintuple. The matching rules of the rx filter can be flexibly configured according to the needs of the user. Including but not limited to support filtering according to port, protocol, etc.
Step S130: matching the data packet to be scheduled with data in a preset cache table;
illustratively, cache data is stored by using per-cpu (per-cpu variable) Hash (Hash) in eBPF (extended Berkeley Packet Filter), and in order to avoid that cache data occupies too many core resources, e-schedule (scheduling function) replaces cache contents by using LRU (Least Recently Used) strategy. The cache content of the e-schedule is stored in a preset cache table, and the preset cache table includes two hash tables, i.e. a redirect _ cache _ Map (mapping table of correspondence between source address and destination address) in table 1, and a key _ content _ Map (mapping table of correspondence between hash value of service data and source address) in table 2. The specific structure is shown in tables 1 and 2:
TABLE 1 mapping table of source address and destination address correspondences
Table 2 mapping table of correspondence between hash value and source address of service data
Table 1 stores the correspondence between the source address and the redirect address. The cached key is combined by protocol, source, destination port, and source IP (address) corresponding to the matching domain computation function. Because value in the cache, the data of key value composed by key-value needs to be composed by the data packets of in-chain and out-chain together, the value field is set to indicate whether the current data is available, and the e-schedule is prevented from misusing incomplete cache data.
The correspondence between the hash of the service data and the key (key, that is, data such as the source address in table 1) in table 1 is stored in table 2.
In the cache entry initialization function, this embodiment records all data except the destination port/IP (destination address) and stores the data in the corresponding cache table.
And matching the data packet to be scheduled with a preset cache table, wherein the matching process is mainly carried out by taking the format of the stored data in the preset cache table as the main mode, and according to the related data of the data packet, a destination address corresponding to the data packet is matched from the preset cache table, wherein the destination address is a destination forwarding address of the data packet, namely the data packet needs to be forwarded to the destination address in a kernel mode.
The matching process is to determine the destination of the data packet from the preset cache table.
Illustratively, the matching the data packet to be scheduled with the data in the preset cache table includes:
a, step a: calculating to obtain key value data of the data packet to be scheduled: the key value data comprises a protocol, a source port, a destination port and a source address;
table 1 in the preset cache table is a corresponding relationship between a source address (key value) and a redirected destination address, and table 2 is a corresponding relationship between a hash value of service data and a key value in table 1, so that when a data packet to be scheduled is matched with the preset cache table, key value data in the data packet needs to be calculated first.
Illustratively, the key-value data includes a protocol, a source port, a destination port, and a source address, the key-value data is from a quintuple of a packet to be scheduled, and the quintuple data of the packet includes: the method comprises the steps of forming a set by five quantities, namely a source address, a source port, a destination address, a destination port and a protocol, analyzing quintuple data of a data packet to be scheduled, and determining key value data of the data packet.
The parsing process may be parsed in a filter for filtering packets that conform to a predetermined data type.
Step b: combining at least two data in the key value data to obtain a retrieval key value;
the key value data comprises four data which are randomly combined, and the combination process can be carried out according to at least two data in the key value data, so that the retrieval key value is obtained.
For example, the data of the retrieval key value may be any two or any three data combinations, for example, a combination of a source address and a source port as the retrieval key value, for example, a combination of a destination port and a source port as the retrieval key value, for example, a random combination of a source address, a source port and a protocol as the retrieval key value, and the like.
Step c: and matching the retrieval key value with data in a preset cache table.
And matching the retrieval key value in the preset cache table by using the retrieval key value, matching the retrieval key value with the data in the preset cache table, determining a destination address in the preset cache table if the matching is successful, and obtaining the destination address from the preset cache table if the matching is failed.
Step S140: and if the first destination address corresponding to the data packet to be scheduled in the preset cache table is obtained through matching, carrying out redirection processing on the data packet to be scheduled, and forwarding the redirected data packet to be scheduled to the first destination address.
And when the first destination address is updated in the preset cache table, the destination address stored in the preset cache table.
In this embodiment, when a first destination address is obtained from a preset cache table in a matching manner, the first destination address is used to perform redirection processing on a data packet to be scheduled, the destination address of the data packet to be scheduled is directly changed in the kernel state, and the redirected data packet to be scheduled is directly forwarded to the first destination address.
In the redirection process, the quintuple data of the data packet to be scheduled is adjusted, and the destination address and the source address are changed, so that the core state directly forwards the data packet to be scheduled to the destination address, the data packet to be scheduled is prevented from passing through a network protocol stack and a plurality of copying processes between the user state and the core state, and the scheduling performance of the data packet is improved.
For example, if the first destination address corresponding to the data packet to be scheduled in the preset cache table is obtained through matching, performing redirection processing on the data packet to be scheduled includes:
step d: and if the first destination address corresponding to the data packet to be scheduled in the preset cache table is obtained through matching, transmitting the data packet to be scheduled to a construction function, so that the construction function can use the first destination address to update the data packet to be scheduled, and use the current node data to update the source port and the source address of the data packet to be scheduled.
And when the first destination address is obtained through matching, the quintuple data of the data packet to be scheduled needs to be changed so as to forward the data packet to be scheduled through a kernel mode directly in the following process.
In order to change a data packet to be scheduled, the data packet is transmitted to a constructor, a first destination address is updated to the data packet to be scheduled through the constructor, and current node data is updated to a source port and a source address of the data packet to be scheduled, so that the data packet to be scheduled is directly forwarded by a kernel state, the process that the data packet to be scheduled is forwarded by copying data between the kernel state and a user state for multiple times through a network protocol stack to obtain scheduling strategy related data from the user state is avoided, the forwarding flow is shortened, and the scheduling performance of the data packet is improved.
Compared with the prior art that multiple data copies are carried out between a kernel mode and a user mode, so that decisions such as a scheduling strategy and a forwarding address made at the user mode are obtained, the kernel mode forwards a data packet according to the decision data, and the scheduling performance of the data packet is low, in the application, an external data packet is obtained, the data packet which accords with a preset data type is determined to be a data packet to be scheduled from the external data packet according to the preset data type, the data packet to be scheduled is matched with data in a preset cache table, if a first destination address which corresponds to the data packet to be scheduled in the preset cache table is obtained through matching, the data packet to be scheduled is redirected, and the redirected data packet to be scheduled is forwarded to the first destination address, namely the first destination address of the data packet to be scheduled is directly determined in the kernel mode, and the data packet to be scheduled is forwarded to the first destination address in the kernel mode, so that the data packet to be scheduled is prevented from being obtained from the user mode, the first destination address of the data packet to be scheduled is also prevented from being copied between the kernel mode, and the performance of the data packet to be scheduled is improved due to the scheduling process of multiple times.
Exemplarily, referring to fig. 2, fig. 2 is a schematic flow chart of a second embodiment of the packet scheduling method of the present application, and the second embodiment is proposed based on the first embodiment of the packet scheduling method of the present application, and the method further includes:
step S210: if the first destination address is not matched, initializing the data packet to be scheduled, and outputting the initialized data packet to be scheduled to a user state;
in the process of matching the data packet to be scheduled with the preset cache table, there is a case that the preset cache table does not contain the data packet to be scheduled, and therefore, in the matching process, the first destination address cannot be determined from the preset cache table, that is, the data packet to be scheduled cannot be directly redirected through the kernel state, and the data packet cannot be directly forwarded as well.
At this time, according to the conventional forwarding mode, the kernel mode and the user mode need to perform information interaction, so as to obtain scheduling decision data of the user mode on the data packet to be scheduled, and forward the scheduling data packet according to the decision data. Therefore, the data packet to be scheduled needs to be transmitted to the user mode through the network protocol stack, and corresponding scheduling decision data needs to be acquired from the user mode.
And in order to avoid the next encountering of the data packets which do not store key values and destination addresses in the preset cache table, a self-learning planning mode is set, a user-mode scheduling decision mechanism is self-learned in the kernel mode, and corresponding scheduling records are recorded to the preset cache table according to the user-mode scheduling decision mechanism, so that when the next data packet to be scheduled is matched with the preset cache table, the destination address can be directly obtained in the kernel mode, and the data copying of the user mode and the kernel mode is avoided for multiple times when the same type of data packet to be scheduled is targeted.
Before outputting the data packet to be scheduled to the user state, performing initialization processing on the data packet to be scheduled, wherein the initialization processing is to process the data packet to be scheduled according to a traditional forwarding mode, meanwhile, updating key value data, hash values of service data and other data in the data packet to be scheduled to a preset cache table, and after scheduling decision data are obtained, completing updating of the preset cache table.
Illustratively, the initialization processing includes storing corresponding data in a preset cache table, and meanwhile, includes performing corresponding processing on a data packet to be scheduled according to a conventional forwarding mode.
For example, if the first destination address is not obtained by matching, the initializing process is performed on the data packet to be scheduled, and the initializing process includes:
step e: if the first destination address is not matched, calculating a first hash value and a first key value group of the data packet to be scheduled;
when the first destination address is not obtained in the matching, the key value data, the service data and other related data of the data packet to be scheduled need to be stored in a preset cache table, and the storage format of the preset cache table includes the key value, the destination address, the service data and other related data of the data packet, so that the first hash value and the first key value group of the data packet to be scheduled need to be calculated.
The first hash value is a relevant data value of the service data, the first key value group is key value data of a data packet to be scheduled, the key value data comprises four items of data of a protocol, a source port, a destination port and a source address, and after the data are obtained through calculation, the four items of data are filled in a corresponding preset cache table.
Step f: and updating the data of the first hash value and the first key value group to the preset cache table so as to complete the initialization processing of the data packet to be scheduled.
In the process of updating data to the preset cache table, the first hash value is updated to the table 2, the first key value group is updated to the tables 1 and 2, and other corresponding parameters except the destination port and the destination address are updated to the preset cache table, so that the initialization processing of the data packet to be scheduled is completed.
Step S220: after the user status works out the initialized scheduling strategy of the data packet to be scheduled, acquiring a second destination address of the scheduling strategy;
and after the data packet to be scheduled is initialized, outputting the data packet to be scheduled to a user mode, wherein the user mode can make a corresponding scheduling strategy according to the data packet to be scheduled, and the scheduling strategy comprises scheduling decision data such as a second destination address and related service data.
Meanwhile, when the scheduling policy is acquired, the corresponding data packet to be scheduled is acquired (the process of copying the data packet for multiple times in the traditional forwarding mode), and when the data packet is acquired, the data packet is screened by using the tx filter, the screening process is the same as the screening process of the rx filter, the filtering mode is the same, and details are not repeated here.
And after the user dynamically works out the scheduling strategy of the initialized data packet to be scheduled, acquiring a second destination address contained in the scheduling strategy.
Step S230: and updating the second destination address to the preset cache table to ensure that the initialized data packet to be scheduled is matched with the preset cache table to obtain the second destination address.
When the data packet to be scheduled is initialized, the key value and the hash value in the data packet are stored in a preset cache table, and the destination address of the data packet needs to be determined in order to schedule the data packet to be scheduled, so that when the second destination address of the scheduling strategy is acquired from a user state, the second destination address is updated into the preset cache table, the data corresponding to the key value and the destination address of the data packet to be scheduled is completed in the preset cache table, the matching of the destination address can be completed directly according to the preset cache table when the data packet to be scheduled is scheduled subsequently, and the redirection and forwarding of the data packet are completed directly by a kernel state.
Illustratively, the updating the second destination address to the preset cache table includes:
step g: analyzing the scheduling strategy to obtain an analyzed second destination address and a second hash value;
when the second destination address is updated to the preset cache table, the update position of the second destination address needs to be determined, that is, the corresponding relationship between the second destination address and the data packet to be scheduled in the preset cache table needs to be determined, so that the scheduling policy needs to be analyzed before the second destination address is updated to the preset cache table, and the second destination address and a second hash value of the scheduling policy in the user state are calculated, where the second hash value is also a hash value of the service data.
Step h: matching a second key value group corresponding to the second hash value in the preset cache table;
the method comprises the steps that a first hash value and first key value group data of a data packet to be scheduled are stored in a current preset cache chain table, when a second hash value is matched with the preset cache table, if a result is matched, the data packet to be scheduled corresponding to a user-mode scheduling strategy is determined to be matched in the preset cache table, the matched result is a second key value group corresponding to the second hash value in the preset cache table, the scheduling strategy given by the user mode is determined according to logic of data updating, the second hash value and a second key value group obtained by matching are both data corresponding to the first hash value and a first destination address, and therefore the data corresponding relation of the data packet to be scheduled in the preset cache table is completed.
Step i: determining a data packet to be updated based on the second key value group;
and after determining the corresponding relation with the data packet to be scheduled according to the second hash value and the second key value group, updating the preset cache table and the data packet to be scheduled.
And the data packet to be updated is a data packet to be scheduled of the data to be updated.
Step j: and updating the second destination address to the data packet to be updated, and updating the second destination address to the preset cache table.
In the updating process, the second destination address is updated to the data packet to be updated, and meanwhile, the second destination address needs to be updated to the preset cache table.
The first hash value and the second hash value, and the first key value group and the second key value group are all related data of the data packet to be scheduled, that is, the hash value and the key value group represent the same type of data packet to be scheduled in the preset cache table, so that when the preset cache table is updated, only the second destination address is updated to a corresponding position.
Illustratively, the updating the second destination address to the data packet to be updated and the updating the second destination address to the preset cache table include:
step k: determining whether the second destination address in the preset cache table is consistent with the data packet to be updated or not based on the second hash value and the second key value group;
before updating data, whether the data in the scheduling strategy corresponds to the data in the preset cache table one by one needs to be determined, the consistency of the data is ensured, and the condition that the data packet is in error when being forwarded by the kernel mode due to the fact that the data such as the data packet and key values in the preset cache table do not correspond is avoided.
And determining the data corresponding positions of the first hash value and the second key value group in a preset cache table according to the second hash value and the second key value group, and determining that the second destination address is consistent with the data packet to be updated after the data of the two parties are successfully matched.
For example, in a process of determining whether data is consistent currently, that is, in a process of matching data in a scheduling policy with data in a preset cache table and data in a data packet to be updated, if it is determined that the data is consistent directly, it is determined that a scheduling policy formulated in a user state already exists in the data packet to be updated and the preset cache table, and if it is not consistent, it is determined that the scheduling policy does not exist in the data packet to be updated and the preset cache table.
And when the matching determination data is consistent, determining that the current scheduling policy is updated to the preset cache table, and further determining the validity of the relevant data of the current scheduling policy.
And when the data are inconsistent according to the matching determination, determining that the relevant data of the scheduling policy do not exist in the current preset cache table, and updating the relevant data in the scheduling policy to the preset cache table.
The relevant data of the scheduling policy includes data such as a hash value, a key value group and a destination address.
Step l: if the data packets are inconsistent, updating the second destination address to the data packet to be updated, updating the second destination address to the preset cache table, and marking the matching times of the updating process as one time;
and when the matching result is inconsistent, updating the second destination address to the data packet to be updated and the preset cache table, and marking the matching times of the updating process as one time after the updating is finished for subsequently judging the validity of the data.
When the matching result is consistent, it is proved that the data related to the current scheduling policy is already stored in the preset cache table and the data packet to be updated, but the matching frequency of the data in the preset cache table is one, and the data is invalid and cannot be directly called, so that the data packet to be scheduled needs to be transmitted to a user state.
Therefore, when the matching results are consistent, the matching times need to be further judged so as to determine the validity of the related data.
Step m: if the second destination address is consistent with the data packet to be updated, determining the matching times of the second destination address and the data packet to be updated;
step n: if the matching times are more than or equal to two times, marking the data packet to be updated to be effective;
step o: and if the matching times are less than twice, increasing the matching times for one time.
If the second destination address is consistent with the data packet to be updated, the second destination address is proved to be applicable to forwarding of the current data packet to be scheduled, but in order to ensure the accuracy of the data, the matching times are required to be determined when the matching is successful.
Illustratively, the user mode does not modify the forwarded traffic data. Therefore, in order to solve the problem that the quintuple of the forwarded data packet (the data packet to be scheduled and processed in the kernel mode) is different from the quintuple of the original data packet (the data packet stored in the preset cache table), the original data packet matched with the forwarded data packet is found by using the hash value of the service data.
In this mechanism, there may be a problem of hash value collision, and therefore, for a case where there is a possible collision, it is required in this embodiment that a packet matching a stream is successfully matched twice (Match _ count > =2, the number of matching times is greater than or equal to 2) in succession, so as to set this cache entry as valid.
When the matching frequency is less than twice, the validity of the data cannot be ensured, so that when the matching frequency is less than twice, the matching frequency is increased once.
Referring to fig. 3, fig. 3 is a flow chart of data update in the present embodiment.
In fig. 3, after obtaining the relevant data of the scheduling policy, filtering the data through a tx filter, where the filtering process is the same as the content explained above, and it is not repeated here, after the data is filtered, parsing the data, where the parsing process includes parsing the scheduling policy and the data packet transmitted by the user mode, calculating a second hash value of the scheduling policy, determining a second key value group from the preset cache table according to the second hash value, determining the data packet to be updated from the preset cache table according to the second key value group, matching the relevant data of the scheduling policy with the data in the preset cache table and the data of the data packet to be updated, determining whether the data packets are consistent, and when the data are inconsistent, updating the preset cache table and the data packet to be updated, marking the matching times in the updating process as one time, further judging whether the matching times is more than or equal to two times when the data are consistent, marking the updated data as valid when the matching times is more than or equal to two times, and adding one time to the current matching times when the matching times is less than two times, so that the subsequent matching times can meet the judgment of validity when the matching times are judged subsequently, the valid data can be used in the preset cache table, the data with the matching times not more than two times are invalid data, and the type of data existing in the preset cache table is not used when the data packet to be scheduled is matched in a kernel mode.
In this embodiment, when the first destination address is obtained by matching, initializing the data packet to be scheduled, storing the relevant data of the data packet to be scheduled in a preset cache table, and outputting the initialized data packet to be scheduled to a user state, after the user state formulates a scheduling policy according to the data packet to be scheduled, obtaining a second destination address in the scheduling policy, and storing the second destination address in the preset cache table, so as to ensure that the second destination address is matched in the preset cache table when the data packet to be scheduled is processed in a subsequent kernel state, thereby achieving a process in which the kernel state self-learns the scheduling policy in the user state, avoiding a process in which multiple data copies are required between the kernel state and the user state when the data packet to be scheduled is subsequently forwarded, and thus recording the scheduling policy in the user state in a self-learning manner, and improving the performance of scheduling of the subsequent data packet.
Exemplarily, referring to fig. 4, fig. 4 is a schematic flow chart of a third embodiment of the packet scheduling method of the present application, and based on the first embodiment and the second embodiment of the packet scheduling method of the present application, the third embodiment is proposed:
fig. 4 includes a data processing procedure and a data interaction procedure of the user mode and the kernel mode.
The user mode contains related data for making a scheduling strategy, so that a data scheduling system exists in the user mode, and the scheduling strategies acquired from the user mode by the kernel mode are all made by the data scheduling system.
A network protocol stack exists between the user mode and the kernel mode to be used as information interaction guarantee, and the kernel mode is directly connected with the network equipment.
The kernel mode comprises two data processing modules, namely an in-chain processing module and an out-chain processing module, and different data processing operations are executed in the two modules respectively.
The data processing process of the in-chain module in the kernel mode is as follows:
after an external data packet is obtained by an XDP hook in a kernel mode, the external data packet passes through an rx filter, part of data types used for direct forwarding in the kernel mode are screened out, matching domain calculation is carried out on the screened data packet, namely, the data packet is matched with data in a preset cache table, after a target address is obtained through matching, the data packet is transmitted to a construction function, the data packet is constructed and redirected, and when the relevant target address cannot be matched in the matching calculation, MISS is reported, namely, mismatching is not successful (when matching is successful, HIT is reported), namely, the fact that the relevant data of the data packet does not exist in the preset cache table is proved, at the moment, the data packet is initialized, and the data packet is output to a data scheduling system in a user mode through a network protocol stack.
The data processing process of the kernel-mode out-link module is as follows:
after a scheduling strategy of a data packet is made by a data scheduling system in a user mode, the scheduling strategy is obtained through a tc hook, the obtaining process also comprises the step of obtaining data such as related data packets, at the moment, data filtering is carried out through a tx filter, the filtering process is the same as the filtering process of an rx filter, the data packets of specific types are screened out, after the screening, the scheduling strategy is analyzed, data such as a destination address and a hash value are determined, corresponding data items are updated according to the matching result of the data and a preset cache table, and the data updating comprises the data such as the preset cache table and the data packet to be scheduled.
In this embodiment, the data packet is directly forwarded in the kernel mode through the in-link processing module, and the scheduling policy self-learning in the user mode is performed in the kernel mode through the out-link processing module, so that more scheduling policies are stored in the kernel mode to deal with different data packets, and multiple data copies between the user mode and the kernel mode are avoided.
In addition, the present application also provides a packet scheduling apparatus, where the packet scheduling apparatus includes:
the acquisition module is used for acquiring an external data packet;
the determining module is used for determining a data packet to be scheduled in the external data packet based on a preset data type;
the matching module is used for matching the data packet to be scheduled with data in a preset cache table;
and the judging module is used for redirecting the data packet to be scheduled and forwarding the redirected data packet to be scheduled to the first destination address if the first destination address corresponding to the data packet to be scheduled in the preset cache table is obtained through matching.
Illustratively, the matching module includes:
the first judgment submodule is used for initializing the data packet to be scheduled and outputting the initialized data packet to be scheduled to a user state if the first destination address is not obtained through matching;
the obtaining submodule is used for obtaining a second destination address of the scheduling strategy after the user status works out the scheduling strategy of the initialized data packet to be scheduled;
and the updating submodule is used for updating the second destination address to the preset cache table so as to ensure that the initialized data packet to be scheduled is matched with the preset cache table to obtain the second destination address.
Illustratively, the determining sub-module includes:
the judging unit is used for calculating a first hash value and a first key value group of the data packet to be scheduled if the first destination address is not obtained through matching;
and the first updating unit is used for updating the data of the first hash value and the first key value group to the preset cache table so as to finish the initialization processing of the data packet to be scheduled.
Illustratively, the update submodule includes:
the analysis unit is used for analyzing the scheduling strategy to obtain an analyzed second destination address and a second hash value;
the matching unit is used for matching a second key value group corresponding to the second hash value in the preset cache table;
a determining unit, configured to determine a data packet to be updated based on the second key value group;
and the second updating unit is used for updating the second destination address to the data packet to be updated and updating the second destination address to the preset cache table.
Illustratively, the second updating unit includes:
a determining subunit, configured to determine, based on the second hash value and the second key value set, whether the second destination address in the preset cache table and the data packet to be updated are consistent;
the first judging subunit is configured to, if the first destination address is inconsistent with the second destination address, update the second destination address to the to-be-updated data packet, update the second destination address to the preset cache table, and mark the matching frequency of the updating process as one time;
the second judging subunit is configured to determine, if the second destination address is consistent with the data packet to be updated, the number of times of matching between the second destination address and the data packet to be updated;
a third judging subunit, configured to mark that the data packet to be updated is valid if the matching frequency is greater than or equal to two times;
and the fourth judging subunit is used for increasing the matching times for one time if the matching times are less than two times.
Illustratively, the matching module includes:
the calculation submodule is used for calculating and obtaining key value data of the data packet to be scheduled: the key value data comprises a protocol, a source port, a destination port and a source address;
the combination submodule is used for combining at least two data in the key value data to obtain a retrieval key value;
and the matching submodule is used for matching the retrieval key value with data in a preset cache table.
Illustratively, the determining module includes:
and the second judgment submodule is used for transmitting the data packet to be scheduled to a constructor if a first destination address corresponding to the data packet to be scheduled in the preset cache table is obtained through matching, so that the constructor can update the data packet to be scheduled by using the first destination address, and update the source port and the source address of the data packet to be scheduled by using the current node data.
The specific implementation of the data packet scheduling apparatus in the present application is substantially the same as that of each embodiment of the data packet scheduling method, and is not described herein again.
In addition, the application also provides a data packet scheduling device. As shown in fig. 5, fig. 5 is a schematic structural diagram of a hardware operating environment according to an embodiment of the present application.
Fig. 5 is a schematic structural diagram of a hardware operating environment of the packet scheduling apparatus, for example.
As shown in fig. 5, the packet scheduling apparatus may include a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 complete communication with each other through the communication bus 504, and the memory 503 is used for storing computer programs; the processor 501 is configured to implement the steps of the packet scheduling method when executing the program stored in the memory 503.
The communication bus 504 mentioned above for the packet dispatching device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus 504 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown, but this is not intended to represent only one bus or type of bus.
The communication interface 502 is used for communication between the above-described packet scheduling apparatus and other apparatuses.
The Memory 503 may include a Random Access Memory (RMD) and a Non-Volatile Memory (NM), such as at least one disk Memory. Optionally, the memory 503 may also be at least one storage device located remotely from the processor 501.
The Processor 501 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
The specific implementation of the data packet scheduling apparatus of the present application is substantially the same as that of each of the embodiments of the data packet scheduling method described above, and is not described herein again.
In addition, an embodiment of the present application further provides a computer-readable storage medium, where a data packet scheduler is stored on the computer-readable storage medium, and when being executed by a processor, the data packet scheduler implements the steps of the data packet scheduling method as described above.
The specific implementation of the computer-readable storage medium of the present application is substantially the same as the embodiments of the data packet scheduling method, and is not described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present application are merely for description, and do not represent the advantages and disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present application may be substantially or partially embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.
The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all the equivalent structures or equivalent processes that can be directly or indirectly applied to other related technical fields by using the contents of the specification and the drawings of the present application are also included in the scope of the present application.
Claims (10)
1. A method for scheduling data packets, the method comprising:
acquiring an external data packet;
determining a data packet to be scheduled which accords with a preset data type in the external data packets;
matching the data packet to be scheduled with data in a preset cache table;
and if the first destination address corresponding to the data packet to be scheduled in the preset cache table is obtained through matching, carrying out redirection processing on the data packet to be scheduled, and forwarding the redirected data packet to be scheduled to the first destination address.
2. The method for scheduling data packets according to claim 1, wherein after matching the data packets to be scheduled with the data in the preset buffer table, the method comprises:
if the first destination address is not obtained by matching, initializing the data packet to be scheduled, and outputting the initialized data packet to be scheduled to a user state;
after the user state works out the scheduling strategy of the initialized data packet to be scheduled, a second destination address of the scheduling strategy is obtained;
and updating the second destination address to the preset cache table to ensure that the initialized data packet to be scheduled is matched with the preset cache table to obtain the second destination address.
3. The method according to claim 2, wherein if the first destination address is not obtained by matching, performing initialization processing on the data packet to be scheduled, includes:
if the first destination address is not matched, calculating a first hash value and a first key value group of the data packet to be scheduled;
and updating the data of the first hash value and the first key value group to the preset cache table so as to finish the initialization processing of the data packet to be scheduled.
4. The method of claim 3, wherein the updating the second destination address to the predetermined buffer table comprises:
analyzing the scheduling strategy to obtain an analyzed second destination address and a second hash value;
matching a second key value group corresponding to the second hash value in the preset cache table;
determining a data packet to be updated based on the second key value group;
and updating the second destination address to the data packet to be updated, and updating the second destination address to the preset cache table.
5. The method as claimed in claim 4, wherein the updating the second destination address to the packet to be updated and the updating the second destination address to the predetermined buffer table comprises:
determining whether the second destination address in the preset cache table is consistent with the data packet to be updated or not based on the second hash value and the second key value group;
if the data packets are inconsistent, updating the second destination address to the data packet to be updated, updating the second destination address to the preset cache table, and marking the matching times of the updating process as one time;
if the second destination address is consistent with the data packet to be updated, determining the matching times of the second destination address and the data packet to be updated;
if the matching times are more than or equal to two times, marking the data packet to be updated to be effective;
and if the matching times are less than two times, increasing the matching times for one time.
6. The method for scheduling data packets according to claim 1, wherein said matching the data packets to be scheduled with the data in a preset buffer table comprises:
calculating to obtain key value data of the data packet to be scheduled: the key value data comprises a protocol, a source port, a destination port and a source address;
combining at least two data in the key value data to obtain a retrieval key value;
and matching the retrieval key value with data in a preset cache table.
7. The method according to claim 6, wherein if the first destination address corresponding to the to-be-scheduled packet in the preset cache table is obtained by matching, performing redirection processing on the to-be-scheduled packet, includes:
and if the first destination address corresponding to the data packet to be scheduled in the preset cache table is obtained through matching, transmitting the data packet to be scheduled to a construction function, so that the construction function can use the first destination address to update the data packet to be scheduled, and use the current node data to update the source port and the source address of the data packet to be scheduled.
8. A packet scheduling apparatus, comprising:
the acquisition module is used for acquiring an external data packet;
the determining module is used for determining a data packet to be scheduled in the external data packet based on a preset data type;
the matching module is used for matching the data packet to be scheduled with data in a preset cache table;
and the judging module is used for redirecting the data packet to be scheduled and forwarding the redirected data packet to be scheduled to the first destination address if the first destination address corresponding to the data packet to be scheduled in the preset cache table is obtained through matching.
9. A packet scheduling apparatus, characterized in that the apparatus comprises: memory, a processor and a packet scheduler stored on the memory and operable on the processor, the packet scheduler being configured to implement the steps of the packet scheduling method according to any of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon a packet scheduler, which when executed by a processor implements the steps of the packet scheduling method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211439802.3A CN115567590B (en) | 2022-11-17 | 2022-11-17 | Data packet scheduling method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211439802.3A CN115567590B (en) | 2022-11-17 | 2022-11-17 | Data packet scheduling method, device, equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115567590A true CN115567590A (en) | 2023-01-03 |
| CN115567590B CN115567590B (en) | 2023-03-10 |
Family
ID=84769680
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211439802.3A Active CN115567590B (en) | 2022-11-17 | 2022-11-17 | Data packet scheduling method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115567590B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115842789A (en) * | 2023-02-23 | 2023-03-24 | 鹏城实验室 | Data packet scheduling method and device and readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015180649A1 (en) * | 2014-05-30 | 2015-12-03 | 华为技术有限公司 | Method for moving data between storage devices, controller and storage system |
| CN106856455A (en) * | 2016-12-27 | 2017-06-16 | 京信通信技术(广州)有限公司 | A kind of method and apparatus of GTPU user face datas forwarding |
| CN107360268A (en) * | 2017-06-23 | 2017-11-17 | 北京奇艺世纪科技有限公司 | A kind of data package processing method, device and equipment |
| CN111262784A (en) * | 2020-01-13 | 2020-06-09 | 杭州朗和科技有限公司 | Message forwarding method, message forwarding device, storage medium and electronic equipment |
| CN113986811A (en) * | 2021-09-23 | 2022-01-28 | 北京东方通网信科技有限公司 | High-performance kernel-mode network data packet acceleration method |
-
2022
- 2022-11-17 CN CN202211439802.3A patent/CN115567590B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015180649A1 (en) * | 2014-05-30 | 2015-12-03 | 华为技术有限公司 | Method for moving data between storage devices, controller and storage system |
| CN106856455A (en) * | 2016-12-27 | 2017-06-16 | 京信通信技术(广州)有限公司 | A kind of method and apparatus of GTPU user face datas forwarding |
| CN107360268A (en) * | 2017-06-23 | 2017-11-17 | 北京奇艺世纪科技有限公司 | A kind of data package processing method, device and equipment |
| CN111262784A (en) * | 2020-01-13 | 2020-06-09 | 杭州朗和科技有限公司 | Message forwarding method, message forwarding device, storage medium and electronic equipment |
| CN113986811A (en) * | 2021-09-23 | 2022-01-28 | 北京东方通网信科技有限公司 | High-performance kernel-mode network data packet acceleration method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115842789A (en) * | 2023-02-23 | 2023-03-24 | 鹏城实验室 | Data packet scheduling method and device and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115567590B (en) | 2023-03-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8861524B2 (en) | Method for TCAM lookup using a key in multi-threaded packet processors | |
| US10834085B2 (en) | Method and apparatus for speeding up ACL rule lookups that include TCP/UDP port ranges in the rules | |
| US10410010B2 (en) | Language-localized policy statements | |
| US11418632B2 (en) | High speed flexible packet classification using network processors | |
| JP5659397B2 (en) | Rule-based content filtering system and method | |
| JP3643507B2 (en) | Packet processing apparatus and packet processing method | |
| US20040100956A1 (en) | Packet search device, packet processing search method used for the same, and program for the same | |
| CN115567590B (en) | Data packet scheduling method, device, equipment and readable storage medium | |
| US20220100852A1 (en) | Distributed security introspection | |
| CN104937572A (en) | Traffic and/or workload processing | |
| CN112367211B (en) | Method, device and storage medium for generating configuration template by device command line | |
| US10142244B2 (en) | Modifying a priority for at least one flow class of an application on a software defined networking controller | |
| CN105939284B (en) | The matching process and device of message control strategy | |
| US20060209827A1 (en) | Systems and methods for implementing counters in a network processor with cost effective memory | |
| US20200073952A1 (en) | Method and Apparatus for Acceleration of Hash-Based Lookup | |
| CN111835729A (en) | Message forwarding method, system, storage medium and electronic equipment | |
| CN116599892B (en) | Server system, routing method, routing device, electronic equipment and storage medium | |
| US9742660B2 (en) | Validating a routing function | |
| US20060048206A1 (en) | Method and computer system for triggering an action on digital communication data | |
| US20210360005A1 (en) | Inferring watchpoints for understandable taint reports | |
| US20050100019A1 (en) | Rule based packet processing engine | |
| JP2001331485A (en) | Distributed data base arithmetic processor | |
| US11210201B2 (en) | Scalable incremental analysis using caller and callee summaries | |
| US11316828B2 (en) | Networking sub-ranges | |
| CN106293635B (en) | Instruction block processing method and processing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |