CN117978694A - CPU resource dynamic control method for DPDK real-time flow - Google Patents

CPU resource dynamic control method for DPDK real-time flow Download PDF

Info

Publication number
CN117978694A
CN117978694A CN202410126429.9A CN202410126429A CN117978694A CN 117978694 A CN117978694 A CN 117978694A CN 202410126429 A CN202410126429 A CN 202410126429A CN 117978694 A CN117978694 A CN 117978694A
Authority
CN
China
Prior art keywords
dpdk
cpu
packet receiving
flow
real
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410126429.9A
Other languages
Chinese (zh)
Inventor
张广兴
姜海洋
毛仁祥
廖志元
王伟
涂楚
李博
王嘉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Future Networks Innovation Institute
Original Assignee
Jiangsu Future Networks Innovation Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu Future Networks Innovation Institute filed Critical Jiangsu Future Networks Innovation Institute
Priority to CN202410126429.9A priority Critical patent/CN117978694A/en
Publication of CN117978694A publication Critical patent/CN117978694A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876—Network utilisation, e.g. volume of load or congestion level
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
    • H04L43/103—Active monitoring, e.g. heartbeat, ping or trace-route with adaptive polling, i.e. dynamically adapting the polling rate
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5018—Thread allocation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Cardiology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Environmental & Geological Engineering (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention relates to a CPU resource dynamic control method of DPDK real-time flow, which polls to obtain real-time flow, inputs the current DPDK real-time flow into a preset LSTM model to obtain predicted future flow; when the current DPDK packet receiving thread capacity does not meet the predicted future flow, the number of the DPDK packet receiving threads is adjusted, the distribution of each CPU core is redistributed according to the adjustment result, and the dynamic adjustment of the CPU resources is completed. Compared with the prior art, the invention has the remarkable advantages that: the number of threads and the CPU core distribution are dynamically adjusted, the flow change is monitored in real time to predict, and the higher system performance is kept under the condition of different flows; meanwhile, tasks are reasonably distributed to a plurality of CPU cores, so that overload of a single core is avoided, and load balancing is realized; the burst flow is better dealt with, the system delay is reduced, the CPU resource waste is avoided, the system stability is improved, and the system is suitable for different workloads.

Description

CPU resource dynamic control method for DPDK real-time flow
Technical Field
The invention relates to the technical field of network traffic processing, in particular to a CPU resource dynamic control method for DPDK real-time traffic.
Background
With the increasing network traffic, conventional software processing methods have failed to meet the requirements of high-speed packet processing. DPDK (DATA PLANE Development Kit) is used as a high-performance data packet processing library, and low-delay and high-throughput data packet processing is realized by directly operating hardware resources. However, during the processing of a large number of data packets, the DPDK polling mechanism may cause the CPU to be fully loaded and full frequency, thereby affecting the stability and energy efficiency of the system; and in network traffic, tidal phenomena often occur, i.e. the traffic increases or decreases over a period of time. This tidal nature presents a significant challenge for the utilization of CPU resources. Conventional static CPU allocation strategies are difficult to accommodate for such tidal changes, which can easily lead to wasted resources or performance bottlenecks. Therefore, it is important to realize a dynamic CPU resource control method based on DPDK real-time traffic.
Document 1: the Chinese patent CN116225198B discloses a CPU load control method and a system, compared with the original sleep mechanism, the method is characterized in that the high-performance transmission is recovered in two or three time periods after the flow change by reducing the CPU power consumption; the high-speed data processing architecture of the data center provides a lower CPU power consumption processing mode while guaranteeing the transmission quality. No study was made of the allocation strategy of CPU load.
Disclosure of Invention
The invention aims to provide a CPU resource dynamic control method of DPDK real-time flow, which is used for carrying out flow monitoring and prediction according to the real-time flow, combining current DPDK packet-receiving related parameters, dynamically adjusting the work load and frequency of a CPU core, and completing dynamic task scheduling so as to realize optimal allocation of resources.
The technical solution for realizing the purpose of the invention is as follows:
The CPU resource dynamic control method of DPDK real-time flow polls to obtain real-time flow, inputs the current DPDK real-time flow into a preset LSTM model to obtain predicted future flow; when the current DPDK packet receiving thread capacity does not meet the predicted future flow, adjusting the number of the DPDK packet receiving threads, and reallocating the distribution of each CPU core according to an adjustment result to finish the dynamic adjustment of CPU resources;
The method comprises the steps of calculating the variance between DPDK real-time flow and predicted future flow obtained after dynamic adjustment of CPU resources is completed: when the calculated variance is larger than a set threshold, inputting the DPDK real-time flow obtained after the dynamic adjustment of the CPU resource into a preset LSTM model to re-predict the future flow, and judging whether the DPDK packet receiving thread capacity after the dynamic adjustment of the CPU resource meets the re-predicted future flow; otherwise, after a preset time interval, inputting the acquired DPDK real-time flow into a preset LSTM model again to re-predict the future flow, and judging whether the DPDK packet receiving thread capacity after the preset time interval meets the re-predicted future flow.
Further, the number of DPDK packet receiving threads is adjusted to be:
If the number of the current DPDK wrapping threads is lower than the number of DPDK wrapping threads required by the predicted future flow, increasing the number of the DPDK wrapping threads, and reallocating CPU cores according to an increase result, so that the distribution number of the reallocated CPU cores meets the use of the predicted future flow;
And if the number of the current DPDK wrapping threads exceeds the number of DPDK wrapping threads required by the predicted future flow, reducing the number of the DPDK wrapping threads, and reallocating CPU cores according to the reduction result, so that the distribution number of the reallocated CPU cores meets the use of the predicted future flow.
Further, the dynamic adjustment of the CPU resource further includes balancing the occupancy rate of each CPU core by executing a CPU resource management policy on a single DPDK wrapping thread, where the CPU resource management policy is:
S41, counting the number of DPDK polling actions and the number of received packets;
S42, judging whether the number of the polling packets is 0, if so, adding 1 to the number of empty packets of the packet receiving queue, entering step S43, otherwise, setting 0 to the number of empty packets of the packet receiving queue, calculating a frequency lifting parameter of the packet receiving queue, and entering step S44;
s43, judging whether the accumulated number of the empty packet times of the packet receiving queue is larger than a set empty packet threshold value,
If yes, calculating dormancy parameters according to the number of empty packets and adding 1 to the number of dormancy, entering into step S44, otherwise returning to step S41 to continue statistics;
S44, comparing the dormancy times with the total number of DPDK packet receiving queues corresponding to all network card ports of the DPDK, if the dormancy times are equal, entering a step S45, otherwise, entering a step S46;
S45, when the number of times of dormancy is equal to the total number of DPDK packet receiving queues corresponding to all the network card ports of the DPDK, traversing all the network card ports of the DPDK, finding out the minimum dormancy parameter in all the network card packet receiving queues, taking dormancy action according to the minimum dormancy parameter, yielding a corresponding CPU time slice, balancing the occupancy rate of a CPU core, accumulating dormancy time, and entering step S47;
S46, when the number of times of dormancy is unequal to the total number of DPDK packet receiving queues corresponding to all network card ports of the DPDK, traversing all the DPDK packet receiving queues, finding out the maximum frequency parameter in all the DPDK queues, adopting corresponding frequency lifting operation according to the maximum frequency lifting parameter, and entering step S47;
S47, judging whether a DPDK timer expires, if so, adjusting the frequency of a CPU core according to the number of the DPDK packets received and the accumulated sleep time of the current polling; otherwise, the process returns to step S41.
Further, the total number of DPDK packet receiving queues corresponding to all network card ports of the DPDK is obtained by calculating the product of the number of all network card ports of the DPDK and the number of DPDK packet receiving queues.
Further, in step S47, the average number of packets currently polled in the timer period is calculated, and the corresponding frequency down operation is performed in combination with the total sleep time, and the total sleep time, the number of polling packet receiving actions, and the number of packets received are set to 0.
An electronic device, comprising:
a memory for storing a computer program;
and the processor is used for realizing the steps of the CPU resource dynamic control method of DPDK real-time flow when executing the computer program.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of a method for dynamic control of CPU resources such as DPDK real time traffic.
Compared with the prior art, the invention has the remarkable advantages that:
1) The number and the distribution of threads are dynamically adjusted, the flow change is monitored in real time and predicted, the CPU core distribution is determined in advance, and the higher system performance is kept under the condition of different flows; meanwhile, tasks are reasonably distributed to a plurality of CPU cores, so that overload of a single core is avoided, and load balancing is realized.
2) By adjusting the CPU occupancy rate and the frequency, the system can flexibly allocate resources according to the actual flow demand, avoid resource waste, better cope with burst flow and reduce system delay; the system can reduce the running speed of the CPU according to the actual demands, thereby reducing energy consumption, temperature, abrasion and noise, improving system stability and adapting to different working loads, being beneficial to saving energy, prolonging the service life of the CPU, reducing running cost and improving system reliability and running efficiency.
Drawings
FIG. 1 is a flow chart of a method for dynamically controlling CPU resources in an embodiment of the invention.
Fig. 2 is a schematic structural diagram of an LSTM model in an embodiment of the present invention.
FIG. 3 is a schematic diagram of a prediction process flow based on the LSTM model in an embodiment of the invention.
FIG. 4 is a flow chart of a CPU resource management policy in an embodiment of the invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present invention and should not be construed as limiting the invention.
These and other aspects of embodiments of the invention will be apparent from and elucidated with reference to the description and drawings described hereinafter. In the description and drawings, particular implementations of embodiments of the invention are disclosed in detail as being indicative of some of the ways in which the principles of embodiments of the invention may be employed, but it is understood that the scope of the embodiments of the invention is not limited correspondingly. On the contrary, the embodiments of the invention include all alternatives, modifications and equivalents as may be included within the spirit and scope of the appended claims.
The computer comprises a CPU core, a memory and input/output equipment, and the purpose of adopting multithreading is to fully utilize the CPU core and improve the utilization rate of computer resources. The CPU core is a place for sending an execution command, and can only read instructions from a memory to execute, so that multithreading scheduling is realized. The multithreading scheduling aims at transmitting the thread number corresponding to the CPU core to a DPDK packet receiving queue and distributing CPU time slices when the CPU core is in an executable state; the thread can be executed until the CPU time slice ends.
The CPU resource dynamic control method of DPDK real-time flow polls to obtain real-time flow, inputs the current DPDK real-time flow into a preset LSTM model to obtain predicted future flow; when the current DPDK packet receiving thread capacity does not meet the predicted future flow, adjusting the number of the DPDK packet receiving threads, and reallocating the distribution of each CPU core according to an adjustment result to finish the dynamic adjustment of CPU resources;
The method comprises the steps of calculating the variance between DPDK real-time flow and predicted future flow obtained after dynamic adjustment of CPU resources is completed: when the calculated variance is larger than a set threshold, inputting the DPDK real-time flow obtained after the dynamic adjustment of the CPU resource into a preset LSTM model to re-predict the future flow, and judging whether the DPDK packet receiving thread capacity after the dynamic adjustment of the CPU resource meets the re-predicted future flow; otherwise, after a preset time interval, inputting the acquired DPDK real-time flow into a preset LSTM model again to re-predict the future flow, and judging whether the DPDK packet receiving thread capacity after the preset time interval meets the re-predicted future flow.
Specifically, as shown in fig. 1, the method for predicting the real-time DPDK traffic through the LSTM model and dynamically adjusting the number of DPDK packet receiving threads and the number and distribution of corresponding CPU cores in combination with the predicted future traffic trend mainly includes the following steps:
S10, collecting and preprocessing historical flow data, and building and training an LSTM model.
S20, collecting real-time flow based on DPDK, and monitoring CPU core load in real time.
S30, predicting real-time flow data counted by DPDK based on an LSTM model to obtain predicted future flow trend.
S40, judging whether the number of DPDK packet receiving threads needs to be increased or decreased according to the predicted future flow trend, if so, executing the step S50, otherwise, returning to the step S20.
S50, determining the number of DPDK packet receiving threads to be increased or decreased according to the predicted future flow trend, and reallocating CPU cores.
S60, repeating the steps S10-S50 to realize continuously optimized CPU core scheduling.
S70, updating the LSTM model according to the collected flow data at regular intervals so as to adapt to new flow conditions.
Specifically, the number of DPDK packet receiving threads is adjusted as follows:
If the number of the current DPDK wrapping threads is lower than the number of DPDK wrapping threads required by the predicted future flow, increasing the number of the DPDK wrapping threads, and reallocating CPU cores according to an increase result, so that the distribution number of the reallocated CPU cores meets the use of the predicted future flow;
And if the number of the current DPDK wrapping threads exceeds the number of DPDK wrapping threads required by the predicted future flow, reducing the number of the DPDK wrapping threads, and reallocating CPU cores according to the reduction result, so that the distribution number of the reallocated CPU cores meets the use of the predicted future flow.
Specifically, the dynamic adjustment of the CPU resource further includes balancing the occupancy rate of each CPU core by executing a CPU resource management policy on a single DPDK wrapping thread, and dynamically adjusting the CPU resource allocation policy according to the monitoring of the DPDK wrapping traffic, when the load condition of any one CPU core changes, so as to ensure that the system can cope with different traffic demands. In addition, the policy also considers the DPDK itself packet receiving parameters such as the receiver descriptor size. The CPU resource management strategy is beneficial to realizing reasonable utilization of CPU resources and improving the stability and performance of the system.
Specifically, executing the CPU resource management policy includes:
S41, counting the number of DPDK polling actions and the number of received packets;
S42, judging whether the number of the polling packets is 0, if so, adding 1 to the number of empty packets of the packet receiving queue, entering step S43, otherwise, setting 0 to the number of empty packets of the packet receiving queue, calculating a frequency lifting parameter of the packet receiving queue, and entering step S44;
s43, judging whether the accumulated number of the empty packet times of the packet receiving queue is larger than a set empty packet threshold value,
If yes, calculating dormancy parameters according to the number of empty packets and adding 1 to the number of dormancy, entering into step S44, otherwise returning to step S41 to continue statistics;
S44, comparing the dormancy times with the total number of DPDK packet receiving queues corresponding to all network card ports of the DPDK, if the dormancy times are equal, entering a step S45, otherwise, entering a step S46;
S45, when the number of times of dormancy is equal to the total number of DPDK packet receiving queues corresponding to all the network card ports of the DPDK, traversing all the network card ports of the DPDK, finding out the minimum dormancy parameter in all the network card packet receiving queues, taking dormancy action according to the minimum dormancy parameter, yielding a corresponding CPU time slice, balancing the occupancy rate of a CPU core, accumulating dormancy time, and entering step S47;
S46, when the number of times of dormancy is unequal to the total number of DPDK packet receiving queues corresponding to all network card ports of the DPDK, traversing all the DPDK packet receiving queues, finding out the maximum frequency parameter in all the DPDK queues, adopting corresponding frequency lifting operation according to the maximum frequency lifting parameter, and entering step S47;
S47, judging whether a DPDK timer expires, if so, adjusting the frequency of a CPU core according to the number of the DPDK packets received and the accumulated sleep time of the current polling; otherwise, the process returns to step S41.
The above steps are cycled.
The total number of DPDK packet receiving queues corresponding to all network card ports of the DPDK is obtained by calculating the product of the number of the network card ports of the DPDK and the number of the DPDK packet receiving queues.
Specifically, in step S47, the average number of packets currently polled in the timer period is calculated, and the corresponding frequency down operation is performed in combination with the total sleep time, and the total sleep time, the number of polling packet receiving actions, and the number of packets received are set to 0.
As shown in fig. 2, tanh and δ are two commonly used activation functions, constructing the LSTM model:
it ＝(wi·[ht-1,xt]+bi) (1)
In the formula (1): i t denotes an input gate, w i denotes a weight of the input gate, h t-1 denotes a hidden state, x t denotes input data, and b i denotes a bias term of the input gate;
ot＝(wo·[ht-1,xt]+bo) (2)
In the formula (2): o t denotes an output gate, w o denotes a weight of the output gate, and b o denotes a bias term of the output gate;
ft ＝σ(wf·[ht-1,xt]+bf) (3)
In the formula (3): f t denotes a forget gate, w f denotes a weight of the forget gate, and b f denotes a bias term of the forget gate;
ht＝ot * tanh(ct) (4)
In the formula (4): h t denotes output, c t denotes cell status.
The following describes the operation procedure of the dynamic control method for CPU resource in detail in combination with the specific application scenario of the present invention.
As shown in fig. 3, the method for dynamically controlling the CPU resource includes the following steps:
A1, collecting historical flow data including but not limited to flow size, flow type and flow change trend, preprocessing the collected data including data cleaning, missing value filling and data normalization to obtain a preprocessed network flow time sequence; and training the preprocessed data by using the LSTM model, and establishing a model capable of predicting the future flow trend.
A2, acquiring real-time flow data and monitoring the load condition of the CPU core in real time.
And A3, inputting the real-time flow data into a trained LSTM model to obtain a predicted future flow trend.
And A4, evaluating whether the current DPDK packet receiving thread capacity can meet the predicted future flow trend according to the predicted future flow trend.
A5, calculating the number of threads which need to be increased or decreased according to the result of the step A4, and reallocating CPU cores after adjusting the number of DPDK packet receiving threads so as to fully utilize the calculation resources.
And A6, collecting actual flow data within 1 minute after the CPU core is scheduled, comparing the actual flow data with predicted future flow trend, and calculating variance. And if the variance is larger than the set threshold, immediately re-executing the step A3, otherwise executing the step A7.
A7, repeatedly executing the steps A3-A6 at intervals of 5 minutes to realize continuous optimized CPU core scheduling.
And A8, periodically adding new flow data into the historical flow data for updating the LSTM model. The LSTM model is retrained using the updated dataset to improve the predictive accuracy of the model.
As shown in fig. 4, the flow of the CPU resource management policy single thread based on DPDK real-time traffic includes the following steps:
b1, counting the number of DPDK polling packet receiving actions (nb_interval_ looped) and the number of packets (nb_rx_processed).
B2, judging whether the number of the polling packets is 0 or not, if so, adding 1 to the number of empty packets (rx_queue- & gt zero_rx_packet_count) of the packet receiving queue, and executing a step B4; otherwise, set (rx_queue→zero_rx_packet_count) to 0, and execute step B3.
B3, calculating the frequency up parameter (rx_queue→freq_up_hit), if the DD bit (Descriptor Done) of the receiver Descriptor at max_pkt_burst) 3 is set to 1, indicating that it has been used, then (rx_queue→freq_up_hit) is the HIGHEST frequency parameter (freq_high), if the DD bit of the Rx Descriptor at max_pkt_burst 2 is set to 1, then the frequency trend (freq_ trend) is increased by 100, if the DD bit of the Rx Descriptor at max_pkt_burst is set to 1, then the frequency trend (freq_ trend) is increased by 1, otherwise, the current frequency is maintained. When the frequency trend (freq_ trend) is accumulated to be greater than 10000, (rx_queue →
Freq_up_hit) is a boost frequency parameter (freq_high), and the frequency trend (freq_ trend) is set to 0; step B6 is performed.
B4, judging whether (rx_queue- & zero_rx_packet_count) is larger than a threshold value (empty_limit, 5) of the number of empty packets, if so, executing the step B5, otherwise, returning to the step B1.
B5, calculating dormancy parameters (rx_queue- & gt idle_hit) of the packet receiving queue according to the number of empty packets, if (rx_queue- & gt zero_rx_packet_count) is smaller than 300, the dormancy parameters (rx_queue- & gt idle_hit) are 1, otherwise, the total dormancy number (Slepp _count) is increased by 1; after the calculation is completed, step B6 is executed.
And B6, comparing the total dormancy number (Slepp _count) with the product of the number of ports of all network cards and the number of corresponding packet receiving queues, if the total dormancy number (Slepp _count) is equal to the product, the step B7 is executed, otherwise, the step B8 is executed, wherein the step B represents that all the packet receiving queues in the current polling are empty.
B7, traversing all the packet receiving queues, and finding out the minimum value rx_queue & fwdarw of the dormancy parameter in all the packet receiving queues
If the sleep parameter minimum value idle_hit_min is smaller than 300, yielding a CPU time slice of microseconds corresponding to the sleep parameter minimum value idle_hit_min, accumulating the total sleep time (total_sleep_time) in the timer period, otherwise switching to an interrupt mode, and switching back to a polling mode after waiting for triggering the interrupt. After the traversal process, step B9 is executed.
B8, traversing all the packet receiving queues, and finding out the maximum value rx_queue & fwdarw of the frequency parameter in all the packet receiving queues
Freq_up_hit (scaleup _hit_max), if the frequency parameter maximum scaleup _hit_max is freq_hit, the corresponding CPU core frequency of the thread is raised to the maximum, if the frequency parameter maximum scaleup _hit_max is freq_hit, the first CPU core frequency is raised. After the traversal process, step B9 is executed.
And B9, judging whether the timer expires, executing the step B10 due to the expiration, and if not, jumping to the step B1.
B10, if the total_sleep_time in the timer period is greater than 25 microseconds, the CPU core frequency is lowered by one level, the average number of polling packets (avg_packet_num) in a single time in the timer period is calculated, and if the average number of polling packets (avg_packet_num) is smaller than MAX_PKT_BURST divided by 10, the CPU core frequency is lowered by one level. The nb_rx_ processed, total _sleep_time, nb_iteration_ looped are set to 0, otherwise the current frequency is maintained and steps B1-B10 are looped.
It should be noted that: the sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. And the foregoing description has been directed to specific embodiments of this application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
The embodiments of the present application are described in a progressive manner, and the same and similar parts of the embodiments are all referred to each other, and each embodiment is mainly described in the differences from the other embodiments. In particular, for apparatus, devices and storage medium embodiments, the description is relatively simple as it is substantially similar to method embodiments, with reference to the description of method embodiments in part.
Those skilled in the art will appreciate that implementing all or part of the above-described methods may be accomplished by way of a computer program, which may be stored on a non-transitory computer readable storage medium and which, when executed, may comprise the steps of the above-described embodiments of the methods. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The above examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.

Claims (7)

1. A dynamic control method for CPU resources of DPDK real-time flow is characterized in that: the CPU resource dynamic control method polls to obtain real-time flow, and inputs the current DPDK real-time flow into a preset LSTM model to obtain predicted future flow; when the current DPDK packet receiving thread capacity does not meet the predicted future flow, adjusting the number of the DPDK packet receiving threads, and reallocating the distribution of each CPU core according to an adjustment result to finish the dynamic adjustment of CPU resources;
The method comprises the steps of calculating the variance between DPDK real-time flow and predicted future flow obtained after dynamic adjustment of CPU resources is completed: when the calculated variance is larger than a set threshold, inputting the DPDK real-time flow obtained after the dynamic adjustment of the CPU resource into a preset LSTM model to re-predict the future flow, and judging whether the DPDK packet receiving thread capacity after the dynamic adjustment of the CPU resource meets the re-predicted future flow; otherwise, after a preset time interval, inputting the acquired DPDK real-time flow into a preset LSTM model again to re-predict the future flow, and judging whether the DPDK packet receiving thread capacity after the preset time interval meets the re-predicted future flow.
2. The method for dynamically controlling the CPU resource of the DPDK real-time traffic according to claim 1, wherein: the number of DPDK packet receiving threads is adjusted as follows:
If the number of the current DPDK wrapping threads is lower than the number of DPDK wrapping threads required by the predicted future flow, increasing the number of the DPDK wrapping threads, and reallocating CPU cores according to an increase result, so that the distribution number of the reallocated CPU cores meets the use of the predicted future flow;
And if the number of the current DPDK wrapping threads exceeds the number of DPDK wrapping threads required by the predicted future flow, reducing the number of the DPDK wrapping threads, and reallocating CPU cores according to the reduction result, so that the distribution number of the reallocated CPU cores meets the use of the predicted future flow.
3. The method for dynamically controlling the CPU resource of the DPDK real-time traffic according to claim 2, wherein: the dynamic adjustment of the CPU resource further comprises balancing the occupancy rate of each CPU core by executing a CPU resource management policy on a single DPDK wrapping thread, wherein the CPU resource management policy is as follows:
S41, counting the number of DPDK polling actions and the number of received packets;
S42, judging whether the number of the polling packets is 0, if so, adding 1 to the number of empty packets of the packet receiving queue, entering step S43, otherwise, setting 0 to the number of empty packets of the packet receiving queue, calculating a frequency lifting parameter of the packet receiving queue, and entering step S44;
s43, judging whether the accumulated number of the empty packet times of the packet receiving queue is larger than a set empty packet threshold value,
If yes, calculating dormancy parameters according to the number of empty packets and adding 1 to the number of dormancy, entering into step S44, otherwise returning to step S41 to continue statistics;
S44, comparing the dormancy times with the total number of DPDK packet receiving queues corresponding to all network card ports of the DPDK, if the dormancy times are equal, entering a step S45, otherwise, entering a step S46;
S45, when the number of times of dormancy is equal to the total number of DPDK packet receiving queues corresponding to all the network card ports of the DPDK, traversing all the network card ports of the DPDK, finding out the minimum dormancy parameter in all the network card packet receiving queues, taking dormancy action according to the minimum dormancy parameter, yielding a corresponding CPU time slice, balancing the occupancy rate of a CPU core, accumulating dormancy time, and entering step S47;
S46, when the number of times of dormancy is unequal to the total number of DPDK packet receiving queues corresponding to all network card ports of the DPDK, traversing all the DPDK packet receiving queues, finding out the maximum frequency parameter in all the DPDK queues, adopting corresponding frequency lifting operation according to the maximum frequency lifting parameter, and entering step S47;
S47, judging whether a DPDK timer expires, if so, adjusting the frequency of a CPU core according to the number of the DPDK packets received and the accumulated sleep time of the current polling; otherwise, the process returns to step S41.
4. The method for dynamically controlling CPU resources of DPDK real-time traffic according to claim 3, wherein: and the total number of DPDK packet receiving queues corresponding to the ports of all network cards of the DPDK is obtained by calculating the product of the number of the ports of all network cards of the DPDK and the number of the DPDK packet receiving queues.
5. The method for dynamically controlling the CPU resource of the DPDK real-time traffic according to claim 4, wherein: in step S47, the average number of packets currently polled in the timer period is calculated, and the corresponding frequency down operation is performed in combination with the total sleep time, and the total sleep time, the number of polling packet receiving actions, and the number of packets received are set to 0.
6. An electronic device, characterized in that: comprising the following steps:
a memory for storing a computer program;
a processor, configured to implement the steps of the method for dynamically controlling CPU resources of DPDK real-time traffic according to any one of claims 1 to 5 when executing the computer program.
7. A computer-readable storage medium, characterized by: the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the method for dynamically controlling CPU resources of DPDK real-time traffic according to any one of claims 1 to 5.
CN202410126429.9A 2024-01-30 2024-01-30 CPU resource dynamic control method for DPDK real-time flow Pending CN117978694A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410126429.9A CN117978694A (en) 2024-01-30 2024-01-30 CPU resource dynamic control method for DPDK real-time flow

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410126429.9A CN117978694A (en) 2024-01-30 2024-01-30 CPU resource dynamic control method for DPDK real-time flow

Publications (1)

Publication Number Publication Date
CN117978694A true CN117978694A (en) 2024-05-03

Family

ID=90859070

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410126429.9A Pending CN117978694A (en) 2024-01-30 2024-01-30 CPU resource dynamic control method for DPDK real-time flow

Country Status (1)

Country Link
CN (1) CN117978694A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118555244A (en) * 2024-06-25 2024-08-27 新华三技术有限公司 Message flow processing method, device, equipment and readable storage medium
CN118842773A (en) * 2024-07-30 2024-10-25 济南浪潮数据技术有限公司 Network traffic management method, device and equipment of cloud platform and storage medium
CN120415548A (en) * 2025-07-01 2025-08-01 远江盛邦安全科技集团股份有限公司 CPU load monitoring method based on dynamic compensation for satellite-borne network equipment

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118555244A (en) * 2024-06-25 2024-08-27 新华三技术有限公司 Message flow processing method, device, equipment and readable storage medium
CN118842773A (en) * 2024-07-30 2024-10-25 济南浪潮数据技术有限公司 Network traffic management method, device and equipment of cloud platform and storage medium
CN120415548A (en) * 2025-07-01 2025-08-01 远江盛邦安全科技集团股份有限公司 CPU load monitoring method based on dynamic compensation for satellite-borne network equipment

Similar Documents

Publication Publication Date Title
CN117978694A (en) CPU resource dynamic control method for DPDK real-time flow
US11886919B2 (en) Directing queries to nodes of a cluster of a container orchestration platform distributed across a host system and a hardware accelerator of the host system
US7191446B2 (en) Method for resource management in a real-time embedded system
KR20130127508A (en) Method and apparatus of smart power management for mobile communication terminals using power thresholds
CN116302578B (en) QoS (quality of service) constraint stream application delay ensuring method and system
CN111580949B (en) Automatic regulating method for network packet receiving mode
CN116185584A (en) A Multi-tenant Database Resource Planning and Scheduling Method Based on Deep Reinforcement Learning
US20210124615A1 (en) Thread scheduling based on performance metric information
CN115225986A (en) Adaptive OSU bandwidth adjustment method and device
CN119583560A (en) Method and system for improving mine communication signals based on edge computing
CN119377045A (en) Convolutional neural network reasoning acceleration system and method based on heterogeneous architecture
CN114185674A (en) CPU task real-time scheduling method, device, equipment and storage medium
CN108241526A (en) ADAPTIVE CONTROL based on model prediction
CN118069291A (en) Elastic capacity expansion and contraction method based on service and result guidance
CN120653370B (en) Virtual machine data migration methods, systems, devices, and storage media
CN119094471B (en) Control method, device, equipment and storage medium of single-chip switch
CN120560939A (en) Server power consumption management method, device, electronic device and storage medium
CN117971484A (en) Threshold adjustment method for transaction current limiting control and electronic equipment
TWI584667B (en) Method for request scheduling and scheduling device
CN118677779A (en) Optimized management method based on network resource scheduling
CN103793041B (en) The method for managing power supply and device of multi-core symmetric multiprocessing system
CN118535333A (en) A method and system for elastic scaling of operators under fluctuating data streams
CN115470006B (en) A Load Balancing Method Based on Microkernel
CN120540859A (en) Capacity adjustment method, device, storage medium and electronic device
CN120179369B (en) A dynamic adaptive interrupt processing method and interrupt processing system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination