CN115037658B

CN115037658B - BPF-based metering master station network detection method and metering master station

Info

Publication number: CN115037658B
Application number: CN202210640720.9A
Authority: CN
Inventors: 孙勇; 李经儒; 刘日荣; 蔡乾乾; 阙华坤; 黄家嘉; 黄友朋; 张捷; 彭策; 唐曦凌
Original assignee: Guangdong Power Grid Co Ltd; Measurement Center of Guangdong Power Grid Co Ltd
Current assignee: Guangdong Power Grid Co Ltd; Measurement Center of Guangdong Power Grid Co Ltd
Priority date: 2022-06-08
Filing date: 2022-06-08
Publication date: 2023-05-02
Anticipated expiration: 2042-06-08
Also published as: CN115037658A

Abstract

The invention discloses a network detection method of a metering master station based on BPF and the metering master station, wherein the method comprises the following steps: loading the compiled BPF function into a Linux kernel of a metering master station, and establishing a key value storage area of the BPF function in the Linux kernel; replacing callback of a data packet receiving processing function by the BPF function to acquire TCP connection information; replacing callback of the fast retransmission function with the BPF function, and storing congestion states before and after executing the fast retransmission function in the key value storage area; and detecting a zone bit corresponding to the slow start exit point by using the BPF function to obtain the exit condition of the mixed slow start. By adopting the method and the device, the events in the Linux kernel can be tracked, detected and even modified by calling the BPF function without recompiling the kernel, so that the network congestion condition of the metering master station can be effectively and efficiently analyzed.

Description

BPF-based metering master station network detection method and metering master station

Technical Field

The invention relates to the technical field of network monitoring, in particular to a network detection method of a metering master station based on a BPF and the metering master station.

Background

The metering master station generally uses a Linux operating system, a current Linux kernel default cubic algorithm is used as a congestion control algorithm, and the kernel is generally required to be recompiled when the problems of a congestion state machine, a TCP (transmission control protocol) transmission buffer queue, mixed slow start and the like of the metering master station adopting the algorithm are analyzed at present, so that the congestion cause is not easy to detect. And specific network events cannot be tracked and located when network congestion analysis is performed.

Disclosure of Invention

The embodiment of the invention provides a BPF-based metering master station network detection method and a metering master station, which are used for tracking modification events in a Linux kernel without recompiling the kernel and tracking and positioning network events.

To achieve the above object, a first aspect of an embodiment of the present application provides a method for detecting a metering master station network based on a BPF, including:

loading the compiled BPF function into a Linux kernel of a metering master station, and establishing a key value storage area of the BPF function in the Linux kernel;

replacing callback of a data packet receiving processing function by the BPF function to acquire TCP connection information;

replacing callback of the fast retransmission function with the BPF function, and storing congestion states before and after executing the fast retransmission function in the key value storage area;

and detecting a zone bit corresponding to the slow start exit point by using the BPF function to obtain the exit condition of the mixed slow start.

In a possible implementation manner of the first aspect, the BPF function is BPF assembly code compiled by an LLVM compiler; and the variable values stored in the key value storage area of the BPF function can be called by other functions in the Linux kernel.

In a possible implementation manner of the first aspect, the replacing a callback of a packet receiving processing function with the BPF function, and obtaining TCP connection information specifically includes:

and extracting a sending queue statistic value of the socket from the TCP structure body, and judging the network congestion degree according to the change of the sending queue statistic value.

In a possible implementation manner of the first aspect, the replacing the callback of the fast retransmission function with the BPF function stores the state of the congestion state machine before and after executing the fast retransmission function in the key value storage area, specifically includes:

and inserting a Kprobe and a Kretprobe probe into an inlet and an outlet of the fast retransmission function respectively, and dynamically storing the state of the congestion state machine in the key value storage area if the same connection control structure exists before and after the fast retransmission function is executed.

In a possible implementation manner of the first aspect, the detecting, with the BPF function, a flag bit corresponding to a slow start exit point to obtain an exit condition of hybrid slow start specifically includes:

detecting a HYSTART_ACK_TRAIN flag bit by using the BPF function to obtain an ACK rank measurement start time stamp and a minimum path transmission delay;

if the difference of the current time minus the ACK rank measurement start time stamp is larger than a rank threshold, the network condition is deteriorated, and slow start and exit are performed; the rank threshold value is related to the minimum path transmit delay.

detecting a HYSTART_DELAY flag bit by using the BPF function to obtain a curr_RTT value and a minimum path transmission DELAY; if the difference between the curr_RTT value and the minimum path transmission delay is greater than a delay threshold, the delay is excessively increased, and the slow start is exited; the value of the delay threshold is related to the minimum path transmission delay.

In a possible implementation manner of the first aspect, the TCP connection information includes TCP basic connection information, process control information, a congestion window, a slow start threshold, and a transmit queue buffer.

In a possible implementation manner of the first aspect, the BPF function is executed as a Kprobe processing program.

The second aspect of the embodiment of the application provides a metering master station based on BPF, which comprises a linux kernel of a BPF function, a network information analysis module, a congestion state analysis module and a slow start analysis module;

the network information analysis module is used for calling the BPF function to replace a callback of a data packet receiving processing function and acquiring TCP connection information;

the congestion state analysis module is used for calling the BPF function to replace a callback of the fast retransmission function and storing the congestion state before and after executing the fast retransmission function in the key value storage area;

and the slow start analysis module is used for calling the BPF function to detect the zone bit corresponding to the slow start exit point and obtaining the exit condition of the mixed slow start.

Compared with the prior art, the BPF-based metering master station network detection method and the metering master station provided by the embodiment of the invention realize a group of enhanced BPF virtual machines in a Linux kernel. After loading the compiled BPF function into a Linux kernel of the metering master station and establishing a key value storage area of the BPF function in the Linux kernel, the Linux kernel triggers an enhanced BPF program defined by the metering master station under a corresponding type of event, such as an enhanced BPF program of a Kprobe type, allows the enhanced BPF program written for the metering master station to serve as a Kprobe processing program, and when the function N is executed, the callback inserted by an early kernel module is not executed, but the enhanced BPF program dynamically inserted by the metering master station mode is executed. The relevant network functions of the Linux kernel of the measuring master station can be detected in real time by utilizing a KBrobe type enhanced BPF program, and network parameter information can be acquired in real time. In addition, events in the Linux kernel can be tracked, detected and even modified by calling the BPF function, and the kernel does not need to be recompiled, so that the network congestion condition of the metering master station is effectively and efficiently analyzed.

Drawings

Fig. 1 is a schematic flow chart of a network detection method of a metering master station based on a BPF according to an embodiment of the present invention;

figure 2 is a schematic diagram of BPF function execution under consideration of an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Referring to fig. 1, an embodiment of the present invention provides a network detection method of a metering master station based on a BPF, including:

s10, loading the compiled BPF function into a Linux kernel of a metering master station, and establishing a key value storage area of the BPF function in the Linux kernel.

S11, replacing callback of the data packet receiving processing function with the BPF function to acquire TCP connection information.

S12, replacing callback of the fast retransmission function with the BPF function, and storing congestion states before and after executing the fast retransmission function in the key value storage area.

S13, detecting a zone bit corresponding to the slow start exit point by using the BPF function to obtain the exit condition of the mixed slow start.

The method of specifying whether and when to call certain kernel functions in certain scenarios is to add log prints to the corresponding functions in the kernel code, which typically requires recompilation of the kernel. However, the advent of Kprobes made it easier to detect. When the Linux kernel executes the specified probe function by inserting the kernel module, a callback function defined by the metering master station is called. The implementation principle of Kprobes in Linux Arm architecture is shown in fig. 1. The code on the enabling point is saved and replaced with a breakpoint instruction. When this breakpoint is executed, a trap instruction will be generated, a register saved, and then the associated detection handler will be skipped. The three methods of the Kprobes mechanism are Kprobe, jprobe and Kret probes, respectively. The Kprobe may be inserted into any instruction to be detected, and the Kret probe may obtain the return value of the probe function, and the Jprobe is used to obtain the input parameter value of the probe function.

The BPF function in this embodiment may be understood as implementing a set of enhanced BPF virtual machines in a Linux kernel to execute the enhanced BPF instructions inserted by the metering master. By programming the enhanced BPF program types, BPF assembly code compiled by the LLVM compiler may be loaded into the Linux kernel of the metering master. The Linux kernel triggers an enhanced BPF program defined by the metering master station under a corresponding type of event, such as an enhanced BPF program of a Kprobe type, and allows the enhanced BPF program written by the metering master station to serve as a Kprobe processing program.

Referring to fig. 2, when a Linux kernel function N of the metering master is detected using Kprobe, when the function N is executed, callback inserted by an early kernel module is not executed, but an enhanced BPF program dynamically inserted by the metering master mode is executed. The relevant network functions of the Linux kernel of the measuring master station can be detected in real time by utilizing a KBrobe type enhanced BPF program, and network parameter information can be acquired in real time.

Illustratively, the BPF functions are BPF assembly code compiled by the LLVM compiler; and the variable values stored in the key value storage area of the BPF function can be called by other functions in the Linux kernel.

Illustratively, the replacing the callback of the packet receiving processing function with the BPF function, and obtaining the TCP connection information specifically includes:

Basic information about network congestion includes basic TCP connection information, the size of the current congestion window, the size of the current slow start threshold, and the amount of data sent to the buffer. In the Linux kernel (version 4.15), a "tcp_rcv_attached" function (packet reception processing function) is used to process the connection state. Table 1 shows some important parameters of the function.

Important parameters of Table 1"TCP RCV ESTABLISHED'

Parameters (parameters)	Purpose(s)
		Structure sock (sk)	Including transport layer protocol related fields
Structure sk_buff (skb)	Representing a current data packet
		Structure tcphdr (th)	TCP header information

As can be seen from the parameters listed in table 1, all the necessary basic information of the network connection can be obtained from the structure sock. Based on these information, the present embodiment obtains network information currently in the TCP fully connected state by using the BPF function. In the Linux kernel of the metering master, congestion related information about the current TCP connection is stored in the structure tcp_sock. From the currently detected functions, there is no such structure in the function parameters, so the structure tcp_sock can only be obtained from the existing structure, and the real-time congestion window, slow start threshold and transmit queue buffer are obtained from the known sock. The data packet processed by the Linux kernel of the metering master station can be represented by skb, and the size of all skb in the sending queue cache is sk_wmem_queue. The size of sk_wmem_queue is dynamically extracted using the eBPF technique. Its fluctuations may indirectly reflect the extent of current network congestion.

In addition, the overall state of the current network can also be detected through the dynamic change relation between the congestion window and the slow start threshold. In the cubic congestion control algorithm, the cubic window growth function is shown in equation 1, where t is the time elapsed since the last congestion window reduction, and C is a known parameter of the equation. In the Linux kernel of the metering master, C is generally set to 0.4, the value of which is determined by statistical verification in a large number of environments, K is the time for the function to increase W without further packet loss, and the calculation formula is shown in formula 2.

W(t)＝C(t-K) ³ +W _max (1)

The next step is to obtain basic information about the current TCP connection procedure. All information about a process may be obtained by obtaining a current process and a process control block for the current process. Once the function to be detected and the intermediate parameters for acquiring the relevant information are determined, an enhanced BPF Map, which is a key value store residing in the kernel, can be defined in the enhanced BPF function of the Kprobe type. Any BPF function that knows these maps can be accessed. Programs running in the metering master space may also access these maps using file descriptors. Any type of data may be stored in the Map as long as the data size is correctly specified in advance. When a BPF function of a Kprobe type is installed on a tcp_rcv_scheduled function, a Linux kernel of a metering master station receives an acknowledgement message from a receiving end and runs the function, triggers an enhanced BPF function, and acquires TCP basic connection information, process control information, a congestion window, a slow start threshold, a transmission queue buffer and the like through the function. These dynamically obtained variable values will be shared with other functions of the metering master core through maps.

Illustratively, the replacing the callback of the fast retransmission function with the BPF function stores the state of the congestion state machine before and after executing the fast retransmission function in the key-value storing area, specifically including:

After the TCP connection is established, the Linux congestion state machine of the metering master station decides whether to reduce the congestion window, maintains the congestion window and continuously increases the congestion window through each state, and the improper processing can cause the data packet to be lost or overtime. The Linux congestion state machine of the metering master station comprises five states, which transmit under certain conditions.

The five states of the congestion state machine change in real time in the Linux kernel of the metering master. The congestion state machine jumps to handle various packet losses and marks the current congestion state of the metering master. For example, after a TCP connection is successful, the network is in an open state and the sender enters the CWR state upon receiving an acknowledgement packet containing a congestion notification flag. When a timeout occurs, the sender will enter the "lost" state from the "on" state. When only a suspicious ACK, such as an unordered ACK, is received, the system enters an unordered state from the open state. When three duplicate acknowledgement packets are received consecutively, the system will enter a recovery state. In any state, the RTO enters a lost state.

The embodiment calls a BPF function for detecting the congestion state machine of the Linux kernel of the whole metering master. When this state jumps, the old congestion state and the new congestion state are recorded. The congestion state machine is controlled by a tcp_failures_alert function in the Linux kernel of the metering master from which the overall jump condition is determined. Since the state switching process needs to be detected, the break point needs to be handled using both Kprobe and Kretprobe mechanisms. The current congestion state is captured as the value of Map before entering the congestion state machine processing function, at which time the structure body Sock pointer is used as the key for that state. It is understood that BPF functions insert Kprobe and Kretprobe at the function's ports and ports, respectively, and that defined BPF functions record the congestion state machine functions before and after execution. If the same connection control structure is found before and after the function execution, the congestion state is dynamically extracted.

The detecting, by using the BPF function, a flag bit corresponding to a slow start exit point to obtain an exit condition of hybrid slow start specifically includes:

In order to solve the problem of a large number of data packets lost due to the rapid growth of slow start, the cubic algorithm proposes a hybrid slow start, which can ensure that the slow start can be completed "safely". Hybrid slow start exits hybrid slow start in two ways, the first being an ACK rank length measurement method. The length of the ACK rank is the sum of the arrival intervals of the closely adjacent ACK packets within one RTT period. A series of ACK packets with a default interval of less than or equal to 2ms is one ACK rank. For each RTT period, the ACK rank length is calculated and compared to the estimated minimum path transmission delay. If the current time minus the ACK rank measurement start timestamp round_start is greater than half the minimum RTT delay, then this indicates that the network condition is deteriorating and slow-start exit is initiated. The second method is to measure the increase in packet delay. When the sampled curr_RTT value is greater than the minimum RTT value plus 1/8 of the minimum RTT value, the delay is considered to be excessively increased, and slow start is exited.

In the Linux kernel of the metering master station, the cubic algorithm exits from slow start through two detection methods. For example, when a slow-start exit point is detected by the ACK rank length measurement method, the flag bit is set to hystart_ack_train. Then, by enhancing the BPF to detect the flag bit, the congestion window, the threshold value and the data of the measured slow-start exit point at this time can be obtained, and the exit condition of the mixed slow start at this time can be intuitively seen.

When a slow-start exit point is detected by a measurement method that measures an increase in packet DELAY, the flag bit is set to HYSTART_DELAY. Then, by enhancing the BPF to detect the flag bit, the congestion window, the threshold value and the data of the measured slow-start exit point at this time can be obtained, and the exit condition of the mixed slow start at this time can be intuitively seen.

Illustratively, the TCP connection information includes TCP basic connection information, process control information, congestion windows, slow start thresholds, and transmit queue buffers.

Illustratively, the BPF function operates as a Kprobe handler.

The embodiment of the application provides a metering master station based on a BPF, which comprises a linux kernel of a BPF function, a network information analysis module, a congestion state analysis module and a slow start analysis module.

And the network information analysis module is used for calling the BPF function to replace a callback of the data packet receiving processing function and acquiring TCP connection information.

And the congestion state analysis module is used for calling the BPF function to replace a callback of the fast retransmission function and storing the congestion state before and after executing the fast retransmission function in the key value storage area.

Compared with the prior art, the metering master station based on the BPF provided by the embodiment of the invention realizes a group of enhanced BPF virtual machines in the Linux kernel. After loading the compiled BPF function into a Linux kernel of the metering master station and establishing a key value storage area of the BPF function in the Linux kernel, the Linux kernel triggers an enhanced BPF program defined by the metering master station under a corresponding type of event, such as an enhanced BPF program of a Kprobe type, allows the enhanced BPF program written for the metering master station to serve as a Kprobe processing program, and when the function N is executed, the callback inserted by an early kernel module is not executed, but the enhanced BPF program dynamically inserted by the metering master station mode is executed. The relevant network functions of the Linux kernel of the measuring master station can be detected in real time by utilizing a KBrobe type enhanced BPF program, and network parameter information can be acquired in real time. In addition, events in the Linux kernel can be tracked, detected and even modified by calling the BPF function, and the kernel does not need to be recompiled, so that the network congestion condition of the metering master station is effectively and efficiently analyzed.

It will be clear to those skilled in the art that, for convenience and brevity of description, reference may be made to the corresponding process in the foregoing method embodiment for the specific working procedure of the metering master station described above, which is not further described herein.

An embodiment of the application provides a server and a client based on a BPF. The server and the client are configured for data communication for the experimental environment. The operating system of both hosts is Ubuntu and the kernel version is linux4.15. In addition, simulation tools are also required to build network congestion environments. Two hosts are provided in the local area network, one acting as a server and the other as a client. The Iperf3 tool is used for communication between the client and the server and the TC tool is used for network congestion simulation. The experiments are divided into three groups, which respectively correspond to network congestion basic information measurement, congestion state machine measurement and hybrid slow start exit point measurement. Each set of corresponding enhanced BPF procedures is written as procedure 1, procedure 2 and procedure 3, respectively. The specific experimental steps are as follows:

step 1: when the network is normal, respectively running a program 1, a program 2 and a program 3;

step 2: the client sends 20971520 bytes (20M) of data to the server;

step 3: closing the program 1, the program 2 and the program 3 to obtain a measurement result;

step 4, setting congestion environment on the client: TCqdisc adds the development 33 root networks with 20%30%;

step 5: running procedure 1, procedure 2 and procedure 3 in a crowded network environment;

step 6: the client sends 20971520 bytes (20M) of data to the server;

step 7: the procedure 1, the procedure 2 and the procedure 3 are closed, and the measurement result is obtained.

In the experiment, the basic network message obtained by the program 1 is compared with the highest command of Linux of the metering master station, and whether the process information obtained by the current program is consistent with the basic network information obtained by tcpdump is verified. Congestion window variation can be obtained from procedure 1. By enhancing the BPF program, obvious phenomenon of packet loss timeout exists in congestion window change under the congestion environment can be obtained. In case of network congestion, the fluctuation of the transmission buffer queue length is much larger than that in the normal state, which means that the enhanced BPF procedure can indirectly reflect the degree of network congestion through the fluctuation of the transmission buffer queue length.

The dynamic transition of the congestion state machine and the calculated values of the normal network state and each state under the network congestion state can be obtained by the program 2 through data statistics. In normal network states, all states switch directly back and forth between tcp_ca_di sor and tcp_ca_open. When the states of tcp_ca_recovery and tcp_ca_loss enter the state of tcp_ca_loss multiple times, it means that packet Loss and timeout frequently occur on the current network. In addition, it is verified whether the enhanced BPF procedure can effectively detect network congestion.

Program 3 dynamically obtains the current mixing slow start and exit times. Equation 3 is a detection formula, and the call BPF function detection data also conforms to the formula.

While the foregoing is directed to the preferred embodiments of the present invention, it will be appreciated by those skilled in the art that changes and modifications may be made without departing from the principles of the invention, such changes and modifications are also intended to be within the scope of the invention.

Claims

1. The metering master station network detection method based on the BPF is characterized by comprising the following steps of:

2. The BPF-based metering master network detection method of claim 1, wherein the BPF function is BPF assembly code compiled by an LLVM compiler; and the variable values stored in the key value storage area of the BPF function can be called by other functions in the Linux kernel.

3. The method for detecting the metering master network based on the BPF as claimed in claim 1, wherein said replacing callback of the data packet receiving processing function with the BPF function, obtaining TCP connection information, comprises:

4. The BPF-based network detection method of metering master station of claim 1, wherein replacing callback of fast retransmission function with the BPF function, and saving congestion state machine state before and after executing fast retransmission function in the key value storage area, specifically comprises:

5. The method for detecting the metering master station network based on the BPF as claimed in claim 1, wherein the detecting the flag bit corresponding to the slow start exit point by using the BPF function, obtaining the exit condition of the hybrid slow start, specifically comprises:

6. The method for detecting the metering master station network based on the BPF as claimed in claim 1, wherein the detecting the flag bit corresponding to the slow start exit point by using the BPF function, obtaining the exit condition of the hybrid slow start, specifically comprises:

7. The BPF-based metering master network detection method of claim 1 wherein the TCP connection information includes TCP basic connection information, process control information, congestion window, slow start threshold and transmit queue buffer.

8. The BPF-based metering master network detection method of claim 1 wherein the BPF function operates as a Kprobe process.

9. The metering master station based on the BPF is characterized by comprising a l inux kernel of a BPF function, a network information analysis module, a congestion state analysis module and a slow start analysis module;

the congestion state analysis module is used for calling the BPF function to replace a callback of the fast retransmission function and storing congestion machine states before and after executing the fast retransmission function in the key value storage area;