CN115037658A

CN115037658A - Metering master station network detection method based on BPF and metering master station

Info

Publication number: CN115037658A
Application number: CN202210640720.9A
Authority: CN
Inventors: 孙勇; 李经儒; 刘日荣; 蔡乾乾; 阙华坤; 黄家嘉; 黄友朋; 张捷; 彭策; 唐曦凌
Original assignee: Guangdong Power Grid Co Ltd; Measurement Center of Guangdong Power Grid Co Ltd
Current assignee: Guangdong Power Grid Co Ltd; Measurement Center of Guangdong Power Grid Co Ltd
Priority date: 2022-06-08
Filing date: 2022-06-08
Publication date: 2022-09-09
Anticipated expiration: 2042-06-08
Also published as: CN115037658B

Abstract

The invention discloses a measurement master station network detection method based on BPF and a measurement master station, wherein the method comprises the following steps: loading the compiled BPF function into a Linux kernel of a metering master station, and establishing a key value storage area of the BPF function in the Linux kernel; replacing the call-back of a data packet receiving and processing function by the BPF function to acquire TCP connection information; replacing the call-back of the quick retransmission function with the BPF function, and storing congestion machine states before and after the quick retransmission function is executed in the key value storage area; and detecting the flag bit corresponding to the slow-start exit point by using the BPF function to obtain the exit condition of the mixed slow start. By adopting the invention, the events in the Linux kernel can be tracked, detected and even modified by calling the BPF function without recompiling the kernel, so that the network congestion condition of the metering master station is effectively and efficiently analyzed.

Description

Metering master station network detection method based on BPF and metering master station

Technical Field

The invention relates to the technical field of network monitoring, in particular to a metering master station network detection method based on BPF and a metering master station.

Background

The metering master station generally uses a Linux operating system, a current Linux kernel default cubic algorithm is used as a congestion control algorithm, and the kernel is generally required to be recompiled when the problems of a metering master station congestion state machine, a TCP (transmission control protocol) sending buffer area queue, hybrid slow start and the like which adopt the algorithm are analyzed at present, so that the cause of congestion is difficult to detect. And specific network events cannot be tracked and located when network congestion analysis is performed.

Disclosure of Invention

The embodiment of the invention provides a metering master station network detection method based on BPF and a metering master station, which can track modification events in a Linux kernel and track and position network events without recompiling the kernel.

In order to achieve the above object, a first aspect of the embodiments of the present application provides a metering master station network detection method based on BPF, including:

loading the compiled BPF function into a Linux kernel of a metering master station, and establishing a key value storage area of the BPF function in the Linux kernel;

replacing the callback of a data packet receiving and processing function by the BPF function to acquire TCP connection information;

replacing the callback of the quick retransmission function with the BPF function, and storing congestion machine states before and after the quick retransmission function is executed in the key value storage area;

and detecting the flag bit corresponding to the slow-start exit point by using the BPF function to obtain the exit condition of the mixed slow start.

In one possible implementation of the first aspect, the BPF function is BPF assembly code compiled by an LLVM compiler; and the variable values stored in the key value storage area of the BPF function can be called by other functions in the Linux kernel.

In a possible implementation manner of the first aspect, the replacing, by the BPF function, a callback of a packet reception processing function to obtain TCP connection information specifically includes:

and extracting the transmission queue statistical value of the socket from the TCP structure body, and judging the network congestion degree according to the change of the transmission queue statistical value.

In a possible implementation manner of the first aspect, the replacing, by the BPF function, a callback of a fast retransmission function, and storing, in the key value storage area, states of congestion state machines before and after the fast retransmission function is executed specifically includes:

inserting KProbe and KretProbe probes into the inlet and outlet of the quick retransmission function, and dynamically storing the state of the congestion state machine in the key value storage area if the same connection control structure exists before and after the quick retransmission function is executed.

In a possible implementation manner of the first aspect, the detecting, by using the BPF function, a flag bit corresponding to a slow-start exit point to obtain an exit condition of mixed slow start specifically includes:

detecting HYSTART _ ACK _ TRAIN flag bits by the BPF function to obtain ACK line measurement starting time stamps and minimum path sending delay;

if the difference value obtained by subtracting the ACK row and column measurement starting timestamp from the current time is larger than the row and column threshold value, the network condition is deteriorated, and the slow start exits; the rank threshold value is related to the minimum path transmit delay.

detecting HYSTART _ DELAY flag bit by using the BPF function to obtain a curr _ RTT value and minimum path sending DELAY; if the difference value between the curr _ RTT value and the minimum path sending delay is larger than a delay threshold value, the delay is increased excessively, and the slow start is quitted; the value of the delay threshold is related to the minimum path transmission delay.

In a possible implementation manner of the first aspect, the TCP connection information includes TCP basic connection information, process control information, a congestion window, a slow start threshold, and a sending queue buffer.

In one possible implementation of the first aspect, the BPF function operates as a Kprobe handler.

A second aspect of the embodiments of the present application provides a metering master station based on a BPF, including a linux kernel with a BPF function, a network information analysis module, a congestion state analysis module, and a slow start analysis module;

the network information analysis module is used for calling the BPF function to replace the callback of the data packet receiving and processing function and acquiring TCP connection information;

the congestion state analysis module is used for calling the BPF function to replace the call back of the quick retransmission function and storing the congestion machine states before and after the quick retransmission function is executed in the key value storage area;

and the slow start analysis module is used for calling the BPF function to detect the zone bit corresponding to the slow start exit point and obtaining the exit condition of the mixed slow start.

Compared with the prior art, the metering master station network detection method and the metering master station based on the BPF provided by the embodiment of the invention realize a group of enhanced BPF virtual machines in a Linux kernel. The method comprises the steps that a compiled BPF function is loaded into a Linux kernel of a metering master station, after a key value storage area of the BPF function is established in the Linux kernel, the Linux kernel triggers an enhanced BPF program defined by the metering master station under the event of a corresponding type, such as a Kbarobe type enhanced BPF program, the enhanced BPF program written for the metering master station is allowed to serve as a Kbarobe processing program, and when a function N is executed, callback inserted by an early kernel module is not executed, but the enhanced BPF program dynamically inserted in a metering master station mode is executed. By utilizing a Kbarobe type enhanced BPF program, the related network functions of the Linux kernel of the metering master station can be detected in real time, and the network parameter information can be acquired in real time. In addition, events in the Linux kernel can be tracked, detected and even modified by calling the BPF function, the kernel does not need to be compiled again, and therefore the network congestion condition of the metering master station can be analyzed effectively and efficiently.

Drawings

Fig. 1 is a schematic flowchart of a measurement master station network detection method based on BPF according to an embodiment of the present invention;

FIG. 2 is a diagram illustrating the execution of a BPF function under consideration in accordance with an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1, an embodiment of the present invention provides a measurement master station network detection method based on BPF, including:

s10, loading the compiled BPF function into a Linux kernel of the metering master station, and establishing a key value storage area of the BPF function in the Linux kernel.

And S11, replacing the call-back of the data packet receiving and processing function with the BPF function to acquire the TCP connection information.

S12, replacing the call back of the quick retransmission function with the BPF function, and storing the congestion machine states before and after the quick retransmission function is executed in the key value storage area.

And S13, detecting the flag bit corresponding to the slow-start exit point by using the BPF function to obtain the exit condition of the mixed slow start.

One way to specify whether and when certain kernel functions are called under certain scenarios is to add a log print to the corresponding function in the kernel code, which typically requires recompiling the kernel. However, the birth of Kprobes makes it easier to probe. When the Linux kernel executes the specified probe function by inserting the kernel module, a callback function defined by the metering master station is called. The implementation principle of Kprobes in the Linux Arm architecture is shown in fig. 1. The code at the enable point is saved and replaced with a breakpoint instruction. When this breakpoint is executed, a trap instruction will be generated, the register saved, and then the associated detection handler will be skipped. The three methods of the Kprobe mechanism are Kprobe, Jprobe and Kret probe, respectively. Kprobe may be inserted into any instruction to be instrumented, the Kret probe may obtain the return value of the probe function, and Jprobe is used to obtain the input parameter value of the probe function.

The BPF function in this embodiment may be understood as implementing a set of enhanced BPF virtual machines in the Linux kernel to execute the enhanced BPF instructions inserted by the metering master. Through programming of the enhanced BPF program type, the BPF assembly code compiled by the LLVM compiler can be loaded into the Linux kernel of the metering master station. The Linux kernel triggers an enhanced BPF program defined by the metering master station, such as a Kprobe-type enhanced BPF program, under the corresponding type of event, and allows the enhanced BPF program written by the metering master station to be used as a Kprobe handler.

Referring to fig. 2, when detecting the Linux kernel function N of the metering master using Kprobe, when the function N is executed, the callback inserted by the early kernel module is not executed, but the enhanced BPF program dynamically inserted in the metering master mode is executed. By utilizing a Kbarobe type enhanced BPF program, the related network functions of the Linux kernel of the metering master station can be detected in real time, and the network parameter information can be acquired in real time.

Compared with the prior art, the metering master station network detection method and the metering master station based on the BPF provided by the embodiment of the invention realize a group of enhanced BPF virtual machines in a Linux kernel. The method comprises the steps that a compiled BPF function is loaded into a Linux kernel of a metering master station, after a key value storage area of the BPF function is established in the Linux kernel, the Linux kernel triggers an enhanced BPF program defined by the metering master station under the event of a corresponding type, such as a Kbarobe type enhanced BPF program, the enhanced BPF program written for the metering master station is allowed to serve as a Kbarobe processing program, and when a function N is executed, callback inserted by an early kernel module is not executed, but the enhanced BPF program dynamically inserted in a metering master station mode is executed. By utilizing a Kprebe type enhanced BPF program, the related network functions of the Linux kernel of the metering master station can be detected in real time, and the network parameter information can be acquired in real time. In addition, events in the Linux kernel can be tracked, detected and even modified by calling the BPF function, the kernel does not need to be compiled again, and therefore the network congestion condition of the metering master station can be analyzed effectively and efficiently.

Illustratively, the BPF function is BPF assembly code compiled by a LLVM compiler; and the variable value stored in the key value storage area of the BPF function can be called by other functions in the Linux kernel.

Exemplarily, the replacing the BPF function for the callback of the packet receiving processing function to obtain the TCP connection information includes:

The basic information about network congestion includes basic TCP connection information, the size of the current congestion window, the size of the current slow start threshold, and the amount of data sent to the buffer. In the Linux kernel (version: 4.15), a "tcp _ rcv _ interested" function (packet reception processing function) is used to process the connection state. Table 1 shows some important parameters of the function.

TABLE 1 important parameters of "TCP RCV ESTABLISHED

Parameter(s)	Purpose(s) to
		Structure sock (sk)	Including transport layer protocol related fields
The structure sk _ buff (skb)	Indicating the current data packet
		Structure tcphdr (th)	TCP header information

As can be seen from the parameters listed in table 1, the basic information of all required network connections can be obtained from the structure sock. According to the information, the embodiment acquires the network information in the TCP full connection state by using the BPF function. In the Linux kernel of the metering master station, congestion related information about the current TCP connection is stored in the fabric TCP _ sock. From the currently detected functions, the function parameters have no structure, so that the structure tcp _ sock can be obtained only from the existing structure, and the real-time congestion window, the slow start threshold and the sending queue buffer are obtained from the known sock. The data packets processed by the Linux kernel of the metering master station can be represented by skb, and the size of all skbs in the sending queue cache is sk _ wmem _ queued. And dynamically extracting the size of the sk _ wmem _ queue by adopting an eBPF technology. The fluctuation of which may indirectly reflect the current degree of network congestion.

In addition, the overall state of the current network can be detected through the dynamic change relationship between the congestion window and the slow start threshold value. In the cubic congestion control algorithm, the cubic window growth function is shown in equation 1, where t is the time elapsed since the last reduction of the congestion window and C is a known parameter of the equation. In the Linux kernel of the metering master, C is generally set to 0.4, the value of which is determined by statistical validation in a large number of environments, and K is the time for the function to increase W without further packet loss, and the calculation formula is shown in formula 2.

W(t)＝C(t-K) ³ +W _max (1)

The next step is to obtain basic information about the current TCP connection process. All information about the process can be obtained by obtaining the current process and the process control block of the current process. Once the function to be tested and the intermediate parameters for obtaining relevant information are determined, an enhanced BPF Map may be defined in the Kprobe type enhanced BPF function, which is a key value storage area residing in the kernel. Any BPF function that knows these maps can be accessed. Programs running in the metering master space may also access these maps using file descriptors. Any type of data can be stored in the Map as long as the data size is correctly specified in advance. When a Kbarobe-type BPF function is installed on a TCP _ rcv _ estableshed function, a Linux kernel of a metering master station receives a confirmation message from a receiving end and operates the function, an enhanced BPF function is triggered, and TCP basic connection information, process control information, a congestion window, a slow start threshold, a sending queue cache and the like are obtained through the function. These dynamically obtained variable values will be shared with other functions of the metering master kernel through Map.

Exemplarily, the replacing the callback of the fast retransmission function with the BPF function, and storing the states of the congestion state machines before and after the fast retransmission function is executed in the key value storage area specifically include:

After the TCP connection is established, the Linux congestion state machine of the metering master station determines whether to reduce the congestion window, maintain the congestion window and continuously increase the congestion window through each state, and data packet loss or overtime can be caused by improper processing. The Linux congestion state machine of the metering master comprises five states which are transmitted under specific conditions.

The five states of the congestion state machine change in real time in the Linux kernel of the metering master station. The congestion state machine jumps to handle various data packet losses and marks the current congestion state of the metering master. For example, after the TCP connection is successful, the network is in an open state, and the sender enters a CWR state when receiving an acknowledgement packet containing a congestion notification flag. When a timeout occurs, the sender will go from the "on" state to the "lost" state. The system enters the out-of-order state from the on state when only one suspected ACK, such as an out-of-order ACK, is received. When three duplicate acknowledgement packets are received in succession, the system will enter a recovery state. In any state, the RTO enters a loss state.

The embodiment calls the BPF function to detect the congestion state machine of the Linux kernel of the whole metering master station. When the state jumps, the old congestion state and the new congestion state are recorded. The congestion state machine is controlled by a tcp _ policies _ alert function in a Linux kernel of the metering master station, and the whole jump condition is judged by the function. Since the stateful switchover process needs to be detected, a combination of Kprobe and Kretprobe mechanisms is used to handle breakpoints. Before entering the congestion state machine processing function, the current congestion state is captured as the value of Map, and at this time, the struct Sock pointer is used as the key of the state. It can be understood that the BPF function inserts Kprobe and Kretprobe respectively at the inlet and outlet of the function, and the defined BPF function records the congestion state machine function before and after execution. If the same connection control structure is found before and after the function execution, the congestion status is dynamically extracted.

Illustratively, the detecting the flag bit corresponding to the slow-start exit point by using the BPF function to obtain the exit condition of the hybrid slow start specifically includes:

detecting a HYSTART _ ACK _ TRAIN flag bit by using the BPF function to obtain an ACK row-column measurement starting timestamp and minimum path sending delay;

if the difference value obtained by subtracting the ACK row and column measurement starting timestamp from the current time is larger than the row and column threshold value, the network condition is deteriorated, and the slow start exits; the row-column threshold value is related to the minimum path transmit delay.

detecting a HYSTART _ DELAY flag bit by using the BPF function to obtain a curr _ RTT value and a minimum path sending DELAY; if the difference value between the curr _ RTT value and the minimum path sending delay is larger than a delay threshold value, the delay is increased excessively, and the slow start is quitted; the value of the delay threshold is related to the minimum path transmission delay.

In order to solve the problem that a large number of data packets are lost due to rapid growth of slow start, the cubic algorithm provides a hybrid slow start, and can ensure that the slow start can be completed safely. The hybrid slow start adopts two modes to quit the hybrid slow start, and the first mode is an ACK row length measuring method. The length of the ACK train is the sum of the arrival intervals of immediately adjacent ACK packets within one RTT period. A series of ACK packets with a default interval less than or equal to 2ms is an ACK rank. For each RTT period, the ACK rank length is calculated and compared to the estimated minimum path transmission delay. If the current time minus the ACK line measurement starting timestamp round _ start is larger than half of the minimum RTT delay, the network condition is deteriorated, and the exit is initiated slowly. The second method is to measure the increase in packet delay. When the sampled curr _ RTT value is greater than 1/8, the delay is considered to increase too much and the slow start is exited.

In the Linux kernel of the metering master station, the cubic algorithm exits the slow start through two detection methods, namely 'safe'. For example, when a slow-start exit point is detected by the ACK rank length measurement method, the flag bit is set to HYSTART _ ACK _ TRAIN. Then, the enhanced BPF detects the flag bit, so that the congestion window, the threshold value and the data of the measured slow start exit point at the moment can be obtained, and the exit condition of the hybrid slow start at the moment can be visually seen.

When an exit point of slow start is detected by a measurement method that measures an increase in packet DELAY, the flag bit is set to HYSTART _ DELAY. Then, the enhanced BPF detects the flag bit, so that the congestion window, the threshold value and the data of the measured slow start exit point at the moment can be obtained, and the exit condition of the hybrid slow start at the moment can be visually seen.

Illustratively, the TCP connection information includes TCP basic connection information, process control information, a congestion window, a slow start threshold, and a transmit queue buffer.

Illustratively, the BPF function operates as a Kprobe handler.

An embodiment of the application provides a metering master station based on BPF, which comprises a linux kernel of a BPF function, a network information analysis module, a congestion state analysis module and a slow start analysis module.

And the network information analysis module is used for calling the BPF function to replace the callback of the data packet receiving and processing function and acquiring TCP connection information.

And the congestion state analysis module is used for calling the BPF function to replace the call-back of the quick retransmission function and storing the congestion machine states before and after the quick retransmission function is executed in the key value storage area.

And the slow start analysis module is used for calling the BPF function to detect the zone bit corresponding to the slow start exit point so as to obtain the exit condition of the mixed slow start.

Compared with the prior art, the metering master station based on the BPF provided by the embodiment of the invention realizes a group of enhanced BPF virtual machines in a Linux kernel. The method comprises the steps that a compiled BPF function is loaded into a Linux kernel of a metering master station, after a key value storage area of the BPF function is established in the Linux kernel, the Linux kernel triggers an enhanced BPF program defined by the metering master station under the event of a corresponding type, such as a Kbarobe type enhanced BPF program, the enhanced BPF program written for the metering master station is allowed to serve as a Kbarobe processing program, and when a function N is executed, callback inserted by an early kernel module is not executed, but the enhanced BPF program dynamically inserted in a metering master station mode is executed. By utilizing a Kbarobe type enhanced BPF program, the related network functions of the Linux kernel of the metering master station can be detected in real time, and the network parameter information can be acquired in real time. In addition, events in the Linux kernel can be tracked, detected and even modified by calling the BPF function, the kernel does not need to be compiled again, and therefore the network congestion condition of the metering master station can be analyzed effectively and efficiently.

It will be clear to those skilled in the art that for the sake of convenience and brevity of description, the specific working process of the metering master station described above may refer to the corresponding process in the foregoing method embodiment, which is not to be appreciated herein.

An embodiment of the application provides a server and a client based on BPF. And the server and the client are configured for the experimental environment to carry out data communication. The operating systems of these two hosts are Ubuntu and the kernel version is linux 4.15. In addition, a simulation tool is also needed to construct a network congestion environment. Two hosts are provided in a local area network, one as a server and the other as a client. The Iperf3 tool is used for communication between clients and servers, and the TC tool is used for network congestion simulation. The experiments are divided into three groups, which respectively correspond to network congestion basic information measurement, congestion state machine measurement and hybrid slow start egress point measurement. Each set of corresponding enhanced BPF programs is written as program 1, program 2, and program 3, respectively. The specific experimental procedure is as follows:

step 1: when the network is normal, respectively operating a program 1, a program 2 and a program 3;

step 2: the client sends 20971520 bytes (20M) of data to the server;

and step 3: closing the program 1, the program 2 and the program 3, and obtaining a measurement result;

step 4, setting a congestion environment on the client: TCqdisc add Devens33 root network loss 20% 30%;

and 5: in a crowded network environment, program 1, program 2, and program 3 are run;

step 6: the client sends 20971520 bytes (20M) of data to the server;

and 7: the procedure 1, the procedure 2, and the procedure 3 were closed, and the measurement results were obtained.

In the experiment, the basic network information obtained by the program 1 is compared with the highest command of the Linux of the metering master station, and whether the process information obtained by the current program is consistent with the basic network information obtained by the tcpdump is verified. The congestion window change can be obtained from procedure 1. By enhancing the BPF program, the obvious packet loss overtime phenomenon existing in the change of the congestion window under the congestion environment can be obtained. When the network is congested, the fluctuation of the length of the transmission buffer queue is much larger than that in the normal state, which shows that the enhanced BPF program can indirectly reflect the degree of the network congestion through the fluctuation of the length of the transmission buffer queue.

Through data statistics, the program 2 obtains the dynamic conversion of the congestion state machine and the calculated values of the normal network state and each state under the network congestion state. In normal network state, all states switch back and forth directly between TCP _ CA _ Di sor and TCP _ CA _ Open. When the states of TCP _ CA _ Recovery and TCP _ CA _ Loss enter the state of TCP _ CA _ Loss multiple times, it indicates that packet Loss and timeout frequently occur on the current network. In addition, it is also verified whether the enhanced BPF procedure can effectively detect network congestion.

Program 3 dynamically obtains the current hybrid slow start and exit times. Equation 3 is a detection equation, and the BPF function called to detect data also conforms to the equation.

While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.

Claims

1. A measurement master station network detection method based on BPF is characterized by comprising the following steps:

replacing the call-back of the quick retransmission function with the BPF function, and storing congestion machine states before and after the quick retransmission function is executed in the key value storage area;

2. The BPF-based metering master station network detection method of claim 1, wherein the BPF function is BPF assembly code compiled by an LLVM compiler; and the variable values stored in the key value storage area of the BPF function can be called by other functions in the Linux kernel.

3. The BPF-based metering master station network detection method according to claim 1, wherein the step of replacing a callback of a packet reception processing function with the BPF function to obtain TCP connection information includes:

and extracting a transmission queue statistical value of the socket from the TCP structure body, and judging the network congestion degree according to the change of the transmission queue statistical value.

4. The BPF-based network detection method for the metering master station according to claim 1, wherein the step of replacing the callback of the fast retransmission function with the BPF function and storing the states of the congestion state machines before and after the fast retransmission function is executed in the key value storage area comprises:

5. The BPF-based metering master station network detection method according to claim 1, wherein the detecting a flag bit corresponding to a slow-start exit point by using the BPF function to obtain a mixed slow-start exit condition specifically includes:

6. The BPF-based metering master station network detection method according to claim 1, wherein the detecting a flag bit corresponding to a slow-start exit point by using the BPF function to obtain a mixed slow-start exit condition specifically includes:

7. The BPF-based metering master station network detection method of claim 1, wherein the TCP connection information includes TCP basic connection information, process control information, congestion window, slow start threshold, and transmit queue buffer.

8. The BPF-based metrology master station network inspection method of claim 1, wherein the BPF function operates as a Kprobe handler.

9. A metering master station based on BPF is characterized by comprising a l inux kernel of a BPF function, a network information analysis module, a congestion state analysis module and a slow start analysis module;