CN114745307B - Container flow monitoring method and bpf controller - Google Patents
Container flow monitoring method and bpf controller Download PDFInfo
- Publication number
- CN114745307B CN114745307B CN202210179635.7A CN202210179635A CN114745307B CN 114745307 B CN114745307 B CN 114745307B CN 202210179635 A CN202210179635 A CN 202210179635A CN 114745307 B CN114745307 B CN 114745307B
- Authority
- CN
- China
- Prior art keywords
- ebpf
- host
- flow
- network card
- mapping
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/142—Network analysis or design using statistical or mathematical methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
- H04L43/0888—Throughput
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- General Physics & Mathematics (AREA)
- Algebra (AREA)
- Physics & Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Pure & Applied Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The application discloses a method for monitoring container flow and a bpf controller, wherein the method comprises the following steps: reading a configuration file of a target host, and writing configuration information in the configuration file into an ebpf mapping table of a kernel, wherein the configuration information at least comprises an internal network segment and an external network segment to be monitored; starting a monitoring event, identifying a host mapping network card without the ebpf program injected into the target host through the monitoring event, and injecting the ebpf program for monitoring the flow for the host mapping network card; the host mapping network card and the container mapping network card have a corresponding relation; and monitoring the internal network flow and the external network flow corresponding to the host mapping network card according to the internal network segment to be monitored and the external network segment by the injected ebpf program, and recording the internal network flow and the external network flow into the ebpf mapping table. The technical scheme provided by the application ensures the stability of the whole system in the process of realizing the flow statistics of the internal and external networks.
Description
Technical Field
The application relates to the technical field of Internet, in particular to a method for monitoring container flow and a bpf controller.
Background
With the gradual maturation of cloud platforms such as IaaS (Infrastructure as a Service ), paaS (Platform as a Service, platform as a service), saaS (Software as a Service ), and the like, container virtualization technology has also been widely used.
At present, in order to distinguish the internal and external network flows of a container, a plurality of network cards are usually set in the container, then the internal network flows and the external network flows are respectively directed to different network cards in a routing mode, and then the internal and external network flows can be counted through an ifconfig command.
However, the existing flow statistics method for creating multiple network cards in a container needs to invasively modify the CNI (Container Network Interface ), which is likely to cause stability problems at the CNI level.
Disclosure of Invention
The application aims to provide a container flow monitoring method and a bpf controller, which ensure the stability of the whole system in the process of realizing the flow statistics of an internal network and an external network.
To achieve the above object, in one aspect, the present application provides a method for monitoring a container flow, where the method is applied to a bpf controller, and the method includes: reading a configuration file of a target host, and writing configuration information in the configuration file into an ebpf mapping table of a kernel, wherein the configuration information at least comprises an internal network segment and an external network segment to be monitored; starting a monitoring event, identifying a host mapping network card without the ebpf program injected into the target host through the monitoring event, and injecting the ebpf program for monitoring the flow for the host mapping network card; the host mapping network card and the container mapping network card have a corresponding relation; and monitoring the internal network flow and the external network flow corresponding to the host mapping network card according to the internal network segment to be monitored and the external network segment by the injected ebpf program, and recording the internal network flow and the external network flow into the ebpf mapping table.
In one embodiment, identifying, by the snoop event, a host mapping network card in the target host into which the ebpf program is not injected includes: and when the monitoring event is started, acquiring all host mapping network cards of the target host, and identifying the host mapping network cards without the ebpf program in all host mapping network cards.
In one embodiment, the method further comprises: if the monitoring event monitors the newly-built host mapping network card, injecting an ebpf program for monitoring the flow for the newly-built host mapping network card.
In one embodiment, the ebpf program for monitoring traffic includes an ingress traffic monitoring program and an egress traffic monitoring program; the monitoring of the intranet flow and the extranet flow corresponding to the host mapping network card comprises the following steps: respectively counting the inflow corresponding to the inner network segment to be monitored and the inflow corresponding to the outer network segment to be monitored under the host mapping network card through the inflow monitoring program; and respectively counting the output flow corresponding to the internal network segment to be monitored and the output flow corresponding to the external network segment to be monitored under the host mapping network card through the output flow monitoring program.
In one embodiment, recording the intranet traffic into the ebpf mapping table includes: and identifying the network card identifier of the host mapping network card, and storing the input flow and the output flow under the external network segment to be monitored and the network card identifier in the ebpf mapping table in an associated mode.
In one embodiment, recording the extranet traffic into the ebpf map includes: and identifying the network card identifier of the host mapping network card, and storing the input flow and the output flow under the external network segment to be monitored and the network card identifier in the ebpf mapping table in an associated mode.
In one embodiment, after recording the intranet traffic and the extranet traffic into the ebpf mapping table, the method further comprises: mapping the ebpf mapping table to the target host in a virtual file mode, so that other programs acquire data in the ebpf mapping table from the target host in a file reading mode.
In one embodiment, the method further comprises: if the configuration information in the configuration file is updated, acquiring updated configuration information, and writing the updated configuration information into the ebpf mapping table to replace the original configuration information in the ebpf mapping table; and monitoring the internal network flow and the external network flow corresponding to the host mapping network card according to the updated configuration information through the injected ebpf program.
To achieve the above object, another aspect of the present application provides a bpf controller, including: the configuration information processing unit is used for reading a configuration file of the target host and writing configuration information in the configuration file into an ebpf mapping table of the kernel, wherein the configuration information at least comprises an internal network segment and an external network segment to be monitored; the program injection unit is used for starting a monitoring event, identifying a host mapping network card without the ebpf program injected into the target host through the monitoring event, and injecting the ebpf program for monitoring the flow for the host mapping network card; the host mapping network card and the container mapping network card have a corresponding relation; and the flow statistics unit is used for monitoring the intranet flow and the extranet flow corresponding to the host mapping network card according to the intranet network segment and the extranet network segment to be monitored through the injected ebpf program, and recording the intranet flow and the extranet flow into the ebpf mapping table.
In one embodiment, the configuration information processing unit is further configured to: if the configuration information in the configuration file is updated, acquiring updated configuration information, and writing the updated configuration information into the ebpf mapping table to replace the original configuration information in the ebpf mapping table; the program injection unit is further configured to: and monitoring the internal network flow and the external network flow corresponding to the host mapping network card according to the updated configuration information through the injected ebpf program.
As can be seen from the above, according to the technical solutions provided in one or more embodiments of the present application, statistics of the network traffic inside and outside the container may be performed by mounting an ebpf (Extended Berkeley Packet Filter, extended bpf) program on the host mapping network card by the bpf (Berkeley Packet Filter ) controller.
Specifically, the host mapping network card and the container mapping network card have a corresponding relationship, and the incoming flow of the host mapping network card corresponds to the outgoing flow of the container mapping network card. In contrast, the outgoing flow of the host mapping network card corresponds to the incoming flow of the container mapping network card, so that the outgoing flow and the incoming flow under the host mapping network card can be counted by mounting the ebpf program for the user-state host mapping network card, and the incoming flow and the outgoing flow under the container mapping network card can be counted correspondingly.
Due to the technical scheme provided by the application, the flow statistics can be completed through the ebpf program running in the kernel mode without depending on the configuration command (for example, the ifconfig command) of the network equipment, so that the CNI is not required to be modified, and the stability of the whole system is ensured. In addition, the statistical data in the kernel mode does not need to be copied to have the user mode, so that the consumption of CPU and memory resources is reduced, and the flow statistical efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a system for monitoring the flow rate of a container in an embodiment of the application;
FIG. 2 is a schematic diagram of a method for monitoring the flow rate of a container in an embodiment of the application;
FIG. 3 is a functional block diagram of a bpf controller in an embodiment of the application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below in conjunction with the detailed description of the present application and the corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, are intended to fall within the scope of the present application.
The method for monitoring the flow of the container provided by the application can be applied to the system shown in fig. 1. The system can be divided into a user mode and a kernel mode. Wherein the bpf controller, the host, the container within the host, and the data collector (metrics collector) may all be in a user state. The monitor event (uevent) started by the bpf controller, the ebpf program installed by the bpf controller and the ebpf mapping table can all run in the kernel mode.
Referring to fig. 1 and 2, a method for monitoring a container flow rate applied to a bpf controller according to an embodiment of the present application may include the following steps.
S1: and reading a configuration file of the target host, and writing configuration information in the configuration file into an ebpf mapping table of the kernel, wherein the configuration information at least comprises an internal network segment and an external network segment to be monitored.
The BPF is a set of instructions that may be used for network packet trace and system call trace. Before the BPF occurs, if the desired data packet is to be filtered from the network card, the data packet received by the network card needs to be copied from the kernel space to the user space, and then the user program filters the data packet in the user space. This approach has a problem in that the packets must be fully replicated to filter out the required packets, and for those packets that are not required, the replication operation is inefficient, wasteful, and replication of the memory data can take up more system resources. The BPF may filter packets directly in the kernel, avoiding some useless copy operations that occupy system resources.
The early bpf function is single, can only act on the filtering of network data packets and the call tracking of other systems, and the expanded bpf (ebpf) function is greatly enriched and can be basically used in each subsystem of Linux. In addition to the functional extensions, the instruction set of the ebpf program also becomes quite complex, so that clang/llvm compilation dedicated to compiling the ebpf program occurs. The ebpf also changes over the framework so that ebpf is no longer comparable to earlier bpfs. Thus, the early bpf is called cbpf, and the expanded bpf is called ebpf.
Ebpf and dock are emerging at almost similar times, and the Ebpf bypasses the netfilter module based on hooks (hooks) provided by the linux kernel in the network aspect, so that the processing speed of the network is increased. The container network is a virtual network built based on iptables in the early stage, and the method can obviously reduce the performance after the iptables rule reaches a certain amount. And ebpf can bypass the iptables module to independently construct a container network, the appearance of ebpf solves the performance problem of the container network.
Based on the method, the ebpf program is introduced into the monitoring method of the container flow, so that the flow statistics process can be realized in the kernel state without changing the CNI.
Specifically, real-time interaction between the user state and the kernel state can be realized by using the ebpf mapping table. In practical applications, the ebpf mapping table may include four different types of forms, where each form may store corresponding data. The four forms are respectively a form for storing an intranet network segment, a form for storing debugging parameters, a form for storing intranet flow and a form for storing extranet flow.
Wherein, deposit the form of intranet network segment: the bpf controller (bpf manager) will load the configuration file of the program when it starts, and there will be intranet ip segment list information in the configuration file, and these information will be converted into ip range information of int digital type, such as ipStart (ip start value) and ipEnd (ip end value). These data are stored in the ebpf map, and configuration information can be read from the ebpf map by only storing the data in the ebpf map and referencing the ebpf map by the ebpf program.
Form for storing debugging parameters: the method can control whether the bpf log is output or not, and the detailed information such as the information content ip information, the network card id information and the packet size, which are judged to be the internal network type traffic or the external network traffic, can be checked through commands such as cat/sys/kernel/debug/trace/trace_pipe.
Form for storing intranet flow: the ebpf program stores the number, the size statistical information and the like of the data packets received and transmitted by the intranet flow in the form.
Form for storing external network flow: the ebpf program stores the number, size statistics, etc. of the data packets transmitted and received by the external network traffic in the form.
In this embodiment, after the bpf controller is started, the configuration file of the target host currently monitored can be read, configuration information such as debug parameters, internal and external network segments and the like can be parsed from the configuration file, and the configuration information is written into a corresponding table of the ebpf mapping table.
In practical application, by modifying the content in the ebpf mapping table, the statistics behavior of the kernel and the debugging of the kernel of the real-time switch can be changed in real time, and when the counted flow is incorrect, the checking can be performed without restarting the process.
Specifically, by sending a specific signal to the bpf controller, after the bpf controller receives the signal, if it is determined that the signal is a signal to open the debug log, the debug switch may be set to on (by configuring the key debug value in the ebpf map to be 1), thereby opening the debug log of the ebpf program. Meanwhile, the tcpdump tool can grasp the packet, count the flow and then compare the flow with information output by the bpf program log, so that whether the counted flow is correct or not is judged.
S3: starting a monitoring event, identifying a host mapping network card without the ebpf program injected into the target host through the monitoring event, and injecting the ebpf program for monitoring the flow for the host mapping network card; the host mapping network card and the container mapping network card have a corresponding relation.
In this embodiment, when the bpf controller is started, a linux uevent monitoring event may be started, and the monitoring event may filter out network events related to the network card in the system, so as to manage the creation and deletion behaviors of the network card.
Specifically, after the monitoring event is started, all host mapping network cards of the target host can be obtained first, and host mapping network cards without the ebpf program injected are identified in all host mapping network cards. For this identified portion of the host map network card, an ebpf program for monitoring traffic may be injected.
In addition, the monitoring event can monitor the newly-added host mapping network card, and after the newly-built host mapping network card is monitored, an ebpf program for monitoring the flow can be injected into the newly-built host mapping network card.
In practice, the ebpf program may include an ingress flow monitor program and an egress flow monitor program in order to distinguish between ingress flow into the container and egress flow from the container. It should be noted that, because the ebpf program is mounted on the host mapping network card, and counts the traffic of the host mapping network card, the traffic of the host mapping network card is actually the incoming traffic of the container, and the incoming traffic of the host mapping network card is actually the outgoing traffic of the container, the incoming traffic monitoring program can monitor the outgoing traffic of the container in a phase-changing manner, and similarly, the outgoing traffic monitoring program can monitor the incoming traffic of the container in a phase-changing manner.
In a specific application scenario, the ingress Traffic monitoring program may be a program written in xdp (eXpress Data Path, fast data channel) code, and the egress Traffic monitoring program may be implemented by an egress filter of tc (Traffic Control) in linux. Thus, the outgoing flow and the incoming flow of the host mapping network card can be monitored respectively by injecting the outgoing flow monitoring program and the incoming flow monitoring program into the host mapping network card respectively.
It should be noted that, the containers may be generally classified into a sandbox (sadbox) container and a service container, where the sandbox container is started before the service container is started, and a network of the service container may be constructed through the sandbox container, that is, the process of creating the host mapping network card is implemented by the sandbox container, and the host mapping network card is already created before the service container is started. In the application, the traffic container is usually needed to monitor the traffic, so that the network card event can be monitored at the first time after the uevent monitoring event is started, and the injection process of the ebpf program on the host mapping network card can be completed before the traffic container is started, thereby avoiding the problem of traffic statistics loss of the traffic container.
S5: and monitoring the internal network flow and the external network flow corresponding to the host mapping network card according to the internal network segment to be monitored and the external network segment by the injected ebpf program, and recording the internal network flow and the external network flow into the ebpf mapping table.
In this embodiment, through the incoming flow monitoring program, the incoming flow corresponding to the internal network segment to be monitored and the incoming flow corresponding to the external network segment to be monitored under the host mapping network card can be counted respectively, and through the outgoing flow monitoring program, the outgoing flow corresponding to the internal network segment to be monitored and the outgoing flow corresponding to the external network segment to be monitored under the host mapping network card can be counted respectively.
In this embodiment, the ingress and egress flows under the internal and external network segments counted by the ebpf procedure may be recorded in the ebpf mapping table. Specifically, when the ebpf program in the kernel mode counts traffic, only the network card identifier of the host mapping network card is concerned, and the container information is not concerned. In view of this, when the traffic is recorded in the ebpf mapping table, the network card identifier of the host mapping network card can be identified, and the incoming traffic and outgoing traffic under the intranet segment to be monitored and the network card identifier are associated and stored in the ebpf mapping table. Similarly, the ingress and egress flows under the external network segment to be monitored can be stored in the ebpf mapping table in association with the network card identifier. However, traffic under the inner network segment and traffic under the outer network segment are typically recorded by different forms. In a specific application scenario, the flow data of the intranet segment recorded in the ebpf mapping table may include the following:
network card identifier of interface_id host mapping network card
Total number of bytes of incoming traffic in internal network segment of internal_rx_bytes
Total number of packets for ingress traffic in internal network segment of internal_rx_packets
Total number of bytes of outgoing traffic in internal network segment of internal_tx_bytes
Total number of packets for outgoing traffic in internal network segment of internal_tx_packets
The traffic of the external network segment recorded in the ebpf mapping table may be as follows:
network card identifier of interface_id host mapping network card
Total number of bytes of incoming traffic under external_rx_bytes external network segment
Total number of packets for ingress traffic under external network segment of external_rx_packets
Total number of bytes of outgoing traffic under external_tx_bytes external network segment
Total number of packets of outgoing traffic under the external network segment of the external_tx_packets
In one embodiment, after the statistical traffic is recorded through the ebpf mapping table, the bpf controller may map the ebpf mapping table to the target host in the manner of a virtual file of the pinned file, so that other programs may directly obtain the data in the ebpf mapping table in the manner of reading the file. Wherein the data in the ebpf mapping table is usually in binary format, and other programs need to be converted into structures corresponding to other programs after reading.
Specifically, in fig. 1, the data collector may use an algorithm in the CNI to calculate a correspondence between the host mapping network card and the container, and may read the data of the pinned file according to a certain period (for example, per minute), and establish a relationship between the read data and the identifier of the container, and report the relationship to a storage system of kubernetes. Thus, the flow information originally associated with the network card identifier can be associated with the corresponding container identifier, so that the inlet and outlet flow of the container can be directly represented.
In one embodiment, if the configuration information in the configuration file is updated during operation of the bpf controller, the bpf controller may acquire the updated configuration information and write the updated configuration information into the ebpf mapping table to replace the original configuration information in the ebpf mapping table. For example, the internal network segment and the external network segment in the configuration information can be updated in real time, so that the internal network traffic and the external network traffic corresponding to the host mapping network card can be monitored according to the updated configuration information through the injected ebpf program.
In one embodiment, the internal and external network segments in the configuration information can be flexibly configured. For example, the external network ip segment to which the machine room belongs may be classified as internal traffic, because such external network ip only interacts inside the machine room, without going through an external network. That is, in the present application, the traffic of the intranet segment is not limited to the common intranet segment (e.g. 10.0.0.0), but may also include a portion of the traffic under the extranet segment.
In the application, the statistical logic of the traffic of the internal and external networks is directly realized by using the ebpf code, the statistical logic is operated in the kernel mode and does not depend on the statistical data of the ifconfig, and the statistical logic can be flexibly customized according to the needs by utilizing the kernel reprogramming capability of the ebpf. And the statistical data based on the kernel mode does not need to be copied to the user mode, so that the CPU resource and the memory resource consumption are reduced. The method is transparent to the CNI, does not need to modify the CNI layer, does not invade the container platform, and avoids the stability problem of the CNI layer.
The ebpf mapping table is used as a bridge for real-time interaction between the user state program and the kernel state program, so that the real-time change of the kernel statistical behavior and the real-time on-off kernel debugging are supported, and when the flow is incorrect, the investigation can be rapidly performed without restarting the process.
The uevent event monitoring mechanism based on the linux kernel, rather than events of the container, is more reliable than monitoring container runtime (container running time) events, is decoupled from container runtime, can monitor creation and deletion events of a container network card in time, ensures instantaneity, and avoids traffic statistics loss of a service container process.
At present, the container network CNI is mainly a container network constructed based on a veth pair, and under the container network, if the flow of the container is to be monitored, only the flow of a host mapping network card in a host space is monitored, so that the flow statistics logic in a kernel mode can not concern the container id information and only the unique id of the network card, and the design ensures that a statistics program does not need to concern whether a container engine uses a dock or a container, and has better universality.
In the application, the bpf controller can read the data of the bpf mapping table according to minutes at the same time, and store the read data into the disk file in the user mode, wherein the disk file can comprise two types of files:
(1) Each host maps the total flow data of one minute on the network card for the use of troubleshooting;
(2) The total flow data of each minute in a single day of each host mapping network card is mainly used for compensating the data acquisition missing in an abnormal time period when the data acquisition device is abnormal.
Referring to fig. 3, the present application further provides a bpf controller, including:
the configuration information processing unit is used for reading a configuration file of the target host and writing configuration information in the configuration file into an ebpf mapping table of the kernel, wherein the configuration information at least comprises an internal network segment and an external network segment to be monitored;
the program injection unit is used for starting a monitoring event, identifying a host mapping network card without the ebpf program injected into the target host through the monitoring event, and injecting the ebpf program for monitoring the flow for the host mapping network card; the host mapping network card and the container mapping network card have a corresponding relation;
and the flow statistics unit is used for monitoring the intranet flow and the extranet flow corresponding to the host mapping network card according to the intranet network segment and the extranet network segment to be monitored through the injected ebpf program, and recording the intranet flow and the extranet flow into the ebpf mapping table.
In one embodiment, the configuration information processing unit is further configured to: if the configuration information in the configuration file is updated, acquiring updated configuration information, and writing the updated configuration information into the ebpf mapping table to replace the original configuration information in the ebpf mapping table;
the program injection unit is further configured to: and monitoring the internal network flow and the external network flow corresponding to the host mapping network card according to the updated configuration information through the injected ebpf program.
As can be seen from the above, according to the technical solutions provided in one or more embodiments of the present application, statistics of the network traffic inside and outside the container may be performed by mounting an ebpf (Extended Berkeley Packet Filter, extended bpf) program on the host mapping network card by the bpf (Berkeley Packet Filter ) controller.
Specifically, the host mapping network card and the container mapping network card have a corresponding relationship, and the incoming flow of the host mapping network card corresponds to the outgoing flow of the container mapping network card. In contrast, the outgoing flow of the host mapping network card corresponds to the incoming flow of the container mapping network card, so that the outgoing flow and the incoming flow under the host mapping network card can be counted by mounting the ebpf program for the user-state host mapping network card, and the incoming flow and the outgoing flow under the container mapping network card can be counted correspondingly.
Due to the technical scheme provided by the application, the flow statistics can be completed through the ebpf program running in the kernel mode without relying on the ifconfig command, so that the CNI is not required to be modified, and the stability of the whole system is ensured. In addition, the statistical data in the kernel mode does not need to be copied to have the user mode, so that the consumption of CPU and memory resources is reduced, and the flow statistical efficiency is improved.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are referred to each other, and each embodiment is mainly described as different from other embodiments. In particular, reference may be made to the description of embodiments of the method described above for an embodiment of the bpf controller.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium which can be used to store information that can be accessed by the computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The foregoing description is only illustrative of the application and is not to be construed as limiting the application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.
Claims (10)
1. The method is characterized in that the method is applied to a bpf controller, the bpf controller, a host and a container in the host are positioned in a user state, and a monitoring event started by the bpf controller, an ebpf program mounted by the bpf controller and an ebpf mapping table run in a kernel state; the method comprises the following steps:
reading a configuration file of a target host, and writing configuration information in the configuration file into an ebpf mapping table of a kernel, wherein the configuration information at least comprises an internal network segment and an external network segment to be monitored;
starting a monitoring event, identifying a host mapping network card without the ebpf program injected into the target host through the monitoring event, and injecting the ebpf program for monitoring the flow for the host mapping network card; the host mapping network card and the container mapping network card have a corresponding relation;
and monitoring the intranet flow and the extranet flow corresponding to the host mapping network card according to the intranet network segment to be monitored and the extranet network segment by the injected ebpf program, and recording the intranet flow and the extranet flow into the ebpf mapping table so as to realize real-time interaction between a user state and a kernel state by using the ebpf mapping table.
2. The method of claim 1, wherein identifying, by the snoop event, a host mapping network card in the target host into which no ebpf program is injected comprises:
and when the monitoring event is started, acquiring all host mapping network cards of the target host, and identifying the host mapping network cards without the ebpf program in all host mapping network cards.
3. The method according to claim 1 or 2, characterized in that the method further comprises:
if the monitoring event monitors the newly-built host mapping network card, injecting an ebpf program for monitoring the flow for the newly-built host mapping network card.
4. The method of claim 1, wherein the ebpf program for monitoring traffic comprises an ingress traffic monitoring program and an egress traffic monitoring program; the monitoring of the intranet flow and the extranet flow corresponding to the host mapping network card comprises the following steps:
respectively counting the inflow corresponding to the inner network segment to be monitored and the inflow corresponding to the outer network segment to be monitored under the host mapping network card through the inflow monitoring program;
and respectively counting the output flow corresponding to the internal network segment to be monitored and the output flow corresponding to the external network segment to be monitored under the host mapping network card through the output flow monitoring program.
5. The method of claim 1 or 4, wherein recording the intranet traffic into the ebpf mapping table comprises:
and identifying the network card identifier of the host mapping network card, and storing the input flow and the output flow under the intranet segment to be monitored and the network card identifier in the ebpf mapping table in an associated mode.
6. The method of claim 1 or 4, wherein recording the extranet traffic into the ebpf map comprises:
and identifying the network card identifier of the host mapping network card, and storing the input flow and the output flow under the external network segment to be monitored and the network card identifier in the ebpf mapping table in an associated mode.
7. The method of claim 1, wherein after recording the intranet traffic and the extranet traffic into the ebpf map, the method further comprises:
mapping the ebpf mapping table to the target host in a virtual file mode, so that other programs acquire data in the ebpf mapping table from the target host in a file reading mode.
8. The method according to claim 1, wherein the method further comprises:
if the configuration information in the configuration file is updated, acquiring updated configuration information, and writing the updated configuration information into the ebpf mapping table to replace the original configuration information in the ebpf mapping table;
and monitoring the internal network flow and the external network flow corresponding to the host mapping network card according to the updated configuration information through the injected ebpf program.
9. The bpf controller is characterized in that the bpf controller, a host and a container in the host are positioned in a user state, and a monitoring event started by the bpf controller, an ebpf program installed on the bpf controller and an ebpf mapping table run in a kernel state; the bpf controller includes:
the configuration information processing unit is used for reading a configuration file of the target host and writing configuration information in the configuration file into an ebpf mapping table of the kernel, wherein the configuration information at least comprises an internal network segment and an external network segment to be monitored;
the program injection unit is used for starting a monitoring event, identifying a host mapping network card without the ebpf program injected into the target host through the monitoring event, and injecting the ebpf program for monitoring the flow for the host mapping network card; the host mapping network card and the container mapping network card have a corresponding relation;
and the flow statistics unit is used for monitoring the intranet flow and the extranet flow corresponding to the host mapping network card according to the intranet network segment and the extranet network segment to be monitored through the injected ebpf program, and recording the intranet flow and the extranet flow into the ebpf mapping table so as to realize real-time interaction between a user state and a kernel state by using the ebpf mapping table.
10. The bpf controller of claim 9, wherein the configuration information processing unit is further configured to: if the configuration information in the configuration file is updated, acquiring updated configuration information, and writing the updated configuration information into the ebpf mapping table to replace the original configuration information in the ebpf mapping table;
the program injection unit is further configured to: and monitoring the internal network flow and the external network flow corresponding to the host mapping network card according to the updated configuration information through the injected ebpf program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210179635.7A CN114745307B (en) | 2022-02-25 | 2022-02-25 | Container flow monitoring method and bpf controller |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210179635.7A CN114745307B (en) | 2022-02-25 | 2022-02-25 | Container flow monitoring method and bpf controller |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114745307A CN114745307A (en) | 2022-07-12 |
| CN114745307B true CN114745307B (en) | 2023-09-22 |
Family
ID=82274908
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210179635.7A Active CN114745307B (en) | 2022-02-25 | 2022-02-25 | Container flow monitoring method and bpf controller |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114745307B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112199668A (en) * | 2020-09-01 | 2021-01-08 | 中国科学院信息工程研究所 | Method and device for detecting DoS attack of CPU consumed by application layer in container |
| CN112564967A (en) * | 2020-12-02 | 2021-03-26 | 杭州谐云科技有限公司 | Cloud service topology self-discovery method and system based on eBPF, electronic device and storage medium |
| CN112860484A (en) * | 2021-01-29 | 2021-05-28 | 深信服科技股份有限公司 | Container runtime abnormal behavior detection and model training method and related device |
| CN113472817A (en) * | 2021-09-03 | 2021-10-01 | 杭州网银互联科技股份有限公司 | Gateway access method and device for large-scale IPSec and electronic equipment |
| CN113568711A (en) * | 2021-08-04 | 2021-10-29 | 上海仪电（集团）有限公司中央研究院 | Load balancing method and device based on eBPF framework and used in K8S |
| CN113992428A (en) * | 2021-11-29 | 2022-01-28 | 北京天融信网络安全技术有限公司 | Intrusion prevention method and device under container environment, electronic equipment and storage medium |
| CN113986459A (en) * | 2021-10-21 | 2022-01-28 | 浪潮电子信息产业股份有限公司 | Control method and system for container access, electronic equipment and storage medium |
| CN114006839A (en) * | 2021-09-27 | 2022-02-01 | 中盈优创资讯科技有限公司 | eBPF-based traffic collection method and device |
| CN114039789A (en) * | 2021-11-17 | 2022-02-11 | 北京天融信网络安全技术有限公司 | Flow protection method, electronic device and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10762193B2 (en) * | 2018-05-09 | 2020-09-01 | International Business Machines Corporation | Dynamically generating and injecting trusted root certificates |
| US11516138B2 (en) * | 2020-04-27 | 2022-11-29 | International Business Machines Corporation | Determining network flow direction |
-
2022
- 2022-02-25 CN CN202210179635.7A patent/CN114745307B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112199668A (en) * | 2020-09-01 | 2021-01-08 | 中国科学院信息工程研究所 | Method and device for detecting DoS attack of CPU consumed by application layer in container |
| CN112564967A (en) * | 2020-12-02 | 2021-03-26 | 杭州谐云科技有限公司 | Cloud service topology self-discovery method and system based on eBPF, electronic device and storage medium |
| CN112860484A (en) * | 2021-01-29 | 2021-05-28 | 深信服科技股份有限公司 | Container runtime abnormal behavior detection and model training method and related device |
| CN113568711A (en) * | 2021-08-04 | 2021-10-29 | 上海仪电（集团）有限公司中央研究院 | Load balancing method and device based on eBPF framework and used in K8S |
| CN113472817A (en) * | 2021-09-03 | 2021-10-01 | 杭州网银互联科技股份有限公司 | Gateway access method and device for large-scale IPSec and electronic equipment |
| CN114006839A (en) * | 2021-09-27 | 2022-02-01 | 中盈优创资讯科技有限公司 | eBPF-based traffic collection method and device |
| CN113986459A (en) * | 2021-10-21 | 2022-01-28 | 浪潮电子信息产业股份有限公司 | Control method and system for container access, electronic equipment and storage medium |
| CN114039789A (en) * | 2021-11-17 | 2022-02-11 | 北京天融信网络安全技术有限公司 | Flow protection method, electronic device and storage medium |
| CN113992428A (en) * | 2021-11-29 | 2022-01-28 | 北京天融信网络安全技术有限公司 | Intrusion prevention method and device under container environment, electronic equipment and storage medium |
Non-Patent Citations (5)
| Title |
|---|
| Web服务中基于流量监控的DDoS攻击防范机制;王秀利;;计算机工程与应用(36);全文 * |
| 分布式网络流量监测;刘军良;肖宗水;;计算机工程(20);全文 * |
| 基于eBPF的容器网络可观测性方法与实践;刘畅;《中国优秀硕士学位论文全文数据库》;全文 * |
| 基于eBPF的网络安全应用研究;于波;《中国优秀硕士学位论文全文数据库》;全文 * |
| 局域网网络监控软件的研究;肖同松;;电脑编程技巧与维护(16);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114745307A (en) | 2022-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103023984B (en) | Terminal application server and application log filtering method thereof | |
| US11366908B2 (en) | Detecting unknown software vulnerabilities and system compromises | |
| CN107704360A (en) | Processing method, equipment, server and the storage medium of monitoring data | |
| CN112181764B (en) | Kubernetes resource data monitoring method and device | |
| CN105184166A (en) | Kernel-based Android application real-time behavior analysis method and system | |
| US20120297254A1 (en) | On demand tracing of application code execution | |
| CN104951395A (en) | Debugging information processing method and device for embedded system | |
| CN104572394A (en) | Process monitoring method and device | |
| CN106777126B (en) | Data online migration method supporting heterogeneous time sequence database | |
| US9946879B1 (en) | Establishing risk profiles for software packages | |
| CN107783880A (en) | A kind of log analysis method of server system, device and server system | |
| CN113656245A (en) | Data inspection method and device, storage medium and processor | |
| CN114385674A (en) | Platform message tracking method, system, device and storage medium | |
| CN111597099A (en) | Non-invasive simulation method for monitoring application running quality deployed on cloud platform | |
| CN114745307B (en) | Container flow monitoring method and bpf controller | |
| WO2022095844A1 (en) | Elastic block service mounting method and apparatus thereof | |
| CN113987401A (en) | Recording method and device of network general log, storage medium and processor | |
| CN117032903B (en) | Simulation debugging method and device, storage medium and electronic equipment | |
| CN111435327B (en) | Log record processing method, device and system | |
| CN110532150B (en) | Case management method and device, storage medium and processor | |
| CN112559565A (en) | Abnormity detection method, system and device | |
| US9965618B1 (en) | Reducing privileges for imported software packages | |
| CN104424234A (en) | Method and device for detecting change of file system and corresponding electronic device | |
| CN105100901B (en) | A kind of virtual memory card configuration method and system | |
| CN115080309A (en) | Data backup system, method, storage medium, and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |