CN115002186B - Network information acquisition method and device, electronic equipment and readable storage medium - Google Patents
Network information acquisition method and device, electronic equipment and readable storage mediumInfo
- Publication number
- CN115002186B CN115002186B CN202210535738.2A CN202210535738A CN115002186B CN 115002186 B CN115002186 B CN 115002186B CN 202210535738 A CN202210535738 A CN 202210535738A CN 115002186 B CN115002186 B CN 115002186B
- Authority
- CN
- China
- Prior art keywords
- information
- network
- kernel
- operating system
- network flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000012544 monitoring process Methods 0.000 claims abstract description 19
- 230000005540 biological transmission Effects 0.000 claims description 30
- 230000006870 function Effects 0.000 claims description 25
- 238000004891 communication Methods 0.000 claims description 16
- 230000004044 response Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 10
- 230000007246 mechanism Effects 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 7
- 238000013500 data storage Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 description 20
- 238000010586 diagram Methods 0.000 description 8
- 238000013459 approach Methods 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002085 persistent effect Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- KLDZYURQCUYZBL-UHFFFAOYSA-N 2-[3-[(2-hydroxyphenyl)methylideneamino]propyliminomethyl]phenol Chemical compound OC1=CC=CC=C1C=NCCCN=CC1=CC=CC=C1O KLDZYURQCUYZBL-UHFFFAOYSA-N 0.000 description 1
- 244000035744 Hura crepitans Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000012098 association analyses Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 201000001098 delayed sleep phase syndrome Diseases 0.000 description 1
- 208000033921 delayed sleep phase type circadian rhythm sleep disease Diseases 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005206 flow analysis Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000003012 network analysis Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Abstract
The application discloses a network information acquisition method, a device, an electronic device and a computer readable storage medium, wherein the method is applied to a proxy program which is deployed on acquired equipment and comprises the following steps: acquiring network flow information through an operating system kernel of the acquired equipment; generating network information respectively corresponding to a plurality of applications in the acquired equipment by utilizing the network flow information; the monitoring content of the operating system is comprehensive and accurate, so that the network flow information directly obtained from the kernel of the operating system by using the agent program has more effective information; meanwhile, the agent program does not need to execute operation with larger expenditure, does not need to actively collect and count information, and only needs to acquire the existing network flow information from the kernel of the operating system, so that the expenditure of computing resources is smaller.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a network information acquisition method, a network information acquisition device, an electronic device, and a computer readable storage medium.
Background
With the development of internet technology, more and more services are provided through a network, in order to ensure the reliability and stability of the network services, the network performance of an application level needs to be effectively detected in time, and before judging the network performance, the network information needed by the judgment needs to be acquired. At present, network information is generally obtained by monitoring network traffic of a device port, however, the network information obtained by the method is less, and the cost of computing resources is high.
Disclosure of Invention
Accordingly, the present application is directed to a network information acquisition method, a network information acquisition device, an electronic device, and a computer readable storage medium, which can obtain more effective information of network information and have less computing resource overhead.
In order to solve the above technical problems, the present application provides a network information acquisition method applied to an agent program, where the agent program is deployed on an acquired device, including:
obtaining network flow information through an operating system kernel of the acquired equipment;
and generating network information corresponding to the plurality of applications in the acquired equipment respectively by utilizing the network flow information.
Optionally, the obtaining, by the operating system kernel of the collected device, network flow information includes:
Acquiring a kernel version number of the kernel of the operating system;
If the kernel version number is higher than a preset version number, acquiring the network flow information by using a target mounting program;
and if the kernel version number is not higher than the preset version number, acquiring the network flow information by using socket connection with the kernel of the operating system.
Optionally, the collecting the network flow information by using the target mounting program includes:
reading a data storage object corresponding to the target mounting program to obtain the network flow information;
the target mounting program is mounted on a mounting point corresponding to the target kernel function, and the network flow information is started and collected after the mounting is successful.
Optionally, the obtaining the network flow information by using a socket connection with the operating system kernel includes:
Sending a network query request to the system kernel through the socket connection;
And receiving response data sent by the system kernel, and analyzing the response data to obtain the network flow information.
Optionally, the method further comprises:
And if the response data corresponding to the network query request is not received, accessing and reading a kernel network state directory corresponding to the kernel of the operating system to obtain the network flow information.
Optionally, the generating, by using the network flow information, network information corresponding to each of the plurality of applications in the collected device includes:
determining index node data corresponding to each network flow respectively by utilizing the network flow information;
Matching the index node data with each socket file to determine a target socket file and target index node data which are in one-to-one correspondence;
And determining corresponding target application information based on the target socket file, and generating the network information by utilizing the target application information and target network flow information corresponding to the target inode data.
Optionally, if the network flow information is transmission control protocol information, the obtaining, by the operating system kernel of the collected device, the network flow information includes:
calling an objective function library to obtain a connection table from the kernel of the operating system;
Obtaining a connection table item from the connection table;
and calling the target function library, and obtaining the transmission control protocol information from the operating system kernel by using the connection table entry.
Optionally, the obtaining the transmission control protocol information from the operating system kernel by using the connection table entry includes:
acquiring connection state information corresponding to each connection table item respectively;
And obtaining the transmission control protocol information from the kernel of the operating system by using the connection table item of which the connection state information is in an effective connection state.
Optionally, the generating, by using the network flow information, network information corresponding to each of the plurality of applications in the collected device includes:
Acquiring application identification information corresponding to each connection table item respectively;
Judging whether corresponding historical application information exists in the memory or not by utilizing each application identification information;
If so, generating the network information by utilizing the historical application information and the target transmission control protocol information corresponding to the connection table item;
Correspondingly, the method further comprises the steps of:
And determining the network information as new historical network information and storing the new historical network information into the memory.
Optionally, the generating, by using the network flow information, network information corresponding to each of the plurality of applications in the collected device includes:
Acquiring application identification information corresponding to each connection table item respectively;
Candidate application information is obtained, the candidate application information is screened by utilizing the application identification information, and target application information corresponding to each connection table item is determined;
And generating the network information by utilizing the target application information and the target transmission control protocol information corresponding to the connection table item.
Optionally, if the network flow information is user datagram protocol information, the obtaining, by the operating system kernel of the collected device, the network flow information includes:
monitoring a network event generated by the kernel of the operating system; the network event tracking mechanism of the operating system kernel is started;
and filtering the network event based on a network protocol to obtain a target network event, and obtaining the network flow information by using the target network event.
The application also provides a network information acquisition device, which is applied to an agent program, wherein the agent program is deployed on acquired equipment and comprises the following components:
The network flow information acquisition module is used for acquiring network flow information through the operating system kernel of the acquired equipment;
And the network information generating module is used for generating network information corresponding to a plurality of applications in the acquired equipment respectively by utilizing the network flow information.
The application also provides an electronic device comprising a memory and a processor, wherein:
the memory is used for storing a computer program;
The processor is configured to execute the computer program to implement the network information acquisition method described above.
The application also provides a computer readable storage medium for storing a computer program, wherein the computer program realizes the network information acquisition method when being executed by a processor.
The application provides a network information acquisition method, which is applied to an agent program, wherein the agent program is deployed on acquired equipment and comprises the following steps: acquiring network flow information through an operating system kernel of the acquired equipment; and generating network information corresponding to the plurality of applications in the acquired equipment respectively by utilizing the network flow information.
It can be seen that the method directly acquires network flow information from the kernel of the operating system, the operating system is deployed on the acquired device, the operation of the device is monitored by the kernel of the operating system, and the monitored content comprises relevant content of network communication. The monitoring content of the operating system is comprehensive and accurate, so that the network flow information directly obtained from the kernel of the operating system by using the agent program has more effective information. Meanwhile, the agent program does not need to execute operation with larger expenditure, does not need to actively collect and count information, and only needs to acquire the existing network flow information from the kernel of the operating system, so that the expenditure of computing resources is smaller. After the network flow information is obtained, the network flow information is split to an application level, and then the network information corresponding to each application can be generated. The method has the advantages that the effective information of the network information is more, and the computing resource cost is smaller.
In addition, the application also provides a network information acquisition device, electronic equipment and a computer readable storage medium, and the network information acquisition device and the computer readable storage medium have the same beneficial effects.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related art, the drawings that are required to be used in the embodiments or the related technical descriptions will be briefly described, and it is apparent that the drawings in the following description are only embodiments of the present application, and other drawings may be obtained according to the provided drawings without inventive effort for those skilled in the art.
Fig. 1 is a flowchart of a network information acquisition method according to an embodiment of the present application;
Fig. 2 is a schematic diagram of an application scenario provided in an embodiment of the present application;
fig. 3 is a flowchart of network information acquisition under a linux system according to an embodiment of the present application;
fig. 4 is a flowchart of network information acquisition under another linux system according to an embodiment of the present application;
fig. 5 is a flowchart of network information acquisition under a Windows system according to an embodiment of the present application;
fig. 6 is a diagram of a visual display effect of network information according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a network information acquisition device according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Currently, network information is generally obtained by monitoring network traffic of a device port, for example, the number of data packets and bytes coming in and going out of each monitored network port can be collected by means of SNMP (Simple Network Management Protocol ) protocol, and the collected information is rough, so that the distribution condition of various different types of customer service applications in the network layer data traffic in the total traffic cannot be distinguished, and the flowing in and out traffic cannot be analyzed. Or may derive flow information in the network device based on the IPFIX (IP Flow Information Export, IP data flow information output) protocol. Compared with the SNMP protocol, the method provides richer information, can refine the granularity of the flow, counts the number of data packets, bytes, flow direction and the like of the flow, but still lacks 7-layer application information, and cannot acquire information of a transmission layer protocol level (such as Round Trip Time (RTT), congestion control algorithm, packet loss rate and the like) of a TCP (Transmission Control Protocol ) protocol with some finer granularity, and cannot support comprehensive flow analysis facing the application. In addition, the network traffic packet grabbing device (such as sniffer, sniffer, a software device monitoring the operation of network data) may be used to perform packet grabbing analysis on the traffic under a certain port of the network device. By acquiring and counting the address distribution, the data packet size, the load and other information in the data packet. However, the packet capturing device generally needs to capture, store and analyze the original data packet, so that the cost of calculation and storage resources is huge, and the user service is greatly influenced.
In order to solve the above problems, the present application provides a network information acquisition method that takes account of the consumption of resources such as the effective data amount of network information and calculation, and uses the characteristics of the kernel of the operating system to acquire more effective data with less overhead. Specifically, an agent (may be referred to as a agent) is deployed on a device to be collected, where the device to be collected may specifically be a virtual device (for example, a virtual machine) or a physical entity device, and the device to be collected has an operating system, where an kernel of the operating system has a capability of monitoring network flow information of itself during running, and the agent interacts or accesses with the kernel of the operating system to obtain network flow information that is monitored and generated by the kernel of the operating system, so as to obtain network information corresponding to each application on the device to be collected respectively.
Referring to fig. 1, fig. 1 is a flowchart of a network information acquisition method according to an embodiment of the present application. The method comprises the following steps:
S101: and obtaining network flow information through an operating system kernel of the acquired equipment.
It will be appreciated that the operating system of the acquired device may have a variety of options, such as the widely used linux system, or the Windows system. Whatever the type of operating system, it has a corresponding operating system kernel, and a computer is composed of various external hardware devices, such as a memory, a processor, a hard disk, etc., so that if each application needs to interface with the hardware devices, the complexity of the computer is greatly increased. In order to avoid the situation, a kernel is arranged in the operating system, and the kernel is used as a bridge for connecting the application with the hardware device, so that the application program only needs to be concerned about interaction with the kernel, and does not need to be concerned about specific details of hardware interaction.
The network flow information refers to flow data formed when the collected device communicates based on various network communication protocols, and the content of the network flow information can include various data packet numbers, sizes, round trip time, network quintuples, byte numbers, flow directions and the like. In the application, the kernel of the operating system has the capability of monitoring network communication to obtain network flow information, and the mode of the capability can be different according to different operating systems, and in some cases, the kernel of the operating system needs to be set to a certain degree to enable the kernel of the operating system to have the capability of monitoring.
When the collected equipment runs, the kernel of the operating system monitors the network communication condition of the collected equipment to obtain corresponding network flow information. The agent may obtain network flow information from the operating system kernel on demand, e.g., in real time, upon detection of a preset event being triggered, or on a periodic basis. The manner in which the network flow information is obtained may vary depending on the version, type, etc. of the operating system, and several specific manners will be described later.
S102: and generating network information corresponding to the plurality of applications in the acquired equipment respectively by utilizing the network flow information.
Since there are multiple applications in the acquired device, each application corresponds to a different network connection. After obtaining the network flow information, the network flow information corresponding to different network connections and the application information of the corresponding application may be combined to obtain the application-level network information, where the application information may include unique identification information of the application, or may further include other content, such as an application type, and the like. The embodiment does not limit the generation mode of the application-level network information, for example, all application information of all applications in the acquired device can be obtained, and all application information and network flow information are compared and matched to obtain the network information; or in a feasible implementation, the network flow information itself comprises application information, and the application-level network information can be obtained by splitting the network flow information.
Specifically, referring to fig. 2, fig. 2 is a schematic view of an application scenario provided in an embodiment of the present application. In this embodiment, the collected device is a virtual machine VM, and the agent deploys on the VM, specifically, may start running in a background service mode, where the agent runs based on configuration information, and the configuration information may include configuration items such as an information collection period and network information content. After the network information is acquired, the agent sends the network information to a server for subsequent treatment.
In one embodiment, the operating system type of the collected device is linux, in which case the collecting process of the network flow information may include:
step 11: and obtaining the kernel version number of the kernel of the operating system.
Step 12: and if the kernel version number is higher than the preset version number, acquiring network flow information by using the target mounting program.
Step 13: and if the kernel version number is not higher than the preset version number, acquiring network flow information by using socket connection with the kernel of the operating system.
For a linux system, the size of the operating version number determines the manner in which network flow information is obtained from the operating system kernel. The kernel version number may be obtained at agent startup. If the kernel version number is higher than the preset version number, the target mounting program can be mounted in the kernel function of the operating system, and then the network flow information is collected by using the target mounting program.
Specifically, the preset version number may be 4.10, and the target mount program may be eBPF (extended Berkeley PACKET FILTER, a revolutionary technique, may run a sandbox program in a Linux kernel without modifying the kernel source code or loading the kernel module) program. After the agent is started, loading the target mounting program, mounting the target mounting program on a mounting point corresponding to the target kernel function, and starting the target mounting program after the mounting is successful. The target kernel function is a kernel function related to network service, and the target mounting program can extract information when the target kernel function is executed, namely, the network flow information can be acquired. After obtaining the network flow information, the target mounting program stores the network flow information in a corresponding data storage object, for example, for eBPF programs, the target mounting program stores the acquired information in the object eBPF map, and the agent can read the data storage object corresponding to the target mounting program to obtain the network flow information. The target mounting program can acquire comprehensive network flow information with extremely low cost, and can efficiently and flexibly add, delete or change indexes to be monitored according to different mounting points. Referring to fig. 3, fig. 3 is a flowchart of network information collection under a linux system according to an embodiment of the present application, a agent loads eBPF a program after starting and hangs on a designated Hook point (i.e. a mounting point corresponding to a target kernel function), and eBPF runs to collect data and store the data in a map. The agent can read data from the map according to the acquisition period, and can screen the map as required to obtain required monitoring indexes so as to form network flow information.
If the kernel version number is not higher than the preset version number, the agent can acquire network flow information by using socket connection with the kernel of the operating system. The socket connection is specifically a netlink connection, and the netlink socket is a special inter-process communication mechanism for realizing communication between a user state process and a kernel process, and is also an interface for a network application program to communicate with the kernel. an agent may encapsulate a network query request (which may be referred to as a netlink request) and send the network query request to the operating system kernel through a socket connection. The linux kernel maintains the statistical information of the network connection of various protocols on the network protocol stack, and after receiving the network query request, the linux kernel feeds back the statistical information of the network connection, namely network flow information, to the agent. and the agent receives the response data sent by the system kernel and analyzes the response data to obtain the network flow information.
When the load of the kernel of the operating system is too high, the kernel of the operating system may not respond to the network query request, which results in failure to acquire network information through the netlink. In this case, the agent may access a kernel network state directory corresponding to the operating system kernel, and read information therein to obtain network flow information. In particular, when a/proc directory in the linux system is used, a virtual file system is obtained, wherein files store the running state of a current kernel, and through the files, information on hardware, application programs, network connection and the like of the system can be understood, wherein content related to a network is recorded under the/proc/net/directory, and the directory is the kernel network state directory. And obtaining the network flow information by reading the information under the directory. This approach consumes more computing resources than the approach of obtaining network flow information through a netlink, due to the need to actively read the information, but is still much less than the approach employing sniffer.
The network flow information obtained in the above manner has the inode data, which can be called inode, and the network connection and each application can be matched by using the inode data, so as to obtain the application-level network information. Specifically, each process corresponds to a network connection, each network connection corresponds to a socket, and in the linux system, the socket is presented in the form of a socket file. The index node data corresponding to each network flow can be determined by using the network flow information, and the index node data is matched with each socket file to determine a target socket file and target index node data which are in one-to-one correspondence, so that the corresponding matching of the network flow (namely, the data flow of network connection) and the application is realized. And determining corresponding target application information based on the target socket file, and generating network information by utilizing the target application information and target network flow information corresponding to the target index node data. The target application information may include unique identification information of the target application, or may also include other information.
Referring to fig. 4, fig. 4 is a flowchart of network information acquisition under another linux system according to an embodiment of the present application. The agent opens the netlink connection with the kernel, sends a network query request, and if a response of the netlink is received, can also analyze the response data packet to obtain network flow information. If the network flow information responded by the netlink is not obtained, the kernel network state directory is read and analyzed, and the network flow information is actively read.
In another embodiment, the operating system type of the acquired device is Windows. In general, the Windows operating system actively monitors the TCP connection, and the obtained network flow information is transmission control protocol information, and the process of collecting the transmission control protocol information may include:
step 21: the call target function library obtains a connection table from the operating system kernel.
Step 22: connection entries are obtained from the connection table.
Step 23: and calling an objective function library, and obtaining transmission control protocol information from the kernel of the operating system by using the connection table entry.
A connection table is maintained in the Windows system kernel for recording the connection condition of the TCP network connection. IPHelper (Internet Protocol Helper ), an object function library, is a Windows-provided function library that can provide applications with the ability to retrieve and modify local network configurations. The function library may be used to obtain a TCP connection table from the Windows kernel, where the TCP connection table includes a plurality of connection entries, and each connection entry includes five tuple information of the TCP connection, connection state information, and application identification information of a corresponding application, for example, a process ID, i.e., pid (Process Identifier ) data. The Windows kernel also performs statistics of extension statistics information for each TCP connection, where the extension statistics information is specific running state information of the TCP connection. The method can determine each TCP connection by using each connection table item, call the target function library and the five-tuple information of the connection table item, and further query the expansion statistical information of the TCP connection, namely the transmission control protocol information, from the kernel of the operating system.
It will be appreciated that there are a number of states in the process of establishing, maintaining and disconnecting a TCP connection, for example, some TCP connections have not been actually established in the process of establishing, and it is not meaningful to obtain corresponding TCP information, where valid data is not present. Therefore, when acquiring the tcp information, the connection state information corresponding to each connection entry may be acquired first, and the tcp information may be obtained from the operating system kernel by using the connection entry whose connection state information is in the active connection state, for example, the connection state information is establish or closewait.
After the transmission control protocol information is obtained, application identification information, such as pid data, may be obtained from the connection table. And acquiring candidate application information, specifically, a designated dynamic link library can be called to obtain the candidate application information, wherein the candidate application information refers to information corresponding to each candidate application program on the equipment to be acquired, the candidate application information comprises application identification information, the candidate application information is screened by using the application identification information, target application information corresponding to each connection table item is determined, and then network information is generated by using the target application information and target transmission control protocol information corresponding to the connection table item.
Although the application processes are in a variable state, some application processes may exist all the time in the process of collecting network information many times, and corresponding network connections also exist all the time. In this case, when generating the network information, the application identification information corresponding to each connection entry may be acquired, and each application identification information may be used to determine whether there is corresponding historical application information in the memory, where the historical application information refers to a portion of the previously acquired network information that matches the application identification information. If so, the network information can be generated by utilizing the historical application information and the target transmission control protocol information corresponding to the connection table item, and the corresponding target application information is not required to be determined from the plurality of candidate application information. Correspondingly, after all new network information is obtained, the new historical network information is determined and stored into a memory for subsequent calling. Referring to fig. 5, fig. 5 is a flowchart of network information acquisition under a Windows system according to an embodiment of the present application. After collection starts, a IPHelper function library is called to obtain a connection table, and when the connection table is not empty, each connection table item is circularly traversed so as to obtain extension information (transmission control protocol information) and process name (application information) of each TCP connection, and then network information is formed. Specifically, the pid is obtained from the connection table, then the process name is obtained according to the pid, the memory is queried preferentially during the obtaining, if the query is not hit, the DLL is called for query, candidate application information is obtained, and then comparison and determination are carried out. And secondly, judging whether the connection state is an effective connection state or not by utilizing the connection state, if so, calling IPHelper a function library to acquire extension statistical information corresponding to the connection table item from the Windows kernel, specifically, transmitting quintuple information of the TCP connection to the Windows kernel and receiving the extension statistical information fed back by the Windows kernel.
In addition to TCP connections, there are applications that communicate based on the UDP (User Datagram Protocol ) protocol. The Windows kernel does not maintain statistics for UDP alone, so a special way needs to be designed to activate the monitoring of UDP by the Windows kernel. The network flow information acquired in this case is user datagram protocol information. In the application, an event tracking mechanism ETW (EVENT TRACING for Windows) in a Windows kernel is started specifically as a network event tracking mechanism. The ETW is a Windows event trace. A trace record mechanism for event objects created by user mode applications and kernel mode drivers is provided. The agent can inform the Windows kernel to start tracking the network event to start the mechanism when starting, and when each network data packet is received or transmitted in Windows, the Windows kernel can generate an event for recording the information of the data packet. The agent can monitor these events and further obtain user datagram protocol information. Specifically, the agent monitors network events generated by the kernel of the operating system, and after the network time tracking mechanism is started, the kernel reports all the network events, which may include events corresponding to a non-UDP protocol, so that the agent can perform filtering processing based on the network protocol on the network events, filter the events not based on the UDP protocol to obtain a target network event, and then analyze the target network event to obtain network flow information. The user datagram protocol information acquired in the mode comprises process name information, and if the application information only comprises the process name, the user datagram protocol can be directly determined to be network information.
Referring to fig. 2, after obtaining the network information in the above manner, the agent may read the configuration file to obtain the address of the monitoring server, and the agent may establish a connection with the server based on an http protocol (HyperText Transfer Protocol ), an RPC protocol (Remote Procedure Call Protocol, remote procedure call protocol), or other available protocols. When the agent collects the network information, the agent can perform local serialization into a protobuf format, and serialization (Serialization) is a process of converting the state information of the object into a form that can be stored or transmitted. During serialization, an object writes its current state to a temporary or persistent storage area. Later, the object may be recreated by reading or de-serializing the state of the object from the storage area. The serialization process may be followed by compression to reduce bandwidth occupation and then sending the network information to the server over the network connection.
The server side can deploy a Kafka message queue and Druid database, the network information reported by the agent is subjected to deserialization and decompression at the server side and then written into the Kafka message queue for buffering, and then Druid takes in data from the Kafka in real time and performs persistent storage. In addition, the network information stored in Druid may be subjected to various association and statistical analyses, and then the network analysis results are presented at Grafana. Referring to fig. 6, fig. 6 is a network information visual display effect diagram provided by the embodiment of the present application, where the visual display content may include an access relationship topology diagram between virtual machines, an access relationship and a traffic state between services (application programs), topN (top N larger) traffic display, abnormal traffic monitoring, and the like. It should be noted that the message queue, database and visual presentation software are only one possible option, and may be replaced by similar content.
By applying the network information acquisition method provided by the embodiment of the application, network flow information is directly acquired from the kernel of the operating system, the operating system is deployed on the acquired equipment, the operation of the equipment is monitored by the kernel of the operating system, and the monitored content comprises related content of network communication. The monitoring content of the operating system is comprehensive and accurate, so that the network flow information directly obtained from the kernel of the operating system by using the agent program has more effective information. Meanwhile, the agent program does not need to execute operation with larger expenditure, does not need to actively collect and count information, and only needs to acquire the existing network flow information from the kernel of the operating system, so that the expenditure of computing resources is smaller. After the network flow information is obtained, the network flow information is split to an application level, and then the network information corresponding to each application can be generated. The method has the advantages that the effective information of the network information is more, and the computing resource cost is smaller.
The following describes a network information collecting device provided by the embodiment of the present application, and the network information collecting device described below and the network information collecting method described above may be referred to correspondingly.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a network information acquisition device according to an embodiment of the present application, including:
A network flow information obtaining module 110, configured to obtain network flow information through an operating system kernel of the collected device;
The network information generating module 120 is configured to generate network information corresponding to each of the plurality of applications in the acquired device by using the network flow information.
Optionally, the network flow information acquisition module 110 includes:
The version number acquisition unit is used for acquiring the kernel version number of the kernel of the operating system if the operating system type is linux;
The mounting program acquisition unit is used for acquiring network flow information by utilizing the target mounting program if the kernel version number is higher than a preset version number;
And the socket acquisition unit is used for acquiring network flow information by using socket connection with the kernel of the operating system if the kernel version number is not higher than the preset version number.
Optionally, the mounting program acquisition unit includes:
The reading subunit is used for reading the data storage object corresponding to the target mounting program to obtain network flow information;
The target mounting program is mounted on a mounting point corresponding to the target kernel function, and network flow information is started and collected after the mounting is successful.
Optionally, the socket collection unit includes:
The sending subunit is used for sending a network query request to the system kernel through socket connection;
And the receiving subunit is used for receiving the response data sent by the system kernel and analyzing the response data to obtain the network flow information.
Optionally, the method further comprises:
and the access reading unit is used for accessing and reading the kernel network state catalog corresponding to the kernel of the operating system to obtain the network flow information if the response data corresponding to the network query request is not received.
Optionally, the network information generating module 120 includes:
The index node determining unit is used for determining index node data corresponding to each network flow respectively by utilizing the network flow information;
The index matching unit is used for matching the index node data with each socket file and determining a target socket file and target index node data which are in one-to-one correspondence;
The first generation unit is used for determining corresponding target application information based on the target socket file and generating network information by utilizing the target application information and target network flow information corresponding to the target index node data.
Optionally, the network flow information acquisition module 110 includes:
the connection table acquisition unit is used for calling the objective function library to obtain a connection table from the kernel of the operating system if the operating system type is Windows and the network flow information is transmission control protocol information;
The table item acquisition unit is used for obtaining a connection table item from the connection table;
and the information acquisition unit is used for calling the target function library and obtaining the transmission control protocol information from the kernel of the operating system by using the connection table entry.
Optionally, the information acquisition unit includes:
The state determination subunit is used for acquiring the connection state information corresponding to each connection table item respectively;
and the effective acquisition subunit is used for acquiring the transmission control protocol information from the kernel of the operating system by utilizing the connection table item of which the connection state information is in the effective connection state.
Optionally, the network information generating module 120 includes:
The identifier acquisition unit is used for acquiring application identifier information corresponding to each connection table item respectively;
The history judging unit is used for judging whether corresponding history application information exists in the memory or not by utilizing the application identification information;
The determining unit is used for generating network information by utilizing the historical application information and the target transmission control protocol information corresponding to the connection table item if the historical application information and the target transmission control protocol information exist;
Correspondingly, the method further comprises the steps of:
And the storage module is used for determining the network information as new historical network information and storing the new historical network information into the memory.
Optionally, the network information generating module 120 includes:
The identifier acquisition unit is used for acquiring application identifier information corresponding to each connection table item respectively;
the candidate screening unit is used for acquiring candidate application information, screening the candidate application information by using the application identification information and determining target application information corresponding to each connection table item respectively;
and the second generation unit is used for generating network information by utilizing the target application information and the target transmission control protocol information corresponding to the connection table item.
Optionally, the network flow information acquisition module 110 includes:
the monitoring unit is used for monitoring network events generated by the kernel of the operating system if the operating system type is Windows and the network flow information is user datagram protocol information; the network event tracking mechanism of the kernel of the operating system is started;
And the filtering unit is used for filtering the network event based on the network protocol to obtain a target network event and obtaining network flow information by utilizing the target network event.
The electronic device provided by the embodiment of the application is introduced below, and the electronic device described below and the network information acquisition method described above can be referred to correspondingly.
Referring to fig. 8, fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Wherein the electronic device 100 may include a processor 101 and a memory 102, and may further include one or more of a multimedia component 103, an information input/information output (I/O) interface 104, and a communication component 105.
The processor 101 is configured to control overall operations of the electronic device 100 to complete all or part of the steps in the network information acquisition method described above; the memory 102 is used to store various types of data to support operation at the electronic device 100, which may include, for example, instructions for any application or method operating on the electronic device 100, as well as application-related data. The Memory 102 may be implemented by any type or combination of volatile or non-volatile Memory devices, such as one or more of static random access Memory (Static Random Access Memory, SRAM), electrically erasable programmable Read-Only Memory (ELECTRICALLY ERASABLE PROGRAMMABLE READ-Only Memory, EEPROM), erasable programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk.
The multimedia component 103 may include a screen and an audio component. Wherein the screen may be, for example, a touch screen, the audio component being for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 102 or transmitted through the communication component 105. The audio assembly further comprises at least one speaker for outputting audio signals. The I/O interface 104 provides an interface between the processor 101 and other interface modules, which may be a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 105 is used for wired or wireless communication between the electronic device 100 and other devices. Wireless Communication, such as Wi-Fi, bluetooth, near field Communication (NFC for short), 2G, 3G or 4G, or a combination of one or more thereof, the corresponding Communication component 105 may thus comprise: wi-Fi part, bluetooth part, NFC part.
The electronic device 100 may be implemented by one or more Application Specific Integrated Circuits (ASIC), digital signal Processor (DIGITAL SIGNAL Processor, DSP), digital signal processing device (DIGITAL SIGNAL Processing Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field Programmable GATE ARRAY, FPGA), controller, microcontroller, microprocessor or other electronic components for executing the network information acquisition method according to the above embodiment.
The following describes a computer readable storage medium provided in an embodiment of the present application, where the computer readable storage medium described below and the network information collection method described above may be referred to correspondingly.
The application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the steps of the network information acquisition method when being executed by a processor.
The computer readable storage medium may include: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality using different approaches for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it is further noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms include, comprise, or any other variation is intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
The principles and embodiments of the present application have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present application and the core ideas thereof; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Claims (11)
1. The network information acquisition method is characterized by being applied to an agent program, wherein the agent program is deployed on an acquired device and comprises the following steps of:
obtaining network flow information through an operating system kernel of the acquired equipment;
generating network information corresponding to a plurality of applications in the acquired equipment respectively by utilizing the network flow information;
The network flow information represents flow data formed when the collected equipment communicates based on various network communication protocols;
If the operating system type is linux, obtaining network flow information through the operating system kernel of the acquired device includes:
Acquiring a kernel version number of the kernel of the operating system;
If the kernel version number is higher than a preset version number, acquiring the network flow information by using a target mounting program;
If the kernel version number is not higher than the preset version number, acquiring the network flow information by using socket connection with the kernel of the operating system;
Or if the operating system type is Windows and the network flow information is transmission control protocol information, the obtaining the network flow information by the operating system kernel of the acquired device includes:
calling an objective function library to obtain a connection table from the kernel of the operating system;
Obtaining a connection table item from the connection table;
calling the target function library, and obtaining the transmission control protocol information from the kernel of the operating system by utilizing the connection table item;
Or if the operating system type is Windows and the network flow information is user datagram protocol information, the obtaining the network flow information by the operating system kernel of the acquired device includes:
monitoring a network event generated by the kernel of the operating system; the network event tracking mechanism of the operating system kernel is started;
and filtering the network event based on a network protocol to obtain a target network event, and obtaining the network flow information by using the target network event.
2. The network information collection method according to claim 1, wherein the collecting the network flow information using the target mounting program includes:
reading a data storage object corresponding to the target mounting program to obtain the network flow information;
the target mounting program is mounted on a mounting point corresponding to the target kernel function, and the network flow information is started and collected after the mounting is successful.
3. The network information collection method according to claim 1, wherein the obtaining the network flow information using a socket connection with the operating system kernel includes:
Sending a network query request to the system kernel through the socket connection;
And receiving response data sent by the system kernel, and analyzing the response data to obtain the network flow information.
4. The network information collection method according to claim 3, further comprising:
And if the response data corresponding to the network query request is not received, accessing and reading a kernel network state directory corresponding to the kernel of the operating system to obtain the network flow information.
5. The network information collection method according to any one of claims 1 to 4, wherein the generating, using the network flow information, network information respectively corresponding to a plurality of applications in the collected device includes:
determining index node data corresponding to each network flow respectively by utilizing the network flow information;
Matching the index node data with each socket file to determine a target socket file and target index node data which are in one-to-one correspondence;
And determining corresponding target application information based on the target socket file, and generating the network information by utilizing the target application information and target network flow information corresponding to the target inode data.
6. The network information collection method according to claim 1, wherein the obtaining the transmission control protocol information from the operating system kernel using the connection table entry includes:
acquiring connection state information corresponding to each connection table item respectively;
And obtaining the transmission control protocol information from the kernel of the operating system by using the connection table item of which the connection state information is in an effective connection state.
7. The network information collection method according to claim 1 or 6, wherein the generating network information corresponding to each of the plurality of applications in the collected device by using the network flow information includes:
Acquiring application identification information corresponding to each connection table item respectively;
Judging whether corresponding historical application information exists in the memory or not by utilizing each application identification information;
If so, generating the network information by utilizing the historical application information and the target transmission control protocol information corresponding to the connection table item;
Correspondingly, the method further comprises the steps of:
And determining the network information as new historical network information and storing the new historical network information into the memory.
8. The network information collection method according to claim 1 or 6, wherein the generating network information corresponding to each of the plurality of applications in the collected device by using the network flow information includes:
Acquiring application identification information corresponding to each connection table item respectively;
Candidate application information is obtained, the candidate application information is screened by utilizing the application identification information, and target application information corresponding to each connection table item is determined;
And generating the network information by utilizing the target application information and the target transmission control protocol information corresponding to the connection table item.
9. The network information acquisition device is characterized by being applied to an agent program, wherein the agent program is deployed on an acquired device and comprises the following components:
The network flow information acquisition module is used for acquiring network flow information through the operating system kernel of the acquired equipment;
the network information generation module is used for generating network information corresponding to a plurality of applications in the acquired equipment respectively by utilizing the network flow information;
The network flow information represents flow data formed when the collected equipment communicates based on various network communication protocols;
The network flow information acquisition module comprises:
The version number acquisition unit is used for acquiring the kernel version number of the kernel of the operating system if the type of the operating system is linux;
the mounting program acquisition unit is used for acquiring the network flow information by utilizing a target mounting program if the kernel version number is higher than a preset version number;
The socket acquisition unit is used for acquiring the network flow information by using socket connection with the operating system kernel if the kernel version number is not higher than the preset version number;
or, the network flow information acquisition module includes:
The connection table acquisition unit is used for calling the objective function library to obtain a connection table from the kernel of the operating system if the operating system type is Windows and the network flow information is transmission control protocol information;
the table item acquisition unit is used for acquiring a connection table item from the connection table;
the information acquisition unit is used for calling the target function library and obtaining the transmission control protocol information from the kernel of the operating system by utilizing the connection table entry;
or, the network flow information acquisition module includes:
The monitoring unit is used for monitoring network events generated by the kernel of the operating system if the operating system type is Windows and the network flow information is user datagram protocol information; the network event tracking mechanism of the operating system kernel is started;
and the filtering unit is used for filtering the network event based on the network protocol to obtain a target network event and obtaining the network flow information by utilizing the target network event.
10. An electronic device comprising a memory and a processor, wherein:
the memory is used for storing a computer program;
the processor is configured to execute the computer program to implement the network information collection method according to any one of claims 1 to 8.
11. A computer readable storage medium for storing a computer program, wherein the computer program when executed by a processor implements the network information acquisition method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210535738.2A CN115002186B (en) | 2022-05-17 | Network information acquisition method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210535738.2A CN115002186B (en) | 2022-05-17 | Network information acquisition method and device, electronic equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115002186A CN115002186A (en) | 2022-09-02 |
| CN115002186B true CN115002186B (en) | 2024-07-09 |
Family
ID=
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110351275A (en) * | 2019-07-11 | 2019-10-18 | 北京脉冲星科技有限公司 | A kind of host port flux monitoring method, system, device and storage equipment |
| CN111162973A (en) * | 2019-12-31 | 2020-05-15 | 奇安信科技集团股份有限公司 | Data flow acquisition method and device, electronic equipment and medium |
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110351275A (en) * | 2019-07-11 | 2019-10-18 | 北京脉冲星科技有限公司 | A kind of host port flux monitoring method, system, device and storage equipment |
| CN111162973A (en) * | 2019-12-31 | 2020-05-15 | 奇安信科技集团股份有限公司 | Data flow acquisition method and device, electronic equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11855850B2 (en) | Systems and methods for networked microservice modeling and visualization | |
| US9578045B2 (en) | Method and apparatus for providing forensic visibility into systems and networks | |
| US9565076B2 (en) | Distributed network traffic data collection and storage | |
| US9641413B2 (en) | Methods and computer program products for collecting storage resource performance data using file system hooks | |
| US9158649B2 (en) | Methods and computer program products for generating a model of network application health | |
| CN103152352A (en) | Perfect information security and forensics monitoring method and system based on cloud computing environment | |
| US20020124078A1 (en) | System for self-monitoring of SNMP data collection process | |
| CN112256542B (en) | eBPF-based micro-service system performance detection method, device and system | |
| CN111124819A (en) | Method and device for monitoring full link | |
| US8661456B2 (en) | Extendable event processing through services | |
| CN111181799B (en) | Network traffic monitoring method and equipment | |
| CN111258851B (en) | Cluster alarm method, device, setting and storage medium | |
| CN1633110A (en) | Flow analysis method based on Linux core | |
| CN112350854B (en) | Flow fault positioning method, device, equipment and storage medium | |
| CN114039900A (en) | Efficient network data packet protocol analysis method and system | |
| EP2634699B1 (en) | Application monitoring | |
| CN113760652A (en) | Method, system, device and storage medium for full link monitoring based on application | |
| CN115002186B (en) | Network information acquisition method and device, electronic equipment and readable storage medium | |
| CN115712646A (en) | Alarm strategy generation method, device and storage medium | |
| US20060053021A1 (en) | Method for monitoring and managing an information system | |
| CN115002186A (en) | Network information acquisition method and device, electronic equipment and readable storage medium | |
| CN112764990B (en) | Target process monitoring method and device and computer equipment | |
| Hirakawa et al. | Advances in visual programming | |
| CN112910842A (en) | Network attack event evidence obtaining method and device based on flow reduction | |
| CN112181789B (en) | Equipment end log collection method based on AWS |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |