CN115002186A - Network information acquisition method and device, electronic equipment and readable storage medium - Google Patents
Network information acquisition method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN115002186A CN115002186A CN202210535738.2A CN202210535738A CN115002186A CN 115002186 A CN115002186 A CN 115002186A CN 202210535738 A CN202210535738 A CN 202210535738A CN 115002186 A CN115002186 A CN 115002186A
- Authority
- CN
- China
- Prior art keywords
- information
- network
- kernel
- network flow
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/146—Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/34—Network arrangements or protocols for supporting network services or applications involving the movement of software or configuration parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/161—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
- H04L69/162—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields involving adaptations of sockets based mechanisms
Abstract
The application discloses a network information acquisition method, a device, electronic equipment and a computer readable storage medium, wherein the method is applied to an agent program, the agent program is deployed in the acquired equipment, and the method comprises the following steps: obtaining network flow information through an operating system kernel of the collected equipment; generating network information corresponding to a plurality of applications in the acquired equipment by using the network flow information; the monitoring content of the operating system is comprehensive and accurate, so that the network flow information directly acquired from the kernel of the operating system by using the agent program has more effective information; meanwhile, the agent program does not need to execute operation with high cost, does not need to actively collect and count information, and only needs to acquire the existing network flow information from the kernel of the operating system, so that the computing resource cost is low.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a network information collecting method, a network information collecting apparatus, an electronic device, and a computer-readable storage medium.
Background
With the development of internet technology, more and more services are provided through a network, in order to ensure the reliability and stability of network services, the network performance at an application level needs to be timely and effectively detected, and before the network performance is judged to be good or bad, network information required for judgment needs to be acquired. At present, network information is generally acquired by monitoring network traffic of a device port, but the method acquires less network information and has higher computational resource overhead.
Disclosure of Invention
In view of the above, an object of the present application is to provide a network information collecting method, a network information collecting apparatus, an electronic device, and a computer-readable storage medium, where effective information of the obtained network information is more and the computing resource overhead is smaller.
In order to solve the above technical problem, the present application provides a network information collecting method, which is applied to an agent program, where the agent program is deployed in a collected device, and the method includes:
obtaining network flow information through an operating system kernel of the collected equipment;
and generating network information corresponding to a plurality of applications in the acquired equipment by using the network flow information.
Optionally, the obtaining, by an operating system kernel of the acquired device, network flow information includes:
acquiring a kernel version number of the kernel of the operating system;
if the kernel version number is higher than a preset version number, acquiring the network flow information by using a target mounting program;
and if the kernel version number is not higher than the preset version number, acquiring the network flow information by utilizing the socket connection between the kernel of the operating system and the kernel of the operating system.
Optionally, the collecting the network flow information by using the target mount program includes:
reading a data storage object corresponding to the target mounting program to obtain the network flow information;
and the target mounting program is mounted at a mounting point corresponding to the target kernel function, and the network flow information is started and collected after the mounting is successful.
Optionally, the obtaining the network flow information by using a socket connection with the kernel of the operating system includes:
sending a network query request to the system kernel through the socket connection;
and receiving response data sent by the system kernel, and analyzing the response data to obtain the network flow information.
Optionally, the method further comprises:
and if the response data corresponding to the network query request is not received, accessing and reading a kernel network state directory corresponding to an operating system kernel to obtain the network flow information.
Optionally, the generating, by using the network flow information, network information corresponding to each of the plurality of applications in the acquired device includes:
determining index node data corresponding to each network flow by using the network flow information;
matching the index node data with each socket file to determine a target socket file and target index node data which correspond to each other one by one;
and determining corresponding target application information based on the target socket file, and generating the network information by using the target application information and target network flow information corresponding to the target index node data.
Optionally, if the network flow information is transmission control protocol information, the obtaining of the network flow information by the operating system kernel of the acquired device includes:
calling a target function library to obtain a connection table from the kernel of the operating system;
obtaining a connection table item from the connection table;
and calling the target function library, and obtaining the transmission control protocol information from the kernel of the operating system by using the connection table item.
Optionally, the obtaining the transmission control protocol information from the operating system kernel by using the connection table entry includes:
acquiring connection state information corresponding to each connection table item;
and obtaining the transmission control protocol information from the kernel of the operating system by using the connection table entry of which the connection state information is in the effective connection state.
Optionally, the generating, by using the network flow information, network information corresponding to each of the plurality of applications in the acquired device includes:
acquiring application identification information corresponding to each connection table item;
judging whether corresponding historical application information exists in the memory or not by utilizing the application identification information;
if the historical application information exists, the network information is generated by using the historical application information and the target transmission control protocol information corresponding to the connection table entry;
correspondingly, the method also comprises the following steps:
and determining the network information as new historical network information and storing the new historical network information in the memory.
Optionally, the generating, by using the network flow information, network information corresponding to each of the plurality of applications in the acquired device includes:
acquiring application identification information corresponding to each connection table item;
acquiring candidate application information, screening the candidate application information by using the application identification information, and determining target application information corresponding to each connection table item;
and generating the network information by using the target application information and the target transmission control protocol information corresponding to the connection table entry.
Optionally, if the network flow information is user datagram protocol information, the obtaining of the network flow information by an operating system kernel of the collected device includes:
monitoring a network event generated by the kernel of the operating system; wherein a network event tracking mechanism of the operating system kernel is started;
and filtering the network event based on a network protocol to obtain a target network event, and obtaining the network flow information by using the target network event.
The application also provides a network information acquisition device, which is applied to an agent program, wherein the agent program is deployed in the acquired equipment, and the network information acquisition device comprises:
the network flow information acquisition module is used for acquiring network flow information through an operating system kernel of the acquired equipment;
and the network information generating module is used for generating network information corresponding to the plurality of applications in the acquired equipment by utilizing the network flow information.
The present application further provides an electronic device comprising a memory and a processor, wherein:
the memory is used for storing a computer program;
the processor is configured to execute the computer program to implement the network information acquisition method.
The present application further provides a computer-readable storage medium for storing a computer program, wherein the computer program is executed by a processor to implement the above-mentioned network information collecting method.
The network information acquisition method provided by the application is applied to an agent program, the agent program is deployed in an acquired device, and the method comprises the following steps: obtaining network flow information through an operating system kernel of the collected equipment; and generating network information corresponding to the plurality of applications in the acquired equipment by using the network flow information.
Therefore, the method directly obtains the network flow information from the operating system kernel, the operating system is deployed on the collected equipment, the operation of the equipment is monitored by the operating system kernel, and the monitored content comprises the related content of network communication. The monitoring content of the operating system is comprehensive and accurate, so that the network flow information directly acquired from the operating system kernel by using the agent program has more effective information. Meanwhile, the agent program does not need to execute operation with high cost, does not need to actively collect and count information, and only needs to acquire the existing network flow information from the kernel of the operating system, so that the computing resource cost is low. After the network flow information is obtained, the network flow information is split to the application level, and then the network information corresponding to each application can be generated. The method has the advantages that more effective information of the network information is acquired, and the cost of computing resources is low.
In addition, the application also provides a network information acquisition device, electronic equipment and a computer readable storage medium, and the network information acquisition device, the electronic equipment and the computer readable storage medium also have the beneficial effects.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments or related technologies of the present application, the drawings needed to be used in the description of the embodiments or related technologies are briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a network information acquisition method according to an embodiment of the present disclosure;
fig. 2 is a schematic view of an application scenario provided in an embodiment of the present application;
fig. 3 is a flow chart of network information acquisition in the linux system according to an embodiment of the present disclosure;
fig. 4 is a flow chart of network information acquisition in another linux system according to an embodiment of the present application;
fig. 5 is a flow chart of network information collection in a Windows system according to an embodiment of the present application;
fig. 6 is a network information visualization display effect diagram provided in the embodiment of the present application;
fig. 7 is a schematic structural diagram of a network information acquisition device according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
Currently, Network information is usually obtained by monitoring Network traffic of an equipment port, for example, the number of data packets and the number of bytes that each monitored Network port enters and exits may be collected by using an SNMP (Simple Network Management Protocol), and the information collected by this method is rough, and generally, not only the distribution status of various types of customer service applications in the Network layer data traffic cannot be distinguished, but also the flow direction of the entering and exiting traffic cannot be analyzed. Alternatively, the Flow Information in the network device may be derived based on an IPFIX (IP Flow Information Export) protocol. Compared with the SNMP Protocol, the method provides richer information, can refine to the granularity of the stream, counts the number of packets, the number of bytes, the flow direction and the like of the stream, but still lacks application information of 7 layers, cannot acquire some information of a Transmission layer Protocol level with finer granularity (for example, RTT (Round Trip Time) of a TCP (Transmission Control Protocol), a congestion Control algorithm, a packet loss rate and the like), and cannot support application-oriented comprehensive flow analysis. In addition, a network traffic packet capture device (e.g., Sniffer, a software device that monitors network data operations) may be used to perform packet capture analysis on traffic under a port of the network device. By acquiring and counting information such as address distribution, data packet size, load and the like in the data packet. However, the packet capturing device generally needs to capture, store and analyze the original data packet, and the computing and storing resource overhead is huge, which may have a large impact on the user service.
In order to solve the above problems, the present application provides a network information collection method that considers both the effective data amount of network information and the resource consumption of computation and the like, which utilizes the characteristics of the kernel of the operating system and can obtain more effective data with less overhead. Specifically, an agent (which may be called agent) is deployed on the acquired device, the acquired device may specifically be a virtual device (for example, a virtual machine) or a physical entity device, the acquired device has an operating system, an operating system kernel has a capability of monitoring network flow information of the acquired device during running, the agent interacts or accesses with the operating system kernel to obtain the network flow information generated by monitoring of the operating system kernel, and then network information corresponding to each application on the acquired device is obtained.
Referring to fig. 1, fig. 1 is a flowchart of a network information collecting method according to an embodiment of the present disclosure. The method comprises the following steps:
s101: and obtaining the network flow information through the operating system kernel of the collected equipment.
It will be appreciated that there are many options for the operating system of the acquired device, such as the widely used linux system, or the Windows system. Regardless of the type of the operating system, the operating system has a corresponding operating system kernel, and the computer is composed of various external hardware devices, such as a memory, a processor, a hard disk, and the like, and if each application needs to interface with the hardware devices through a communication protocol, the complexity of the computer is greatly increased. In order to avoid the situation, the kernel is arranged in the operating system and serves as a bridge for connecting the application with the hardware equipment, so that the application program only needs to care about interaction with the kernel and does not need to care about specific details of hardware interaction.
The network flow information refers to streaming data formed when the acquired device communicates based on various network communication protocols, and the content of the network flow information may include multiple types, such as the number of packets, the size, the round trip time, the network quintuple, the number of bytes, the flow direction, and the like. In the application, the kernel of the operating system has the capability of monitoring network communication to obtain network flow information, and the manner of having the capability may be different according to different operating systems, and in some cases, the kernel of the operating system needs to be set to a certain extent so as to have the monitoring capability.
When the collected equipment runs, the network communication condition of the collected equipment is monitored by the operating system, and corresponding network flow information is obtained. The agent may obtain network flow information from the operating system kernel as needed, e.g., in real time, upon detecting that a predetermined event is triggered, or periodically. The manner of acquiring the network flow information is different according to the version and type of the operating system, and specific manners will be described later.
S102: and generating network information corresponding to the plurality of applications in the acquired equipment by using the network flow information.
Since there are multiple applications in the captured device, each application corresponds to a different network connection. After the network flow information is obtained, the network flow information corresponding to different network connections and the application information corresponding to the application may be combined to obtain the network information at the application level, and the application information may include unique identification information of the application, or may also include other content, such as the application type. The embodiment does not limit the generation manner of the application-level network information, and for example, all application information of all applications in the acquired device may be acquired, and all application information and network flow information are compared and matched to obtain network information; or, in a possible implementation, the network flow information itself includes the application information, and the network flow information is split, so that the application-level network information can be obtained.
Specifically, please refer to fig. 2, and fig. 2 is a schematic view of an application scenario provided in the embodiment of the present application. In this embodiment, the collected device is a virtual machine VM, and the agent is deployed on the VM, and specifically, may be started and operated in the form of a background service, and the agent operates based on configuration information, where the configuration information may include configuration items such as an information collection period and network information content. After the network information is acquired, the agent sends the network information to the server so as to carry out subsequent treatment.
In one embodiment, the type of the operating system of the acquired device is linux, in which case, the acquisition process of the network flow information may include:
step 11: and acquiring the kernel version number of the kernel of the operating system.
Step 12: and if the kernel version number is higher than the preset version number, acquiring the network flow information by using the target mounting program.
Step 13: and if the kernel version number is not higher than the preset version number, acquiring the network flow information by utilizing the socket connection with the kernel of the operating system.
For linux systems, the size of the operation version number determines the way network flow information is obtained from the operating system kernel. The kernel version number may be obtained at agent boot. If the kernel version number is higher than the preset version number, a target mounting program can be mounted in the kernel function of the operating system, and then the target mounting program is utilized to collect the network flow information.
Specifically, the preset version number may be 4.10, and the target mount program may be an eBPF (extended kernel Packet Filter, a revolutionary technology that can run a sandbox program in the Linux kernel without changing kernel source codes or loading kernel modules) program. After the agent is started, the target mount program is loaded and mounted at the mount point corresponding to the target kernel function, and the target mount program is started after the mount is successful. The target kernel function is a kernel function related to network services, and the target mount program can extract information when the target kernel function is executed, namely, network flow information can be acquired. After obtaining the network flow information, the target mount program stores the network flow information in a corresponding data storage object, for example, for an eBPF program, the target mount program stores the acquired information in an object, which is an eBPF map, and the agent may read the data storage object corresponding to the target mount program to obtain the network flow information. The target mounting program can acquire comprehensive network flow information with extremely low overhead, and indexes to be monitored can be efficiently and flexibly added, deleted or changed according to different mounting points. Referring to fig. 3, fig. 3 is a flow chart of collecting network information in a linux system according to an embodiment of the present application, an agent loads an eBPF program after being started and hangs the eBPF program at a specified Hook point (i.e., a hanging point corresponding to a target kernel function), and the eBPF can collect data after running and store the data in a map. The agent can read out the data from the map according to the acquisition cycle, and can also screen the data as required to obtain the required monitoring index, thereby forming network flow information.
If the kernel version number is not higher than the preset version number, the agent can acquire the network flow information by using the socket connection with the kernel of the operating system. The socket connection is specifically a netlink connection, and the netlink socket is a special interprocess communication mechanism for realizing communication between a user mode process and a kernel process, and is also an interface for communication between a network application program and the kernel. agent may encapsulate a network query request (which may be referred to as a netlink request) and send the network query request to the operating system kernel over a socket connection. The linux kernel maintains the statistical information of the network connection of various protocols on the network protocol stack, and after receiving the network query request, the linux kernel feeds the statistical information of the network connection, namely the network flow information, back to the agent. and the agent receives the response data sent by the system kernel and analyzes the response data to obtain the network flow information.
When the load of the kernel of the operating system is too high, the kernel of the operating system may not respond to the network query request, which may result in failure to obtain the network reservation information through the netlink. In this case, the agent may access the kernel network state directory corresponding to the kernel of the operating system, and read information therein to obtain the network flow information. Specifically, a/proc directory in the linux system is a virtual file system, in which files store the current running state of a kernel, and information on hardware, application programs, network connection and the like of the system can be understood through the files, wherein network-related content is recorded under the/proc/net/directory, and the directory is a kernel network state directory. By reading the information in the directory, the network flow information can be obtained. This approach consumes more computing resources than the approach of acquiring network flow information via a netlink, since it requires active reading of information, but is still much less than the approach using sniffers.
The network flow information obtained in the above manner has index node data, which may be called inode, and the network connection and each application may be matched by using the index node data to obtain application-level network information. Specifically, each process corresponds to one network connection, and each network connection corresponds to one socket (socket), which is presented in the form of a socket file in the linux system. By using the network flow information, index node data corresponding to each network flow can be determined, and the index node data is matched with each socket file to determine a target socket file and target index node data corresponding to each other one by one, so that corresponding matching between the network flow (i.e. network connection data flow) and the application is realized. And determining corresponding target application information based on the target socket file, and generating network information by using the target application information and target network stream information corresponding to the target index node data. The target application information may include unique identification information of the target application, or may also include other information.
Referring to fig. 4, fig. 4 is a flowchart of network information collection in another linux system according to an embodiment of the present application. and the agent opens the netlink connection with the kernel, sends a network query request, and can analyze the response data packet to obtain network flow information if receiving the response of the netlink. And if the network flow information responded by the netlink is not acquired, reading and analyzing the kernel network state directory and actively reading the network flow information.
In another embodiment, the operating system type of the acquired device is Windows. In general, a Windows operating system may actively monitor TCP connections, and the obtained network flow information is transmission control protocol information, and the acquisition process of the transmission control protocol information may include:
step 21: and calling the target function library to obtain the connection table from the kernel of the operating system.
Step 22: and obtaining a connection table item from the connection table.
Step 23: and calling a target function library, and obtaining the transmission control protocol information from the kernel of the operating system by using the connection table item.
A connection table is maintained in the Windows system kernel and used for recording the connection condition of TCP network connection. An ip Helper (Internet Protocol Helper), i.e. an objective function library, is a function library provided by Windows, and can provide an application program with the capability of retrieving and modifying the local network configuration. The function library is utilized to obtain a TCP connection table from the Windows kernel, where the TCP connection table includes a plurality of connection table entries, and each connection table entry includes five-tuple information of the TCP connection, connection state information, and application identification information of a corresponding application, such as a Process ID (Process Identifier) data. The Windows kernel also can carry out statistics of extended statistical information aiming at each TCP connection, and the extended statistical information is the specific running state information of the TCP connection. And determining each TCP connection by using each connection table entry, calling quintuple information of the target function library and the connection table entry, and further inquiring the extended statistical information of the TCP connection, namely transmission control protocol information from the kernel of the operating system.
It can be understood that there are multiple states in the process of establishing, maintaining and disconnecting TCP connections, for example, some TCP connections are not actually established in the process of establishing, and it is meaningless to obtain their corresponding TCP information, where there is no valid data. Therefore, when the tcp information is obtained, the connection state information corresponding to each connection table entry may be obtained first, and the tcp information is obtained from the kernel of the operating system by using the connection table entry of which the connection state information is in the valid connection state, for example, the connection state information is estabilish or closed wait.
After obtaining the tcp information, the application identification information, such as pid data, may be obtained from the connection table. The method includes the steps of obtaining candidate application information, specifically, calling a designated dynamic link library to obtain the candidate application information, wherein the candidate application information refers to information corresponding to each candidate application program on equipment to be acquired, and includes application identification information, screening the candidate application information by using the application identification information, determining target application information corresponding to each connection table item, and further generating network information by using the target application information and target transmission control protocol information corresponding to the connection table items.
Although the application process is in a changing state, some application processes may exist all the time in the process of collecting network information many times, and the corresponding network connections also exist all the time. In this case, when the network information is generated, the application identification information corresponding to each connection table entry may be obtained, and whether corresponding historical application information exists in the memory is determined by using each application identification information, where the historical application information refers to a part of the previously obtained network information that is the application information that matches the application identification information. If the connection table entry exists, the network information can be generated by using the historical application information and the target transmission control protocol information corresponding to the connection table entry, and the corresponding target application information does not need to be determined from the plurality of candidate application information. Correspondingly, after all the new network information is obtained, the new network information is determined as new historical network information and is stored in the memory for subsequent calling. Referring to fig. 5, fig. 5 is a flowchart illustrating network information collection in a Windows system according to an embodiment of the present disclosure. After the collection is started, calling an IPhelper function library to obtain a connection table, and circularly traversing each connection table item when the connection table is not empty so as to obtain the extension information (transmission control protocol information) and the process name (application information) of each TCP connection, thereby forming network information. Specifically, pid is obtained from a link table, a process name is obtained according to the pid, the pid is firstly inquired from a memory during obtaining, if the inquiry is not hit, a DLL is called for inquiry, candidate application information is obtained, and then comparison and determination are carried out. Secondly, judging whether the connection state is an effective connection state or not by using the connection state, if so, calling an IPhelper function library to acquire the extended statistical information corresponding to the connection table item from the Windows kernel, specifically, sending quintuple information of the TCP connection to the Windows kernel, and receiving the extended statistical information fed back by the Windows kernel.
In addition to TCP connection, there are also applications that communicate based on the UDP (User Datagram Protocol) Protocol. The Windows kernel does not separately aim at maintaining statistical information of the UDP, so a special mode needs to be designed to activate the monitoring of the Windows kernel on the UDP. The network flow information obtained in this case is user datagram protocol information. In the present application, the event tracking mechanism etw (event tracking for Windows) started in the Windows kernel is specifically a network event tracking mechanism. ETW is a type of Windows event trace. A trace record mechanism for user-mode applications and kernel-mode driver created event objects is provided. When agent starts, the agent can inform the Windows kernel to start the tracking of the network event so as to start the mechanism, and when one network data packet is received or sent in Windows, the Windows kernel can generate an event for recording the information of the data packet. The agent can monitor these events to obtain the user datagram protocol information. Specifically, the agent monitors a network event generated by an operating system kernel, and after a network time tracking mechanism is started, the kernel reports all network events, which may include events corresponding to a non-UDP protocol, so that the agent can perform filtering processing based on the network protocol on the network event, filter the events of the non-UDP protocol to obtain a target network event, and further analyze the target network event to obtain network stream information. The user datagram protocol information obtained by the method comprises the process name information, and if the application information only comprises the process name, the user datagram protocol can be directly determined as the network information.
Referring to fig. 2, after the network information is obtained in the above manner, the agent may read the configuration file to obtain an address of the server of the monitoring server, and the agent may establish a connection with the server based on an http Protocol (HyperText Transfer Protocol), an RPC Protocol (Remote Procedure Call Protocol), or other available protocols. When the agent collects the network information, the process can be locally serialized into a protobuf format, and Serialization (Serialization) is a process of converting the state information of an object into a form which can be stored or transmitted. During serialization, the object writes its current state to a temporary or persistent store. The object may later be recreated by reading or deserializing the state of the object from storage. The serialization process may be followed by compression to reduce bandwidth usage, and then network information may be sent to the server over the network connection.
The server side can be used for deploying a Kafka message queue and a Druid database, network information reported by the agent is deserialized and decompressed at the server side and then written into the Kafka message queue for buffering, and then the Druid intakes data from the Kafka in real time and stores the data in a persistent mode. In addition, the network information stored in the Druid may be subjected to various association analyses and statistical analyses, and then the network analysis results are presented on Grafana. Referring to fig. 6, fig. 6 is a network information visualization display effect diagram provided in the embodiment of the present application, and the content of the visualization display may include an access relationship topology diagram between virtual machines, an access relationship and a traffic state between services (application programs), TopN (top N large) traffic display, abnormal traffic monitoring, and the like. It should be noted that the message queue, the database and the visualization software are only a feasible option, and they can be replaced by the same kind of content.
By applying the network information acquisition method provided by the embodiment of the application, the network flow information is directly acquired from the operating system kernel, the operating system is deployed on the acquired equipment, the operation of the equipment is monitored by the operating system kernel, and the monitored content comprises related content of network communication. The monitoring content of the operating system is comprehensive and accurate, so that the network flow information directly acquired from the operating system kernel by using the agent program has more effective information. Meanwhile, the agent program does not need to execute operation with high cost, does not need to actively collect and count information, and only needs to acquire the existing network flow information from the kernel of the operating system, so that the computing resource cost is low. After the network flow information is obtained, the network flow information is split to the application level, and then the network information corresponding to each application can be generated. The method has the advantages that more effective information of the network information is acquired, and the cost of computing resources is low.
In the following, the network information acquisition device provided by the embodiment of the present application is introduced, and the network information acquisition device described below and the network information acquisition method described above may be referred to correspondingly.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a network information collecting device according to an embodiment of the present application, including:
a network flow information obtaining module 110, configured to obtain network flow information through an operating system kernel of the collected device;
the network information generating module 120 is configured to generate, by using the network flow information, network information corresponding to each of the plurality of applications in the acquired device.
Optionally, the network flow information obtaining module 110 includes:
a version number obtaining unit, configured to obtain a kernel version number of an operating system kernel if the operating system type is linux;
the mounting program acquisition unit is used for acquiring network flow information by using a target mounting program if the kernel version number is higher than a preset version number;
and the socket acquisition unit is used for acquiring the network flow information by utilizing the socket connection with the kernel of the operating system if the kernel version number is not higher than the preset version number.
Optionally, the mounted program collecting unit includes:
the reading subunit is used for reading a data storage object corresponding to the target mounting program to obtain network flow information;
the target mounting program is mounted at a mounting point corresponding to the target kernel function, and after the mounting is successful, the network flow information is started and collected.
Optionally, the socket collecting unit includes:
the sending subunit is used for sending a network query request to the system kernel through socket connection;
and the receiving subunit is used for receiving the response data sent by the system kernel and analyzing the response data to obtain the network flow information.
Optionally, the method further comprises:
and the access reading unit is used for accessing and reading the kernel network state directory corresponding to the kernel of the operating system to obtain the network flow information if the response data corresponding to the network query request is not received.
Optionally, the network information generating module 120 includes:
the index node determining unit is used for determining index node data corresponding to each network flow by using the network flow information;
the index matching unit is used for matching the index node data with each socket file to determine a target socket file and target index node data which correspond to each other one by one;
and the first generating unit is used for determining corresponding target application information based on the target socket file and generating network information by using the target application information and target network flow information corresponding to the target index node data.
Optionally, the network flow information obtaining module 110 includes:
the connection table obtaining unit is used for calling the target function library to obtain a connection table from the kernel of the operating system if the type of the operating system is Windows and the network flow information is transmission control protocol information;
the table item acquisition unit is used for acquiring a connection table item from the connection table;
and the information acquisition unit is used for calling the target function library and acquiring the transmission control protocol information from the kernel of the operating system by using the connection table item.
Optionally, the information obtaining unit includes:
the state determining subunit is used for acquiring connection state information corresponding to each connection table entry;
and the effective acquisition subunit is used for acquiring the transmission control protocol information from the kernel of the operating system by using the connection table entry of which the connection state information is in the effective connection state.
Optionally, the network information generating module 120 includes:
the identification acquisition unit is used for acquiring application identification information corresponding to each connection table item;
the history judging unit is used for judging whether corresponding history application information exists in the memory by utilizing each application identification information;
the determining unit is used for generating network information by utilizing the historical application information and the target transmission control protocol information corresponding to the connection table item if the network information exists;
correspondingly, the method also comprises the following steps:
and the storage module is used for determining the network information as new historical network information and storing the new historical network information into the memory.
Optionally, the network information generating module 120 includes:
the identification acquisition unit is used for acquiring application identification information corresponding to each connection table item;
the candidate screening unit is used for acquiring candidate application information, screening the candidate application information by using the application identification information, and determining target application information corresponding to each connection table item;
and the second generating unit is used for generating the network information by using the target application information and the target transmission control protocol information corresponding to the connection table item.
Optionally, the network flow information obtaining module 110 includes:
the monitoring unit is used for monitoring a network event generated by an operating system kernel if the operating system type is Windows and the network stream information is user datagram protocol information; wherein, a network event tracking mechanism of an operating system kernel is started;
and the filtering unit is used for filtering the network event based on the network protocol to obtain a target network event and obtaining network flow information by using the target network event.
In the following, the electronic device provided by the embodiment of the present application is introduced, and the electronic device described below and the network information acquisition method described above may be referred to correspondingly.
Referring to fig. 8, fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. Wherein the electronic device 100 may include a processor 101 and a memory 102, and may further include one or more of a multimedia component 103, an information input/information output (I/O) interface 104, and a communication component 105.
The processor 101 is configured to control overall operations of the electronic device 100, so as to complete all or part of the steps in the above network information acquisition method; the memory 102 is used to store various types of data to support operation at the electronic device 100, such data may include, for example, instructions for any application or method operating on the electronic device 100, as well as application-related data. The Memory 102 may be implemented by any type or combination of volatile and non-volatile Memory devices, such as one or more of Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic or optical disk.
The multimedia component 103 may include a screen and an audio component. Wherein the screen may be, for example, a touch screen and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may further be stored in the memory 102 or transmitted through the communication component 105. The audio assembly further comprises at least one speaker for outputting audio signals. The I/O interface 104 provides an interface between the processor 101 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 105 is used for wired or wireless communication between the electronic device 100 and other devices. Wireless Communication, such as Wi-Fi, bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of one or more of them, so that the corresponding Communication component 105 may include: Wi-Fi components, Bluetooth components, NFC components.
The electronic Device 100 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components, and is configured to perform the network information collecting method according to the above embodiments.
In the following, a computer-readable storage medium provided by an embodiment of the present application is introduced, and the computer-readable storage medium described below and the network information acquisition method described above may be referred to correspondingly.
The present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the network information acquisition method are implemented.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it should be further noted that, in this document, relationships such as first and second, etc., are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Also, the terms include, or any other variation is intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
The principle and the implementation of the present application are explained herein by applying specific examples, and the above description of the embodiments is only used to help understand the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (14)
1. A network information acquisition method is applied to an agent program, the agent program is deployed in an acquired device, and the method comprises the following steps:
obtaining network flow information through an operating system kernel of the collected equipment;
and generating network information corresponding to a plurality of applications in the acquired equipment by using the network flow information.
2. The method according to claim 1, wherein obtaining network flow information through an operating system kernel of the device to be acquired comprises:
acquiring a kernel version number of the kernel of the operating system;
if the kernel version number is higher than a preset version number, collecting the network flow information by using a target mounting program;
and if the kernel version number is not higher than the preset version number, acquiring the network flow information by utilizing the socket connection between the kernel and the operating system.
3. The method according to claim 2, wherein the collecting the network flow information by using the target mount program comprises:
reading a data storage object corresponding to the target mounting program to obtain the network flow information;
and the target mounting program is mounted at a mounting point corresponding to the target kernel function, and the network flow information is started and collected after the mounting is successful.
4. The method of claim 2, wherein the obtaining the network flow information by using a socket connection with the operating system kernel comprises:
sending a network query request to the system kernel through the socket connection;
and receiving response data sent by the system kernel, and analyzing the response data to obtain the network flow information.
5. The network information collection method of claim 4, further comprising:
and if the response data corresponding to the network query request is not received, accessing and reading a kernel network state directory corresponding to an operating system kernel to obtain the network flow information.
6. The method according to any one of claims 1 to 5, wherein the generating, by using the network flow information, network information corresponding to each of the plurality of applications in the device to be acquired comprises:
determining index node data corresponding to each network flow by using the network flow information;
matching the index node data with each socket file to determine a target socket file and target index node data which correspond to each other one by one;
and determining corresponding target application information based on the target socket file, and generating the network information by using the target application information and target network flow information corresponding to the target index node data.
7. The method according to claim 1, wherein if the network flow information is tcp information, obtaining the network flow information through an os kernel of the device to be acquired comprises:
calling a target function library to obtain a connection table from the kernel of the operating system;
obtaining a connection table item from the connection table;
and calling the target function library, and obtaining the transmission control protocol information from the kernel of the operating system by using the connection table entry.
8. The method of claim 7, wherein said obtaining the tcp information from the os kernel by using the connection table entry comprises:
acquiring connection state information corresponding to each connection table item;
and obtaining the transmission control protocol information from the kernel of the operating system by using the connection table entry of which the connection state information is in the effective connection state.
9. The method according to claim 7 or 8, wherein the generating network information corresponding to each of the plurality of applications in the device to be acquired by using the network flow information includes:
acquiring application identification information corresponding to each connection table item;
judging whether corresponding historical application information exists in the memory or not by utilizing the application identification information;
if the historical application information exists, the network information is generated by using the historical application information and the target transmission control protocol information corresponding to the connection table entry;
correspondingly, the method also comprises the following steps:
and determining the network information as new historical network information and storing the new historical network information in the memory.
10. The method according to claim 7 or 8, wherein the generating network information corresponding to each of the plurality of applications in the device to be acquired by using the network flow information includes:
acquiring application identification information corresponding to each connection table item;
acquiring candidate application information, screening the candidate application information by using the application identification information, and determining target application information corresponding to each connection table item;
and generating the network information by using the target application information and the target transmission control protocol information corresponding to the connection table entry.
11. The method according to claim 1, wherein if the network flow information is user datagram protocol information, obtaining the network flow information through an operating system kernel of the device to be acquired comprises:
monitoring a network event generated by the kernel of the operating system; wherein a network event tracking mechanism of the operating system kernel is started;
and filtering the network event based on a network protocol to obtain a target network event, and obtaining the network flow information by using the target network event.
12. A network information acquisition device is applied to an agent program, the agent program is deployed in an acquired device, and the network information acquisition device comprises:
the network flow information acquisition module is used for acquiring network flow information through an operating system kernel of the acquired equipment;
and the network information generating module is used for generating network information corresponding to the plurality of applications in the acquired equipment by utilizing the network flow information.
13. An electronic device comprising a memory and a processor, wherein:
the memory is used for storing a computer program;
the processor is configured to execute the computer program to implement the network information acquisition method according to any one of claims 1 to 11.
14. A computer-readable storage medium storing a computer program, wherein the computer program is configured to implement the network information collection method according to any one of claims 1 to 11 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210535738.2A CN115002186A (en) | 2022-05-17 | 2022-05-17 | Network information acquisition method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210535738.2A CN115002186A (en) | 2022-05-17 | 2022-05-17 | Network information acquisition method and device, electronic equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115002186A true CN115002186A (en) | 2022-09-02 |
Family
ID=83027719
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210535738.2A Pending CN115002186A (en) | 2022-05-17 | 2022-05-17 | Network information acquisition method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115002186A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116016702A (en) * | 2022-12-26 | 2023-04-25 | 浪潮云信息技术股份公司 | Application observable data acquisition processing method, device and medium |
| CN116419291A (en) * | 2023-06-09 | 2023-07-11 | 阿里巴巴（中国）有限公司 | Method, equipment and system for extracting runtime parameters |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140019609A1 (en) * | 2012-07-10 | 2014-01-16 | Nathaniel C. Williams | Methods and Computer Program Products for Analysis of Network Traffic by Port Level and/or Protocol Level Filtering in a Network Device |
| US20170094034A1 (en) * | 2015-09-30 | 2017-03-30 | International Business Machines Corporation | User datagram protocol (udp) application handling during live kernel update |
| CN110351275A (en) * | 2019-07-11 | 2019-10-18 | 北京脉冲星科技有限公司 | A kind of host port flux monitoring method, system, device and storage equipment |
| WO2020094035A1 (en) * | 2018-11-07 | 2020-05-14 | 上海连尚网络科技有限公司 | Wireless network connection method for terminal |
| CN111162973A (en) * | 2019-12-31 | 2020-05-15 | 奇安信科技集团股份有限公司 | Data flow acquisition method and device, electronic equipment and medium |
| KR20200069702A (en) * | 2018-12-07 | 2020-06-17 | 엘에스웨어(주) | System and method for collecting Tor network traffic |
| CN111800490A (en) * | 2020-06-23 | 2020-10-20 | 深信服科技股份有限公司 | Method and device for acquiring network behavior data and terminal equipment |
| CN112350859A (en) * | 2020-10-28 | 2021-02-09 | 武汉绿色网络信息服务有限责任公司 | Method, device, equipment and storage medium for managing network function entity |
| WO2021189257A1 (en) * | 2020-03-24 | 2021-09-30 | 深圳市欢太科技有限公司 | Malicious process detection method and apparatus, electronic device, and storage medium |
| CN113703813A (en) * | 2021-09-07 | 2021-11-26 | 北京天融信网络安全技术有限公司 | Kernel upgrading method, device, equipment and computer readable storage medium |
| CN114285619A (en) * | 2021-12-20 | 2022-04-05 | 北京安天网络安全技术有限公司 | Network information display method and device and electronic equipment |
| CN114285621A (en) * | 2021-12-20 | 2022-04-05 | 北京安天网络安全技术有限公司 | Network threat monitoring method and device and electronic equipment |
-
2022
- 2022-05-17 CN CN202210535738.2A patent/CN115002186A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140019609A1 (en) * | 2012-07-10 | 2014-01-16 | Nathaniel C. Williams | Methods and Computer Program Products for Analysis of Network Traffic by Port Level and/or Protocol Level Filtering in a Network Device |
| US20170094034A1 (en) * | 2015-09-30 | 2017-03-30 | International Business Machines Corporation | User datagram protocol (udp) application handling during live kernel update |
| WO2020094035A1 (en) * | 2018-11-07 | 2020-05-14 | 上海连尚网络科技有限公司 | Wireless network connection method for terminal |
| KR20200069702A (en) * | 2018-12-07 | 2020-06-17 | 엘에스웨어(주) | System and method for collecting Tor network traffic |
| CN110351275A (en) * | 2019-07-11 | 2019-10-18 | 北京脉冲星科技有限公司 | A kind of host port flux monitoring method, system, device and storage equipment |
| CN111162973A (en) * | 2019-12-31 | 2020-05-15 | 奇安信科技集团股份有限公司 | Data flow acquisition method and device, electronic equipment and medium |
| WO2021189257A1 (en) * | 2020-03-24 | 2021-09-30 | 深圳市欢太科技有限公司 | Malicious process detection method and apparatus, electronic device, and storage medium |
| CN111800490A (en) * | 2020-06-23 | 2020-10-20 | 深信服科技股份有限公司 | Method and device for acquiring network behavior data and terminal equipment |
| CN112350859A (en) * | 2020-10-28 | 2021-02-09 | 武汉绿色网络信息服务有限责任公司 | Method, device, equipment and storage medium for managing network function entity |
| CN113703813A (en) * | 2021-09-07 | 2021-11-26 | 北京天融信网络安全技术有限公司 | Kernel upgrading method, device, equipment and computer readable storage medium |
| CN114285619A (en) * | 2021-12-20 | 2022-04-05 | 北京安天网络安全技术有限公司 | Network information display method and device and electronic equipment |
| CN114285621A (en) * | 2021-12-20 | 2022-04-05 | 北京安天网络安全技术有限公司 | Network threat monitoring method and device and electronic equipment |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116016702A (en) * | 2022-12-26 | 2023-04-25 | 浪潮云信息技术股份公司 | Application observable data acquisition processing method, device and medium |
| CN116419291A (en) * | 2023-06-09 | 2023-07-11 | 阿里巴巴（中国）有限公司 | Method, equipment and system for extracting runtime parameters |
| CN116419291B (en) * | 2023-06-09 | 2023-10-31 | 阿里巴巴（中国）有限公司 | Method, equipment and system for extracting runtime parameters |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9578045B2 (en) | Method and apparatus for providing forensic visibility into systems and networks | |
| Kutare et al. | Monalytics: online monitoring and analytics for managing large scale data centers | |
| CN115002186A (en) | Network information acquisition method and device, electronic equipment and readable storage medium | |
| CA2503987C (en) | System and method for performance management in a multi-tier computing environment | |
| CA2753019C (en) | Monitoring of distributed applications | |
| CN100576819C (en) | Flow analysis method based on linux kernel | |
| US9112894B2 (en) | Real time distributed network monitoring and security monitoring platform (RTD-NMS) | |
| US20050049924A1 (en) | Techniques for use with application monitoring to obtain transaction data | |
| CN112256542B (en) | eBPF-based micro-service system performance detection method, device and system | |
| CN111258851B (en) | Cluster alarm method, device, setting and storage medium | |
| CN109271243B (en) | Cluster task management system | |
| CN112350854B (en) | Flow fault positioning method, device, equipment and storage medium | |
| WO2021169275A1 (en) | Sdn network device access method and apparatus, computer device, and storage medium | |
| EP2634699B1 (en) | Application monitoring | |
| CN116418700A (en) | Distributed data capturing method based on DPDK | |
| KR101916799B1 (en) | Apparatus And Method For Big Data Server Load Balancing Control | |
| CN115712646A (en) | Alarm strategy generation method, device and storage medium | |
| CN112969172A (en) | Communication flow control method based on cloud mobile phone | |
| CN112764990B (en) | Target process monitoring method and device and computer equipment | |
| CN113986653A (en) | Openstack load balancing data monitoring method, system, storage medium and equipment | |
| CN113656241B (en) | Container terminal full life cycle management and control system and method | |
| CN110932927B (en) | Service processing method, device, equipment and readable storage medium | |
| CN109635015B (en) | Determination method and device for attribute data using object and server | |
| CN117032989A (en) | Time delay perception scheduling algorithm | |
| Li et al. | Network Security Intrusion Detection and Mass Alarms Under Cluster Computing Platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |