CN115454630A

CN115454630A - Data auditing method, apparatus, system, device, medium and program product

Info

Publication number: CN115454630A
Application number: CN202211075580.1A
Authority: CN
Inventors: 郑天文; 王竟成; 李海龙; 黄梓锋
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2022-09-02
Filing date: 2022-09-02
Publication date: 2022-12-09

Abstract

The disclosure provides a data auditing method which can be applied to the technical field of computers. The data auditing method comprises the following steps: when the network connection state with external auditing equipment is ensured, acquiring operation is carried out on the data flow of an application program to obtain real-time flow data; and executing a forwarding operation on the real-time flow data, and forwarding the real-time flow data to an external auditing device so that the external auditing device audits the real-time flow data, wherein the acquisition operation and the forwarding operation are realized based on a preconfigured flow guidance tool, and the preconfigured flow guidance tool is realized based on a protocol stack of a kernel bypass. The present disclosure also provides a data auditing apparatus, system, device, medium and program product.

Description

Data auditing method, apparatus, system, device, medium, and program product

Technical Field

The present disclosure relates to the field of computer technology, particularly to the field of data security technology, and more particularly to a data auditing method, apparatus, system, device, medium, and program product.

Background

In the field of data security, "auditing" is to perform auditing analysis on the flow of an application program, and the auditing content generally comprises user operation, specific program response content and the like, so that historical backtracking of interaction between the application program and a client of a terminal in the whole server can be realized.

For this reason, in the prior art, auditing of traffic is usually achieved by using a technology based on a Data Plane Development Kit (DPDK).

However, in the prior art, a plurality of schemes for auditing the flow based on the DPDK technology exist, but the schemes have the problems of poor auditing timeliness, occupation of server memory resources, interception of the flow by a Linux/Unix kernel network protocol stack and the like.

Disclosure of Invention

In view of the foregoing, the present disclosure provides data auditing methods, apparatus, systems, devices, media, and program products that improve auditing efficiency and reduce server memory resource usage.

According to a first aspect of the present disclosure, there is provided a data auditing method, the method being applied to a server, the method comprising: when the network connection state with external auditing equipment is ensured, acquiring operation is carried out on the data flow of an application program to obtain real-time flow data; and executing a forwarding operation on the real-time flow data, and forwarding the real-time flow data to an external auditing device so that the external auditing device audits the real-time flow data, wherein the acquisition operation and the forwarding operation are realized based on a preconfigured flow guidance tool, and the preconfigured flow guidance tool is realized based on a protocol stack of a kernel bypass.

According to the embodiment of the disclosure, the method for ensuring the network connection state with the external auditing equipment comprises the following steps: and judging whether the first network card identification number is successfully matched with the second network card identification number, wherein the first network card identification number and the second network card identification number are the network card identification numbers of the same external auditing equipment, the first network card identification number is a locally pre-stored network card identification number, and the second network card identification number is a locally actually connected network card identification number.

According to an embodiment of the present disclosure, the acquiring data traffic of the application program to obtain real-time traffic data includes: when detecting that a first flow is received, acquiring the first flow to obtain the real-time flow data, wherein the first flow is the flow sent to the application program by an external client; and when receiving a second flow is detected, acquiring the second flow to obtain the real-time flow data, wherein the second flow is the flow sent to an external client by the application program.

According to an embodiment of the present disclosure, the performing a forwarding operation on the real-time traffic data and forwarding the real-time traffic data to an external auditing device includes: distributing port information and a network card structure; and forwarding the real-time flow data to the external auditing equipment through the port information based on the network card structural body.

According to a second aspect of the present disclosure, there is provided another data auditing method, applied to an external auditing device, the method comprising: receiving real-time flow data, wherein the real-time flow data is obtained by executing acquisition operation and forwarding operation based on a pre-configured flow guiding tool in a server, and the pre-configured flow guiding tool realizes flow guiding based on a protocol stack of a kernel bypass; extracting flow characteristics in the real-time flow data; classifying and storing the real-time flow data through the flow characteristics to obtain a temporary file; and analyzing the temporary file to obtain analysis content.

According to the embodiment of the present disclosure, the traffic characteristics include quintuple information and time information, the quintuple information includes protocol information, and the classifying and storing the real-time traffic data according to the traffic characteristics to obtain a temporary file includes: classifying the real-time traffic data based on the protocol information; and storing the classified real-time flow data according to the time sequence based on the time information.

According to an embodiment of the present disclosure, the parsing the temporary file to obtain parsed contents includes: merging the plurality of temporary files based on the same protocol information to obtain a first temporary file; and analyzing the first temporary file to obtain the analysis content.

According to a third aspect of the present disclosure, there is provided a data auditing method, comprising: when the server ensures that the server is in a network connection state with external auditing equipment, acquiring data traffic of an application program to obtain real-time traffic data; the server executes forwarding operation on the real-time flow data, and forwards the real-time flow data to external auditing equipment so that the external auditing equipment audits the real-time flow data, wherein the forwarding operation is performed according to a pre-bound network card identification number; extracting, by the external auditing device, traffic features in the real-time traffic data from the server; classifying and storing the real-time flow data by the external auditing equipment through the flow characteristics to obtain a temporary file; and analyzing the temporary file by the external auditing equipment to obtain analyzed content, wherein the acquisition operation and the forwarding operation are realized based on a pre-configured drainage tool, and the pre-configured drainage tool realizes drainage based on a protocol stack of a kernel bypass.

According to a fourth aspect of the present disclosure, there is provided a data auditing apparatus, the apparatus being applied to a server, the apparatus including: the system comprises an acquisition module, a monitoring module and a control module, wherein the acquisition module is used for executing acquisition operation on data flow of an application program when ensuring that the external auditing equipment is in a network connection state, so as to obtain real-time flow data; and the forwarding module is used for executing forwarding operation on the real-time flow data and forwarding the real-time flow data to external auditing equipment so as to enable the external auditing equipment to audit the real-time flow data, wherein the acquisition operation and the forwarding operation are realized based on a pre-configured drainage tool, and the pre-configured drainage tool realizes drainage based on a protocol stack of a kernel bypass.

According to the embodiment of the disclosure, the acquisition module is further configured to determine whether a first network card identification number is successfully matched with a second network card identification number, where the first network card identification number and the second network card identification number are network card identification numbers for the same external audit device, the first network card identification number is a locally pre-stored network card identification number, and the second network card identification number is a locally actually connected network card identification number.

According to the embodiment of the disclosure, the forwarding module is further configured to acquire a first traffic to obtain the real-time traffic data when detecting that the first traffic is received, where the first traffic is a traffic sent to the application program by an external client; and when detecting that a second flow is received, acquiring the second flow to obtain the real-time flow data, wherein the second flow is the flow sent to an external client by the application program.

According to the embodiment of the disclosure, the forwarding module is further configured to distribute port information and a network card structure; and forwarding the real-time flow data to the external auditing equipment through the port information based on the network card structural body.

According to a fifth aspect of the present disclosure, there is provided a data auditing apparatus, the apparatus being applied to an external auditing device, the apparatus comprising: the real-time flow data receiving module is used for receiving real-time flow data, the real-time flow data is obtained by executing acquisition operation and forwarding operation based on a pre-configured drainage tool in the server, and the pre-configured drainage tool realizes drainage based on a protocol stack of a kernel bypass; the flow characteristic extraction module is used for extracting flow characteristics in the real-time flow data; the classified storage module is used for classifying and storing the real-time flow data through the flow characteristics to obtain a temporary file; and the analysis module is used for analyzing the temporary file to obtain analysis content.

According to an embodiment of the present disclosure, the traffic characteristics include quintuple information and time information, the quintuple information includes protocol information, and the classification storage module is further configured to classify the real-time traffic data based on the protocol information; and storing the classified real-time flow data according to the time sequence based on the time information.

According to an embodiment of the present disclosure, the parsing module is further configured to merge the multiple temporary files based on the same protocol information to obtain a first temporary file; and analyzing the first temporary file to obtain the analysis content.

According to a sixth aspect of the present disclosure, a data auditing system is provided, where the system includes a server and an external auditing device, where the server is configured to perform an acquisition operation on data traffic of an application program to obtain real-time traffic data when ensuring that the server is in a network connection state with the external auditing device; the server is further configured to perform forwarding operation on the real-time traffic data, and forward the real-time traffic data to external auditing equipment so that the external auditing equipment audits the real-time traffic data, wherein the forwarding operation is performed according to a pre-bound network card identification number; the external auditing device is configured to extract traffic characteristics from the real-time traffic data from the server; the external auditing equipment is also configured to classify and store the real-time flow data through the flow characteristics to obtain a temporary file; and the external auditing equipment is also configured to analyze the temporary file to obtain analysis content, wherein the acquisition operation and the forwarding operation are realized based on a pre-configured drainage tool, and the pre-configured drainage tool realizes drainage based on a protocol stack of a kernel bypass.

In a seventh aspect of the present disclosure, there is provided an electronic device comprising: one or more processors; memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the above-described data auditing method.

In an eighth aspect of the present disclosure, there is also provided a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the above-mentioned data auditing method.

In a ninth aspect of the present disclosure, there is also provided a computer program product comprising a computer program which, when executed by a processor, implements the above-mentioned data auditing method.

In the embodiment of the disclosure, the flow is led to the external auditing equipment for auditing, so that the occupation of processing resources of the server can be avoided, and the normal operation of the server is ensured. In the initialization process, the smoothness of a link is ensured, and the real-time performance of flow data acquisition can be ensured. The flow guiding tool of the protocol stack based on the kernel bypass can bypass the Linux/Unix kernel protocol stack, reduce the security sensitivity of the Linux/Unix system and avoid the false identification and false alarm of the flow guiding process. In conclusion, the influence on the native service processing of the server can be reduced to the maximum extent.

Drawings

The foregoing and other objects, features and advantages of the disclosure will be apparent from the following description of embodiments of the disclosure, which proceeds with reference to the accompanying drawings, in which:

FIG. 1A schematically illustrates an application scenario diagram of a data auditing method according to an embodiment of the present disclosure;

FIG. 1B schematically illustrates a data interaction diagram of a data auditing system, according to an embodiment of the present disclosure;

FIG. 2 schematically illustrates a flow diagram of a data auditing method according to an embodiment of the present disclosure;

FIG. 3 schematically illustrates a forwarding function logic diagram according to an embodiment of the disclosure;

FIG. 4 schematically illustrates a flow diagram of a data auditing method according to an embodiment of the present disclosure;

FIG. 5 is a block diagram schematically illustrating a structure of a data auditing apparatus according to an embodiment of the present disclosure;

FIG. 6 is a block diagram schematically illustrating the structure of a data auditing apparatus according to an embodiment of the present disclosure; and

FIG. 7 schematically illustrates a block diagram of an electronic device suitable for implementing a data auditing method according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.

Where a convention analogous to "at least one of A, B, and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, and C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.).

Before embodiments of the present disclosure are disclosed in detail, a description of key technical terms used in the embodiments of the present disclosure is needed, as follows:

data plane development kit: (Date Plane Development Kit, abbreviated as DPDK) is an accelerated software library for packet data processing. The DPDK is an open source project of the Linux foundation, and the main purpose of developing the DPDK is to provide a simple and complete framework for rapid packet processing in data platform applications.

Peripheral component interconnect standard: (Peripheral Component Interconnect, abbreviated PCI), a standard bus. In the embodiments of the present disclosure, the hardware network card may be referred to as a PCI network card, i.e., a PCI slot network card, which is a network component operating in a data link layer and is an interface for connecting a computer and a transmission medium in a local area network.

RFC documents: (Request For Documents, abbreviated as RFCs) is currently the most comprehensive and authoritative document maintained officially For a technology and has been developed into the official records of Internet specifications, standards, procedures and events.

Sysfs: is a virtual file system provided in Linux. This file system can not only export information of devices (devices) and drivers (drivers) from the kernel to the user space, but also can be used to make settings for the devices and drivers.

BPF compiler set: (BPF Compiler Collection, BCC for short) is a kernel tracing and debugging tool based on Linux eBPF characteristics.

Kprebe: dynamically attaching to kernel call point functions, such as checking parameters before kernel exec system calls, cut-in can be done with the BPF program set SEC ("kprobe/sys _ exec") header.

And (4) Hook: a special message processing mechanism can monitor various event messages in a system or process, intercept messages sent to a target window and process the messages, so that the transmission of the events is intercepted and monitored before the events are transmitted to a terminal.

A large page of memory: in the Linux system, the physical memory is managed with a page as a basic unit. By default, the page size is 4KB. Then 1MB of memory can be divided into 256 pages; 1GB of memory can be divided into 256000 pages. The CPU has a built-in Memory Management Unit (MMU) for storing a list of these pages (page table), each page having a corresponding entry address. Pages of 4KB in size are reasonable when the "paging mechanism" is proposed, since the memory size at that time is less than a few tens of megabytes. However, the physical memory capacity of current computers has increased to the level of GB and even TB, which can result in a waste of memory if the operating system still uses 4KB size as the basic unit of a page, which results in insufficient page space in the MMU in the CPU to store all address entries. Meanwhile, when an application program with a large memory requirement is operated on a Linux operating system, a default 4KB page is adopted, so that more TLB Miss and missing page terminals are generated, and the performance of the application program is greatly influenced. When the operating system takes 2MB or more as a paging unit, the number of TLB Miss and page fault interrupts can be greatly reduced, and the performance of the application program can be obviously improved. In order to solve the above problems, since Linux Kernel 2.6, the concept of Huge Pages has been introduced, which aims to adapt to larger and larger memory space by using large-page memory to replace the conventional 4KB memory page. Huge Pages have two specifications, 2MB and 1GB, with a 2MB size (default) suitable for GB-class memory and a 1GB size suitable for TB-class memory.

In the prior art, the DPDK-based packet capturing or auditing method generally includes the following three methods:

the first scheme is as follows: by modifying the DPDK bottom-layer source codes, the network data packets can be backed up in a large-page memory, and the reserved network data packets are released regularly by a specific method, so that packet capture or audit is realized.

Scheme II: and (3) adopting a remote injection technology to carry out Hook on a packet receiving/sending function of the DPDK, and releasing a network data packet by injecting a specific code so as to realize packet capturing or auditing.

And a third scheme is as follows: the DPDK data packet capture development framework, namely library _ pdump, is adopted, and a DPDK _ pdump tool is provided, and the tool can be used for capturing the specified interface and queue DPDK data packets which are taken over by a DPDK application program.

For the above first, second and third schemes, certain disadvantages exist, which are as follows:

aiming at the first scheme: the method needs to consume an extra large page of memory for backing up the network data packet, and needs to release the backed-up network data packet at regular time, so that more memory is occupied, and the packet capturing timeliness is poor.

Aiming at the second scheme: remote injection techniques are required, but for a higher security operating system or in an operating system with antivirus software installed, the collected data packets will be intercepted.

Aiming at the third scheme: network data packets cannot be directionally mirrored, and auditing requirements cannot be met.

In order to solve technical problems in the prior art, an embodiment of the present disclosure provides a data auditing method, where the method includes: when the network connection state with external auditing equipment is ensured, acquiring operation is carried out on the data traffic of an application program to obtain real-time traffic data; and executing a forwarding operation on the real-time flow data, and forwarding the real-time flow data to an external auditing device so that the external auditing device audits the real-time flow data, wherein the acquisition operation and the forwarding operation are realized based on a preconfigured flow guidance tool, and the preconfigured flow guidance tool is realized based on a protocol stack of a kernel bypass.

Fig. 1A schematically illustrates an application scenario of a data auditing method according to an embodiment of the present disclosure.

As shown in fig. 1A, the application scenario 100 according to this embodiment may include

terminal devices

101, 102, 103, a network 104, a server 105, and an external audit device 106. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

A user may use

terminal devices

101, 102, 103 to interact with a server 105 over a network 104 to receive or send messages or the like. The

terminal devices

101, 102, 103 may have installed thereon various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).

The

terminal devices

101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.

The server 105 may be a server providing various services, such as a background management server (for example only) providing support for websites browsed by users using the

terminal devices

101, 102, 103. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device.

External auditing device 106 may be a hardware device that is directly connected to server 105, with external auditing device 106 being used to audit traffic in server 105.

It should be noted that the data auditing method provided by the embodiment of the present disclosure can be generally executed by the server 105 and/or the external auditing device 106 on a case-by-case basis. Accordingly, the data auditing apparatus provided by the embodiments of the present disclosure may be provided in the server 105 and/or the external auditing device 106 on a case-by-case basis in general. The data auditing method provided by the embodiment of the present disclosure may also be performed by a server or server cluster that is different from the server 105 and is capable of communicating with the

terminal devices

101, 102, 103 and/or the server 105. Correspondingly, the data auditing device provided by the embodiment of the present disclosure may also be disposed in a server or a server cluster that is different from the server 105 and is capable of communicating with the

terminal devices

101, 102, 103 and/or the server 105, or the data auditing device provided by the embodiment of the present disclosure may also be disposed in a server or a server cluster that is different from the external auditing device 106 and is capable of communicating with the server 105.

It should be understood that the number of terminal devices, networks, and servers in FIG. 1A are merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

FIG. 1B schematically shows a data interaction diagram of a data auditing system according to an embodiment of the present disclosure.

As shown in fig. 1B, the data auditing system includes: a server 105 and an external auditing device 106,

wherein,

the server 105 is configured to perform acquisition operation on the data traffic of the application program to obtain real-time traffic data when the server is ensured to be in a network connection state with external auditing equipment;

the server 105 is further configured to perform a forwarding operation on the real-time traffic data, and forward the real-time traffic data to an external auditing device, so that the external auditing device audits the real-time traffic data, where the forwarding operation is performed according to a pre-bound network card identification number;

the external auditing device 106 configured to extract traffic characteristics in the real-time traffic data from the server;

the external auditing device 106 is further configured to perform classified storage on the real-time traffic data through the traffic characteristics to obtain a temporary file; and

the external auditing device 106 is further configured to parse the temporary file to obtain parsed contents,

wherein the collecting operation and the forwarding operation are implemented based on a preconfigured drainage tool that is drained based on a protocol stack of a kernel bypass.

As shown in fig. 1B, at least network card 1 and network card 2 are included on server 105, and at least network card 3 is included on external audit device 106.

The network card 1 is used for interacting with a client application in the terminal 101/102/103, and receiving an incoming flow of an application program or sending an outgoing flow of the application program. For example, first, the network card 1 receives the traffic in the terminals 101/102/103, then the network card transmits the traffic to the traffic directing tool, and finally the traffic directing tool sends the traffic to the application program as the incoming traffic of the application program. For another example, the traffic guidance tool receives the outgoing traffic of the application program, then the traffic guidance tool sends the outgoing traffic to the network card 1, and finally the network card 1 sends the traffic to the corresponding interactive terminal 101/102/103.

The network card 2 forwards the acquired outgoing flow and incoming flow of the application program to the pre-bound network card 3. For example, the traffic guidance tool first collects the traffic of the application program, then the auditing device 600 forwards the traffic of the application program to the network card 2, and finally the network card 2 sends the traffic of the application program to the network card 3. For another example, the traffic guidance tool first collects the traffic of the application program, then the auditing device 600 forwards the traffic of the application program to the network card 2, and finally the network card 2 sends the traffic of the application program to the network card 3. Of course, the network card 3 may also be called an audit network card.

The application, the drainage tool, and the data auditing apparatus 700 in fig. 1B are computer programs, and are indicated by dashed boxes in fig. 1B.

The data auditing method of the disclosed embodiment will be described in detail below with fig. 2-4 based on the scenarios described in fig. 1A and 1B.

FIG. 2 schematically shows a flow diagram of a data auditing method according to an embodiment of the present disclosure.

Fig. 3 schematically illustrates a forwarding function logic diagram according to an embodiment of the disclosure.

As shown in fig. 2, the data auditing method of this embodiment includes operations S210 to S220, which may be performed by the server 105 described above.

In operation S210, when it is ensured that the network connection state with the external auditing device is maintained, a collecting operation is performed on the data traffic of the application program, so as to obtain real-time traffic data.

Specifically, the application program may be an application program that performs external communication based on the preconfigured drainage tool protocol stack. For example, as shown in fig. 1B, the application program may be in communication with the terminal 101/102/103 through the protocol stack of the preconfigured drainage tool.

Specifically, a PCI number may be used as the network card identification number, and then, the network card identification number (i.e., the PCI number) may be self-discovered through some related functions when the external auditing device is determined to be in the network connection state. For example, as shown in fig. 1B, when the network card 3 is normally bound to the server 105 and the connection is normal, the server 105 may check the PCI number of the network card 3 (i.e., the second network card number), and determine that the external auditing device is in the network connection state through the pre-stored PCI number of the local network card 3 (i.e., the first network card number).

For example, as shown in fig. 1B, a running environment is built by using modified DPDK source codes, and two network cards are required, one is a network card 1 for maintaining an original service, and the other is a network card 2 for forwarding traffic, where the network card 2 is connected to a network card 3 of an auditing device. When devbind the network card by executing devbind, the kernel interacts with the kernel through sysfs, and the kernel binds the network card by using a specified driver, so that the binding of the network card 2 and the network card 3 is realized. And after the network card is bound, executing a devbind. Py-s command, checking the PCI number of the network card 3, and informing a program of forwarding the flow to which network card is needed when the DPDK application program is started subsequently.

In the DPDK, the process of ensuring that the server and the external auditing device are in the network connection state occurs at the initialization stage of the DPDK. The implementation logic is as follows: adding processing on a parameter dump-enable in a resource allocation initialization function of a DPDK source code, and storing a PCI number of an auditing network card into a global variable dump-PCI-id in a character string form; the resource allocation initialization function calls a network card discovery function to discover all PCI network card information, and an audit network card also comprises the PCI network card information; the network card discovery function searches information of network card equipment according to the PCI number, such as vendor, device, subsystem _ vendor, subsystem _ device, class and the like; if the searched PCI number is consistent with the dump _ PCI _ id, setting the is _ dump to true when the PCI structural body is distributed; and finally calling a network card structure distribution function by the network card discovery function to store personal information formats for all network card distribution structures, wherein the port id is a member variable in the structure, and when the PCI device is _ dump is found to be true, assigning the allocated port id to the global variable dump _ port.

According to an embodiment of the present disclosure, the acquiring data traffic of the application program to obtain real-time traffic data includes: when detecting that a first flow is received, acquiring the first flow to obtain the real-time flow data, wherein the first flow is the flow sent to the application program by an external client; and when detecting that a second flow is received, acquiring the second flow to obtain the real-time flow data, wherein the second flow is the flow sent to an external client by the application program.

Specifically, for an application program, the first flow rate is an input flow rate of the application program, and the second flow rate is an output flow rate of the application program. The above collecting operation can be regarded as that when the application program normally communicates with the external client, the flow of the original normal communication is also forwarded to the external auditing equipment.

As shown in fig. 3, in the case of port binding, the traffic packet receiving function transmits the traffic packet to the application program, and simultaneously, the traffic packet is forwarded from the network card 2 to the external auditing device; under the condition of port binding, the flow packet sending function sends the flow packet to the network card 1, and simultaneously, the original flow packet is also forwarded to the auditing equipment from the auditing network card. For example, the logic implemented is as follows: in a receiving function rte _ eth _ rx _ burst, if the dump _ portlet is not-1, calling a rte _ eth _ tx _ burst function to send out a received data packet through the dump _ portlet; in the sending function rte _ eth _ tx _ burst, if the dump _ portlet is not-1, calling the rte _ eth _ tx _ burst function to send the received data packet out through the dump _ portlet.

As shown in fig. 1B, it can be seen that the communication process of the server 105 and the terminals 101/102/103 for entering and exiting normal traffic and the flow guiding process of the server 105 and the network card 3 are executed in parallel, in real time, and do not interfere with each other.

In operation S220, a forwarding operation is performed on the real-time traffic data, and the real-time traffic data is forwarded to an external auditing device, so that the external auditing device audits the real-time traffic data, where the collecting operation and the forwarding operation are implemented based on a preconfigured flow guidance tool, and the preconfigured flow guidance tool is implemented based on a protocol stack of a kernel bypass.

Wherein the protocol stack of the kernel bypass may be capable of bypassing a Linux/Unix kernel protocol stack.

According to an embodiment of the present disclosure, the performing a forwarding operation on the real-time traffic data, and forwarding the real-time traffic data to an external auditing device includes: distributing port information and a network card structure; and forwarding the real-time flow data to the external auditing equipment through the port information based on the network card structural body.

Specifically, the network card structure may be regarded as a data format for storing real-time traffic information, and the port information is a real-time traffic information export port. When a DPDK application program is started, bring-dump-enable = PCI number of the network card 3, when the modified code detects that the program has-dump-enable, the modified code executes relevant code logic, and when real-time data traffic exists, the modified code is forwarded to the network card 3.

It is understood that the preconfigured drainage tool described above includes, but is not limited to, DPDK, and any tool that can bypass the Linux/Unix kernel protocol stack and perform drainage is included in the preconfigured drainage tool in this disclosure.

In the embodiment of the disclosure, the flow is led to the external auditing equipment for auditing, so that the occupation of processing resources of the server can be avoided, and the normal operation of the server is ensured. In the initialization process, the smooth link is ensured, and the real-time performance of flow data acquisition can be ensured. The flow guiding tool of the protocol stack based on the kernel bypass can bypass the Linux/Unix kernel protocol stack, reduce the security sensitivity of the Linux/Unix system and avoid the false identification and false alarm of the flow guiding process. In conclusion, the influence on the native service processing of the server can be reduced to the maximum extent.

FIG. 4 schematically shows a flow diagram of a data auditing method according to an embodiment of the present disclosure.

As shown in fig. 4, the data auditing method of this embodiment includes operations S410 to S440, and the data auditing method may be applied to the external auditing apparatus 106.

In operation S410, real-time traffic data is received, where the real-time traffic data is obtained by performing an acquisition operation and a forwarding operation based on a pre-configured flow guidance tool in a server, and the pre-configured flow guidance tool implements flow guidance based on a protocol stack of a kernel bypass.

In operation S420, a traffic feature in the real-time traffic data is extracted.

In operation S430, the real-time traffic data is classified and stored according to the traffic characteristics, so as to obtain a temporary file.

According to an embodiment of the present disclosure, the traffic characteristics include quintuple information and time information, the quintuple information includes protocol information, and the classifying and storing the real-time traffic data according to the traffic characteristics to obtain a temporary file includes: classifying the real-time traffic data based on the protocol information; and storing the classified real-time flow data according to the time sequence based on the time information.

Specifically, the five-tuple information includes a source address IP, a destination address IP, a source port number, a destination port number, and a protocol.

In operation S440, the temporary file is parsed to obtain parsing contents.

According to an embodiment of the present disclosure, the parsing the temporary file to obtain parsed contents includes: when the temporary files reach a preset threshold value, combining a plurality of temporary files based on the same protocol information to obtain a first temporary file; and analyzing the first temporary file to obtain the analysis content.

Specifically, the preset threshold may be set based on a specific development or operation and maintenance scenario, and is not described herein again.

For example, the execution logic in the external auditing device is as follows: in an audit equipment system, detecting a function related to network flow receiving in a kernel by using a Kprobe based on a BCC technology, and intercepting flow entering external audit equipment; state information related to network traffic exists in the detected function, the information of the function context can be obtained through a BCC technology, and then in the process of detecting a network traffic receiving function by using a Kbarobe, corresponding information including information such as a source address IP, a target address IP, a source port number, a target port number and a protocol is obtained by using a variable of the network traffic receiving function; classifying according to a flow protocol, and storing relevant information of flow and message content in a temporary file according to a time sequence for subsequent use; after the flow is stored for a certain amount, merging packets and analyzing contents, such as an http protocol, according to different protocol rules for different protocols, merging according to the RFC standard, and further acquiring the request or reply contents, including the http link, the protocol version, the request parameters and the like, and storing the contents in a database respectively for different protocol stacks, so as to realize playback of the whole operation process.

In the embodiment of the disclosure, the flow is dragged to the external auditing equipment for auditing, so that the processing resource of the server can be avoided from being occupied, and the normal operation of the server is ensured. In the initialization process, the smooth link is ensured, and the real-time performance of flow data acquisition can be ensured. The flow guiding tool of the protocol stack based on the kernel bypass can bypass the Linux/Unix kernel protocol stack, reduce the security sensitivity of the Linux/Unix system and avoid the false identification and false alarm of the flow guiding process. In conclusion, the influence on the native service processing of the server can be reduced to the maximum extent.

Based on the data auditing method, the disclosure also provides a data auditing device. The data auditing device will be described in detail below with reference to fig. 5 and 6.

Fig. 5 is a block diagram schematically illustrating a structure of a data auditing apparatus according to an embodiment of the present disclosure.

As shown in fig. 5, the data auditing apparatus 500 of this embodiment includes an acquisition module 510 and a forwarding module 520.

The collecting module 510 is configured to perform a collecting operation on the data traffic of the application program to obtain real-time traffic data when the external auditing device is ensured to be in a network connection state. In an embodiment, the acquisition module 510 may be configured to perform the operation S210 described above, which is not described herein again.

The forwarding module 520 is configured to perform a forwarding operation on the real-time traffic data, and forward the real-time traffic data to an external auditing device, so that the external auditing device audits the real-time traffic data, where the collecting operation and the forwarding operation are implemented based on a preconfigured flow guidance tool, and the preconfigured flow guidance tool is implemented based on a protocol stack of a kernel bypass. In an embodiment, the forwarding module 820 may be configured to perform the operation S220 described above, which is not described herein again.

According to the embodiment of the present disclosure, the acquisition module 510 is further configured to determine whether a first network card identification number is successfully matched with a second network card identification number, where the first network card identification number and the second network card identification number are network card identification numbers for the same external auditing device, the first network card identification number is a locally pre-stored network card identification number, and the second network card identification number is a locally actually connected network card identification number.

According to an embodiment of the present disclosure, the forwarding module 520 is further configured to, when it is detected that a first traffic is received, acquire the first traffic to obtain the real-time traffic data, where the first traffic is a traffic sent to the application program by an external client; and when detecting that a second flow is received, acquiring the second flow to obtain the real-time flow data, wherein the second flow is the flow sent to an external client by the application program.

According to an embodiment of the present disclosure, the forwarding module 520 is further configured to allocate port information and a network card structure; and forwarding the real-time flow data to the external auditing equipment through the port information based on the network card structural body.

According to an embodiment of the present disclosure, any multiple modules of the collection module 510 and the forwarding module 520 may be combined into one module to be implemented, or any one of the modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 510 and the forwarding module 520 may be implemented at least partially as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by any other reasonable manner of integrating or packaging a circuit, such as hardware or firmware, or implemented by any one of three implementations of software, hardware, and firmware, or any suitable combination of any of them. Alternatively, at least one of the acquisition module 510 and the forwarding module 520 may be at least partially implemented as a computer program module, which when executed may perform a corresponding function.

Fig. 6 is a block diagram schematically illustrating the structure of a data auditing apparatus according to an embodiment of the present disclosure.

As shown in fig. 6, the data auditing apparatus 600 of this embodiment includes a real-time traffic data receiving module 610, a traffic feature extraction module 620, a classification storage module 630 and a parsing module 640.

The real-time traffic data receiving module 610 is configured to receive real-time traffic data, where the real-time traffic data is obtained by performing an acquisition operation and a forwarding operation based on a pre-configured drainage tool in a server, and the pre-configured drainage tool is used for realizing drainage based on a protocol stack of a kernel bypass. In an embodiment, the real-time traffic data receiving module 610 may be configured to perform the operation S410 described above, which is not described herein again.

The flow characteristic extraction module 620 is configured to extract a flow characteristic from the real-time flow data. In an embodiment, the flow characteristic extraction module 620 may be configured to perform the operation S420 described above, which is not described herein again.

The classification storage module 630 is configured to perform classification storage on the real-time traffic data according to the traffic characteristics, so as to obtain a temporary file. In an embodiment, the classification storage module 630 may be configured to perform the operation S430 described above, which is not described herein again.

The parsing module 640 is configured to parse the temporary file to obtain parsed contents. In an embodiment, the parsing module 640 may be configured to perform the operation S440 described above, which is not described herein again.

According to an embodiment of the present disclosure, the traffic characteristics include five-tuple information and time information, the five-tuple information includes protocol information, and the classification storage module 630 is further configured to classify the real-time traffic data based on the protocol information; and storing the classified real-time flow data according to the time sequence based on the time information.

According to an embodiment of the present disclosure, the parsing module 640 is further configured to merge the multiple temporary files based on the same protocol information to obtain a first temporary file; and analyzing the first temporary file to obtain the analysis content.

According to the embodiment of the present disclosure, any plurality of the real-time traffic data receiving module 610, the traffic feature extracting module 620, the classification storing module 630, and the parsing module 640 may be combined and implemented in one module, or any one of the modules may be split into a plurality of modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the real-time flow data receiving module 610, the flow feature extraction module 620, the classification storage module 630, and the parsing module 640 may be at least partially implemented as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or implemented by any one of three implementations of software, hardware, and firmware, or an appropriate combination of any several of them. Alternatively, at least one of the real-time traffic data receiving module 610, the traffic feature extraction module 620, the classification storage module 630 and the parsing module 640 may be at least partially implemented as a computer program module, which, when executed, may perform a corresponding function.

FIG. 7 schematically illustrates a block diagram of an electronic device suitable for implementing a data auditing method, according to an embodiment of the present disclosure.

As shown in fig. 7, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. The processor 701 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 701 may also include on-board memory for caching purposes. The processor 701 may comprise a single processing unit or a plurality of processing units for performing the different actions of the method flows according to embodiments of the present disclosure.

In the RAM 703, various programs and data necessary for the operation of the electronic apparatus 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other by a bus 704. The processor 701 performs various operations of the method flows according to the embodiments of the present disclosure by executing programs in the ROM 702 and/or the RAM 703. It is noted that the programs may also be stored in one or more memories other than the ROM 702 and RAM 703. The processor 701 may also perform various operations of method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.

Electronic device 700 may also include input/output (I/O) interface 705, which input/output (I/O) interface 705 also connects to bus 704, according to an embodiment of the present disclosure. The electronic device 700 may also include one or more of the following components connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.

The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM 702 and/or the RAM 703 and/or one or more memories other than the ROM 702 and the RAM 703 described above.

Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the method illustrated by the flow chart. When the computer program product runs in a computer system, the program code is used for causing the computer system to realize the method provided by the embodiment of the disclosure.

The computer program performs the above-described functions defined in the system/apparatus of the embodiments of the present disclosure when executed by the processor 701. The systems, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.

In one embodiment, the computer program may be hosted on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted in the form of a signal on a network medium, distributed, downloaded and installed via the communication section 709, and/or installed from the removable medium 711. The computer program containing program code may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711. The computer program, when executed by the processor 701, performs the above-described functions defined in the system of the embodiment of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.

In accordance with embodiments of the present disclosure, program code for executing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, these computer programs may be implemented using high level procedural and/or object oriented programming languages, and/or assembly/machine languages. The programming language includes, but is not limited to, programming languages such as Java, C + +, python, the "C" language, or the like. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.

The embodiments of the present disclosure are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the disclosure, and these alternatives and modifications are intended to fall within the scope of the disclosure.

Claims

1. A data auditing method, which is applied to a server, comprises the following steps:

when the network connection state with external auditing equipment is ensured, acquiring operation is carried out on the data traffic of an application program to obtain real-time traffic data; and

performing a forwarding operation on the real-time traffic data, forwarding the real-time traffic data to an external auditing device, so that the external auditing device audits the real-time traffic data,

wherein the collecting operation and the forwarding operation are implemented based on a preconfigured drainage tool, and the preconfigured drainage tool is drainage based on a protocol stack of a kernel bypass.

2. The method of claim 1, wherein the method of ensuring network connectivity with an external audit device comprises:

and judging whether the first network card identification number is successfully matched with the second network card identification number, wherein the first network card identification number and the second network card identification number are the network card identification numbers of the same external auditing equipment, the first network card identification number is a locally pre-stored network card identification number, and the second network card identification number is a locally actually connected network card identification number.

3. The method of claim 1 or 2, wherein the performing an acquisition operation on the data traffic of the application to obtain real-time traffic data comprises:

when detecting that a first flow is received, acquiring the first flow to obtain the real-time flow data, wherein the first flow is the flow sent to the application program by an external client; and

and when detecting that a second flow is received, acquiring the second flow to obtain the real-time flow data, wherein the second flow is the flow sent to an external client by the application program.

4. The method of claim 1 or 2, wherein said performing a forwarding operation on said real-time traffic data, forwarding said real-time traffic data to an external auditing device, comprises:

distributing port information and a network card structure body; and

and forwarding the real-time flow data to the external auditing equipment through the port information based on the network card structural body.

5. A data auditing method is applied to external auditing equipment, and comprises the following steps:

receiving real-time flow data, wherein the real-time flow data is obtained by executing acquisition operation and forwarding operation based on a pre-configured flow guiding tool in a server, and the pre-configured flow guiding tool realizes flow guiding based on a protocol stack of a kernel bypass;

extracting flow characteristics in the real-time flow data;

classifying and storing the real-time flow data through the flow characteristics to obtain a temporary file; and

and analyzing the temporary file to obtain analysis content.

6. The method of claim 5, wherein the traffic characteristics include five tuple information and time information, the five tuple information including protocol information,

the classifying and storing the real-time flow data through the flow characteristics to obtain a temporary file comprises the following steps:

classifying the real-time traffic data based on the protocol information; and

and storing the classified real-time flow data according to a time sequence based on the time information.

7. The method of claim 6, wherein parsing the temporary file to obtain parsed content comprises:

merging the temporary files based on the same protocol information to obtain a first temporary file; and

and analyzing the first temporary file to obtain the analysis content.

8. A data auditing method, comprising:

when the server is in a network connection state with external auditing equipment, acquiring data traffic of an application program to obtain real-time traffic data;

the server executes forwarding operation on the real-time flow data, and forwards the real-time flow data to external auditing equipment so that the external auditing equipment can audit the real-time flow data, wherein the forwarding operation is performed according to a pre-bound network card identification number;

extracting, by the external auditing device, traffic features in the real-time traffic data from the server;

classifying and storing the real-time flow data by the external auditing equipment through the flow characteristics to obtain a temporary file; and

the temporary file is analyzed by the external auditing equipment to obtain analysis content,

9. A data auditing apparatus, the apparatus is applied to a server, the apparatus comprises:

the acquisition module is used for executing acquisition operation on the data traffic of the application program to obtain real-time traffic data when ensuring that the external auditing equipment is in a network connection state; and

a forwarding module for performing a forwarding operation on the real-time traffic data and forwarding the real-time traffic data to an external auditing device so that the external auditing device audits the real-time traffic data,

10. A data auditing apparatus, the apparatus is applied to external auditing equipment, the apparatus includes:

the real-time flow data receiving module is used for receiving real-time flow data, the real-time flow data are obtained by executing acquisition operation and forwarding operation based on a pre-configured drainage tool in the server, and the pre-configured drainage tool realizes drainage based on a protocol stack of a kernel bypass;

the flow characteristic extraction module is used for extracting flow characteristics in the real-time flow data;

the classified storage module is used for classifying and storing the real-time flow data through the flow characteristics to obtain a temporary file; and

and the analysis module is used for analyzing the temporary file to obtain analysis content.

11. A data auditing system, the system includes a server and an external auditing device,

wherein,

the server is configured to execute acquisition operation on the data traffic of the application program when ensuring that the server is in a network connection state with external auditing equipment, so as to obtain real-time traffic data;

the server is further configured to perform forwarding operation on the real-time traffic data, and forward the real-time traffic data to external auditing equipment so that the external auditing equipment audits the real-time traffic data, wherein the forwarding operation is performed according to a pre-bound network card identification number;

the external auditing device is configured to extract traffic features in the real-time traffic data from the server;

the external auditing equipment is also configured to classify and store the real-time flow data through the flow characteristics to obtain a temporary file; and

the external auditing device is also configured to parse the temporary file to obtain parsed contents,

12. An electronic device, comprising:

one or more processors;

a storage device for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-8.

13. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 8.

14. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 8.