CN115883255B - Data filtering method, device and computer readable medium - Google Patents
Data filtering method, device and computer readable medium Download PDFInfo
- Publication number
- CN115883255B CN115883255B CN202310051101.0A CN202310051101A CN115883255B CN 115883255 B CN115883255 B CN 115883255B CN 202310051101 A CN202310051101 A CN 202310051101A CN 115883255 B CN115883255 B CN 115883255B
- Authority
- CN
- China
- Prior art keywords
- data packet
- service data
- xdp
- module
- filtered
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a data filtering method, equipment and a computer readable medium, wherein the method is applied to a terminal; the hardware of the terminal comprises a network card; the kernel space of the terminal comprises a network card driving module, an XDP module and a network protocol stack; the user space of the terminal comprises an application program; the method comprises the following steps: the XDP module acquires the service data packet received by the network card through the capture point of the service data packet; the capture point of the service data packet is a hook function for acquiring the service data packet; the XDP module filters the obtained service data packet according to the filtering rule to obtain a filtered service data packet; the network card driving module sends the filtered service data packet to a network protocol stack; the network protocol stack sends the filtered service data packet to the application program. According to the method and the device, data filtering is performed in the kernel space, and the efficiency of receiving the effective service data packet by the application program is improved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data filtering method, apparatus, and computer readable medium.
Background
In the user space in the system framework of the terminal, there are typically installed a plurality of applications that implement self-supporting service functions by using some service data packets. Specifically, in the prior art, the application program uses the service data packet: the service data packet is transmitted from the bottom layer hardware to the upper layer user space, after receiving the service data packet transmitted from the bottom layer, the application program analyzes and filters the service data packet to obtain a filtered service data packet, and then the service function is realized by using the filtered service data packet.
As can be seen from the foregoing description of the application program using the service data packet, only the service data packet after filtering is valid for the application program, and the service data packet after filtering is invalid for the application program. However, in the prior art, in the process of transmitting the service data packet from the bottom layer of the terminal to the user space of the upper layer, a large amount of invalid service data packets are transmitted to the user space, so that the efficiency of receiving the valid service data packet by the application program is very low, and the performance of the application program is seriously affected for the application program needing to use the valid service data packet at high frequency.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data filtering method, device and computer readable medium, which filter service data packets through an XDP module, so as to improve the efficiency of receiving effective service data packets by an application program.
In order to achieve the above object, the embodiment of the present invention provides the following technical solutions:
in a first aspect, the application discloses a data filtering method applied to a terminal; the hardware of the terminal comprises a network card; the kernel space of the terminal comprises a network card driving module, an XDP module and a network protocol stack; the user space of the terminal comprises an application program; the data filtering method comprises the following steps:
the XDP module acquires the service data packet received by the network card through a capture point of the service data packet; the capture point of the service data packet is a hook function for acquiring the service data packet;
the XDP module filters the acquired service data packet according to the filtering rule to obtain a filtered service data packet;
the network card driving module sends the filtered service data packet to the network protocol stack;
the network protocol stack sends the filtered service data packet to the application program;
The application program uses the filtered service data packet to execute a service function related to the filtered service data packet.
Optionally, in the above data filtering method, before the XDP module performs filtering processing on the acquired service data packet according to a filtering rule to obtain a filtered service data packet, the method further includes:
the XDP module analyzes the acquired service data packet and analyzes the contract number of the acquired service data packet;
the XDP module performs filtering processing on the acquired service data packet according to a filtering rule to obtain a filtered service data packet, and includes:
and the XDP module filters the acquired service data packet by using a filtering rule according to the analyzed contract number of the service data packet to obtain a filtered service data packet.
Optionally, in the above data filtering method, the XDP module obtains, through a capture point of a service data packet, the service data packet received by the network card, including:
when the network card receives a service data packet, triggering the XDP module to process the service data packet according to a hook function parameter type through a capture point of the service data packet to obtain the processed service data packet;
The XDP module performs filtering processing on the acquired service data packet according to a filtering rule to obtain a filtered service data packet, and includes:
and the XDP module filters the processed service data packet according to a filtering rule to obtain a filtered service data packet.
Optionally, in the above data filtering method, before the XDP module performs filtering processing on the processed service data packet according to a filtering rule to obtain a filtered service data packet, the method further includes:
the XDP module determines an IP head pointer in the processed service data packet according to an Ethernet frame head pointer corresponding to the hook function parameter entry type;
the XDP module determines a transmission protocol head pointer of the processed service data packet according to the IP head pointer;
the XDP module determines the payload head pointer of the processed service data packet according to the transmission protocol head pointer;
the XDP module analyzes the contract number of the processed service data packet according to the payload head pointer and the contract number offset position of the processed service data packet; the contract number offset position of the processed service data packet is predetermined according to the protocol format of the service data packet;
The XDP module performs filtering processing on the processed service data packet according to a filtering rule to obtain a filtered service data packet, and the method includes:
and the XDP module filters the processed service data packet by using a filtering rule according to the analyzed contract number to obtain a filtered service data packet.
Optionally, in the above data filtering method, before determining the transport protocol header pointer of the processed service data packet according to the IP header pointer, the XDP module further includes:
the XDP module determines the transmission protocol type of the processed service data packet according to the IP head pointer;
wherein the XDP module determines, according to the IP header pointer, a transport protocol header pointer of the processed service data packet, including:
if the transmission protocol type of the processed service data packet is a specific transmission protocol type, the XDP module determines a transmission protocol head pointer of the processed service data packet according to the IP head pointer; wherein the specific transmission protocol type is the transmission protocol type of the service data packet used by the application program.
Optionally, in the above data filtering method, the creating process of the capture point of the service data packet includes:
starting a kernel debugging function;
determining a transmission path of a service data packet in the kernel space according to kernel debugging information; the kernel debugging information comprises functions used when the service data packet is transmitted;
and creating a capture point of the service data packet according to the transmission path of the service data packet in the kernel space.
Optionally, in the above data filtering method, the XDP module performs filtering processing on the obtained service data packet according to a filtering rule, to obtain a filtered service data packet, including:
the XDP module judges whether the obtained service data packet meets the release condition specified in the filtering rule;
the XDP module determines the obtained service data packet as a filtered service data packet if the obtained service data packet meets the release condition specified in the filtering rule;
the XDP module performs filtering processing on the acquired service data packet according to a filtering rule, and after obtaining the filtered service data packet, the XDP module further includes:
and if the obtained service data packet does not meet the release condition specified in the filtering rule, discarding the obtained service data packet by the XDP module.
Optionally, in the above data filtering method, before the network card driving module sends the filtered service data packet to the network protocol stack, the method further includes:
and the XDP module returns the filtered service data packet to the network card driving module.
In a second aspect, the present application discloses a computer readable medium having stored thereon a computer program, wherein the program when executed by a processor implements a method according to any of the first aspects described above.
In a third aspect, the present application discloses a data filtering apparatus comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of the first aspects described above.
Based on the data filtering method provided by the embodiment of the invention, the XDP module obtains the service data packet received by the network card through the capture point of the service data packet, wherein the capture point of the service data packet is a hook function for obtaining the service data packet. And then the XDP module filters the acquired service data packet according to the filtering rule to obtain a filtered service data packet, and then the network card driving module sends the filtered service data packet to the network protocol stack, and the network protocol stack sends the filtered service data packet to the application program to execute related service functions. Because the filtering processing is performed in the kernel space of the terminal, invalid service data packets are not transmitted in the subsequent transmission path from the XDP module to the application program, and the efficiency of receiving the valid service data packets by the application program is improved. In addition, the data filtering method of the embodiment of the application realizes the acquisition and filtering of the service data packet through the capture point of the XDP module in the kernel space, so that the mode has better compatibility, and when the XDP module is required to acquire and filter the service data packets with different protocol formats, the quick adaptation can be realized by only adaptively modifying the XDP module, the development of hardware of a terminal is not required, and the consumed development workload is very low.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a data filtering method disclosed in an embodiment of the present application;
fig. 2 is a schematic diagram of a frame system of a terminal according to an embodiment of the present application;
fig. 3 is a flow chart of a method for creating a capture point according to an embodiment of the present application;
fig. 4 is a schematic diagram of a data format in a service data packet according to an embodiment of the present disclosure;
fig. 5 is a flow chart of a process of resolving a contract number according to an embodiment of the present application;
fig. 6 is a schematic diagram of a transmission flow of a service data packet according to an embodiment of the present application;
fig. 7 is a schematic diagram of a frame of a data filtering system according to an embodiment of the present disclosure.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In this application, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Referring to fig. 1, an embodiment of the present application proposes a data filtering method, which may be used for filtering scenes of service data packets applicable to different service types. For example, the method can be applied to filtering scenes of network quotation data packets, filtering scenes of video data packets and the like. The network market data is the data input of the market-making transaction system and comprises real-time data such as buying price, buying quantity, selling price, selling data and the like. The network quotation data is contained in the network quotation data packet.
The data filtering method shown in fig. 1 is applied to a terminal. The terminal is understood to be a terminal device, which may be a mobile phone, a desktop computer, a notebook computer, etc. The system framework of the terminal in this embodiment of the present application is shown in fig. 2, and includes Hardware (Hardware), kernel space (Kernel space), and User space (User space). Where kernel space may also be referred to as the kernel layer, and user space may also be understood as the application layer. The hardware of the terminal comprises a network card, the kernel space of the terminal comprises a network card driving module, a rapid data path (eXpress Data Path, XDP) module and a network protocol stack, and the user space of the terminal comprises an application program. The application program may be understood as an application for performing a service function using a service data packet. The system type of the terminal in the embodiment of the present application may be a linux operating system. Specifically, the data filtering method shown in fig. 1 includes the following steps:
S101, the XDP module acquires the service data packet received by the network card through the capture point of the service data packet, wherein the capture point of the service data packet is a hook function for acquiring the service data packet.
The XDP module obtains the service data packet received by the network card by directly or indirectly calling the capture point of the service data packet. The service data packet includes service data, which may be understood as data for implementing a service function. The service data packet may consist of one or more parts of IP, user datagram protocol (UDP, user Datagram Protocol), transmission control protocol (TCP, transmission Control Protocol), payload, etc. The payload carries specific data content in the data packet, for example, network quotation data in the network quotation data packet is carried in the payload.
The service data packets of different service types may be different in the IP portion, the UDP portion, the TCP portion, the payload portion, etc., and the service data packets of different protocol formats under the same service type may be different in the IP portion, the UDP portion, the TCP portion, the payload portion, etc. Taking the network quotation data packet as an example, the field of the contract number corresponding to the protocol format of the network quotation data packet of the futures and the field of the contract number corresponding to the protocol format of the network quotation data packet of the option have different field characteristics.
The capture points for the service data packets may be pre-created. The capture point of the service data packet may be in the network card driving module. The XDP module calls a capturing point of the service data packet through the network card driving module to capture the service data packet received by the network card. The network card driver module may be understood as a module for running a network card driver. The network card driving program is a program for enabling the hardware network card to function normally, and the network card can work normally only to receive the service data packet. The network card driving module comprises a plurality of hook functions. A hook function (hook) is a procedure for handling system messages when it is captured in the system and then performs some operations, which can be understood as a procedure. The hook function in the network card driving module is instantiated in advance to obtain the hook function for acquiring the service data packet, and the hook function is used as a capturing point of the service data packet in the step S101.
After the capture point of the service data packet is established in advance, after the network card receives the service data packet, the XDP module can capture the service data packet received by the network card through the capture point of the service data packet, and then execute subsequent steps S102 to S105 on the captured service data packet.
Alternatively, the hook function used as the capture point in step S101 may be a fast data path (eXpress Data Path, XDP) or may be TC, and the specific type of the hook function is not limited in the embodiments of the present application.
Optionally, in a specific embodiment of the present application, an implementation manner of step S101 is performed, including:
when the network card receives the service data packet, the XDP module is triggered to process the service data packet according to the hook function entry type through the capture point of the service data packet, and the processed service data packet is obtained.
Specifically, when the network card receives the service data packet, an interrupt event is triggered, and then an entry of a capturing point of the service data packet is triggered, the service data packet is obtained in real time, the service data packet is transmitted to the XDP module according to the hook function entry type, and the service data packet is processed according to the hook function entry type, so that the processed service data packet is obtained. The format of the processed service data packet is the hook function parameter type format.
The hook function entry type may be understood as an entry format type of a capture point of the service data packet. After processing the service data packet into a data packet conforming to the hook function parameter entry type, the capture point of the service data packet can be used for acquisition and subsequent filtering processing.
Referring to fig. 2, taking an example of taking a capture point of a service data packet as an XDP hook function, a specific process of implementing step S101 through an XDP mechanism may be: the hardware buffer area of the network card buffers the received service data packet, the network card receives the service data packet to trigger an interrupt event, the service data packet is triggered to enter an inlet of an XDP hook interception point (namely a capturing point of the service data packet) through the network card driving module, the service data packet is transmitted into a network data packet filtering (Extended Berkeley Packet Filter, eBPF) virtual machine according to the type of the XDP hook function entry in real time, and then the step S102 is executed in the eBPF virtual machine. Wherein, the eBPF virtual machine and the XDP hook intercept point are both running in kernel mode. Since the XDP intercept point (i.e., XDP hook intercept point) has functions of passing, discarding, and forwarding the data packet, the filtered data packet can be discarded, passing the filtered data packet.
It should be noted that, in step S101, the number of service data packets acquired by the XDP module is not limited in the embodiment of the present application, and all service data packets received by the network card may be acquired by the XDP module through a capture point of the service data packet.
Referring to fig. 3, in an embodiment of the present application, a process for creating a capture point of a service data packet includes the following steps:
S301, starting a kernel debugging function.
The kernel debugging function is mainly used for acquiring kernel debugging information. Kernel debug information may be understood as related data within the kernel space of the terminal. The kernel debugging function can be realized by adopting a kernel debugging tool, and kernel debugging is carried out on the terminal through the tool to acquire kernel debugging information. The tool may not be installed inside the terminal, or may be installed inside the terminal, which is not limited in the embodiment of the present application.
S302, determining a transmission path of the service data packet in a kernel space according to kernel debugging information, wherein the kernel debugging information comprises a function used when the service data packet is transmitted.
The kernel debugging information comprises information of functions (which can be understood as called) used when the service data packet is transmitted in the kernel space, and the transmission path of the service data packet in the kernel space can be determined by analyzing the dependency relationship, calling sequence and the like among the functions used when the service data packet is transmitted. The transmission path of the service data packet in the kernel space can be understood as the order of use of the functions used when transmitting the service data packet in the kernel space.
S303, creating a capturing point of the service data packet according to the transmission path of the service data packet in the kernel space.
According to the transmission path of the service data packet in the kernel space, one function is selected as an inlet for receiving the service data packet in the network card driving module, and the inlet is instantiated into a hook function capable of acquiring the service data packet, so that a capturing point of the service data packet is created. Illustratively, the capture point of the service data packet may be created by selecting the function that is first called (used) in the network card driver module according to the transmission path. The closer the capture point of the created service data packet is to the underlying hardware network card, the higher the efficiency of the subsequent application program to receive the valid service data packet.
For example, the format of the IP portion 401 in the service data packet may be shown in fig. 4, and includes an IP header, a version number of 4 bits, a header length (Internet Header Length, IHL) of 4 bits, and a DS field of 6 bits … …, which are described in detail with reference to the IP portion 401 format shown in fig. 4, and will not be described herein. Referring to fig. 4, the UDP portion 402 may include a UDP header, a source port number, a destination port number, a UDP length, and a UDP checksum, and details thereof are shown in fig. 4 and will not be described herein. Referring to fig. 4, the TCP portion 403 may include a TCP header, a source port number, a destination port number, a sequence number, an acknowledgement function, a header length, a reserved, a control field, and the like, and details thereof are shown in fig. 4 and will not be described herein.
It should be noted that, the embodiment of the present application does not limit the specific format of the service data packet.
S102, the XDP module filters the acquired service data packet according to the filtering rule to obtain a filtered service data packet.
A filtering rule may be understood as a rule for specifying a condition for a traffic data packet to be allowed to pass through to a network protocol stack, or for specifying a condition for a traffic data packet to not be allowed to pass through to a next traffic data packet receiving node (in this embodiment, a network protocol stack). According to the filtering rule, filtering all the service data packets obtained in the step S101, wherein the service data packets which are in accordance with the filtering rule and can be released to the network protocol stack are used as the filtered service data packets, but the service data packets which are not in accordance with the rule and can be released to the network protocol stack are filtered, and are not used as the filtered service data packets, and are not released to the network protocol stack.
Illustratively, performing one embodiment of step S102 includes:
the XDP module judges whether the acquired service data packet meets the release condition specified in the filtering rule, if the acquired service data packet meets the release condition specified in the filtering rule, the acquired service data packet is determined to be the filtered service data packet, and if the acquired service data packet does not meet the release condition specified in the filtering rule, the acquired service data packet is discarded.
The filtering rule includes a release condition, which can be understood as a condition of a service data packet that can be released to the network protocol stack. The service data packets meeting the release condition are effective service data packets which can be used by the application program, and the service data packets which do not meet the release condition need to be filtered, namely, are not released to the network protocol stack. A traffic data packet that does not meet the release condition is understood to be a traffic data packet that is not valid for the application, i.e. a traffic data packet that is not used by the application when performing the traffic function. And screening out all the service data packets meeting the release conditions specified in the filtering rules from the step S101 as the filtered service data packets so as to flow the service data packets into the next data receiving node. And the service data packets which are not screened, namely, do not meet the release condition are directly discarded.
For example, if the capture point of the service data packet used in step S101 is an XDP hook function, an XDP filter may be preset by using an XDP mechanism, and a filtering rule may be configured in the XDP filter, and further, the filtering rule may be used to filter all the acquired service data packets, so as to filter service data packets that do not meet the release condition specified in the filtering rule. For example, as shown in fig. 2, after the XDP hook intercept point (i.e. the capture point of the service data packet) intercepts the service data packet received by the network card of the hardware layer, the service data packet is filtered by the XDP filter program in the eBPF virtual machine, and the filtering rule is used to implement discarding of the invalid service data packet, and the valid service data packet is reserved as the filtered service data packet, and the filtered service data packet is continuously transmitted to the subsequent data packet receiving node through the network card driving module until being transmitted to the application program.
Optionally, in an embodiment of the present application, before performing step S102, the method further includes:
the XDP module analyzes the acquired service data packet and analyzes the contract number of the acquired service data packet. When the XDP module executes step S102, the XDP module is configured to: and the XDP module filters the acquired service data packet by using a filtering rule according to the contract number of the analyzed service data packet to obtain a filtered service data packet.
Specifically, in some embodiments, the release condition in the filtering rule is set according to the contract number of the service data packet. After the XDP module obtains the service data packet, analyzing the contract number of the effective load part in the service data packet, checking whether the contract number of the service data packet accords with the release condition in the filtering rule, if so, determining the service data packet as the filtered service data packet, and if not, removing the service data packet and discarding the service data packet.
For example, when the filtering rule is that the market data packet of the futures allows passage, the market data packet of the options is discarded, the contract number of the market data packet of the futures has an A field, and the contract number of the market data packet of the options has a B field. Therefore, after the XDP module obtains the service data packet, the contract number of the payload part in the service data packet is parsed, and then whether the contract number has an a field or not can be checked after the contract number of the service data packet is parsed, if the contract number has the a field, the clearance condition is met, and the service data packet is determined to be the filtered service data packet. If there is no A field, it is discarded directly.
Optionally, the contract number of the service data packet is parsed by determining an offset position of the contract number in the service data packet in advance. The offset position of the contract number may be understood as the offset position of the contract number in the payload section. For example, when the service data packet is a network quotation data packet, the contract number is offset by 3, that is, after the offset by 3 bits at the head pointer of the payload, the contract number of the network quotation data packet can be read.
The corresponding relation exists between the offset position of the contract number and the protocol format of the service data packet, and the offset position of the contract number of the service data packet can be determined according to the corresponding relation between the offset position of the contract number and the protocol format of the service data packet by determining the protocol format of the service data packet in advance. For example, the protocol format of the service data packet may be predetermined, and when the XDP module parses the acquired service data packet, the contract number (Instrument ID) of the acquired service data packet may be parsed according to the offset position of the contract number corresponding to the protocol format.
Optionally, in an embodiment of the present application, if in step S101, when the network card receives the service data packet, the XDP module triggers the XDP module to process the service data packet according to the hook function entry type through the capture point of the service data packet, so as to obtain a processed service data packet, an implementation manner of step S102 is performed, including:
And the XDP module filters the processed service data packet according to the filtering rule to obtain a filtered service data packet.
That is, in the filtering process, step S102 filters the service data packet processed according to the hook function entry type. Specific filtering processes may refer to the foregoing descriptions, and are not repeated here.
Optionally, referring to fig. 5, in an embodiment of the present application, the XDP module performs filtering processing on the processed service data packet according to a filtering rule, and before obtaining the filtered service data packet, the method further includes:
s501, the XDP module determines an IP head pointer in the processed service data packet according to the Ethernet frame head pointer corresponding to the hook function parameter entry type.
The format of the processed service data packet is processed into the format of the hook function entry type through the capture point. The format of the hook function entry type is fixed, so that the ethernet frame header pointer corresponding to the hook function entry type can be determined according to the format corresponding to the hook function entry type. And the XDP module can locate the position of the IP head pointer from the processed service data packet according to the Ethernet frame head pointer corresponding to the hook function parameter type. The IP header pointer may point to the location of the IP header.
S502, the XDP module determines a transmission protocol head pointer of the processed service data packet according to the IP head pointer.
As can be seen from the foregoing format of the IP portion shown in fig. 4, the length information of the IP portion may be read in the format of the IP portion, and thus the IP length information may be read after the IP header pointer is located, and thus the position of the transport protocol header pointer following the IP portion in the processed service data packet may be located according to the IP length information.
In this embodiment of the present application, the IP portion of the processed service data packet is followed by the transport protocol portion, and since the IP header pointer of the processed service data packet is known, and the IP length information is also known through the IP header pointer, the position of the transport protocol header pointer of the processed service data packet can be deduced, and thus the transport protocol header is located.
Optionally, in an embodiment of the present application, before S502 after the performing deficiency, the method further includes:
the XDP module determines the transmission protocol type of the processed service data packet according to the IP head pointer. When the XDP module executes step S502, the XDP module is configured to: if the transmission protocol type of the processed service data packet is a specific transmission protocol type, the XDP module determines the transmission protocol head pointer of the processed service data packet according to the IP head pointer. Wherein the specific transmission protocol type is the transmission protocol type of the service data packet used by the application program.
The XDP module can confirm which transmission protocol the processed service data packet is according to the IP head pointer. The transmission protocol type is UDP, TCP, etc. The types of transmission protocols for the service data packets applicable in different data filtering scenarios are different. For example, when the service data packet is a network quotation data packet, the transmission protocol used by the network quotation data packet is UDP, so when the data filtering method provided in the embodiment of the application is executed on the network quotation data packet, it may be first confirmed whether the processed service data packet is a UDP type transmission protocol before filtering.
Specifically, the IP portion of the service data packet carries information of the transport protocol type, so after the XDP module locates the IP header according to the IP header pointer, the information of the IP portion can be read to determine the transport protocol type of the processed service data packet. Subsequently, when step S502 is executed, it is determined whether the transport protocol type of the processed service data packet is a specific transport protocol type, and if so, the XDP module determines, according to the IP header pointer, the transport protocol header pointer of the processed service data packet. If not, forwarding to other nodes or directly flowing to the next flow node. The specific transmission protocol type refers to a transmission protocol type of a service data packet used by an application program in the embodiment of the application program when executing a service function. For example, the transmission protocol used by the network market data packet is UDP, and when the data filtering method provided in the embodiments of the present application is performed on the network market data packet, the specific transmission protocol type is UDP.
S503, the XDP module determines the payload head pointer of the processed service data packet according to the transmission protocol head pointer.
The XDP module locates a transport protocol header position based on the transport protocol header pointer. The transport protocol part is followed by the payload, due to the fixed format of the processed service data packets. And further, after the header of the transport protocol is positioned, the length information of the transport protocol part is read, and then the payload header pointer can be positioned.
S504, the XDP module analyzes the contract number of the processed service data packet according to the payload head pointer and the contract number offset position of the processed service data packet, wherein the contract number offset position of the processed service data packet is predetermined according to the protocol format of the service data packet.
The payload part comprises a contract number, and the offset position of the contract number in the payload part is preset according to the protocol format of the service data packet, so that the position of the contract number can be offset to the position of the contract number in the payload head part according to the payload head pointer and the contract number offset position of the processed service data packet, and the contract number of the processed service data packet can be read.
The XDP module performs filtering processing on the processed service data packet according to a filtering rule, and is used for: and the XDP module filters the processed service data packet by using a filtering rule according to the analyzed contract number to obtain a filtered service data packet.
After the processing of step S102 in the embodiment of the present application, the number of the filtered service data packets obtained by the XDP module in step S102 is reduced compared with the number of the service data packets obtained in the original step S101, so that the data volume of the subsequent transmission is also significantly reduced (that is, the subsequent transmission is only the filtered service data packets, and the number of the transmitted data packets is significantly reduced), thereby improving the efficiency of receiving the effective service data packets by the application program packets.
In addition, the embodiment of the application is used for acquiring and filtering the service data packet by adopting the hook function in the kernel space, has higher portability, and can be suitable for being used in the kernel space of different operating systems. For example, if the capture point of the service data packet is XDP, the XDP can run on an operating system with kernel of the linux operating system greater than 4.15 without any modification to the operating system. In addition, the data filtering method provided by the embodiment of the application is low, and the currently used network cards support the use of hook functions and the XDP mechanism, so that no special requirements are required for the hardware network card, the purchase of a specific type of network card with high cost is not required, and the re-adaptability development of the network card is not required. In addition, the capture point of the XDP module can acquire and filter the service data packet, the mode has good compatibility, and when the XDP module is required to acquire and filter the service data packet with different protocol formats, the quick adaptation can be realized only by adaptively modifying the XDP module, the development of hardware of a terminal is not required, and the consumed development workload is low.
S103, the network card driving module sends the filtered service data packet to a network protocol stack.
The filtered service data packet belongs to a service data packet which is valid for the application program. The number of the service data packets after filtering is reduced compared with the number of the service data packets received by the original network card. Compared with the mode of filtering the service data packet in the application program in the prior art, the data volume sent to the network protocol stack in the embodiment of the application is reduced, and the efficiency of transmitting the effective service data packet is improved. After the XDP module filters the filtered service data packet in step S102, the XDP module may return the filtered service data packet to the network card driving module, where the network card driving module may send the filtered service data packet to the network protocol stack.
Illustratively, as shown in fig. 2, the eBPF virtual machine (which is included in the XDP module) returns the filtered service data packet to the network card driver module, which sends the filtered service data packet to the network protocol stack buffer.
In other embodiments, a kernel firewall may be further included in the kernel space to further improve the security of the data transmission process in the terminal. If the kernel space further includes a kernel firewall, the method of executing step S103 is as follows: the XDP module sends the filtered service data packet to the network card driving module, and the network card driving module sends the filtered service data packet to the kernel firewall, and the kernel firewall processes the service data packet and then sends the service data packet to the network protocol stack.
And S104, the network protocol stack sends the filtered service data packet to the application program.
Illustratively, as shown in fig. 2, the network protocol stack and the application program communicate through a Socket to implement sending the filtered service data packet from the network protocol stack to the application program.
S105, the application program uses the filtered service data packet to execute the service function related to the filtered service data packet.
After receiving the filtered service data packet sent by the network protocol stack, the application program can directly use the filtered service data packet to execute related service functions without filtering. Because the invalid service data packet is filtered out in the XDP module in advance in the process of transmitting the service data packet from the network card to the application program, the speed of acquiring the valid service data packet by the final application program is obviously improved.
For example, if the service data packet is a network quotation data packet, the process of executing step S105 may be that after the application program completes login identity authentication between the API login interface and the login server, the application program checks the filtered network quotation data packet received by the application program, and analyzes the correct quotation field. Where a correct market field is understood to be a field in an interpretable format that is applicable to an application. And pushing the analyzed detailed market data to a strategy platform module for transaction.
The process of completing login identity authentication between the application program and the login server through the API login interface is as follows: the application program establishes TCP connection with the login server, the login server sends the network data packet for identity authentication to the application program, after the authentication is completed by the application program and the login server, the filtered service data packet sent by the network protocol stack is analyzed by the credentials sent by the login server, and the detailed quotation data is distributed to the policy platform module.
In order to further describe the transmission process of the service data packet in the embodiment of the present application, referring to fig. 6, taking the capture point of the service data packet as an XDP as an example, the transmission process of the service data packet in the system frame of the terminal is as follows: the network card sends the received service data packet to a network card driving module, the network card driving module starts an initial (Native) loading mode, loads an XDP program (the XDP program runs in the XDP module), runs the XDP mode, achieves acquisition and filtration of the service data packet through the XDP mode, then sends the filtered service data packet to a kernel firewall, the kernel firewall sends the service data packet to a network protocol stack, and finally the network protocol stack adopts a socket communication mode and application program communication, and transmits the filtered service data packet to the application program.
Based on the data filtering method provided in the embodiment of the present application, as shown in fig. 7, taking a network market data packet as an example, a filtering process of the network market data packet is as follows: the network quotation data packet is multicast by the exchange machine to the terminal, the network card in the terminal receives the multicast network quotation data packet, and then triggers the XDP quotation filtering module (the XDP quotation filtering module is the same as the XDP module) to acquire the network quotation data packet and filters the acquired network quotation data packet by using a filtering rule to acquire the filtered network quotation data packet. The XDP quotation filtering module transmits the filtered network quotation data packet to a quotation receiving program (namely an application program) through a kernel firewall and a network protocol stack, and the quotation receiving program logs in a login server (namely a login server) to execute corresponding service functions.
According to the data filtering method provided by the embodiment of the invention, the XDP module acquires the service data packet received by the network card through the capture point of the service data packet, wherein the capture point of the service data packet is a hook function for acquiring the service data packet. And then the XDP module filters the acquired service data packet according to the filtering rule to obtain a filtered service data packet, and then the network card driving module sends the filtered service data packet to the network protocol stack, and the network protocol stack sends the filtered service data packet to the application program to execute related service functions. Because the filtering processing is performed in the kernel space of the terminal, invalid service data packets are not transmitted in the subsequent transmission path from the XDP module to the application program, and the efficiency of receiving the valid service data packets by the application program is improved. In addition, the data filtering method of the embodiment of the application realizes the acquisition and filtering of the service data packet through the capture point of the XDP module in the kernel space, so that the mode has better compatibility, and when the XDP module is required to acquire and filter the service data packets with different protocol formats, the quick adaptation can be realized by only adaptively modifying the XDP module, the development of hardware of a terminal is not required, and the consumed development workload is very low.
The embodiment of the application also discloses a computer readable medium, on which a computer program is stored, wherein the program is executed by a processor to implement the data filtering method according to any one of the embodiments of the application.
The embodiment of the application also discloses a data filtering device, which comprises: one or more processors; a storage device having one or more programs stored thereon; the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the data filtering method as set forth in any of the embodiments of the present application described above.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for a system or system embodiment, since it is substantially similar to a method embodiment, the description is relatively simple, with reference to the description of the method embodiment being made in part. The systems and system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (3)
1. The data filtering method is characterized in that the method is applied to a filtering scene of a network quotation data packet, wherein the network quotation data packet is real-time data of a marketing transaction system and is applied to a terminal; the hardware of the terminal comprises a network card; the kernel space of the terminal comprises a network card driving module, an XDP module and a network protocol stack; the user space of the terminal comprises an application program; the data filtering method comprises the following steps:
the XDP module acquires the service data packet received by the network card through a capture point of the service data packet; the capture point of the service data packet is a hook function for acquiring the service data packet;
the XDP module filters the acquired service data packet according to the filtering rule to obtain a filtered service data packet; setting a release condition of the filtering rule according to a contract number of the service data packet;
the network card driving module sends the filtered service data packet to the network protocol stack;
the network protocol stack sends the filtered service data packet to the application program;
the application program uses the filtered service data packet to execute service functions related to the filtered service data packet;
The XDP module obtains the service data packet received by the network card through the capture point of the service data packet, and the XDP module comprises:
when the network card receives a service data packet, triggering an interrupt event and triggering an entry of a capture point of the service data packet, acquiring the service data packet in real time, transmitting the service data packet into the XDP module according to the entry type of a hook function, and processing the service data packet according to the entry type of the hook function to obtain the processed service data packet;
the XDP module filters the acquired service data packet according to a filtering rule, and before obtaining the filtered service data packet, the XDP module further comprises:
the XDP module analyzes the acquired service data packet and analyzes the contract number of the acquired service data packet;
the XDP module performs filtering processing on the acquired service data packet according to a filtering rule to obtain a filtered service data packet, and includes:
the XDP module filters the acquired service data packet by using a filtering rule according to the analyzed contract number of the service data packet to obtain a filtered service data packet;
The XDP module filters the acquired service data packet according to a filtering rule to obtain a filtered service data packet, and the method comprises the following steps:
the XDP module carries out filtering treatment on the treated service data packet according to a filtering rule to obtain a filtered service data packet;
the XDP module filters the processed service data packet according to a filtering rule, and before obtaining the filtered service data packet, the XDP module further comprises:
the XDP module determines an IP header pointer in the processed service data packet according to an Ethernet frame header pointer corresponding to the parameter entering type of the hook function;
the XDP module determines a transmission protocol head pointer of the processed service data packet according to the IP head pointer;
the XDP module determines the payload head pointer of the processed service data packet according to the transmission protocol head pointer;
the XDP module analyzes the contract number of the processed service data packet according to the payload head pointer and the contract number offset position of the processed service data packet; the contract number offset position of the processed service data packet is predetermined according to the protocol format of the service data packet;
The XDP module performs filtering processing on the processed service data packet according to a filtering rule to obtain a filtered service data packet, and the method includes:
the XDP module filters the processed service data packet by using a filtering rule according to the analyzed contract number to obtain a filtered service data packet;
the XDP module determines the transmission protocol head pointer of the processed service data packet according to the IP head pointer, and further comprises:
the XDP module determines the transmission protocol type of the processed service data packet according to the IP head pointer;
wherein the XDP module determines, according to the IP header pointer, a transport protocol header pointer of the processed service data packet, including:
if the transmission protocol type of the processed service data packet is a specific transmission protocol type, the XDP module determines a transmission protocol head pointer of the processed service data packet according to the IP head pointer; wherein the specific transmission protocol type is the transmission protocol type of a service data packet used by the application program;
the creation process of the capture point of the service data packet comprises the following steps:
Starting a kernel debugging function;
determining a transmission path of a service data packet in the kernel space according to kernel debugging information; the kernel debugging information comprises functions used when the service data packet is transmitted;
creating a capturing point of the service data packet according to the transmission path of the service data packet in the kernel space;
the XDP module filters the acquired service data packet according to a filtering rule to obtain a filtered service data packet, and the method comprises the following steps:
the XDP module judges whether the obtained service data packet meets the release condition specified in the filtering rule;
the XDP module determines the obtained service data packet as a filtered service data packet if the obtained service data packet meets the release condition specified in the filtering rule;
the XDP module performs filtering processing on the acquired service data packet according to a filtering rule, and after obtaining the filtered service data packet, the XDP module further includes:
the XDP module discards the obtained service data packet if the obtained service data packet does not meet the release condition specified in the filtering rule;
before the network card driving module sends the filtered service data packet to the network protocol stack, the network card driving module further includes:
And the XDP module returns the filtered service data packet to the network card driving module.
2. A computer readable medium, having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the method of claim 1.
3. A data filtering apparatus, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310051101.0A CN115883255B (en) | 2023-02-02 | 2023-02-02 | Data filtering method, device and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310051101.0A CN115883255B (en) | 2023-02-02 | 2023-02-02 | Data filtering method, device and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115883255A CN115883255A (en) | 2023-03-31 |
| CN115883255B true CN115883255B (en) | 2023-06-23 |
Family
ID=85758624
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310051101.0A Active CN115883255B (en) | 2023-02-02 | 2023-02-02 | Data filtering method, device and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115883255B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114401245A (en) * | 2021-12-22 | 2022-04-26 | 互联网域名系统北京市工程研究中心有限公司 | Method, device, computer equipment and storage medium for realizing high-performance DNS service |
| CN114640716A (en) * | 2022-05-11 | 2022-06-17 | 军事科学院系统工程研究院网络信息研究所 | Cloud network cache acceleration system and method based on fast network path |
| WO2022267815A1 (en) * | 2021-06-21 | 2022-12-29 | 中兴通讯股份有限公司 | Data packet filtering method and apparatus, and electronic device and computer-readable storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11055292B2 (en) * | 2017-11-21 | 2021-07-06 | Gto Llc | Systems and methods for generating and arbitrating among multiple market-data feeds |
| CN113489737A (en) * | 2021-07-14 | 2021-10-08 | 芯河半导体科技(无锡)有限公司 | Netfilter-based packet filter implementation method |
| CN114244560B (en) * | 2021-11-10 | 2024-04-16 | 奇安信科技集团股份有限公司 | Flow processing method and device, electronic equipment and storage medium |
| CN115603985A (en) * | 2022-09-30 | 2023-01-13 | 北京天融信网络安全技术有限公司（Cn） | Intrusion detection method, electronic device and storage medium |
-
2023
- 2023-02-02 CN CN202310051101.0A patent/CN115883255B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022267815A1 (en) * | 2021-06-21 | 2022-12-29 | 中兴通讯股份有限公司 | Data packet filtering method and apparatus, and electronic device and computer-readable storage medium |
| CN114401245A (en) * | 2021-12-22 | 2022-04-26 | 互联网域名系统北京市工程研究中心有限公司 | Method, device, computer equipment and storage medium for realizing high-performance DNS service |
| CN114640716A (en) * | 2022-05-11 | 2022-06-17 | 军事科学院系统工程研究院网络信息研究所 | Cloud network cache acceleration system and method based on fast network path |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115883255A (en) | 2023-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11082436B1 (en) | System and method for offloading packet processing and static analysis operations | |
| CN109347817B (en) | Method and device for network security redirection | |
| US20190303221A1 (en) | Methods and apparatus for preventing packet spoofing with user space communication stacks | |
| US8307417B2 (en) | Port enablement | |
| US20070083924A1 (en) | System and method for multi-stage packet filtering on a networked-enabled device | |
| CN102387045B (en) | Embedded point to point (P2P) flow monitoring system and method thereof | |
| US11005813B2 (en) | Systems and methods for modification of p0f signatures in network packets | |
| CN102761534B (en) | Realize the method and apparatus of media access control layer Transparent Proxy | |
| US8544078B2 (en) | Flexible network security system and method for permitting trusted process | |
| US20220263823A1 (en) | Packet Processing Method and Apparatus, Device, and Computer-Readable Storage Medium | |
| US20120140640A1 (en) | Apparatus and method for dynamically processing packets having various characteristics | |
| WO2020206849A1 (en) | Method and system for processing dhcp data with vlan tag | |
| JP3581345B2 (en) | Packet transfer device and packet transfer method | |
| CN115883255B (en) | Data filtering method, device and computer readable medium | |
| CN114710356B (en) | Data processing method and device of vehicle-mounted firewall and vehicle-mounted firewall equipment | |
| CN113923273A (en) | Data packet control method and related device | |
| US10491654B2 (en) | Communicating with a remote service through a hypertext transfer protocol (HTTP) session | |
| CN109756454B (en) | Data interaction method, device and system | |
| CN112217782A (en) | Apparatus and method for identifying attacks in a computer network | |
| CN110545256A (en) | Data transmission method, system, electronic device, transfer server and storage medium | |
| CN110943895A (en) | Network shutdown and Linux system-based network data processing method and device | |
| CN114143048B (en) | Method, device and storage medium for managing safety resources | |
| CN115580563B (en) | Mirror image data processing method and device of cloud network and electronic equipment | |
| CN108366074B (en) | Anti-hijacking method and device for network data packet | |
| CN117411694A (en) | Intrusion detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |