CN110971487B

CN110971487B - Network protocol identification method and device

Info

Publication number: CN110971487B
Application number: CN201911175552.5A
Authority: CN
Inventors: 罗佳; 许泽文
Original assignee: WUHAN HONGXU INFORMATION TECHNOLOGY CO LTD
Current assignee: WUHAN HONGXU INFORMATION TECHNOLOGY Co.,Ltd.
Priority date: 2019-11-26
Filing date: 2019-11-26
Publication date: 2021-10-26
Anticipated expiration: 2039-11-26
Also published as: CN110971487A

Abstract

The embodiment of the invention provides a network protocol identification method and a network protocol identification device. The method comprises the following steps: acquiring a data packet from a network card based on a data plane development kit; judging the protocol type of a transmission layer of the data packet, and determining that the protocol type of the transmission layer is TCP or UDP; if the TCP is adopted, scanning the data packet based on the flow mode of the Hyperscan engine, and identifying the protocol which is adopted by the data packet and runs on the TCP protocol; and if the UDP is adopted, scanning the data packet based on the block mode of the Hyperscan engine, and identifying the protocol which is adopted by the data packet and runs on the UDP protocol. According to the network protocol identification method and device provided by the embodiment of the invention, the data packet is acquired based on the data plane development kit, and the data packet is scanned based on two modes of the Hyperscan engine, so that the identification result of the network protocol can be acquired more efficiently and quickly in a mass data environment.

Description

Network protocol identification method and device

Technical Field

The present invention relates to the field of computer network technologies, and in particular, to a network protocol identification method and apparatus.

Background

With the development of computer networks, the speed of the networks is faster and faster, the data volume transmitted by the networks is larger and larger, and the application of high-speed networks is wider.

The traditional network protocol identification method is based on a deep packet analysis mode, and identification can be carried out only after data is recombined. Under the condition that the data volume transmitted by the network is larger and larger, the traditional network protocol identification method is low in identification speed and efficiency and cannot adapt to the mass data environment.

Disclosure of Invention

The embodiment of the invention provides a network protocol identification method and a network protocol identification device, which are used for solving or at least partially solving the defects of low identification speed and low efficiency in a mass data environment in the prior art.

In a first aspect, an embodiment of the present invention provides a network protocol identification method, including:

acquiring a data packet from a network card based on a data plane development kit;

judging the protocol type of the transmission layer of the data packet, and determining that the protocol type of the transmission layer is TCP or UDP;

if the TCP is adopted, scanning the data packet based on a flow mode of a Hyperscan engine, and identifying a protocol which is adopted by the data packet and runs on a TCP protocol; and if the UDP is adopted, scanning the data packet based on a block mode of a Hyperscan engine, and identifying a protocol which is adopted by the data packet and runs on the UDP protocol.

Preferably, the determining that the protocol type of the transport layer is TCP or UDP, and if the protocol type of the transport layer is TCP, scanning the data packet based on a stream mode of a Hyperscan engine, and identifying a protocol running on a TCP protocol adopted by the data packet, further includes:

and reforming the data packet into ordered data.

Preferably, if the packet is TCP, scanning the packet based on a stream mode of a Hyperscan engine, and the specific step of identifying a protocol running on a TCP protocol adopted by the packet includes:

scanning the reformed data packet based on a flow mode of a Hyperscan engine and a pre-acquired flow mode protocol identification rule to acquire an identification rule corresponding to the data packet;

and determining the TCP protocol corresponding to the identification rule as the protocol which is adopted by the data packet and runs on the TCP protocol.

Preferably, if the packet is UDP, scanning the packet based on a block pattern of a hyperspcan engine, and identifying a protocol running on a UDP protocol used by the packet includes:

scanning the data packet based on a block mode of a Hyperscan engine and a pre-acquired block mode protocol identification rule to acquire an identification rule corresponding to the data packet;

and determining the UDP protocol corresponding to the identification rule as the protocol which is adopted by the data packet and runs on the UDP protocol.

Preferably, the specific step of scanning the reformed data packet based on the Hyperscan engine flow pattern and the pre-obtained flow pattern protocol identification rule includes:

and if the data packet is judged and known not to be HTTP data, scanning the reformed data packet based on the flow mode of the Hyperscan engine and a pre-acquired flow mode protocol identification rule.

and if the data packet is judged and known to be HTTP data, scanning the decoded reformed data packet based on a Hyperscan engine stream mode and a pre-acquired stream mode protocol identification rule after HTTP decoding is carried out on the data packet.

Preferably, the data plane development kit-based method for acquiring a data packet from a network card and determining a protocol type of a transport layer of the data packet further includes:

and preprocessing the IP layer of the data packet.

In a second aspect, an embodiment of the present invention provides a network protocol identification apparatus, including:

the data acquisition module is used for acquiring a data packet from the network card based on the data plane development kit;

the session management module is used for judging the protocol type of the transmission layer of the data packet and determining that the protocol type of the transmission layer is TCP or UDP;

the data scanning module is used for scanning the data packet based on a flow mode of a Hyperscan engine if the TCP is adopted and identifying a protocol which is adopted by the data packet and runs on a TCP protocol; and if the UDP is adopted, scanning the data packet based on a block mode of a Hyperscan engine, and identifying a protocol which is adopted by the data packet and runs on the UDP protocol.

In a third aspect, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the computer program is executed, the steps of the network protocol identification method provided in any one of the various possible implementations of the first aspect are implemented.

In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the network protocol identification method as provided in any one of the various possible implementations of the first aspect.

According to the network protocol identification method and device provided by the embodiment of the invention, the data packet is acquired based on the data plane development kit, and the data packet is scanned based on two modes of the Hyperscan engine, so that the identification result of the network protocol can be obtained more efficiently and quickly in a mass data environment.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.

Fig. 1 is a schematic flowchart of a network protocol identification method according to an embodiment of the present invention;

fig. 2 is a schematic diagram illustrating an initialization process of a network protocol identification method according to an embodiment of the present invention;

fig. 3 is a schematic flow chart illustrating a data acquisition step in the network protocol identification method according to an embodiment of the present invention;

fig. 4 is a schematic flowchart illustrating a session management step in the network protocol identification method according to an embodiment of the present invention;

fig. 5 is a schematic flowchart illustrating an output step in the network protocol identification method according to an embodiment of the present invention;

fig. 6 is a schematic flowchart illustrating a TCP flow scanning step in the network protocol identification method according to an embodiment of the present invention;

fig. 7 is a schematic flowchart illustrating a UDP block scanning step in the network protocol identification method according to an embodiment of the present invention;

fig. 8 is a schematic flowchart illustrating a preprocessing step in a network protocol identification method according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of a network protocol identification apparatus according to an embodiment of the present invention;

fig. 10 is a schematic structural diagram of a network protocol identification apparatus according to an embodiment of the present invention;

fig. 11 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In order to overcome the above problems in the prior art, embodiments of the present invention provide a network protocol identification method and apparatus, and the inventive concept is to implement efficient and fast automatic network protocol identification in a mass data environment based on a data plane development kit and a Hyperscan.

Fig. 1 is a flowchart illustrating a network protocol identification method according to an embodiment of the present invention. As shown in fig. 1, the method includes: and S101, acquiring a data packet from the network card based on the data plane development kit.

Step S101 is a data acquisition step. The data acquisition step can be specifically realized through a data acquisition thread.

It should be noted that the main execution body of the network protocol identification method provided by the embodiment of the present invention is a network protocol identification device.

The data packet obtained in step S101 may be a batch data table.

A Data Plane Development Kit (DPDK) is used for a function library and a drive set for fast Data packet processing, so that the Data processing performance and throughput can be greatly improved, and the working efficiency of a Data Plane application program is improved.

Before step S101, the network protocol recognition device needs to be initialized. Fig. 2 is a schematic diagram illustrating an initialization process of a network protocol identification method according to an embodiment of the present invention. As shown in fig. 2, the initialization process includes:

step S200, start. The configuration files required by each step are given, and the static protocol identification rule configuration files are given.

Step S201, reading and analyzing the configuration file. Reading configuration files required by each step to determine the number and parameters of threads required by each step and specific interaction relations among the threads, thereby constructing an overall process of network protocol identification; and reading the static protocol identification rule configuration file to generate a protocol rule table.

Step S202, initializing the operating environment. Generating a flow mode protocol identification rule database and a block mode protocol identification rule database required by a Hyperscan scanning engine according to a protocol rule table; generating an EAL (environmental Abstraction Layer) parameter required by the DPDK platform according to the parameters in the configuration file required by each step, and initializing the EAL; initializing a memory pool, a network card, a lock-free message queue and the like according to the resource parameters; and starting corresponding threads on different CPU cores to construct an overall process of network protocol identification.

And step S203, monitoring the dynamic rule. Dynamically adding or deleting rules through a preset interface, and keeping a monitoring state all the time; whether dynamic rules are monitored: if yes, receiving a new dynamic rule, executing the step S204, and issuing the rule; otherwise, directly performing the state output in step S205; and then jumps to step S203.

And step S204, issuing rules. And modifying the protocol rule table according to the new dynamic rule to generate a new flow pattern protocol identification rule database and a new block pattern protocol identification rule database which are respectively used for the specific protocol identification of the step S103.

And step S205, outputting the state. And carrying out unified terminal display output or log file output on the whole running state of the network protocol identification device and various statistical data of each functional module of the network protocol identification device.

Fig. 3 is a schematic flow chart illustrating a data acquisition step in the network protocol identification method according to an embodiment of the present invention. And the data acquisition thread acquires data from the high-speed network card and distributes the acquired data packets in a balanced manner. As shown in fig. 3, the data acquisition step specifically includes:

step S300, start. And providing starting resource preparation for all the data acquisition threads, and sequentially triggering the starting of all the data acquisition threads.

And step S301, initialization. Creating a data acquisition thread on a corresponding logic core according to a logic core ID parameter of a pre-designated data acquisition thread; initializing all input ports of the current data acquisition thread, namely binding all initialized network card receiving queues associated with the current data acquisition thread; initializing all output ports of the current data acquisition thread, namely binding all initialized data packet message queues associated with the current data acquisition thread; initializing other parameters which are initialized currently, such as the maximum number of batch packets.

Step S302, network card queue batch packet receiving. And the data acquisition threads sequentially and circularly receive and distribute the data packets from the input ports in batches.

Receiving packets in batches from the corresponding network card queue according to the current input port ID, determining the input port ID of the next packet receiving, and then performing data distribution in the step S303; then, the process jumps to step S302 and enters the next loop.

And step S303, data distribution. And according to the number of the output ports of the current data acquisition thread, the data packets are distributed to all the output ports in a balanced manner.

And when the distribution is balanced, the same source and sink are ensured, namely the data with the same five-tuple is sent to the only output port.

The quintuple includes a local IP, a remote IP, a local port, a remote port, and a transport layer protocol.

Step S102, judging the protocol type of the transmission layer of the data packet, and determining that the protocol type of the transmission layer is TCP or UDP.

Specifically, step S102 is a session management step. The session management step may be specifically implemented by a session management thread.

Fig. 4 is a schematic flowchart of a session management step in the network protocol identification method according to an embodiment of the present invention. And the session management thread sequences the TCP streams, merges the maximum ordered data into a giant frame, uniformly forwards the giant frame to all TCP stream scanning threads associated with the giant frame, and directly and uniformly forwards the UDP message to the UDP block scanning thread. As shown in fig. 4, the session management step specifically includes:

step S400, start. Providing starting resource preparation for all session management threads and triggering the starting of all session management threads in turn.

And step S401, initialization. Creating a session management thread on a corresponding logic core according to a logic core ID parameter of a pre-designated session management thread; initializing all input and output ports of the current session management thread, namely binding all initialized message queues associated with the current session management thread; initializing other parameters of the current session management thread, such as a hash table, the maximum number of batch packets, and the like.

And step S402, processing overtime of the session table node.

The session management thread uses the hash table to implement session management for the TCP flow. The network data packet has various uncertainties, so for the TCP stream session node, timeout processing needs to be performed, that is, the hash table is scanned to locate an invalid stream node and deleted, and meanwhile, an end data packet of the stream is generated and forwarded to a subsequent thread.

The overtime processing uses a mechanism of block scanning, because the number of nodes in the hash table is very large, the cost of scanning all the nodes every time is high, the hash table is decomposed into a plurality of blocks, and all the nodes in one block are scanned every time by maintaining the block ID of the hash table to be scanned.

And S403, batch packet receiving of the message queue. And the session management module receives the data packets from each input port in batches in a sequential and circular manner, processes and forwards the data packets one by one, and enters the next circulation after all the data packets are processed.

And receiving packets in batches from the corresponding message queue according to the current input port ID, determining the input port ID of the next packet receiving, and then entering the step S404 to process the message. After the processing is completed, the process proceeds to step S402.

And step S404, judging whether the message is processed.

If yes, jumping to step S402;

otherwise, sequentially executing:

and S405, identifying the four-layer protocol, namely judging the protocol type of the transmission layer. If the protocol type of the transport layer of the data packet is UDP, step S406 is executed, the data packet is forwarded to the UDP block scanning thread, and the process skips to step S404; and the TCP continues to process the subsequent steps.

Step S407, determine whether the current TCP packet is an end packet, that is, determine whether the current TCP packet is an end packet of the stream. If yes, executing step S408, deleting the node and forwarding the end packet, namely finding the stream node corresponding to the end packet, deleting the node and forwarding the end packet to a subsequent thread, and then jumping to step S404; if not, continuing to process the subsequent steps.

And step S409, searching session nodes. If the current node is found, executing step S411, and performing direct data mounting, namely, sequentially inserting the current data into a data linked list of the current node; if not, step S410 is executed to create a node, and step S411 is executed after the node is created to mount data.

Step S103, if the TCP is adopted, scanning the data packet based on the flow mode of the Hyperscan engine, and identifying the protocol which is adopted by the data packet and runs on the TCP protocol; and if the UDP is adopted, scanning the data packet based on the block mode of the Hyperscan engine, and identifying the protocol which is adopted by the data packet and runs on the UDP protocol.

In particular, Hyperscan is a high performance regular expression matching library.

If the Protocol type of the transport layer of the data packet is UDP (User Datagram Protocol), the UDP block scanning thread scans the data packet based on the block mode of the Hyperscan engine, and identifies the Protocol running on the UDP Protocol adopted by the data packet.

Protocols running over the UDP protocol include: DHCP, NTP, BOOTP, etc.

The Protocol type of the transport layer of the data packet is TCP (Transmission Control Protocol), and the TCP stream scanning thread scans the data packet based on the stream mode of the Hyperscan engine to identify the Protocol used by the data packet and running on the TCP scanning Protocol.

Protocols running over the TCP protocol include: HTTP, HTTPS, FTP, POP3, SMTP, Telnet, SSH, etc.

The protocol recognition result may be output by the outputting step, and the result may be output to an external user using a network interface. The outputting step may be specifically implemented by an output thread.

Fig. 5 is a schematic flowchart of an output step in the network protocol identification method according to an embodiment of the present invention. As shown in fig. 5, the outputting step specifically includes:

step S500, start. Providing starting resource preparation for all output threads and triggering the starting of all output threads in turn.

Step S501, initialization. According to the logic core ID parameter of the output thread appointed in advance, an output thread is established on the corresponding logic core; initializing all input ports of the current output thread, namely binding all initialized message queues associated with the current output thread; initializing a network output interface of a current output thread; and initializing other parameters of the current output thread, such as the maximum number of batch packets and the like.

And step S502, batch packet receiving of the message queue. And the output module thread sequentially and circularly receives the protocol identification result messages from each input port in batch, processes the messages one by one, and enters the next circulation after all the messages are processed.

And receiving packets from the corresponding message queue in batch according to the current input port ID, determining the input port ID of the next packet receiving, and then executing the step S503 to process the message. After the processing is completed, the process proceeds to step S502.

Step S503, judging whether the message is processed.

If yes, jumping to step S502;

if not, calling a network output interface, executing the step S504, and outputting the identification result; it jumps to step S503.

The embodiment of the invention has the following advantages: firstly, the coupling is low, an asynchronous message interaction mechanism between modules is realized on the basis of a DPDK lock-free message queue, and the coupling is lower; secondly, the expansibility is high, the modularization is realized by reasonable functional module division, and the expansibility is realized by dynamically constructing a data processing model example, namely, the most appropriate processing model can be constructed according to the actual network data environment and the load condition of the module; thirdly, the flexibility is realized, the static protocol identification rule is supported, and the dynamic protocol rule is also supported; fourthly, high efficiency is achieved, and automatic identification processing of high-speed data is achieved on a general x86 platform based on DPDK and Hyperscan.

The embodiment of the invention acquires the data packet based on the data plane development kit, scans the data packet based on two modes of the Hyperscan engine, identifies the protocol running on the TCP protocol and the protocol running on the UDP protocol adopted by the data packet, and can more efficiently and quickly acquire the identification result under the environment of mass data.

Based on the content of the foregoing embodiments, determining that the protocol type of the transport layer is TCP or UDP, and if the protocol type of the transport layer is TCP, scanning the data packet based on the stream mode of the Hyperscan engine, and identifying a protocol running on the TCP protocol adopted by the data packet, further includes: the data packet is reformed into ordered data.

Specifically, as shown in fig. 4, step S412 is further included after step S411 to determine whether there is ordered data.

If not, directly jumping to the step S404; if yes, the data packet is rearranged to obtain the maximum ordered data, step S413 is executed to send the maximum ordered data to the TCP stream scanning thread, and then the process directly jumps to step S404.

The embodiment of the invention can more efficiently and quickly identify the protocol running on the TCP protocol based on the Hyperscan engine by reforming the TCP data packet into the ordered data.

Based on the content of the foregoing embodiments, if the TCP is the TCP, the data packet is scanned based on the stream mode of the Hyperscan engine, and the specific step of identifying the protocol that is used by the data packet and runs on the TCP protocol includes: and scanning the reformed data packet based on the flow mode of the Hyperscan engine and the pre-acquired flow mode protocol identification rule to acquire the identification rule corresponding to the data packet.

Specifically, the embodiment of the present invention performs the TCP flow scanning step. The TCP stream scanning step may be implemented by a TCP stream scanning thread.

And the TCP stream scanning step is used for carrying out cross-packet scanning on the effective load of the TCP data packet, and sending the identified protocol type result to an output thread after the scanning is finished.

Fig. 6 is a flowchart illustrating a TCP flow scanning step in the network protocol identification method according to an embodiment of the present invention. As shown in fig. 6, the TCP flow scanning step specifically includes:

step S600, start. And providing starting resource preparation for all TCP stream scanning threads, and sequentially triggering the starting of all TCP stream scanning threads.

And step S601, initialization. Creating a TCP stream scanning thread on a corresponding logic core according to a pre-specified logic core ID parameter of the TCP stream scanning thread; initializing all input and output ports of the current TCP stream scanning thread, namely binding all initialized message queues associated with the current TCP stream scanning thread; initializing other parameters of the current TCP stream scanning thread, such as a hash table, the maximum number of batch packets, local resources of a Hyperscan engine and the like.

Step S602, flow table node timeout processing.

The embodiment of the invention realizes cross-packet scanning of TCP data by using a hypScan stream mode, namely, realizes cross-packet identification of a protocol under the condition of not carrying out TCP stream recombination. The scan mode requires caching of context, maintained through a hash table. The network data environment is complex, so that the time-out processing is required to be carried out on the failed context.

And step S603, batch packet receiving of the message queue. And the TCP stream scanning module receives the data packets from each input port in batch in a sequential and cyclic manner through the thread, processes the data packets one by one, and enters the next cycle after all the data packets are processed.

And receiving packets in batches from the corresponding message queue according to the current input port ID, determining the input port ID of the next packet receiving, and then entering the step S604 to process the message. After the process is completed, the process proceeds to step S602.

And step S604, judging whether the message is processed.

If yes, jumping to S602;

otherwise, using the latest issued stream mode protocol identification rule base to perform stream mode scanning on the effective load of the current TCP data packet and determine the hit identification rule.

And determining a TCP protocol corresponding to the identification rule as a protocol running on the TCP protocol and adopted by the data packet.

Specifically, a protocol rule table is searched according to the ID of the hit identification rule, the type of the TCP protocol corresponding to the hit identification rule is obtained, the protocol running on the TCP protocol is identified and used as an identification result, and the result is sent to the output thread.

The embodiment of the invention identifies the network protocol based on the Hyperscan engine, and can identify the protocol running on the TCP protocol more efficiently and quickly.

Based on the content of the above embodiments, if the UDP is the UDP, the specific steps of scanning the data packet based on the block mode of the Hyperscan engine, and identifying the protocol used by the data packet and running on the UDP protocol include: scanning the data packet based on the block mode of the Hyperscan engine and the pre-acquired block mode protocol identification rule, and acquiring the identification rule corresponding to the data packet.

Specifically, the embodiment of the present invention performs the UDP block scanning step. The UDP block scanning step may be implemented by a UDP block scanning thread.

And the UDP block scanning step directly scans the payload of the UDP message in a block mode, and sends the identified protocol type result to an output thread after scanning.

Fig. 7 is a flowchart illustrating a UDP block scanning step in the network protocol identification method according to an embodiment of the present invention. As shown in fig. 7, the UDP block scanning step specifically includes:

step S700 starts. Starting resource preparation is provided for all the UDP block scanning threads, and starting of all the UDP block scanning threads is triggered in turn.

And step S701, initialization. Creating a UDP block scanning thread on a corresponding logic core according to a logic core ID parameter of a pre-designated UDP block scanning thread; initializing all input and output ports of the current UDP block scanning thread, namely binding all initialized message queues associated with the current UDP block scanning thread; initializing other parameters of the current UDP block scanning thread, such as the maximum number of batch packets, local resources of a Hyperscan scanning engine and the like.

And step S702, batch packet receiving of the message queue. And the UDP block scanning module thread sequentially and circularly receives the data packets in batches from each input port, processes the data packets one by one, and enters the next circulation after all the data packets are processed.

And receiving packets in batches from the corresponding message queue according to the current input port ID, determining the input port ID of the next packet receiving, and then entering the step S703 to process the message. After the processing is completed, the process proceeds to step S702.

And step S703, judging whether the message is processed.

If yes, jumping to step S702;

if not, executing step S704, performing Hyperscan block mode scanning processing and sending a hit result, that is, using the latest delivered block mode protocol identification rule base, performing block mode scanning on the payload of the current UDP data packet, and determining a hit identification rule.

And determining a UDP protocol corresponding to the identification rule as a protocol which is adopted by the data packet and runs on the UDP protocol.

Specifically, a protocol rule table is searched according to the ID of the hit identification rule, the type of the UDP protocol corresponding to the hit identification rule is obtained, the protocol running on the UDP protocol is identified and used as an identification result, and the result is sent to an output thread; it jumps to step S703.

The embodiment of the invention identifies the network protocol based on the Hyperscan engine, and can identify the protocol running on the UDP protocol more efficiently and quickly.

Based on the content of the above embodiments, the specific step of scanning the reformed packet based on the Hyperscan engine flow pattern and the pre-acquired flow pattern protocol identification rule includes: and if the data packet is judged and known not to be HTTP data, scanning the reformed data packet based on the flow mode of the Hyperscan engine and a pre-acquired flow mode protocol identification rule.

Specifically, the following steps are also included after step S604:

step S605, determine whether the current TCP packet is an end packet of the stream. If yes, executing step S606, deleting the node, and then jumping to step S604; if not, continuing to process the subsequent steps.

And step S607, searching a session node. If not, executing step S508 to create a node; if the result is found, the subsequent steps are continued to be processed.

Step S609, determine whether the HTTP data is present. If not, continuing to process the subsequent steps.

Step S611, Hyperscan stream mode scanning processing and hit result sending, namely using a newly issued stream mode protocol identification rule base to perform stream mode scanning on the payload of the current TCP data packet, determining a hit identification rule, searching a protocol rule table according to the ID of the hit identification rule, obtaining the type of the TCP protocol corresponding to the hit identification rule, identifying the protocol running on the TCP protocol as an identification result, and sending the result to an output thread; it jumps to step S604.

The embodiment of the invention is based on the Hyperscan engine, and can more efficiently and quickly identify the protocol running on the TCP protocol.

Based on the content of the above embodiments, the specific step of scanning the reformed packet based on the Hyperscan engine flow pattern and the pre-acquired flow pattern protocol identification rule includes: and if the data packet is judged and known to be HTTP data, scanning the decoded reformed data packet based on the stream mode of the Hyperscan engine and a pre-acquired stream mode protocol identification rule after HTTP decoding is carried out on the data packet.

Specifically, if the determination result in step S609 is yes, step S610 is executed to perform an HTTP process, that is, an operation such as decoding HTTP data.

After the decoding is completed, step S611 is executed, the Hyperscan stream mode scan processing and the hit result transmission.

After the HTTP data is decoded, the embodiment of the invention identifies the network protocol based on the Hyperscan engine, and can identify the protocol running on the TCP protocol more efficiently and quickly.

Based on the content of the foregoing embodiments, based on the data plane development kit, acquiring the data packet from the network card, and determining the protocol type of the transport layer of the data packet, further include: and preprocessing the IP layer of the data packet.

Specifically, after the data packets are acquired from the network card, the acquired batch data packets are preprocessed through the preprocessing step, and the session management step is executed after the preprocessing. The preprocessing step may be specifically implemented by a preprocessing thread.

The preprocessing thread recombines the IP fragment message, strips the IP protocol head of the IP message and then distributes the IP protocol head to all the session management threads related to the IP message in a balanced manner.

Fig. 8 is a schematic flowchart of a preprocessing step in the network protocol identification method according to an embodiment of the present invention. As shown in fig. 8, the preprocessing step specifically includes:

step S800, start. Providing starting resource preparation for all preprocessing threads, and sequentially triggering the starting of all preprocessing threads;

step S801, initialization. Creating a preprocessing thread on a corresponding logic core according to a logic core ID parameter of a preassigned preprocessing thread; initializing all input and output ports of the current preprocessing thread, namely binding all initialized message queues associated with the current preprocessing thread; initializing other parameters of the current preprocessing thread, such as the maximum number of batch packets and the like.

And S802, receiving packets in batch by the message queue. And the preprocessing thread sequentially and circularly receives the data packets from each input port in batch, preprocesses the data packets one by one and forwards the data packets until all the data packets are processed, and then enters the next circulation.

And receiving packets in batches from the corresponding message queue according to the current input port ID, determining the input port ID of the next packet receiving, and then entering the step S803 to preprocess the message. After the process is completed, the process proceeds to step S802.

Step S803 determines whether the message has been processed. For the network data packets received in bulk, it is necessary to perform preprocessing, i.e. reassembly of IP fragments and stripping of IP headers, one by one. And then carrying out balanced distribution of the same source and the same sink according to the number of the output ports.

Judging whether the message is processed or not:

if yes, go to step S802;

if not, sequentially executing:

step S804, IP fragmentation checking, checking the data packet IP layer head protocol field and the fragmentation flag bit, and determining whether the data packet is a fragmentation packet; if yes, go to step S805; if not, step S807 is performed.

And S805, IP fragment packet recombination, namely searching an IP fragment cache table, and orderly inserting the fragment packets into corresponding positions.

Step S806, checking whether the recombination is completed; if yes, go to substep S807; otherwise, jumping to step S803;

and S807, stripping and forwarding the IP header, and offsetting the data pointer of the complete IP message to directly point to the load part. And meanwhile, balanced distribution is carried out according to the number of output ports. It jumps to step S803.

The embodiment of the invention can judge the transport layer protocol more quickly by recombining the IP fragment message, thereby identifying the protocol running on the TCP or UDP protocol and obtaining the identification result more efficiently and quickly.

Fig. 9 is a schematic structural diagram of a network protocol identification apparatus according to an embodiment of the present invention. Based on the content of the foregoing embodiments, as shown in fig. 9, the apparatus includes a data acquisition module 901, a session management module 902, and a data scanning module 903, where:

a data acquisition module 901, configured to acquire a data packet from a network card based on a data plane development kit;

a session management module 902, configured to determine a protocol type of a transport layer of the data packet, and determine that the protocol type of the transport layer is TCP or UDP;

the data scanning module 903 is configured to scan a data packet based on a stream mode of a Hyperscan engine if the TCP is the TCP, and identify a protocol running on a TCP protocol adopted by the data packet; and if the UDP is adopted, scanning the data packet based on the block mode of the Hyperscan engine, and identifying the protocol which is adopted by the data packet and runs on the UDP protocol.

Specifically, the data acquisition module 901, the session management module 902, and the data scanning module 903 are electrically connected in sequence.

The data acquisition module 901 acquires data from the high-speed network card and distributes the acquired data packets in a balanced manner.

The session management module 902 sequences the TCP streams, merges the maximum ordered data into a jumbo frame, and uniformly forwards the jumbo frame to all TCP stream scanning threads associated therewith, and directly and uniformly forwards the UDP packet to the UDP block scanning thread.

The data scanning module 903 includes two sub-modules, which are respectively used for scanning the data packet based on the stream mode of the Hyperscan engine, identifying the protocol used by the data packet and running on the TCP protocol, and scanning the data packet based on the block mode of the Hyperscan engine, and identifying the protocol used by the data packet and running on the UDP protocol.

The network protocol identification apparatus provided in the embodiments of the present invention is configured to execute the network protocol identification method provided in each of the above embodiments of the present invention, and specific methods and processes for implementing corresponding functions by each module included in the network protocol identification apparatus are described in the embodiments of the network protocol identification method, and are not described herein again.

The network protocol identification device is used for the network protocol identification method of the foregoing embodiments. Therefore, the description and definition in the network protocol identification method in the foregoing embodiments can be used for understanding the execution modules in the embodiments of the present invention.

Fig. 10 is a schematic structural diagram of a network protocol identification apparatus according to an embodiment of the present invention. Based on the content of the above embodiments, as shown in fig. 10, the apparatus includes: the system comprises an operation and maintenance module 10, a data acquisition module 20, a preprocessing module 30, a session management module 40, a TCP stream scanning module 50, a UDP block scanning module 60 and an output module 70.

The TCP stream scanning module 50 and the UDP block scanning module 60 together implement performing step S103.

The data acquisition module 20, the preprocessing module 30, the session management module 40, the TCP stream scanning module 50 or the UDP block scanning module 60, and the output module 70 interact in sequence, the whole data processing process adopts a pipeline model, the front-end function module produces messages, and the back-end function module consumes messages. The data acquisition module 20 acquires data from the high-speed network card and distributes the data to all the preprocessing modules 30 associated with the data acquisition module in a balanced manner. The preprocessing module 30 reassembles the IP fragment packet, and after stripping the IP protocol header of the IP packet, the IP fragment packet is distributed to all session management modules 40 associated with the IP fragment packet in a balanced manner. The session management module 40 orders the TCP flows, merges the largest ordered data into jumbo frames, and forwards it evenly to all TCP flow scanning modules 50 associated therewith. Meanwhile, the session management module 40 forwards the UDP packet directly to the UDP block scanning module 60 in a balanced manner. The TCP flow scanning module 50 performs cross-packet scanning on the payload of the TCP data packet, wherein http decoding is performed before the data with the payload of http protocol is scanned. The UDP block scanning module 60 may directly perform block mode scanning on the payload of the UDP packet. TCP stream scanning module 50 and UDP block scanning module 60 forward the protocol type identified by the scan to output module 70. The output module 70 outputs the result to an external user using a network interface.

The operation and maintenance module 10 interacts with the TCP flow scanning module 50, and the operation and maintenance module 10 receives the dynamic rules, updates the protocol identification rule table, sequentially generates the latest flow pattern rule database, and sends the latest flow pattern rule database to the TCP flow scanning module 50. The TCP flow scanning module 50 updates the local resource of the Hyperscan scanning engine by using the latest rule base issued by the operation and maintenance module 10, and starts to scan data by using the latest rule base.

The operation and maintenance module 10 interacts with the UDP block scanning module 60, and the operation and maintenance module 10 receives the dynamic rule, updates the protocol identification rule table, generates the latest block mode rule database based on the dynamic rule, and sends the latest block mode rule database to the UDP block scanning module 60. The UDP block scanning module 60 updates the local resources of the Hyperscan scanning engine using the latest rule base issued by the operation and maintenance module 10, and starts to scan data using the latest rule base.

And the operation and maintenance module 10 is the most basic module. The operation and maintenance module 10 initializes the resources in an initialization phase, and prepares for starting other modules. And meanwhile, generating a processing model example graph according to the configuration file of the data processing module, and triggering the starting of all other modules.

In the embodiment of the invention, a data processing model example is generated by adopting a cordwood dynamic construction mode, and a plurality of functional modules can exist at the same time except for an operation and maintenance module. And the operation and maintenance module builds a data processing model example graph and initialization resources when initializing according to the specific model configuration file, and builds a specific data processing model example according to the example graph.

Meanwhile, the operation and maintenance module is also responsible for outputting the state and the log of the whole program, receiving the dynamic rule of the user and sending the dynamic rule to the TCP stream scanning module and the UDP block scanning module.

The data acquisition module 20 uses DPDK user plane driver, acquires data from the high-speed network card under the condition of reducing packet loss to the maximum extent, and uniformly forwards the data to the preprocessing module 20 associated with the data acquisition module.

The preprocessing module 30 is mainly responsible for processing the IP layer of the network data packet, i.e. recombining the IP fragments and stripping the IP header, and uniformly forwards the preprocessed data to the session management module 40 associated therewith.

The session management module 40 is responsible for sequencing and reassembling the TCP data, maintains the TCP data stream using the hash table, sequences sequentially-reached TCP data, reassembles the maximum ordered data, generates a huge data frame, and forwards the huge data frame to the TCP stream scanning module 50.

The session management module 40 is also responsible for forwarding UDP, and forwards the received UDP packet to the UDP block scanning module 60 in a balanced manner.

The TCP flow scanning module 60 performs packet-crossing scanning on TCP data by using a flow mode based on a Hyperscan technology, that is, an object scanned each time is not application layer data after TCP is completely reassembled, but is a payload of a single data frame sent by the session management module, so as to achieve higher performance; for HTTP data on TCP, decoding the HTTP data subjected to encoding processing before scanning; the results of hits during the scan are forwarded to output module 70.

The UDP block scanning module 60 is configured to scan TCP data in a block mode based on a Hyperscan technology, that is, a scanned object is a payload of a UDP packet forwarded by the session management module; the results of hits during the scan are forwarded to output module 70.

And the output module 70 forwards the scanning and recognizing results of the TCP stream scanning module 50 and the UDP block scanning module 60 to other external services through a network communication interface.

For example, if the data processing model configuration file specifies that each module starts an instance, the whole system includes 7 task blocks, that is, 1 operation and maintenance module, and 6 data processing task blocks: a data acquisition module 20, a pre-processing module 30, a session management module 40, a TCP stream scanning module 50, a UDP block scanning module 60, and an output module 70. And starting the operation and maintenance module 10 on the No. 1 logic CPU core, reading the configuration file of the data processing model, generating an example graph, and initializing resources such as a network card queue, a memory pool, a message queue and the like required by 6 data processing task blocks. The operation and maintenance module 10 triggers 6 data processing task blocks, and respectively starts an instance on the No. 2-7 logic CPU core. After the starting is finished, the front-end module is used as a producer and the rear-end module is used as a consumer, and the front-end module and the rear-end module are sequentially interacted with each other, so that the automatic protocol identification of the network data packet is realized. Where each data processing task block is both a producer and a consumer. For the dynamic rules issued by the user, the operation and maintenance module instance processes the dynamic rules and then respectively forwards the processed dynamic rules to the TCP stream scanning module 50 instance and the UDP block scanning module 60 instance.

Fig. 11 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention. Based on the content of the above embodiment, as shown in fig. 11, the electronic device may include: a processor (processor)1101, a memory (memory)1102, and a bus 1103; wherein, the processor 1101 and the memory 1102 complete communication with each other through the bus 1103; the processor 1101 is configured to invoke computer program instructions stored in the memory 1102 and executable on the processor 1101 to perform the network protocol identification methods provided by the above-described method embodiments, including, for example: acquiring a data packet from a network card based on a data plane development kit; judging the protocol type of a transmission layer of the data packet, and determining that the protocol type of the transmission layer is TCP or UDP; if the TCP is adopted, scanning the data packet based on the flow mode of the Hyperscan engine, and identifying the protocol which is adopted by the data packet and runs on the TCP protocol; and if the UDP is adopted, scanning the data packet based on the block mode of the Hyperscan engine, and identifying the protocol which is adopted by the data packet and runs on the UDP protocol.

Another embodiment of the present invention discloses a computer program product, the computer program product includes a computer program stored on a non-transitory computer readable storage medium, the computer program includes program instructions, when the program instructions are executed by a computer, the computer can execute the network protocol identification method provided by the above-mentioned method embodiments, for example, the method includes: acquiring a data packet from a network card based on a data plane development kit; judging the protocol type of a transmission layer of the data packet, and determining that the protocol type of the transmission layer is TCP or UDP; if the TCP is adopted, scanning the data packet based on the flow mode of the Hyperscan engine, and identifying the protocol which is adopted by the data packet and runs on the TCP protocol; and if the UDP is adopted, scanning the data packet based on the block mode of the Hyperscan engine, and identifying the protocol which is adopted by the data packet and runs on the UDP protocol.

Furthermore, the logic instructions in the memory 1102 may be implemented in software functional units and stored in a computer readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or make a contribution to the prior art, or may be implemented in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods of the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

Another embodiment of the present invention provides a non-transitory computer-readable storage medium, which stores computer instructions, the computer instructions causing a computer to execute the network protocol identification method provided by the foregoing method embodiments, for example, including: acquiring a data packet from a network card based on a data plane development kit; judging the protocol type of a transmission layer of the data packet, and determining that the protocol type of the transmission layer is TCP or UDP; if the TCP is adopted, scanning the data packet based on the flow mode of the Hyperscan engine, and identifying the protocol which is adopted by the data packet and runs on the TCP protocol; and if the UDP is adopted, scanning the data packet based on the block mode of the Hyperscan engine, and identifying the protocol which is adopted by the data packet and runs on the UDP protocol.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. It is understood that the above-described technical solutions may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method of the above-described embodiments or some parts of the embodiments.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A network protocol identification method, comprising:

if the TCP is adopted, scanning the data packet based on a flow mode of a Hyperscan engine, and identifying a protocol which is adopted by the data packet and runs on a TCP protocol; if the UDP protocol is adopted, scanning the data packet based on a block mode of a Hyperscan engine, and identifying a protocol which is adopted by the data packet and runs on the UDP protocol;

if the TCP is adopted, scanning the data packet based on a flow mode of a Hyperscan engine, and identifying a protocol which is adopted by the data packet and runs on a TCP protocol comprises the following specific steps:

determining a TCP protocol corresponding to the identification rule as a protocol which is adopted by the data packet and runs on the TCP protocol;

if the data packet is UDP, scanning the data packet based on a block mode of a Hyperscan engine, and identifying a protocol which is adopted by the data packet and runs on a UDP protocol comprises the following specific steps:

2. The method according to claim 1, wherein the determining that the protocol type of the transport layer is TCP or UDP, and if TCP, scanning the packet based on a stream pattern of a Hyperscan engine, and identifying a protocol used by the packet and running on top of a TCP protocol, further comprises:

and reforming the data packet into ordered data.

3. The method according to claim 1, wherein the step of scanning the reformed packet based on the Hyperscan engine flow pattern and the pre-obtained flow pattern protocol identification rule comprises:

4. The method according to claim 3, wherein the step of scanning the reformed packet based on the Hyperscan engine flow pattern and the pre-obtained flow pattern protocol identification rule comprises:

5. The method according to any one of claims 1 to 4, wherein between the step of acquiring the data packet from the network card and the step of determining the protocol type of the transport layer of the data packet based on the data plane development kit, the method further comprises:

and preprocessing the IP layer of the data packet.

6. A network protocol identification device, comprising:

the data scanning module is used for scanning the data packet based on a flow mode of a Hyperscan engine if the TCP is adopted and identifying a protocol which is adopted by the data packet and runs on a TCP protocol; if the UDP protocol is adopted, scanning the data packet based on a block mode of a Hyperscan engine, and identifying a protocol which is adopted by the data packet and runs on the UDP protocol;

the data scanning module is specifically used for scanning the reformed data packet based on a flow mode of a Hyperscan engine and a pre-acquired flow mode protocol identification rule to acquire an identification rule corresponding to the data packet;

the data scanning module is further specifically configured to scan the data packet based on a block mode of a Hyperscan engine and a pre-acquired block mode protocol identification rule, and acquire an identification rule corresponding to the data packet;

7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the network protocol identification method according to any of claims 1 to 5 when executing the program.

8. A non-transitory computer readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, implementing the steps of the network protocol identification method according to any one of claims 1 to 5.