CN112749028A - Network traffic processing method, related device and readable storage medium - Google Patents
Network traffic processing method, related device and readable storage medium Download PDFInfo
- Publication number
- CN112749028A CN112749028A CN202110033277.4A CN202110033277A CN112749028A CN 112749028 A CN112749028 A CN 112749028A CN 202110033277 A CN202110033277 A CN 202110033277A CN 112749028 A CN112749028 A CN 112749028A
- Authority
- CN
- China
- Prior art keywords
- network
- thread
- network data
- network traffic
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims description 20
- 238000001514 detection method Methods 0.000 claims abstract description 44
- 238000000034 method Methods 0.000 claims abstract description 35
- 238000012545 processing Methods 0.000 claims description 30
- 238000011161 development Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 abstract description 14
- 238000005111 flow chemistry technique Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- ZPUCINDJVBIVPJ-LJISPDSOSA-N cocaine Chemical compound O([C@H]1C[C@@H]2CC[C@@H](N2C)[C@H]1C(=O)OC)C(=O)C1=CC=CC=C1 ZPUCINDJVBIVPJ-LJISPDSOSA-N 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
Abstract
The application discloses a network flow processing method, related equipment and a readable storage medium, in a multi-thread network intrusion detection system, a network flow receiving queue and a connection tracking table are arranged for each thread, network data of the same network connection is received by one network flow receiving queue, and for the same network connection, a plurality of threads can not process the network data, and when each thread processes the network data, only the connection tracking table corresponding to each thread needs to be operated, the threads can not interfere with each other, so that the threads in the system can process the network data in the network flow receiving queues in parallel, and further the performance of the multi-thread network intrusion detection system is improved.
Description
Technical Field
The present application relates to the field of network security technologies, and in particular, to a network traffic processing method, a related device, and a readable storage medium.
Background
With the rapid development of internet services, Network traffic is increasing explosively, and it is necessary to fully analyze Network traffic to detect an attack existing in a Network, so that NIDS (Network Intrusion Detection System) is produced at the outset. When the NIDS performs network intrusion detection, the NIDS generally includes two processes of processing network traffic and rule matching, and in the process of processing network traffic, it needs to perform connection tracking on network data (messages, data packets, etc.), determines and stores connection tracking information of the network data, so that in the subsequent rule matching process, the network data of each network connection is recombined based on the connection tracking information of the network data to obtain the network traffic of the network connection, and intrusion detection is performed on the network traffic of each network connection. Currently, NIDS systems use a unified connection tracking table to store connection tracking information for network data.
With the development of multi-core processors, multi-threaded NIDS (such as NIDS implemented based on a worker mode of a surfaca engine) are also widely used, in the processing process of network traffic, different network data of the same network connection may be received and processed by different threads, in this case, different threads may operate the connection tracking table at the same time, and in order to ensure the correctness of the connection tracking information, the connection tracking table needs to be locked, which results in that only one thread may operate the connection tracking table at each time, and the performance of the multi-threaded NIDS is affected.
Therefore, how to improve the processing procedure of the multi-threaded NIDS on the network traffic to improve the performance of the multi-threaded NIDS becomes a technical problem to be solved by those skilled in the art.
Disclosure of Invention
In view of the foregoing problems, the present application provides a network traffic processing method, a related device, and a readable storage medium. The specific scheme is as follows:
a network traffic processing method is applied to a multi-thread network intrusion detection system, wherein each thread in the system corresponds to a network traffic receiving queue, network data of the same network connection is received by one network traffic receiving queue, and each thread in the system corresponds to a connection tracking table, and the method comprises the following steps:
for each thread, acquiring first network data from a network traffic receiving queue corresponding to the thread;
analyzing the first network data to obtain connection tracking information of the first network data;
and storing the connection tracking information of the first network data into a connection tracking table corresponding to the thread.
Optionally, the manner of setting the corresponding network traffic receiving queue for each thread is as follows:
and setting a corresponding network flow receiving queue for each thread based on a network intrusion detection engine and a data plane development tool set.
Optionally, the setting a corresponding network traffic receiving queue for each thread based on the network intrusion detection engine and the data plane development toolset includes:
initializing the data plane development tool set, and setting a network flow receiving queue in a network card, wherein the network flow receiving queue is consistent with the number of working threads in the network intrusion detection engine;
and setting a working thread in a network intrusion detection engine corresponding to each network traffic receiving queue as a thread corresponding to the network traffic receiving queue.
Optionally, each thread in the system corresponds to a timer, and then the method further includes:
for each thread, after the connection tracking information of the first network data is stored in a connection tracking table corresponding to the thread, starting the timer to start timing;
if the connection tracking information of the second network data is stored in the connection tracking table corresponding to the thread before the timing duration of the timer reaches the preset duration, restarting the timer after the connection tracking information of the second network data is stored in the connection tracking table corresponding to the thread;
if the connection tracking information of the second network data is not stored in the connection tracking table corresponding to the thread when the timing duration of the timer reaches the preset duration, deleting the connection tracking information of the network connection corresponding to the first network data recorded in the connection tracking table corresponding to the thread;
the second network data and the first network data are network data connected with the same network.
Optionally, before each thread acquires the first network data from the network traffic receiving queue corresponding to the thread, the method further includes:
collecting network data;
for each network data, determining a target network traffic receiving queue corresponding to the network data, and receiving the network data by the target network traffic receiving queue;
the target network traffic receiving queue is one of the network traffic receiving queues corresponding to the threads; each network data of the same network connection corresponds to the same target network traffic receiving queue.
Optionally, the determining, for each network data, a target network traffic receiving queue corresponding to the network data includes:
calculating a symmetric hash value of the network data;
determining a network traffic receiving queue corresponding to the symmetric hash value of the network data as a target network traffic receiving queue in the network traffic receiving queues corresponding to the threads;
and the symmetric hash values of the network data connected with the same network are the same.
A network traffic processing apparatus applied to a multi-threaded network intrusion detection system, wherein each thread in the system corresponds to a network traffic receiving queue, network data of the same network connection is received by one network traffic receiving queue, and each thread in the system corresponds to a connection tracking table, the apparatus comprising:
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring first network data from a network traffic receiving queue corresponding to each thread;
the analysis unit is used for analyzing the first network data to obtain the connection tracking information of the first network data;
and the storage unit is used for storing the connection tracking information of the first network data into a connection tracking table corresponding to the thread.
Optionally, the apparatus further comprises:
and the network flow receiving queue setting unit is used for setting a corresponding network flow receiving queue for each thread based on the network intrusion detection engine and the data plane development tool set.
Optionally, the network traffic receiving queue setting unit includes:
the initialization unit is used for initializing the data plane development tool set and setting a network flow receiving queue in a network card, wherein the network flow receiving queue is consistent with the number of working threads in the network intrusion detection engine;
and the setting unit is used for setting one working thread in the network intrusion detection engine corresponding to the network traffic receiving queue as a thread corresponding to the network traffic receiving queue aiming at each network traffic receiving queue.
Optionally, each thread in the system corresponds to a timer, and then the apparatus further includes:
the timing processing unit is used for starting the timer to start timing after storing the connection tracking information of the first network data into a connection tracking table corresponding to each thread; if the connection tracking information of the second network data is stored in the connection tracking table corresponding to the thread before the timing duration of the timer reaches the preset duration, restarting the timer after the connection tracking information of the second network data is stored in the connection tracking table corresponding to the thread; if the connection tracking information of the second network data is not stored in the connection tracking table corresponding to the thread when the timing duration of the timer reaches the preset duration, deleting the connection tracking information of the network connection corresponding to the first network data recorded in the connection tracking table corresponding to the thread; the second network data and the first network data are network data connected with the same network.
Optionally, the apparatus further comprises:
the network data acquisition unit is used for acquiring network data before each thread acquires first network data from a network traffic receiving queue corresponding to the thread;
a target network traffic receiving queue determining unit, configured to determine, for each network data, a target network traffic receiving queue corresponding to the network data, and receive the network data by the target network traffic receiving queue; the target network traffic receiving queue is one of the network traffic receiving queues corresponding to the threads; each network data of the same network connection corresponds to the same target network traffic receiving queue.
Optionally, the target network traffic receiving queue determining unit includes:
the computing unit is used for computing a symmetric hash value of the network data;
a determining unit, configured to determine that, in the network traffic receiving queues corresponding to the threads, a network traffic receiving queue corresponding to a symmetric hash value of the network data is a target network traffic receiving queue; and the symmetric hash values of the network data connected with the same network are the same.
A network flow processing system comprises a plurality of threads, each thread in the system corresponds to a network flow receiving queue, network data of the same network connection is received by one network flow receiving queue, and each thread in the system corresponds to a connection tracking table.
A network traffic processing device comprising a memory and a processor;
the memory is used for storing programs;
the processor is configured to execute the program to implement the steps of the network traffic processing method.
A readable storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of the network traffic processing method as described above.
By means of the technical scheme, the application discloses a network flow processing method, related equipment and a readable storage medium, in a multi-thread network intrusion detection system, a network flow receiving queue and a connection tracking table are arranged for each thread, network data of the same network connection are received by the network flow receiving queue, multiple threads cannot process the network data of the same network connection aiming at the same network connection, and when each thread processes the network data, only the connection tracking table corresponding to the thread needs to be operated, the threads cannot interfere with each other, the threads in the system can process the network data in the network flow receiving queues in parallel, and further the performance of the multi-thread network intrusion detection system is improved.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 is a schematic structural diagram of a multi-threaded network intrusion detection system according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a network traffic processing method disclosed in an embodiment of the present application;
FIG. 3 is a block diagram of another multithreaded network intrusion detection system as disclosed in an embodiment of the present application;
fig. 4 is a schematic flow chart illustrating another network traffic processing method disclosed in the embodiment of the present application;
fig. 5 is a schematic structural diagram of a network traffic processing apparatus according to an embodiment of the present application;
fig. 6 is a block diagram of a hardware structure of a network traffic processing device according to an embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
For convenience of understanding, the structure of the multi-threaded network intrusion detection system is first described in the present application, and referring to fig. 1, fig. 1 is a schematic structural diagram of a multi-threaded network intrusion detection system disclosed in an embodiment of the present application.
As shown in fig. 1, the multi-threaded network intrusion detection system includes N threads, each thread corresponds to a network traffic receiving queue, and each thread corresponds to a connection tracking table.
As an implementation manner, a manner of setting a corresponding network traffic receiving queue for each thread is proposed in the present application, that is, a corresponding network traffic receiving queue is set for each thread based on a network intrusion detection engine and a data plane development tool set. Specifically, the data plane development tool set may be initialized, and a network traffic receiving queue that is consistent with the number of working threads in the network intrusion detection engine is set in a network card; and setting a working thread in a network intrusion detection engine corresponding to each network traffic receiving queue as a thread corresponding to the network traffic receiving queue.
It should be noted that the network intrusion detection engine may be a surfata engine, a Snort engine, or the like, and the Data Plane Development toolset may be a DPDK (Data Plane Development Kit), or the like.
As an example, in the present application, a corresponding network traffic receiving queue may be set for each thread based on a surfata engine worker mode and a DPDK (Data Plane Development Kit). Specifically, the subcatea engine is used as a main engine, and the DPDK is called as a lib library. When the SURICA engine is initialized, initializing the DPDK, setting network traffic receiving queues in a network card, wherein the number of the network traffic receiving queues is consistent with that of workers in the SURICA engine, and setting one worker in the SURICA engine corresponding to each network traffic receiving queue as a thread corresponding to each network traffic receiving queue.
As an implementable manner, a connection tracking table may be created for each thread in the present application. It should be noted that each connection tracking table can only be operated by the corresponding thread.
In addition, in the present application, network data of the same network connection is received by one network traffic receiving queue. That is, there may be only one network data of a network connection in one network traffic receive queue, or there may be network data of multiple network connections, and the network data of the same network connection is not received by two or more network traffic receive queues.
In the application, before performing network traffic processing or during performing network traffic processing, a system further needs to acquire network data, determine a target network traffic receiving queue corresponding to the network data for each network data, and receive the network data by the target network traffic receiving queue; the target network traffic receiving queue is one of the network traffic receiving queues corresponding to the threads; each network data of the same network connection corresponds to the same target network traffic receiving queue.
In this application, network data of the same network connection may be received by the same network traffic receiving queue using a symmetric hash (systematic RSS) technology, and based on this, for each network data, the manner of determining the target network traffic receiving queue corresponding to the network data may include: calculating a symmetric hash value of the network data; determining a network traffic receiving queue corresponding to the symmetric hash value of the network data as a target network traffic receiving queue in the network traffic receiving queues corresponding to the threads; and the symmetric hash values of the network data connected with the same network are the same.
In addition, with respect to the symmetric hashing technique, any version may be adopted in the present application, for example, dpdk18.11 version. In this version, the inventor performed relevant verification on both the 82599 network card and the X710 network card of intel, and thus it was possible to receive network data of the same network connection from the same network traffic reception queue. Specifically, 82599 uses a scheme for modifying the default hash key of the network card; x710 uses Microsoft Toeplitz algorithm (Microsoft Toeplitz Based Hash) to directly replace the Hash function.
In addition, each thread also comprises a network data analysis module and a rule matching module, wherein the network data analysis module is used for analyzing the network data acquired from the network traffic receiving queue to obtain connection tracking information, and the rule matching module is used for performing rule matching on the network traffic based on the connection tracking information in the connection tracking table to obtain an intrusion detection result.
Based on the multi-thread network intrusion detection system, the inventor provides a network traffic processing method. Next, a network traffic processing method provided by the present application is described by the following embodiments.
Referring to fig. 2, fig. 2 is a schematic flowchart of a network traffic processing method disclosed in the embodiment of the present application, where the method is applied to the multi-threaded network intrusion detection system, and the method may include:
step S201: and aiming at each thread, acquiring first network data from a network traffic receiving queue corresponding to the thread.
In this application, network data of the same network connection is received by one network traffic receive queue. That is, there may be only one network data of a network connection in one network traffic receive queue, or there may be network data of multiple network connections, and the network data of the same network connection is not received by two or more network traffic receive queues. The first network data is any network data already received in the network traffic receiving queue corresponding to the thread, and the network data may be network data of any network connection existing in the network traffic receiving queue, and may be in various forms, such as a data packet, and the like.
Step S202: and analyzing the first network data to obtain the connection tracking information of the first network data.
In the application, after a thread acquires first network data from a network traffic receiving queue corresponding to the thread, the thread needs to analyze the first network data to obtain connection tracking information of the first network data, where the connection tracking information of the first network data is used to indicate network traffic to which the first network data belongs.
Step S203: and storing the connection tracking information of the first network data into a connection tracking table corresponding to the thread.
In the present application, one thread can only operate the connection tracking table corresponding to the thread, and therefore, after the first network data is analyzed to obtain the connection tracking information of the first network data, the thread needs to store the connection tracking information of the first network data in the connection tracking table corresponding to the thread.
It should be noted that the connection tracking table may be a flow-manager flow table.
The embodiment discloses a network traffic processing method, in a multi-thread network intrusion detection system, a network traffic receiving queue and a connection tracking table are arranged for each thread, network data of the same network connection is received by one network traffic receiving queue, and network data of the same network connection is not received by two or more network traffic receiving queues, so that for the same network connection, multiple threads do not exist for processing the network data, and when each thread processes the network data, only the connection tracking table corresponding to each thread needs to be operated, the threads do not interfere with each other, the network data in each network traffic receiving queue is processed in parallel by each thread in the system, and further the performance of the multi-thread network intrusion detection system is improved.
Referring to fig. 3, fig. 3 is a schematic structural diagram of another multi-threaded network intrusion detection system disclosed in this embodiment of the present application, and as shown in fig. 3, the multi-threaded network intrusion detection system includes N threads, each thread corresponds to a network traffic receiving queue, network data of the same network connection is received by one network traffic receiving queue, each thread corresponds to a connection tracking table, and each thread corresponds to a timer, which can be implemented in various ways. As an example, when the system sets a corresponding network traffic receiving queue for each thread based on the surfata engine worker mode and the DPDK, the timer may be a timer in the DPDK.
In another embodiment of the present application, based on the above multithreading network intrusion detection system, the present inventors propose another network traffic processing method. Referring to fig. 4, fig. 4 is a schematic flowchart of another network traffic processing method disclosed in the embodiment of the present application, where the method may include:
step S401: for each thread, after the connection tracking information of the first network data is stored in a connection tracking table corresponding to the thread, starting the timer to start timing;
step S402: and if the connection tracking information of the second network data is stored in the connection tracking table corresponding to the thread before the timing duration of the timer reaches the preset duration, restarting the timer after the connection tracking information of the second network data is stored in the connection tracking table corresponding to the thread.
It should be noted that the second network data and the first network data are network data connected to the same network.
Step S403: and if the connection tracking information of the second network data is not stored in the connection tracking table corresponding to the thread when the timing duration of the timer reaches the preset duration, deleting the connection tracking information of the network connection corresponding to the first network data recorded in the connection tracking table corresponding to the thread.
This step illustrates that the traffic processing of the network connection corresponding to the first network data is overtime, and this overtime may be caused by that all the traffic processing of the network connection corresponding to the first network data is completed, or may be caused by that other network data of the network connection corresponding to the first network data is not successfully received into the network traffic receiving queue corresponding to the first network data due to a network failure.
In this embodiment, the flow timeout may be managed within each thread itself.
In summary, if the network intrusion detection system with multiple threads adopts the network traffic processing method disclosed in the present application, the performance of the whole system will also achieve the effect of horizontal increase along with the increase of the number of threads.
The network traffic processing device disclosed in the embodiment of the present application is described below, and the network traffic processing device described below and the network traffic processing method described above may be referred to correspondingly.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a network traffic processing device disclosed in the embodiment of the present application. As shown in fig. 5, the network traffic processing apparatus is applied to a multi-threaded network intrusion detection system, where each thread in the system corresponds to a network traffic receiving queue, network data of the same network connection is received by one network traffic receiving queue, and each thread in the system corresponds to a connection tracking table, and the network traffic processing apparatus may include:
an obtaining unit 11, configured to obtain, for each thread, first network data from a network traffic receiving queue corresponding to the thread;
the analyzing unit 12 is configured to analyze the first network data to obtain connection tracking information of the first network data;
a storage unit 13, configured to store the connection tracking information of the first network data in a connection tracking table corresponding to the thread.
Optionally, the apparatus further comprises:
and the network flow receiving queue setting unit is used for setting a corresponding network flow receiving queue for each thread based on the network intrusion detection engine and the data plane development tool set.
Optionally, the network traffic receiving queue setting unit includes:
the initialization unit is used for initializing the data plane development tool set and setting a network flow receiving queue in a network card, wherein the network flow receiving queue is consistent with the number of working threads in the network intrusion detection engine;
and the setting unit is used for setting one working thread in the network intrusion detection engine corresponding to the network traffic receiving queue as a thread corresponding to the network traffic receiving queue aiming at each network traffic receiving queue.
Optionally, each thread in the system corresponds to a timer, and then the apparatus further includes:
the timing processing unit is used for starting the timer to start timing after storing the connection tracking information of the first network data into a connection tracking table corresponding to each thread; if the connection tracking information of the second network data is stored in the connection tracking table corresponding to the thread before the timing duration of the timer reaches the preset duration, restarting the timer after the connection tracking information of the second network data is stored in the connection tracking table corresponding to the thread; if the connection tracking information of the second network data is not stored in the connection tracking table corresponding to the thread when the timing duration of the timer reaches the preset duration, deleting the connection tracking information of the network connection corresponding to the first network data recorded in the connection tracking table corresponding to the thread; the second network data and the first network data are network data connected with the same network.
Optionally, the apparatus further comprises:
the network data acquisition unit is used for acquiring network data before each thread acquires first network data from a network traffic receiving queue corresponding to the thread;
a target network traffic receiving queue determining unit, configured to determine, for each network data, a target network traffic receiving queue corresponding to the network data, and receive the network data by the target network traffic receiving queue; the target network traffic receiving queue is one of the network traffic receiving queues corresponding to the threads; each network data of the same network connection corresponds to the same target network traffic receiving queue.
Optionally, the target network traffic receiving queue determining unit includes:
the computing unit is used for computing a symmetric hash value of the network data;
a determining unit, configured to determine that, in the network traffic receiving queues corresponding to the threads, a network traffic receiving queue corresponding to a symmetric hash value of the network data is a target network traffic receiving queue; and the symmetric hash values of the network data connected with the same network are the same.
Referring to fig. 6, fig. 6 is a block diagram of a hardware structure of a network traffic processing device according to an embodiment of the present application, and referring to fig. 6, the hardware structure of the network traffic processing device may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;
in the embodiment of the application, the number of the processor 1, the communication interface 2, the memory 3 and the communication bus 4 is at least one, and the processor 1, the communication interface 2 and the memory 3 complete mutual communication through the communication bus 4;
the processor 1 may be a central processing unit CPU, or an application Specific Integrated circuit asic, or one or more Integrated circuits configured to implement embodiments of the present invention, etc.;
the memory 3 may include a high-speed RAM memory, and may further include a non-volatile memory (non-volatile memory) or the like, such as at least one disk memory;
wherein the memory stores a program and the processor can call the program stored in the memory, the program for:
for each thread, acquiring first network data from a network traffic receiving queue corresponding to the thread;
analyzing the first network data to obtain connection tracking information of the first network data;
and storing the connection tracking information of the first network data into a connection tracking table corresponding to the thread.
Alternatively, the detailed function and the extended function of the program may be as described above.
Embodiments of the present application further provide a readable storage medium, where a program suitable for being executed by a processor may be stored, where the program is configured to:
for each thread, acquiring first network data from a network traffic receiving queue corresponding to the thread;
analyzing the first network data to obtain connection tracking information of the first network data;
and storing the connection tracking information of the first network data into a connection tracking table corresponding to the thread.
Alternatively, the detailed function and the extended function of the program may be as described above.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A network traffic processing method, applied to a multi-threaded network intrusion detection system, wherein each thread in the system corresponds to a network traffic receiving queue, network data of the same network connection is received by one network traffic receiving queue, and each thread in the system corresponds to a connection tracking table, the method comprising:
for each thread, acquiring first network data from a network traffic receiving queue corresponding to the thread;
analyzing the first network data to obtain connection tracking information of the first network data;
and storing the connection tracking information of the first network data into a connection tracking table corresponding to the thread.
2. The method of claim 1, wherein the corresponding network traffic receive queue is set for each thread as follows:
and setting a corresponding network flow receiving queue for each thread based on a network intrusion detection engine and a data plane development tool set.
3. The method of claim 2, wherein setting a corresponding network traffic receive queue for each thread based on the network intrusion detection engine and the data plane development toolset comprises:
initializing the data plane development tool set, and setting a network flow receiving queue in a network card, wherein the network flow receiving queue is consistent with the number of working threads in the network intrusion detection engine;
and setting a working thread in a network intrusion detection engine corresponding to each network traffic receiving queue as a thread corresponding to the network traffic receiving queue.
4. The method of claim 1, wherein there is one timer for each thread in the system, the method further comprising:
for each thread, after the connection tracking information of the first network data is stored in a connection tracking table corresponding to the thread, starting the timer to start timing;
if the connection tracking information of the second network data is stored in the connection tracking table corresponding to the thread before the timing duration of the timer reaches the preset duration, restarting the timer after the connection tracking information of the second network data is stored in the connection tracking table corresponding to the thread;
if the connection tracking information of the second network data is not stored in the connection tracking table corresponding to the thread when the timing duration of the timer reaches the preset duration, deleting the connection tracking information of the network connection corresponding to the first network data recorded in the connection tracking table corresponding to the thread;
the second network data and the first network data are network data connected with the same network.
5. The method of claim 1, wherein before each thread retrieves first network data from a network traffic receive queue corresponding to the thread, the method further comprises:
collecting network data;
for each network data, determining a target network traffic receiving queue corresponding to the network data, and receiving the network data by the target network traffic receiving queue;
the target network traffic receiving queue is one of the network traffic receiving queues corresponding to the threads; each network data of the same network connection corresponds to the same target network traffic receiving queue.
6. The method of claim 5, wherein the determining, for each network data, a target network traffic receive queue corresponding to the network data comprises:
calculating a symmetric hash value of the network data;
determining a network traffic receiving queue corresponding to the symmetric hash value of the network data as a target network traffic receiving queue in the network traffic receiving queues corresponding to the threads;
and the symmetric hash values of the network data connected with the same network are the same.
7. A network traffic processing apparatus, applied to a multi-threaded network intrusion detection system, wherein each thread in the system corresponds to a network traffic receiving queue, network data of the same network connection is received by one network traffic receiving queue, and each thread in the system corresponds to a connection tracking table, the apparatus comprising:
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring first network data from a network traffic receiving queue corresponding to each thread;
the analysis unit is used for analyzing the first network data to obtain the connection tracking information of the first network data;
and the storage unit is used for storing the connection tracking information of the first network data into a connection tracking table corresponding to the thread.
8. A network traffic processing system is characterized in that the system comprises a plurality of threads, each thread in the system corresponds to a network traffic receiving queue, network data of the same network connection is received by one network traffic receiving queue, and each thread in the system corresponds to a connection tracking table.
9. A network traffic processing device comprising a memory and a processor;
the memory is used for storing programs;
the processor, configured to execute the program, and implement the steps of the network traffic processing method according to any one of claims 1 to 7.
10. A readable storage medium, having stored thereon a computer program, characterized in that the computer program, when being executed by a processor, carries out the steps of the network traffic processing method according to any of the claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110033277.4A CN112749028A (en) | 2021-01-11 | 2021-01-11 | Network traffic processing method, related device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110033277.4A CN112749028A (en) | 2021-01-11 | 2021-01-11 | Network traffic processing method, related device and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112749028A true CN112749028A (en) | 2021-05-04 |
Family
ID=75650722
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110033277.4A Pending CN112749028A (en) | 2021-01-11 | 2021-01-11 | Network traffic processing method, related device and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112749028A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113518130A (en) * | 2021-08-19 | 2021-10-19 | 北京航空航天大学 | Packet burst load balancing method and system based on multi-core processor |
| CN115037782A (en) * | 2022-08-12 | 2022-09-09 | 南瑞轨道交通技术有限公司 | Real-time data transmission method and system based on dual-network dual-link |
| CN115150198A (en) * | 2022-09-01 | 2022-10-04 | 国汽智控(北京)科技有限公司 | Vehicle-mounted intrusion detection system, method, electronic device and storage medium |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1937574A (en) * | 2005-09-19 | 2007-03-28 | 北京大学 | Network flow classifying, state tracking and message processing device and method |
| US20080181245A1 (en) * | 2007-01-31 | 2008-07-31 | Claude Basso | System and Method for Multicore Communication Processing |
| US20100293353A1 (en) * | 2009-05-18 | 2010-11-18 | Sonnier David P | Task queuing in a network communications processor architecture |
| CN103166845A (en) * | 2013-03-01 | 2013-06-19 | 华为技术有限公司 | Data processing method and device |
| CN104767659A (en) * | 2015-04-28 | 2015-07-08 | 重庆邮电大学 | Predictive type dynamic high-speed network flow detecting method and device |
| CN107317759A (en) * | 2017-06-13 | 2017-11-03 | 国家计算机网络与信息安全管理中心 | A kind of thread-level dynamic equalization dispatching method of network interface card |
| CN108647104A (en) * | 2018-05-15 | 2018-10-12 | 北京五八信息技术有限公司 | request processing method, server and computer readable storage medium |
| CN109117270A (en) * | 2018-08-01 | 2019-01-01 | 湖北微源卓越科技有限公司 | The method for improving network packet treatment effeciency |
| CN109407970A (en) * | 2018-09-12 | 2019-03-01 | 新华三技术有限公司成都分公司 | Read-write requests processing method, device and electronic equipment |
| CN109688069A (en) * | 2018-12-29 | 2019-04-26 | 杭州迪普科技股份有限公司 | A kind of method, apparatus, equipment and storage medium handling network flow |
| CN110022330A (en) * | 2018-01-09 | 2019-07-16 | 阿里巴巴集团控股有限公司 | For the processing method of network packet, device and electronic equipment |
| CN110022267A (en) * | 2018-01-09 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Processing method of network data packets and device |
| CN111193668A (en) * | 2019-12-10 | 2020-05-22 | 中移（杭州）信息技术有限公司 | Flow distribution method and device, computer equipment and storage medium |
| CN112000429A (en) * | 2020-08-06 | 2020-11-27 | 北京浪潮数据技术有限公司 | Connection tracking deletion method, device and equipment of cloud platform management platform |
-
2021
- 2021-01-11 CN CN202110033277.4A patent/CN112749028A/en active Pending
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1937574A (en) * | 2005-09-19 | 2007-03-28 | 北京大学 | Network flow classifying, state tracking and message processing device and method |
| US20080181245A1 (en) * | 2007-01-31 | 2008-07-31 | Claude Basso | System and Method for Multicore Communication Processing |
| US20100293353A1 (en) * | 2009-05-18 | 2010-11-18 | Sonnier David P | Task queuing in a network communications processor architecture |
| CN103166845A (en) * | 2013-03-01 | 2013-06-19 | 华为技术有限公司 | Data processing method and device |
| CN104767659A (en) * | 2015-04-28 | 2015-07-08 | 重庆邮电大学 | Predictive type dynamic high-speed network flow detecting method and device |
| CN107317759A (en) * | 2017-06-13 | 2017-11-03 | 国家计算机网络与信息安全管理中心 | A kind of thread-level dynamic equalization dispatching method of network interface card |
| CN110022330A (en) * | 2018-01-09 | 2019-07-16 | 阿里巴巴集团控股有限公司 | For the processing method of network packet, device and electronic equipment |
| CN110022267A (en) * | 2018-01-09 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Processing method of network data packets and device |
| CN108647104A (en) * | 2018-05-15 | 2018-10-12 | 北京五八信息技术有限公司 | request processing method, server and computer readable storage medium |
| CN109117270A (en) * | 2018-08-01 | 2019-01-01 | 湖北微源卓越科技有限公司 | The method for improving network packet treatment effeciency |
| CN109407970A (en) * | 2018-09-12 | 2019-03-01 | 新华三技术有限公司成都分公司 | Read-write requests processing method, device and electronic equipment |
| CN109688069A (en) * | 2018-12-29 | 2019-04-26 | 杭州迪普科技股份有限公司 | A kind of method, apparatus, equipment and storage medium handling network flow |
| CN111193668A (en) * | 2019-12-10 | 2020-05-22 | 中移（杭州）信息技术有限公司 | Flow distribution method and device, computer equipment and storage medium |
| CN112000429A (en) * | 2020-08-06 | 2020-11-27 | 北京浪潮数据技术有限公司 | Connection tracking deletion method, device and equipment of cloud platform management platform |
Non-Patent Citations (2)
| Title |
|---|
| 王明贞;赵国鸿;唐勇;: "基于多核网络处理器的高效流管理技术研究", 小型微型计算机系统, no. 12, 15 December 2012 (2012-12-15) * |
| 田野;张玉军;: "多线程实现网络入侵检测的负载均衡", 微电子学与计算机, no. 03 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113518130A (en) * | 2021-08-19 | 2021-10-19 | 北京航空航天大学 | Packet burst load balancing method and system based on multi-core processor |
| CN115037782A (en) * | 2022-08-12 | 2022-09-09 | 南瑞轨道交通技术有限公司 | Real-time data transmission method and system based on dual-network dual-link |
| CN115150198A (en) * | 2022-09-01 | 2022-10-04 | 国汽智控(北京)科技有限公司 | Vehicle-mounted intrusion detection system, method, electronic device and storage medium |
| CN115150198B (en) * | 2022-09-01 | 2022-11-08 | 国汽智控(北京)科技有限公司 | Vehicle-mounted intrusion detection system, method, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112749028A (en) | Network traffic processing method, related device and readable storage medium | |
| CN108804299B (en) | Application program exception handling method and device | |
| CN106657057B (en) | Anti-crawler system and method | |
| US9280438B2 (en) | Autonomic hotspot profiling using paired performance sampling | |
| WO2019200799A1 (en) | Short message verification code pushing method, electronic device and readable storage medium | |
| US10216600B2 (en) | Linking single system synchronous inter-domain transaction activity | |
| CA2560747A1 (en) | Profile based capture component for monitoring events in applications | |
| CN111641621B (en) | Internet of things security event identification method and device and computer equipment | |
| US20200012784A1 (en) | Profile generation device, attack detection device, profile generation method, and profile generation computer program | |
| US20170104771A1 (en) | Network monitoring device, network monitoring method, and network monitoring program | |
| WO2020232871A1 (en) | Method and device for microservice dependency analysis | |
| US11531676B2 (en) | Method and system for anomaly detection based on statistical closed-form isolation forest analysis | |
| US11108787B1 (en) | Securing a network device by forecasting an attack event using a recurrent neural network | |
| CN113992340B (en) | User abnormal behavior identification method, device, equipment and storage medium | |
| CN105550628A (en) | Fingerprint inputting and recording method and apparatus | |
| US20180189498A1 (en) | Device monitoring policy | |
| CN111885034B (en) | Internet of things attack event tracking method and device and computer equipment | |
| CN110691090B (en) | Website detection method, device, equipment and storage medium | |
| Zali et al. | Real-time intrusion detection alert correlation and attack scenario extraction based on the prerequisite-consequence approach | |
| CN107688481B (en) | Multi-node-supporting KVM virtual machine hiding process detection system | |
| CN113660134B (en) | Port detection method, device, electronic device and storage medium | |
| CN108234341B (en) | Nginx dynamic passive current limiting method and system based on equipment fingerprint | |
| CN110391952B (en) | Performance analysis method, device and equipment | |
| Nolting et al. | Context-based A/B test validation | |
| CN109190366B (en) | Program processing method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |