CN113641672A - Multi-dimensional rapid matching method and device and storage medium - Google Patents
Multi-dimensional rapid matching method and device and storage medium Download PDFInfo
- Publication number
- CN113641672A CN113641672A CN202110862429.1A CN202110862429A CN113641672A CN 113641672 A CN113641672 A CN 113641672A CN 202110862429 A CN202110862429 A CN 202110862429A CN 113641672 A CN113641672 A CN 113641672A
- Authority
- CN
- China
- Prior art keywords
- data
- matching
- dimension
- bitmap
- application object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000013507 mapping Methods 0.000 claims abstract description 18
- 238000004590 computer program Methods 0.000 claims description 6
- 238000006073 displacement reaction Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 3
- 230000000875 corresponding effect Effects 0.000 description 63
- 238000010586 diagram Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 5
- 238000003780 insertion Methods 0.000 description 5
- 230000037431 insertion Effects 0.000 description 5
- 238000003491 array Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000001413 cellular effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2264—Multidimensional index structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2237—Vectors, bitmaps or matrices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
Abstract
The invention relates to a multi-dimensional fast matching method, a device and a storage medium, wherein the method comprises the following steps: acquiring an application object and configuring corresponding multi-dimensional matching conditions; carrying out configuration initialization aiming at the multi-dimensional matching condition to form a configuration structure array; inserting quintuple data of the application object into the configuration structure array, determining a corresponding set bitmap, and mapping to obtain unique corresponding definition ID data; and matching the data packet to be matched with each dimension of the application object, acquiring a matching bitmap of each dimension, determining corresponding matching ID data, comparing the matching ID data with the definition ID data, and determining whether the data packet belongs to the application object. The invention carries out multidimensional application identification, quickly identifies which application the conversation flow in the network belongs to, and simultaneously utilizes the setting of the bitmap bit to reduce the memory space.
Description
Technical Field
The invention relates to the technical field of computer matching, in particular to a multi-dimensional rapid matching method, a multi-dimensional rapid matching device and a storage medium.
Background
Identifying network applications and classifications is the most fundamental function in modern network management and security systems. A common network application identification accuracy is 50% to 80%. However, with the development of the internet, higher requirements are put on network security and network application identification, and both accuracy and stability must be higher. Conventional application recognition systems have been unable to meet the increasingly complex network environments. In the prior art, for the AC algorithm, the function advantage is that a large number of character strings are counted and ordered, so that the AC algorithm is used for text word frequency search statistics in an advantageous scene, and the matching mode is precise matching. And the application matching in our network mainly depends on five-tuple to determine which application is input by one session, and the multi-dimensional matching is realized. The AC algorithm cannot accomplish range matching. For bloom filtering, the setting of its big bit array is not uniquely determined because its value is calculated by hash, and when some element is not in the set, but the calculated hash value is the same, there is a case in which it is being matched, i.e. the accuracy of bloom filtering is not 100%. For the naive bayes algorithm, the independence of the attributes of the data set is difficult to satisfy in many cases, because the attributes of the data set are often correlated with each other, and if the problem occurs in the classification process, the classification effect is greatly reduced. Therefore, how to provide a method for quickly and accurately identifying application traffic is an urgent problem to be solved.
Disclosure of Invention
In view of the above, it is necessary to provide a multidimensional fast matching method, apparatus and storage medium, so as to overcome the problems of inaccurate and unstable application traffic identification in a complex network environment in the prior art.
The invention provides a multi-dimensional rapid matching method, which comprises the following steps:
acquiring an application object and configuring corresponding multi-dimensional matching conditions;
carrying out configuration initialization aiming at the multi-dimensional matching condition to form a configuration structure array;
inserting quintuple data of the application object into the configuration structure array, determining a corresponding set bitmap, and mapping to obtain unique corresponding definition ID data;
and matching the data packet to be matched with each dimension of the application object, acquiring a matching bitmap of each dimension, determining corresponding matching ID data, comparing the matching ID data with the definition ID data, and determining whether the data packet belongs to the application object.
Further, the configuring the corresponding multidimensional matching condition includes:
setting the five-tuple data and the configuration data packet direction of the application object session, wherein the five-tuple data comprises source IP data, source port data, destination IP data, destination port data and protocol configuration data;
and generating corresponding matching conditions of each dimension according to the source IP data, the source port data, the destination IP data, the destination port data, the protocol configuration data and the configuration data packet direction to form the multi-dimension matching conditions.
Further, the configuring and initializing the multidimensional matching condition, and forming a configuration structure array includes:
applying a corresponding structure storage array aiming at the matching condition of each dimension, wherein the structure storage array is used for storing configuration parameters of the matching condition of the corresponding dimension;
initializing the configuration parameters of each dimension to 0 to form the configuration structure array, wherein the configuration parameters comprise bitmap pointers of the corresponding dimension.
Further, the inserting the five-tuple data of the application object into the configuration structure array comprises:
and performing recursive search based on a dichotomy, and sequentially inserting the source IP data, the source port data, the destination IP data, the destination port data, the protocol configuration data and the configuration data packet direction of the application object according to the size sequence of the IP data to form a stored data array.
Further, the determining the corresponding set bitmap, and the mapping to obtain the unique corresponding definition ID data includes:
according to the stored data array, applying for a corresponding new memory to store a dimension bitmap of a corresponding dimension aiming at each dimension;
and performing bit AND on the definition ID data of the application object and the dimension bitmap through left shifting by N bits to obtain the setting bitmap.
Further, the matching the data packet to be matched with each dimension of the application object, and acquiring a matching bitmap of each dimension includes:
determining corresponding quintuple data according to the data packet to be matched;
matching quintuple data corresponding to the data packet to be matched with the stored data of each dimensionality of the application object based on binary recursive search, determining a corresponding bitmap pointer which is not empty, and forming the matched bitmap of the corresponding dimensionality;
and carrying out bit AND operation on the matched bitmap of each dimension to determine a unique value.
Further, the performing a bit and operation on the matching bitmap of each dimension, and determining a unique value includes:
and performing bit AND operation on the matched bitmap of each dimension according to the reverse displacement opposite to the set bitmap, and determining the unique value.
Further, the determining the corresponding matching ID data, comparing with the definition ID data, and determining whether the corresponding matching ID data belongs to the application object includes:
according to the unique value, determining matching ID data corresponding to the data packet to be matched;
and comparing the matching ID data with the definition ID data, wherein when the matching ID data is equal to the definition ID data, the data packet to be matched belongs to the application object corresponding to the definition ID data.
The invention also provides a multi-dimensional fast matching device, which comprises:
the acquisition unit is used for acquiring the application object and configuring corresponding multi-dimensional matching conditions;
the processing unit is used for carrying out configuration initialization aiming at the multi-dimensional matching condition to form a configuration structure array; the configuration structure array is also used for inserting the quintuple data of the application object into the configuration structure array, determining the corresponding set bitmap, and mapping to obtain unique corresponding definition ID data;
and the matching unit is used for matching the data packet to be matched with each dimension of the application object, acquiring a matching bitmap of each dimension, determining corresponding matching ID data, comparing the matching ID data with the definition ID data, and determining whether the data packet belongs to the application object.
The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the multi-dimensional fast matching method as described above.
Compared with the prior art, the invention has the beneficial effects that: firstly, aiming at different application objects, setting matching conditions of multiple dimensions, and ensuring accurate identification of a flow message from the multiple dimensions; then, carrying out configuration initialization aiming at the matching condition of each dimension to form an initialized configuration structure array; further, aiming at the initialized configuration structure array, inserting quintuple data of the application object to form corresponding stored data, determining a set bitmap of each dimension, and forming unique definition ID data of the application object; and finally, carrying out matching identification from multiple dimensions on the data packet to be matched, forming matched ID data by using the matched bitmap of each dimension, comparing the matched ID data with the defined ID data, and determining the defined ID data which can be matched with the data packet to be matched so as to identify the hit application object. In conclusion, the invention carries out multi-dimensional application identification, quickly identifies which application the session flow in the network belongs to, and simultaneously utilizes the setting of the bitmap bit to reduce the memory space.
Drawings
FIG. 1 is a schematic view of an embodiment of an application system of a multi-dimensional fast matching method according to the present invention;
FIG. 2 is a schematic flow chart illustrating an embodiment of a multi-dimensional fast matching method according to the present invention;
FIG. 3 is a flowchart illustrating an embodiment of step S1 in FIG. 2 according to the present invention;
FIG. 4 is a flowchart illustrating an embodiment of step S2 in FIG. 2 according to the present invention;
FIG. 5 is a flowchart illustrating an embodiment of the mapping definition ID data of step S3 in FIG. 2 according to the present invention;
FIG. 6 is a diagram illustrating mapping of definition ID data according to an embodiment of the present invention;
FIG. 7 is a schematic flowchart of an embodiment of obtaining a matching bitmap for each dimension in step S4 in FIG. 2 according to the present invention;
FIG. 8 is a flowchart illustrating the step S4 in FIG. 2 determining whether the object belongs to an embodiment of an application object according to the present invention;
FIG. 9 is a diagram illustrating a relationship between a structure array, a bitmap, and an application ID according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of an embodiment of a multi-dimensional fast matching apparatus provided in the present invention.
Detailed Description
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate preferred embodiments of the invention and together with the description, serve to explain the principles of the invention and not to limit the scope of the invention.
In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Further, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Reference throughout this specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the described embodiments can be combined with other embodiments.
The invention provides a multidimensional rapid matching method, a multidimensional rapid matching device and a storage medium, which are applied to the technical field of computer matching. The following are detailed below:
an embodiment of the present invention provides an application system of a multidimensional fast matching method, and fig. 1 is a scene schematic diagram of an embodiment of an application system of a multidimensional fast matching method provided by the present invention, where the system may include a server 100, and a multidimensional fast matching device, such as the server in fig. 1, is integrated in the server 100.
The server 100 in the embodiment of the present invention is mainly used for:
acquiring an application object and configuring corresponding multi-dimensional matching conditions;
carrying out configuration initialization aiming at the multi-dimensional matching condition to form a configuration structure array;
inserting quintuple data of the application object into the configuration structure array, determining a corresponding set bitmap, and mapping to obtain unique corresponding definition ID data;
and matching the data packet to be matched with each dimension of the application object, acquiring a matching bitmap of each dimension, determining corresponding matching ID data, comparing the matching ID data with the definition ID data, and determining whether the data packet belongs to the application object.
In this embodiment of the present invention, the server 100 may be an independent server, or may be a server network or a server cluster composed of servers, for example, the server 100 described in this embodiment of the present invention includes, but is not limited to, a computer, a network host, a single network server, a plurality of network server sets, or a cloud server composed of a plurality of servers. Among them, the Cloud server is constituted by a large number of computers or web servers based on Cloud Computing (Cloud Computing).
It is to be understood that the terminal 200 used in the embodiments of the present invention may be a device that includes both receiving and transmitting hardware, i.e., a device having receiving and transmitting hardware capable of performing two-way communication over a two-way communication link. Such a device may include: a cellular or other communication device having a single line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific terminal 200 may be a desktop, a laptop, a web server, a Personal Digital Assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, and the like, and the type of the terminal 200 is not limited in this embodiment.
Those skilled in the art can understand that the application environment shown in fig. 1 is only one application scenario of the present invention, and does not constitute a limitation on the application scenario of the present invention, and that other application environments may further include more or fewer terminals than those shown in fig. 1, for example, only 2 terminals are shown in fig. 1, and it can be understood that the application system of the multi-dimensional fast matching method may further include one or more other terminals, which is not limited herein.
In addition, as shown in fig. 1, the application system of the multidimensional fast matching method may further include a memory 200 for storing data, such as multidimensional matching conditions, configuration structure arrays, bitmap setting, and the like.
It should be noted that the scene diagram of the application system of the multidimensional fast matching method shown in fig. 1 is only an example, and the application system and the scene of the multidimensional fast matching method described in the embodiment of the present invention are for more clearly illustrating the technical solution of the embodiment of the present invention, and do not form a limitation on the technical solution provided in the embodiment of the present invention.
An embodiment of the present invention provides a multidimensional fast matching method, and referring to fig. 2, fig. 2 is a schematic flow chart of an embodiment of the multidimensional fast matching method provided by the present invention, including steps S1 to S4, where:
in step S1, an application object is acquired, and a corresponding multidimensional matching condition is configured; the multidimensional matching conditions generally comprise five matching conditions of a source ip, a source port, a destination ip, a destination port and a protocol in a session five-tuple;
in step S2, performing configuration initialization on the multidimensional matching condition to form a configuration structure array;
in step S3, inserting the quintuple data of the application object into the configuration structure array, determining the corresponding set bitmap, and mapping to obtain unique corresponding definition ID data; wherein, the bitmap bits are set, each bit represents an application id, and the occupied space is small;
in step S4, the data packet to be matched is matched with each dimension of the application object, a matching bitmap of each dimension is obtained, the corresponding matching ID data is determined, and the matching ID data is compared with the definition ID data to determine whether the data packet belongs to the application object.
In the embodiment of the invention, firstly, aiming at different application objects, the matching conditions of multiple dimensions are set, and the accurate identification of the flow message is ensured from the multiple dimensions; then, carrying out configuration initialization aiming at the matching condition of each dimension to form an initialized configuration structure array; further, aiming at the initialized configuration structure array, inserting quintuple data of the application object to form corresponding stored data, determining a set bitmap of each dimension, and forming unique definition ID data of the application object; and finally, carrying out matching identification from multiple dimensions on the data packet to be matched, forming matched ID data by using the matched bitmap of each dimension, comparing the matched ID data with the defined ID data, and determining the defined ID data which can be matched with the data packet to be matched so as to identify the hit application object.
It should be noted that the five-tuple data includes source IP data, source port data, destination IP data, destination port data, and protocol configuration data. Therefore, matching is effectively carried out from multiple dimensions, and the accuracy of application identification is guaranteed.
It should be noted that the bitmap pointer is a data structure representing a dense set (dense set) in a finite field, each element appears at least once, and no other data is associated with the element.
As a preferred embodiment, referring to fig. 3, fig. 3 is a schematic flowchart of an embodiment of step S1 in fig. 2 provided by the present invention, and step S1 specifically includes steps S11 to S12, where:
in step S11, the five-tuple data and the configuration packet direction of the application object session are set, where the five-tuple data includes source IP data, source port data, destination IP data, destination port data, and protocol configuration data;
in step S12, a matching condition for each corresponding dimension is generated according to the source IP data, the source port data, the destination IP data, the destination port data, the protocol configuration data, and the configuration packet direction, so as to form the multi-dimension matching condition.
In the embodiment of the invention, the matching condition of each dimension is set for different application objects, and the multi-dimension matching condition is set according to the corresponding quintuple information, so that the accuracy of information matching is ensured.
In a specific embodiment of the present invention, the configuration of the custom application a is taken as an example, and the ID is 65, that is, for the application object a, the ID data is defined as 65. The application object may be predefined, e.g., QQ, Payment, etc., whose matching conditions have also been predefined. It may also be a custom application, the name of which is determined by itself, e.g. to satisfy port 21, protocol ftp, which may be defined as "ftp application". In the process of configuring the multidimensional matching condition, a session is uniquely determined by using a session five-tuple, such as: source/destination Ip: 1.1.1.1-1.1.1.100, source/destination port: 8080, protocol configuration: any, configure packet direction: request data (when the message direction is not distinguished, the source ip and the destination ip, the source port and the destination port are the same, if the packet source is distinguished, the above configuration is only matched in a single direction).
As a preferred embodiment, referring to fig. 4, fig. 4 is a schematic flowchart of an embodiment of step S2 in fig. 2 provided by the present invention, and step S2 specifically includes steps S21 to S22, where:
in step S21, for the matching condition of each dimension, applying for a corresponding structure storage array, where the structure storage array is used to store configuration parameters of the matching condition of the corresponding dimension;
in step S22, the configuration parameter of each dimension is initialized to 0 to form the configuration structure array, where the configuration parameter includes a bitmap pointer of the corresponding dimension.
In the embodiment of the invention, aiming at the matching condition of each dimension, corresponding storage data is applied to store the set data to form a structure storage array, the structure storage array is utilized to carry out subsequent matching identification, and a bitmap pointer of each corresponding dimension is stored to form the set bitmap which is used for effectively mapping the unique definition ID data of the corresponding object.
In a specific embodiment of the present invention, before configuration is issued, a larger structure array is applied for data of each dimension for storing parameters of each configuration, wherein one member is a bitmap pointer, and initialized data of each dimension is 0.
As a preferred embodiment, the inserting of the quintuple data of the application object in step S3 specifically includes: and performing recursive search based on a dichotomy, and sequentially inserting the source IP data, the source port data, the destination IP data, the destination port data, the protocol configuration data and the configuration data packet direction of the application object according to the size sequence of the IP data to form a stored data array. In the embodiment of the invention, the dichotomy is utilized to carry out recursive search to quickly insert the relevant configuration parameters of each dimension, thereby ensuring effective storage and facilitating subsequent identification and matching.
In a specific embodiment of the present invention, after configuration is successful, the insertion is required according to the size order of ip. The binary method is used for carrying out recursive search, the data at the beginning are all 0, the minimum is, the newly inserted data must be larger than 0, the array index of the binary search must be 0, namely, 1.1.1.1 ip should be stored in the position of the array index of 0. After the insertion, 1.1.1.100 is inserted in the same way, but it should be noted that one more ip1.1.1.101 is actually inserted, but the bitmap pointer of 1.1.1.101 is empty. The protocols of the ports of other dimensions are all represented by numerical values, the port 8080 is stored in an 8080-8081 range, the actual valid port is 8080, and the protocol any is 1-maximum (255).
As a preferred embodiment, referring to fig. 5, fig. 5 is a schematic flowchart of an embodiment of mapping definition ID data in step S3 in fig. 2 provided by the present invention, where step S3 specifically includes steps S31 to S32, where:
in step S31, according to the stored data array, for each dimension, applying for a corresponding new memory to store a dimension bitmap of the corresponding dimension;
in step S32, the definition ID data of the application object is bit-anded with the dimension bitmap by left-shifting N bits, so as to obtain the setting bitmap.
In the embodiment of the invention, the set bitmap of each dimension is utilized, and the unique definition ID data of the application object is used for mapping. It should be noted that each custom application generates an id, and the id is unique and corresponds to the application uniquely. After the ip is stored in the structure array, a new memory is applied for storing the bitmap, and then the id obtains a new bitmap with the old bitmap by shifting the bits to the left by n bits. Since all ip's in the range of addresses 1.1.1.1-1.1.1.100 can match this id, it is also necessary to traverse the address range, mapping the bitmap of each address to the id, in effect traversing only two addresses 1.1.1.1 and 1.1.1.100. Ports and protocols also map id on bitmaps of their own dimension.
In an embodiment of the present invention, referring to fig. 6, fig. 6 is a mapping diagram of an embodiment of defining ID data provided by the present invention, where u8 data type arrays are used, one u8 is one byte, and one byte has 8 bits, which can represent 8 IDs of 0 to 7, and the arrays u8 ID _ ip _ main _ array [ NUM ], where NUM is the length of the arrays, and is set by itself according to the required size. Then the number of ids that can be represented is NUM 8. And the ID is 65, and the NUM value is at least 9 at the minimum. If the mapping ID is 65, it indicates that the 65 th position of the entire ID _ ip _ may _ array needs to be 1. First, it is necessary to know to which u8 the 65 th bit belongs, and the computer counts from 0, i.e. the id is also reduced by 1 to get the true location stored in the computer. The index (index) of the array is obtained and recorded by using the integer part of (65-1)/8, namely the array index 8, and then the index is shifted to the left according to the remainder. (65-1)% 8 gets the remainder 0, then shift left by 0 bits, which is the specific position of the 65 th bit in the array, as shown in FIG. 6, in the formula: id _ ip _ may _ array [ (id-1)/8] | -1 UL < ((id-1)% 8). Where 1UL denotes unsigned long shaping 1, which can be understood as 1.
As a preferred embodiment, referring to fig. 7, fig. 7 is a schematic flow chart of an embodiment of the invention, provided by the present invention, in which step S4 in fig. 2 acquires a matching bitmap of each dimension, and step S4 specifically includes steps S41 to S43, where:
in step S41, determining corresponding quintuple data according to the packet to be matched;
in step S42, based on binary recursive search, matching quintuple data corresponding to the to-be-matched packet with the stored data of each dimension of the application object, determining a corresponding bitmap pointer that is not empty, and forming the matching bitmap of the corresponding dimension;
in step S43, the matching bitmap for each dimension is subjected to a bit and operation to determine a unique value.
In the embodiment of the invention, multidimensional matching is carried out on any data packet to be matched to obtain the bitmap pointer corresponding to the data packet, namely the matching bitmap, so that the corresponding unique value is determined based on the multidimensional matching bitmap to judge whether the data packet belongs to the corresponding application object.
It should be noted that the matching process is similar to the insertion process described above, and the designated array index is found from the ip value by binary recursive search, so that the member bitmap can be obtained, and this is true for each dimension.
As a preferred embodiment, a boundary segmentation mode is adopted to search the bitmap pointer which is most matched with each dimension of the data packet to be matched, and a corresponding matching bitmap is formed. In a specific embodiment of the present invention, if the data packet ip is 1.1.1.101, the dichotomy finds an array with 1.1.1.101, but since its bitmap is empty, it is determined that no application id corresponds to it; if the ip of the data packet is 1.1.1.5, although there is no ip in the array, the binary search will find a closest ip, which is 1.1.1.1, and its bitmap is not null, then 1.1.1.5 is considered as one between 1.1.1.1 and a certain ip range, although the ending range cannot be determined, but this is not the focus of the present invention, and the present invention only concerns whether 1.1.1.5 is in ip configuration, which is why the present invention inserts 1.1.1.1 and 1.1.1.100 directly during the insertion process, because all bitmaps in this interval are the same as 1.1.1.1, and the purpose of inserting 1.1.1.101 is to determine the range boundary, i.e. boundary segment, greater than 1.1.1.1.100. Wherein, the single port number 8080 is also segmented into 8080 and 8081 in the same way, and the bitmap of 8081 is set to null. When the bitmap of each dimension is acquired, the bitmap of the ip, the port and the protocol is positioned in the operation, and the determined application ID can be obtained, so that effective identification is guaranteed.
As a preferred embodiment, step S43 specifically includes: and performing bit AND operation on the matched bitmap of each dimension according to the reverse displacement opposite to the set bitmap, and determining the unique value. In the embodiment of the invention, after the steps are carried out, bitmap values of multiple dimensions can be obtained, and a unique value, namely the application ID, can be obtained through reverse displacement opposite to the setting of bitmap and position operation. A specific numerical example is used for explaining, when the value is different from the ID of the application a, it indicates that the data packet does not belong to the application a, and the data packet is identical to the application, thereby completing the identification.
As a preferred embodiment, referring to fig. 8, fig. 8 is a schematic flowchart illustrating that step S4 in fig. 2 determines whether the application object belongs to an embodiment of the application object, where step S4 further includes step S44 to step S45, where:
in step S44, determining matching ID data corresponding to the data packet to be matched according to the unique value;
in step S45, the matching ID data is compared with the definition ID data, and when equal, the packet to be matched belongs to the application object corresponding to the definition ID data.
In the embodiment of the invention, the bitmap pointer is used for determining the matched ID data, and the matched ID data is matched with the defined ID data of different application objects, so that the effective flow application identification is realized.
In a specific embodiment of the present invention, referring to fig. 9, fig. 9 is a schematic diagram of a relationship between a structure array, a bitmap, and an application ID according to an embodiment of the present invention, where ip is 1.1.1.5 as an example, then the same binary search method of "sequential insertion" is used to find the bitmap of 1.1.1.1, and return an array index of 8, ID of 65. At this time, only the value of the 65 th bit needs to be paid attention to, and the id _ ip _ may _ array [8] position where 65 should be located is found by using the same principle as the binary method, and the binary representation is 10000000. Other dimensions whatever data is configured, with an ID equal to 65, will find the location of array [8 ].
In the first case, if the other dimensions are not matched, that is, if the port _ bitmap of the other dimensions is empty, the ID of the application a is not matched to be 65;
in the second case, in other application B, id ═ 66, if the bitmap of the port dimension hits, the binary of port _ array [8] is 01000000, and the result of the two phases with id _ ip _ array [8] & port _ array [8] (00000001 & 00000010) is finally 0, and still does not conform to application Aid 65;
in the third case, (00000001 &00000001) hit in the port of application a, and the result is not 0, indicating that there is an application id corresponding thereto. At this time, the result needs to be circularly shifted to the right by 1 bit each time, and when the result is 0, the number of shifts is the number of bits in the array with index being 8, so + index x 8 is also needed, which is the value of id;
it should be noted that each array structure has a bitmap, each bit of the bitmap represents each application id, and if the first bit and the second bit are set to 1, the corresponding relationship is that the first bit identifies the application with the application id of 1. The second bit identifies application 2. Conversely, if not set, it indicates that the application at the corresponding location will not be matched.
An embodiment of the present invention further provides a multidimensional fast matching device, and with reference to fig. 10, fig. 10 is a schematic structural diagram of an embodiment of the multidimensional fast matching device provided by the present invention, including:
an obtaining unit 1001 configured to obtain an application object and configure a corresponding multidimensional matching condition;
the processing unit 1002 is configured and initialized according to the multidimensional matching condition to form a configuration structure array; the configuration structure array is also used for inserting the quintuple data of the application object into the configuration structure array, determining the corresponding set bitmap, and mapping to obtain unique corresponding definition ID data;
a matching unit 1003, configured to match the data packet to be matched with each dimension of the application object, obtain a matching bitmap of each dimension, determine corresponding matching ID data, compare the matching ID data with the definition ID data, and determine whether the data packet belongs to the application object.
For a more specific implementation manner of each unit of the multi-dimensional fast matching apparatus, reference may be made to the description of the multi-dimensional fast matching method of the present invention, and similar beneficial effects are obtained, and details are not repeated here.
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the multi-dimensional fast matching method as described above.
Generally, computer instructions for carrying out the methods of the present invention may be carried using any combination of one or more computer-readable storage media. Non-transitory computer readable storage media may include any computer readable medium except for the signal itself, which is temporarily propagating.
A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages, and in particular may employ Python languages suitable for neural network computing and TensorFlow, PyTorch-based platform frameworks. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The embodiment of the invention also provides a computing device, which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the processor executes the program, the multi-dimensional fast matching method is realized.
According to the computer-readable storage medium and the computing device provided by the above embodiments of the present invention, the content specifically described for implementing the multi-dimensional fast matching method according to the present invention may be referred to, and the beneficial effects similar to those of the multi-dimensional fast matching method described above are achieved, and are not described herein again.
The invention discloses a multi-dimension quick matching method, a device and a storage medium, firstly, aiming at different application objects, setting multi-dimension matching conditions, and ensuring accurate identification of flow messages from multiple dimensions; then, carrying out configuration initialization aiming at the matching condition of each dimension to form an initialized configuration structure array; further, aiming at the initialized configuration structure array, inserting quintuple data of the application object to form corresponding stored data, determining a set bitmap of each dimension, and forming unique definition ID data of the application object; and finally, carrying out matching identification from multiple dimensions on the data packet to be matched, forming matched ID data by using the matched bitmap of each dimension, comparing the matched ID data with the defined ID data, and determining the defined ID data which can be matched with the data packet to be matched so as to identify the hit application object.
The technical scheme of the invention carries out multidimensional application identification, quickly identifies which application the conversation flow in the network belongs to, simultaneously utilizes the setting of the bitmap bit to reduce the memory space, carries out multidimensional application matching identification on the flow passing through the network equipment, can identify various illegal accesses in the network through the network application identification, purifies the network environment and maintains the network security
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention.
Claims (10)
1. A multi-dimensional fast matching method is characterized by comprising the following steps:
acquiring an application object and configuring corresponding multi-dimensional matching conditions;
carrying out configuration initialization aiming at the multi-dimensional matching condition to form a configuration structure array;
inserting quintuple data of the application object into the configuration structure array, determining a corresponding set bitmap, and mapping to obtain unique corresponding definition ID data;
and matching the data packet to be matched with each dimension of the application object, acquiring a matching bitmap of each dimension, determining corresponding matching ID data, comparing the matching ID data with the definition ID data, and determining whether the data packet belongs to the application object.
2. The multidimensional fast matching method according to claim 1, wherein the configuring the corresponding multidimensional matching condition comprises:
setting the five-tuple data and the configuration data packet direction of the application object session, wherein the five-tuple data comprises source IP data, source port data, destination IP data, destination port data and protocol configuration data;
and generating corresponding matching conditions of each dimension according to the source IP data, the source port data, the destination IP data, the destination port data, the protocol configuration data and the configuration data packet direction to form the multi-dimension matching conditions.
3. The multidimensional fast matching method according to claim 2, wherein the initializing configuration for the multidimensional matching condition and forming a configuration structure array comprises:
applying a corresponding structure storage array aiming at the matching condition of each dimension, wherein the structure storage array is used for storing configuration parameters of the matching condition of the corresponding dimension;
initializing the configuration parameters of each dimension to 0 to form the configuration structure array, wherein the configuration parameters comprise bitmap pointers of the corresponding dimension.
4. The multidimensional fast matching method according to claim 3, wherein the inserting the five tuple data of the application object into the configuration structure array comprises:
and performing recursive search based on a dichotomy, and sequentially inserting the source IP data, the source port data, the destination IP data, the destination port data, the protocol configuration data and the configuration data packet direction of the application object according to the size sequence of the IP data to form a stored data array.
5. The multidimensional fast matching method according to claim 4, wherein the determining of the corresponding set bitmap and the mapping to obtain the uniquely corresponding definition ID data comprises:
according to the stored data array, applying for a corresponding new memory to store a dimension bitmap of a corresponding dimension aiming at each dimension;
and performing bit AND on the definition ID data of the application object and the dimension bitmap through left shifting by N bits to obtain the setting bitmap.
6. The multidimensional fast matching method according to claim 1, wherein the matching of the data packet to be matched with each dimension of the application object, and the obtaining of the matching bitmap for each dimension comprises:
determining corresponding quintuple data according to the data packet to be matched;
matching quintuple data corresponding to the data packet to be matched with the stored data of each dimensionality of the application object based on binary recursive search, determining a corresponding bitmap pointer which is not empty, and forming the matched bitmap of the corresponding dimensionality;
and carrying out bit AND operation on the matched bitmap of each dimension to determine a unique value.
7. The multi-dimensional fast matching method according to claim 6, wherein said performing a bit and operation on said matching bitmap of each dimension, determining a unique value comprises:
and performing bit AND operation on the matched bitmap of each dimension according to the reverse displacement opposite to the set bitmap, and determining the unique value.
8. The multidimensional fast matching method of claim 7, wherein the determining corresponding matching ID data, compared with the definition ID data, to determine whether the application object belongs to comprises:
according to the unique value, determining matching ID data corresponding to the data packet to be matched;
and comparing the matching ID data with the definition ID data, wherein when the matching ID data is equal to the definition ID data, the data packet to be matched belongs to the application object corresponding to the definition ID data.
9. A multi-dimensional fast matching device, comprising:
the acquisition unit is used for acquiring the application object and configuring corresponding multi-dimensional matching conditions;
the processing unit is used for carrying out configuration initialization aiming at the multi-dimensional matching condition to form a configuration structure array; the configuration structure array is also used for inserting the quintuple data of the application object into the configuration structure array, determining the corresponding set bitmap, and mapping to obtain unique corresponding definition ID data;
and the matching unit is used for matching the data packet to be matched with each dimension of the application object, acquiring a matching bitmap of each dimension, determining corresponding matching ID data, comparing the matching ID data with the definition ID data, and determining whether the data packet belongs to the application object.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a multidimensional fast matching method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110862429.1A CN113641672A (en) | 2021-07-30 | 2021-07-30 | Multi-dimensional rapid matching method and device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110862429.1A CN113641672A (en) | 2021-07-30 | 2021-07-30 | Multi-dimensional rapid matching method and device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113641672A true CN113641672A (en) | 2021-11-12 |
Family
ID=78418766
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110862429.1A Pending CN113641672A (en) | 2021-07-30 | 2021-07-30 | Multi-dimensional rapid matching method and device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113641672A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102571531A (en) * | 2010-12-16 | 2012-07-11 | 上海博达数据通信有限公司 | Classified matching method for access control list |
| CN103685224A (en) * | 2013-09-05 | 2014-03-26 | 北京安博达通科技有限责任公司 | A network invasion detection method |
| CN104579970A (en) * | 2013-10-29 | 2015-04-29 | 国家计算机网络与信息安全管理中心 | Strategy matching method and device of IPv6 message |
| CN108628966A (en) * | 2018-04-20 | 2018-10-09 | 武汉绿色网络信息服务有限责任公司 | A kind of quick matching and recognition method and device based on character string |
| CN109617927A (en) * | 2019-01-30 | 2019-04-12 | 新华三信息安全技术有限公司 | A kind of method and device matching security strategy |
-
2021
- 2021-07-30 CN CN202110862429.1A patent/CN113641672A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102571531A (en) * | 2010-12-16 | 2012-07-11 | 上海博达数据通信有限公司 | Classified matching method for access control list |
| CN103685224A (en) * | 2013-09-05 | 2014-03-26 | 北京安博达通科技有限责任公司 | A network invasion detection method |
| CN104579970A (en) * | 2013-10-29 | 2015-04-29 | 国家计算机网络与信息安全管理中心 | Strategy matching method and device of IPv6 message |
| CN108628966A (en) * | 2018-04-20 | 2018-10-09 | 武汉绿色网络信息服务有限责任公司 | A kind of quick matching and recognition method and device based on character string |
| CN109617927A (en) * | 2019-01-30 | 2019-04-12 | 新华三信息安全技术有限公司 | A kind of method and device matching security strategy |
Non-Patent Citations (1)
| Title |
|---|
| 李丽娟 等: "基于多维度的网络数据融合筛选技术", 《中国新技术新产品》, pages 34 - 36 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9634902B1 (en) | Bloom filter index for device discovery | |
| CN110191428B (en) | Data distribution method based on intelligent cloud platform | |
| CN108268216B (en) | Data processing method, device and server | |
| CN111443899B (en) | Element processing method and device, electronic equipment and storage medium | |
| CN109150962B (en) | Method for rapidly identifying HTTP request header through keywords | |
| CN111859033B (en) | IP library query method and device and IP library compression method and device | |
| CN111737564A (en) | Information query method, device, equipment and medium | |
| CN110765076A (en) | Data storage method and device, electronic equipment and storage medium | |
| CN114817651B (en) | Data storage method, data query method, device and equipment | |
| CN114817657A (en) | To-be-retrieved data processing method, data retrieval method, electronic device and medium | |
| CN113572721B (en) | Abnormal access detection method and device, electronic equipment and storage medium | |
| EP3418909A1 (en) | A method for accessing a key in a cuckoo hash table | |
| CN110022343B (en) | Adaptive event aggregation | |
| CN113641672A (en) | Multi-dimensional rapid matching method and device and storage medium | |
| CN113742332A (en) | Data storage method, device, equipment and storage medium | |
| US20160301658A1 (en) | Method, apparatus, and computer-readable medium for efficient subnet identification | |
| CN113726885A (en) | Method and device for adjusting flow quota | |
| CN113342275B (en) | Method, apparatus and computer readable storage medium for accessing data at block link points | |
| US11822803B2 (en) | Method, electronic device and computer program product for managing data blocks | |
| CN116055191B (en) | Network intrusion detection method and device, electronic equipment and storage medium | |
| CN113256301B (en) | Data shielding method, device, server and medium | |
| CN114528258B (en) | Asynchronous file processing method, device, server, medium, product and system | |
| CN111131197B (en) | Filtering strategy management system and method thereof | |
| CN115622977A (en) | Method, system, device and medium for detecting and recording IPv4 address space survival | |
| CN116600031A (en) | Message processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |